Keypair generation - crowdsourcing some data

PC i7-4770 @3.40 GHz Win 10

0.432102601 seconds to generate 10000 sodium keys
23142 sodium keys per second

0.383055007 seconds to generate 10000 dalek keys
26105 dalek keys per second

4 Likes

PC i7-4790 @3.8ghz - Gentoo w/ libsodium 1.0.11

Run 1:

1.319460804 seconds to generate 10000 sodium keys
7578 sodium keys per second

0.386879183 seconds to generate 10000 dalek keys
25847 dalek keys per second

Run 2:

1.365471169 seconds to generate 10000 sodium keys
7323 sodium keys per second

0.37937801 seconds to generate 10000 dalek keys
26358 dalek keys per second

Run 3:

1.352970868 seconds to generate 10000 sodium keys
7391 sodium keys per second

0.393035926 seconds to generate 10000 dalek keys
25442 dalek keys per second

4 Likes

I just got to the bottom of this. Unfortunately I was right; there was a bug in our build script for libsodium meaning optimisation was effectively disabled for all platforms except Windows.

Some halfwit (who mainly works on Windows) created the bug about a year and a half ago :man_facepalming: so this could actually have significant repercussions for our notions of performance of all the testnets in that timespan!

I’ve got a pull request up for the fix now… better late than never I guess!

9 Likes

Hey @jlpell

When you say you used libsodium 1.0.11, is that what you have installed on your system? Unless you modified @mav’s source code, your installed version of libsodium would have been ignored. Going by your results, it looks like it’s used our build script, but I just want to check :slight_smile:

1 Like

Who will remain nameless hey :stuck_out_tongue_closed_eyes::tired_face:

Good thing I asked wasn’t it. :grin:

Still my Win 7 times were close for each method rather than the “3 times” difference.

2 Likes

Absolutely! :smile:

Windows builds fortunately weren’t affected by this bug since rather than building libsodium from source, our build script pulls down prebuilt binaries provided by the library maintainer.

2 Likes

Oh it was linux that was affected. I read it as “the person” worked mainly on windows meant it was the windows build.

You know what they say about people who assume …

2 Likes

I already did find it odd that this Sodium keys generation test was slower on the Linux OS’es than comparable systems with a Windows OS.
So retest this with the fix on non Windows systems?
And I assume you’ll be double checking for potential comparable mistakes at other places, if applicable?

3 Likes

Well done @neo and @mav for helping uncover this bug. This community is great. :slight_smile:

8 Likes

I’ll let @mav answer this. I’d guess yes, but with the fix, the results appear to be similar to those for Dalek, so maybe he’s satisfied with what info is already gathered.

Well, none of the rest of our backend libraries have build scripts which compile native dependencies, so the issue should be contained to just rust_sodium. Having said that, I’ve been wanting to set up test machines in-house to run a variety of continuous tests, including benchmarks, but haven’t had time to date. Running something like the self-encryption benchmark tests should have picked up a regression like this, and also would have the benefit of picking up regressions in any third party dependencies we use.

3 Likes

It’s great to see open source software working as intended. It’s not just about the source code, it’s about the full gamut and this whole community is a real asset to the safe network project.

Given enough eyeballs, all bugs are shallow.

14 Likes

I’m satisfied with the results. I’ll release a new version once the change is merged but the data already gathered is useful enough as it is.

4 Likes

Yes, that was the version of libsodium on my system. I figured…
perhaps since it wasn’t the most recent version available I was such getting such poor performance compared to the windows folks with similar hardware. Why else would my natively compiled gentoo system be so slow? :sweat_smile:

2 Likes

Ah - thanks - makes sense! :smile:

@anon86652309
Not that I know what I’m doing in rust or SAFE just yet… but per your original question I modified mav’s benchmark to use libsodium-ffi instead of rust_sodium so that my system’s version of libsodium v1.0.11 would be used. Based on my reading of the libsodium-ffi docs I think I picked the correct corresponding function to what was being used by rust_sodium. Here was the result:

25519_brute_force
Started generating keys

3.95828689 seconds to generate 100000 sodium keys
25263 sodium keys per second

3.751911443 seconds to generate 100000 dalek keys
26653 dalek keys per second

The changes I made to mav’s src/main.rs were:

//extern crate rust_sodium;
//extern crate sodium;
//use rust_sodium::crypto::sign;
extern crate libsodium_ffi;
let mut pk: u8 =0; let mut sk: u8 =0;
let pk_ptr : *mut u8 = &mut pk; let sk_ptr : *mut u8 = &mut sk;
let total_tests = 100000;
for _ in 1…total_tests + 1 {
//sign::gen_keypair();
unsafe {libsodium_ffi::crypto_sign_keypair(pk_ptr,sk_ptr);}}

EDIT:
Just realized reading through the thread that it wasn’t very clear about the order of events for the benchmarks I ran. My first set of benchmarks were using the same rust_sodium as everyone else on linux, and I reported the version of libsodium on my system because I assumed it was being used. @anon86652309 pointed out that was not the case and that my results were for rust_sodium since I would have needed to do something silly like modify the mav’s source code for std libsodium to be pulled in. This made me think, “hey why not try something silly like that to get another datapoint…”. So this post (above) just shows the performance of std libsodium on the same hardware.

2 Likes

For me a big part of the dalek use is the ability to work on more platforms without difficult build scripts. Also the ability to use no std and use the code on micro controllers etc.

In terms of validation dalek has had 1 validation (external audit). Libsodium has recently had one validation as well, so we hope these are good audits, but who knows, nist vectors seem ok, but the ability to run on more platforms for me is a big win.

Alongside the miscreant lib (rust) means we can ditch a lot of c libs (for symmetric encryption), use no std etc. so we can work on many platforms, but also get benefits of nonce reuse etc.

So move to pure rust is a win all round, then we attack brotteli (compression) and get to pure rust and use of no std where possible to allow much better and easier compilation on more platform types. It would not be 100% cross platform, but it is closer.

6 Likes

IMO based on severe QA & Release Management pain endured in a previous software development life, the decision to ditch c libs is a very wise decision given what is being built (an overlay network), while sticking to rust 100% to accomplish the task. This homogenity of language use dramatically simplifies and lowers the OPEX cost of the backend development machinery of build/compile/test reporting and related effort and has more people able to respond to “all hands on deck” bugs as they occur, also allows devs to switch off to different tasks between release cycles quite easily making manpower adjustments so much easier , and overall makes the whole effort that more agile and competitive.

3 Likes