Simple web-based tool for BLS keys

Came across EIP2333 today, basically BIP32 for BLS keys

“for deriving BLS private keys from a single source of entropy … allows for a practically limitless number of keys to be derived for many different purposes while only requiring knowledge of a single ancestor key in the tree.”

Mostly my interest here is whether there’s a standard way to represent a BLS keypair and how it feeds into the concepts of poly / commitment in the threshold_crypto library, and whether Safe might benefit from this standard representation (existing libraries etc).

I came across EIP2333 from chia.net - they have a bunch of tests for bls keys in bls-signatures test.cpp and also refer to https://tools.ietf.org/html/rfc5869 describing a HKDF (ie a Hashed Message Authentication Code (HMAC)-based key derivation function, got that?!) used by EIP2333

7 Likes

I’m still trying to track down a solid test vector for bls12-381 serialization. Not having much luck so far.

I found this from chia mapping secret key to public key, however threshold_crypto does not generate a matching public key:

I also came across this issue from Feb 2019 with ethereum2.0 (who will also be using bls12-381): https://github.com/ethereum/eth2.0-specs/issues/605
“This is a heads-up that the BLS spec is likely to change. Many blockchain projects are converging towards BLS12-381 (Zcash Sapling, Chia, Ethereum 2.0, Filecoin, Dfinity, Algorand, etc.) and there is a strong desire for everyone to standardise on the fine details.”
“I encourage anyone interested in this discussion to email Sergey Gorbunov (sgorbunov@uwaterloo.ca) who seems to be leading the effort.”
Might be worth someone at maidsafe getting in touch.

I’ve started comparing blst vs threshold_crypto, since blst has a lot of to_bytes and from_bytes methods that can be compared with threshold_crypto bincode serialization. It’s a shame threshold_crypto doesn’t have any to_bytes or from_bytes since those methods would be a lot more reliable than arbitrary serialization (in this case using bincode).


Also to extend on EIP2333 (ie BIP32-HDkeys-for-BLS) there’s also EIP2335 BLS12-381 Keystore which specifies “a JSON format for the storage and interchange of BLS12-381 private keys. … This specification is designed not only to be an Ethereum 2.0 standard, but one that is adopted by the wider community who have adopted the BLS12-381 signature standard. It is therefore important also to consider the needs of the wider industry along with those specific to Ethereum.”
I’m not sure if this is worth picking up and using but it’s worth knowing about this ongoing work around key management for bls12-381 keys. Whatever problem (real or not) these proposals are addressing we can be pretty sure to be running into similar issues ourselves.

edit: to clarify why I’m pursuing ‘other peoples’ standardization, one of the benefits is hardware wallets will be interoperable between these other cryptos and Safe (so we don’t have wonder how to do hardware wallets, they’ll just be there already).

11 Likes

A bit of a large edit so I’ll put it in a separate post

I found this test for bls keypairs from chia which maps secret key to public key, however threshold_crypto does not generate a matching public key however the secret key must be converted to little endian before being used in the web tool. See this issue comment in threshold_crypto for more info.

The SecretKey in that test in little endian form is
045690f6a8fb6fac9ce7c1171740e4e2e1f572036240e6a7a091c0dae33b354a
which gives the matching public key from the test when used in the BLS web tool.

The current sn_api implementation serializes secret keys using bincode which defaults to little endian; the code is fairly deeply nested but ends up at sn_data_types/utils.rs L28.

5 Likes

Version 0.3.0 is out, see BLS - Threshold Crypto

The main change is to be compatible with all other bls12-381 tools / libraries (which use big endian rather than little endian, eg ethereum2, chianet, zcash, algorand, …).

This means the chianet test posted above now passes here as well. Any hardware wallet, fpga, dev tools etc developed for other bls-based networks will also work with this tool (and presumably Safe Network too).

It’s a bit of a tricky situation because of these (unfortunately quite technical) reasons:

  • sn_api does not show keys to people any more so it’s hard to say if this tool is now compatible or not. I’m sure sn_api will end up importing / exporting keys the same way as all other bls12-381 projects, so it will end up being compatible with this tool (and other bls12-381 tools such as hardware wallets etc).
  • secret_key, public_key, signatures are clearly displayed as big endian in all places I can find, but it’s not clear how to display commitments and polys (as in, I can’t find any other bls12-381 tests for how these would be represented).
  • bincode is fine for simple structs like SecretKey but for more complex structs like Commitment and Poly bincode is almost certainly not what will be commonly used, so this is likely to change in the future. But it’s not clear how it will change.
11 Likes

This is something in the TODO list for the CLI as well, right now as a temporary solution the keypair the CLI obtains when authorised with authd is stored at ~/.safe/cli/credentials but with a non-standard format (we just serialise our KeyPair struct), so I guess apart from what you suggest about importing/exporting keys, I guess this is a good standard format for storing the CLI credentials file too?

Or what would be a good format to store diff type of keys, e.g. BLS and Ed25519 so it includes this info?

6 Likes

Hey @mav! Really happy to see that you’re maintaining and bringing new feats to this really nice multi-purpose web-app there! As for the pain points listed:

secret_key, public_key, signatures are clearly displayed as big endian in all places I can find, but it’s not clear how to display commitments and polys

threshold_crypto folks seem to have it implemented a basic Debug for poly and commitments, it shouldn’t be a problem to look into this and raise a PR to the crate, will get this in my to-do list (:

bincode is fine for simple structs like SecretKey but for more complex structs like Commitment and Poly bincode is almost certainly not what will be commonly used

bincode was our choice for all serializations in SAFE Network(due to it’s compactness and speed) and we made use of the same for our bls_dkg crate as well. But as you said so, it’s not the final version and we are defo looking out for alternatives to bincode(msg_pack, cookie_factory, etc,.) passively that’ll be more robust for complex structs. So do keep an eye if out for changes in the futures.

Hope it makes sense. Thanks! :slight_smile:

4 Likes

Yes it would be.

The thing is, if we want to do hardware signing, threshold_crypto gives no easy way to get the bytes in a standard way for use with that hardware.

threshold_crypto::SecretKey.reveal() gives the ‘standard’ big endian hex value that matches all other places. But it’s given as SecretKey(Fr(0x<the hex>)), so isn’t ergonomic to work with (have to strip the start/end, then parse the hex to bytes). Why not just give us the bytes directly with SecretKey.to_bytes()!

hex(bincode(SerdeSecret(SecretKey))) does not match SecretKey.reveal().

So we just need to be careful about how the secret is managed. There’s always some way to recover/convert the info so let’s not sweat it too much just yet!

I’m currently working on an EIP2333 tool that will complement the threshold_crypto_ui tool nicely. I’m pretty sure EIP2333 will gain rapid traction, it’s a great standard.

That would be amazing.

I’ve poked around a bit with the idea of SecretKey.to_bytes() and SecretKey.from_bytes() in #104 but the presence of SerdeSecret makes me feel those changes may not be so easy to get through. threshold_crypto seems very cautious about accidentally leaking secrets.

Hopefully these helper methods can be added somehow, I would love it :slight_smile:

8 Likes

You confused me a bit with this part, so for hardware signing (hardware wallet right?) we wouldn’t need the secret key, perhaps not even the pk, what scenario is it you are imagining here?

1 Like

Oh yes sorry this is confusing! I meant to say offline signing, or any non-maidsafe-based signing. In that case people need a standard ‘view’ of the secret key to be exported/imported to/from maidsafe/other tools.

If we show users hex(bincode(SerdeSecret(SecretKey))) the value would not be importable anywhere else (because they expect big endian but bincode gives little endian).

So a basic compatibility test might be

  • get a key from maidsafe software (not currently possible but will be in the future)
  • try to import that key somewhere else (maybe a piece of zcash software)
  • (optional) use zcash software and the maidsafe key to sign a message
  • (optional) bring the signature back into maidsafe for verification (and broadcast if desired)

If we can do this then we will have access to a large ecosystem of bls tools that will ‘just work’ with maidsafe software. If we can’t do this people may need to do some preprocessing of keys coming out of maidsafe software before being used elsewhere.

9 Likes

Chianet has a bls test with a secret key, public key, message and signature in this test.

threshold_crypto (and the web tool) can read the secret key and convert it to the correct public key.

But threshold_crypto cannot verify the signature.

I’m trying to find out why. I started a conversation in the threshold_crypto repo but if anyone here has additional info that’d be great too.

4 Likes

EIP2333 bls key generator for easy bls key management.

It’s like BIP39 / BIP32 but for bls keys instead of bitcoin keys. Very handy indeed.

I’ll be adding signing / verifying / encryption / decryption in a future release but be aware these operations will not currently be compatible with the threshold_crypto_ui tool (despite both being bls12-381 based).

8 Likes

There doesn’t seem to be much likelihood of threshold_crypto becoming compatible with other bls implementations such as ethereum, zcash, chia, algorand etc.

https://github.com/poanetwork/threshold_crypto/issues/110#issuecomment-808743875

I agree it would be good to make it compatible. Not sure if anyone is working on this repository right now, though.

There are several people I could see being able to implement the necessary changes to become compatible, maybe with funding it might be possible to get them to submit pull requests, I’m not sure, but I also don’t think it’s necessarily needed or desired.

But, for now, I’ve made threshold_crypto compatible to the degree I feel necessary with the repo blsttc - "poanetwork/threshold_crypto using supranational/blst for sign+verify "

Of interest is commit 27c64dc - Use BLST for sign and verify

This allows us to keep the existing api of threshold_crypto (no code changes to sn_* code) but get the performance and standardization benefits of blst.

Why use blsttc?

  • faster, around 2x to 12x faster to sign and verify than threshold_crypto (see this bls performance post)
  • based on a standardized implementation, so the inputs and outputs can be used with many other libraries, hardware devices, tooling etc where threshold_crypto would not.
  • compiles with musl

Why not use blsttc?

  • uses unsafe a lot (73 instances in lib.rs), may or may not be a problem in real life.
  • comes with caveats, eg from the blst readme
    • The library deliberately abstains from dealing with memory management and multi-threading, with the rationale that these ultimately belong in language-specific bindings.
    • Another responsibility that is left to application is random number generation.
    • The essential point to note is that it’s the caller’s responsibility to ensure that public keys are group-checked with blst_p1_affine_in_g1
  • State of audit / security is not as clear as threshold_crypto
    • blst: “Formal verification of this library is planned and will utilize Cryptol and Coq to verify field, curve, and bulk signature operations.”
    • threshold_crypto: “An official security audit has been completed on threshold_crypto by Jean-Philippe Aumasson.”

I have no expectation of blsttc being used but I personally find it useful so maybe others will too.

8 Likes

I do think we need to consider that move. Using an unmaintained (even if it is audited) lib is not a good move. This would be a great contender for BGF for sure.

4 Likes

Sorry I didn’t really get to explaining this properly. Let me explain why I don’t think we need to do major low-level changes to threshold_crypto.

threshold_crypto is a really nice lib. It has a clean interface and is well understood by maidsafe devs. These are the main differences:

  • dkg via poly+commitments (can be added to other libs, the initial poly.rs code comment is very helpful, but currently these features are unique to threshold_crypto). SN uses dkg extensively which is a good reason to keep threshold_crypto at least as an interface. Other bls libraries have the required low level functions, but do not have any sort of higher level Poly or Commitment types.

  • bls encrypt+decrypt functionality is unique to threshold_crypto (eg blst does not offer these methods). Sure, SN doesn’t use it (instead using ed25519 or symmetric encryption), so this is only a minor point. If we can use a standard then great we should, but no other library offers these methods (eg bls in javascript api does not offer encrypt+decrypt) so I’m not sure if there is a standard for bls encrypt+decrypt.

  • other bls libraries all seem to offer public key and signature aggregation, which threshold_crypto does not. This feature isn’t needed for current network operations but might be very useful later on, I’m not sure yet.

So I feel we shouldn’t necessarily aim to ‘fix’ threshold_crypto, since it’d probably still be slower than blst which is heavily optimized. But we probably should aim to keep threshold_crypto api since it’s nicer than the alternatives and it’s already used extensively by maidsafe. I guess blsttc satisfies both these in a very simple way, standardizing and speeding up the most common operations (sign+verify) but keeping the public interface identical to before.

5 Likes

This is important for sure. Dealerless key generation is critical to these libs IMO.

The weirdness part? We do this ourselves and I would prefer to use a crypto lib for that as we don’t want to hire cryptographers for this. This is a failing of threshold_crypto but we have a solution for now.

5 Likes

Small update to the tool that allows different encodings for keys (hex/bytes) and messages (ascii/hex/bytes). Very handy to have a simple switch when using these values in dev tools or hardcoded in rust.

https://iancoleman.io/threshold_crypto_ui/

For example the chia test specifies the signed message as bytes [7,8,9] (can’t type those ascii chars easily from a keyboard), so it’s nice to be able to test that sort of stuff.

7 Likes

This is not a bad learning resource for those interested in what going on behind in this excellent tool:

3 Likes

And after all this bigendian stuff, turns out filecoin uses littleendian bls keys…

paulmillr/noble-bls12-381 - Add a note on filecoin - “Filecoin uses little endian byte arrays for private keys - so ensure to reverse byte order if you’ll use it with FIL.”

:man_facepalming:

4 Likes

Yip, typical. :man_facepalming:

3 Likes

New version is out, and with that a move to blsttc_ui

The renamed site is https://iancoleman.io/blsttc_ui/
New renamed repo is GitHub - iancoleman/blsttc_ui: UI for blsttc rust library
Standalone file is in the releases 0.4.0: Release 0.4.0 · iancoleman/blsttc_ui · GitHub

v0.4.0 is a really exciting step and feels close to being more than just a prototype.

The big change is to move from threshold_crypto to blsttc library, which was done for two main reasons:

  • data serialization is done in a consistent / documented / tested way using to_bytes and from_bytes. This means bincode is not used any more (yay!).

  • signatures are now bbs+ signatures which are compatible with virtually every other user of bls12-381 (eg chia, eth2 etc).

The serialization thing is pretty huge imo. Being able to settle on a particular format for Poly and Commitment data (and Ciphertext etc) makes them more portable and usable beyond just this tool (although I don’t know anyone using these kind of keys yet). Poly is sorta equivalent to bip32 xprv and Commitment is sorta equivalent to bip32 xpub.

Prior to this we only had portable secret keys and public keys, so hopefully we can see Poly and Commitment style keys become more widely used.

Now we can start to really think seriously about key management. EIP2333 is another similar key management tool but is a little constrained since it doesn’t have the concept of master public key for deriving child keys (only a master secret, at least as far as I know). So I’m really pumped about Poly and Commitment hopefully becoming easier to use.

Shoutout to github user glitch003 for giving some tips on how to get blst working with wasm in this issue. It was incredibly helpful and good timing considering this was an unprompted issue he raised at just the right time.

One small detail is I tried adding a single-threaded feature flag to blst, which worked for tests, but when I tried to compile to wasm using that feature flag it didn’t seem to omit the threading. I’ll have to look further into that (any help is welcome), but in the meantime I’ll be using this single_thread branch in the tool, which I’ll do my best to keep up to date with blst master branch.

16 Likes