Update 14 October, 2021

We’ve alllmost reached a mini-milestone today (90% of the way there!) with the merging of the API and the CLI into the main Safe Network repo meaning no more version mismatches. What does that mean and why does it matter? @ChrisO explains all below.

General Progress

As you will probably be aware, with features like AE the network is designed to be ‘CRDT-like’ with eventual consistency guaranteed. This means that if we send a message or a chunk of data and it doesn’t get received immediately, it doesn’t matter because we simply wait or send it again. No complex synching required.

Much of the work we are doing at the moment is getting this CRDT-like behaviour working properly. The looping behaviour we saw with node-to-node messaging seems to be mostly fixed now.

AE messages let the client know whether it needs to update its understanding of the sections it’s talking to. We’ve added a few tweaks to the client so that it pre-contacts any bootstrap nodes to get an updated view of the network before allowing it to move on to fire any commands. We’ve also added a tiny wait on PUTting any data to the network, as the command was previously returning straight away, while the AE-message handling happens in the background. This hasn’t been an issue with the safe_network library tests as they all keep the client around, but the CLI doesn’t so this could have been leading to issues where a client didn’t know about a part of the network, and missed handling/re-sending the command when those AE-messages came in.

We’re also seeing some anomalies with the CI (continuous integration) tests. It can be hard to tell whether the problem lies with the network or the CI tests themselves. It’s the sort of thing testnets help us figure out. Thanks again to everyone who’s been getting stuck in. There should be another candidate to check out shortly, as we get the API+CLI merged. We’re also considering options for a cloud ELK cluster to store community testnet logs and make them searchable.

@Jimcollinson has been continuing to look into n of m authentication flows using BLS key shares. To unlock your Safe you might choose to go with a password + passphrase (or QR code) combo with a device key on your phone in reserve (2 of 3). If you forget your password you could use passphrase and device key. With your primary credentials (password and passphrase) you can also create new keys on additional devices so, if you’re really forgetful, you could unlock your safe using multiple devices, or backup passphrases. The idea here is achieving the flexibility to accommodate various security threat models—balanced with convenience—and yet be tolerant of credential loss.

On the DBC front @danda has been digging further into denominations. Expect more details down the line.

A Single Repository

Back in June, we had completed some work to merge several repositories that constituted the code for the network, into a single large repository. Due to the code in these repositories being so closely related, it didn’t make sense for them to evolve independently, and in fact, it was actively problematic for them to do so. It made the network difficult to debug and test, and keeping all the versions up-to-date and synchronised was a significant chore. The single repository combines all this code into different modules of the same crate, with a single version number.

However, in this endeavour, there were a couple of repositories left behind, namely sn_api and sn_cli. The API and the CLI are actually a bit different from the previously mentioned repositories, in that they’re not core components of the network itself, but rather their purpose is to enable us to use the network. Due to constrained resources focussed mainly on development of the network, the code in both the API and CLI had gone out-of-date, and it was a significant piece of work to bring them up-to-speed. So even though we did also want to merge them, we weren’t really in a position to do so.

We’ve been working to bring the CLI and API up-to-date, so thankfully, now we are in a position where they can be merged in, and we’re happy to say that this is almost there. sn_api and sn_cli will sit alongside the code for the network, so the safe_network repository consists of three crates that will be updated and deployed at the same time.

It’s important to be aware, this is actually quite a significant change and not simply an administrative convenience. Some of the benefits:

  • Everything in the same crate removes confusion about which versions of the CLI and API are compatible with which version of the network. All those in any release will be compatible.
  • When the code for the network is updated with breaking changes, the developer will be forced to make an update to the API and CLI. This prevents the API and CLI code from going stale again.
  • New versions of the network are immediately available for testing with a CLI that is compatible with the network.

On the last point there: now that we’ve seen and welcomed the community being more involved in launching testnets, with a CLI/API/network combination being released at the same time, we can create a much shorter test and release cycle, where essentially any new release in the safe_network repository is immediately available for community testing. Ultimately this has big potential to help us iterate faster towards a launch candidate.


Useful Links

Feel free to reply below with links to translations of this dev update and moderators will add them here:

:russia: Russian ; :germany: German ; :spain: Spanish ; :france: French; :bulgaria: Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!

73 Likes

Let’s try a First!

Great work guys! Enjoying these posts.

28 Likes

90% is a great number. Great work to all!

18 Likes

Nice, thank you for the update!

17 Likes

Very nice, I think this is both technically sweet and another example of how MaidSafe are always looking for max decentralisation in every aspect of the project. No-one else comes close, no other project has such broad capabilities. :partying_face:

30 Likes

Thank you to all concerned for this update - glad to see its all continuing to come together.

18 Likes

Does that one repository spawn one binary?.. assets on current release show sn_node but I can’t see enough on the phone atm to be sure if a lite cli is there for those not with node. Perhaps that’s just for future.

Nice to see talk of safeid handling too… been thinking a lot about this myself recently.

:+1:

13 Likes

Music to my ears especially with @josh, @vort and @Southside smashing the community testnets :clap:t2:

18 Likes

It’s great this is getting attention. What good is a decentralized internet if all us forgetful folks can’t unlock our safes?! The focus on usability is always unprecedented. Thanks @maidsafe

15 Likes

And this faster iteration means efficient logging - and more importantly logging analysis - becomes essential.
Unfortunately I have been distracted with paying work and then a very liquid funeral and wake yesterday has meant I have not moved as far with ELK as I had hoped. Hope to get that sorted soon after my hangover subsides.

13 Likes

The work hasn’t been completed yet. Once completed, both the safe and sn_node binaries will be available on the safe_network releases.

14 Likes

Be sure to remember and ping me if you need any help too!

15 Likes

Fantastic work all. Bugs squashed, Features (mostly) locked down and into a synced release pipeline, reevaluating test architecture… it’s a good place to be.

11 Likes

Thanks so much to the entire Maidsafe team for all of your hard work and great progress. Keep working hard, but remember to rest and relax too. :racehorse:

11 Likes

As usual I’m late to the party but what a fantastic update I was lacking on the time needed to join in for the last community test net but am ready to try my hand next time round.

Well done to all the team

14 Likes

Oh don’t worry, I will :slight_smile:

You may live to regret this offer.

10 Likes

It would be good to do another one next week, upon completion of the merge of sn_cli into safe_network.

14 Likes

I’ve got the tool firing on all 8 and purring like a kitten.

15 Likes

We don wan no steenkeeng gringo V8 vibrating like a very shoogly thing.
We demand a thoroughbred Italian V12 that sings…

4 Likes