Update 11 November, 2021

Something a bit different this week and an opportunity to get up to date with all the developments in the network and how they fit together. JPL has been badgering us all for info over the past few weeks, but all in a good cause as the Primer has now been updated! He explains the main changes below.

General Progress

New Hire: We have appointed a finance person now. Andrew James, from Troon of all places! Andrew will be heavily fintech/onboarding focussed after a short period sorting out all the mess our CEO has probably made of the accounts in the last few months :wink: This will release David again to get a focus again on the technology which he tells us “he cannot wait to focus once more”, as admin for him is “a living hell”. Everyone to their own! The only downside is that Andrew is on a 3 month notice period, but we hope that can be reduced.

@lionel.faber has been looking to speed up the CI process by using an AWS EC2 instance to build the entire sn_repo, uploading the artefacts to S3 and then downloading it in all the parallel builds so the GitHub Action machines need no compilation. It should speed CI up significantly at a low cost thanks since AWS bills the instances per second and does not charge for instances that have been powered off.

@chriso is also digging into testnet improvements and release the process. The CLI merge is still a little too buggy for primetime just yet.

Back in the Labs, @danda and David Rusu have been looking further into ring signatures and how they might combine with a Rust BLS cryptography library blstrs and commitments.

And in the bug squishing department, @bochaco, @yogesh and @qi_ma are sifting through Anti-Entropy and the node joining process, with some DKG messaging issues being pinpointed just today, which look like they could be solved by a sprinkle of AE, so we’ll be diving in there next.

Safe Network Primer

The Safe Network Primer has been updated :tada: and boy did it need doing. The last serious update was just over a year ago but so much has changed since then. In putting it together I’ve been struck by all the simplifications that have happened. It just feels a lot more streamlined and cohesive now, and from the point of view of its biographer, a fair bit easier to understand - although some parts still give me a headache.

If you follow the Thursday updates you’ll be familiar with many of these changes already, but the original idea of the Primer was to have everything in one place and to strip out a lot of the heavier stuff so that non-techies and semi-techies could understand how it all fits together. A lot of the early heavy lifting was done by @polpolrene and subsequent updates have been a lighter load as a result of his hard work.

So, what’s changed?

Well you can read for yourself at primer.safenetwork.org, but in summary:

CRDTs were just a twinkle in @dirvine’s eye at the time of the last update, but now they are a fundamental part of the Network and crucial in achieving consistency without requiring a heavy-duty, network wide algorithm. Using CRDTs has enabled a shift to Register and Multimap data types, which are a lot simpler and more flexible than their predecessors.

Another part of the process of maintaining consistency and stability is Anti-Entropy, which ensures that a section is stable before changes are made within it. This is fundamental to every operation, and fortunately it lends itself to pretty pictures too, so AE has a chapter of its own.

Then we have DBCs. These have been covered in great depth in previous updates, and are still a work in progress, so the Primer gives an overview of what they are and what they can do. I suspect they will merit a chapter of their own in a future update.

In combination, AE and DBCs have precipitated a change from ‘pay on GET’ to ‘pay on PUT’. Nodes are now paid for storing data rather than delivering it.

The Node Age chapter has also been updated. This is something the team has been working on recently and a few processes have been altered as a result of experiments.

And then there’s Safe credentials. While not part of the core network, a good UX will be an absolutely crucial ingredient if the Network is to take off. @JimCollinson covered current plans in some depth last week, so the Primer just covers the core concept of n-of-k credentials. Once again, this will need its own chapter once the plans are implemented.

A couple of things have been removed too. First, the authentication process is being rejigged and is likely to look rather different from the old authenticator, but it’s early days and rather than add to confusion, I shelved that part for now. Second, the chapter on the API was hopelessly out of date, and trying to understand the new one was too much for my semi-techie head, so that can come later.

As ever, there will be typos, there will be mistakes, there will be badly written wibble. @StephenC has done a sterling job of typo-riddance, but there will be more. There always are. If you spot any, let me know.

Thanks to the team, for their help and explanations. It’s been fun and I’ve been overawed to see how closely everyone works together and how quickly ideas take shape, but now I need to go and lie down.


Useful Links

Feel free to reply below with links to translations of this dev update and moderators will add them here:

:russia: Russian ; :germany: German ; :spain: Spanish ; :france: French; :bulgaria: Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!

71 Likes

First! Still can’t believe I actually managed to be first this time :smiley:

Congratz on the new hire & GL on the new job Andrew, you’re becoming part of a wonderful team for sure. Rest of the team great work (as usual), looking forward to the next update.

21 Likes

Wow. Second? That’s a first for me.

Looking forward to dig into node age changes on the revised primer.

17 Likes

Thirdddd yes!!!

15 Likes

Looks good https://primer.safenetwork.org/ :+1:

The must enable javascript is unfortunate default but the level of detail on that is great to see. Towards the bottom, the twitter link tempted for safenetworktech is pointed to the maidsafe twitter.

I keep plugging away at Tauri and it continues to impress as a good option for spawning Rust/React apps.

Random thought this week which I feel daft asking because perhaps is obvious… do xor addresses have an associated like a public key for a private key - given there complexity I wonder if they are of a kind that a public key can follow from. I guess there is some tool that can work a public key from private key that might answer this??

15 Likes

On
https://primer.safenetwork.org
I see “Last update: August 2020”.

11 Likes

It’s best to think of them as 32byte addresses. In our case, some are actually public keys (all nodes are).

You generally can, many of the algorithms used such as BLS allow you to query the private key for the public key.

14 Likes

Excited to read the primer @JPL! Thanks for all the effort you put into maintaining what you and Rene started together.

Congrats to the new hire! Great to have you here.

18 Likes

Try a hard refresh

13 Likes

Thanks so much to the entire Maidsafe team for all of your hard work! The updates are just fantastic. :racehorse:

13 Likes

Good to hear the primer is updated. I look forward to finding how GETs will work if they aren’t incentivised in the same way as before.

Still think it would be best to have a wiki for this tech, as intrinsically invite users to keep them up-to-date apart (along with a load of other benefits).

10 Likes

I was way to slow this week!!

Congrats to all the team and @JPL for all the hard work.

Great to hear @dirvine has been released from admin and now free to get back to the coal face :slight_smile:

19 Likes

Thank you team… we are willing you on week after week and enjoying watching the pieces fail into place.

Great work @JPL :clap:

19 Likes

Great news on all fronts!

Special thanks to @JPL for updating the primer! :clap: :clap: :clap:

I just gave a very brief look on the AE chapter, and it immediately cleared some doubts I was having. Going to dive in the Primer on better time.

18 Likes

Thanks Team and @JPL!
A note on the primer: I find this section a bit confusing:

" At no point does the Network ever store a user’s password."

And then a few sentences down:

“With Self-Authentication, the user generates his or her own credentials which are stored on the Safe Network in such a way that they are cryptographically obscured […]”

Edit: spelling in Consensus and Quorum, “recieving”

12 Likes

Thx 4 the update Maidsafe devs

Welcome Andrew James
Thank GOD that David can focus again on the tech

Sleep SAFE @jpl, thx for the primer update

Keep hacking super ants, thanks again for all your hard work
:stuck_out_tongue_closed_eyes:

17 Likes

I will read it carefully. I looked at it for a second and saw a multimap, CRDT type register, for Spentbook of DBCs. Thanks @dirvine

6 Likes

But @JPL in the Primer it says,

Running a Node in the Safe Network is called Farming because users look after the data until it is needed, at which point they may earn payment for their efforts

Am I understanding this right? Farmers get paid when you store the data, no longer via the old system. Is this new info or did I miss it in previous discussions.

6 Likes

I can’t find that - are you looking at the current version Nov 2021? Maybe need a browser refresh.

Yeah I think that’s wrong. @dirvine?

4 Likes