Update 06 October, 2022

Most of the team are occupied in refactoring Authority, the code that tells us which actors can do what actions and what level of authorisation they require. We hand over to a young chap called @dirvine to explain further.

General progress

Other fixes are ongoing too.

@anselme has fixed an error in the sn_sdkg crate that involved making vote handling recursive. With this, a node doing DKG on its own would recursively reach termination upon handling their first vote in one call! (Not good).

@bochaco is continuing to debug communications between nodes, together with @qi_ma, and also correcting issues found with single-threaded commands and queries/queries in the continuous integration (CI) pipeline.

Meanwhile @bzee and @chriso continue to examine the workings of Quinn, the Rust implementation of the Quic protocol for establishing and maintaining connections between peers, with a view to refactoring qp2p.

Refactoring Authority

At the moment we have per-message Authority and that authority is split between node, client, section and section_part. It’s both confusing and bug prone. More recently, we have been digging into Authority, what it means and what it buys us.

The first step in this clean up is to remove message authority and focus on Operations. By that we mean the operation in the message is where we want to check authority and also be able to apply it directly to data. This part saves us, 1: having the authority (signatures) twice and 2: ensuring any data mutation (including storage) has the relevant Authority.

To break this down a little. To store chunks or a container (a register) we require that clients pay. When they have paid, each elder returns a part section signature. The client then aggregates the signature to create a section signature. That section signature stays with the data and makes the data NetworkValid. So there is no sense in signing the whole message, just the part that says Store ABC where ABC is the data name. So each piece of data now has SectionAuthority and we don’t care what message or format we receive the signed operation in.

So what, some may ask. Well in essence what we are doing now is quite exciting from several points of view:

  1. Less code again
  2. More concrete data types
  3. Removing the requirement of any message signatures (caveat in a minute).
  4. And the big one … see below

The astute may at this point notice a big win for decentralised networks here. With SectionSigned data elements and the SectionTree from a few updates back we have network agnostic valid data. i.e. any network that also trusts our SectionTree, even the majority of it (in the case of a mega hack) can validate and trust the NetworkValid data. This feature allows data to move to other networks, or even Safe II, who knows?

However, we do have nodes sign messages, and we think of that as Evidence. Nodes message each other and clients with data or suggestions (like node X is dead or node Y wants to join etc.) and we treat their messages as infrastructure commands (i.e. transient) or client service commands (give me data). As these messages lead to consequences on the network we require the nodes to behave. If they are found to misbehave we must have irrefutable proof of bad behaviour so we can punish them. The signed message gives us irrefutable proof of a node’s behaviour.

Taking all this into account the current refactor is rather quick, it’s only a few days, but it buys us a ton of simplicity and at the same time gives us more flexibility in storing/republishing data. Imagine you find some old data on your node, it’s not on the network but should be. Then you should not need to pay to store it, you should just say here is SectionSigned data, you MUST store it. It’s the network’s contract with the planet.

Not only that, the clients who pay to store, actually pay to get SectionAuthority on the data, making it NetworkValid. Now this means they can add the section authority to their data and choose to store it at any time they wish. They can also republish it if the network somehow failed them.

There are 3 or 4 of us taking this task this week and already, we are seeing improvements in clarity and possibly finding some weird bugs as we remove the old code from way back (messaging still has some very old code there). It’s a really neat refactor that will make bug hunting so much more simple as it makes the network operations more simple. It also makes the network simpler to explain.


Useful Links

Feel free to reply below with links to translations of this dev update and moderators will add them here:

:russia: Russian ; :germany: German ; :spain: Spanish ; :france: French; :bulgaria: Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!

54 Likes

Early update first :+1:t2:

16 Likes

Thanks so much to the entire Maidsafe team for all of your hard work! :racehorse:

14 Likes

third? 20charcharchar

yeah third! now to read!

wow I am blown away by this update!!! Go Safe! Go Safe!

16 Likes

The SectionAuthority changes sound very elegant technically. :clap: I think this often happens when you manage to take something complex and find a simpler solution, but is often very hard so :clap: again.

Can you summarise what ‘data’ is in this context, for example for immutable data is it a chunk?

For mutable?

And does this also wrap up anything else with the data such as permissions and ownership (now or in future), or is that entirely separate?

Another encouraging update, thanks team. :pray:

19 Likes

You guys are so fast… Fifth. Nice work @maidsafe

13 Likes

Sixth I’m losing my touch. Another great update

Already thinking about safe II this is great as geting from Fleming to Maxwell on a running network always sounded like trying to change the piston rings on a running engine to me.

Great work team :slight_smile:

17 Likes

First! (…first to post 7th on an update at this time of day)

16 Likes

Its a beautiful thing being built, pitty that it is so hard to see.
Taken for granted is what it will be.

20 Likes

To aspire to poetry like Josh, I fail,
Fuckyou haiku, Canny dae yu.

14 Likes

sounds like music!

14 Likes

Nice update!

Wondering if this is paving the road for Maxwell (network upgradability) :thinking: … so getting us a lot closer to beta.

Great work ants! Keep a hackin’.

19 Likes

Sorry folks, at a long meeting this afternoon. But yes immutable data == chunk and mutable data == register.

Yes the register container has the policy and owner contained in it. So you pay for the empty container and it defines who can update it.

Kinda, it means even if everything totally breaks it can restart with all the old data. So no upgrades yet, but looking at total network failure resistance and recovery, should we ever need it. It would be a freak world event or a significant software bug to make that happen.

The bottom line is we are looking for secure, simple and concrete handling of network-valid data.

The nice part is this is again, making it easier to fork, upgrade or compete with Safe. We all win if we all wish the vision to happen.

25 Likes

Thx 4 the update and all your hardwork Maidsafe devs

Great to see the work to make the network simpler to explain.

Are forks like with blockchains different entities (bitcoin/lightcoin) or can SAFE forks data interoperate with persistence of the existing SAFE data on the new fork?

Keep hacking super ants

11 Likes

I think either is possible. The persistence of data, though, is now going to be possible.

13 Likes

Hihi can’t wait for a Deepmind fork of SAFENET, giving you super ants a well deserved time out for celebration… :sweat_smile:

Maybe can’t hurt as a community to in the future approach them to optimize SAFENET, they seem to be solving a lot of problems :exploding_head:

9 Likes

Spectacular. To infinity :+1:

12 Likes

@dirvine Just curious, I remember a couple years ago it being mentioned that we had about 2 years of runway left, how much runway do we have left at the current burn rate? Just curious about the longevity of the project and whether or not we have enough resources to get the network launched and out there!

3 Likes

We are fine with our runway at the moment. Well in excess of 2 years.

18 Likes