Update 11 August, 2022

With activity ongoing in several disparate, if not unrelated, areas, we thought it would be a good time for a round-up of the overall state of the project covering the main strands of work, where they’re at and how they connect.

General Progress

@yogesh has been streamlining ElasticSearch/Kibana server setup to allow us to more easily monitor our droplets, and is finalising the docs for traceroute.

@joshuef has uncovered a process in sn_node that’s using up about 1/3 of CPU time and is probably unnecessary as we have implemented that functionality elsewhere, so he’s seeing what happens if we remove it.

@oetyng and @davidrusu have been leading the charge on conceptualising Genesis DBC creation and payouts, which the team have been fleshing out this week (see below), while @anselme is in the final stages of refactoring DKG with multithreading removed for simplicity, as well as removing timeouts (see below).

Activity Breakdown

Membership, AE and consensus - joining the dots

Membership and Anti-Entropy (AE) are about the system keeping up to date, while consensus is used to decide on decisions that change the makeup of a section, and/or the data it holds.

Every system message that is sent across the network contains the section key. The client sends what it thinks is the current section key when it firsts communicates with the section, and if it is out of date the section elders send it the current key in the section authority provider (SAP), which also contains a list of the elders in the section, so it can update its records before trying again. This is AE.

We’re now looking at adding Membership into this information exchange too, to close the gap between what nodes understand and vote upon, and what is shared within the SAP. Membership, you may recall, is how elders keep track of the adults and other elders in their section, so they can handle new joins, splits, churn and promotions.

Particularly when we’re looking at payments to individual adults, it will be quicker if we know where they are - adults churn more quickly than elders and may have been relocated to another section, or promoted. In addition, adults have a node age, and their payment will be proportional to that. So, we’re looking at how to include some information about the current generation of the Section adults in messages about storage and payments.

Adults can claim their payments at any time, even if they have moved sections, etc, because the DBC is reissued to their public key and they can prove they were in the section at that particular time, as part of that particular generation. So this measure is more about reducing the number of messages flying around.

As well as bringing Membership into the information exchange process, we have also introduced a consensus mechanism into the way it works. Membership uses consensus to accept and remove members in a section in a synchronised way. It makes sure all elders have the same view on the order of the joins and leaves in the section. This is to prevent inconsistencies in each elder’s member set since the member change log is an append-only chain where consensus with supermajority is required to add a new item.

Calling time on timeouts - increasing certainty on DGK rounds

In general DKG doesn’t use consensus, instead DKG participants make sure no one is cheating by sending out signed messages and once they obtain all the other’s messages they send out a set with everybody’s messages and make sure everyone signed the same set. If there’s a single inconsistency, they stop participating in this DKG round, it is considered failed and we never reach any form of consensus on this round. This makes sure a DKG round with cheating members never reaches termination, we don’t want cheaters as elders after all.

Many DKG rounds can happen concurrently, and it is a race between them to reach termination (or die out). When we’ve got a completed DKG, handover consensus can begin and decide on the next elders. Since handover uses consensus it makes sure only one DKG outcome is selected, so we can have multiple concurrent DKG completions but only one of them will be selected for handover.

Distributed key generation (DKG) is how elders create the section key in a secure manner where each elder only knows their own keyshare. In testing, we have found a few instances where DKG was inconsistently failing and retrying due to the use of timeouts when waiting for messages.

We are revamping our DKG with a cleaner, timer-free implementation. For example, if the oldest seven nodes in a section are not the current elders, they will attempt a DKG run to create a new section key. If not all elder candidates participate, we allow this DKG to not terminate, and this is fine because we don’t want inactive nodes to become elders. We can wait for another membership churn to happen again and trigger another DKG. We embrace having concurrent DKGs racing to become the new elders. Handover consensus will decide which one wins ultimately.

@anselme has spent a lot of time cleaning up DKG, removing unneeded message types from the DKG code, and integrating the new DKG. It’s looking a lot better now and we will soon be able to test it.

Node changes

We’ve made some changes to our sn_node binary. Now you need to pass the contacts file path to each node that is joining a network (--network-contacts-file). Rather than the node expecting contacts to be found somewhere, other tools are in charge of making sure the path is provided to the node.

We’ve updated our sn_launch_tool to do this, so if you are using it (or the CLI which makes use of it), the tool now provides each joining node with the path to --network-contacts-file.

--network-contacts-file is simply a file that gives a joining node or bootstrapping client the information it needs to join the network. This information is found in the SAP, and the SAP part of PrefixMap.

For new nodes we use the Genesis node’s PrefixMap file to provide this information (it’s located at ~/.safe/node/local-test-network/sn-node-genesis/prefix_map), but the tool also copies this file to the location where then the clients/CLI will find it by default (~/.safe/prefix_maps/default).

We can use the same PrefixMap to allow both a node to join a network and a client to bootstrap to the network, as it contains the SAPs they can connect to. From their perspective, it’s the same as a network contacts list.

In the future, clients and nodes may support other types of files and formats. Whatever changes there, they will always need a network contacts list to bootstrap/connect to a network.

As described in a recent update, PrefixMap recently evolved from a simple file that maps section prefixes to the current SAPs, to a more complex structure that includes the complete section key DAG/tree - so this file will likely be renamed soon.

Finalising network payout of the 70% rewards

70% of SNT will ultimately be created by the network in the process of clients uploading data.

When clients upload data they must first pay the sections where the data will be stored. These payments are shared between the nodes in each section (most likely the elders and adults storing the given data, pro-rata based upon age.

However, a certain proportion (to be decided) of payments will also result in the minting of new SNT in the form of a DBC. The total value of the new DBC will be the same as the payment amount. The new DBC will be shared between all nodes in the section.

To select those payments that will result in a new SNT, a test is carried out, for example some kind of match involving a hash of the payment and the section address prefix. The probability of passing the test will depend on the desired rate of SNT creation (previously the farming rate).

Elders check every payment to see if it passes this test. The incentive for this check (which is fast and requires minimal resources) is the chance to earn SNT. Most payments will not pass the test and will proceed as normal, but when there is a match, the elders use that payment to mint a Genesis DBC. This Genesis DBC is brand new and increases the total amount of SNT available to spend. Its total amount is the same as the amount of the upload payment, and its value is split between all nodes in the section, weighted by node age.

For a DBC to be spent it must be reissued to the public keys of the new owner(s). For this, we pass the Genesis DBC back to the client and have it reissue new DBCs to the nodes - plus one DBC to itself as an incentive. The amount of this incentive needs to be worked out.

Once the DBCs have been reissued, the recipient nodes are notified and can now spend their DBC as and when they wish to.

Minting new SNT only occurs with new data storage. SNT transfers do not result in a new DBC being created.

We will probably change the name Genesis DBC as there are many such DBCs so the word ‘genesis’ is confusing.

Finalising farming/paying for storage

With the concept of token emission coming along nicely, we can also see how our plans for data payments fit and happily we see that they fit rather well. It’s a flow we’ve spoken about previously, with clients requesting quotes from the section for a given set of data, and then making the appropriate payments for this. What’s nice is that this is inherently batched, so one set of DBC reissues can work for any amount of data.

This process gives us a receipt with which we can validate a given piece of data has been paid for, and that we can attach to the data effectively making it “Network Valid”, ie, it can be re-uploaded without having to pay for it, for example.

This receipt will then form a part of the network payout process as outlined above.

Using gossip rounds for intra-section membership and inter-section updating of PrefixMap

We introduce a gossip approach to new SAP creation to ensure the network is always reachable from all sections. This mechanism ensures information is spread logarithmically (very fast and efficient). As an elder receives a new SAP from any part of the network, it will select two random elders from any section and forward the message to them. AE will ensure that nodes update in both directions and also terminate the gossip round. This is inter-section gossip and only concerns elders.

Internal to the section, we have membership changes that include adults. These changes will follow the same pattern to ensure all members are up-to-date with any membership changes. This is intra-section gossip.

Useful Links

Feel free to reply below with links to translations of this dev update and moderators will add them here:

:russia: Russian ; :germany: German ; :spain: Spanish ; :france: French; :bulgaria: Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!


Gold today. Feeling lucky.


Happy to see the Maidsafe family pushing trough and closer every day to the final network!

Love for everyone


Bronze and boy this is a big update!!!


Keep chuggin‘ along.
I lost track a long time ago but seems to be coming together. Well done.


Think I see what you did there


Interesting developments here. This appears to be one of the automated ways of avoiding having the Foundation hang onto lots of SNT and finding ways to issue it over time. Is that correct?

Not covered in the explanation is:

  • how to ensure newly minted SNT never exceed the promised cap.
  • how a ‘bad’ section is prevented from minting as much as it wants, or at least is limited to an amount that makes it not worthwhile taking over a section.

I’m sure these have been thought of but am not clear myself, and can’t immediately think of how to do this :man_shrugging:t2:

Oh, are we close to the point it makes sense to restart community tests?


Mother of updates :grimacing:


Thanks to all for a comprehensive update of where we are at across the entire project.

@happybeing You ask the questions I wish I had thought of myself. Enjoy the Pimms… I myself have polished off 2 x 32Gb Weihenstaphaner and many wee bratwurst.

Chomping at the bit for a new testnet.


Looks that way to me and clever manipulation of the difficulty of the test may allow flexibility in the money supply for SNT. Even better if an algorithm can be devised so its automated and does not have to become the responsibility of the Foundation.


Where are we at with the governance work, white paper revision, and Swiss Foundation?

When was the last official Testnet and when do we expect the next one?

Would Maidsafe be willing to consider an outside look at the strategies and approach?
A proper unbiased cost benefit analysis of the project from all angles so that me might improve progress and tighten management efficiency.

I’m not trying to be negative here. I really would like to see what other professionals, capable of understanding what is trying to be achieved, think about how this is being approached and managed. I’d like to see some outside looking in constructive criticism. I think that without a fresh outside look at all of it we can get offtrack as to how things can be achieved. Again I hope this does not offend anyone in anyway.

What do the folk at Maidsafe think? What does the community think?
With all this being said I’m really looking forward to the next official testnet.


So clients pays to nodes, which store client’s data for the first time.
And when all these nodes go offline, they transfer data to other nodes, but this time for free, right?

But they results in using storage of nodes, right? Also for free.
I understand that it is not possible to pay for payments and that used storage will be relatively small, but it still makes less space for paid data.


Absolutely. It’s one we feel looks nice simple and secure. Now we are in the weeks of the distribution and payment algorithms. All coming on well.

They will only exist in certain addresses and those addresses are limited.

The algorithm to allow minting also has the requirement that it is not “minable” and is computationally prohibitive. We are looking at rewards at a level of less than the payment and the payment (which includes the elders and adults wallets) signature is part of the “mint” process. As the sig is not known up from and not guessable, that covers point 1, also though as the reward is less than the payment we go a long way to cover part 2 (too expensive to brute force).

The key parts to finalise are

  1. The actual minting algorithm (but we will likely base on payment sig plus reward less than payment amount). We are looking at making the non minable thing not matter about which section mines these, but we also are considering the mint section being different form the payment section. We have options to iron out but looks good.
  2. The payment balance algorithm, i.e. what does the first chunk cost and how does the next one get costed? We already have a bunch of work done there with quotes and payments, but we just need to tie it firmly down.

Soon, but not yet


Happening and close to submission to the authorities.

soon I hope, I don’t have date of last one.

Of course

By whom though?

Understood man, no worries.

YEs, mostly. Clients pay for the storage authorisation (section sig) and then store at any time as their data has the network authority to be stored. That means we can reupload any lost data for free, if we ever lost data that is.

Hopefully, the above helps answer this point.


Forgot what day it was! Yeeehaaa :cowboy_hat_face:


I’m interested in what happens with transaction data, when nodes goes offline.
Client’s data paid (which is the limit), then cloned as many times as needed.
Transactions are not paid, but still needs to be reuploaded when nodes go offline? So what is limiting their amount?
Sorry if my question is not correct by itself, I did not watched development closely lately.


Wish I knew what all this meant :woozy_face: still looking forward to the next testnet/comnet it’s been a while :+1:t2:


Could we be vulnerable to a dust attack here?

presumably “weeds” - typo #1

I’m guessing “up front” here -typo #2

As messages from @dirvine always contain one and only one typo, how do we know this is a genuine message and that the real David Irvine has not been kidnapped?

Yours paranoically, Southside


Thanks so much to the entire Maidsafe team for all of your hard work! :racehorse: