With activity ongoing in several disparate, if not unrelated, areas, we thought it would be a good time for a round-up of the overall state of the project covering the main strands of work, where they’re at and how they connect.
General Progress
@yogesh has been streamlining ElasticSearch/Kibana server setup to allow us to more easily monitor our droplets, and is finalising the docs for traceroute.
@joshuef has uncovered a process in sn_node that’s using up about 1/3 of CPU time and is probably unnecessary as we have implemented that functionality elsewhere, so he’s seeing what happens if we remove it.
@oetyng and @davidrusu have been leading the charge on conceptualising Genesis DBC creation and payouts, which the team have been fleshing out this week (see below), while @anselme is in the final stages of refactoring DKG with multithreading removed for simplicity, as well as removing timeouts (see below).
Activity Breakdown
Membership, AE and consensus - joining the dots
Membership and Anti-Entropy (AE) are about the system keeping up to date, while consensus is used to decide on decisions that change the makeup of a section, and/or the data it holds.
Every system message that is sent across the network contains the section key. The client sends what it thinks is the current section key when it firsts communicates with the section, and if it is out of date the section elders send it the current key in the section authority provider (SAP), which also contains a list of the elders in the section, so it can update its records before trying again. This is AE.
We’re now looking at adding Membership into this information exchange too, to close the gap between what nodes understand and vote upon, and what is shared within the SAP. Membership, you may recall, is how elders keep track of the adults and other elders in their section, so they can handle new joins, splits, churn and promotions.
Particularly when we’re looking at payments to individual adults, it will be quicker if we know where they are - adults churn more quickly than elders and may have been relocated to another section, or promoted. In addition, adults have a node age, and their payment will be proportional to that. So, we’re looking at how to include some information about the current generation of the Section adults in messages about storage and payments.
Adults can claim their payments at any time, even if they have moved sections, etc, because the DBC is reissued to their public key and they can prove they were in the section at that particular time, as part of that particular generation. So this measure is more about reducing the number of messages flying around.
As well as bringing Membership into the information exchange process, we have also introduced a consensus mechanism into the way it works. Membership uses consensus to accept and remove members in a section in a synchronised way. It makes sure all elders have the same view on the order of the joins and leaves in the section. This is to prevent inconsistencies in each elder’s member set since the member change log is an append-only chain where consensus with supermajority is required to add a new item.
Calling time on timeouts - increasing certainty on DGK rounds
In general DKG doesn’t use consensus, instead DKG participants make sure no one is cheating by sending out signed messages and once they obtain all the other’s messages they send out a set with everybody’s messages and make sure everyone signed the same set. If there’s a single inconsistency, they stop participating in this DKG round, it is considered failed and we never reach any form of consensus on this round. This makes sure a DKG round with cheating members never reaches termination, we don’t want cheaters as elders after all.
Many DKG rounds can happen concurrently, and it is a race between them to reach termination (or die out). When we’ve got a completed DKG, handover consensus can begin and decide on the next elders. Since handover uses consensus it makes sure only one DKG outcome is selected, so we can have multiple concurrent DKG completions but only one of them will be selected for handover.
Distributed key generation (DKG) is how elders create the section key in a secure manner where each elder only knows their own keyshare. In testing, we have found a few instances where DKG was inconsistently failing and retrying due to the use of timeouts when waiting for messages.
We are revamping our DKG with a cleaner, timer-free implementation. For example, if the oldest seven nodes in a section are not the current elders, they will attempt a DKG run to create a new section key. If not all elder candidates participate, we allow this DKG to not terminate, and this is fine because we don’t want inactive nodes to become elders. We can wait for another membership churn to happen again and trigger another DKG. We embrace having concurrent DKGs racing to become the new elders. Handover consensus will decide which one wins ultimately.
@anselme has spent a lot of time cleaning up DKG, removing unneeded message types from the DKG code, and integrating the new DKG. It’s looking a lot better now and we will soon be able to test it.
Node changes
We’ve made some changes to our sn_node binary. Now you need to pass the contacts file path to each node that is joining a network (--network-contacts-file). Rather than the node expecting contacts to be found somewhere, other tools are in charge of making sure the path is provided to the node.
We’ve updated our sn_launch_tool to do this, so if you are using it (or the CLI which makes use of it), the tool now provides each joining node with the path to --network-contacts-file.
--network-contacts-file is simply a file that gives a joining node or bootstrapping client the information it needs to join the network. This information is found in the SAP, and the SAP part of PrefixMap.
For new nodes we use the Genesis node’s PrefixMap file to provide this information (it’s located at ~/.safe/node/local-test-network/sn-node-genesis/prefix_map), but the tool also copies this file to the location where then the clients/CLI will find it by default (~/.safe/prefix_maps/default).
We can use the same PrefixMap to allow both a node to join a network and a client to bootstrap to the network, as it contains the SAPs they can connect to. From their perspective, it’s the same as a network contacts list.
In the future, clients and nodes may support other types of files and formats. Whatever changes there, they will always need a network contacts list to bootstrap/connect to a network.
As described in a recent update, PrefixMap recently evolved from a simple file that maps section prefixes to the current SAPs, to a more complex structure that includes the complete section key DAG/tree - so this file will likely be renamed soon.
Finalising network payout of the 70% rewards
70% of SNT will ultimately be created by the network in the process of clients uploading data.
When clients upload data they must first pay the sections where the data will be stored. These payments are shared between the nodes in each section (most likely the elders and adults storing the given data, pro-rata based upon age.
However, a certain proportion (to be decided) of payments will also result in the minting of new SNT in the form of a DBC. The total value of the new DBC will be the same as the payment amount. The new DBC will be shared between all nodes in the section.
To select those payments that will result in a new SNT, a test is carried out, for example some kind of match involving a hash of the payment and the section address prefix. The probability of passing the test will depend on the desired rate of SNT creation (previously the farming rate).
Elders check every payment to see if it passes this test. The incentive for this check (which is fast and requires minimal resources) is the chance to earn SNT. Most payments will not pass the test and will proceed as normal, but when there is a match, the elders use that payment to mint a Genesis DBC. This Genesis DBC is brand new and increases the total amount of SNT available to spend. Its total amount is the same as the amount of the upload payment, and its value is split between all nodes in the section, weighted by node age.
For a DBC to be spent it must be reissued to the public keys of the new owner(s). For this, we pass the Genesis DBC back to the client and have it reissue new DBCs to the nodes - plus one DBC to itself as an incentive. The amount of this incentive needs to be worked out.
Once the DBCs have been reissued, the recipient nodes are notified and can now spend their DBC as and when they wish to.
Minting new SNT only occurs with new data storage. SNT transfers do not result in a new DBC being created.
We will probably change the name Genesis DBC as there are many such DBCs so the word ‘genesis’ is confusing.
Finalising farming/paying for storage
With the concept of token emission coming along nicely, we can also see how our plans for data payments fit and happily we see that they fit rather well. It’s a flow we’ve spoken about previously, with clients requesting quotes from the section for a given set of data, and then making the appropriate payments for this. What’s nice is that this is inherently batched, so one set of DBC reissues can work for any amount of data.
This process gives us a receipt with which we can validate a given piece of data has been paid for, and that we can attach to the data effectively making it “Network Valid”, ie, it can be re-uploaded without having to pay for it, for example.
This receipt will then form a part of the network payout process as outlined above.
Using gossip rounds for intra-section membership and inter-section updating of PrefixMap
We introduce a gossip approach to new SAP creation to ensure the network is always reachable from all sections. This mechanism ensures information is spread logarithmically (very fast and efficient). As an elder receives a new SAP from any part of the network, it will select two random elders from any section and forward the message to them. AE will ensure that nodes update in both directions and also terminate the gossip round. This is inter-section gossip and only concerns elders.
Internal to the section, we have membership changes that include adults. These changes will follow the same pattern to ensure all members are up-to-date with any membership changes. This is intra-section gossip.
Useful Links
Feel free to reply below with links to translations of this dev update and moderators will add them here:
Russian ; 
German ; 
Spanish ; 
French; 
Bulgarian
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!
still looking forward to the next testnet/comnet it’s been a while 
