Update 14 July, 2022

There are no secrets in open source. As some of you spotted, the old config file for connecting is no more, replaced by PrefixMap. So what’s going on? @roland is here to explain more.

Another thing that’s headed for the exit is the unmaintained and buggy sledDB. But you also knew that already. @yogesh explains what is replacing it, and why we are avoiding the database option for now at least.

General progress

The whole team is engaged in simplifying the code by stripping out unneeded multithreading. We’ll refer you to @joshuef’s post here to explain why. The good news is, removing this cruft has already been highly positive in terms of performance.

@bzee has developed a new tool that lets us inspect a chunk and see where it’s stored on the network and with which adult. We hope to get this out to the community soon for testing.

Also on observability @joshuef has been digging into heaptrack, which profiles memory use. Memory is generally much better now but there are still occasional spikes.

@bochaco and @chriso are ploughing ahead with DBC integration, and we hope to have the next iteration ready for community testing very soon - thanks a lot for your help with this so far guys :bowing_man:

And @heather_burns is in the media again, this time talking about the UK’s Draconian (and now thankfully delayed or possibly even abandoned) Online Safety Bill.


We have replaced the old config file with PrefixMap, partly as a simplification of the codebase, partly to optimise the connections and messaging. Config contained IPs and ports of nodes to connect to, plus the genesis key of the network. The PrefixMap contains all of the above plus the section authority provider (SAP), which provides a list of all the current elders in a section and the current section key. Moreover, since the genesis key can be provably linked to the section key in the SAP, the client can verify it is in the right place, in the right network and everything is up-to-date (via anti-entropy).

This also allows the client to home in on the nearest elders straight away, so we can avoid hammering all the elders in a section for every request.

For new nodes joining the network, the PrefixMap provides an overview of the network topology which it can update as it needs to, again, skipping a few interim steps.

@roland explains the background.

The config file, which contained the network’s genesis key and some bootstrap IP addresses was the way we connected to any public/private testnet. But at the same time, we were also maintaining the PrefixMap, a structure which contained the latest SAPs of the sections we are aware of. Since the SAPs contain the IPs of all the elders, we can use this to bootstrap to the network instead.

This is advantageous over the config file because we are not hammering the fixed set of nodes used to bootstrap to the network. Instead we are able to query the PrefixMap to get the closest SAPs to a given XOR-name and use these to bootstrap into the network. This balances the load while bootstrapping.

We also plan shortly to move the SectionChain into the PrefixMap which will resolve some issues in the client, which currently has to go through the anti-entropy (AE) update process to ensure it’s up-to-date, which can add up to a huge amount of AE messages. This is a work in progress.

How does this change the Comnets?

These changes should not affect the running of local and community testnets (Comnets), although you will notice a few small differences.

  • The only new process is that we will be using the PrefixMap to connect to local/public testnets instead of the node_connection_info.config file.
  • We use the Genesis Key of the network to name the PrefixMap files, hence they might look something like this PublicKey(0123..4567)

Connecting to the Comnet:

  • Make sure you do not have the old config file ~/.safe/cli/config.json and old prefix maps inside ~/.safe/prefix_maps/. You can clear out the entire .safe dir using rm -r ~/.safe/
  • You can add the Comnet to your networks list by providing the URL/path of its PrefixMap, as provided by whoever sets up the Comnet (e.g. curl https://).
    Use the following CLI command to do so, networks add my_comnet "<path/url>"
  • You can then switch to the added network using networks switch my_comnet

Running local baby-fleming testnet:

  • Make sure you don’t have the old files, as stated above.
  • From the CLI, install the latest version of sn_node using the command node install
  • Use killall sn_node to stop any running instances of sn_node
  • Run the testnet using node run-baby-fleming
  • You can now use networks command to view all the networks that are available to you and use networks switch network_name to switch between them

Note: The CLI uses the ~/.safe/prefix_maps/ directory to update the list of networks and select the current network it’s connected to, hence renaming files inside this directory can lead to undesirable results.

Ditching sled

The final piece in the sled removal process, storing registers, is now almost in place and the buggy and unmaintained sledDB will shortly be fully replaced with our own in-house disk storage. As we said last week, this is a keep-it-simple implementation with no bells and whistles that performs on par with the other alternatives, while freeing us from an external dependency at this stage. There’s nothing to prevent us looking at other databases down the line when we need one.

We’re confident that this change will remove one of the common causes of testnet failure. But before we get too excited, it’s probably not the only one, and we’re looking into Membership to see if perhaps chunks are sometimes being sent to the wrong adults.

Useful Links

Feel free to reply below with links to translations of this dev update and moderators will add them here:

:russia: Russian ; :germany: German ; :spain: Spanish ; :france: French; :bulgaria: Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!


Never been first before


Place #2 maybe now read.

Thx dev later

Thx Maidsafe devs for all your hard work.

Really love the progress

Keep hacking super ants


Ok I’ll take bronze today. Hoping for a testnet soon.


Thanks so much to the entire Maidsafe team for all of your hard work! :racehorse:


Lucky 7 !!!

Thanks for a most informative update and I am 100% agog to get testing these latest changes.
Thank you to all involved.
Not forgetting @Heather_Burns Get tore in amongst them, Heather!!


If I remember correctly, sn_node was not using files to start, instead it used command line, like ... --hard-coded-contacts "[ \"\", ... \" ]" --genesis-key 8041e439a6b363630d...c0a47af94efa6c43bcab .... Did this mechanism changed? Where node will get its startup information after PrefixMap change?


Nice update!

Keep hacking super ants … we got yer back!


I concur with this. Previous config file still worked after many network changes (new, deleted or relocated nodes, split sections, …) as long as at least one node in the config was still alive, even if it didn’t belong to its original section anymore.

Is it still true?

I am afraid now that a static copy of PrefixMap becomes useless right from the first section modification. Can someone clear my concerns about this?


Great works, Thx all… I have been reading all updates since signing up.


Hard coded contacts should not be necessary with a prefixmap (nor should it have been w/ a node config file). It was a way to start without that info.

Prefixmap is useful so long as there are live nodes, same as the config file.

The only thing that should reallistically have changed here is that instead of us 1) having a config file that is only ips, (and maintaining and writing a prefixmap apart), we 2) use the prefixmap for the same functionality as prev, since it has all the same IPs AND some network knowledge to be able to verify that nodes are who they say they are and which sections they are in.

It’s like an organised version of the node config info.


So now, without --hard-coded-contacts, sn_node will search for prefixmap file in some default location and use it if it found it?


That’s what I wanted to hear. Thanks.


Great progress! So sledDB was really the main cause of the testnet failures?


Yeh this was the behaviour before w/ the config file. It should be unchanged (cc @roland to confirm!)


Yes, you’re right. We read the prefix map from ~/.safe/prefix_maps/default file. To give a little extra context, default is a hardlink (symlink requires elevated access in windows) pointing to a prefix map file. It’s used to select the current network when multiple prefix maps (networks) are present. We went in that route route to avoid providing an env variable / arg everytime you start a node/client.


I’m using NSSM to launch node, there are many benefits for such method: there is no console window, which I can accidently close, I can set environmental variables to control logging levels, redirect console outputs to file and I can start and stop such service with default OS mechanisms. Maybe I forgot some.
The problem is that services are launched with “system” user. So node binary will never find correct settings.
Are there any other methods of launching node without console window? I did not looked for different methods because was satisfied with NSSM.


Steady progress. Looking forward to an official testnet soonish!

@19eddyjohn75 is your profile pic from an ODB album? Love ODB.


I don’t think this was ever a conclusion made.
AFAIK there was the post 3000 ops hanging, and then we saw it was unmaintained and decided we might just as well switch it out. Also it’s a rather large code base. So that’s about it I think.

Its batch-ops was far outperforming all other alternatives though. So that would have been nice to have. Something that could be extracted/found later perhaps, if needed.