It’s time to pull together all those mysterious acronyms, revisitings and baffling announcements that have peppered the forum over the last couple of extraordinary weeks to give you - and us - a chance to reflect on what the hell’s going on.
General progress
The entire team has been heads down in libp2p, Kademlia and related topics this week, so there’s less to itemise than usual.
@bzee is focused on NAT traversal capabilities in libp2p for hole punching, @oetyng is currently integrating DBCs into the new system. He just finished a first round of improved documentation for DBCs, and clarifying naming and increasing type safety to reduce cognitive load. That’s also removed the bearer DBC concept and a ton of API + code for that. Next is to remove the Elder signatures from DBCs, as that won’t be used any more.
@roland is tidying up the code and digging into data republishing along with @chriso. And @bochaco has just gotten the chunk/register code ported across. Once republishing is in place we’ll be firing into some heavy internal testing.
@qi_ma is experimenting with the libp2p crate and playing with testnets from the stableset_net code base, investigating the chunk upload/fetch flow on top of that.
More about libp2p et al
Thus far, our plans for libp2p have emerged in bursts of excited chatter, so it’s time to put it all in one place to give a better idea about how it fits with DBCs, Sybil protection and all the rest.
First the why. In short, as we’ve modified DBCs and data to be backed by DBCs, that’s opened up the door to a simpler approach to the underlying network. And it’s this that allows us to start stripping out some of the more complex parts of the network, meaning a lot less for the team to worry about, which is a huge win. Our team is dedicated but small, and now they can focus on the value-add.
Libp2p itself is used by Filecoin, Eth and Avalanche which not only means there are lots of eyes on the code, but also gives us the opportunity for collaborations and interoperability down the road.
Libp2p now helps out with QUIC - the library that manages connections between nodes and one of the main struggles we were having with messaging. Perhaps equally importantly, it can handle hole punching. How it does that is quite complicated but it’s covered here allowing nodes that are behind a firewall or a home router to connect in a p2p fashion. Most people should be able to connect without having to mess around with port forwarding. Hopefully over time, the crate will improve to support even more routers than it does now.
Then there’s denial of service (DoS) protection, where it has controls to limit the number of active connections between nodes and, optionally, rate-limit inbound connections.
As many of you will know, Safe is based on the Kademlia distributed hash table (DHT) which allows XOR routing and content addressable storage. This is supported by libp2p too, if not quite out of the box, at least at a level we can work with and enhance. Importantly, it implements the Kademlia feature of refreshing nodes, meaning that dead nodes don’t remain in the routing table for very long.
This refreshing is important in view of another attack vector: people generating billions of keys offline then trying to brute force their way in as new Sybil nodes. As a test for this, we are looking at implementing a gatekeeper called verifiable random functions (VRF).
A VRF takes an existing key + input data and outputs a new public key + proof it was derived from the old key plus input data. So, the node that wants to join creates a key and finds the closest group of nodes to that key. The VRF takes the key plus the IDs of this group and outputs a new key pair plus a proof. This new public key plus the proof is then sent to that close group to join, whereupon the nodes in that group can check it was generated from a valid old key and the IDs of the nodes close to that old key.
Since nodes are churning rapidly (much faster now), keys must be generated and used quickly. They have a very limited lifespan, thus mitigating the ‘offline key generation’ attack. VRFs are an innovation by Algorand.
Next on the list of goodies is decentralised IP-based public key infrastructure (PKI), which provides protection against man-in-the-middle attacks. As mentioned a couple of weeks back, this effectively lets us use our group consensus mechanism as a way of becoming our own CA, certifying messages as secure.
Our path towards specialised nodes such as for archiving and audit suddenly becomes a lot simpler, as does the tricky problem of upgrading the protocol, with various libp2p functionalities helping us here.
So libp2b and other projects like rustls and VRFs take care of a lot of the hard low-level networking problems, meaning we can focus on the real innovation on top, notably our use of DBCs.
This is where Safe stands out with a massively parallel and scalable transaction throughput. Because Safe is a data network first and foremost, we use DBCs to secure the data as well as paying for storage, in this case with the recent addition of the “reason” field which stores the name of the data it paid for. The data also stores a link back to the DBC to complete the circle.
We also pass a lot of responsibility for security to the client, with client signatures over DBCs, and their verification securing all the data on the network. In doing so we have removed network sigs entirely, simplifying things and securing against crypto-cracking quantum computers when they come along.
For a UX unmatched by anything else currently out there, we have multisig functionality through BLS keys to make this a truly useful and usable system.
Add that to earlier innovations like self encryption of chunks and the picture is pretty much complete. This missing link of libp2p simplifies the offering into an easily explainable network and does so in a massively scalable fashion, whilst providing an incredibly high level of security with almost no historic state (apart from the DBC transactions).
This feels and looks like the Safe Network actually waking up now and shedding all the research parts and tests we have done to provide a small, robust node that will allow users to simply join the network and start earning and storing data almost immediately, regardless of how long the node will be online for. Everyone is going to get Secure Access to a new approach to humanity’s most precious element, knowledge.
Useful Links
Feel free to reply below with links to translations of this dev update and moderators will add them here:
Russian ; 
German ; 
Spanish ; 
French; 
Bulgarian
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!
