Safe Network Dev Update - November 5, 2020

Summary

Here are some of the main things to highlight since the last dev update:

  • Our adoption of proptesting has been bearing fruit with some important issues identified this week.
  • Significant performance improvements to the network startup process have been merged here, with further proposed improvements in progress here.
  • Our CRDT consultant has begun working with us again this week and dived straight into enhancing Deterministic Secure Broadcast (DSB) with support for dynamic membership.
  • Kudos to forum member @treslumen for their contribution to sn_api here. :clap:

Safe Client, Nodes and qp2p

Safe Network Transfers Project Plan
Safe Client Project Plan
Safe Network Node Project Plan

We’ve been shoring up our Sequence data this week, starting with various proptests to ensure the CRDT data converges as expected under various different circumstances. We’ve unearthed a possible bug in how permission policies were being applied and have been working on a fix, plus more specific tests for that particular area.

Alongside, we’ve started investigating how we can use the Sequence Data’s permission system for Map data, with the aim of making it CRDT ready. Initial investigations indicate it’s looking like we can apply this wholesale, so right now we’re setting up the basics for that and, assuming no blockers, we’ll start firing in with tests afterwards.

On sn_node this week, we’ve been zeroing-in on an inter-section message passing bug that intermittently prevents clients from uploading data to the network, highlighted by failing data tests in our internal testnet. All of the network’s messages maintain a list of entities/authorities that it has traversed to reach its destination, which wasn’t being updated correctly for its reply to come back. A refactor and a fix is underway for this and, once squashed, it should put us back on track to continue pushing the testnet forward.

We’ve also happily accepted a PR from forum member @treslumen, removing some obsolete code after the recent sn_client updates. Thanks for that @treslumen, very much appreciated :+1:

Transfers

Continuing on from last week, there has been further work on improving anonymity and consistency.
We’ve also returned to look deeper at Digital Bearer Certificates (DBCs) and see how we can use them now that we’ve made a lot of progress with CRDTs and AT2.

A DBC is a mechanism whereby a token or certificate can be held and then used. What DBCs can give us, if we manage to solve a couple of things, is full anonymity and offline transactions.

CRDTs

This week our CRDT consultant has begun work to enhance Deterministic Secure Broadcast (DSB) with support for dynamic membership. Dynamic membership is the process by which new peers may securely join or leave a peer group (section) by voting/agreement of existing members.

The leave case is most interesting because it entails detecting when a peer is faulty and then other members vote to exclude the faulty peer. A peer may be considered faulty for many reasons, including becoming non-responsive, using the wrong protocol, sending bad data, or sending invalid signatures.

As a reminder, DSB is a Byzantine fault tolerant (BFT) protocol that is presently used for securing AT2 transfers. It is also useful for securing CRDT algorithms.

It is expected that DSB with dynamic membership will form a solid foundation for future routing improvements as sections must regularly agree on adding and removing elders and other nodes.

Routing

Project Plan

To ensure routing level tests are using the same network setup as the planned testnet configuration, we decided to remove NetworkParams and use const value for testnet. This enforces the use of the same constant values for a section’s size and a section’s elder_size, across the routing and node level tests.

This week we also tweaked the network startup process to improve the startup performance. We replaced the DKG unknown message bounce-back with a simple backlog during the startup, reducing the time taken for a first section to fully form with all its elders from minutes to seconds. This also adds a proptest for DKG to further ensure the code quality and network reliability.

Further improvements to the network startup phase have also been proposed in this PR, which is currently working through testing and peer review. During the startup phase, every infant that joins is instantly relocated back to the same section, but with increased age. This changes the process slightly so that the new age is pseudo-randomly generated from a given age range, therefore preventing the peers from having the same age, which would cause them to be relocated at the same time. It also updates the relocation calculation so only the oldest nodes are relocated.

As mentioned in the Nodes section above, an issue was found on an internal testnet where the payload of section-to-section messages from nodes to routing were not always identical across elders and clients. It was initially thought that this was a bug in the routing layer, but after some investigation, we traced it back to nodes and this is now in their capable hands.

Useful Links


Feel free to reply below with links to translations of this dev update and moderators will add them here:

:bulgaria: Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!

74 Likes

Jippy first

Thx Maidsafe devs for giving me that first call.

Now read :crazy_face:

@treslumen :vulcan_salute:

26 Likes

Great to see nice progress again.

19 Likes

The world’s in lockdown and still we can rely on Safe Network updates to cheer us up! :smiley:

Good work, keep it up.

20 Likes

Once again a great update. Finding stuff in testing is great, having a design where fixes happen in the same week is better.

DBCs are next-level stuff. If it is possible in the SAFE network, then it’s difficult to over-hyperbolize the potential of that. If not, well instant scalable anonymous transactions of a currency with a demanded resource backing it is more than enough of a consolation prize.

Awesome as always SAFE team. Thanks for what you do.

25 Likes

The return of the DBC!!!
Cash may not die after all! :smirk:

All I have to say here is thank you @maidsafe for doing some of the most important work to securing our freedoms and rights.

Props to @treslumen for the api PR!

24 Likes

Thanks so much to the entire Maidsafe team for all of your hard work! :racehorse:

Here is the latest meme:

4a87xp

16 Likes

Christmas may come early this year.

22 Likes

Sounds like the DBC will be a great core development for AT2.

14 Likes

Thrilled to see technical members increasingly getting involved.

Excellent!! Last mile, folks :smile:

16 Likes

If DBCs work out, does that mean that anyone will be able to print/write one out, whatever amount of future safecoin they wish, and give it to a friend or whatever? (Basically looking for a DBC for dummies here, or a bit of spitballing as to how it might look). Very exciting stuff, if I am understanding correctly, amazing work maidsafe :laughing:

13 Likes

With a DBC you can re-issue it or more correctly ask for a reissue. So you have one or several for lots of safeocin. You want to pay somebody 4 safecoin. You take a DBS with >=4 safecoin and have it re-issued to the new id that’s gonna own it.

In fact here is a good read on more detail (just imagine each section is a bank) https://opaque.link/post/digitalmoneydbc/

19 Likes

Juicy update. So exciting to see the collaborative efforts of the team continue to manifest and begin to crystalize. I can also rest easy tonight knowing that David is making DBCs a priority. Epic times ahead.

12 Likes

Great update as per usual, thanks Maid team!

I love the sound of this. Wondering how it works in more theoretical detail though … it sounds like magic! :wink:

oops, got ahead of myself, perhaps this query was already addressed … will go read that.

13 Likes

I had searched around and found that guys blog, was just coming back here to link it actually! Funny, and yeah he explains it in a pretty understandable way. I had been guessing a section might work as the mint, makes sense. Scrit looks interesting too, I can’t help but think the guy there would be excited about what you’re all up to here

10 Likes

I sent him an email with a link to the update letting him know his article is linked in the comments section. I also told him Maidsafe is including DCBs in the Safenetwork. :slight_smile:

14 Likes

Great update again, thanks for your work!
Very cool to see @treslumen contributing code! :ok_hand:

“Inter-section” sounds like you have a testnet running with multiple sections. Would be nice to hear a bit more about that, just in some wide strokes how it is going. And of course, is that maybe something the community can expect when then public testnet comes?

9 Likes

About coins in circulation?

“As long as we don’t do DBC (digital bearer certificates) then an approximation is available. Could perhaps be with DBC as well, but that is beyond what we know now.” Citation from @oetyng awhile back.

Any new knowledge if it will be possible to know anything about coins in circulation with DBC?

2 Likes

Not at this stage. Something worth considering though. Could be a difficult genesis but that’s probably the price. I feel bulletproofs can give us some validation of initial mint though. That and perhaps no DBC exists until we have X sections (mints). Something along these lines anyway.

7 Likes

And that elder will be closed off after a week of testing, leaving everyone stranded again…