Update 23rd March, 2023

We know you’re all champing at the bit to get testnetting again, but there are one or two fixes we need to make first to ensure it will provide valuable learnings for us. One in particular is getting node ageing working reliably. @joshuef explains more about the hurdles and how we are preparing to leap over them as we head for greener pastures.

General progress

There’s loads going on as always, including some deep dives into the fundamentals. As some of you will know, our qp2p networking library is based on Quinn, a Rust implementation of the QUIC industry standard that was developed by Google.

The good thing about adopting widely used components is they have lots of eyes on them and are constantly being updated, but a drawback is that they don’t always work quite the way we want them to. In this case, the TLS library relies on DNS and that requires a certificate authority (CA), both of which in their standard form are a no-no for a p2p network. But… David has a cunning plan to use our group consensus as a way of becoming our own CA, which should allow us to secure traffic and sign with our ed25519 keys, at least when an updated version of rustls comes out, which should be soon. Ultimately we want to minimise our use of qp2p and this will be a step towards doing so.

@bzee has also been delving into qp2p and is adding a stripped down version to the stableset_net repo to improve the efficiency of messaging. @davidrusu is also working on the stable set, including getting the Stateright tool to offload its work queue to disk to allow us to test more elaborate models.

On DBCs, @oetyng has made good progress with updating the sn_dbc crate as well as clarifying the language we are using to describe blinding (hiding the amount transacted) and unblinding to make it easier to follow. He’s now working on updating the command flow between clients and elders when transacting with DBCs.

@bochaco is working on exposing gRPC for safe_node and adding a step in our testnet tool to check launched codes using such a service.

Roland has been beavering away on telemetry, improving our visibility at the node and function level. Traces allow us to see what’s going on with every function, but in their raw form they are hard to read, so Roland is pulling them into OpenSearch where they can be aggregated by node and time to give us a highly granular picture of what’s happening where.

And with the legal stuff out of the way @JimCollinson is turning his attention to branding. What should our core messages be, and how should we present them? There’s a lot to learn from others who have done it right, so if there are any companies or individuals you find particularly inspiring, let us know in this thread.

Progress towards an ageing network

There are several things afoot that we’d like to get more solid for any new testnet. The main issue we saw in the last testnet was that our relocation code, and therefore node age, were not working as expected.

The basic reason for this was that we just did not have enough churn prior to opening up the network. But it opened up the question of how to reliably achieve that and relocation going forward.

One simple change has been reducing the node’s starting age, this gets more relocation faster… but this itself also comes with various costs and tradeoffs - especially until we have tiered data storage in place. But we also hit some other issues there.

We’ve seen that our now previous “two step” approach to membership was causing us issues (we have Membership where voting happens and also SectionPeers which was supposed to be up-to-date for membership changes and based upon our SectionAuthorityProvider (SAP), but these two could get out of sync). As such it’s great that @qi_ma 's PR has been merged, which should reduce such discord.

We also ran into an issue with membership voting decisions getting incredibly large, and causing far too much traffic, sending verification time through the roof. This itself has been refactored back to sane levels, and also inadvertently exposed some blocking we had going on.

That process blocking has also been nixed, so things are working much more smoothly around membership voting and DKG now too.

As such, we’re back at looking at relocation velocity and trying to ensure we’re not seeing any loops as nodes move around the network. Once that is in place, we’ll be in a much better place for handling churn and storing data.

This in itself is a great place to be, but we’re not stopping there. Work on a separate POC for the stable set has begun, as some folks here have noted! This aims to be the simplest node (and it could well be vastly simpler if we can avoid a lot of the network knowledge maintenance and DKG that comes from our main branch). It’s still early days there, but we’re keen to get the simple implementation in and trash the hell out of it to see how things are looking.

Both paths are quite exciting and will hopefully yield another, more “aged” testnet in the near future!


Useful Links

Feel free to reply below with links to translations of this dev update and moderators will add them here:

:russia: Russian ; :germany: German ; :spain: Spanish ; :france: French; :bulgaria: Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!

48 Likes

Awesome stuff. (now to read!)

Sounds like solid progress. I’m keen to see another test network, but might as well make some serious strides forward and test when it’ll be more fruitful vs rushing. Hopefully with these tweaks the next test network will age well :slight_smile:

Despite all progress in the crypto world, there’s still nothing that comes close to what you’re working on. Keep up the amazing work towards this hugely important goal MaidSafe!

21 Likes

Didn’t even know it was Thursday and grabbed second! :rofl:

21 Likes

Thanks so much to the entire Maidsafe team for all of your hard work! :horse_racing:

20 Likes

Early this week :slight_smile:
Thanks to all for their hard work, keeping an eye on whats new is difficult, there is just so much on so many fronts.

Keep it up folks, you are an inspiration.

15 Likes

Lots of interesting ideas/fixes as usual. Folks are indeed chomping at the bit for an official testnet - we can taste the excitement as we get closer to launch.

Thanks for all the hard work team. Please keep hitting it hard - but also take some time to relax too :wink:

Cheers
image

16 Likes

Something that would not be a worry with the stable set approach?

Do I have it right that with the current approach there are no age differences inside each “generation” even though some of the nodes arrived earlier than others? But with stable set every node has an “queue ticket” that they get in order of appearance?

5 Likes

Is that kind of discord tolerable to some extent, or not at all? My gut says not at all.

7 Likes

Thx 4 the update Maidsafe devs

For years I’ve wondered how the CA problem would be solved. Got pretty depressed when I saw the decentralized solution, a while back.

Again you super ants have a elegant solution :clap: :clap: :clap:

:partying_face:

Really love the community TestNets and individual TNs

Keep hacking super ants

17 Likes

Node ageing occurs till a point where the node is considered stable and trustworthy enough to be an elder. At that age it joins the stable set for the section at the bottom of the queue. There was a suggestion that this would be age 20 which means a long time being a node according to David.

9 Likes

Curious, will there be a way for the human owner of a device to tell what age his nodes are, and whether any are elders?

2 Likes

At the moment this is present in the node logfile and can be viewed using vdash. At launch? :man_shrugging:t3:

7 Likes

I don’t know much at all about Certificate Authorities but doesn’t this open up the possibility of any safe specific browser or plug-in being able to verify the authenticity of an official site?

Assuming there is a manner of determining actual Identity of say Amazon during registration, then when you visit safe://amazon the browser could verify (with little green lock and cert info) that it is the actual Amazon and not the scam site safe://amaz0n?

Obviously NRS allows for anonymous registration as well so this could still be applied to sites with longest genuine traffic perhaps?

I suppose besides anonymous registration through the safe ‘protocol’ then maybe Maidsafe would or could act as a registrar and have their Elders sign for any sites porting to SN?

Idk, got my gears turning. Just good to be able to provide the same reassurances and follow similar mental models to peoples everyday interaction with the World Wide Web.

7 Likes

It’s an interesting one. So basically these are the main points

  1. We use a self signed IP based certificate authority (rustls is releasing that ability soon) (i.e. no DNS)
  2. On Safe the network knows and can confirm to anyone who the current elders etc. are (elders plus next ~20 adults in the stable set)
  3. Nodes known by their public key
  4. The public key becomes a TLS 1.3 certificate
  5. If you get the elders from Safe then you can inject those public keys into the certificate authority vector for your connection.

So this way we get a certificate authority with self signed certificates, which seems like an oxymoron (as does much of Safe). The self signed certificates are node names we can attest are Safe nodes and attest that via group consensus (majority of elders tell you these guys are OK). This latter part turns self signed certificates into trusted certificates as Safe tells us they are trusted.

In short with group consensus we can turn IP based self signed certificates into a trusted set of known certificates that carry authority/trust.

Therefore in Safe we do have the ability to green check mark browsers etc. but should not need to as using this mechanism and the XOR based DHT with network valid data then all sites are secure and safe.

tl;dr Where we are going we don’t need no checkmarks :slight_smile: However it’s only in Safe really and not for the clear web, but who knows what devs will do with that?

14 Likes

Thanks for the clarification!

This :point_up: :fire:

7 Likes

This reminds me of something I was thinking about, a long time ago now…

10 Likes

And age is simply a number of relocations, right?

1 Like

Yes i believe that is right in that the node is relocated as it ages.

There can be cases where a node will lose some age and in these cases the number of relocations will not be its age

4 Likes

Thank you for the heavy work team MaidSafe! I add the translations in the first post :dragon:


Privacy. Security. Freedom

4 Likes