Safe Network Dev Update - January 14, 2021

Summary

Here are some of the main things to highlight since the last dev update:

  • We continue to move all client messages definitions, and serialisation logic, to the new sn_messaging crate.
  • Work to remove stream management in nodes, as mentioned in last week’s update, has been completed and merged
  • We’ve been working through and fixing the issues highlighted by our new routing stress-testing tool.
  • A massive shout out and thank you to a couple of community members for PRs this week - @mav and @tfa both chipped in with multiple PRs across different repos to help out and improve our work. Always inspiring to see :heartbeat:
  • We’re still working away testing a final bug resolution that we hope will mean that we can release the next iteration of the testnet.

Safe Client, Nodes and qp2p

Safe Network Transfers Project Plan
Safe Client Project Plan
Safe Network Node Project Plan

Over the last week, we spent some more time improving the qp2p examples so that they better demonstrate the library’s features separately. Along with this, we extended the echo service example to support manual port forwarding. While implementing this, we identified and implemented a small extension that verified the endpoint provided by the user as input. This allows us to return an error much earlier instead of waiting for the network to identify and report the node as unreachable.

We also continued with moving all client messages definitions, and serialisation logic, to the new sn_messaging crate. We’ve improved the protocol and we are now using Msgpack for serialising all messages that are sent by clients to nodes. This is an ongoing task though, as some inner parts of the client messages are still using the bincode serialisation and we therefore need to migrate those to also use Msgpack.

Work has continued with the distributed actor transitions at elder change, i.e. signing over the section funds to the new elder constellations. A small bug hindering the transition (due to failing sig validation) has been taking some time to root out but has just now been found. Next steps in the transition are now to be tested.

Lastly, we’ve been progressing with updating client/node to remove client challenges and to re-enable proper onboarding of clients, after the work to remove stream management in nodes, as mentioned in last week’s update, was completed and merged. This in turn has turned up some potential issues in client message receival or parsing, so we’re digging in there.

API and CLI

This week we were able to successfully generate musl builds for CLI and authd in our CI, and therefore it’s all setup and ready for our very next release. This means both the CLI and authd applications should be compatible with any Linux platform out of the box from the release package, we’ll need the community to help us validate this is actually the case though by trying it out on as many different Linux platforms as possible. We’ll release the new versions in due course.

BRB - Byzantine Reliable Broadcast

Work continued this week to further polish the BRB crates. The DataType API was changed so that validation error details can be returned to the caller. Further along these lines, the rust-crdt crate has been enhanced so that each CRDT type we use now has a validate() method that can be called prior to applying an operation. Pull Requests for CI/CD (Continuous Integration and Continuous Deployment & Delivery) have been made for each BRB crate. We have a few more changes to finish up this week, after which we plan to make the first release to crates.io.

Routing

Project Plan

Last week we mentioned how the stress test tool uncovered some hidden issues in routing. This week we set out to track them down and fix them. One issue was that non-elder members of a section would not know that a split happened due to a bug in how the Sync messages were created. This was fixed and merged. There is another PR currently in progress which fixes a couple more of those issues. The remaining issues after that are probably all related to the way we handle section membership changes. We hope to resolve these by integrating the new BRB membership algorithm that was also mentioned last week. This work will start very soon, possibly as soon as next week, if all goes well.

Similar to our recent move to have all client messages definitions and serialisation logic in a separate crate (sn_messaging), we started looking at doing the same with sn_routing messages. The goal is to be able to separate all the message definition and serialisation logic, from the core routing logic. This should allow us to have a clear definition of the interface exposed by routing services, and therefore how any implementation (in any programming language) can be made compatible with our existing sn_routing implementation. We are in the very early stages of this process, but wanted to share our goal here.

Useful Links


Feel free to reply below with links to translations of this dev update and moderators will add them here:

:russia: Russian ; :germany: German ; :spain: Spanish ; :france: French ; :bulgaria: Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!

72 Likes

First! 20 char

18 Likes

Nice too see the steady progress.

Nice work @mav and @tfa !

28 Likes

Third!! Now to read :smile:

This is super exciting! More grease to your elbows, MaidSafe :smile:

Here’s a link to the post supporting this on Twitter, so you can like, retweet, and comment too!

https://twitter.com/safenetworktech/status/1349794221486694401?s=21

22 Likes

A public multi-section network is so close! Amazing progress. Thanks @maidsafe and community members

25 Likes

Almost there…well done all.

17 Likes

Well done everyone! :clap: looking forward to test net!

19 Likes

Thanks so much to the entire Maidsafe team for all of your hard work! :racehorse:

4p8qo9

15 Likes

Great stuff @MaidSafe, and to have @mav and @tfa get stuck into the code. :clap:

21 Likes

Thank you once again!

There was so much in this update, that it is difficult to see what is supposed to be done before next iteration. Would be nice to get some clarification.

10 Likes

With some of the new developments in the tech sphere, the need for decentralized internet is more urgent than ever. Here is an article about the need for decentralized internet, called web 3.0 in the article. :racehorse:

22 Likes

Thx 4 the update Maidsafe devs.

Love the progress

We need more @mav’s :vulcan_salute:and @tfa’s :vulcan_salute:

P.S. let’s not forget the og’s @bochaco and @oetyng :stuck_out_tongue_closed_eyes:

Keep hacking super ants

17 Likes

Beautiful work as always, team—thanks for another great update.

:safe:

12 Likes

It’s all clean up and bugfix right now. We had rewards switched off and they may be back again in next iter, but it won’t stop us.

20 Likes

@mav @tfa Future core Developers receiving some dev rewards on the Safe Network???

Thanks for your efforts @tfa and @mav

And of course a big thank you for the ongoing efforts of the team.

I am talking with some youtube creators and when i mentioned the future Safe Network they were like “WHEN can we start using it?” (EDIT: and they are not at risk of being kicked)

The way youtube and twitter are banning people for questionable reasons, we will have a line up of people wanting to use these services on Safe. Guess they will line up with the torrents people and all the others.

27 Likes

Questionable? Inciting an insurrection against a legally elected government is questionable? There will be YouTubes and Twitters on the Safe Network given enough time and they will be faced with the same kind of decisions during times of crises.

3 Likes

Can you please take your politics to a political thread @VaCrunch … questionable means that people have differing opinions on the topic and that’s true. You on the other hand have a firm opinion on the topic and you are voicing it in this thread.

And you were the one complaining the other day about people and their self-serving posts. Hypocrisy.

15 Likes

Moving both posts to a political thread is fine with me, Tyler. Happy to see you remember my posts.

1 Like

In my case it was only small modifications, not in the core of the network code:

  • Improve logging of number of elders and adults
  • Rename a function so that it corresponds to what it computes and how it is used (remaining_space_ratio => used_space_ratio)
  • Execute GitHub actions only from Maidsafe repositories
  • Possibility to inhibit safe update/install commands (which also reduces binary size by 30%)
24 Likes