SAFE Network Dev Update - April 23, 2020

Summary

Here are some of the main things to highlight since the last dev update:

  • We released a new stable NuGet package of the SAFE C# API.
  • We added a new small feature in the mobile browser enabling users to browse the files from a FilesContainer. This should be available in the next release.
  • We worked on adapting the dog command to show every step in the resolution chain.

Vaults Phase 2

Project plan

In the last week, we’ve been continuing to chip away at data handling by Adult Vaults. We have made good progress there, identifying requirements in Routing and reaching out to @adam and @qi_ma who help satisfy our requirements instantly. While we continue working on this front, we are also thinking about other components of the next phase such as handling network splits etc. Stay tuned!

SAFE API

Project plan

After the latest release of SAFE CLI v0.12.0 last week, we started working on some new features and enhancements proposed by the community on the CLI dog command. It currently shows information about how a URL is resolved to locate the target content, but when this resolution means more than one indirection, it doesn’t show every step in such a resolution chain. Thus we are now trying to make sure it does show the complete resolution chain so the user can fully understand what content is being targeted by a URL, and how.

Some other minor additions are being made to the APIs for parsing SAFE-URLs, such as parsing and being able to manipulate the query params and/or the fragment. These are not used by any of the features we currently expose on the CLI but it’s something that will be helpful for app developers when dealing with SAFE-URLs.

CRDT

The efforts in trying to have a first PoC implementation for a CRDT data type continued during this last week. The sunny day scenarios for a Sequence data type are already working in this PoC implementation, thus we started working on the addition of vector clocks to properly handle and support concurrent operations. An example scenario where this could be required would be if two different insert operations coming from two different clients need to be applied in all replicas (Elders/vaults in our case), but the order in which these two new items are added must be the same on each of the replicas, with no conflicts. We have a first draft implementation for covering some basic cases of concurrent operations, and we are now trying to cover all edge cases by writing some decent tests which should help us in taking the implementation forward.

We’ve also been progressing with an AT2 implementation of safecoin, which has involved a large refactor of SAFE Client Libs to split things up and make it easier to understand. With that under our belt, we have a conversion to the basic setup for AT2 (without any validation etc… basically a conversion of what we had ready for the next step). This is working well and puts us in good stead.

In parallel we’ve been working on setting up new tests and “chaos” whereby we can more assuredly verify that our AT2 implementation is working as expected once we have the consensus steps setup. This should involve simulating dropped messages, to ensure we still converge on consensus, or clients/nodes acting maliciously so we can very that such behaviours are rejected as expected.

Furthermore on the back of that we’ve also started some modernisation of the SCL Rust futures implementation to make various facets of this simpler. That in itself is quite a big task. But it’s one we’ve had pending for a while now so it’s good to get it underway.

SAFE Browser / SAFE Authenticator (mobile)

Browser Project Plan
Authenticator Project Plan

This week, we fixed multiple issues in the mobile browser reported by the community. The README file for the repo and FAQ forum page were also updated to fix some broken links and to add some extra content.

Previously when trying to connect to the MaidSafe hosted section, we were showing the same error messages for different issues. To improve the error dialogs and make them easier to troubleshoot, we broke the connection process into two steps. Downloading the connection file from S3, and connecting to the SAFE Network. Now both steps show their separate progress and error dialogs.

We added a new small feature in the mobile browser enabling users to browse the files from a FilesContainer. Using this, users can now browse any FilesContainer using its XOR-URL/NRS-URL and can browse the files from a list.

We’ll be continuing to add more fixes and features before releasing a new version of the apps in due course.

We also updated and added to the SAFE Authenticator FAQ forum page, alongside fixing a broken link in the README.

SAFE App C#

Project plan

We released a new stable NuGet package this week. The updated package contains the fix for the decode auth response API. Also, we updated the XorUrlEncoder APIs to support subnames. Previously in the C# APIs, it wasn’t possible to generate a XOR-URL with subnames; with the recent changes now you can supply subnames to generate a new XOR-URL. We also updated the XorUrlEncoder APIs to use a C# list of string instead of using JSON strings when generating a XorUrlEncoder from a XOR-URL.

Routing and quic-p2p

Routing Project Plan

Last week we merged the test improvements we talked about in the last dev update and it already helped us identify another subtle bug we had, which we promptly fixed. We then implemented two small changes: one to expose a method to retrieve the current section adults and another one to modify the way nodes age during network startup - both requirements from the Baby Fleming team.

There is also another refactoring task in progress. This one aims to bring more sanity into the chain module. This module started a very long time ago as a simple data structure holding the history of section member changes, but over time accumulated a lot of unrelated functionality. The refactoring will break it down into several self-contained modules each with clearly defined responsibilities, resulting in cleaner and simpler code. We expect to raise the PR with this work very soon.

BLS - Distributed Key Generation

Since last week we have added more test scenarios to the DKG crate. Refactoring work is continuing here to improve it further. We are also making use of BLS-DKG to write up a Routing Simulation tool that would help us to investigate and integrate this crate with the routing module. This would then be used alongside Parsec making the network’s consensus process much lighter. Initial setup would be to simulate a newly forming section of Elders, simulate a churn by adding Elders as the DKG session proceeds, and test to see how the section performs in achieving consensus. This also would help us to check the crate’s usability within a churning section as it would in a real world scenario.

Useful Links


Feel free to send us translations of this Dev Update and we’ll list them here:

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the SAFE Network together :tada:

73 Likes

First…my first time…finally :slight_smile:

Edit: lot of small steps to Fleming, can’t wait…great work Maidsafe team

27 Likes

Thank you everyone. Steady progress getting made.

21 Likes

Nice progress fixing these small issues, is great to see.

I’ve been impressed that there is no apparent data loss… pulling and pushing at this I can’t break the fundamentals of getting back what I put up. Are we not providing enough chaos that you have to fake it!? :smiley:

:+1:

23 Likes

End users always find new ways to cause chaos that no developer or machine can predict! :laughing:

30 Likes

It’s quite neat to see that after all these pieces are stitched together how things are modularized or refined to make it more maintainable and clear to outsiders. The optimizations happening for data types and Safecoin, consensus in general I suppose, are just great. I’m guessing this will result in a significant and robust performance boost which is obviously welcome!

Have any other projects implemented AT2? Just curious. It’s hard to gauge progress of implementation but it sounds like it’s already off to a good start! Best of luck team @maidsafe

Curious how Nakita is coming along on IGD in quic-p2p as well.

Cheers everyone!

18 Likes

Really picked the right time to give up the office space :stuck_out_tongue: Things seem to be chugging along without any significant interruption since people were already working from home anyways.

29 Likes

True, remarkably prescient!

16 Likes

Thanks for all the hard work and such solid continued progress. Probably been finding the last couple of months harder than any other point over the last couple of years to follow all the strands, new types of consensus etc, but getting more of a sense now of how it’s all coming together.

For anyone that’s interested I’ve been quietly working away on my little search app during lockdown, really getting to grips with front end stuff whilst waiting for data types etc. to settle down. May be a little while before there’s anything to share, but still chipping away!

For the sticklers @joshuef and @happybeing I’ve also had what I think is a promising idea for making a genuinely decentralised search index, which as much as anything would just be so much more interesting than a budget version of Duck Duck Go or whatever! I’ll try and start that conversation again one day when I’m feeling coherent!

25 Likes

Thanks so much to the entire Maidsafe team for all of your hard work!

16 Likes

Thanks for another great update Maidsafe devs,

Could the current test safecoin coexist with this new implementation? Polkadot has something similar with Kusama which is used as cutting edge & breakneck speed testing, but still has economic value.

Cheers and keep up the good work

2 Likes

Whre can I check CRDT part on github? thanks in advance.

Also here

safe-nd is a good place to start. That spills over into vaults and SCL too though I believe as they need to know how to handle these new data types. Sometimes work being done is in certain team members repos before being merged into the Maidsafe repo, just fyi.

6 Likes

The answer appears to be, well!

Cheers @nbaksalyar

8 Likes

Do you think it will spill over to parsec eventually?

2 Likes

It means we don’t require parsec for crdt data types, so we get faster and concurrent updates that are guaranteed to merge. Merge means all operations succeed. The reason in many cases is causal order as opposed to strict order.

So where we don’t require strict and total order we can skip parsec. That means we can operate faster and with less memory bloat etc. Where we do need strict order then parsec fills that gap nicely.

20 Likes

So another performance upgrade for “free” then?

6 Likes

Thank you for the heavy work team MaidSafe! You are the CHANGE I want to see in the world! :bowing_man: :muscle: :bowing_man:

I add the translation into Bulgarian in the first post :dragon:

9 Likes

So this thing existed before I reinvented it :sweat_smile: Eventually consistent global counter ("SafeTime") was esentially the same thing :rofl:

2 Likes

Onward and upward team, well done, keep chipping away!

4 Likes