SAFE Network Dev Update - April 30, 2020

Summary

Here are some of the main things to highlight since the last dev update:

  • Today we released v0.5.1 of the mobile Browser, and v0.3.1 of the mobile Authenticator :tada:
  • The initial implementation of Vault Phase 2b Adults and their behaviour while handling Immutable Data requests has been completed.
  • We have our first PoC Sequence data type implemented as a CRDT and are able to start integrating it in the libraries stack.
  • We now have a test version of AT2 working well with the CLI.
  • A PR to add UPnP to quic-p2p is now under review - this brings connecting vaults from different homes into a single section much closer.
  • @jimcollinson started working on refinements to the candidate designs he shared previously (see the SAFE Network App UX section).

A quick note to say that Nikita finished his contract with MaidSafe this week, with his final contribution being the submission of this UPnP PR in quic-p2p (discussed further in the quic-p2p section below). Nikita has worked for MaidSafe for 4 years and I’m sure you will agree he has made a huge contribution to our project in that time. Last year Nikita switched to part-time to be able to spend time on his own projects, and he believes now is the right time for him to take the leap and change direction. We know he won’t like being mentioned like this :blush: but please join us in thanking @nbaksalyar for his contributions over the years and wishing him the very best success in the future.

Vaults Phase 2

Project plan

The past week we have continued progressing nicely towards our next Phase. We have completed the initial implementation of Adults and their behaviour while handling Immutable Data requests. The next step is to design and implement the way the section behaves during section splits.

We have some additional hands helping us progress development in Vaults. @ravinderjangra has joined us to help design and implement various features required for the next phase. @oetyng has also been helping, starting off with a refactor of the client handler which has been merged today. We are excited for the days ahead and we hope you are too.

SAFE API

Project plan

Some refactoring has been taking place in our safe-api repo, specifically to the XorUrlEncoder API which is used to manage SAFE-URLs. We are adding some more functions to it so it can be used by any developer to deal with SAFE-URLs, i.e. both NRS-URLs and XOR-URLs, to parse, manipulate and also generate the corresponding XOR-URLs if needed. This is not affecting any of the functionality exposed by the CLI but just that specific API.

CRDT

As mentioned over the last few weeks, we’ve been trying to get a first PoC Sequence data type implemented as a CRDT and we now seem to have something that is functional and good enough to start integrating it in the libraries stack. This is why we have now started working on the implementations required to expose such a Sequence data type on our core libraries, starting from safe-nd and safe-vault. We are planning to add support for this new data type in the full stack without removing or changing any of the current data types or operations supported, but instead, having it first as an additional data type which we can use to start validating and testing e2e in a single-section network environment.

Once this new data type is fully integrated and validated, we’ll then start migrating all our client APIs which make use of the AppendOnlyData to start using the Sequence data type. E.g. FilesContainers and NRS Containers are currently stored as AppendOnlyData and they will be migrated to use the Sequence data type. This won’t affect how the API is exposed, nor the CLI commands, but just how that content is stored on the network.

As we’ve done with other data types, we will aim at having the public version of the Sequence fully working first, before trying to expose the private (a.k.a. unpublished) scope of it.

Otherwise, our initial AT2 implementation is looking like a solid replacement for the current safecoin implementation. We have a test version working well with the CLI, and we’re now working around the edge cases in the SCL tests themselves to ensure this is solid. From there, we’ll be looking at the next steps for AT2 which will include gossipping between nodes to ensure all updates are passed on, which should also enable double spend protection.

SAFE Network App UX

MVE Feature Status Tracker

If you’ve been keeping an eye on the SAFE Network App feature tracker, you’ll notice quite a sea of green in the Essential Features row. That’s good news of course, but it also means we need to start pulling this puppy together.

Up until now, it’s been designed and prototyped around some pretty stock componentry, with the Material Design system underpinning it. This was the pragmatic choice to enable us to relatively rapidly build out a multi-platform app which is usable, familiar, and without too much of a maintenance overhead.

This approach does introduce some constraints of course — on the whole design constraints are a good thing though — the major drawback being, well, the unavoidably Googley vibes, and a little less control over the aesthetic.

Overall, this is a tradeoff worth making, given the team size, and the overall mission. But now as the time approaches to broaden out the design to further platforms (if you may recall, we’ve approached this mobile first) it’s a sensible point to put some time into the overall look and feel of the UI. It will avoid duplication of work as we move between platforms, and also allows time for the design to mature iteratively too.

So, here’s some of the rough criteria we’re working towards, when starting the upper UI layers, and adding that satisfying sparkle:

  1. The designs should prioritise usability, and functionality above all else.
  2. They should translate well across platforms, utilising core OS elements where required for native usability.
  3. They should also have a familiar SAFE feel, taking cues from existing brand assets, having enough uniqueness to orient users but not at the expense of 1 and 2.
  4. Thought should be given to adapting to multilingual options when the time comes.
  5. Maintenance overheads should be kept low.
  6. Iterative, incremental improvements, rather than big bangs.

So with that in mind, here are some sneaky peeks at the progress so far. As our first pass, we’re taking the existing prototypes and bending the Material System as far as is practicable, thinking about typography, depth, transparency, basic shapes, colour, as well as a smattering of texture too.

The idea here is to take some cues from the work we did on the website, while keeping utility and usability as priority. So a little less rad, and a little more workmanlike.

This will all keep on evolving as we iterate week by week as hands get dirtier, and of course when we are able to resume testing with users.

Futures in Rust

Last week saw us start some larger refactoring to bring us inline with modern Rust conventions on futures. We’ve started the process of moving SAFE Client Libs over to standard library futures and the latest version of tokio, too. This is no small task, but has been progressing well. As part of this we had to do a similar futures update in the Self Encryption library, which SCL depends upon, and now that we’ve unblocked that we’re continuing with changes in SCL.

SAFE Browser / SAFE Authenticator (mobile)

Browser Project Plan
Authenticator Project plan

Today we released v0.5.1 of the mobile Browser, and v0.3.1 of the mobile Authenticator :tada:

You may recall that the recent v0.5.0 release of the mobile Browser included auto-update for the first time, as detailed here. Today’s release of mobile Browser v0.5.1 gives you the first opportunity to see this auto-update process in action, if you currently have v0.5.0 installed.

To auto-update, you should launch Browser v0.5.0, then move the Browser to the background, e.g. by returning to your device’s home screen. When you then resume the Browser it should prompt you to update to the new version. Note that we were expecting a prompt to update on app launch, and not to have to move the app to the background and then resume it. We have raised an issue in the project here to investigate why this is not happening.

If you currently have a Browser version < v0.5.0, or you do not have the Browser on your device yet, you can download the latest version using the download links/QR code from the README file.

In this latest Browser release, we updated the files container page to include links to files. The user can now fetch the files from the files container using any nrs or xorurl. We made some textual changes in the authentication dialog popup to make the authentication step clearer. We also updated multiple error dialogs throughout the app to be more precise, which should make it easier for users to understand the problem and troubleshoot. Finally, the FAQ link in the app’s settings page was also updated to the correct forum thread.

Alongside the Browser, we also released v0.3.1 of the mobile Authenticator app today. Note that the mobile Authenticator app does not include auto-update - this app is being used as a temporary measure to authenticate, with the medium term plan being to roll this authentication process up into the SAFE Network App, so we have not spent unnecessary time including auto-update in it.

Authenticator v0.3.1 includes some small updates to the text used throughout the app, to make it consistent with the Browser and more accurate for the current stage of the project, i.e. we are now connecting to networks, not a single vault as we were in Vaults phase 2a.

You can download the latest version using the download links/QR code from the README file.

Routing and quic-p2p

Routing Project Plan

We are currently addressing some areas in Routing that were too strict in tests and had become less relevant to the real world (assumption that all nodes see the same network at a common point in time). In addition, naming conventions are being addressed at pace, this allows a much wider audience to understand the code base there.

We are also transitioning the section elders + key information to be network wide and updated in real time, only when required. This allows us to constantly and consistently update information in a secure manner that does not require synchronisation or strict order and in addition allows the network to self-heal as opposed to fail if a single important message fails to be delivered and acted on. This process should see significant speed increases in the Routing layer as well as again removing a need for strict consensused order and instead rely on cryptographic proofs of agreement in real time. It’s a very slick approach that is simpler code but will give us faster and more secure message handling.

As mentioned at the start of the update, Nikita has been working hard to finish the implementation of UPnP in quic-p2p over the last few weeks, and signed out with this PR, which is under review. UPnP gives us automatic port forwarding, which means that the process of connecting vaults from different homes into a single section becomes much simpler and therefore feasible. After reviews are completed, we will begin internal testing, and all going well it’s then over to the community.

BLS - Distributed Key Generation

Project plan

This week a major refactoring PR has been merged to greatly improve the BLS-DKG crate. It presents a more precise interface to use, and makes the implemented DKG generation flow follow the DKG Protocol paper more accurately. The initial work of introducing quic-p2p as the transport layer has been raised and is currently in progress. Once this is merged, the crate will need to be put through rigorous testing to check how the transport layer usage needs to be tuned and configured in such a way it supports real-world Routing like simulations. Nodes being dropped or delays in messages reaching their destination due to churn events are just some of the real-world network problems that we need to simulate and take into consideration when designing the DKG flow. We are in the process of writing such happy/unhappy path simulations to be tested on a DKG session before we can confidently have this crate employed in the network.

Useful Links


Feel free to send us translations of this Dev Update and we’ll list them here:

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the SAFE Network together :tada:

76 Likes

First! Now I read the update :slight_smile:

21 Likes

Not bad for an old guy :wink:

Goodbye and very best wishes to you Nikita @nbaksalyar. Thank you for all your hard work. Good luck with your projects and please check back and let us know what you are up to when you can.

Thanks Maidsafe team, progress on so many fronts is great to see. I shall now disappear down the SPARQL hole again…! :crazy_face:

But first I…

  • updated mobile authenticator
  • updated to mobile browser 0.5 so I could try…
  • auto update of mobile browser to 0.51

All good! But I am now using the shared vault not section.

Do I add the shared section in the authenticator? I’m a bit confused because mobile browser 0.50 asked if I wanted to use Shared Section or Authenticator and now I don’t see how to get to Shared Section.

The authenticator only shows Shared Vault so I’m guessing I need to download the config file but maybe some hints could be included in the UI at least for now?

34 Likes

Sad to see another great developer leave the project.

4 Likes

Just updated the mobile browser - seamless :+1: Best of luck with your new venture @nbaksalyar ! Whatever it is I’m sure it will benefit greatly from your talents.

27 Likes

Fifth! Now going to read. :slight_smile:

13 Likes

Thanks to all of the entire team for all of your hard work!

16 Likes

Does exactly that prompt in Android 7… though the activity bar for that download doesn’t seem to evidence progress and it’s taking a long while from here, while not clear it’s downloading successfully.
There’s a odd issue that a mobile firewall, which allows SAFE Browser, blocks the download… perhaps not for you to fix but surprised that I needed to unlock the firewall for the Browser to action its own upgrading download.

Edit: got the 0.5.1 but had to try a couple of time to see it action… unclear what the difference was… but nice improvements.

:smiley:

Sometime later, perhaps should point to a safe:// domain rather than jumping out to unSAFE environment. :thinking:

@nbaksalyar good luck and thanks for your contribution. Hopefully your projects can be SAFE related and we’ll see more of you again in future. :+1:

24 Likes

Damn, really hate seeing another talented developer moving on to personal projects but if there was a good bang to go out on it would be integrating universal plug n play with a newer transmission protocol. Good stuff to PR as a farewell.

Best of luck in your future projects @nbaksalyar , hopefully you come around from time to time to check in on progress here.

However the march must go onward. Every week is progress, every week gets us closer to the realization of SAFE Network.

22 Likes

Good luck out there @nbaksalyar you have brought a lot of positive direction and progress to the project. Hope you pop in or return someday in whatever capacity suits you best.

On the AT2 implementation, is it planned to use the already existing safe-gossip for the gossiping between nodes? I’m chanting your name @oetyng as you push forward on this! Huge props once again, so many amazing contributions.

You too @joshuef. You and Gabe are like a two man dream team.

Can’t wait to play with all this fun UI @JimCollinson @ravinderjangra. Not a google fan but the google look on the other hand, I quite like :blush: so I’m glad you guys went Material.

Vaults phase 2b looking like it’s getting closer and closer. CRDT progress looking great as well as far as I can tell. Great stuff @bochaco!

Nice update to the prompt to connect to the shared section for mobile browser :+1:

Thanks to everyone @maidsafe!

31 Likes

Thank you everyone for the kind words and good wishes! :blush:

Of course, it goes without saying – I’m fully prepared to launch my Vault from home :slightly_smiling_face:

50 Likes

Great to hear!

Thanks for your time and effort! Good luck in your future ventures!

Edewcate_english_rhymes_The_Ants_go_Marching_One_by_One_Song_Nursery_Rhyme

23 Likes

Oh man, @nbaksalyar is a loss. Sharp, and dedicated. Whatever you do next you will do well. I do hope you’ll stick around in the community.

Awesome stuff otherwise.

23 Likes

Great update again :+1:

Found a great intro on CRDT’s:

16 Likes

Best of luck, @nbaksalyar!! Nikita has done so much incredible work to push the SAFE Network forward, from development to community support. For example, Nikita helped the launch of SAFE Network Chicago by video conferencing in to present and answer questions. We definitely look forward to seeing you around as a much valued member of the SAFE Community!

With regards to this week’s other updates, it’s exciting to see the steady press forward! More and more it seems like we’re entering into the “last mile” of delivery. :smile:

32 Likes

Any reason why:


Are not listed under the MaidSafe Org people?

Just not been a priority to keep parity of folks “in” the org?

Just helps me keep track of high level who is deving besides looking up PRs and commits.

7 Likes

Keep going maidsafe. 100% backing the whole team. Can’t wait to see this thing in the real world. :kissing_heart:

22 Likes

Hey @jeremyjpj0916, currently 6 out of 15 natural persons are not publicly visible in the organisation, among them me and @adam. That’s why you don’t see us there :blush:

19 Likes

But we see the work you do :slight_smile:

Thank you to you and all the other devs.

@nbaksalyar we will miss you. Thank you for all your efforts and best wishes for the future. I look forward to your E-book on Asynchronous Rust.

18 Likes

Love it.

^^^ :point_up:
Before I get giddy and fall over myself, are you saying vaults from home before we have a multi-section network?

19 Likes