SAFE Network Dev Update - January 9, 2020

Summary

Here are some of the main things to highlight this week:

  • Happy New Year and may you all have a prosperous 2020!
  • @danda has joined the team :tada:
  • @oetyng shared the Data Types Refinement pre-RFC. Please give it a read and say what you think!
  • The safe files add command now accepts a file’s content from stdin which enables piping other commands and uploading new files onto the network.
  • The CLI User Guide was updated with a new section for the safe xorurl command.
  • We’ve begun early concept and design work on the farming distribution aspect of Safecoin.

Recruitment

We’re delighted to welcome @danda to the team! You’ll most likely recognise his name, as he’s been an active contributor to the safe-api repo for almost two months now! He’ll be initially working with @oetyng on the farming aspect of Safecoin - see his first contribution to a dev update in the new Safecoin Farming section below.

Vaults Phase 2

Project plan

It’s been an exciting couple of weeks on the Vaults front. We are seeing the different parts such as Routing, PARSEC, Vaults and Clients come together as we edge towards the end of Phase 2a, i.e. Clients communicating with a single section. With the routing-vault integration fully complete and the new bootstrap process for clients implemented, vaults have the ability to spawn up and connect to each other forming a section. With the help of any one node’s connection information, a client can connect to the node requesting the connection information of the other nodes in the section and then resend the request to multiple vaults and establish a connection to the network. With this, it can start sending requests for data and coin operations, which is precisely what we’ve been doing via the tests in SAFE Client Libs. We are seeing good results with the client tests against a single section, however, there are a few failures as well due to random request timeouts. Debugging this has given us some good leads to follow up on, which are keeping us on our toes all day! We’ve tested the multiple vaults setup on Digital Ocean droplets as well and we’re seeing only the same few failures which means we’ll be ready to deploy a working test net once we’ve resolved these last few errors. So, fingers crossed!

At the same time, we are continuing to improve QuicP2P, our networking library. @adam moved the mock networking implementation from Routing to the quic-p2p codebase, making the Routing public API simpler and more self-contained. Now we can use the mock network (which emulates the data exchange at the low level) in vaults, clients, or other projects without having to depend on Routing. With that, and the other improvements introducing async code, we should be ready to release the new version of quic-p2p soon.

SAFE API

Project plan

These past few days, we focused on finalising a couple of PRs that we started before the end of year break. A PR prepared by @danda last year was merged which allows the safe files add command to accept a file’s content from stdin which enables piping other commands and uploading new files onto the network.

Another feature which we started a few weeks back (suggested by the community in this forum), and now merged, was the incorporation of a new safe xorurl subcommand for the CLI which comes in handy when wishing to calculate the XOR-URLs for local files, without the need to upload them to the network. The CLI User Guide was updated with a new section for this new command. Note that for now this command requires the CLI to be authorised, we plan to enhance it in the future to remove this limitation.

safe-authd has been the main focus in the past few days, as its update command is not working and therefore it is not possible to update it in the same way as the CLI. We started fixing this by adding support for AWS S3 when deciding where to fetch the new releases from, as that’s where we are publishing safe-authd released binaries. We are also planning to start using S3 for the CLI binaries as it’s faster for users to download them than from the GitHub releases page.

Once we have safe-authd fixed so it can be updated, we’ll add a couple of new subcommands to the CLI to be able to install and update the Authenticator daemon by automatically pulling the corresponding binary from S3. This will improve and simplify the UX for the CLI and authd users.

Labelled Data, Indexing and Token Authorisation

RFC, Project Plan

@joshuef has been ramping up within SAFE Client Libs, learning the codebase and making a start on a basic implementation of Auth Tokens, and its integration into the codebase.

Thus far, we have a simple token/caveat setup along the lines of macaroons. This is now being integrated into SAFE Client Libs more solidly, using our existing AppKeys setup for signing the token, and validation, which should simplify things down the line.

The initial goal here is to replace our authentication mechanism with the tokens, which can then later be expanded to hold further info such as allowed labels, etc. (As we build out labels/indexing separately).

Data Types Refinement

RFC

@oetyng has written a new RFC for consolidating the various data types and the changes that they have undergone during development. Aside from that the RFC also introduces simpler naming for the data types. The discussions have already started on the forum, so head over now, give it a read and say what you think.

SAFE Client Libs

Project plan

We ended 2019 on a great note with an amazing community contribution to the Client Libs codebase that helped us clear up our backlog. We’ve also added some small but useful features that would be very helpful in the application layers across the CLI, desktop and mobile platforms. Some of them are: supporting dry run when storing Immutable Data, more information when fetching the list of applications from the authenticator, and support for the GET_NEXT_VERSION constant for mutable data operations.

During his last couple weeks at MaidSafe, @marcin has mainly focused on tying up loose ends, reviewing PRs, and closing out his remaining assignments. This included some infrastructure work such as CI and GitHub Actions fixes. He also left some new documentation in the SCL wiki.

SAFE Browser (mobile)

Project plan

In previous versions of the mobile browser, the information required to connect to the network was hardcoded in the app but now to enable users and devs to browse content from the local/shared vaults, the mobile browser gets the vault connection info from the authenticator app using an unregistered response.

As we are getting close to a release of the mobile browser with pWeb features, we have updated the app to support navigation between different versions of a web page on iOS. The Android platform already had this feature for the last couple of weeks.

We also fixed an issue related to the Rust logs which are stored on the user’s device when running the app, so in the case of any issue these can be sent to the devs for troubleshooting and bug fixes.

SAFE Network App (mobile)

Project plan

This week we worked on some more improvements in the SAFE Network App for mobile and it’s looking even better now that we’ve ported most of the images from the Figma design to the app. We also made some changes in the platform-specific code to improve the navigation UI/UX.

SAFE Authenticator (mobile)

Project plan

The mobile authenticator app now shows the test coin permissions on the app information page for all authenticated apps. We made some improvements on the vault selection page to make it more smooth and in the process removed one 3rd party package which was causing some delay in the UI updating.

We have updated the minimum supported platform versions for Android and iOS. Now the minimum supported version for Android is 5.0+ and for iOS it’s 11.0+. With these changes, we have finished development for all the targeted features/changes from the current project board and we are planning for a release in the coming weeks :smile:.

SAFE App C#

Project plan

The safe_app_csharp library has been updated to the latest changes in thesafe-api master branch and the C# published immutable data put API now supports the dry-run flag. We are currently facing an issue with the bindings when trying to decode the authentication response message from the authenticator. Once this bug is fixed, we will be releasing a new Release Candidate package.

In parallel, we are looking into automating our GitHub release process from the Azure DevOps build pipeline on a version change commit. This is still under testing and once completed, we may deploy this for all the C# repos for a better release and deploy experience.

Safecoin Farming

RFC

We’ve begun early concept and design work on the farming distribution aspect of Safecoin, as described in RFC-0057.

This week we have been going over the RFC line by line, making notes, discussing, starting on design docs such as a data flow document and also writing some prototype pseudo-code for discussion purposes.

Our general goal is to implement the RFC as specified. Once that is completed and working we can test, model and tweak algorithms such as store cost and reward distribution as necessary.

Node Ageing

Project plan

Following our initial work on promotion/demotion, we have focussed on cleaning up code and fixing the remaining test failures to support the remaining work for Node ageing.

We also addressed, with numerous PRs, some long-standing issues: removing some no longer useful code, upgrading some outdated crates, and continuing to clean up our routing API.

We are now looking at completing the items in promotion/demotion, with the parsec pruning issue already addressed, and some work in progress to establish trust between Adults and Elders.

Useful Links


Feel free to send us translations of this Dev Update and we’ll list them here:

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the SAFE Network together :tada:

91 Likes

First of 2020!

28 Likes

second ----20chrs etc

19 Likes

Congratulations on an important milestone.

26 Likes

Today was my last day at MaidSafe. Thank you all for the last two years, it’s been quite a ride and I’m thrilled to have been part of it! Bye everyone :wave:

76 Likes

Good luck in the future and don’t be a stranger! :grinning:

22 Likes

Great to see a community contributed delivering code and then joining the team again. This feels like a great way for devs to learn the codebase, prove they are capable, then take a more active role on the project. Really pleased to see this!

Likewise, it is great to hear a community developer has contributed on the client libs. I hope you drop in on this thread to say hi! :grin:

Also it is awesome to read that test nets are getting tantalizingly close… I’m looking forward to having something to play with again! Whenever a release drops, test or otherwise, it gives the whole project and community a shot in the arm - much needed after a poloniex’s attempts at being The Grinch!

Looking good for 2020! Heads up with lots to look forward to!

31 Likes

:partying_face::partying_face::partying_face::partying_face::partying_face::partying_face::partying_face::partying_face::partying_face::partying_face::partying_face::partying_face::partying_face::partying_face::partying_face::partying_face::partying_face::partying_face::partying_face:

31 Likes

I guess there is no telling yet what kind of mix of CPU/RAM/Storage will be a minimum and whether there is a point beyond which there is little added value in having more CPU/RAM :thinking:

10 Likes

Sorry to see you go @marcin, All the very best in your next position and as has been said before, don’t be a stranger.

Welcome @danda, your contributions appear to be making a difference, thank you.

31 Likes

Thanks @marcin and all the best for the future. Hope to see you around as well.

Welcome @danda. You’ve already made some great contributions, best of luck in your new role!

And thanks again all at @maidsafe for this update. The excitement is building towards new testnets yay! :partying_face:

32 Likes

So far The Best dev update this Year :tada:

Welcome @danda on a board :rocket:

29 Likes

Thanks so much to the team for all of your hard work! It’s going to be a great 2020!

21 Likes

Count me in :fireworks: :sparkler: :tada: :confetti_ball: :zap:

17 Likes

Thanks @marcin, I hope that your future will be good :slight_smile:

19 Likes

Congrats, Maidsafe team! :slight_smile:

19 Likes

This is such amazing news! :smiley: these test will have been some of the most anticipated in history. 2020 has long been seen as a date that represents an advanced future and to think it can be SAFE’s birth year is all too fitting.

Was looking at Safecoins RFC Farming section and it made me curious what priority push notifications/messages are for the network?
Fantastic to have @oetyng on this and @danda as well, congrats on the hire btw!

Thank you @marcin for all you’ve done. I’m glad you were able to get RDF SPARQL stuff set up to punt down the road with Nikita. Wishing you well.

The UI/UX stuff is looking top notch @ravinderjangra and @JimCollinson. Can’t wait to see what you come up with for vault UI Jim, I am particularly partial to your design style :yum:

The way the tokens are being implemented with labels in mind is clever stuff! Rooting for you Josh :love_you_gesture:

Man, this is going to be sweet.

35 Likes

@Nigel

Where can I learn more? I read the Safecoin Revised RFC and I don’t recall it. Is it similar to Colored Coins?

8 Likes

This is referring more to authorization, permissions, and the labeling of data so as to ensure data can have more granular control, indexing, and avoid siloing of ones data by applications.

14 Likes

I see “early stages of safecoin farming” and I’m thinking “omg it’s coming, it’s coming”. I know it’s all early and stuff but it’s still exciting.

20 Likes