SAFE Network Dev Update - August 13, 2020

[maidsafe]

Summary

Here are some of the main things to highlight since the last dev update:

• We have finalised a first implementation of our new quic-p2p async API, another huge step in simplifying our codebase. Work to get our libraries up to speed with the new APIs is underway.
• We have an update on the latest progress on the SAFE Network App UX.
• Work has commenced on implementing the tree CRDT in Rust.
• @JPL updated the SAFE Network Primer to include all the latest developments.
• Last Friday was @ravinderjangra’s last day at MaidSafe. We wish him all the best for his future and thank him for his involvement in the development of the SAFE Network over the last 2 years, especially everything related to the mobile apps and APIs!

CRDT

safe-fs

This week work has begun implementing the tree CRDT in Rust. Yesterday a milestone was reached when the code correctly executed the two problematic concurrency conditions described (and solved) in the paper. Further refinements and test cases are in the works.

Farming

SAFE Transfers Project plan
SAFE Client Libs Project plan
SAFE Vault Project plan

SAFE Client Libs and Vault integration

With the async quic-p2p development complete this week, we are now working on getting the libraries up to speed with the newer APIs from quic-p2p. Previously SAFE Client Libs had the implementation of two communication choices - send which sends a Query and waits for a response and send_only which just sends a Command and doesn’t wait for a response (more like the Fire-and-Forget strategy). Now all of the hassle has been moved to quic-p2p and been made much simpler with the async/await paradigm, while also making the Client Libs lighter. With that done, we will no longer be seeing the blocking error from quic-p2p that happened previously during integration, as mentioned in the previous Dev update.

Meanwhile, the Vaults have also been given an improvement where we streamlined the integration of the Vaults and Routing layer in PR#1064. All the modules inside need to use and communicate with the Routing layer which made it crucial for us to stabilise the flow of the objects. With this change, all that the Vaults would need now to achieve the same is to pass the Network type which then holds all the logic for making the usage of Routing APIs simple.

To make the process of integration testing easier, we are also working on a minor (multi-threaded) framework under a feature-flag in Vaults that spins-up a SAFE section at the call of a function and runs the specified client-side tests that are present in the SAFE Client Libs on that section. This maintains the essence of realness of the network with real QUIC connections and with real clients, removing the need for any mock implementation and moreover making this work both ways to significantly fast track the development and integration of SCL and Vaults simultaneously.

Rewards

We’ve implemented the wallet registration for reward payouts.
A node wishing to join the network is able to specify a wallet address in the config. When the node has joined a section in the network, this wallet address is sent to the Elders of the section, for the eventual payout of rewards.

SAFE Network App UX

Feature Tracker / Screens and User Flows

First off, if you’re delving into the Figma files above to follow along with the latest changes you should notice a decent performance improvement after our relentless pestering of senior Figma developers.

A few disparate updates to share with you:

App Permissions are now Capabilities

We’ve previously walked you through how you’ll be in control over your data, and how and when each app can be used by you to access it.

Until now, we’ve been calling these App Permissions. But in order to not anthropomorphise Apps, and more accurately describe what’s going on, we’re changing this to Capabilities.

So rather than giving an App Permission, to say edit an image —when really you’re just giving yourself the ability to edit the image using that app— you’ll give it the Capability to do that.

It also allows us to reserve the term Permission for when you are sharing, or managing access to data, with another user or group of people.

Floating Action Buttons now with descriptions

We’re using floating action buttons (FAB) in a number of different contexts. While the material guidelines favour just a single icon, given the number of different contexts, we felt it enhanced the clarity and usability to include an action description within the FAB, so you’ll see this change roll out through various screens and flows.

Login prompts from External Apps

We’ve also been working on flows for triggering log in events from external apps, and how these are handled, both in terms of maintaining an understandable flow when they happen, and popping the user back out into a helpful place after it’s complete.

Publishing Timer Snackbar

And a final wee update. We’ve made a change to the publishing process that includes a snackbar with a countdown timer, enabling the user a small cooling-off period before perpetual data is published. This is instead of the alternative of a short ‘undo’ window, which we’d previously mooted, and more accurately reflects what is going on behind the scenes.

Routing

Project Plan

As mentioned in last week’s dev update, we have been working on some Routing improvements, with two further improvements merged this week: remove bounced_unknown_messages cache and remove unnecessary send_neighbour.

There is also on-going work to remove SendNeighbourInfo votes, which currently has several random tests failing but we expect those to be resolved and merged soon.

Since last week’s debate on the relocation refactoring work (Remove relocate_queue and make relocation independent on the order of churn events), we found a promising solution which Adam has already started implementing. There are a bunch of failing tests to be overcome first (as relocation is a key feature which affects the testing framework a lot), before raising a formal PR for it.

This past week we finalised a first implementation of our new quic-p2p async API, which as in all other crates is helping immensely in simplifying our codebase. This new API follows the pattern used in the async world, exposing a stream of messages the consumer can just iterate over asynchronously, which for us means removing any type of event loop implementation of our own. We’ve been testing this with our client libraries to make sure it fits well in our software architecture and it has been showing very good results so far.

We are now trying to do the same migration in the routing library to not only use the new async quic-p2p API but also to expose its own API as async, which as said above will simplify the codebase massively both in routing and vault crates. As you may realise this is all internal refactoring, the benefits of which can be difficult to see for end users, but we are very positive this will allow us to find and solve any issues we find in the future more easily, as well as make it easier to maintain and evolve existing features.

Useful Links

• SAFE Network Website
• SAFE Network Primer
• Network Fundamentals
• Roadmap
• Glossary

---

Feel free to reply below with links to translations of this dev update and moderators will add them here:

Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the SAFE Network together!

[happybeing]

Tada! Still life in this old dog Thanks all at Maidsafe - I particularly like the refactoring around async as I’ve just used it and can see how much it could simplify the code. Rust has some very nice idioms (e.g. match) that make coccurrency really easy and ‘safe’ as well as ‘SAFE’ . The UI changes sound cool too. And of course @danda is still tearing through the code - I didn’t know he had something working in Rust. That’s great news.

Bye bye @ravinderjangra, thank you for all your excellent work on SAFE and good luck. I hope you’ll stay in touch.

And I have a report of my own…

I’m not up to writing proper Rust for the TreeCDRT / filesystem yet, but I am writing code in Rust and very pleased with my progress and the language.

It is a lot to get used to all at once, and some things are a pain to figure out, and there’s lots of fiddly characters to get in the correct place n stuff, but… the compiler and other tools, and the language itself are excellent. In no time I’m using async style functions (which I know from JavaScript) to create multiple tasks in a little dashboard app I’m writing as a learning exercise.

So I recommend it, and feel free to join me here. I’ve posted a little update on the app I’m making for my Rustification on that topic.

[KafkaLee]

second!!! someone said it is impossible network, but it is becoming possible network!

[Nigel]

Sorry to see you go @ravinderjangra but I wish you well in your future endeavors. You have brought mobile to us and much much more, massive gratitude.

I like the UI design changes and clarifications. Glad this is still evolving to bring more clarity to the user.

Farming, wallets, AT2, simplified codebase oh my! Keep it coming @maidsafe we will test so hard and spin up more vaults than imaginable once it’s good and ready. Need to nip those connection issues too though so anyone can join and run a vault. In due course. Cheers!

Oh and huge thank you to @JPL!!! I’m excited to look over the primer again tonight.

[davidpbrown]

@ravinderjangra Good luck

@maidsafe

in order to not anthropomorphise Apps

until the day of AI Apps.

@maidsafe

a small cooling-off period before perpetual data is published.

Prompts a thought of whether there is a simple switch for making mutable data immutable.

[Nigel]

Couple more thoughts. I’m glad the login permission stuff is being thought about and handled now rather than later!

Also I can’t wait to store my files on safe-fs

[Southside]

Thanks to everyone for their hard work this and every week - sorry to see you go @ravinderjangra - all the best and don’t be a stranger.

Just a few of the points that caught my eye

@maidsafe

We’ve implemented the wallet registration for reward payouts.

Another of the fundamental building blocks in place

@maidsafe

It also allows us to reserve the term Permission for when you are sharing, or managing access to data, with another user or group of people.

This is good, more natural use of language.

@maidsafe

removing any type of event loop implementation of our own

• simplify and add lightness -

Thanks again to everyone, the progress shown here emphasises my view that noting around the ERC-20 token debate must be allowed to distract the devs

EDIT: ooops - forgot to add my thanks to @JPL for the updated Primer - I’ll go through that again and I m looking forward to sharing it again as a “must-read” to get up to speed on SAFE.

[davidpbrown]

@Nigel

I can’t wait to store my files on safe-fs

Having somewhere to store files safely
it’s a huge plus… small capabilities in new utility will remind people of alsorts they are missing without knowing it. I’ve had too many hard drives fail… a pina fixing the difference from backups.

[19eddyjohn75]

Thx for the great update Maidsafe devs

@ravinderjangra’ wish you the best and hopefully to see you have a SAFE project in the future

@JPL

@maidsafe

A node wishing to join the network is able to specify a wallet address in the config.

Could a hacker easily change this WA?

@maidsafe

We’ve made a change to the publishing process that includes a snackbar with a countdown timer

Love this feature

Keep hacking super ants, we’re almost there

[davidpbrown]

@19eddyjohn75

Could a hacker easily change this WA?

Could crypto-sign the config?.. nice idea to lock the config in some way, given it’s running for a long while… and might necessarily reboot on occasion.

[Nigel]

Secure Enclave’s are in future work. That would be pertinent, no?

[Nigel]

So far Maidsafe is on top. Don’t be complacent though, be sure to vote. Every little win counts and it could help save some money.

[Secretariat415]

Thanks so much to the entire Maidsafe team for all of your hard work and great progress!

[Nigel]

@JPL just glimpsing through the primer and great work! The section proof chain was a point of clarification for me and the CRDT and AT2 were also excellent explanations. I think it was in the data type section though, immutable data was included in the data types being replaced and I don’t believe that’s the case but if you could confirm that, it would be good to know. Thanks for updating this amazing resource!

[JPL]

Thanks @nigel - it’s been fascinating going through it all and seeing how the new features fit in with the overarching plan. The team are pushing through some pretty fundamental changes all aimed at simplicity, rationalisation and performance and it’s great to see how it’s all gelling. Data types are a moving target, particularly with regard to making them CRDTs, but that only applies to the mutable types. As far as I understand it, the only agreed change to ImmutableData is that its name has changed to Blob, making it a noun rather than a verb/adjective as with Map and Sequence. In all other respects it’s the same. I’m expecting to update this chapter quite frequently as things change though as it’s very much a live issue.

Please ask any other questions or make any other suggestions here, as small changes are easy to make on the live site. If need be I’ll make a separate thread for this, but for now if anyone spots anything just let me know here.

[Sotros25]

@maidsafe

@JPL updated the SAFE Network Primer to include all the latest developments.

This is great! I know a lot of people will find this very helpful.

All the best, @ravinderjangra!! Thank you for all you’ve done to help make the SAFE Network a reality! Hope to still see you around.

Thanks to the MaidSafe team for continuing to press forward!

[JPL]

All the best @ravinderjangra - it’s been good following your work. Best of luck wherever you’re going next. Hope you keep in touch.

[Nigel]

@JPL I got that sense reading it too. Really amazing indeed.

Thanks for the knowledge on the name change from immutable data to blob.

Also there was a mention of merging in the vault persona section I believe. Besides that I didn’t catch anything that jumped out. Great work.

[davidpbrown]

@JPL

small changes are easy

It’s good work … I wonder sometimes how many visitors fall over the must use javascript error default but perhaps most people have not control of scripts and those who do don’t mind. Some sites perhaps doesn’t fall back easily…

[JPL]

Good spot - corrected now More eyes = fewer errors. Found and mopped up couple of other rogue merges that were still hanging around too.