SAFE Network Dev Update - October 31, 2019

MaidSafe announcement, please read

Summary

Here are some of the main things to highlight this week:

Marketing

It’s Halloween guys :jack_o_lantern: And hopefully, you’ll find this week’s update devilishly full of treats to get your fangs into.

First up, we have an updated primer! A massive thank you to @JPL who has updated it with the now fully ABFT consensus algorithm PARSEC, ways in which the Network defends itself against common forms of attack, the new AppendOnly data type and the introduction of the quic-p2p routing library which replaces Crust. Absolutely fantastic work by @JPL as always. The primer is a staple marketing tool we use when we’re speaking to people about the Network, or when we just need to refresh our memories about different parts of the network, it’s always close by. It’s so good, it’s scary :skull:.

We’re also almost ready to send out another October Newsletter. There’s a couple of tweaks to make (and we want to get this out 100% perfect) so this will likely reach your inbox tomorrow, just in time for the witching hour.

And finally on this ghoulish night (ok we’ll stop with the Halloween references), it was a pleasure to see some of you switch on to the new season of Silicon Valley. We, of course, acted as advisors on the last season, but we’re sure this season will continue to drive the conversation forward about the need for decentralised networks which is always great news for us.

Vaults Phase 2

Project plan

We have succeeded in running Vaults with the integrated Routing. There is still some work required to set up proper sections and to make the SAFE Client Libs aware of multiple Vaults, but we are getting the first promising results: the nodes are exchanging messages and voting on events through PARSEC and these results demonstrate that we are getting close to fulfilling the goal of the Phase 2a, that is running multiple real Vaults within a single network section.

With Vault phase 2a nearing its end and the rest of the tasks being sequential, the team now has some time to add a few enhancements and newer APIs to the SAFE Client Libs and thus are temporarily switching to tackle down the new project board aimed towards the next version of SCL. This board will be used to track the upcoming & newer high-level APIs and also issues from SCL met by the CLI team.

SAFE CLI (Command Line Interface)

Project plan

The development of authd (Authenticator daemon) continued with good pace and we were focused on adding support for Windows. We are very happy that it can now also run as a Windows service. Creating a service on Windows wasn’t that complicated but we did have some minor challenges to face, e.g. Windows needs the service to first be installed before it can be started (as opposed to Linux/macOS where it can be simply launched), so we had to add a couple of additional commands (install/uninstall) to authd which are supported only for Windows now. Windows also requires admin permissions to not only install the service but also to start and stop it, so we’ll be adding some Windows-specific details to the User Guide to cover these aspects.

authd keeps track of the authorisation requests received in memory, and it also has the list of the endpoints subscribed to receive notifications for these authorisation requests (e.g. SAFE Network App would be a subscriptor), it’s crucial it keeps those lists tidy in order to not allow them to grow indefinitely. Thus we worked on making some enhancements to cope with a few scenarios where auth requests need to time out and automatically be removed from the list, as well as automatically unsubscribe endpoints which became unresponsive when a notification was attempted to be sent. This all helps authd to be more stable and to use a reduced amount of memory and resources.

We are now trying to start with some more sophisticated integration tests, ideally trying to make sure authd works nicely with SAFE CLI, SAFE Network App, SAFE Browser and web apps, and if all works as expected we will be able to share it very soon for you to try it out.

This point seems like a good time to bring QA into the process so they are actively going through the latest authd developments and putting it through its paces.

In addition, @marcin has lately been involved with the Foreign Function Interface portion of this crate, doing code reviews and helping the team with FFI-related questions. This week, he did a small refactor moving FFI-specific code to the FFI module. He has also raised some minor PRs to fix style issues, making the Rust code more idiomatic.

We are also having some design discussions in relation to how to keep evolving the safe-api. We are now looking at ways to expose more APIs that will allow users to manipulate all the data types supported by the Network, i.e. apart from the ImmutableData APIs we are already exposing, we would like to start exposing APIs to directly manipulate published/unpublished AppendOnlyData, and MutableData. We are of course trying to make sure they are as friendly and simple to use as possible, providing nice abstractions for the most common use cases first. We’ll be sharing more details once we have an initial draft of all this.

SAFE Network App (desktop)

Project plan

The past design sprint has seen our focus turn to Vaults, and more specifically how a Vault can be installed and run easily from the SAFE Network App.

We started first by addressing the onboarding flow for a new user. How can they offer up some of their computer’s resources and earn their way to a fresh account?

We looked at several approaches to this initially, but settled on reusing some of the language and journey flows from previous user research, by enabling the user to use a Vault to generate an account invite—prepackaged to give them all they need to get up and running. This will, of course, need to be tested with real users in due course, but for now we build out the full Vault suite with this as our baseline.

From there, the user can keep the Vault running, earning them Safecoin to fund their day-to-day Network use.

But this onboarding flow is only one small part of the complete Vault UI picture. What is an ostensibly simple program, has various modes of operation, which we need to take into account. So we’re building and considering flows for:

  • Allocating hard disk space and storage location
  • Starting and stopping the vault
  • Seeing the Vaults status, a log of events, and current earnings
  • Changing basic settings, such as enabling auto-restarts
  • Electing to send earnings to a specific wallet or SafeID
  • Starting and operating the Vault while logged out, and then ‘linking’ it to your account later, allowing it to be protected, and sweeping funds to the default account wallet.
  • How we might allow for monitoring and configuration of Vaults running remotely

Of course we’re putting things together with the core functionality in mind, but we’re still thinking about how we allow for more advanced monitoring and richer statistics in short order.

We aim to make things solid, useable and understandable. We won’t get there in just one leap, but a series of steps and feedback loops. We look forward to showing you all those pieces coming together very soon.

On the implementation side this week, we’ve been syncing with the latest authd changes as described above and have started implementing flows for simply accepting or rejecting authentication requests.

SAFE Network App (mobile)

Project plan

We’re really excited :smile: to announce that the development work for the SAFE Network App Mobile has kicked off, as the C# APIs are available and the NuGet package isn’t too far away.

We have already completed the initial project setup and this week we spent some time to scope out and plan the tasks for the very first version of the SAFE Network App on mobile. We are focusing on the UI design first and have started to work on a couple of screens.

We’ll let you know more details in the coming weeks about how the SAFE Network Mobile App is going to be a game-changer and a one of a kind application on mobile. The design put together by the UI team is really incredible and it is going to be a real joy to develop and use the app.

SAFE Browser (desktop)

Project plan

This week we’ve released a new version of the SAFE Browser (v0.15.2), which updates Perpetual Web resolution logic and finally exposes the XorUrlEncoder. It also updates Electron to v6.1.0 and fixes several bugs. And on macOS, you will notice that the app is now notarized - no more errors when opening on macOS Catalina!

When opening a previous version of the perpetual web browser you will automatically be prompted to update to this latest version, or if you are downloading for the first time, use the link above to the release on GitHub.

Beyond this release, we’ve also written some more detailed tests for the new resolution logic and started the path to upgrading to Electron 7. We’ve also been working on some new features to more easily test and check releases without complications arising from the auto-update mechanisms.

SAFE App C#

Project plan

A lot of work has been going on, over the past couple of months in safe_app_csharp and as you can see in the project board we are very near to the fruits of success (in this case, release :smile:) and along the way we have tackled different problems on different platforms.

For the past couple of weeks, we were having issues with the iOS CI build and test pipeline and we weren’t able to test out APIs on the CI. This week after trying different methods and tools we managed to fix (#159, #162) the issue and now we are successfully running the tests for the new C# APIs on all supported desktop and mobile platforms.

We have also exposed a new API for setting the vault configuration file path that is needed for mobile platforms to connect to the local/shared vault (#158).
This wouldn’t have been possible without @lionel.faber and the SCL team who helped in creating a function for setting the vault configuration file path in SCL (#1051). Using this API, we have finally managed to connect a mobile device to the shared vault. We still have some work left in connecting to a local vault, which we hope will be resolved very soon.

We have also exposed the newly added permissions for apps to access the balance and perform mutations (#156).

Node Ageing

Project plan

We have been continuing our efforts to enable the relocation mechanism and to implement Elder promotions and demotions.

On that first front, the mechanism itself was implemented last week, now we are working on actually triggering relocations when applicable. This is progressing, but there are still details that need to be polished and worked out.

With the Elder promotions and demotions, we ran into problems with how they are playing with our current way of handling churn. As it turned out, the multi-step process we have in the current code often leads to sections becoming stalled when promotions and demotions are enabled. We are currently working on simplifying this and removing the risk of a section getting stalled when the set of Elders changes.

Additionally, the first part of the work on simplifying connections between Elders is now complete, unblocking the work to handle multiple joining notifications at the same time. This new test also proved useful in uncovering corner cases that we are working on fixing. Finally, we made further progress on tracking whether Elder nodes are still responsive or need to be removed.

Useful Links


Feel free to send us translations of this Dev Update and we’ll list them here:

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the SAFE Network together :tada:

59 Likes

Thanks for your efforts, @JPL! Super important to have the primer; I think I’m going to read it some more soon!

36 Likes

2nd.……

11 Likes

Better 3rd than never.

11 Likes

@JPL - You are well on the way to create the first SAFENetwork text book. You should publish. You could add a few chapters on development for the safe network. Maybe we could all contribute with safenetwork tutorials and exercises.

Great update. Thanks for the treats.

23 Likes

6th… Happy Halloween :jack_o_lantern: :ghost:
Congrats for the awesome work @JPL !
Now reading further

Thanks for your valuable contributions to the app @lionel.faber ! You rock!

This is what we’re here for! Open source is the way towards great things!

18 Likes

Thanks so much to the team for all of their hard work!

And Happy Halloween! :skull_and_crossbones: :ghost: :jack_o_lantern:

14 Likes

Such an important project; amazing leadership; and one that keeps delivering solid progress… supporting it, is an easy decision. :+1:

I wonder we all appreciate the transparency and real world issues… it’s a rare project leadership that know to do that.

All those changes made and especially the focus on product, will only help.

Congratulations and a big thank you to all those involved past; present; and future!

:smiley:

19 Likes

Thanks for all the hard work @JPL … definitely when we enter the stage where it is profitable to sell this idea to other people I see myself making lots of use of that. Really should send you a cut of the profits!

4 Likes

Hurr durr derp derp Hurr derp

3 Likes

Great work !! I really happy to see primer

6 Likes

wow… :+1:

(default needs a fix? : You need to enable JavaScript to run this app.)

5 Likes

The pace continues even as the team slims down, and looking from here handled impeccably.

I’m very sorry to see such competent and lovely people let go. I am grateful to each of you for the marketing we’ve seen and the behind the scenes activity we don’t. All the best to each you. I hope you will still feel part of this great community.

This project is becoming even more laser focused, and will I believe continue at this pace or likely accelerate in delivery of tasks, milestones and… products!

I’ve been enjoying picking up tidbits and learning to use the new API without disturbing the team, although they do still pop up with hints and tips - thank you for that. And now brave @folaht has joined me (on the Dev forum), so we can help each other while the team gets on with things. It’s great to hear they are looking at exposing more of the API, although that hasn’t been a limitation for me so far.

So thanks to all are Maidsafe, those leaving, those lucky to stay and work your socks off. :wink:. The progress this week is amazing, and exciting to see.

And special thanks @JPL, the primer is such a valuable resource to be able to share - which I do regularly, so knowing it is up to date is great :slightly_smiling_face:

25 Likes

This is both positive and negative news.

The negative is on the people side of things and a big thank you for all the hard work for those no longer with the project.
On the positive side, and to all the Nervous Nellie’s out there, you now have a clear timeline to launch (or not).
I can’t wait!!!

8 Likes

Any updates on the academic review of PARSEC?

5 Likes

This is essential for JAMS and obviously others as well! Really great to hear this and we are super anxious to get on this (not having to try and hack up workarounds) and reveal an alpha once we can get at these!

Sorry to hear that offices are closing and there will be departures but this community can easily help take this the extra mile and I encourage anyone with a bright idea to build, collaborate, and share the excitement of your ideas to drive more competition and inspire others.

19 Likes

“succeeded in running Vaults with integrated routing…” This is so exciting. Can’t wait to see what transpires in the next few weeks.

10 Likes

This probably is needed for a .NET Python too :hugs:

3 Likes

Strange post. The marketing was giant waste of time and money. Why would anyone try to sugarcoat it?

That’s how I see it, you are welcome to say what it is you disagree with, but just saying you disagree doesn’t add anything.

2 Likes