SAFE Network Dev Update - October 24, 2019

Marketing

This week, if you are one of the privileged individuals that are signed up to our newsletter then you’d have received a summary of the recent deliverables over the past few weeks. For those of you that aren’t signed up - why not? Head over to safenetwork.tech and register for regular emails which vary between updates, like yesterday’s, or more thought leadership editorial musings about the state of today’s Internet.

There’s still time to vote for @dimitar and his nomination for the site of the year. I know many of you have already voted (ourselves included!) but make sure you don’t forget - so why not head over and do it now? After you’ve finished reading the rest of the dev update of course :smile:

Vaults Phase 2

Project plan

This week we have made good progress on the Routing & Vaults integration: the required API changes are done and we’ve also fixed the affected Routing tests. We are continuing with the Vaults code now and expect to start running the first integration tests soon.

With issue 882 closing, all client requests now go through the consensus mechanism. The test environment with mock-routing now performs client operations on all client handlers when it receives a request. The next step is for a client to connect to 2/3rd of its client handlers in the section and send the requests to the nodes in its connected list. This implementation has started in PR 886 and brings us one step closer to a client connecting to a section.

As phase two of Vaults keeps chugging along, we’ve not forgotten about the shared vault that everyone is using via the SAFE Browser (with Perpetual Web functionality) or the CLI app. Since the phase one release, we’ve added some features to SAFE Client Libs and the Vault such as granularity of permissions for apps, coin refunds for failed operations and more. These updates are currently being adapted by the CLI and the language APIs so expect updated releases all around very soon.

SAFE CLI (Command Line Interface)

Project plan

During the last week, we focused completely on the implementation of the authd (Authenticator daemon) and the APIs to interact with it as a client. We were able to do some refactoring and restructuring of the authd code as well, since the initial draft implementation was needing it, but now that we got to the point of having all the functionality for the basic app authorisation flows in place, it was required to be able to keep evolving it.

The last API we were working on was to allow applications to subscribe to receive notifications from authd when an authorisation request was sent by any SAFE application. An application which subscribes to these notifications can then prompt the user to allow/deny the authorisation request before requesting authd to give the credentials to the app or not. As you know this is what is required by any Authenticator UI, like the SAFE Network App. This was the last feature we wanted to have before moving forward with the next steps in getting authd ready.

We are now focusing on making authd stable, fixing some known bugs, adding support for Windows, and having the CLI expose auth commands for all the operations authd supports, which would finally deprecate the safe_auth CLI, leaving a single SAFE CLI for all operations.

We are really looking forward to sharing this with all the community as we believe it’s a nice step towards simplifying the user experience around app authorisation flows, as well as making it as simple as possible for apps to exchange authorisation requests/responses with authd from any device.

SAFE Network App

Project plan

We’ve been finalising the update behaviour with an implementation underway for apps (initially just the browser right now) to update themselves in the background silently. All we have left to do here is finalise some of the data management in the SAFE Network App and put everything through some internal testing. We’ll then have application updates capable of being managed from one location!

SAFE Browser (desktop)

Project plan

This last week has seen us grappling with both Travis and Apple in an attempt to get their new notarization system integrated with our CI. We’re still seeing some issues there, but we have it properly integrated into the build system now, which means we can do releases!

We are working on a final few bugs but will be releasing a new version of the browser in the coming days. The soon-to-be-released version will integrate a new safe-nodejs update, exposing the XorUrlEncoder for apps to utilise. It will also update some of the pWeb resolution logic, blocking resources requested by a page from another location, if those resources are not themselves versioned. This also led us to do a bit of a refactor on the page resolution logic to be able to make this testable and catch regressions here more easily. Those tests will make their way in soon, alongside some other updates to dependencies and the required updates for SAFE Network App compatibility.

You can check out the new resolution rules when the latest browser is released! Keep an eye on the forum and our social media accounts.

SAFE App C#

Project plan

This week we started with some repo restructuring tasks and merged the code for all new APIs in the master branch and the stable code for the alpha-2 network is available in a new alpha-2 branch.

We added some more tests for the fetch and inspect API and found an issue where the inspect API was returning the wrong result. The issue was resolved and the tests are passing now for all desktop platforms.

With a working setup on desktop now in place, we have moved on to integration testing on mobile platforms and it has been exciting so far. We required some changes in the Rust libraries that the Client Libs team quickly sorted out. We then identified that the native libraries generated for Android weren’t compliant with the default device environment. With the right amount of tinkering we resolved this along with a few other bugs that surprised us.

We now have the tests in the C# library running successfully on Android and iOS with the exception of a couple of APIs. This means we’re not far away from a package release so keep an eye on the SAFE App C# repository for updates.

Node Ageing

Project plan

The work on node relocation has been progressing nicely. The actual relocation flow was put into place last week, and this week we implemented the relocation trigger - that is, the decision of when a relocation should happen, which node(s) should be relocated and to which section they should be relocated to. We also discovered a flaw in how the proof for the target section was constructed, but we promptly fixed it. The last remaining test failures are due to us sometimes being too eager with the relocation, but a fix for this is being worked on as well. The final piece of the relocation story is to implement the mechanism for the upper layers to request more nodes into the section. The work on this has already started.

We are also making progress in further simplifying establishing communication channels between elders. We are working to package the information needed to send a direct message with the notification indicating which Elder to communicate with. This can be either within the node’s own section or its neighbours’. This will help eliminate a category of bugs we’ve been facing regularly.

We are also progressing the improvement work to support Node Ageing and BLS: We are making good progress on detecting whether nodes are responsive enough to keep being Elders, as well as better handling multiple notifications that nodes have joined or left at the same time.

Useful Links


Feel free to send us translations of this Dev Update and we’ll list them here:

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the SAFE Network together :tada:

84 Likes

Am I actually first?

Now I’ve had a read, looks like you have all had another busy week.
Thank you team for all your effort, once again I am very happy with the progress made.
I’ve not done any testing of all the cool new stuff you keep on creating, I did try to help out with the community network, but was away for many months caring for sick family and could not reconnect my vault once it fell offline.
Really looking forward to Fleming though.

Hope you all have a great weekend.

32 Likes

From second I would say yes. Who is on first :slight_smile:

24 Likes

What’s on second :stuck_out_tongue:
[20 chars]

18 Likes

A leather jacket? :wink: ‘What’s on second’ is an antique store apparently.
Fine update by the way :+1:

15 Likes

Just in case you haven’t seen this classic … https://www.youtube.com/watch?v=kTcRRaXV-fg
Abbott & Costello: Who’s on first.

20 Likes

for-your-hard-work-thank-you-for-your-hard-work-52868436

16 Likes

This is great. Finally the surface is becoming simple (well, simpler). Careful though, this kind of stuff combined with the UI/UX mockups and it may actually become hard to convince people how powerful it is under the hood haha. Another fantastic update.

23 Likes

Thanks so much for all of your hard work. There will be a pot of gold at the end of this rainbow.

11 Likes

Thank you team. All areas coming together fast :smile: but I couldn’t wait for the SAFE Browser changes that are coming in v0.15.2. So today I’ve been peeking at (and breaking) the upcoming browser release, but for devs here are the Safe() object APIs for v0.15.2 (not yet released):

auth_app
connect
fetch
files_container_add
files_container_add_from_raw
files_container_create
files_container_get
files_container_sync
files_get_published_immutable
files_put_published_immutable
inspect
keypair
keys_balance_from_sk
keys_balance_from_url
keys_create
keys_create_preload_test_coins
keys_transfer
nrs_map_container_add
nrs_map_container_create
nrs_map_container_get
nrs_map_container_remove
validate_sk_for_url
wallet_balance
wallet_create
wallet_get
wallet_get_default_balance
wallet_insert
wallet_transfer
parse_and_resolve_url
parse_url
xorurl_base

Note:
bold: new since v0.15.1
strikeout: removed since v0.15.1

Note also the new XorUrlEncoder API mentioned in the OP.

NEW dWeb Blog: Why Solid on SAFE?

There’s a new post on dWeb Blog: Why Solid on SAFE? the content of which is self explanatory. Please read and share if you think it interesting to others.

My thanks to:

  • @david-beinn for being guest editor and vastly improving the piece
  • @dimitar for the header image (credited at the top if you look carefully)
  • @Nigel for his earlier safegit image (belatedly credited at the top of that post!)
33 Likes

Just to clarify, they were never working and we commented them out, but we should be exposing them as soon as we have the chance.

15 Likes

Unlucky 13th*

*base9

Another very fine update, full of solid progress both in Ayr and associated offices and from the happy community — Sincere thanks to you all

10 Likes

Thanks for your hard work Maidsafe devs,

When Pwn2Own?

:sweat_smile:

6 Likes

We are working to package the information needed to send a direct message with the notification indicating which Elder to communicate with. This can be either within the node’s own section or its neighbours’. This will help eliminate a category of bugs we’ve been facing regularly.

This sounds extremely flaky. Is not the existing network picture/consensus model enough to know who to communicate with?

Why does it result in bugs instead of expected and anticipated error handling?

What are the security implications of taking orders who to communicate to instead of deriving it yourself from your understanding of the network you exist in?

3 Likes

It is subtle, your view is your view in some space and time, consensus view is guaranteed to be a view all others see in the same order. Consensus view may never see your view (you were wrong) so we use consensus as a this happened and agreed by quorum it happened. I hope that makes sense.

14 Likes

Anywhere I can read about the design of this? I couldn’t see anything in RFCs that stood out as an obvious place to look.

I’m especially interested in the new nodes allow / disallow rules since that’s a very important aspect of network growth and adoption.

9 Likes

lol watched the whole thing. awesome

4 Likes

Not yet, except code… Basically the impl for this so far is we accept nodes up to SAFE_SECTION_SIZE and then the upper layers have to tell routing to accept new nodes (so it will accept new Infants and/or try to relocate a node from another section). This is all similar to farming right now. So all the mechanics in place, an initial algorithm and then we test/debate the whole thing in testnets/releases through to launch.

14 Likes

How is the office in India? Someone just asked me this

3 Likes

I can dig that, sometimes you are put into a position where waiting for community agreement and consensus is going to be too time consuming. Implement a solution the MAID group deems optimal for an MVP. As long as the code itself in the complex portions gets a few comments from those developing on it, community can always scan the code and rehash alternative improvements down the road.

:ship: it.

9 Likes