SAFE Network Dev Update - July 4, 2019

Summary

Here are some of the main things to highlight this week:

  • We’re beginning the design process for the Perpetual Web milestone with the simplest set of user stories—centred around the read-only mode of use. You can head on over to the GitHub repo, and follow along with the progress as the designs unfold.
  • We’re expecting the current milestone of the SAFE CLI, which brings the files put, files sync and cat commands, to be ready early next week.
  • We’ve been working on the implementation of our own encoding for the XOR-URLs. We have just implemented our own encoding which incorporates not only the XOR name and content type in the XOR-URL string but also the type tag.
  • The SAFE Client Libs team has begun adapting and implementing the FFI layer for the new data types starting from Mutable Data. Which means that the day for the non-Rust folks out there to interface with the new data types is arriving soon.

Marketing

This week, the marketing team have been exploring in more detail the 4 milestones @JimCollinson shared last week. The first one we’ve dug into is Perpetual Web (here is the Medium link and forum one). This is really exciting and opens up a new world where public data lives on indefinitely. It cannot be deleted, removed, or tampered with. A network of truthfulness. Also, who loves the video from 2007? It’s great, isn’t it?! But seriously, it just shows how long these ideas have been swimming around our heads and how we’re determined to follow through on our original ambitions in building the SAFE Network. We’ve got another three of these posts exploring the other milestones Take Control of Your Data, Private Communications and the New Digital Economy to come over the coming weeks so stay tuned…

@SarahPentland has also published our monthly round-up, so for those who want a neat summary of all we’ve achieved in June, go check it out.

And good news for those who reside in the Glasgow area - we have another meetup :tada:. We’re just sorting out speakers but for those who attended the last time, it was absolutely fantastic meeting the community and we’re looking forward to seeing you all again, and hopefully even more of you! It’s on the 22nd July but head over to meetup for all the details.

A quick update on exchanges in case you’ve missed it. We’ve taken the decision to remove all of our assets from the Bitker Exchange. Instead of clogging up the weekly dev update, you can read about our reasons on this forum post.

Finally, a piece of team news to wrap up with. We’re happy to announce that with the recent move by @dugcampbell to COO, @Cgray has now been promoted to Head of Marketing. Congratulations Ceilidh! :tada:

User Experience

Following on from last week’s UX roadmap announcements, we’re losing no time in getting stuck right into the making.

So, here we go, let’s start taking the wraps off the first piece of the Perpetual Web.

You can head on over to the GitHub repo, and follow along with the progress as the designs unfold, but for the non-git-hubbers amongst us, let me just drop some of the detail right here.

The Perpetual Web Browser

The Perpetual Web is, of course, the first end-user focused milestone on the way to the launch of the SAFE Network. It will showcase and enable the perpetual, versioned nature of all public data.

The app at the heart of this is the SAFE Browser. So first off, we’ll be dealing with the features that will extend the browser, creating the Perpetual Web UX.

User Personas and Usecases

We have three proto-personas for the Perpetual Web milestone.

  1. Laurel: The Consumer
  2. Tony: The Contributor
  3. Yanni: Web Developer

If Personas and how designers and organisations use them is new to you, here’s a wee primer, courtesy of Wikipedia.

Just think of them as a way of synthesising a large group of real human target users, in such a way that a whole team can have a common understanding of the needs, goals, and behaviours of the individuals we are making things for. Part design tool, part internal communication tool, and a way of keeping on track, together.

Wireframes — Laurel: Read Only Version History

We’re beginning the design process with the simplest set of user stories—centred around the read-only mode of use—catering for the needs of Laurel. It’s a solid place to start as it will involve the important set of features around version history, understanding authorship identities, and exploring linked data, but without the complexity of account creation, publishing, etc.

Fleshing out the stories and user flows for Laurel also helps inform the requirements for Tony and Yanni.

It should be noted that these are ‘mid-fidelity’ wireframes. They should enable you to grasp how the user interface will function, and picture it in use, but they are relatively plain vanilla in their look and feel. We are focusing on function and interaction first, and then aesthetic will come in later iterations.

Viewing Version History History

How does Laurel navigate the version history for a page, or asset? How does she know she is on the latest version, and see what has changed since the previous versions?

What can she learn about the publisher of content, and the owner of the site?

View Full Screen

Error Responses Page

Here we start to detail out the error responses Laurel may encounter, including missing, moved, or outdated content.

You will also begin to see some hooks into Tony’s persona, including creating a Public Name, or accessing Private Data.

View Full Screen

SAFE CLI

Project plan

The CLI has seen some interesting progress this week. The implementation of the files put and files sync command has been finalised and is now going through our internal testing/QA process. The cat command has also been implemented to support both published ImmutableData and FilesContainer XOR-URLs, i.e. files and folders can be uploaded with files put and files sync, obtaining the resulting XOR-URLs which can then be used to fetch the data with the cat command.

The cat command even supports providing a path with a XOR-URL, e.g. a file published in a FilesContainer with path/filename “/myfolder/myfile.txt” can be fetched using the XOR-URL of the FilesContainer with the file’s path appended to it:

$ cargo run --features scl-mock -- cat safe://hbyw8kkqr3tcwfqiiqh4qeaehzr1e9boiuyfw5bqqx1adyh9sawdhboj5w/myfolder/myfile.txt
This is a test to create a text file with some content to work with files put and cat.

Another aspect that has been worked on this past week was the implementation of our own encoding for the XOR-URLs. Those who were involved in the XOR-URL RFC discussion should remember the proposal to use multiformats CID for encoding the XOR name, content type, etc. We have just implemented our own encoding which incorporates not only the XOR name and content type in the XOR-URL string but also the type tag. This means that it’s not necessary to put the type tag as a port number in the URL as all the information is included in the XOR-URL string. This is implemented in an attempt to find out what the best format/encoding for the XOR-URL is while we move forward with the implementation of the CLI. The current encoding scheme is as follows (variable length from 35 to 42 bytes, base32z base encoding by default):

  • 1 byte for version
  • 1 byte for content type (ImmutableData, FilesContainer, Wallet, etc.)
  • 32 bytes for the content’s XOR name
  • and up to 8 bytes for type tag, only if it’s applicable

We resumed efforts this week, a bit more aggressively this time, to integrate the CLI with safe_client_libs. The plan is to continue with this for the next few days and it will be also one of the main goals of our next milestone, which will be outlined next week.

We were originally aiming to finish the current milestone by end of this week, but we decided to invest a few more days to make sure the main bugs are fixed, and some edge case scenarios are well-tested, before encouraging you to start using the new commands. We believe this should be finalised early next week, when we’ll notify everyone so that you can enjoy the new commands available, and hopefully provide some feedback.

SAFE Browser

There is little to update you all on when it comes to the Browser this week. After our release last week, we were simply analysing the next steps for the Browser and the overall priorities across different fronts.

We plan to work on a new release that fixes some very high priority bugs only, while investing more of the front-end resources on some of the other projects we’ve prioritised currently (like the efforts related to the Perpetual Web deliverables). We’ll be sharing more details about our plans for the next Browser release in the next few days though, as before, by simply sharing a link to the corresponding project board.

Mock interface

Project plan

SAFE Clients Libs has had its own share of action this week. To begin with, FFI are in the works! Which means that the day for the non-Rust folks out there to interface with the new data types is arriving soon.

@marcin has begun adapting and implementing the FFI layer for the new data types starting from Mutable Data. Secondly, we have been focusing, reiterating and improving on how Wallet/Account creation and handling could be done in the SAFE Network. Multiple methods are being brainstormed to overcome any disadvantages and inefficiencies. RPCs for the same are being defined and are soon to be integrated with the mock interface so that you can try them out as soon as they are ready.

In addition to that, the next major milestone of Adapting SAFE Client Libs to the new data types is progressing well and we are halfway through. Some of the above tasks are also part of this milestone as well. This project requires the rest of the components in SCL such as safe_app and safe_authenticator to make use of the new data types and slowly replace the legacy ones. As this work proceeds, we’ll simultaneously be working on refactoring the data types as per the needs and optimizations. We’re also fixing pesky bugs and working on correcting the semantics of the data types as we stumble upon them during the milestone tasks. This lets us deliver an issue-free mock-interface once all the hustle ends.

Subtle changes have been made to the data types RFCs (Published/Unpublished Data & Immutable Data) to fix issues and semantic errors we came across during the development. These RFCs are still open for receiving your opinions as well. Please do check them out to see what the changes are, and do give us a nudge if you see something that doesn’t fit well or something that might fit better :smile:

Vaults

Project plan

This week has seen plenty of progress in vaults. We now have four team members working in this area (@adam, @fraser, @jon, and @nbaksalyar) and to an extent in safe-nd, since it’s tightly coupled to the vault crate (it defines the types which are common to vaults and SAFE Client Libs (SCL) - so mainly the RPC’s and chunk types).

We’ve completed most of the work for the first couple of RPCs to support basic coin operations, Immutable Data and accounts data. With these in place, implementing the remaining RPC handlers should be somewhat quicker, as we have fewer unknowns and more concrete ideas about how to implement them.

We also now have a test framework in place which will let us build some more confidence in the code, although we’re expecting to be able to run the SCL test suite against an instance of a vault very soon (hopefully within a week or two). Since that test suite is more mature and fairly extensive, it’ll be a great way to check that the vault code is doing what is expected of it.

Integration of quic-p2p into Routing

Project plan

This week, we’ve been continuing the work on proper resending of messages on failures, which bore fruit today in the form of a PR. This is important for Reliable Message Delivery, as sometimes a node will try to pass a message to another node and fail (for example, because the other node disconnected in the meantime). If the node didn’t retry, the message would get lost. Since we dropped the routes and acks mechanism when introducing Reliable Message Delivery, we had no way to resend attempts when sending failed - this now changes with this PR.

Reliable Message Delivery

Project plan

We decided that the last issue still opened in RMD fits the scope of Secure Message Delivery better than RMD. It concerned signature gathering so we moved it to the other project. This completes the RMD project! :tada:

BLS cryptography

The BLS RFC now moved to the implementation phase and we have a brand new Project plan:

Project plan

As we have been highlighting over the last few weeks, we started by working on the feature that is least likely to be altered, so the initial task is already closed. This week we completed the follow-up task: a user of PARSEC can now start a BLS Distributed Key Generation with all PARSEC voters. When complete, each peer can retrieve the group threshold public key, and that peer’s secret key. This will be essential to keep secure message between Disjoint sections lightweight. With this scheme, a group can sign messages with a single signature.

Community contributions

Finally, we have been seeing some regular community PRs in quic-p2p which is great for us to see as it shows that the library is viewed as interesting and that people are spending time reading the code. PR #59, PR #56 and PR #57 are examples. Thanks go to @d1vyank! Plus a big thanks go to @mav who has once again carried out some fascinating research on chunk distribution within sections. Great work!

76 Likes

Firsttttttt!!now reading
Edit: congrats @Cgray !
Nice illustrations, I like them! I love to see you making progress and showing them with transparency to the community
Good job!

20 Likes

Second for the first time!

16 Likes

Very cool, very real progress shows! Delightful.

17 Likes

Great progress, go MAID Safe!

14 Likes

Amazing progress team! Will the RMD then get a happy green check mark? Cheers!

14 Likes

I’m gonna have to work more on how to present these Figma design files in the Forum etc… it’s not ideal. Will have a think about how to improve it.

If anything needs more explain’ let me know.

17 Likes

Wow, I really like how you implement Version History and ownership in the Perpetual web browser! This looks so good!

20 Likes

Indeed, would be cool if this is reflected on the safenetwork.tech website. Keep up the good work!

BTW, speaking of the safenetwork.tech website, I noticed that the page is often quite unresponsive. It takes quite long to load and when clicking on to top navigation items (e.g. roadmap) nothing happens. If I then wait for some seconds it jumps to the requested section. I think this is due to the gif animations of the background which take longer to load. I’ve created a screen-capture of the behavior: http://recordit.co/3TrmrI8NpT

In the video above I not only move the mouse cursor, but click several times on different navigation items without any effect.

9 Likes

Great progress team! @Cgray - congrats on promotion too!

Loving the versioning mock ups! This will be a killer feature for users and it will give real usability to the perpetual web! Excellent work!

21 Likes

It is really nice to be able to take a look at this from here. The error messages look very readable and simple to understand. The Url bar changes depending on versions are very clean looking too.

Again it allows us to sort of touch the future , which creates a mix of hope and excitment . Well done !

As a side note, I switched my pc for a raspberry pi, and I had to use my old regular pc to be able to watch the slides on figma.com, as these bring the pi to its knees.

11 Likes

Those Figma things don’t seem to work on my tablet. Tried tapping on them, and the full screen version. I can get two buttons to display ( ‘>’ and ‘Restart’) but no further.

6 Likes

Does this mean the RMD entry in the roadmap can be turned green?

19 Likes

I had that for a while (it takes an age to download though mind). I had some js blocker and things that prevented it working.

10 Likes

Beautiful and beautiful.

Thanks to everyone helping make this “impossible” network possible.

16 Likes

I waited several minutes for the progress bar to complete, and then some. Still no joy so gave up.

7 Likes

I suppose the Figma slides could be exported to some dumb jpg slideshow ?

7 Likes

It’s a shame, these Figma prototype/slideshows end up being buggy. I wonder if they are perhaps struggling with the numbers of simultaneous users?

Will have it try something else for the publication/sharing. It’s a shame as often you can end up sinking half a day in outputting static mocks, assembling, notating, etc. Whereas using a ‘active’ Figma file fits more directly into the workflow.

Ho hum!

15 Likes

Maybe they can be recorded as a screencast?

3 Likes

Yeah that was my first thought

6 Likes