SAFE Network Dev Update - July 11, 2019

Summary

Here are some of the main things to highlight this week:

  • @jimcollinson created a YouTube screencast to give a sneak peek at what the UX team has been working on during this design sprint: Editing and Publishing sites on the SAFE Network.
  • The milestone for having files containers supported by the CLI has now been fully implemented. See the CLI User Guide for detailed information about how to use the new files put, files sync and cat commands.
  • We released SAFE Browser v0.14.1 to fix a few minor bugs.

Marketing

A short but sweet update from the marketing team this week. We put out our fortnightly tweetstorm today exploring some of the top stories that weā€™ve read this week. Weā€™d love the community to be part of this so please send any news our way and weā€™ll include it!

Work hasnā€™t stopped on the exploring the Frontend milestones either. While last week we published a spotlight on the Perpetual Web (and @JimCollison has some more amazing updates to share below), next up weā€™ll be exploring Take Control of Your Data: this takes a look at usersā€™ private data and really explores how our data today lives in the hands of others - and what this means.

This isnā€™t just blue-sky thinking or an impossible idea. No, weā€™re well underway with the work to bring control back to you, the user, and make it a reality. Our post will be live tomorrow so weā€™ll update you with a link when itā€™s out :wink:

The next two themes weā€™ll explore are Private Communications and The New Digital Economy, so we hope you enjoy these as much as we have enjoyed writing them.

And finally, a reminder about the SAFE Network: Glasgow meetup thatā€™s taking place next Monday (15th July). Our very own @Jean-Philippe will be chatting about what heā€™s working on, together with a few folk from Ayr HQ. Getting to meet the community face to face is an incredible luxury so we are excited to see you all there!

Project Management overview

Its been a few weeks, so time for a wee project overview update :wink:

In the interests of continual improvement weā€™ve tweaked the gantt chart so you can see a today line, a high level progress bar for each project and a nice big green tick for completed projects (so satisfying!!)

You can see progress compared to the last update posted as well as having added in the Frontend milestones :tada:

As you can appreciate this is still at a high level, but it should continue to give you a visual on whatā€™s going on. Enjoy :smile:

User Experience

In this design sprint for the Perpetual Web Browser, weā€™ve turned our focus from Laurelā€™s ā€˜Read-Onlyā€™ Persona, to Tony and his needs as a content contributor.

How can we make it fast and straightforward for Tony to create, edit, and publish a site on the SAFE Network, taking advantage of the serverless infrastructure, and the versioned perpetual data?

After last weekā€™s challenges with presenting the wireframes, @jimcollinson whipped up a screencast to walk you through it:

Feel free to ask your questions, or make observations in this dedicated forum thread.

SAFE CLI

Files Command project plan
NRS project plan

The milestone for having files containers supported by the CLI has now been fully implemented. During the last week we were able to take it through our internal testing process to fix some issues and stabilise it, so we are very proud to share this with you today. Please do give it a try! As a quick reminder, with this milestone finished, we are now able to perform the following operations from the SAFE CLI (still using an internal mock rather than using SCL):

  • Create (published) FilesContainers by uploading files and entire folders with the $ safe files put command
  • Retrieving and rendering the content of FilesContainers and files (published ImmutableData) using XOR-URLs with the $ safe cat command
  • Updating an already created FilesContainer by sync-ing it up with local changes made to files and folders with the $ safe files sync command

Why would we really like you all to try this out? Well, this should show you how easy it will be to upload data onto the SAFE Network and keep it in sync with local changes. It has never been so easy until right now, you should definitely check it out!

The CLI User Guide has been updated with detailed information about how to use these new commands (look for the Files and Cat sections in the ToC). We strongly suggest to read it thoroughly before trying them out. If you have already pulled the CLI code and built it, remember to pull the latest code from the master branch to get this functionality.

$ safe cat safe://hbyzg58n3kk8d7jiopppdtdry3ckiwp1z7mq3g8oiao5mrrm55twthgj5w/lets/safe/the/world.txt
       _,--',   _._.--._____
.--.--';_'-.', ";_      _.,-'
.'--'.  _.'    {`'-;_ .-.>.'
     '-:_ SAFE Network '=.
       ) >     {_/,     /~)
       |/               `^ .'

Weā€™ve also been working on a new milestone and project plan which has been agreed and set up this week. This new milestone is about NRS (Name Resolution System) and the integration we started with SCL API. The main goals can be summarised as follows:

  • Implementation of a first PoC for the $ safe nrs create, $ safe nrs add and $ safe nrs remove commands (note that add and remove work on subnames)
  • Support for being able to $ safe cat NRS-URLs, with subnames and paths
  • Have all existing commands and APIs work with SCL API, which would allows us to also run CLI tests with SCLā€™s ā€œmock-networkā€ feature

Note that in the same way as the previous milestone for FilesContainers, we will be using a pseudo-RDF for the NRS Containersā€™ data representation for now. This is just to allow us to focus on the UX and UI of these commands, and we will be able to adapt the internal implementation to use RDF in the medium term without affecting the APIs or the CLI user experience.

SAFE Browser

Today we released another version of the SAFE Browser. You can download the latest release, v0.14.1, from here.

This small release was created to fix a bug we picked up internally in the last few days, where logs were not being populated on Windows platforms. We also included a small fix for a tabbing issue.

Mock interface

Project plan

The mock interface in SAFE Client Libs has been through a bit of maintenance this week. A number of internal improvements and semantic fixes were made.

The team has been working very closely with the Vault team which helped identify and fix a few issues. For example, previously the different variants of the same data type used the same address namespace. This resulted in the potential for address collisions that needed to be handled in the Mock Vault. After some discussion, we decided to use separate namespaces for each of the data type variants.

We also added some fixes to Append Only Data RPCs to fetch values for specific keys and to properly check the required indexes when appending to the permissions and owner fields. You can learn more about all of the changes in the project plan linked above. These changes are necessary for a hassle-free integration in the future with the real network.

A couple of feature additions were also done during the week, where we worked on the LoginPacket RPC handling. Formerly known as the AccountPacket, it is being revamped to fit the changes defined in the RFC, including the name change. This will now enable us to move on to the safe_authenticator module which makes use of the now called LoginPacket for logging in clients via self authentication.

Next in line, we will start planning quic-p2p integration. Mock quic-p2p will be replacing the current Mock Routing which Client Libs makes use of, to simulate the real network while handling all the RPCs in the Mock Vault. This will be a separate milestone for which we will be defining tasks soon.

Vaults

Project plan

Work on the Phase 1 of Vault has been coming along nicely. We completed a major feature this week with the implementation of the mock Safecoin (as defined in the RFC): now we support operations for balance creation and coin transfers.

Thereā€™s a trick, though: in the first iteration we allow all operations to pass through even if a user doesnā€™t have a sufficient balance, thatā€™s including the balance creation. So yes, it is possible to create an unlimited Safecoin balance (isnā€™t that nice?).

We also completed Immutable Data (both published and unpublished) and implemented operations for the Login Packet data type (renamed from ā€œaccountsā€ to cause less confusion).

We are now moving on to Mutable and Append-Only data. Once thatā€™s done, weā€™ll be ready to move onto integration testing with the SAFE Client Libs.

Integration of quic-p2p into Routing

Project plan

The work on resending messages on failures is almost complete. We merged 2 PRs on that front: one that implements the actual resending logic (#1705) and one that fixes some performance issues (#1709). At this point, the Routing part is done.

There is still work required on the quic-p2p side to update the API to include a feature that Routing now expects. While quic-p2p was functionally complete, we needed an enhancement to quic-p2p to complete the integration: quic-p2p will have to emit a confirmation that a sent message will no longer fail when it knows that this is the case.

With this almost out of the way, our efforts are shifting towards implementing Secure Message Delivery.

BLS cryptography

Project plan

On the BLS front, we continue implementing our DKG (Distributed Key Generation). The DKG is a way for the peers to collectively generate their secret and public keys.

We now have two new PRs merged. One enables us to record scenarios involving DKG and replay them in tests - this is important for our tests and benchmarks suite. The other one enables running a DKG with only a part of the section participating. This feature will be required for handling splits: The two halves of a section will need to run separate instances of DKG in order to generate keys for the new sections that will be born in the split.

Next up, we are looking at allowing peers that are not yet full members of a section to participate in the DKG. This will be essential anytime a new peer is joining. Theyā€™ll be able to obtain their own secret key and provide a share of the section signatures.

At this point weā€™re looking to split this big item into more manageable sub-tasks now that all the foundational work is complete and we have much better visibility.

81 Likes

first !!!
First time ever!!!

34 Likes

Not First! :champagne: ā€¦ but not without having to jump through hoops and provide something that ā€˜looksā€™ like a sentence.

This game is getting tricker! :wink:

Solid on SAFE mini update:

Today I checked in code which allows a Solid app to work on SAFE (mock) just by using a SAFE compatible version of the solid-auth-client library (on branch: add-safe-compatibility)

The Solid app Iā€™m using is solid-filemanager which lets you browser your storage, and allows you to create, edit and delete files.

It is just a proof of concept, but it works.

I also have a modified version of solid-web-id-manager so that when you create a WebID it creates a storage location and refers to this in your WebID profile. That makes some SAFE storage available to any Solid app that looks it up based on the WebID you have selected in the SAFE Browser drop-down.

Iā€™ll put all these on alpha2 in due course but want to test with another app or two first.

Maidsafe Update

Every week reason to celebrate. Thanks everyone for maintaining such momentum. It is very exciting to see how much is being pulled together in such a short time. Meanwhile

[cough] SAFE Drive [cough] :stuck_out_tongue_winking_eye:

And whatā€™s thisā€¦ half complete!!!
Screenshot%20moble%20browser%20plan-edited

Q: Also, what is ā€œSafe Network appā€?

A: See here and scroll down to ā€œWhat Next?ā€ (h/t @david-beinn)

49 Likes

What a great update!

Updated project overviewšŸŒž

20 Likes

Probably the first who has actually read it through - said he in a rather self-satisfied passive aggressive toneā€¦
Seriously folks - another great update. Just about to go play with the SAFE CLI - suddenly Iā€™m not hungry and dinner can wait.
Talking to some folks today who agreed with me that SAFE is going to be a game changer and are looking to at the very least store their sensitive user data in SAFE vaults.
Thanks to all involved In Ayr Chennai and remotely.

EDIT: and thanks also to @happybeing for his updates on SOLID. My discussions today included SOLID as well as SAFE. Serious interest from some very interesting players.

35 Likes

Fifth for the first timeā€¦

25 Likes

:star_struck:. Amazing. Just wonderful.

18 Likes

Itā€™s envisioned to be like an access point of the SAFE Network which will include on-boarding, authentication, and ready access to the core suite of apps such as Wallet, Vault, Data Manager, Browser etc. ( starts of as a download manager/launcher).

The cat is out of the bag finally! brainchild of @ravinderjangra :bowing_man:.

32 Likes

We really need to double the size of the UX team to keep up with the bloke!

23 Likes

10th! :raised_hands: Now to read ā€¦

Another great update guys and gals! Thanks as always.

9 Likes

Technically you are 9th.

As always, great update.

8 Likes

Then Iā€™m :elf:
Again a fine update!
@JimCollinson: pay attention for a clear pronunciation difference between the 1st and the 3d persona in another possible future video (Laurel and Yanni) :wink:

13 Likes

I think this post has most information about it (concept drawings.) I did wonder if it had kind of passed folk by, due to the lack of people absolutely losing their minds over it!

Thanks again for all the great work everyone!

14 Likes

rather impressed with the progress visible in the Gantt chart. Great way to illustrate. Please keep those charts coming (like every last Thursday of the month maybe?)
:+1::+1::+1::+1:

18 Likes

Lot of hard and great work. Almost hard to follow and I only have to read about it :stuck_out_tongue:.

Will have a look at the video right now. Thanks!

15 Likes

Love the project management overview with those satisfying green ticks like my character has finally gained enough experience to acquire new spell in a RPG! I think everyone on these forums loves it to. The question now is how can we pipeline the general crypto community into that jpg. I think too many people think this project stubbed its toe and just stopped sometime since 2015. We need to do better at showing potential investors there is serious momentum building. Iā€™ll at least tweet that puppy out to do my part!

16 Likes

Really great update again! So much progress is being made and it feels like the product plan is providing a great platform for the devs to build on. Really positive to see!

Mobile browser?! :exploding_head:

19 Likes

Great to see the safe n solid mini update! Keep up the great work, Mark!

20 Likes

This is a really great update. The progress is accelerating. Thanks so much to all of the team for all of their hard work!

9 Likes

I mostly browse the forum on mobile and everytime I encountered a safe URL I wished I had a mobile safe browser! :sweat_smile: Very cool to see that this is tackled now! :slight_smile:

14 Likes