SAFE Network Dev Update - May 16, 2019

Development Updates
1

Summary

Here are some of the main things to highlight this week:

UX and Website

Those of you who were quick off the mark will have noticed that we’ve just rolled out an update to the safenetwork.tech website.

It’s been bubbling away alongside other work for some months now, so it’s great to have it out in the wild. Think of it as an evolution of the site and brand, as well as bringing out some of the original intent of the site with a bit more focus.

We’ve sharpened up the typography, refined the palette, simplified the grid, and added a little more of a flourish to the homepage.

While much of the content remains the same, we’ve taken the time to overhaul the How It Works page, improved the on-boarding process when folks are ready to Get Involved, and of course can introduce the anticipated new Roadmap page.

The Roadmap page itself reflects much of what was revealed in last week’s update, but we hope it lays things out in a more digestible, accessible way. It’s a living document that will be updated continually as the project ramps up towards beta.

Hopefully, it will be helpful for you all to have a one-page overview of all the moving parts as they start falling into place; easy access to the relevant RFCs, discussions, and designs; as well as having the pleasure of seeing the green ‘completed’ ticks appearing!

These site launches are a big undertaking, with lots of work from across multiple teams; so thank you all who contributed, in particular @shankar, @dugcampbell, @JimCollinson, @Karamu, @SarahPentland, and @nadia. That Oxford Comma was just for you Nadia.

As always we appreciate your bug and typo reports, drop them in as a GitHub Issue if you sniff one out. If you don’t use GitHub then feel free to add a comment to this post and we’ll pick it up.

Marketing

Required listening for all this week as @ustulation was invited back onto @fergish’s excellent SAFE Crossroads podcast, this time for a conversation about why we’ve made the move to incorporate QUIC-P2P within the Network. We’ve also been focusing heavily on the SAFE Network Fundamentals this week on Twitter - just search for #SAFEFundamentals. We also put out a couple of Medium posts on the Fundamentals - you’ll find Part 1 here and Part 2 here. Just to recap: we’re republishing (under the medium-repost tag) so that those who don’t like Medium don’t need to visit it. However, for everyone else - please do continue to click through and support with claps on Medium as this helps the distribution of our messages!

Quick comment on exchanges: Bitker is currently waiving fees on MAID trading so worth a look. And a final reminder about the upcoming meetups: @ustulation and @nbaksalyar speaking at SAFE Network: Brighton on Thurs 23rd May, @nbaksalyar sharing the stage with Ruben Verborgh of Solid fame at the first SOLID London meetup on Wed 22nd May. And last but by no means least, @sotros25 is doing yet more sterling work, running the next edition of the SAFE Network: Chicago meetup with the Chicago Blockchain Project on Thursday, May 30th. Thanks in advance for all the work throughout the community helping to publicise each of these events!

SAFE API & Apps

Several of us were focusing on reviewing several aspects of the upcoming new data types, including the AppendOnlyData data type needed to have a perpetual web! This would have an impact on some of our implementations on the front-end side, especially around the new APIs required to support it, and how some of our applications and data representation would need to be adapted to this new data type and the whole concept of perpetuity of data.

We therefore started a new forum topic where we highlight, as an overview, what is it about these new data types that makes them different and how they may impact current SAFE APIs and applications. Please join us in this discussion to clarify doubts and/or expand on any of the aspects touched upon.

This type of research tasks will for sure continue in the next few days and weeks, along with some effort to both update some of the already published RFCs, or to create new ones like the one we are working on for a SAFE data CLI as briefly mentioned before in these dev updates.

The browser has finally gotten some cleaner/more flexible tab and window data structures. Which have solved a raft of wee browser niggles as well as opening the door for some more standard browser features later (e.g. tab reorganisation/dragging). We’ve also gotten one major step closer to a clean and shiny codebase, with all browser code linted and CI tests for the same. Otherwise we’ve been investigating if automated website tests might be possible with the SAFE Browser and TestCafe (which, while looking much more positive, is still proving to be an elusive dream). Auto-updating browsers are on the way too! Initial drafts are working well on macOS, but need some further digging for Windows and Linux.

We published a patch release of the SAFE Authenticator mobile application earlier this week. It includes a fix to prevent the Authenticator app from crashing when sending the response back to the app and the app url is not found. Also, now the authenticator identifies an invalid container name in authentication and container requests. And it provides information whenever an app requests access to another app’s container. For more information have a look at the changelog.

SAFE Client Libs

Over the past week, we have been focussing on the RFCs for a major change in the network data types. The RFCs have been published today: RFC 54 and RFC 55. With that, we’ll finally adhere to the network fundamental of data perpetuity. With all of the community feedback taken into account, we decided to address it without breaking the compatibility with our existing APIs, but by extending them instead.

There are several major changes: first, we are looking to introduce a notion of unpublished and published data. Unpublished data is any kind of data that can be read only by the owner (or multiple owners) or by a private group which has read permissions. Published data is stored in perpetuity and it can never be deleted or changed–even by the owners. Thus, with this approach, the well-familiar MutableData will be available only for private use, as an unpublished data type. Second, we’re looking to introduce a new data type, AppendOnlyData. While it’s also a key-value store, it will have different semantics from MutableData: instead of changing the existing keys, new entries need to be inserted to change the data. But even with these changes, all past entries will be always available.

This planned change is big, but it opens up a lot of possibilities in the network. For example, you can browse all past versions of a website–just like on the Internet Archive, but for the entire Web. It also addresses the problem of link rot, which is also very important for Linked Data and the true Semantic Web.

The RFCs are not final and we’ll be looking for your comments and opinions. We also plan to start implementing some proof-of-concepts in parallel, so these are exciting times for the Client Libs and the SAFE Network in general!

Work on the PoC for RDF support in SAFE Client Libs also continues in parallel. We have implemented Rust bindings for the Redland API to parse RDF documents into RDF graph objects and vice versa. Initial implementations for SPARQL querying on RDF graphs are showing promising results. We are currently implementing a proper approach for graceful error-handling. We have also integrated some RDF functionality into SAFE Client Libs that enables storing, retrieving and updating RDF graphs using Mutable Data. We will soon plan integrations with the upcoming data types too!

We’d like to sign off by welcoming @marcin who’s now back in action after a well-deserved vacation!

Routing

Implementation of initial aspects of Fleming and work on design continues.

In Routing, we created a few PRs for some code cleanup and further preparations for the implementation of Node Ageing. These improve our tests and our handling of candidates attempting to pass the resource proof - as the candidate’s state should be shared between the members of the section, changes to it should be a result of PARSEC consensus, and this is what PRs 1628 and 1629 aim to achieve.

There is also ongoing work on a bug in PARSEC uncovered by Routing tests some time ago. This bug proved to be very elusive and difficult to identify precisely, but we believe we have finally found the cause and are working on fixing it.

On the design front, we had some internal discussions on RFCs about Safecoin and Message Relay that are in the making. Some issues have been raised and we are fixing them. The Secure Message Relay RFC has been published today, and the Safecoin RFC will be following soon.

We also continue our efforts on designing Node Ageing flows in routing_model. There are still some missing pieces in the flows that we are continuously working to complete - a list of these can be found in Issue #7.

QUIC P2P

There are more things to work on in quic-p2p, but now that we have released the first public crate version, we’ll be slowing down the development for the time being, putting our resources to more critical things.

Viv

Next, we have a piece of sad news that we’d like to bring to your attention. @Viv, our current CTO, has decided to step down from his role at MaidSafe. As everyone who’s followed the project knows, Viv’s contribution to the company over the past 8 years has been exceptional, having worked his way up from a junior development position to lead the development teams. He has been an invaluable member of the team and he will be missed.

There has been internal discussion around this topic for a while now and given his commitment and belief in the project, Viv has not taken the decision to leave lightly. However, after lengthy conversation, he’s of the opinion that stepping down will enable the executive team to become more streamlined in design and project decisions moving forwards.

Viv will continue to work for MaidSafe until the end of July to ensure a smooth transition to @dirvine who will initially assume the role as acting CTO. We don’t plan on hiring a replacement straight away and will take some time to consider the next steps, which could include promoting from within the company.

It’s been a lengthy journey that we’ve been on and it’s always sad when someone leaves the team to return to the community. But this project continues to be built on the hard collective efforts of many and we’re always so grateful for everyone that has put their shoulder to the wheel and put in a hard shift, particularly when it comes over the course of many years - and that very much applies to @Viv.

Project Management

Finally, with @dirvine taking over as acting CTO, we’ve reorganised slightly in order to facilitate project management taking a much more active role in delivery.

We haven’t hired new people, and we aren’t taking dev time away, but we have worked on clarifying some project-related responsibilities for some staff to help us ensure we deliver our goal. The Project Manager has a support team of two (ensuring front and backend have dedicated focus), three staff have project coordinator roles to ensure tasks in the plan are correct, have reasonable estimates and the priorities are correct.

We also have a ‘strikeforce’ team of four (two for frontend and two for backend) who will use their knowledge and experience to work with the teams to continue to improve the delivery quality in addition to providing support to the teams when tasks are hitting issues or looking to overrun.

The RFC process will come under far greater focus as part of the design process. For transparency, we now have a private Pre-RFC repo where these discussions take place internally before we make them public. This ensures that the team can be extremely focused on ensuring that the topic has been interrogated deeply before the draft RFC is made public - at which stage we all, as a community, can debate things thoroughly so that we collectively end up with the most comprehensive solutions.

78 Likes
3

Fiiiiiiiiiiiiiiiiirst! Now to read :slight_smile:

Another great update! I think we’re spoiled with these updates frankly 
 the best in crypto, hands down. It’s easy to take quality communication like this for granted, but the depth, clarity and obvious integrity are appreciated.

The podcast with @ustulation was excellent, so thanks to everyone who made that possible.

And very sorry to see @Viv is moving on, but I wish him all the best in future endeavours, and a sincere thanks for his time with the project.

30 Likes
4

Second
,


7 Likes
5

Another great update, so many good things happpening. Really sorry to see you go Viv but I’m sure you’ve left the project in good stead to deliver all goals
All the best in whatever the future holds for you.

19 Likes
6

We all are, he has given everything and asked for little in return, a true maidsafe stalwart. As I have previously said it is like watching a machine work. All in all though, time moves on, things change etc. but Viv is as committed to the success of SAFE as anyone and that says an awful lot. A shock for many of us, but understandable as well, that pressure is immense and he took it on his shoulders for a very long time. The company will do whatever it can for him as you would expect. He knows the internals of the current routing more detail than anyone and his experience still helps us in transitioning that to the release code with the RFCs we have posted and more.

49 Likes
7

Love it!

Well done!!!

16 Likes
8

Guys I think you made a great job with the website. Way cleaner, smarter, easier and more understandable by new users. At first seems that not much has changed in style, but if you look closer you’ll find out that space is used well and words are chosen wisely to explain easily something technically hard. Nice!

21 Likes
9

:broken_heart:

Thank you for all you hard work.
Good luck for the future @Viv

26 Likes
10

Great to read about the new data types and versioning - it sounds like they bring a benefits to both public and private data, which will be well received.

It is really sad to hear about @Viv moving on though. Such a senior figure leaving close to the release of Fleming seems a little odd, but I am sure both parties have their reasons. I just hope that the transition is smooth and progress continues to be made.

25 Likes
11

Great to see a detailed roadmap to the future with all the milestones and explanation of each! Makes a lot more sense what still needs to be done! For the developers it is an extra challenge to get the green marks :white_check_mark:! Keep on building the free future!

15 Likes
12

great update, lots of progress this week! sad for sure to hear about Viv leaving.
But yeah, I imagine 8 years at MaidSafe is like 16 years at another company.
All the best to you @Viv and keep rolling MaidSafe :wink:

19 Likes
13

Excellent update - apart from that Oxford Comma which, as all right-thinking people know, is an abomination. Another big thumbs up for the redesigned site. Much cleaner on the desktop (haven’t checked mob yet) and I love the animated graphics. Like everyone else, very sorry to see @Viv go - all the best with life’s next chapter.

19 Likes
14

I’m sooooooo with you on that! :wink:

10 Likes
15

And you can add my name to that list as well
 :wink:

8 Likes
16

So far that’s @JPL, @Nadia, and @dugcampbell.

Anyone else?

11 Likes
17

I fear we may be about to witness a transatlantic schism here 


6 Likes
18

I don’t have a Github acct but work as a professional literary editor, so thought I would share some suggestions:

“Freedom of expression, control of personal data, private and secure communications; and a whole new economy.”

Semicolons should only be used between independent clauses, and the text following the semicolon above isn’t independent. Two solutions: use a comma, or (my suggestion) use an em dash. The latter would look like this:

“Freedom of expression, control of personal data, private and secure communications—and a whole new economy.”

“See How it Works”

“It” is a pronoun and should be capitalized in title case. (The hyperlink at the bottom of main page with the same wording correctly capitalizes “It.”)

“Everyone should be able to collaborate freely and share ideas; free and unrestricted access to information for everyone around the world.”

Incomplete sentence. I suggest deleting the semicolon, adding a comma instead, and following the comma with “enabling”:

“Everyone should be able to collaborate freely and share ideas, enabling free and unrestricted access to information for everyone around the world.”

“David Irvine — Founder”

OK tiny correction here: Delete the space between the em dash and “Founder”:

“David Irvine —Founder”

“When we’re given the tools to communicate privately and securely we gain the freedom to collaborate and innovate all as one.”

I suggest adding a comma after “securely.”

“Browse and access anonymously”

I suggest reversing the order of “browse” and “access” for sequential accuracy. (First you access, then you browse.) Unless you want the alliteration of “access anonymously.”

“Get free unrestricted access from any connected device around the world.”

Optional, and for stylistic reasons you may want to ignore this suggestion, but I suggest adding a comma after “free.”

“You’ll pay a minimal, one-time, upload charge but you’ll get permanent secure backups with no more monthly fees.”

Delete comma after “one-time” and add a comma after “charge.” Reasons are slightly more complicated but DM me if you want clarification—no problem.

“Instead, you’ll use Safecoin, a crypto currency that is integrated into and distributed entirely by the network.”

I suggest making “crypto currency” one word as this is now the standard.

“Users are compensated for providing their resources and can use this to pay developers to use their apps in a virtuous and self sustaining cycle.”

Three problems: Triple use of “use” or derivative, second “use” technically refers to “their resources” as opposed to Safecoin, and “self sustaining” should be hyphenated. Suggestion:

“Users are compensated with Safecoin for providing their resources, and can then exchange their Safecoin to pay developers to use their apps in a virtuous and self-sustaining cycle.”

“By running a Vault, you can offer the network your unused resources—like hard drive space and bandwidth—and be automatically be rewarded with Safecoin.”

Delete one of two “be”s.

“As there is no human interaction, the Network autonomously creates a perfect market. Prices are set continually dependent on available resources and user demand.”

Delete one of the two spaces between sentences to standardize with house style.

General note: When using the shortened “the network,” sometimes “network” is capitalized, sometimes it isn’t. I would standardize. My suggestion would be to use the capitalized version. I also suggest reviewing all pages on website, searching for “the network,” and replacing all lower-case instances (or vice versa, if you choose to standardize with lower case). Not a huge deal, but hardening house style make sense IMO, and avoids any potential confusion down the road.

Please feel free to DM me if there is confusion or uncertainty about any of these suggestions—I’d be happy to provide rationale or citations for why I suggested them. Otherwise the website looks amazing—thanks to everyone who helped make this possible!

28 Likes
19

Blimey what an update. :sweat_smile: Amazing work going on, so much that it will take a while to dig into and digest but I want to say two things right away:

I went straight to the new roadmap and think it is fantastic. Very clear, streamlined and informative. It also seems to show Fleming and beta are more in reach than last week’s task boards - I’m not sure why, but it does. I think we should make more of it somehow. Maybe link to it in every update in a mini section that itemises any changes (items changing status, new items etc). Or a ‘roadmap updated’ tweeted whenever something is checked off the list, with a link to the roadmap. How about including a countdown of items left to do until beta too? This could feature in the tweet, in the weekly Dev update and on the website. Anyway, great work, and keeping it updated is a great opportunity to make more of it.

Secondly but much more important I’m really sorry to hear that @viv is stepping down. I had the pleasure and privilege of a few calls with Viv, and have to say he’s a really great personality as well as having been an engine at the heart of this development - for eight years! I’ll miss you Viv but hope you will still be around in some way or other, and wish you all the best with what you choose to do next. If you are passing near Gloucester any time I hope you’ll let me buy you a beer. :beer:

34 Likes
20

My love for em-dashes will never die.

7 Likes
21

great feedback. Many many thanks

6 Likes