SAFE Network Dev Update - March 7, 2019

Summary

Here are some of the main things to highlight this week:

Marketing

Over the past few months, a number of you have been asking about the plans for SAFE DevCon. After a great deal of conversation around the topic - and various calls with event organisers in different locations (Berlin turned out to be favourite), we’ve now made a decision to take an alternative approach this year. As a result, we’ll be organising a SAFE Hackathon in London (probably in mid-September).

There are two key reasons behind the decision. First, supporting the growth of the community - specifically around developer engagement - is a key area for us. There’s no doubt that a conference with talks would help to bring the community together. However, we’d really love to see more DApps being developed out in the wild. Talks can be given in a range of places and times - from meetups to conferences to straight-to-YouTube - but in order to attract more SAFE Developers, we need to continually work to support more independent SAFE DApp developers building out their ideas. With that said, the second key factor here is timing for the event. Running a May event was the original plan. However, we need to be certain that Appendable Data has been released before the event - hence the target date of September. That’s not because we believe it will take that long, but we want to completely avoid any issues with people being away on Summer holidays, etc.

Why is the release of Appendable Data key? Simply because any DApps developed before that time will require updating afterwards. So the choice is between running an event where most DApps need ‘fixed’ afterwards - or putting something where the DApps created can survive (and, from a marketing perspective, help to show off the Network).

We appreciate this might be slightly disappointing to those of you who were looking forward to the next community get-together. Hopefully though the reasons will make sense - and, of course, there’s nothing to stop any of us holding regular SAFE meetups with talks etc during this time. Speaking of which, quick reminder that next Thursday, March 14, @dugcampbell and @bart will be speaking at the SAFE Network: Brighton meetup for those that are in the area!

One final thing: @dgeddes and @frabrunelle have been working on putting far more detail into the Discourse Discobot greeting that each new Forum signup receives. Now, each newcomer will have a more defined path towards achieving their trust level status in order to access the Alpha 2 Network. We’re also working on another way to streamline things on this front - watch this space!

SAFE API & Apps

The SAFE Authenticator Mobile UI/UX enhancement milestone has come to an end and we are glad to announce that v0.1.1 of the application was released yesterday. See this post for more information. You can download the latest APK from this GitHub release. Have a look at the changelog for the list of feature improvements.

This week we have been checking off the last few tasks in the safe_app_java milestone. With the project’s test suite considerably improved, we will set it down to simmer as we move our focus to other design aspects.

The browser has seen more code quality enhancements, with further refinements to linting in JS and TypeScript, ready for us to get this all squashed so we can implement stricter checks on the code as part of the automated CI process. We’ve also been wrestling Travis’ Windows implementation with the aim of reinstating automated windows builds for the next release.

There is and has been an ongoing effort to generate more technical documentation for some of our front-end projects, especially those that can help new developers know where to start from when contributing to these packages/libraries. We recently finished a first draft with enhancements on the safe-app-nodejs package which is currently under review. Suggestions and help from the community to review this PR would be very welcome and appreciated.

We’ve been following closely and learning from all the contributions made on the forum by @oetyng, with a lot of ideas and experiments he’s been working on, which may be applicable to many projects or to solve some challenges developers will face when storing big data on the Network. Therefore we wanted to not only highlight it but also show our humble appreciation to his valuable contributions to the forum. For your reference: Infinitely appending data, DataStore over appendable data.

SAFE Client Libs

This week we found success in implementing a proof-of-concept for RDF on the SAFE Network, developed in Rust. We were able to arrive at a coarse, yet working demo that currently lacks testing, error handling, etc. RDF triples were successfully serialised to key/value pairs and stored on disk. We were also able to retrieve them, deserialise them into triples, and handle them as an abstract RDF model (which allows to read, add, edit, and query the data). Next steps would be to enable storing RDF in the network by stashing it in a Mutable Data and experiment running SPARQL queries on it. Because the key/value representation of triples we use perfectly translates into Mutable Data, this task should be less complicated. An exciting time awaits! RDF is coming!

On the documentation front, we continued publishing new pages to the Client Libs wiki including “Mutable Data”, “Client trait and implementations”, and “GET and PUT reference”, and wrote several soon-to-be published brand-new docs.

In addition to this, the first draft of the RDF RFC has been completed and the team have had a chance to look it over internally. After addressing a few comments, the RFC should be ready to publish publicly very soon – we can’t wait for your opinions and suggestions!

Routing

Full steam ahead in the Routing team!

Happy that we had built a sufficient understanding of the Sybil implications of different designs, we decided to conclude the Sybil investigations for the moment. We released a first post on Sybil resilience to set the scene. This post already prompted a few interesting conversations. We are very grateful to the community for their invaluable insights :heart:. A follow-up post will highlight the key insights we gained during the last few weeks, so if this week’s post wets your appetite, stay tuned :smile:. We will also be very interested in hearing what you all have to say about these.

So with Sybil sorted for now, our Fleming discussions took a slight turn. If you remember, the initial aim of these discussions was to get a holistic look at a number of big questions to ensure that the way we put together Fleming won’t hinder future milestones. We have now had the time to discuss most of these in some depth and gained the clarity we were seeking. With this, we are focusing the discussion on the scope of Fleming itself and what practical decisions we will take for that milestone, often leaving a range of possibilities that testnets should help narrow down.

We continued working on the PARSEC whitepaper. We added a section on the common coin, making it fully asynchronous and wrote the associated proofs. Since we were touching up the whitepaper, we took the opportunity to conduct a thorough review and address some open issues with the initial wording. For instance, we addressed some of the points raised by the community and some other imprecisions we noticed. We are still doing a few touch ups and reviewing the entire paper thoroughly to make sure the changes are not introducing any inconsistencies. PARSEC v2.0 should be coming soon and we are very proud of it :smile:

On the PARSEC side, improvements keep happening. We fixed a hole in our concrete coin protocol by adding another type of sync event: Requesting. The issue was that it would be possible for a malicious node to pretend they had received many Request events. This could lead them to pretend they were more responsive than other nodes and decide the value of the coin flip. With the new Requesting event, inventing a Request is immediately detectable. There is a negative performance impact (roughly 30% degradation), but it is necessary in the current state of affairs.

Speaking of performance, we merged a change that makes PARSEC compute multiple stable blocks at the same time if there are multiple interesting payloads in a single meta-election. This change results in a 70% improvement in benchmarks that focus on reaching consensus on many events, but also opens the door for an even better improvement that we teased last week: the next step will be to bundle multiple observations into single gossip events so that the artificial limit on the number of transactions per second that we have at the moment may be lifted. Work on that front is still ongoing.

We merged a couple of improvements to the benchmarks themselves. One created some benchmarks with a larger number of events so we could profile the improvement from the changes we have in mind. Another change worked around a little bug in criterion that had our benchmarks report incorrect results for a couple of days.

Crust

We continue investigating into our dynamic connection manager where we were planning to have active (a definite session exists between the peers) and passive (cached connection info, but no active session) connections so as not to overwhelm systems with 1000s, or more, connections (as might be necessitated by Routing section sizes and connectivity story) but we are also keeping a keen eye on QUIC.

It seems QUIC has become a lot more mature with great stuff in its design spec that might solve many of our problems.

Some notable mentions could be (a) it builds on UDP which can give us the ability to keep socket-descriptor exhaustion under strict check by multiplexing many QUIC connections on single underlying UDP sockets (b) reliable and congestion controlled ( c) Address migrations and session suspensions and resumptions (d) in-built TLS handshake so that we don’t do our own secure-serialisation (e) multistreaming and stream prioritisation over the same single connection to a peer, etc.

The Rust crate we are looking into is quinn. It doesn’t have for example stream prioritisation implemented yet so there might be some features in the QUIC spec not yet in quinn, which we need, but they might be pretty easy to PR to quinn. This investigation along with dynamic connections is keeping us busy in Crust just now.

89 Likes

First! ! Now I’m gonna read it with peace in my mind :smile:

This is great stuff.

27 Likes

Second!

Haven’t done that in a while :stuck_out_tongue:

Hoping for Fleming soon!

Still excited to dig into the mobile toolkit soon :joy: haven’t had time yet but that was a huge release that I haven’t fully enjoyed or utilised yet

23 Likes

Wooot! Looking forward to it!

The mobile release updates look so great! Great job to the Chennai office!

Huge props to @oetyng! Another extremely valuable member of the community.

I’m so anxious to see PARSEC go fully async :grimacing:

Exciting stuff happening in Client Libs and Crust. Like a kid in a candy shop with updates like these.

Also, the Hackathon is a great idea! Question though, when Appendable Data is implemented will it be usable on the Alpha 2 Network?? Should dapp devs continue building with MD and then switch to AD in SAFE-Maxwell or wait to continue building in AD that may be on Alpha 2??

25 Likes

damn so I have to wait all the way to sept for the next major emotion fueled pump? :stuck_out_tongue: I do actually agree with the way you are going about this. For people to want to make Dapps, at least a lot of them, I think need to have some assurance they are making something that can be profitable in the end. I am sure some would make simple Dapps to help test out the network, but real enterprise sized things just can’t be developed then redeveloped for zero cost. It is good that you are working on firming up dates when people can expect it to be worth taking this plunge. I am sure some people are like “ya I will make a Dapp when the safe network beta starts.” Hopefully you can give these people some assurance their work is not wasted before that. I would like to see beta start with a good number of Dapps that are more then “hello world” apps. That will be an important point for marketing and if you are like ya we got the chicken but no eggs yet, people will be much more skeptical.

13 Likes

Nice to see all the medium term stuff coming into focus, and really exciting to hear about the progress on RDF!
Anything about Sybil attacks and maintaining decentralisation (they seem so closely linked) sends me spinning off on all kinds of confused tangents that are more social and economic than technological, but it’s fascinating to see all the continuing work in this area, and also great to have such a clear explanation of it.

11 Likes

Since all the major tech is done, and nothing more needs to be invented, each new dev post feels like the thread to me. All that’s left is for the network to be sewn together, a few pieces at a time! Great progress team!

13 Likes

knowing little of what that is it sounds really cool … is that a combo of break it; make it; and bug hunt ?

Define: hackathon

/ˈhakəθɒn/

an event, typically lasting several days, in which a large number of people meet to engage in collaborative computer programming.

A hackathon is a design sprint-like event in which computer programmers and others involved in software development, including graphic designers, interface designers, project managers, and others, often including domain experts, collaborate intensively on software projects.

That is a good idea… intensive introduction and study of what is possible! :+1:

10 Likes

Just around the corner :sweat_smile::relaxed:

4 Likes

An onsite hackathon in September. Does that mean we will be past Fleming or is it pre Fleming testing?

1 Like

Coincidence that it would be two years since alpha 2? I suspect they are aiming to release something but won’t say fleming incase they are not ready.

1 Like

Neat update Maidsafe devs.

A SAFE Hackathon would be super fun and luckily there is enough breathing room before it starts.

Let’s not forget this (just a little feedback Sarah & Ceilidh the roundup sounds more natural this time around :+1:)

Maybe it would also help to inform investors on bnktothefuture that they can check out the SAFE Network forum to know more about progress. Somebody commented on bnktothefuture…

:stuck_out_tongue:

15 Likes

Nick just presented the roadmap in the investor zone (on March 6th):

Alpha 2 (Safe Authenticator) - current milestone, a decentralised network which contains some early SAFE apps and websites.

​Alpha 3 - (SAFE Fleming) a technical release that provides a completely decentralised Routing layer for transient data. So this will test SAFEs core networking libraries in the wild and enable users to get involved by running a Routing node from home. This network will not have data storage (it will be transient) but will connect together users computers to form the network.

Alpha 4 (SAFE Maxwell) essentially a combination of alpha 2 and alpha 3. A decentralised network with data storage.

Alpha 5 (unnamed) Alpha 4 with test Safecoin. The next release after this will be Version 1. It is at this point we will become revenue earning.

Also someone made this suggestion a few days ago (on March 5):

I suggest you visit their forums to ask questions; you’ll be able to get quicker and more detailed technical responses from community members. https://forum.autonomi.community/

14 Likes

Well, as much as I don’t like to make predictions, and even less put unnecessary pressure onto the team…

Which, to me means data layer by August, and therefore Flemming before then (possibly May)!

2 Likes

I’m very glad for the mention. I’m trying to find problems to solve, where I can apply my enthusiasm with what I love to do, and it’s absolutely great to finally have some more time to do it with SAFENetwork. The SAFENetwork IMO is in the very top of important and worthy projects, all categories, all levels.
I try to do something that can be of use so I am really happy to know that it’s appreciated.

Thanks @Nigel for those words!
It’s a very inspiring community to be part of, all the different souls and personalities, so many realising the importance of this and wanting to see it happen, and I do feel very happy to be able to contribute.

36 Likes

Nice update as usual and thanks to the team for their continued hard effort towards Fleming!

5 Likes

It is great to see you having the time and creating so much @oetyng. Your knowledge, experience and skill had been apparent for a long time, but now we are beginning to see what you can do with that, and how important your contributions are to building powerful services and applications on SAFE.

I hope you keep enjoying this, and finding the time for it. Great stuff man, and thanks from me too :smile:

You bring a professionalism we can all learn from. :clap:

17 Likes

But isn’t data a feature of alpha 4 maxwell? Alpha 3 is routing layer only I thought

I believe that’s correct, but that doesn’t contradict what I say above…Maidsafe stated they’re planning to have AD implemented in some form by September (conservatively). AD is a data layer. Flemming, and this is the only arguable point, needs to be in place before AD is implemented. Obviously AD could be implemented on top of current consensus and routing, but that could have been done anytime since alpha 2, while it hasn’t and it still isn’t the priority.

1 Like

I think Flemming needs to be out there before the hackathon. Without this, people will dwell on what there isn’t, rather than what there is and what it will become.

5 Likes