SAFE Network Dev Update - February 28, 2019

Summary

Here are some of the main things to highlight this week:

Marketing

Another week of building up knowledge and materials on top of the usual daily business-as-usual for the Marketing Team. Ultimately this work will translate into documentation that fills some of the existing gaps in explaining the existing (and coming) tech for newcomers to the project. In addition, the next SAFE Network: Brighton Meetup is now arranged for Thursday, March 14: @dugcampbell will be giving a general introduction to the SAFE Network slanted towards a non-tech audience before @bart calls in via video and gives an update on PARSEC that youā€™ll not want to miss!

SAFE API & Apps

Over the last few days, @vigneshwara has been gunning down tasks in the safe_app_java milestone. The unit tests now demonstrate additional scenarios such as sharing of mutable data, manipulation of permissions, etc. Developers can use these unit tests as API usage reference while building applications of their own. We have also completed integration with coveralls.io for code coverage. Additionally, with some help from community member @ogrebgr we are also exploring a new route for the packaging and distribution of the Java desktop library.

We are working on project scope documentation of all the C# repositories which will help community developers to understand the scope and structure of different projects. The documentation will also be helpful for new contributors bootstrapping on the various projects. Also, we are wrapping up the UI/UX improvement milestone for the SAFE Authenticator mobile app.

A new version of the SAFE Browser was released, containing a number of small fixes and enhancements. Read the changelog for more details.

Recently weā€™ve started some exploratory tasks looking at the possibility of having some form of CLI for some of the most common and/or most used features available to end users and client apps. For example, developers creating desktop SAFE apps need the Authenticator and Browser running to be able to authorise an app and test it, thus we are starting to look at how a simple Authenticator CLI could help in this regard. We are in the very early stages of such research and depending on how this evolves we will be sharing more details and plans for it.

SAFE Client Libs

We are continuing to make good progress on adding RDF stack support to SAFE Client Libs. This week we solved a blocker in the proof-of-concept project for Mutable Data based triple storage implementation. We are considering taking a bit simpler approach in integration with Redland (a standard-conforming RDF library written in C), utilising the existing code and making a number of required changes minimal for the first iteration. Once this problem is solved, we should be able to start using SPARQL queries on the triples stored on the network, making it a new way of consuming and analysing data on the SAFE Network. We will provide more interesting use cases and examples soon, demonstrating why native Linked Data support in the Client Libs is important.

This week weā€™re also preparing to publish the draft of the first RFC on RDF on SAFE to start a public discussion. This RFC defines RDF support in a more broad and abstract sense, proposing an API and listing the benefits it brings to our development stack and the network in general.

We also continued reviewing and publishing our documentation to the Client Libs wiki. Weā€™ve updated the Readme to make it less verbose, linking to the wiki to save on space and to be less overwhelming to newcomers. We moved our safe_core module-level documentation to the wiki as well (though the docs.rs page will not be updated until we do a version publish).

Weā€™re continuing to work on the comprehensive documentation for Client Libs. Note, however, that for the time being the wiki is still being worked on, and some links to other pages may not work yet ā€“ please be patient while we continue publishing all of our docs. As a bonus, @marcin took some time this weekend to create a nice overview of how all of our crates fit together :slight_smile:

Routing

Continuing on from last week weā€™ve been further focusing the scope and features of Fleming, aided by the simulations and probability modeling for Sybil attacks and message delivery. We have a few alternatives for reliable message delivery that weā€™re weighing up the pros and cons of and we expect to enter a phase soon where some of these will be made more concrete in code.

The updated Parsec whitepaper is nearing completion. It improves the algorithm to make it fully asynchronous using threshold cryptography. As explained earlier, the updated whitepaper wonā€™t result in an immediate change in the implementation as we are prioritising features that are must-haves for the release of SAFE Fleming. In the meantime, we are drafting an RFC to capture all the knowledge gained when looking at the implementation closely a couple of weeks ago.

The ongoing performance optimisations for Parsec is still a priority and is progressing nicely, with both algorithmic and implementation improvements. A few changes that are in the pipeline increase the bandwidth of the algorithm significantly, especially when membership changes are less frequent than other events; which we think will be a realistic scenario in a live network. The main idea is to bundle together multiple opaque payloads inside single gossip events so that the bandwidth isnā€™t limited by the CPU cost of processing a given gossip graph. Fundamentally, this allows us to have a configurable trade-off between throughput and latency. We are still preparing these changes for integration into the codebase. Once they are integrated, we think we may be approaching the point where we believe the scalability to be sufficient, prior to gaining more data by running actual test networks.

Crust

As we continue to move toward Fleming, we are currently focusing on detailing the requirements to enable dynamic connections (and the API changes), which will allow Crust to scale up to thousands of connections. We are almost ready to start drafting the RFCs to bring in wider engagement and discussion.

In addition, we simplified bootstrap cache and we believe we made it more robust at the same time. So now if cached peers are inactive for a prolonged time, they wonā€™t be kicked out of the cache because they might become alive in a short time again. The only time peers will be evicted from the cache is when it is full and we want to cache new peers that we definitely know are connectible.

81 Likes

First for the first time! Keep up the good work Safers!!

19 Likes

second!

Anyways I really applaud you going out to show this project to non-techy people. I would love if you could also dumb down these updates for people like me lol. Or at least after you are done talking like star trek ā€¦ ya know: we integrated the RFH21 conduits with the warp coreā€¦ then say something about what that does in plain English like now we can travel at warp 6 with these upgrades. You do this sometimes and I give you credit for that, but please try to do it more and reach out to even lower IQs (like me lol)

22 Likes

Thirdā€¦ 20ā€¦ I am third.

10 Likes

Do you have any idea when alpha 3 could be released? I mean could it be in 1 month, 6 months, 1 years, more?

seems like test nets will begin soonish, so lets hope the alpha 3 can be released in 1 month or so

7 Likes

Nice! Keep plugging away @maidsafe youā€™re getting so close!

11 Likes

Iā€™d like to see an alpha 3 in one month!!! But Iā€™m not that confident seeing how the development is so long :persevere:

1 Like

yes, its taking much more time all of us would hope, but ā€¦lets hope that what takes long to build now will also last much longer ā€¦than so many hyped blockchain projects with no real purpose. They mentioned test networks for the first time in this update, that should also tell you that we are getting thereā€¦

12 Likes

Go scalability, go test networks! Coming soon. Great job team!

7 Likes

Check out the crate graph :exploding_head: :smiley:

16 Likes

I am sure we will see Alpha 3 SAFE -Fleming within 12 monthsšŸ˜…

2 Likes

The grindstone wheel is whirring away at high speed as usual! Thanks for the update and keep at it - the SafeNetwork blade is getting a keen edge.

7 Likes

I suspect itā€™s within 3 months. Theyā€™ve been de-prioritising big things to keep the focus on the fleming release, so it must be fairly close.

11 Likes

Yeah I am glad to see them focus on the here and now and moving towards a milestone, there will always be ā€œnice to haveā€ features popping up in a project of this scope and as a dev myself I know my tendency to want to dive right into them forgetting the big picture :laughing: . But keep focused and meet goals and note the improvements and new good features thoughts up in some github issues/project board for later review and implementation.

13 Likes

how was that graph generated?

3 Likes

You can check out the details here, including how it was generated (under More Info) and a legend of what all the colors mean :slight_smile:

8 Likes

thx for sharing it with other Rust users! i had actually searched crates.io without finding it. i recommend adding https://crates.io/keywords/visualization.

4 Likes

No problem! Iā€™ll add the keyword next time I publish a new version for cargo-deps. Thanks for the suggestion! If I have time this weekend Iā€™ll also add some new features that should make it easier to use :slight_smile:

8 Likes

New MaidSafe podcast
First :wink:

Just realized: This is from 2014 oof

4 Likes