SAFE Network Dev Update - October 18, 2018

Summary

Here are some of the main things to highlight this week:

  • The first version of the Crust Test will be going offline tomorrow morning. Before releasing the next test, we’d like to work on a couple of improvements in order to increase the information returned from the test.
  • SAFE-Fleming is our new name for Alpha 3.
  • Next week, @dugcampbell will be heading across to Berlin for the Web3 Summit - so if you’re going, say hi!
  • @m3data is running a survey to gauge interest in a SAFE Network: Sydney meetup.
  • The Routing team estimates that they have tackled around a third of the kinds of malice they detected during their brainstorming.

Release of the Crust Test

First up, we’d like to thank everyone who has taken the time to get involved with the release of the Crust Test over the last 24 hours. Excitement reached fever pitch as people from all over the world successfully connected to each other by testing out the SAFE Network’s hole punching functionality. There have been a few comments on the Forum along the lines of ‘it’s amazing to watch the birth of the new internet’ - and we couldn’t agree more :slight_smile: The test is the visible culmination of many months of work by the Crust team, with everyone across the business working together on preparing the release over the last couple of weeks so we hope you’ve enjoyed it as much as we have!

We’re delighted to say that we’ve now achieved the goals that we set for the initial part of this test. We’d like to work on a couple of improvements in order to increase the information returned from the test - so as of early tomorrow morning, we will be stopping the first version of this Crust Test.

One of the most important iterations we’d like to make to the test is to enable the next version to record direct connections between nodes. In addition, we knew that the Dashboard would have certain performance constraints so we’d like to ensure that this is a little easier for you all to read (essentially the next Dashboard version won’t have to load all the data with each refresh). Finally, in addition to displaying data for all tests that have taken place, we’d like to let individual users filter the results in order to show their own connection results in isolation.

It’s important to point out however that this is another stage on the route to SAFE-Fleming (our new name for Alpha 3). Oh, and don’t forget to give both posts explaining this week’s activities the full 50 claps on Medium if you feel they deserve it ;-). So whilst we’re looking at making a few improvements for the next iteration of the text, the goal here is not to provide a fully-functional all-singing all-dancing dashboard. Instead, the goal is to provide a basic feedback mechanism for the community that provides a feel for what is happening - and just why it is so important.

Marketing

Unsurprisingly, the Marketing team has pretty much been working on the Crust Test release exclusively this week - drafting Medium posts, messaging for social media, storyboarding and recording the video, updating DevHub - together with a myriad of many other details relating to the release and promotion.

Next week, @dugcampbell will be heading across to Berlin for the Web3 Summit - so if you’re going, say hi! Also, it’s worth pointing out that @m3data is running a survey to gauge interest in a SAFE Network: Sydney meetup.

SAFE API & Apps

This past week we focussed on improving the packaging for the safe_app_java Android Library and its compatibility with the jcenter and maven repositories. Internal review/testing of the example mobile application gave way for some fixes and suggestions that are now being implemented in the application.

@shriram2301 has been ramping up and showing good progress in understanding the SAFE Browser codebase, as well as the safe_app_nodejs API. He’s already been assigned a first bug to fix which he is using to also understand the development process we follow in the team.

There has been an improvement implemented in safe_app_nodejs recently (PR #295) to remove the need of setting a NOVE_ENV env variable to download the native libraries to use the Mock network. This has been causing problems to users in the past and we are hoping this will allow applications to not need this environment variable when they are being executed for the Mock network.

Part of the team has been trying to resume its efforts on the initiatives for supporting RDF data & APIs, as well as XOR URLs to better support linked data. As you probably can imagine, now that the Crust Test has been made public, this was one very important milestone for the project, and for the company, which required some help from the frontend team as well. This slowed us down or even distracted us from being that active in relation to RDF and linked data related tasks. Apart from the discussions and details we’ve already been sharing in our forums, we are planning to trigger some discussions about these topics also on the Solid Community Forum in order to get some feedback from that community as there are many people there with good knowledge of RDF and linked data.

We would like to encourage anyone interested in these topics to participate in the discussions we are having. We try to be focused and pragmatic in the process, with concrete details of how things can be changed and implemented and not just from a theoretical angle, but with code snippets, data structures examples, etc. We will shortly be creating a new post on the Dev Forum to trigger a discussion around a proposal for migrating our DNS data to an RDF representation, and it will be important to have you all keep an eye on it and provide your valuable technical input and feedback.

SAFE Client Libs

This week we’ve been continuing to refine and polish the API for the next Client Libs release. We added a new constant, GET_NEXT_VERSION, which can be passed to the API functions dir_update_file and dir_delete_file to automatically fetch the next version to be used from the network. Previously, one could pass the special value 0 to dir_update_file, which led to some confusion from community members who thought this should be a bug. Also, these two functions dir_update_file and dir_delete_file will now return the new version of the file entry.

Routing

The main task that has kept the Routing team busy this week has been to continue and implement detection of various types of Malice in PARSEC and implement functional tests for each of these.

Some forms of Malice have required important additions to the code, such as the addition of “per peer membership list” that started last week. In simple terms, this allows any node to place itself in another node’s shoes in order to figure out if they respected the protocol at all time. We started implementing this task last week but left aside some aspects of the task that we have been picking up this week. Since it is a rather important change to the code, we will only really be able to call it done once we’ve implemented the tests for it.

For implementing these tests, we need to add support to “per peer membership list” in the generation and the parsing of our dot format graph files. This task is currently in progress. We are also using the opportunity to further refine our graph files syntax, so as to help with any new tests we will be writing.

As we are coding all these new features, we are continuously refactoring the codebase to keep the code maintainable and allow us to maintain velocity for the upcoming features.

In addition to these malice related tasks, we extended PARSEC’s API with two methods that we expect will be needed for integration with the Routing repository. One allows checking whether there are any observations in the graph that haven’t obtained consensus yet. The other one retrieves all unpolled observations created by a given peer.

This should allow us to start work on integrating PARSEC with Routing in parallel with the continuing work on malice detection. This is the reason why we tackled dynamic membership in priority over malice detection. Expect to see some of this progress along the road to SAFE-Fleming :wink:

Having a look at the bigger picture for milestone 2, we have started to make a dent into malice detection as we have tackled around a third of the kinds of malice we detected during our brainstorming. This should keep us busy for a while longer, after which we will be turning our gaze at the other aspects of milestone 2. Namely: performance analysis and enhancements, production-ready documentation, additional testing and enhancements to the whitepaper based on all the feedback we have been gathering.

Crust

We’ve been working hard to make the Crust Test happen. We made a special application that makes use of the Crust and p2p crates, attempts hole punching and sends the collected information to our server-side software. It then collects all of those results which I’m sure you all saw on our live dashboard :slight_smile: Our main focus was hole punching but at the same time we collected UPnP availability stats which we’re sure the Marketing team will summarize later.

We’re also aware of several bugs and inconveniences that prevented some people from running the test clients on their computers. Because the client binaries require rather recent versions of operating systems, the users who prefer more stable OS releases like Debian, Ubuntu 16.04, or macOS High Sierra ran into a variety of problems trying to execute them. We have a plan for lowering the OS requirements and making the Crust client accessible to a much wider audience for the next iteration of the test.

85 Likes

First for the first time, in this awesome weekly update thank you all the team of maidsafe and all the community for making this great project possible!

29 Likes

Thanks so much for all of your hard work!

20 Likes

ok so no alpha 3 this year

Aaaaaaarrrrrrghhhhhhhh [20 characters]

25 Likes

Nobody said that it would be, but Ill bite :slight_smile: what was said here to make that your only comment?

If we are reading the same update, things are looking great!

20 Likes

Nope no alpha 3 but we might get SAFE-Fleming if we’re nice :slight_smile:

Great work @maidsafe this week has been a thrill. Looking forward to what’s next already! Glad we can be of use.

Add on: welcome @shriram2301 :slight_smile: and good luck to @dugcampbell in Berlin!

36 Likes

This was a very encouraging update, and week in general, I feel.

We may not get SAFE-Fleming this year, in fact it looks almost certain. However, I don’t think it will take too much longer into 2019, based on their pace of work. The upside is, Alpha 3 appears (to me) to be the bulk of the “unknown” work that is left. Alpha 4 is vaults from home, which has already been demonstrated before, and is not too difficult conceptually. Additionally, Alpha 4 may contain the initial Safecoin implementation, which may require some more research, but should be relatively little actual coding work.

With the continual team growth, and pushing through these research hurdles, I feel pretty good about a late 2019 delivery for a Beta or RC1 phase.

31 Likes

Great update this week.

Would like know how much about 41% successful connection rate for hole punction can be improved in future.

4 Likes

Another great update! The Crust hole_punch test was fun, and amazing, bypassing servers, and connecting to personal computers, this is a big deal. it works! The Dawn of a new internet is upon us. This project is really starting to shape up. Thanks for the opportunity to participate.

23 Likes

Great week and I completely forgot it was update day in all the crust test excitement! Glad to hear routing is coming along nicely too!

Keep up the great work!

22 Likes

Where does it say that? :sweat_smile:

4 Likes

Great progress! Was cool to be part of the Crust test and see connections taking place, definitely makes progress more tangible.

Be great to get some feedback on the survey I have draft for The Sydney meetup.

I look forward to the proposal on RDF representation and providing input. Also for the insights and potential collaborations that come out of the web3 summit.

Onwards @maidsafe team!

15 Likes

Is it just me or does RDF DNS sound like some next level sh*t?

11 Likes

What’s 6 to 9 months in the scale of things. Getting it right is key, and I’m glad Maidsafe are crossing and dotting the absolutely necessary things that are required

9 Likes

Great update. Is this the story behind naming it Safe Fleming?

From the medium article: " after Sir Alexander Fleming, the noted Scottish Nobel Prize winner from Ayrshire who discovered penicillin"

Interested to know how the name was decided.

11 Likes

Wow, how great was it having software to enjoy this past 48 hours?
Seeing familiar names and new names showing up on my desktop was amazing.

I was listening to some American radio today, we have a lot of great long term supporters in the United States continuing to bang the table for the Safe Network.

Really looking forward to the concrete mix being ready for the foundations of a new internet.
And once the mix is poured … it’s skyscraper time for developers and supporters!

@abennett Maybe It’s the penicillin for the internet’s illnesses, without a sugar lump though, because Safecoin comes later. :blush:

19 Likes

Yes! I am interested to know, I was thinking it was because his discover saved the lives of so many people :slight_smile:

5 Likes

Alright … So after seeing that pic at the top … I really have to know …

How many of the team are “staying late” at the office in order to play video games on that HUGE screen monitor!!!??

10 Likes

IMO, building for LTS releases is the best way forward. When we get to beta and people start putting up ‘farms’ in the cloud, LTS supported builds will be expected.

6 Likes