MaidSafe Dev Update - December 6, 2016

Good news! It looks like we’ll be able to keep using the old “8 closest nodes”-type groups in safe_vault (and still use the secure message routing mechanism, without safe_vault even knowing about it). This means that we can reuse almost all of the existing algorithms in safe_vault. There are still edge cases that we’ll need to look into, but so far it looks promising :slight_smile:

SAFE Authenticator

Team leader: @Krishna_Kumar

We have been working on integrating SAFE Authenticator with the safe_browser. Since safe_core is not ready yet, we have been making some progress on the UI implementation and mocking the backend responses. Likewise, we have also started working on an example application that will allow users to upload web content that can be viewed from SAFE Browser. This example will showcase the new authentication mechanism and the usage of the new SAFE APIs. Unlike the demo app (which allows users to store private files), this app will only deal with web hosting and the files will be created in the _public container.

SAFE Core

Team leader: @ustulation

Low-level (FFI) access to manipulation of MutableData, provision of standard user directories and MaidManager simulation in mock-vault for app key authentication have been completed (but not thoroughly tested). We are proceeding to code the Authenticator flow as well the App flow, which will involve a new crate for system-uri calls to open other binaries. If we do manage to complete that this week, then we might move over to unit testing, which might expose minor flaws here and there. Or if anything else needs to be done, we might begin unit testing sometime next week.

Routing & Vault

Team leader: @AndreasF

As we mentioned last week, we tried an alternative approach to port safe_vault to the new routing, where the vault would keep storing each data chunk in exactly the eight closest nodes to the data, and routing would use the new structure as in RFC 37. It looks like this is the way to go: We got the mock Crust tests (with a simulated network, without actual TCP connections) to almost always work – the complex churn test that sends messages while creating and removing nodes still occasionally fails, so there are still edge cases that we’ll need to look into. At the same time, we’re extending the tests in both routing and safe_vault beyond what we had before RFC 37, to help us catch bugs earlier in the future. And while doing that, too, we expect to find a few more issues right away.

However, we’ve also started testing on droplets to make sure that actual nodes behave like the simulated ones, and to see whether there are any new issues that don’t show up in the mock Crust tests.

And there’s another caveat: this port is very much a set of quick hacks to test whether the new approach for safe_vault makes sense. Now that we are convinced that it does, we will replace these hacks with proper solutions. For example, routing will need to explicitly know and handle both kinds of “groups”…

As you can see, it’s becoming awkward to use the same word for both these concepts. So we decided to rename the new one! We’ll update the RFC accordingly. Here’s the new terminology:

  • RFC 37, “Disjoint Sections”, describes a way to partition the network: each node is always a member of exactly one section. And each section corresponds to a binary prefix: the section comprises all nodes whose names in binary start with that prefix. A section (except while churn handling is ongoing) always consists of at least MIN_SECTION_SIZE nodes. (Currently 8.)

  • A group of nodes is given by a number k ≤ MIN_SECTION_SIZE and some address in the network, and consists of the k nodes closest to that address. That is the same notion of a group as before RFC 37. Groups can overlap: A node will usually be a member of several different groups.

However, a group is always a subset of a section, and in a perfectly balanced network of exactly MIN_SECTION_SIZE * 2n nodes, the sections are exactly the groups with k = MIN_SECTION_SIZE.

So safe_vault will now keep using groups for managing data and user accounts, and routing will use sections for lots of upcoming features: securely relaying messages, making decisions about node relocation, banning bad nodes, managing data chains, etc.

And finally, we have started with the implementation of another big feature: resource proof. At first, this will be used only to measure the performance of nodes that are newly joining the network, and make sure that they have enough bandwidth and CPU to be useful. Later, this challenge will likely be applied repeatedly, e.g. when relocating nodes to a different address in the network.

This should help making the next test network more stable: A user starting 1000 nodes on one machine won’t be able to join with all of them at the same time anymore, and a phone on slow mobile data won’t either. In the long run we want the network to use all the help it can get, of course! But we first need to get the basic structure right, and at the current state, slow nodes are still doing more harm than good.

SAFE Browser

As mentioned last week, Krishna’s team reviewed the safe_browser implementation and passed a few clarification questions to @joshuef. We now have answers to most of our questions and @joshuef also cleaned up a few minor nitpicks that we had. We are looking forward to bring safe_browser under a MaidSafe repository soon.

Dev Outreach

Ben is giving 3 presentations this week:

And Nick is giving a presentation at Bitcoin Wednesday Amsterdam #42.

Ben will then be speaking at the Microservices Conference
in Berlin
(February 16-17, 2017). The title of his talk is “Beyond the microservice: decentralising the entire infrastructure”.

After that, he will be speaking at BOB Conf Berlin on February 24, 2017 about Privacy-first data structures on SAFE.

Blockchain technology like Bitcoin popularised the concept of decentralised consensus. Together with the rise of peer-to-peer technologies, it becomes possible to entirely decouple data-ownership from its physical storage and reorganise information in a privacy-sensitive manner. SAFE is a decentralised communication and storage network with a local-group-consensus mechanism to ensure consistency. But without central servers how do you organise user-generated content – like comments on a blog?

In this talk, we will explore the privacy-first data structures employed on SAFE, how you can use cryptography to provide access control on the data layer directly and thus rethink the idea of “central storage in the cloud”. Finally, we will take a short look into the important role functional programming will play in these networks in the future.

Recruitment

Michael Sproul is the newest member to join the MaidSafe team! He’ll be working remotely from Australia. He’s likely to start in Andreas’ team (Routing & Vault) this Thursday.

We also have a few more candidates going through the recruitment process. We are looking at getting more people in Spandan’s team (SAFE Core & Crust).

55 Likes

Welcome Aboard Michael!

9 Likes

very nice to hear the team is getting even bigger and that more outreach is planned too :slight_smile:

6 Likes

Great work guys! I’m hyping way too much because I see so much progress and am anxious to get my greedy little hands on the new at home vaults. I’ll be sitting quietly and patiently until I see @frabrunelle put a :tada: testnet 12!!! On a future dev update :confounded: this is me trying to contain myself. Glad another team member is on board and starting soon! Welcome Michael!

7 Likes

Welcome Michael! Congratulations @joshuef and thanks @frabrunelle for the update!

5 Likes

Great news!

Good luck with all presentations and sharing the story about SAFE. Also a warm welcome for Michael :+1:.

If remember correctly “Node Ageing” was part of the upcoming Vault binaries as well. But I guess we’ll see a test when Disjoint Groups is fully implemented and tested together with resource proof?? I mean, that’s quite a big update.

6 Likes

@frabrunelle Very helpful and well written update Francis, thanks! :slight_smile:

10 Likes

Thanks so much to everyone for all of your hard work. Be sure to take a long vacation over the holidays.

5 Likes

The word that comes to mind as we experience the ever increasing, steady progress, both tech-wise and organizational, is “SOLID”. And I mean that in all of the good ways and none of the bad. :wink:

15 Likes

Okay I’m confused with all these new developments with the authenticator and integrating it with the SAFE browser and all. Will we have a “demo app” that we can upload public and private data with still? Are you simply creating a new type of app or retiring the old one? And if you’re replacing the old one how does one then upload private data? Will we even have a launcher still? What is the process for signing up to the network with the new authenticator in place? This is just all so confusing.

1 Like

You’ll have all the functionality from the Launcher and Demo App in the browser. So you start the SAFE Beaker Browser and notice it has a icon in some way or the other where you can do all in the browser itself. Including browsing websites. it removes complexity. Start the browser and you’re good.

14 Likes

Thank you all for the warm welcomes! :smile:

I can’t wait to get started writing code for the SAFE network!

21 Likes

Just don’t wait! :stuck_out_tongue_winking_eye:

8 Likes

Thanks Maidsafe team for all your hard work, soon this will pay off.

Welcome @michaelsproul tell your coding friends about this project

Wow can’t wait for SAFE Browsers integration :stuck_out_tongue:

7 Likes

Feel free to give a sneak peek of the UI :slight_smile:

Sensible approach and if the next test uses hosted nodes, please enable a CDN POP that can be turned on and off to properly test performance down under as we dont have as many nodes down here[quote=“frabrunelle, post:1, topic:11921”]

SAFE Browser

As mentioned last week, Krishna’s team reviewed the safe_browser implementation and passed a few clarification questions to @joshuef. We now have answers to most of our questions and @joshuef also cleaned up a few minor nitpicks that we had. We are looking forward to bring safe_browser under a MaidSafe repository soon.
[/quote]

Please enable a stretch goal for a ‘SAFE only version’, you know it makes sense

Recruitment

@michaelsproul from Australia…hooray :tada: …hopefully he’s from a real footy state originally :slight_smile:

7 Likes

Okay… That’s awesome and all but doesn’t that put way too much dependency on the browser? Maybe ya’ll are crazy about web apps. Personally I like stand alone applications I can quit and start up independently.

2 Likes

I’m not sure but maybe think of the browser as replacing the launcher with regards to stand alone apps. So overall it’s one less so to run, but you can still run stand alone apps too.

7 Likes

Will this be recorded?

6 Likes

Fantastic progress as always from the team! Look forward to spending more time delving into the new tutorials and SAFE APIs with the devs Meeco :slight_smile: Our Founder/CEO has asked me to reach out to the team and look at collaboration. Exciting times :slight_smile:

15 Likes

In the Meeco blog post: Immutable Me they sound fairly invested in Blockchain technology.

Are they investigating the possibilities here or maybe they have done extensive research into SAFE already?

2 Likes