MaidSafe Dev Update - November 2, 2017

SAFE Browser v0.7.0 was released earlier this week (with binaries both for the mock and actual network). This is the first update to SAFE Browser since the beginning of Alpha 2 and it includes a bunch of fixes/updates.

We also have a new Android developer, called Janmejoy, starting next week based in India. This reflects our ongoing commitment to grow the team and adding more resource to our mobile team. We continue to recruit for a number of roles – Software Testing and Release Manager, UI Designer and a Software Support Engineer – to be based at our HQ and our new marketer @SarahPentland has been working with @victoria to increase the visibility of these opportunities while Victoria has been headhunting suitable candidates.

The other exciting news is that we have decided to build a fully customised SAFE Browser which will cater very specifically to SAFE functionality. Some very high-level details and links are below, however, the first step is to work toward a Proof of Concept (PoC) which will check our assumptions and further define the best path forward. Further positive news from the front-end team as they make really good progress on the Java bindings generator, and their tests into providing backward compatibility between networks suggest we may not need to shut down vaults to update SAFE APIs, although as you will see there is still some work to be done in this area.

SAFE Authenticator & API

@bochaco is updating safe_app_nodejs to the changes in the master branch of safe_client_libs. The DOM API will be updated to use the latest safe_app_nodejs after the Node.js integration is tested. The changes are not just internal, it affects the API and thus the test cases must be updated.

@shankar is working on the last phase of refactoring the web hosting example. Once that is out of the way, he is planning to start working on the developer website by grouping all documents and examples under one roof. The prototype of the dev website will be made available in a couple of weeks.

@hunterlester is focusing on advancing the CI systems to get the tests running. The browser CI is also being improved for auto-publish to GitHub releases on new tags.

We are also making some progress with the mobile platforms. @rachit has set up a sample using Gradle and we will be starting to integrate with the JNI bindings next week. On the other hand, the C# API internals are being refactored and a test framework is being integrated. The remaining APIs that were not integrated during Alpha 2 will be integrated in the following weeks.

@srini is working on an editable comments plugin, to demonstrate how versioning can be implemented using the APIs.

After last week’s Dev Forum post on Browser development and some helpful community responses there, we’ve decided to start in the direction of a fully custom SAFE Browser. So far we were using a fork of the Beaker Browser.

While browsers like Firefox and Brave were initially enticing, the complexity of using these would involve either:

  • A lot of workarounds.
  • Diving into core code to enable basic functionality.

Neither of which are particularly desirable.

Furthermore, when we get to look at more advanced functionality (streaming/SAFE-specific functionality) things start to look even more complex.

As such we’re planning to start working on a fresh Electron-based browser. This will let us use the knowledge we’ve gained so far, but work in a much cleaner fashion and focus on what SAFE: needs. So we’re planning to release a basic SAFE only browser as a first step in the next couple of weeks. We consider this as a POC for the custom browser. Based on the stability and feedback of the custom browser, we will be deciding on whether to continue on this route or to rethink on other options.

All going well there, we’ll then look to reintegrate the authenticator functionality before planning if/what other features we want to see in the new browser.

SAFE Client Libs

We are continuing to think about how to maintain backward compatibility between the networks. Our recent tests show very promising results: the recent changes in the master branch are compatible with the data currently stored on the Alpha 2 network, which means that we won’t need to shut down vaults to upgrade our APIs. @adam is currently working on serialisation tests to confirm this and to make sure that this compatibility is maintained in the future. However, as promising as it is, we’ve yet to find a way to make core data structures future-proof, like in the case when we want to introduce multi-sig to Mutable Data. We’re looking for various options to make it unnecessary to shut down Vaults even in the case when we need to introduce new capabilities for our data structures. As mentioned in the previous update, we plan to write and publish a comprehensive RFC document on this topic at some point in the future.

We expanded our safe_app API by implementing new functions to the crypto module (MAID-2412). We started by adding a new object cache handle for secret signing keys named SignSecKeyHandle and replacing the existing SignKeyHandle with SignPubKeyHandle. We provided functions for dealing with the new secret key handle, such as sec_sign_key_get, which we modeled after the equivalent functions for public key handles. We also added the function sign_generate_key_pair which generates a pair of such keys and returns the handles. Finally, we added two new functions sign and verify. sign can sign data using either an app’s own secret key or any arbitrary secret key handle, while verify takes some signed data and returns either the original plaintext or an error if it could not be verified. Overall, the new API functions should feel familiar as their usage is consistent with existing functions.

In addition to the above, we have merged a fix for app reauthorisation using 2 PUTs and an API change replacing value structs with pointers.

Lastly, we’re having good progress with the Java language bindings generator and we should see first tangible results in the next week.

Routing, Vault & Crust

The Routing design discussions have continued this week with the focus being on the relative pros and cons of the two remaining proposals, namely vanilla Data Chains and the Section Graph variant. They’re intentionally very similar options, but have a couple of fundamental differences. We’re now leaning towards Data Chains, but still have to flesh out some more details before we embark on the real low-level design task. The areas we’re looking into include penalties for misbehaving peers (including how to detect them, how this could be exploited, the severity of penalties), peer ageing and handling mass peer loss across the network.

No major changes in Crust this week but lots of minor ones. We’ve been hard at work improving documentation and expanding our test suite, which has enabled us to catch and fix a couple of bugs. Thanks to that, connections should no longer hang when they’re being closed, and config-file monitoring should now work reliably. In the next few days, Crust should get to the point where we’ll be ready to start integrating our NAT traversal and uTP libraries.

84 Likes

First!

Great to see mobile app development getting traction. Welcome Janmejoy! I feel this will really speed adoption when the time comes. So many projects don’t have it that it will be a real differentiator, IMO.

14 Likes

Good stuff - While there is obviously still plenty to do I get the impression that the network is really starting to solidify and stabilise in many different areas now.

My key takeaways from an initial read through:

We also have a new Android developer, called Janmejoy, starting next week based in India

Welcome Janmejoy, as @Tom_Carlson says supporting mobile is very important

@shankar will start working on the developer website by grouping all documents and examples under one roof.

Hallelujah! Much needed.

So we’re planning to release a basic SAFE only browser as a first step in the next couple of weeks.

Only takes two weeks? Blimey. Will it be made available for us to try out at that point? Shame you couldn’t adapt something already there, but I understand browsers are now very complex beasts and SAFE might get locked into something it couldn’t control - look forward to seeing the new one in action.

the recent changes in the master branch are compatible with the data currently stored on the Alpha 2 network, which means that we won’t need to shut down vaults to upgrade our APIs.

Another sign of increasing maturity and solidity.

Cheers everyone :slight_smile:

23 Likes

Great news on custom browser! I hope that goes really well as I know some folk that will depend on the expanded features it will bring more organically. Curious how the iOS mobile app is coming in the approval process @maidsafe. Can’t wait to finally see data Chains fully fleshed out, my assumptions were so far off for when it’d be close. :sleepy: I just want to rub it in all the haters faces. All joking aside, great work to the team! We’re always eagerly absorbing as much as we can to be an informed and active community.

19 Likes

Probably more than two weeks :stuck_out_tongue: I think this part should be read as “in the next few weeks” :slightly_smiling_face:

I think the plan is to test it internally first and then release it to the community.

/cc @Krishna_Kumar

14 Likes

You know what ? I’m happy to read this :sweat_smile:

oh yes, the documentation really needs being raked in a unified place… When I stop working on Safe for a few days, and get back at it, it takes me a few days to remind / rediscover where the docs are…

17 Likes

Great update maidsafe!

11 Likes

Why do you think data chains is far away from release?

2 Likes

I don’t think it is but I thought it would have been implemented by at least october based off many things I had read but predictions are stupid especially coming from me :stuck_out_tongue_closed_eyes:
I can follow along quite well but I don’t know what the big boys know or have a crystal ball. It seems close enough to me even though I wrongfully assumed otherwise. Don’t panic, these guys know what they’re doing.

8 Likes

Brilliant update MaidSafe. Really moving forward on many fronts in parallel.

Excited about the new Android dev (welcome Janmejoy!), and that data chains conceptual discussions are homing in on the final approach to choose.

It must be considered quite far from release when the overall approach is still undecided, and there are potential show-stoppers that haven’t yet been overcome.

I’m sure they’ll get through the challenges, but it may take some time.

Thanks again for sharing such detailed updates!

10 Likes

Welcome to the club! Let us know if we need to test something.

Great update, as always. So much going on. Think making your own SAFE Browser is a good idea.

9 Likes

Hello!

Maybe too early, but on maidsafe.net version 0.6.0 for download and when I click in browser “Check for Updates” got error ‘Error: Can not find Squirrel’

First, as always congrats on progress. :clap: Regarding data chains and other previously highlighted tasks. Has secure message propagation and chain linearization been solved? What are the remaining problems that are being worked on aside from malicious node participation? I feel stuck in a haze. Data chains deserves a detailed blog post methodically working through all of the challenges and proposed solutions. It is the backbone of SAFE after all. :blush:

9 Likes

Wow, I think Captain Obvious must have paid a visit…a 100% autonomous SAFE network is back on track.

Maybe the Captain, being a numismatist at heart, could the find time to address the team on just how exciting Safecoin is as a marketing strategy.

8 Likes

Hmmm - sorry, you’ve lost me there :thinking:

Thanks Maidsafe devs for all your hard work.

Good news and welcome Janmejoy

Prediction: In the future every browser manufacturer will integrate SAFE vault, because it will be a source of income. Only question is, will they share this revenue with users? :stuck_out_tongue: Hmmmm some will probably be greedy and take up 50Gb of users storage and not give them a dime. :kissing_heart:

8 Likes

Is this what was originally option A? as in section graph was originally B.
Or are these two different variants of option B.
Confuzzled as it was said that option A was open to too many attacks. :thinking:.

2 Likes

Yes this is all eventual consistency and natural design. Option A was more tangeroa/pbft etc. which were designs to force sync between nodes at every step. So linear data chains or branching data chains (graph like) is where we are at. They are very similar in many ways so we are finalising the code complexity and nuances of each one.

15 Likes

HI There,
i appreciate everything that you guys are doing. I understand its a challenging task and thats why i am still holding ALL of my maid since ICO from years ago HOWEVER as much as i love the seemingly progressive updates a real big question as to be raised as to WHY is this project taking so long! Its been in the works for ten years !!! Just release something already - this is getting rediculous and your going to get overtaken by other projects.

2 Likes

Rewriting the internet’s code is a big job.

The Maidsafe team is trying to do something insanely great that will alter the world.

They deserve support and patience.

10 Likes