MaidSafe Dev Update - November 9, 2017

Today, we are releasing an editable comments plugin. It’s similar to the comments plugin we released a few weeks ago (see this dev update), except that it allows users to edit their own comments and to see the version history of each comment.

This JavaScript plugin can be integrated into any safe:// web page by following the instructions here. We’ve also set up an example at safe://blog.safe-examples (on Alpha 2) for community members who want to try a hosted version of the editable comments plugin. To host your own version of this example, you can download example-blog.zip from here and upload it to the SAFE Network (Alpha 2) using the Web Hosting Manager application.

SAFE Authenticator & API

During the last few days, we’ve been progressing well on making the necessary changes in safe_app_nodejs to be able to upgrade to the latest code from safe_client_libs. There is only one minor issue detected which is being investigated at the moment. The corresponding changes in the authenticator plugin were also made to make use of the latest authenticator code from safe_client_libs.

We started working on the bindings to expose new functions from the safe_app_nodejs API which are provided by the new safe_client_libs, like the functions to manage signing keys (generating them, signing and verifying signatures), the ability to query for containers permissions granted just using the authorisation URI without the need to connect beforehand, and the ability to access the account’s information which contains the number of mutations done and available.

After finishing with all changes in the safe_app_nodejs API, we plan to make the necessary changes to our example applications so they can make use of the latest version of safe_app_nodejs and safe_client_libs. All these changes will be compatible with the Alpha 2 network and stored data, therefore during the integration tests, backward compatibility tests will also need to be performed.

The scripts to automate the tasks of packaging and publishing both of the SAFE Browser’s plugins on the S3 servers have been put in place, and they are ready to be used in our next release. They are automatically triggered when we push a new tag to each of the plugins’ repositories. Analogously, a script to automatically create the safe-node-app package and publish it on npmjs.com is in place and ready to be used in our next release.

@srini has created an editable-comments-plugin example to demonstrate how versioned data can be handled using the APIs. @shankar has started to work with @shona in planning the developer website.

@rachit and @krishna_kumar have set up the Java repository for safe-app using Gradle. Composite Build Setup of Gradle is used to set up the project. Since this feature of Gradle is still in incubating stage of development, only the latest versions of the IDEs support the same. Android Studio version 3.0 and IntelliJ 2017.2 support composite builds. This setup helps us to have one shared codebase for both Android and desktop library files. The repository is set up to get JAR files of all combinations (macOS/Windows/Linux, Mock/NonMock, x86/x64) for desktops and mock/non-mock versions for Android.

With @joy (Janmejoy) joining the team this week we will be able to focus on Java and C# in parallel.

@joshuef has been focusing on the custom browser and has been progressing well. The POC consists of a basic HTML browser with a simple UI. The new browser has a comprehensive test suite and more robust logging, both of which should help speed up development of new features. Now we’re progressing with a much tighter integration of SAFE protocols, which should also lend itself to a more streamlined development process going forward.

SAFE Client Libs

During this week, we’ve been working on catching and fixing bugs, including the ones reported by the community. The first bug concerns the apps revocation feature: in some rare circumstances, users could be locked out of the ability to revoke any apps. Thanks to the very detailed report by community member @JPL, we’ve been able to pinpoint the cause of it quickly, and it turned out that our revocation code didn’t account for the way entries are stored in the apps’ containers by some of the community apps. So that when the re-encryption process kicked in, it tried to decrypt those entries, and because some of them were unencrypted, it failed, effectively failing the entire revocation process. Now that we have errors recovery, this re-encryption was attempted repeatedly, resulting in continuously failing revocation attempts. This bug is fixed and merged now and should be integrated into the SAFE Browser soon.

We also fixed a bug with the is_mock_build() function in safe_core not working correctly. The cause was simple (invalid syntax which the compiler didn’t catch) and the fix was simpler.

Another bug was in the system_uri library and was present only on the macOS platform. The underlying cause of it turned out to be quirky, so it took some time to investigate it: the open function exported by the library didn’t work correctly and resulted in a segmentation fault, which basically means that system_uri wasn’t usable on macOS and because of that the front-end team had to invent some temporary workarounds. Apparently, the name of the open function itself clashed with some system functions on Mac, so just renaming that function did the trick and the bug is now fixed.

In the background, we’re also continuing our work on the bindings generator. It is progressing quite well and we’re almost done with the Java part. The major remaining thing to be done is the JNI glue code generator. It required some time to research and decide which way forward is the best as we have two options: either generate C++ JNI binding code (which is a standard practice in the Java world) or write this code in Rust (which is, by and large, experimental for now). Both of these options have their pros and cons, but ultimately we decided that keeping our language stack small and focussed outweighs the possible disadvantages of writing JNI code in Rust. Another benefit that we have with Rust JNI bindings is that it’s much easier to generate Rust code programmatically because we have full access to the relevant APIs of the Rust compiler (namely, the syntax parser and the pretty-printer). In other good news, the code generation framework is essentially complete and soon @adam will start working on the C# bindings generator. The current progress with it can be tracked in this repository.

Routing, Vault & Crust

We’ve been pushing ahead with the Data Chains specification this week, having chosen that proposal over the Section Graph one. We’re well on the way through the mid-level design; somewhere between the high-level overview of the protocols and the low-level programming tasks which will be finally produced. So far, this effort to produce more detailed plans has resulted in several lengthy design discussions involving the full design team. The majority of this work is complete now, and we’re hopeful that in the next few days the actual tasks can be specified so that we can begin turning this proposal into code.

The migration of Crust to the new Tokio + Futures architecture is complete. Compared to the previous iteration of Crust, the code is much smaller and more easily maintainable. This will make it much easier to implement the future improvements that are planned for Crust. Speaking of which, a first prototype of the NAT-traversal crate is now complete. Although further testing is needed before we can be confident in deploying it, it is ready to start being used in the development branch of Crust. In the coming weeks, with our new NAT-traversal and uTP crates, we should have a version of Crust which can also run entirely over UDP (in addition to TCP).

85 Likes

Sneaky little early update. :grinning:

Great news!

19 Likes

@MaidSafe Please check your links - editable-comments-plugin - link here is broken:

@srini has created an editable-comments-plugin example to demonstrate how versioned data can be handled using the APIs. @shankar has started to work with @shona in planning the developer website.

9 Likes

Thanks for letting us know, I just fixed the broken link :slight_smile:

14 Likes

Indeed it is, this is actually a huge thing. Much simpler code, no timers or durations but all event driven logic. It is very promising indeed. The measure twice approach should (pray) pay off for us here. Brilliant thing tho is we have multiple options to do the impossible network, and that feels great.

50 Likes

Thats an update all to itself.
Should be framed :+1:

Edit:

Im sure it is well worth the stresses of getting here!

24 Likes

Great update Maidsafe. You guys are rocking it!

13 Likes

Can’t wait to see the NAT library get merged into master! Wooot!!! Great news on Data Chains as well :smile: :sunny:

Also welcome @joy
The custom browser will end up being a very positive thing in the end so best of luck to @joshuef and kudos to @JPL for being an active community bug hunter. I’m such a fan boy of this project, I get a little excited :stuck_out_tongue:

14 Likes

@nicklambert will your talk at next weeks meetup be recorded?

7 Likes

Yes, he answered on another topic.

8 Likes

This the best update in a long time for me. What seemed like a mist of activity now has an easily discernible form. Feels almost like we’re wrapping things up after a long years work. Having put everything back in it’s place, looking at a completed architectural blueprint with pride right before turning off the lights and closing the door.

This is when the peices truly come together. I’m sure the team are walking with a pep in their step. Congrats on all the progress ladies and gentlemen. You’ve approached the last mile in the corridor leading to the gates of freedom. Champaign and cheers await… :slightly_smiling_face:

18 Likes

Just about ready to start writing code for Data Chains. How good is that. Bring it on

11 Likes

Awesome! :slight_smile:

9 Likes

Alpha 3 is coming soon , cheer’s

7 Likes

Top work chaps! Onwards and upwards as always :slight_smile:

8 Likes

Here’s another update with user stats:

TL;DR: We’re growing in a nice pace…

11 Likes

Depends how long it takes to specify the tasks, but they haven’t started that yet so I wouldn’t be expecting that writing code is about to start.

1 Like

Oh. I must have been taking the words too literally. " hopeful that in the next few days the tasks can be specified". A few days here and there is no big deal.

I hope it turns out to be so, but once the code is working, I’d think there could be quite a few tests running alongside Alpha 2 required to get data chains working well & thoroughly tested ahead of Alpha 3. This whole process could take quite a while, so whether it’s soon or not depends on your definition of ‘soon’.

Saying that, it’s very exciting progress, and it’ll be fantastic to hear about / see the first tests that show the concept is alive & functioning well.

Well done Maidsafe team, and keep up the amazing work!

9 Likes

What’s your best guess of “quite a while”. It definitely sounds like a lot longer than “soon”.