MaidSafe Dev Update - June 7, 2018


#1

Here are some of the main things to highlight this week:

  • If you missed it, the biggest piece this month was definitely the TechCrunch article last Saturday on PARSEC. It was a great in-depth piece following up on our launch of PARSEC a couple of weeks ago and definitely one to read and share if you haven’t already.
  • This week, we were able to build a new Peruse package which contains all the enhancements and refactored code we’ve been working on to improve performance and stability. We started our internal testing phase on it and we’re working on fixing the minor issues we encounter to be able to then share it with the community.
  • The release of safe_app 0.7.0, safe_authenticator 0.7.0, safe_core 0.30.0, and ffi_utils 0.6.0 should become available soon, with all features that have been added since these crates were published in February this year. The major change from the previous version is the licensing change from dual GPLv3/commercial to MIT/BSD for SAFE App and GPLv3 for other Client Libs. We’ve also implemented the bindings generator framework for SAFE Client Libs, with C headers, Java, and C# as our main targets.
  • Half the Routing team is working on implementing PARSEC. The tasks for “Milestone 1” (consensus in a static section) have been specified in Jira and the implementation is going smoothly in our new parsec GitHub repo.

Marketing

Let’s start this month as we always do with a review of the stats across various social platforms. @sarahpentland has written a summary of the month on Medium and we’re happy to say that everything continues to show steady overall growth:

If you missed it, the biggest piece this month was definitely the TechCrunch article last Saturday on PARSEC. It was a great in-depth piece following up on our launch of PARSEC a couple of weeks ago and definitely one to read and share if you haven’t already. As we said when at the release of the White Paper, we’re keen to engage with any projects that are facing similar challenges. It’s open source for precisely that reason, to enable people to kick the tyres so to speak and use it themselves so getting a large piece in the major tech press has been hugely valuable. We’d also like to say thanks to @Eureka18 who applied his/her design chops to spruce up the recent PARSEC slides.

Also, check out @happybeing’s recent work on the SAFE CLI Boilerplate, creating an easy way to make cross-platform command line SAFE Apps for Windows, Linux and macOS. And a shout out should also go to those who helped him with this (@bzee, @davidpbrown, @southside, @draw, @hunterlester).

User Experience & Website Design

The website development continues apace. The past week has seen a fusion of the visual style, information architecture, and content development of previous sprints to produce a final candidate homepage layout.

Resplendent in colour, crisp of fonts, orderly in typography, with a soupçon of motion; We’re pleased with where it’s headed, and when it’s time to pull the covers off we’re sure you will be too.

A single, flat static mockup, pored over and interrogated by the marketing team, red pen in hand, then became a multitude of adaptive layouts as it went through the meticulous process of designing for an infinite number of device sizes. So, you’ll be pleased to know, it will work just fine on the smallest of screens, right through to monitors a full parsec wide.

We’ve also invested some time in micro-interaction design, pattern developments, and quite a lot of brain power on messaging: crafting, re-working and then crafting some more. SAFE is a complex, multifaceted project with such broad-ranging implications that this task cannot be underestimated. Anyone who thinks it’s easy, either hasn’t given it a go or hasn’t watched Mad Men before trying.

We’ve also had to make some decisions on the technical underpinnings of the site. So, rope tied around his waist, we sent @shankar off into the depths of the internet to find just the right solution to match our deployment ambitions. He returned unscathed, clutching a progressive React static site generator that should fit the bill.

Wasting no time he got straight to work on implementation; while the rest of the design team moved on to laying out deeper pages to keep the whole machine cranking along, velocity set to ‘brisk’.

SAFE Browser, Authenticator & API

This week, we were able to build a new Peruse package which contains all the enhancements and refactored code we’ve been working on to improve performance and stability. We started our internal testing phase on it and we’re working on fixing the minor issues we encounter to be able to then share it with the community. We are also making all the last adjustments to finally merge all the Peruse code onto the MaidSafe safe_browser repository, under the peruse branch, in the next couple of days.

The support for multipart Range requests, which was a missing piece for the support of content streaming, has also been merged into Peruse.

On the safe_app_nodejs side, we continued with some internal refactoring and some enhancements we would like to have ready for our next version. One of the internal refactors we made was in our automated tests to reduce their execution time to 1/3.

A new feature was also implemented to optionally skip the download of the system_uri library, this is particularly useful when safe_app_nodejs is used as a dependency from a mobile application which may not need the system_uri utilities as it may use the mobile native libraries to send and receive the authorisation requests/responses. When installing the safe_app_nodejs package, setting NODE_ENV to either mobile_prod or mobile_dev skips the download of the system_uri library and disables the APIs normally exposed.

@ravinderjangra has been scoping the next set of tasks and also doing some clean up in the safe_mobile and safe_app_csharp repositories. The Rust scripts in the CI are being replaced with Cake scripts and he is also focusing on automating tasks using the Cake scripts. Also, the code formatting is also being reviewed and standardised.

@lionel.faber has been a lone warrior with the Java bindings. There are bugs in the JNI layer which are being addressed by @nbaksalyar. Meanwhile, @lionel.faber is trying to get the other aspects which are not dependent on the JNI layer progressing. He is looking into packaging and application integration by trying to build a sample app with the Java bindings. This will help us to identify the integration issues and make the process much easier for the developers.

SAFE Client Libs

This week we’ve defined the roadmap and plans for the next two releases of SAFE Client Libs.

The release of safe_app 0.7.0, safe_authenticator 0.7.0, safe_core 0.30.0, and ffi_utils 0.6.0 should become available soon, with all features that have been added since these crates were published in February this year. The major change from the previous version is the licensing change from dual GPLv3/commercial to MIT/BSD for SAFE App and GPLv3 for other Client Libs. We’ve also implemented the bindings generator framework for SAFE Client Libs, with C headers, Java, and C# as our main targets.

The next scheduled release, with safe_app 0.8.0, safe_authenticator 0.8.0, and safe_core 0.31.0 should include more. The main goal for this release is to simplify APIs and significantly improve the documentation for Rust developers. It should also include the improvement we’re aiming for in regard to minimising the usage of C dependencies, replacing rust_sodium with the new crypto library safe_crypto, described in the previous update. We’re done with our research for safe_crypto and we’re starting the implementation phase soon.

We’ve also completed the migration of all libraries to the latest stable Rust, 1.26.1. This allows us to use all the new features available in the newer compiler versions. However, during our upgrade process, the Rust team has released a new patch, fixing an issue in the borrow checker, so soon we’ll start updating our code to use this version with the bug fixes.

Routing

The Routing team has been working on two tasks in parallel:

Half the team is working on implementing PARSEC. The tasks for “Milestone 1” (consensus in a static section) have been specified in Jira and the implementation is going smoothly in our new parsec GitHub repo.

The other half has been working on the algorithm itself. We are pursuing two goals in parallel:

  • On one hand, we are solidifying the existing algorithm based on peer review we received, mostly from Twitter. We would like to thank @VladZamfir, @socrates1024, @emacbrough and @zmanian for the valuable insights they have been sharing with us. From their feedback, we have been looking at being more specific in our Liveness proof, as well as in our efficiency claims.
  • On the other hand, we are also looking ahead at the next milestone and giving some attention to the proofs for dynamic membership within a section.

We also have to thank https://crates.io/users/csherratt and https://crates.io/users/kvark who already had a “parsec” crate published on crates.io, and who kindly allowed us to take ownership of it. This has probably saved a few dozen dev-hours of bikeshedding about what to name our crate if we couldn’t use “parsec”!

Crust

Over the past couple of months, we’ve been spending a lot of time making tokio-utp more robust: fixing bugs, documenting code, writing tests, etc. This week, we finally implemented graceful connection shutdown. Now we can use connections correctly: when both peers decide to terminate a connection, a shutdown procedure is run where both peers exchange and acknowledge FIN packets - just like in TCP. Also, it’s worth mentioning that tokio-utp will soon be moved to the MaidSafe organization on GitHub: https://github.com/maidsafe.

Remember that in Crust everything is encrypted, even NAT traversal communication. So both our crates, Crust and p2p, provide encryption. We started implementing the planned encryption changes on the p2p crate first. Currently, it defines Rust traits for encryption which will probably be moved to https://github.com/maidsafe/safe_crypto in the near future. There’s also an implementation of those traits that uses rust_sodium as a backend. Finally, work on integrating encryption traits in Crust is already in progress. Although, if you want to use Crust with generic encryption traits, you probably want to wait until traits are moved to the safe_crypto repository. Otherwise, expect more changes in the future :slight_smile:

One issue we had with netsim was its unergonomic build process: it required libclang and bindgen which were used to build Rust FFI bindings for C types. Instead, we removed bindings generation and implemented them in Rust manually. As a result build process is simpler and doesn’t require libclang anymore :muscle:


#2

baduffba … :sunglasses:

ps:

so nice to see the libs maturing =)

impatient as i always am I am glad that progress is being made and am looking forward to the bright future of my favorite project :slight_smile:


#3

First yeah oh yeah ha ha


#4

Second or third 20 characters


#6

Great to hear that implementation is going smoothly. Great work as always team!


#7

So much good stuff going on here. Well done team! :wink:


#8

I love the writing for the Website Redesign section.


#9

Me too, was pretty comical :joy: great update!!!


#10

Keep up the good work! I have some plans for these java bindings when they are ready! :slight_smile:


#11

Agreed! It was my favourite part of the update and had me smiling! :slight_smile:


#12

This cracked me up. I can almost smell that yummy page now.


#13

While I like reading these updates, most of it goes over my head, but as an analogy from Jurassic Park goes:
“If it’s heavy it must be expensive”

Then this holds true to me, that:
“if it’s complicated it must be damn good!”


#14

This update was @SarahPentland 's doing. I believe I can detect her enthusiasm, creative spirit, and her thoroughness a mile away.

Am I right?

I can’t wait to hear about the refinement of PARSEC. This is the stuff I stick around for!!! :laughing:


#15

I saw what you did here! Is super safe! Whole parsec safe! :smiley:


#16

The author clearly enjoys cooking and elaborate food :slight_smile: I can see a vegan salad here, with some fried onions.

Thanks for the neat update guys, seems the engine is in great condition :wink:


#17

Love you guys. Keep up the hard work! (From a caravan in lake District!)


#18

Very well assembled. I can see the progress just by the tone of these updates. Adding the flair will only benefit the audience.

One suggestion might be to refer to these as simply “Updates” or similar and drop dev. There has been a marked departure from “dev” and a refined approach could appeal to the broader audience.

Well done ladies and gents.


#19

If it’s an issue with a crate in the future, I would suggest following the current naming convention [as I understand it] and prepending “safe_” to the desired-yet-unavailable identifier.

Could the design of QUIC [0] inform the work on tokio-utp?

[0] https://www.youtube.com/watch?v=EHgyY5DNdvI


#20

I’m not sure if that is a coincidence or on purpose. But if Parsec would still be a problem, you have Spacer :wink:


#21

Another fantastic update! I love reading these every week; they’re so marvelously written.

Sounds like the preamble to a marvelous dish! Can’t wait to see the new website :yum::smile: