SAFE Network Dev Update - November 22, 2018

Summary

Here are some of the main things to highlight this week:

  • MaidSafeCoin will soon be on BitUniverse’s Link service. With 80% of their users based in Asia, this should hit a very different demographic from the other services that we’ve recently started using.
  • We want to draw your attention to the amazing work that @happybeing is doing on his SAFE Drive project. Please go read the post, try to understand what he’s working on and if at all possible give him a hand!
  • The safe_app_java library has been updated to work with the latest version of SAFE Client Libs. This is a significant milestone for the Java library as v0.9.0 of Client Libs includes fixes to the issues we have been facing in the JNI layer.
  • The safe_app_csharp library has also been updated to work with v0.9.0 of SAFE Client Libs. Pull requests have been raised in the safe-authenticator-mobile and safe_app_csharp repositories.
  • The safe_app native lib was also upgraded to v0.9.0 in the safe_app_nodejs package and the last PR that is planned to be included in the next release of this package is under review and testing at the moment.
  • The SAFE Client Libs team has been focused on understanding which RDF capabilities we need to include in the SAFE Network core libraries to better support integration with Solid and the W3C standards.

Marketing

The push to share updates and content across a much wider range of distribution channels continues. We’ll also now be sharing our updates on BitUniverse from the launch of its Link service. The BitUniverse app is available on either iOS or Android and with 80% of their users based in Asia, this should hit a very different demographic from the other services that we’ve recently started using. The service launches on November 24th.

We’ve been spending time this week working on the distribution strategy for a variety of messages/formats/platforms, together with planning for the next piece of more substantive pieces of content around the road to SAFE-Fleming. More on this very soon. We’ve also just got our Steemit account approved (@thesafenetwork). We’re slightly stretched in terms of resources for publishing original content but we plan to experiment here a little as well over the next few weeks.

Out and about this week: @dugcampbell was at BBC Radio Scotland on Wednesday morning giving a few thoughts in the business section of the ‘Good Morning Scotland’ show and he’s also on a panel at Edinburgh Uni with three other crypto businesses on Monday 26th November, when he’ll be giving a quick 15 minute intro to the SAFE Network before debating the future of cryptocurrencies.

We’re happy to announce that @opacey has secured a venue for the London Meetup next Wednesday (28th November) at CampFire Co-working. Sign up here if you’d like to hear @nbaksalyar give a talk on Crust and discuss the recent Crust test results.

Finally, we want to draw your attention to the amazing work that @happybeing is doing on his SAFE Drive project. Not content with leading the way by collaborating with TBL’s Solid project, he’s also been working hard at building a framework that means that the vision of say a decentralised Dropbox and a decentralised GitHub on the SAFE Network are getting closer by the day. Following his recent breakthrough, this is huge news and within the team, we’re all continually amazed by the work he’s putting in. So we’ll end with a plea to everyone reading this: please go read the post, try to understand what he’s working on and if at all possible give him a hand!

Recruitment

Our new starts are all settling in well but we still have a couple of weeks yet until we can welcome our other new recruits to the company.

As for the Network Engineer role, we are still in the recruitment process for this role. We knew this would be a tough one to fill and we will get there in the end.

SAFE API & Apps

The safe_app_java library has been updated to work with the latest version of SAFE Client Libs. This is a significant milestone for the Java library as v0.9.0 of Client Libs includes fixes to the issues we have been facing in the JNI layer. We have also raised a pull request which includes code comments and automation for the generation and publishing of API documentation.

The safe_app_csharp library has also been updated to work with v0.9.0 of SAFE Client Libs. Pull requests have been raised in the safe-authenticator-mobile and safe_app_csharp repositories. As you might be aware, we have already deprecated support for x86 devices. However to improve developer experience we have decided to add support for devices with the x86_64 architecture.

The safe_app native lib was also upgraded to v0.9.0 in the safe_app_nodejs package and the last PR that is planned to be included in the next release of this package is under review and testing at the moment. We are hoping to be able to publish it in the next few days with the list of changes and enhancements made to it. The main feature for this release is the addition of all the RDF/WebIDs utilities exposed as experimental APIs.

Several fixes and functional enhancements have been implemented and already merged into the SAFE Browser repository, and we are trying to finalise the implementation of a switch in the UI which would allow users to enable the experimental APIs in the DOM as well as any experimental features we may want to expose in the browser like the existing WebID selector. We are planning to upgrade safe_app_nodejs as soon as the new version is published in the next few days, and this will lead us to a phase of exploratory testing in preparation for the next release of the browser. The safe_authenticator native library will also be upgraded in the browser to v0.9.0 in the next few days.

SAFE Client Libs

After the latest Client Libs release, we have been focused on understanding which RDF capabilities we need to include in the SAFE Network core libraries to better support integration with Solid and the W3C standards. We have started with defining the scope of the future work and we decided to base it on the mock version of SAFE Client Libs. This approach will allow us to try multiple ideas without any need to touch Vaults or Routing code, and the tests will be entirely self-contained.

Our primary goal for now is the research that should answer two important questions:

  1. How to store the RDF triples on the SAFE Network efficiently? There are many known ways of doing this (and the front-end team already used one of them in the safe_app_nodejs RDF emulation layer), but we’ll need to find an approach that works best while meeting the network fundamentals. This involves storing and indexing RDF triples in Mutable Data, and most importantly, storing triples in the encrypted form.
  2. How to implement the SPARQL query language for the SAFE Network? The Vaults will need to have some form of support for the query protocol, but it is defined with an assumption that the data is public and anyone can query it. This is not always the case on the SAFE Network, and we want to query our data without transferring any encryption keys to Vaults and compromising our security and privacy.

We already have some good ideas on how to answers these questions. We’ll be trying different ways of solving the problems and we’ll keep sharing our findings with the SAFE and Solid communities.

Routing

This week, we continued with our work on four main fronts: extending the test suite to more thoroughly test dynamic membership, performance improvements, Routing integration, and malice handling.

The work on the test suite update has been progressing steadily, and this fairly significant update should be merged to master hopefully by the end of the week. This will allow some further test tasks to be started too, since we deliberately held back on these to avoid creating extra work in needing to update new tests after this work was completed.

We completed the optimisation work as far as the easily-identifiable parts are concerned. These did, as expected, allow for the test suite to run significantly quicker, and also makes Parsec more readily usable by Routing. We also identified an issue with our proptest setup whereby it was consuming huge amounts of memory and being killed occasionally. This fix has allowed us to re-enable the proptest code, increasing the effectiveness of our soak tests once more.

Having spent some time improving benchmark tests and examining their results as part of the optimisation push, we now have a very basic process in place to help us avoid regressing badly on performance as we create new code. This setup will be given some further work in the future to make it more automated and robust.

Work has also progressed on integrating Parsec into Routing. We continue to find a wee bit of friction there at the API, and are tweaking things to suit Routing requirements while maintaining as generic an API as possible. So far, issues have been limited to missing functionality exposed by Parsec, and have been fairly trivial to address. There will doubtless be some more give and take over the coming weeks as Routing and Parsec become more closely integrated, but so far, this appears to be going well.

Finally, we have been continuing to pick off tasks which address malice detection and handling. These haven’t yielded any real surprises or problems, and we’re well on the way to completing the final few now.

Crust

We started off this week with implementing encryption in our sockets library - socket-collection. The library has to be flexible enough to fit p2p and Crust needs. Thus before merging the encryption PR, we attempted to integrate it into the p2p crate. To make sure the changes work as expected, we wrote some more tests. Then after experimenting with socket-collection and p2p, we gathered valuable insights to refine our crypto API in socket-collection.

78 Likes

First for the first time :slight_smile:

33 Likes

Insider trading maidsafe style :smiley: :smiley:

30 Likes

Very powerful update! Thanks @maidsafe Team for the well done job! :slight_smile:

22 Likes

Great to see these pieces moving forward and HUGE shout out to @happybeing on his Safe Drive!!

22 Likes

Great progress - it sounds like routing and PARSEC are coming together well!

Keep up the great work @happybeing too!

20 Likes

Love this part! Thank you @maidsafe team and @happybeing
Keep up the great work!

21 Likes

Thanks so much for all of your hard work! The efforts of the Maidsafe team are one of many reasons I have to be thankful today!

It’s great you got a Steemit account for Maidsafe! Don’t forget about www.minds.com, they are also an up-and-coming forum!

14 Likes

This is a very encouraging update, and I don’t mean because I’m mentioned :smile: – although thank you very much Maidsafe for highlighting my bit – but because it shows significant progress and upcoming releases in so many areas. This is brilliant.

Coming to SAFE Drive, thank you everyone for your encouragement and help. Quite a few people have now jumped in to help with testing, porting to Windows and MacOS etc, so it really has been much more than just me. Special mention to @JPL who has given a lot of time over the last few weeks, and to @isntism and @draw who got stuck in in no time and have already helped spot some bugs. But to all who have had a go, you know who you are. Things are going amazingly well, and there are plenty of things to do, and which people can help with.

Thanks for the messages of support. Public and private, these help a lot :slight_smile:

Although I am now worried in case I can’t deliver. :fearful:

But I have sorted a few more bugs even today, and the regression tests are passing [phew] which is amazing.

SAFE Drive for a decentralised git and github

What I’m keen to find time to do is to check out the feasibility of a decentralised git (fairly straightforward) and a decentralised github (more involved) because it means setting up a web UI for git repos using something open source, such as gitea or sr.ht. If anyone fancies having a go with those, or any other web UI, please head over and reply to this post on the dev forum.

SAFE Drive for decentralised Dropbox

This is ready to go for anyone who wants to play with rsync and cron. If somebody could create a script based UI or something to make that easy for others, or just to test it more rigorously, please jump in and reply to this dev forum post.

SAFE Drive help with testing

If you would like to try SAFE Drive out and do a bit of testing – and even trying to set it up is valuable testing – myself and others will help you get going and be very happy for any time you put into this, however small. By now I think there are a few people who can help others get SAFE Drive working and to do some tests, so if you would like to help, see the instructions on this dev forum post.

45 Likes

David Irvine, unfornately I am like your grandmother…I will have to wait till safenetwork is as easy as swiping a finger on the screen … for me to use it! . I have no knowledge whatsoever on programming! The easy part I am able to do like downloading the browser and stuff and trying the links availabe… ! I logged on to the crust test… that was awsome!

14 Likes

I just added a note on the future for SAFE Drive, see:

10 Likes

Great update Maidsafe devs,

@happybeing super to see SAFE Network projects “code first”, instead of asking for money. A Github substitute on the SAFE Network is a guaranteed success, keep up the good work sir. :+1: :+1: btw I wanted to help, unfortunately it just not that easy to get it started on my computer, what always help is an step by step instruction vid. I know that your super busy, but are you also thinking about creating a course for this as you go along? It would help other devs understand it how to get started, there are just not enough SAFE courses, jpl’s was extremely good.
:stuck_out_tongue:

12 Likes

I cannot stop being excited by this. This alone will bring hoards of people willing to buy SAFEcoin to subscribe to the service.
Maybe early SAFEtestnet github should have the current SAFE source code uploaded. A network that hosts its own code as well as running on it.

6 Likes

Thanks @19eddyjohn75, I appreciate your support and the suggestion. Actually, a getting started video like this might be something somebody else could do to help. I could do it of course, but just mention the opportunity.

Mean time myself and others can help anyone get going.

That helps because it encourages to streamline and improve the written instructions, and to think about doing a video. :wink:

On reflection, a reason to favour written rather than video instructions atm is that while I know some people find them easier, things are always changing at this stage and it is much harder to update video than write a new reply to a post. Or to edit the README.md, which I’m actually already quite bad at updating - so that’s something else could do with some love!

7 Likes

Me too. I had been thinking about this for ages before I realised I was working on something (SAFE Drive) that would deliver this in a useful form. Dur!

I’m working through some bugs that git is helping me find while using SAFE Drive to host repositories as I write (yes I have a brain implant that connects me directly to my laptop while I type on this tablet :wink:).

What would help while I’m fixing that is for one or two brave souls to be working out how to set up Gitea and/or sr.ht so we can also try out using those as a Web UI like github. If anyone has experience setting up a hosted package (locally on your own machine) and is interested, see: here

10 Likes

:scream: :scream_cat: Oooh no they got @happybeing also # :zipper_mouth_face:

Please also realize that this and SAFE press are #github.io
https://www.youtube.com/watch?time_continue=60&v=2MsN8gpT6jY
but on the SAFE network
:stuck_out_tongue:

4 Likes

Yesterday was Thanksgiving Day over here, and I’m certainly thankful for all the awesome work the MaidSafe crew continues to deliver! Very excited to hear the SAFE good word will be spreading even further with BitUniverse, and I’ll be sure to point people with technical chops in SAFE Drive’s direction too. :smile:

12 Likes