SAFE Network Dev Update - November 29, 2018

Summary

Here are some of the main things to highlight this week:

  • @fergish interviewed @joshuef and @bochaco on the RDF/Solid connection with SAFE.
  • A new version of the safe_app_nodejs package (v0.10.0) has been published this week on npm, which incorporates some enhancements mainly around the experimental APIs. Please refer to the changelog to see the complete list of changes.
  • The SAFE Client Libs team has been progressing well with the research on adding RDF/SPARQL support to SAFE Client Libs. Their progress can be tracked through the GitHub issues.
  • Malice detection, testing and routing integration continue to be the main focus of the Routing team.

Marketing

Big news this week that you might have heard about - with the SAFE Network logo appearing in the new #1 US film hit, ā€˜Ralph Breaks the Internetā€™! Props to @nigel who first spotted it and @Harvindar who got some photo evidence. Last night the story started to be picked up by the press. Weā€™d love to find out the connection so if anyone knows, please give us a shout!

Itā€™s been a packed week for getting out and about:

The rest of the week for marketing has been focused on work around messaging (more on that to come soon), producing content around Dynamic Membership in Parsec (ditto) and content strategy for the new newsletter subscription list (reminder: you can sign up in the footer at https://safenetwork.tech/). Weā€™ve also been working at improving the video work for the weekly updates. Real progress this week and you can take a look at the improvements weā€™ve made by checking out this unlisted video if youā€™re interested. We have a few further things weā€™re planning to do over the next couple of weeks to raise the quality even further but hopefully youā€™ll agree that the production values are improving. Weā€™re keen to invest more in video updates moving forwards of various types - so the foundations weā€™re laying here should stand us all in good stead.

Finally, one quick reminder: SAFE Network: Brighton meetup is next Wednesday where @nbaksalyar will be running through Crust - and you can still sign up here.

SAFE API & Apps

The safe_app_java repository now includes the code comments from which the Javadocs will be generated and published. safe_app_csharp was updated to support x86_64 for Android along with the API being updated to safe_app v0.9.0. Improvements were made in handling error messages on the Mobile Authenticator. With the API stabilized and the tutorials falling in place, we are currently standardizing the project management processes, before we start planning the final testing and release process for the C# and Java APIs.

A new version of the safe_app_nodejs package (v0.10.0) has been published this week on npm, which incorporates some enhancements mainly around the experimental APIs: they are all now exposed behind a flag so developers and users can use them at an early stage, while being explicitly aware of the fact that they are still in an experimental stage. Please refer to the changelog to see the complete list of changes. The following is a summary of the highlights of this release:

  • Upgrade safe_app lib to latest v0.9.0.
  • Every experimental API is exposed behind a flag/option.
  • XOR-URLs can be obtained for any ImmutableData and MutableData on the network.
  • Experimental RDF API is also exposed which allows applications to create and manipulate data in RDF format.
  • Experimental WebID API which provides an abstraction for creating/updating WebIDs on the SAFE Network, even allowing the application to publish them with a publicName ā€œsafe://ā€-URL.
  • Fetch data using publicName-URLs or XOR-URLs. This functionality is supported by the ā€˜webFetchā€™ and ā€˜fetchā€™ functions exposed.

We have also upgraded safe_app_nodejs to v0.10.0 and safe_authenticator lib to v0.9.0 in the SAFE Browser, which enables all the experimental APIs in the DOM API to be exposed. Weā€™re working on allowing users to have the control to enable/disable them for the use of web apps. We are now trying to finalise this functionality and doing a lot of exploratory testing to be able to release it soon.

Since the new version of safe_client_libs (v0.9.0) makes use of a new type of encoding (base32 as opposed to base64) for the authorisation requests/responses, we will therefore also be releasing a new version of our Web Hosting Manager application which will be compatible with the new SAFE Browser being released.

There has been a lot of work also going on around the automation of tests for the SAFE Browser, which will help us a lot to assure that changes made donā€™t affect the existing functionality. We are planning to keep working on enhancing this in the next few weeks, this is one of the main enhancements weā€™ve been working on in the last months on the browser. As part of this effort, we are trying to move away from AppVeyor onto using Travis for all platforms (Windows, Linux and macOS).

SAFE Client Libs

We have been progressing well with the research on adding RDF/SPARQL support to SAFE Client Libs. Weā€™ve started to work on Rust bindings for the open RDF C libraries, librasqal and librdf, which we want to reuse for some of the basic functions such as query parsing.

While using Rust libraries is always a better choice because of better security guarantees, we havenā€™t found libraries implementing the RDF standards completely, and at this point it is not feasible for us to produce them ourselves. Instead, we decided to focus on implementing basic support for SPARQL queries and triple storage on the SAFE Network as quickly as possible. Then we can start testing RDF-based apps and the ideas from RFCs proposing an RDF-based replacement for DNS. And with simple demo apps, we can iterate on these features further, providing better performance, having better API design, and more complete and seamless integration with the SAFE Network.

Our progress can be tracked through the GitHub issues.

Routing

Malice detection, testing and routing integration continue to be the main focus of the Routing team. We also had multiple interactions with the SAFE Network community and the Rust community.

On the routing-integration side, we implemented two new small API changes in Parsec to facilitate the use of this crate by routing. We are also currently looking at a third one.
One simplified the configuration of the input to the consensus protocol. Instead of giving routing the full configurability of a custom function, we now only expose a choice between a couple of options. Like any API improvement, it makes our interface harder to misuse while maintaining a sufficient amount of control over the crateā€™s behaviour.
Another one added a method to indicate which peers are ready to receive gossip communication from a given nodeā€™s perspective. Prior to the change, routing would just try any node it knew about until one of the communications succeeded. This approach is simpler and more expressive. It also help us make our tests more robust as we can now verify that nodes that should be able to process our communications.

Finally, we are working on a modification that allows Parsec to inform Routing when new nodes become aware of previous consensus decisions. It is not information that is essential to expose in the strict definition of a consensus algorithm, but we noticed that it would make the use of Parsec by Routing a little bit easier.

Overall, it is fair to say that we are quite excited with the progress on the Routing front, even though we are aware of more opportunities for improving the performance that might well prove necessary before we can release the network to the world :smiley:

We kept looking at various forms of Malice and systematically improving Parsec by making it able to deter nodes from engaging in these malicious activities. We implemented detection of Spam malice. Thatā€™s when a node deliberately sends us more information than they know we need to incur extra-work for us. We also have a number of changes that should make it into master in the coming days.

We also continued to improve the testing framework and expect to merge these changes soon too.

From the community side, we were happy to merge a pull request from @u-amine on GitHub that updated our tutorial and Readme since the way we run tests and the example have changed since the tutorial was written.
@pierrechevalier83, @jonhaggblad, Andreas and @povilasb got to mingle with the Rust community by attending RustFest in Rome over the weekend. It was a great motivation boost and an instructive experience.

@pierrechevalier83 also did a presentation about the synergy between the Rust programming language and the MaidSafe values first to an audience of Rust programmers on Tuesday at the Rust London User Group Meetup, then to a SAFE Network enthusiasts audience on Wednesday at the SAFE Network London Meetup. The slides from the talk can be found on GitHub. Both talks were filmed and the videos should be available shortly.

Crust

@povilasb took 3 days off to take a trip to Rome to visit the RustFest conference and exchange ideas with fellow Rustaceans. Now back, heā€™s been solely working on the development of socket-collection. We finally merged the encryption changes. Now different crypto contexts can be passed with socket config structure when constructing UDP or TCP sockets. Also, socket-collection now encrypts the packet header as well, which is just the packet size currently. Now at this point, all the data on the wire will seem absolutely random to the observer. We also added more UDP tests and expanded the docs. Eventually, we introduced the socket trait which will abstract TCP and UDP sockets and reduce the duplicate code.

78 Likes

First :smile: Muhahahahhaha

32 Likes

Nice update chaps! It sounds like routing and parsec are coming togethet nicely!

16 Likes

Iā€™m very glad I can be one of the first to congratulate you for the great job! Go, go, go Team MaidSafe! :smiley: :rocket: :smiley:

14 Likes

This is great stuff! This will help devs keep up to date and on the bleeding edge. Hopefully will help understand and plan out refactors too.

Excellent progress in Routing and Crust! Get well soon @nbaksalyar.

18 Likes

sixth - a perfect sixth :slight_smile:

11 Likes

Good, cos I havenā€™t been able to get anywhere with WHM for days and that was making testing SAFE Drive a bit difficult.
SAFE Drive is an essential part of the whole SAFE caboodle so I urge you all to help @happybeing help us all .
Check out this thread on the dev forum

11 Likes

@Southside, just to be clear, this should affect you only if you were using the latest version of safe_app_nodejs or building the browser yourself form our repo, which I assume you are not.

If you are using Peruse v0.7.0 and WHM v0.5.0 you shouldnā€™t have any issues at all.

12 Likes

great job everyone !

8 Likes

Thanks so much to the whole Maidsafe team for all of your hard work.

8 Likes

Well done you guys and gals. I sense 2019 is going to be a Happy New Year with even more intensity and focus on the network. Work hard for the next 24 days and David may even give some of you Christmas Day off.

8 Likes

No need to tell them to work hard. Their knuckles are already raw enough. Working hard for other people is their equivalent of taking it easy. :wink:

Itā€™s like dragging a boulder up a hill. You squeeze every ounce of energy out to make it there. Frothing at the mouth knowing that every step taken is one closer to a respite you await with great alacrity.

Stay strong and passionate @maidsafe . It wont be too long before you plant your flag, rest, and drink a crisp cool glass of reinvigorating water before you set off on the next adventure. Weā€™re all here cheering for you and will continue to provide any boost we can muster with the little we have.

When you get there look around you before you peer off into the landscape. For those in your immediate circle who have remained throughout the journey have proven to be robust.

Shout cheers with pride! :beers: Giddy inebriation is but one of the many rewards deserved and made much more enjoyable with an extended family. :v: :crazy_face:

:facepunch: :sunglasses:

11 Likes

Iā€™m just wondering how far the progress bar has moved towards beta and launch?

Iā€™m sorry Iā€™m more pragmatic and not fanboying it up as much.

7 Likes

4.78 inches :wink:

More seriously though, the roadmap to alpha 3 has 3 milestones, most of which the updates show as nearing completion. The front end side with RDF/SOLID is making very good progress as well as mobile and generalised app APIs and languages. The marketing and admin teams are almost at full strength. There is not much in terms of recruitment that is required and progress now should be obvious in the updates, now also video updates.

The alpha 3 release is a big deal and mostly invisible to the more ordinary/less technical person and that is where you may be thinking itā€™s not clear.

45 Likes

Incremental progress across all areas. I buzz inside knowing there is momentum building on the RDF/Solid workstreams. I shall listen to the crossroad podcast this weekend as the chaos kidlets are sound asleep.

Keep pushin through @maidsafe team!

11 Likes

4.78 out of 5? :joy:

10 Likes

Once again, seriously loving the tone here SAFE. Gettinā€™ 'er done.

4 Likes

Another amazing update team!

MAIDSAFE is literally a stones throw away from alpha 3 in terms of development. Alpha 3 is more imporatnt than alpha 4. I believe alpha 3 de-risks most of the tech for the network. Weā€™re getting close. :clinking_glasses:

11 Likes

I donā€™t wanna be an ass but you are using ā€œliterallyā€ wrong :stuck_out_tongue:
It is figuratively a stoneā€™s throw away, unless you are standing in front of MaidSafeā€™s office holding a stone in your hand.

2 Likes

To be fair and I only looked this up because I use it as Micheal did often.

_

//a word that can be used both literally and figuratively

_

3 Likes