SAFE Network Dev Update - January 31, 2019

Summary

Here are some of the main things to highlight this week:

  • We released an updated C# API and new tutorials for .NET mobile and desktop development on the SAFE Network. See this post for more details.
  • The Routing team released a first forum post to start shedding some light on the work that they are doing to prepare for the Fleming milestone.
  • A few MaidSafe team members will be at FOSDEM this weekend. See this forum post for more info.

Marketing

It was fantastic to see Devolution, a new DApp developed by @Glen_Simister, pop up on the Forum out of nowhere this week! If you want to see his vision of how an open-source application can be used to build communities with political autonomy, you can read more about it in this forum topic.

This afternoon, @fergish has been recording a lengthy new SAFE Crossroads podcast with both @dirvine and @viv. It should be published (at least the first part) early tomorrow morning (UK time) so please keep an eye open, have a listen and feel free to share it widely!

We also supported the C# release this week, together with another monthly summary of SAFE Network goings-on that you should see published within the next couple of days. Video content has also consumed a fair amount of time this week. For example, after a very successful SAFE Browser hangout last Friday afternoon, the video needed a little creative editing to get around the initial technical difficulties we had - but the call itself was really successful, with lots of great questions from the participants. Weā€™re hoping to work with the Devs to get more of these calls on a more regular basis moving forwards so please shout if there are any topics in particular youā€™d like us to get arranged.

We all enjoyed the Data Privacy Day banter online earlier this week - and interesting to see many people hoping that regulation will solve the problems of a broken internetā€¦ Also, @dugcampbell has just been confirmed to speak at the Landing Festival (3-4th April 2019), which will have over 1,500 attendees. Weā€™re considering maybe running a SAFE Meetup in Berlin the night after the festival - so give us a shout if youā€™re in the vicinity and it might be something youā€™d be up for coming along to!

SAFE API & Apps

This week we released an updated safe_app_csharp API and API documentation for .NET mobile and desktop development. We also released example apps for desktop (Windows) and mobile (Android, iOS). New tutorials have been added to the DevHub to help developers get started with development on desktop and mobile. Check out this forum post for more details.

Scoping continues for the next milestone for safe_app_java. The team has put together a couple of PoCs that will improve our test suite and CI setup. In parallel, we are also putting together some documentation for the project structure to help developers find their way around the code and start contributing.

Browser wise weā€™ve been looking at where we can further stabilise things, which has seen us taking a closer look at tabbing in the application, and what we can simplify therein. Weā€™re also trying to get the browser running against newer versions of Electron itself, which while not being as simple as one would hope, weā€™re making progress on.

Similarly, weā€™re scoping out a new release of the FFI library in safe_app_nodejs which should provide compatibility for newer Node versions (hello 10.x!), as well as being a precursor for the aforementioned Electron updates.

SAFE Client Libs

Weā€™ve been continuing to work on the Proof of Concept for storing RDF triples in network Mutable Data. This week was spent working out the serialisation internals of librdf (the RDF implementation we are building on top of). Something in our implementation was (and is) causing this step to fail and our priority is to fix this as soon as possible. The next step will be to reuse RDFā€™s triple hashing so that we can more efficiently store triples as key/value pairs in Mutable Data (the scheme we currently have in the POC is a placeholder for this). Progress on this POC can be followed here. In the meantime, the RFC for bringing RDF to SAFE Client Libs is getting closer to completion. Weā€™re going to publish it soon and start the discussion process.

Recently @yogesh joined our team and he started with helping us with some outstanding tasks. The first one is switching SAFE Bindgen to another parsing library. This is required because recently weā€™ve switched to always using the latest available stable Rust compiler, and the parsing library we currently use in Bindgen is deprecated and not supported anymore. Consequently, it does not recognise some of the new syntax constructs that appeared in the recent Rust versions, and it fails to parse the SAFE Client Libs code using this syntax. Getting this work done will allow us to completely switch Client Libs and the rest of our repositories to the latest Rust version.

In parallel, weā€™re continuing to document the core concepts of SAFE Client Libs and scoping out the remaining parts of the libraries to cover. Weā€™re considering this an important milestone and weā€™re aiming to make this documentation public soon.

Routing

The Routing team continues sharing effort between Fleming preparations and the implementation of Parsec.

One aspect of our work that overlaps between both consists of addressing the scalability of Parsec with various section sizes. As noticed here by some astute community members, we have continued to make serious progress on this front as we merged a change that drastically reduces the number of calls to the function is_interesting_payload and improves performance by > 50% for larger section sizes.
As we can now easily measure the scalability with section size, we can see that more work is still needed here. We are already following interesting leads to get even further improvements. For instance, we used Callgrind to establish that the function meta_votes_since_round_and_step is called much more often than expected. This is really good as it gives us an opportunity to keep improving the code.

Since last week, we continued troubleshooting soak testing failures. We fixed a problem with the test framework that was causing some of these failures. This allowed us to identify another one, apparently coming from a bug in the production code. At first glance, it looks like a peer reaches consensus on the same block twice. We are still tracking that particular bug down.

We are also continuing to investigate a performance issue related to ā€œaccompliceā€ malice handling.

We are also continuing to prepare for the Fleming milestone with design discussions.

Yesterday, we released a first forum post to start shedding some light on the work that we are doing in this area. This article sets the scene. It will be followed by more articles that will dive into specific technical aspects of our work as we move forward.

Finally, we continued running simulations to estimate the impact of various Routing design parameters such as the section size, the number of elders per section and such on the security of the Network. We are refining our conclusions and will continue in that direction by varying a number of parameters in our simulations to get a more complete picture. This work is still progressing. We expect to communicate more details about it when it allows us to crystallize certain specific aspects of the design.

Crust

Finally together with Lukaā€™s help we did some bootstrap cache improvements. Now Crust will try to maintain the list of the most active 200 publicly reachable peers and use those to bootstrap off.

Weā€™ve been spending a great amount of time refining our nearest roadmap. We are defining more concrete milestones and prioritizing them to make the Fleming release happen sooner. Some of the things that need to be taken into consideration are node restarts, network upgrades, connection limits, etc.

87 Likes

First I guess :smiley:

14 Likes

Thanks so much for all of the hard work that the team puts in!

11 Likes

Great work again. Looking forwards to the next set of fleming articles. Also looking forwards to the routing simulation results.

14 Likes

Any way you can share some of the scalability per section size numbers? Itā€™d be great to have a sense of how the early parsec-based consensus scales compared to others. A potential issue with keeping the numbers fully under wrap is that it can give the sense that the numbers do not compare favorably with the competition, even if early. But I suspect that even these non-optimized numbers are likely better. But I could be wrong.

8 Likes

Getting these two on together can only mean exciting times are at our doorstep. Canā€™t wait to listen.

21 Likes

I have never listened to a SAFE crossroads podcast. I see that it is available as an actual podcast, so I have just subscribed in DogCatcher, my android podcast app.

But, to be honest, when Iā€™m on the road, Iā€™m listening to a book, and when Iā€™m home, Iā€™m on Youtube.

Thatā€™s my long winded way of asking @fergish, to please consider putting his stuff on Youtube in video format. I think there are a lot of people like me that are not listening to his content, just because we have become spoiled by video and video has become our main means of educating ourselves when home.

But I will definitely try to listen to the David/Viv one.

4 Likes

SoundCloud is really easy to listen from home or in the car. Better still, it plays in the background on your android phone, unlike YouTube.

12 Likes

Then no one can interview naked! Seems like a big ask to meā€¦ :face_with_raised_eyebrow:

19 Likes

I kind of agree to a certain extent. I do listen to the odd podcast from letā€™s talk bitcoin, but I do much prefer a video style like the Joe Rogan podcasts.

2 Likes

Great updateā€”really looking forward to @fergishā€™s conversation with @dirvine and @viv!

8 Likes

Somebody turned me on to NewPipe, which is a 3rd party Youtube player that allows playing in background. It is available on FDroid.

3 Likes

I donā€™t think that follows.

You Americans are so conservative.

14 Likes

Great update team! :clap:

Quick question. In what order will features of fleming be implemented? I imagine network upgrades will take priority over the others considering that all other components (i.e. node restarts, etc) could then be deployed without inturrupting subsequent testnets too much.

3 Likes

They could, but the number of viewers might drop

Our TV did not censor a cricketer being fully dacked during an interview and barring all waist down.

Anyhow I find the podcasts great and allows me to be doing other less meaningful tasks while listening (eg housework)

4 Likes

Thanks, @thc.

I donā€™t usually do video for a few reasons, but all current Crossroads podcasts post to youtube automatically with a logo screen at this channel link:

https://www.youtube.com/channel/UCTDgrcxepLgaREjpwjpBWvg

But as @Nigel says, Soundcloud is pretty easy to use as well, with feeds from both SAFE Crossroads and Technology & Choice:

https://soundcloud.com/safecrossroads/safe-crossroads-51-safe

13 Likes

This is good news. Great work team!

6 Likes

Welcome @yogesh!

I love that the team geeks out over unexpected optimizations :slightly_smiling_face: Iā€™m still anxiously awaiting the TPS tests after performance is up to Maidsafeā€™s standards!

6 Likes

When i read that i felt weā€™re close. Todayā€™s update i get the sense its still farā€¦too technical for me :rofl:

3 Likes

Its a bit like building a high rise building where there are many floors and as they get higher the lower floors can start having the cladding attached and then the windows. If you just hear about the cladding and windows then it sounds near completion, but if you look further then you see the higher floors still being concreted etc with some floors not even there yet. And of course then there is the fitting out inside still to be done.

Well SAFE sounds like its well into the process and some of the components are near completion while some are still being ā€œconcretedā€ (PARSEC) and some still to do (SAFEcoin)

12 Likes