SAFE Network Dev Update - August 9, 2018

Here are some of the main things to highlight this week:

  • This week, we’ve published the monthly summary of July activities on the SAFE Network.
  • Thanks to @opacey, we now have a London Meetup booked in for Wednesday 29th August. It will be held at ThinkRise in Shoreditch (https://thinkrise.com/) and will have @dugcampbell giving an intro to the SAFE Network for the newcomers before @pierrechevalier83 talks about PARSEC. The link will go live in the next day so if you haven’t signed up yet, please join the meetup group (https://www.meetup.com/SAFE-Network-London/) to get an invite direct to your inbox.
  • Last week we released and published a new version of the safe-node-app package (v0.9.0) which incorporates several changes, enhancements and some bug fixes, including the changes required to upgrade the safe_app native lib to v0.8.0.
  • Also, note we updated the code in our Electron quickstart app tutorial as well as in the tutorial published on our DevHub website so it can be used with the latest safe-node-app package.
  • @dugcampbell, @bart and @pierrechevalier83 will gather in London on Tuesday the 11th of September to present “An exploration of the SAFE Network and its open consensus protocol: PARSEC”. You can read the abstract and register for free on the event’s Eventbrite page. The talk itself will be 1h30 mins followed by about half an hour for questions.
  • This week, we finally finished designing the SAFE Crypto changes. Now we will be able to use SAFE Crypto in all of our libraries. This time we are not abstracting the primitive crypto types, we are merely wrapping them so that the underlying library could be easily changed.

Marketing

First off this week, we have attached a copy of the monthly digital statistics for July 2018.

This week, we’ve published the monthly summary of July activities on the SAFE Network. We’ve also been carrying out a range of follow-up activities from the Decentralised Web Summit in order to build upon what was a hugely successful few days in San Francisco.

We also have advance notice of a couple of upcoming talks to tell people about. Thanks to @opacey, we now have a London Meetup booked in for Wednesday 29th August. It will be held at ThinkRise in Shoreditch (https://thinkrise.com/) and will have @dugcampbell giving an intro to the SAFE Network for the newcomers before @pierrechevalier83 talks about PARSEC. The link will go live in the next day so if you haven’t signed up yet, please join the meetup group (https://www.meetup.com/SAFE-Network-London/) to get an invite direct to your inbox. And, as you’ll see from the Routing update below, there will be another talk in London a couple of weeks later, whilst we have a couple more talks to be confirmed in the next couple of days - so if you’re out and about and want to meet up with any of the team, we’d love to see you there.

SAFE API & Apps

Last week we released and published a new version of the safe-node-app package (v0.9.0) which incorporates several changes, enhancements and some bug fixes, including the changes required to upgrade the safe_app native lib to v0.8.0. This upgrade to safe_app v0.8.0 implied some minor breaking changes in the API, therefore please consider looking at our CHANGELOG where we described each of these changes to make it easier for any app developer to upgrade to this new safe-node-app package. As usual, if you have any issues when doing this please post your questions in our dev forum that way we’ll be able to help.

Also, note we updated the code in our Electron quickstart app tutorial as well as in the tutorial published on our DevHub website so it can be used with the latest safe-node-app package.

We are now actively working on trying to fix issue #258 which was being exposed more easily with our POC applications we recently released for the DWS (Decentralised Web Summit). Once we have that solved we plan to release a new patch version of Peruse with all the changes and fixes we’ve been working on during the last 6 weeks or so mainly around making it more stable from a functional point of view.

This week we started planning what are the main enhancements we want to incorporate to Peruse to finally deprecate our Beaker Browser fork, as well as going deeper into the aspects of the evolution of our data representation with RDF and linked data using the knowledge and experience we obtained from the development of the POC apps shared right before the DWS. We would also like to thank you all who participated in testing and playing with these apps. The Patter app was just a POC but we still believe it is one of the apps we may want to keep improving and enhancing as it can be a very powerful tool to showcase many use cases.

We have solved the issues we faced with the dynamic linking of shared libraries for safe_app_java. With the help of the SAFE Client Libs team, the Android work is also back on track. The latest code can be found in this branch.

We have nearly completed our .NET developer guide. This guide demonstrates the use of the MaidSafe.SafeApp package. We are working on an example .NET Framework console app which will help you to understand how to use this NuGet package and perform various network operations. We are planning to release both of these resources as soon as internal review is finished.

SAFE Client Libs

During this week, we have been focused on PARSEC foreign function interface (FFI) and language bindings.

We have finished the bulk of work on it. We are happy with how it has shaped up and the integration tests are now completely passing. Currently, we’re polishing it, improving the documentation, and finalising the automated tests to be sure that PARSEC can be used from other languages. We’ll be starting with C/C++ because it is the common language of all systems, and knowing that a library is usable from C, we can safely assume it will work with virtually any other language. With that in mind, we’ll be building a simple integration test in C to demonstrate it works as intended.

We have also checked our bindings generator (SAFE Bindgen) is fully compatible with the new interface convention we use for PARSEC (it differs from Client Libs because we don’t have to make it asynchronous). The required minimal amount of changes have been applied and a new version of Bindgen is going to be released in the coming days, too.

In parallel, we have been catching bugs that blocked the mobile team progress: thanks to the precise bug reports from @lionel.faber, we’ve been able to pinpoint the exact causes pretty quickly and apply the fixes. The bugs were caused by the strictness of Java Native Interface usage rules on Android. The fix was simple and it is now merged.

Routing

The Routing team are still scoping the work for Milestone 2.
We had already covered Foolproof handling of malice and Dynamic membership last week. We continued our brainstorming sessions and additionally fleshed out the description of what we want to do for Performance, Extensive tests and Extensive documentation.

A few additional discussions will happen on some specifics of Performance and Extensive tests, but we reached a sufficient amount of detail that we felt ready to start raising Jira tasks for individual items of works in each of the areas covered by Milestone 2.

All of these tasks are visible under the Parsec milestone 2 epic, so you can now start to get a feel for the scope of the milestone.
Some of the tasks had been raised previously, but we now have 49 tasks under that epic, and counting.

By the end of this milestone, we aim for Parsec to be production ready for integration in Routing during milestone 3, as well as for any open-source project that would like to benefit from our innovations. There is still a long way to go, but we can’t wait to get there :smile:

As we mentioned last week, there has been ongoing work on using Proptest for running our tests in PARSEC. This work is now coming to an end, and it might replace soak testing (i.e. running the test suite thousands of times) in the future. We will also be looking at leveraging Proptest in other crates, such as Routing itself. For more details about what Proptest is and does, check the last update.

We mentioned a possible presentation for “Work on blockchain” in the last dev update. It is now confirmed.
@dugcampbell, @bart and @pierrechevalier83 will gather in London on Tuesday the 11th of September to present “An exploration of the SAFE Network and its open consensus protocol: PARSEC”. You can read the abstract and register for free on the event’s Eventbrite page. The talk itself will be 1h30 mins followed by about half an hour for questions. In the great London tradition, it will be followed by a couple of beers :beer:, so feel free to join in if you happen to be in this part of the world :smiley:

Crust

This week, we finally finished designing the SAFE Crypto changes. Now we will be able to use SAFE Crypto in all of our libraries. This time we are not abstracting the primitive crypto types, we are merely wrapping them so that the underlying library could be easily changed.

We also somewhat progressed with Crust integration into Routing. First, if we want to connect two peers together, we need to know the address and the public encryption key of a peer we’re connecting to. Then, Crust encryption keys are usually generated randomly. So we had a problem with deploying a test network when we wanted to connect multiple Vaults together but we did not know their public encryption keys. So we implemented a new option which makes Crust write its encryption keys to a file. That allowed us to implement automated scripts to deploy and connect multiple Vaults together. As a result, we ran end-to-end tests on a test network. Finally, testing revealed some bugs in the compatibility layer which we are now working on.

Speaking of bugs, there was a subtle one in the tokio-utp crate that we fixed this week. On macOS specifically, if you connect a UDP socket, you’re not allowed to use the sendto()/recvfrom() functions. Instead, you are supposed to use the regular send()/recv(). So we implemented a check to see if the UDP socket used by tokio-utp is not connected to any specific address.

Finally, we released another netsim version which slightly changed port-restricted NAT behavior. When you use a port-restricted NAT, it will only remember the last port mapping. So say you send a UDP packet from local endpoint L:1234 to a remote one R1:5678, the NAT will store a mapping R1:5678 -> L:1234. But if you send another packet from the same local endpoint L:1234 to some other remote endpoint R2:9876, the first mapping R1:5678 -> L:1234 will be replaced by a new one R2:9876 -> L:1234. So subsequent packets coming from remote endpoint R1:5678 will be discarded. I know, this is a little bit involved, but if you’re planning to use netsim, this information might become useful :wink:

62 Likes

First!!! In addition to working so much, I hope you can rest … Sleep is very important for health! Go-Go @MaidSafe Team!

13 Likes

oh i am first!!! not…

1 Like

Third lol. Ok let’s give this a read before they serve lunch.

1 Like

No medal for me. But I have participated anyway :wink:

Hope everyone enjoys this summer which is even hotter than this update.

4 Likes

Can’t wait for this! :slight_smile: Sounds good!

14 Likes

Any signs of germination yet?

15 Likes

:sunglasses: great work guys!

11 Likes

thought that team said milestone 2 is not so complex and time consuming as milestone 1 but now it seems that this one will take alot longer than milestone 1 or am I misstaking…

1 Like

Can you link to where that was stated please?

Don’t yet know if its going to take longer than milestone-1, will hopefully know better about that when the task list is complete, but yes the expectation here was to make parsec cater to dynamic network membership(rather than the fixed members expectation of milestone-1) quickly, which was spec’d and extended to the white-paper and was simple enough luckily. However, to make the crate itself feature complete(or very close to) to start integrating it with routing, malice handling and performance benchmarking/updates… which are also required from the listed readme requirements are being taken into the same JIRA epic/milestone to hopefully prevent us from having to iterate back when they need to get coded in accordingly.

17 Likes

Was great getting the chance to participate in the Patter.
Not been as shaken up (in a good way) since running the “vaults from home”.

And thanks for the exciting San Francisco event, looks like you made a strong impression with
significant people, creating the chance down the road a bit for some powerful win - win (the best way to roll) opportunities.
Thanks for everything, everyone, and all the best (patter :wink: ) for upcoming London presentations.

13 Likes

Super exciting to see all of the activity coming out of the DWS as well as the upcoming events in London!! :smile:

9 Likes

I totally agree. Silliness, nonsense, goofing off, and rest are all very underrated.

“A little nonsense now and then is relished by the best of men.”

3 Likes

Fourteenth!! Eat my shorts, fifteenth onward.

:slight_smile:

5 Likes

Oh yes, it’s a little bit tricky and we will have to take some precautions before we release info about this. Not that it’s not something that the world should love, but we have to speak to the EFF lawyers (already have started) to check the PR etc. will be OK.

23 Likes

Great Update Maidsafe team!!

2 Likes

After announcement of PARSEC I read a3 was around the corner. Never guessed it would take another few months.

2 Likes

I think it’s safe to say they don’t experience time.

3 Likes

Thanks so much for all of your hard work! And I look forward to the London meetup.

2 Likes