Guided by the team leads, the devs continue to focus on the tasks that need to be completed to move the SAFE Network forward. As always, thank you to the team leads for taking the time to provide these updates - so without further ado, on with the update…
##Crust, Core: Andrew, Spandan (tl) & Vinicius.
Crust now functions correctly with direct connections. The team have completed the integration with Routing and ran tests which create over 50 Routing nodes on a local machine and everything went smoothly. We tried contrasting this async with the previous blocking version of Crust and observed that (for starters) the async version has exactly six threads per Routing node compared to almost ten times more in the non-async code. These threads directly translate to CPU load, with a greater number of threads equating to higher CPU load. The blocking Crust has the CPU screaming with 1000’s of threads in total (for 50 nodes) while the async version is very benign with almost 10x less - we only occasionally hear the fan kick in indicating CPU is relaxing . Although it would be a little hasty to mention anything about benchmarks at this early stage it is interesting to note that async ran a little over a minute faster than the blocking Crust.
Hole punch tests are all passing on Linux and OSX, but are causing random issues on Windows. With a few team members off on leave last week and absent for the beginning of this week, progress has slowed slightly here. We are currently working to resolve the hole punch issues on Windows (mainly due to crashes from inside the
mio crate), and once this has been completed, the team’s focus will shift onto prioritizing message sending globally across all peers so that Routing gets an API through which it can specify which messages are most important. It should also preempt the transmission of lower priority messages which will either then be discarded, or transmitted sometime in future, if it is still valid to do so.
A while back we found a bug in the
mio code. The crash site is due to an
unwrap() in the Windows part of the code. While
mio uses this internally for windows) returned an
Option indicating there is a possibility of
mio simply unwrapped this leading to the panic. With that sorted out (in our fork of
mio currently) we are progressing further with hole punch debugging on Windows. Once completed we will inform the
mio maintainers about this.
##Routing, Vaults: Adam, Andreas (tl), Fraser & Qi.
We are still mostly focused on cleanups and Crust, however, the guys have now also started to port Routing to the current Crust “async” branch to begin testing and we are seeing promising results so far! We have added a new message type so that clients can retrieve their data chunk count and allowance.
Next we are going to attack some efficiency problems: Routing messages of up to 1 MB in size use a lot of bandwidth in the network and create a lot of lag, so we will take two measures to ameliorate that problem:
- Large messages will be split up: Instead of waiting for a whole megabyte to arrive before starting to send it on to the next hop in a message’s route, for example, nodes will be able to start transferring the first small chunk(part) of it on to the next hop immediately, while downloading the second one. Also, important messages will be able to squeeze in between the chunks instead of having to wait for the whole file transfer.
- If a group sends a large message to prove to the recipient that there is a consensus on that message, only one member will send the message itself and the others will just send a signature.
##Client : Krishna (tl), Scott & Shankar.
The RFC for launcher version 0.5 is being discussed. @cretz has been very helpful with suggestions here. If there are no more changes / suggestions in the next couple of days, the team will begin planning the implementation and update the status from proposed to active.
The major points covered in the RFC are:
- Removing encryption between launcher and applications.
- Improving security validation from launcher and proxy.
- Refactor NFS API and also include streaming abilities.
- Presenting the user account storage stats in the launcher.
- Configuration for enabling and disabling the proxy from launcher even from login the page.
We also plan on improving the UX of launcher based on the issues that have been reported and also from our own internal list of bugs and desired enhancements.
The CI integration has exposed few bugs in the APIs. We will fix these issues and complete the CI integrated for SAFE launcher. The planning for the RFC will be completed this week, with implementation work starting soon after so the client team expect the next version of launcher to roll out pretty soon.
In summary, with the Crust
mio change and the efficiency enhancements in Routing, we will be starting vault droplet tests tomorrow. All things going well this will lead directly into community TEST 4. So great progress this week, but we do need to verify that we haven’t had any code regression.
Thanks as always for all of your support, here is a link to the weekly transcript.