This week’s Dev Update has very much been a collaborative affair, with contributions from the entire team; I can take very little credit, especially the Routing section with @anon86652309 leading the update charge here. So without further ado let’s get into it…
##Routing
As @dirvine stated yesterday in a Dev Update leak 
Routing: the patient is now stable and off the critical list, this is now very much the case with brilliant progress made this week, which has been borne out in real testing. We have run 65 (one more than group size) nodes with a close group size of 8 on droplets, all nodes could populate close group number of nodes in their routing tables. 
This is a significant breakthrough and a testament to the hard work of the team. Does it mean we are fully out of the bug infested woods? Nearly, but not quite. Over to @anon86652309 for the detail.
This week has again seen significant effort put into cleaning up and fixing Routing with most of the normal working hours consumed in a multi-dev hangout with one dev (normally Spandan) coding and drafting documentation in a screen-share, so that all the others can view and discuss the progress in real-time. There have been several night-time code pushes too where simpler tasks have been identified late afternoon, implemented overnight, and reviewed by all five devs the following morning. Despite the heavy focus on discussion and documentation, there have been around 45 non-merge commits made to the ‘cleanup’ branch in Routing and we’re starting to see positive results.
From my own perspective, a significant step forward is that we now have five core devs with fairly intimate knowledge of the codebase in Routing rather than one, along with the knowledge and (draft) documentation of the process flows and design rationale which has gone into the library. This puts us in a very strong technical position moving forward.
On a more tangible basis, we now have much less code in Routing, we’ve significantly reduced the complexity, and we’ve ironed out quite a few issues and to-dos, some of which have been fairly long-standing. We’re not finished yet, although I hope we’ll soon be at a stage where the remaining tasks can be documented and implemented by just one or two of us rather than consuming the time of five developers.
The main features not covered at the moment are churn (handling nodes connecting and disconnecting from the network), swarm (handling message distribution amongst a close group of nodes) and caching (handling data being cached and retrieved on an opportunistic basis). When I say “not covered”, I mean that these haven’t been reviewed and documented in the same way as most of the other Routing processes during this current effort. Much, if not all of this code is implemented already - we just need to check its validity, make fixes where required and document the results.
I sincerely expect that next week’s update will contain news that we have implemented a working test network of Routing nodes!
For the more technical-minded, a brief summary of the week’s actual code changes are:
- reduced complexity of process for joining the network (including process for relocating nodes)
- reduced complexity of connection establishment
- incorporated helpers provided by the new maidsafe_utilities crate, the main one being using its event sender to avoid a suboptimal event loop containing a micro-sleep
- incorporated changes to Crust (we now explicitly tell Crust when to start listening and we handle failed connect attempts)
- fixed bug in RoutingTable
- reduced number of memory allocations
- removed several unused functions and unwanted lint exceptions
- removed all
unwrap()calls from the non-test code and replaced those in the test code with more useful macros - refactored some excessively large functions into smaller, more legible ones
- refactored or removed existing tests
- improved debug output
##Crust
From a Crust perspective work continues on the Crust speedup RFC, with the guys hoping to implement enough of it so that we can run benchmarks and compare results against the updated Crust this week. Crust has also seen a few updates to better suit Routing and its requirements (e.g. removing default acceptors). As mentioned in last week’s update the team continued to investigate the RUST-UTP issues and work to ensure that we have a rock solid UTP connect function and are making good headway.
##Client
The RFC for exposing SAFE Network Data types was raised last week, which details the IPC APIs which would be exposed from the Launcher. The APIs would allow developers to create and manage custom structured data types and also store raw data / immutable data to the network. Another RFC to expand the DNS APIs has been raised too.
This week we are aiming for a couple more RFCs: one to provide authenticated access from Launcher and another to make the safe_nfs API non-blocking. The aim of the authenticated access RFC is to present developers with a simple and common approach to authorise applications (e.g. standalone apps, add-on, etc).
A comparative analysis on developing browser add-ons / extensions for browsers (Chrome, Safari) is being explored, with the aim being to increase understanding of the restrictions which other platforms impose, for example, Chrome does not support custom scheme (safe:) handling. So common standards have to be defined for SAFE Network add-ons on how and what would be exposed from the add-on.
##MaidSafe Website
With the completion of all the implementation tasks, we now have an initial draft of the end user site. This will allow us to more easily review the site and make changes to the design and implementation where appropriate. A considerable amount of effort has been put in by Scott overhauling all the visuals and ensuring that the new site works well across all platforms.
While we very much hope that you like what has been produced, the new site should be viewed as a starting point, rather than a finished project. With the roll out of the network commencing soon, information regarding getting started, installer downloads and refinement of the FAQs are likely to be among the first revisions. We look forward to your comments once the site goes live.
##Documentation
As mentioned last week we have significantly increased our focus on Documentation. So this week we have had a couple of planning meetings and been evaluating readme.io to see if it fits the bill as a good tool to create and importantly maintain good SAFE Network documentation. As a proof of concept we have created a custom theme and are working on some initial “Getting started” guides for SAFE Launcher CLI and the SAFE_DNS and SAFE_NFS examples. So far it looks like this could be a good platform to use to quickly and create easy to follow documents, which also allows for community contributions; a good start.
All in all things feel like they are coming together after a couple of weeks of focus in specific areas. The feeling of confidence that the team now have about Routing is palpable and with Crust also getting a lot of focus we hope to have solid, working and efficient transport layers which we can build upon. It also feels good to start producing documentation that will enable developers to easily integrate with the network and our code. While what we are producing is very much a starting point, the importance of engaging with the community and third party developers cannot be over-emphasised and we will be looking to increase momentum as the rollout of the network develops.
