Here are some of the main things to highlight this week:
- @Bart has published the 4th installment of the Road to Fleming series all about Network restarts.
- We published the SAFE Network March Round Up on YouTube and Medium.
With @dugcampbell away in Berlin talking all things SAFE at the Landing Festival, and two of the team away on their holidays it’s been a slightly quieter week in Marketing. So holding the fort has been @SarahPentland and @cgray, with them working on plans for content over the month of April. We’ve already got a few volunteers lined up to join us on the sofa for the SAFE Buzz video series, but if there is someone in particular you’re keen to hear from or you want to understand more about one of the teams, just add them to the thread below and we’ll see what we can do. As we’re now (already?!) in April, we have the usual regular monthly update on YouTube with a round-up of things that happened in March. For those who prefer the written word, we’ve got an overview on Medium so head over and check that out. As always, feedback welcome . We pushed out another Tweetstorm, this time on a simplified, non-technical view of the SAFE Network - what do you think?
Work continues at a fine clip on a major update to the website. It’s designed as an evolutionary enhancement to the existing site, but it’ll also contain some new content which we think the community will be particularly delighted with.
The site has always been a key tool for the Marketing team to explain the Network and describe its vision and impact. That’ll become even more crucial in the coming months, as milestones converge, and we’re happy to be doing our bit to making it all happen. Exciting times!
SAFE API & Apps
We’ve been working on enhancing the documentation on the browser and we finally merged that into its master repo. The README now contains a lot more information not only for users trying to build it and/or use it, but also for developers interested in contributing to the project. There are details now about the structure of the codebase, its internal design, and how the development tasks are organised in the project board.
This week has also seen @manav_chhuby digging into the internal tab logic of the browser. This is one of the oldest parts of the codebase, and is responsible for no small number of bugs due to how it was initially set up. Manav has been working hard to implement some increased sanity here, as well as providing even more tests.
@hunter has also polished off a PR that achieves a couple of nice wee things: 1) It blocks all HTTP/S requests by default (no longer triggering your clearnet browser). It still shows a notification for any navigation events that occur, while throwing an error for any
fetch events triggered via the DOM. And 2) It deprecates the ugly notification prompt we all know and firmly have mixed feelings about. And instead integrates ant designs notification component, which is decidedly less blocking and somewhat easier on the eyes too.
On the auth CLI, we have a PR in progress and almost ready for review, which would allow users to store their SAFE account credentials in a config file so they don’t need to be entered by the user every time. We are also planning to work on adding more unit tests to the project.
SAFE Client Libs
init() has been called on Client Libs! This week, we have started to define tasks for our next milestones. We’re starting to think about the implementation of the XOR URLs RFC which will make it possible to use SAFE URLs in RDF resources and link data stored on the network together. And speaking of RDF: it will be picking up pace again, as we’re continuing to work on the proof-of-concepts and demos, fixing the remaining blockers and bugs. @lionel.faber and @Yogesh will also be joining this effort to quicken the development progress. Update of SAFE Bindgen to use a modern parser library is nearing completion and we take it as a priority because it’s part of another larger milestone of porting Client Libs to use the latest stable version of the Rust compiler.
Reshaping of our Rust APIs to be more developer friendly is another thing we’re aiming to implement. We’re starting with simpler steps, exposing more internal helper functions that simplify interaction with the network. We’re also making sure all FFI APIs are available in Rust. In the longer term, when Rust has more language features, we’ll be streamlining our interfaces further. For example, one of the things we’re looking forward to is the inclusion of async/await syntax in Rust which should make it much more straightforward to write asynchronous code, without having to learn difficult concepts like futures. Considering the asynchronous nature of all network operations, it will be an obvious improvement to our API.
On the documentation front, we are nearing completion as a major part of the work has been dealt with by @marcin. @lionel.faber has been wrapping his head around the code this week to better understand how the gears move in the Client Libs implementation. This will be helpful for the upcoming refactoring milestone.
Our involvement with the community continues as @mav has been very patiently responding to review comments on his code submission. We also made edits to the Guide to contributing page, adding new sections and more links to make it even more accessible for people without much development experience. We also added a note about this guide which appears when opening a new issue or PR. In addition, we migrated a significant chunk of our old Jira issues to GitHub, making them open for the community. Should anyone want to tackle them, a good chunk have been marked
good first issue or
This week we made some interesting progress on the Fleming front, but let’s start with what is happening in PARSEC.
The PARSEC subteam worked on making the algorithm still more resilient to malice. We implemented a fix to a bug present in the original whitepaper, which made it possible to create split consensus by forking the gossip graph. There was a hole in the proofs of correctness, which we fixed in the to-be-released 2.0 paper and backported the fix to current PARSEC. There is also ongoing effort in improving the handling of accusations cast by nodes and in creating tests for the malice handling code.
On the Fleming side, we went full steam ahead in detailing the changes required for Node Ageing, and even started ticking off some initial implementation tasks.
In the implementation, we started moving the responsibilities of the Routing Table to the Chain. With the introduction of the Chain, which holds the history of membership changes in the section, some functionality was duplicated between it and the Routing Table. This move effectively deduplicates some code, making it easier to maintain and develop it.
Another implementation task is focused on changing the message relay mechanism. The current mechanism of passing a message between two distant sections relies on a chain of relay nodes, one per section, across the network. This has the drawback of a single node being able to break the chain, so the mechanism additionally involved waiting for an acknowledgement of reception of the message, and if it couldn’t be received, re-sending the message using a different route (a different chain of relay nodes), with up to 8 routes (the sender gives up if the 8th route fails).
The new mechanism gets rid of the acknowledgements and timeouts - to make them unnecessary, instead of a single relay node per section, we use
N/3 of them (
N being the size of the section). Since we always expect every section to contain more than
2N/3 honest nodes, this means that we expect at least a single honest relaying node in every intermediate section (which we call a hop). All of the relaying nodes then pass the message to all of the relaying nodes in the next hop, making it certain that the message will make it to the recipient, as long as our security assumption (more than
2N/3 honest nodes per section) holds.
On the Node Ageing design front, we are analysing the flows needed to satisfy our requirements. We prepared some documentation in the form of flowcharts that will guide our implementation, which you can check out here (bear in mind that we are still working on making them more readable, explaining the context, etc.). Reviews, comments and contributions are very welcome! (If you wish to contribute, the source might be useful.) We also created an implementation of the flows in actual code, in order to be able to test their correctness before touching the Routing codebase.
This week we’ve been investing a lot of time into quinn – the Rust QUIC protocol library. While quinn still won’t work as we’d like it to, we keep analysing the failures we observe under the high load. We are in constant communication with the quinn team and we filed a number of issues and proposals. To highlight a few:
- Random panics once connection is explicitly closed
- Occasional ‘specified instant was later than self’ panic
- Panic in unwrap (for
- Panic on connection timeout
- Resolving quinn::Incoming
All of our bug reports get quick responses and most of them have been fixed and closed already, so we’ll keep testing the changes and running our test scenarios. Overall, this work should also bring better QUIC support to Rust and benefit the community.