In addition, we are also working on a Getting started or User walkthrough content and we are happy to share the initial draft of the same. This content will be improved in the weeks to follow and we will eventually take this to the dev website.
On another note, we received a package from @beekeeper who sent three jars of honey all the way to Troon. This sweet (see what we did there) gesture is very much appreciated and the fact that the honey came from Ron’s very own hives makes it all the more special! Now, back to the serious stuff…
SAFE Authenticator & API
The safe_app_nodejs and DOM APIs are now up-to-date with the
dev branch of safe_client_libs. A few issues with the authenticator and the DOM APIs were fixed. We also identified that the network status callback is not propagated in the DOM API, and so we will be implementing this at the earliest. We are testing the APIs with small code snippets which you can find in the Getting started draft mentioned in the introduction.
@Kumar has resumed work and he has wired up the MutableData API for safe_app_java. We are expecting to wind this week with a feature complete safe_app_java API and start with the test cases for MutableData and NFS emulations next week.
SAFE Client Libs & Vault
This week’s major change has been the merge of the large pull request for the safe_vault repository. This hits an important milestone of bringing support for MutableData to vaults, which were the only missing component required for transition to the new apps authorisation paradigm. We continue to test these features and we’re running soak tests to catch any remaining bugs and inconsistencies, as we want to make sure they won’t ruin the user experience. After these tests are done, we’re going to run an internal network and test it in a real-world environment.
The mutable-data branch in vaults also needs a few minor tweaks to catch up on changes that were made in the master branch, such as invitation-based accounts registration, so that’s also what we’ll be focused on for the rest of this week.
Upcoming changes in routing will also affect tests in vaults. Improved message delivery randomisation in mock-crust allows to run soak tests more efficiently and it already helped us in catching some rare bugs in vaults.
In parallel, we’re starting to investigate a possible new feature for SAFE Client Libs to support transactional mutations. We are considering it as a solution to the operation recovery problem. It might help in guaranteeing that sequences of operations are executed in full (or not executed at all), without an option to leave data in an incorrect state.
We keep extending the simulation and added tools to produce graphs that visualise how each section reaches consensus on its member list and prefix. With its help, we made a few more tweaks to the rules of Option B to reduce the number of messages that need to be sent.
This graph shows the number of messages in flight (queue size) and sent as well as the total number of nodes in each step of the simulation. The y-axis is the index of the current step, i.e. time flows from the left to the right.
The spikes show the casting of votes by nodes in cases of churn, and subsequently trying to agree on valid blocks (representing the outcome of that event) eventually. Each spike roughly corresponds to a churn event, i.e. a node joining or leaving or a section splitting or merging.
To look at the development of the simulated network in more detail, we can also produce a graph that shows each valid block.
This one actually represents the same run of the simulation, but allows you to retrace in detail how the network gained nodes, split into six sections, lost nodes and merged back into one again. Time flows roughly from the top to the bottom, although two blocks of different sections (e.g. prefix
11) with the same version (e.g. 63, in this example) didn’t necessarily become valid at the same time.
It also illustrates a central difference between option B and option A: In option B (simulated here) you occasionally see two or three “contradictory” blocks with the same version and prefix which both became valid, i.e. the nodes didn’t strictly decide, for example, whether node 1 left before or after node 2: they declared both sequences of events to be valid. This slight “uncertainty” yields a simpler message flow, fewer rounds of message exchanges and faster accumulation of valid blocks.
In routing master, we are further extending the mock Crust tests in order to cover even more possible sequences in which messages could be handled. In the current implementation, for example, the tests don’t simulate situations where two nodes exchange several messages back and forth while a message from a third node is in flight. If we make the test suite general enough to take all kinds of extreme message delay scenarios into account, we hope to find potential problems much sooner in the future.