Summary
Here are some of the main things to highlight since the last dev update:
- Some big updates to the Safe Network Lexicon are ready to be unveiled! Watch the video here or read the transcript in the dedicated forum thread here.
- Work is underway to support multiple clients using the same public key to connect concurrently and mutate data.
- Further network startup performance improvements have been merged to sn_routing.
Updating the Safe Network Lexicon
We’re updating the Safe Network lexicon for end users, and replacing the terms Account, Safecoin, and Vault.
This will probably feel like a biggy, so @jimcollinson has made a video setting out all the details on why we need to do this, and what we are proposing to replace them with.
Please feel free to add your thoughts, and comments, and observations in the dedicated thread on the subject, where you’ll also find a full transcript of the video.
Safe Client, Nodes and qp2p
Safe Network Transfers Project Plan
Safe Client Project Plan
Safe Network Node Project Plan
With the section startup changes merging into sn_routing last week, it brought in a few breaking changes to sn_node where newly joining infant nodes undergo relocation back to the same section immediately to induce ageing. This relocation has to be worked around or ignored in sn_node as it takes a relocating node to be someone from a different section and so proceeds to fetch the wallet IDs, rewards, stats, etc. from the previous section, not realising that it is an infant undergoing induced ageing. We are well underway with a refactor to take this scenario into consideration, alongside a few more fixes for blob data flow that re-enables chunk replication at adult nodes.
A parallel task we have just started on is to support having several clients which make use of the same public key when connecting concurrently from different devices/instances to mutate data. For example, a user may have a public key which owns his/her emails, and he/she uses an email application from a mobile and laptop at the same time. Currently there is a limitation for this use case, and we are trying to now support it by allowing the application to provide some form of application instance identifier. Such an identifier would be used internally in the CRDT operations to make sure both instances of the app can mutate data concurrently without creating data merge conflicts.
CRDTs
The initial Sequence CRDT proptests have been merged into sn_data_types, which increase confidence and coverage. Part of this test setup led to us improving permission handling to avoid potential branching issues when old permission ops were applied, now also merged to master.
This work has also shone a light on the need for CRDT operations to be signed to be able to verify the entity that initiated the operation. This gives us more confidence, but also shifts permission checks into the data itself, which should make testing this much simpler. As such, we’ve started work on getting the sequence ops signed, with the aim of verifying validity of ops against permitted public keys.
We’ve also got a POC draft of a Map CRDT type compiling with some basic proptests passing there. There are still changes to be made w/r/t the internals of the Map, but this has the same style of permissions as the Sequence type, which means once we have augmented that with sigs and verification for ops, we should be able to port it over to Map easily enough.
DSB Dynamic Membership
Our CRDT consultant has been plugging away. This week he has implemented an example node that communicates using QP2P and supports both peer join and removal (kill). A particular issue has been identified regarding how to deal with messages initiated before a voting membership change. The Generation Clock is thought to be a solution here, but that still needs to be proven out with tests.
In parallel, the original bft-crdts crate is being broken apart into smaller, more focused library crates. The idea is to loosely couple (a) the particular secure broadcast implementation, (b) the data being secured, and (c) the network transport (e.g. QUIC, TCP/IP, or in-memory network for simulated tests).
Routing
As proposed in last week’s update, the work to further improve the startup phase has now been merged. This improves the startup performance by giving new infant nodes a wider pseudo-randomly generated age range and only relocating the oldest nodes, therefore avoiding too many nodes being relocated at the same time.
In progress at the moment is work to allow a node to rejoin with the same name. This is designed based on the Subsequently Joining the Network section of the Node Ageing RFC. The intention is that a node trying to rejoin with the same name will be immediately relocated with age halved, as long as the halved age is greater than the MIN_AGE (currently 4).
There is also ongoing work to improve lost peer detection, with the support of qp2p. The basic plan here is to ping peers on connection loss to confirm if they really have gone offline. With lots of aspects needing to be considered and balanced, particularly as this is such a key factor of faulty peer detection, there are ongoing internal discussions and debate to find the best way forward.
Useful Links
Feel free to reply below with links to translations of this dev update and moderators will add them here:
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!
or the start… the awakening!
