Here are some of the main things to highlight since the last dev update:
- We’ve found and merged a workaround for a
lagissue that had been doing its best to remain unsolved.
- A draft PR for the rewards flow has been raised, and is nearing completion.
- Some broken client network tests, which have prevented us from progressing with testing the fix for a
DataSectionmessaging issue (the final issue blocking a public testnet), should now be resolved and testing will now resume.
- We merged the CI/CD PRs in each of our BRB crates, which resulted in automated test coverage, versioning and our first releases of the BRB crates on crates.io.
- @JimCollinson provides another captivating update on the progress of the Safe Network App & UX.
Safe Client, Nodes and qp2p
Safe Network Transfers Project Plan
Safe Client Project Plan
Safe Network Node Project Plan
Solving of lag issues
There’s been a lot of deep diving in the last week attempting to find the source of the significant lag we’ve been experiencing while running internal testnets. And while we’re not wholly ‘fixed’ we do have a much better idea here, and a simple enough workaround has now been merged.
The crux of this issue was something blocking a node’s handling of client requests. This may be related to some sub-optimal node concurrency (which we’re also looking at fixing), but the root seems to stem from dropped connections blocking the ability of node’s to respond, and so they’ve been effectively waiting on the connection to be fully abandoned by the lower layers - this wait is the lag that we’ve been seeing.
It seems that a simple tweak to a node’s idle-timeout (lowering it from 30s to ~5s…we’re experimenting to find optimal numbers now) will get the nodes sufficiently unblocked that we can again get client tests running nicely. So while we’re digging in to see if there’s an issue in
qp2p’s connection pool, and refactoring
sn_node somewhat, we’re able to move forwards now with testing everything once again.
We’ve previously mentioned the ongoing work on the rewards flow. The rewards are paid by the Elders in a section, by them using a distributed AT2 actor - (i.e. the
section actor) and aggregated signatures. Every time there is a change in the Elder constellation, the section key changes, and the distributed actor transitions to the new key by transferring all the tokens to it. There’s a different initial path on the transfer flow for the section actor (and any multi-owner actor), compared to single owner actors, where validation of a transfer is proposed by a majority, before there is a validation at the replicas.
The transition from genesis node as well as tests for this are completed and confirmed, now we are working on the subsequent transitions.
While working on the above, we found that the Elder state was not consistent between the dependent operations in such a transition, and thus this week we refined the representation and handling of this state. A draft PR for the rewards flow was raised, containing all of the above mentioned work, as it is now nearing completion.
DataSection messaging issue
In a previous dev update, we mentioned that we were investigating removing the possible duplicated signature-aggregate work carried out by
sn_node during exchanging messages among
DataSection. There is a WIP PR removal signature aggregate raised for this purpose. This PR passes
sn_node unit tests, and we had been using
sn_client network tests to carry out further verification. However, due to other refactoring work in
sn_client, carried out in parallel, the
sn_client network tests have been broken for other reasons, so the verification work has had to be put on hold for a short while.
We believe that this duplication removal work will address a
DataSection messaging issue which we observed during our internal testnet runs at the end of last year, something we should be able to verify via the
sn_client network test. This is currently the expected final remaining blocker to being able to host a public testnet, so we haven’t been able to progress with internal testing here in the meantime.
However, with the recent merge of some refactoring work in both
sn_client, we are now back onto this as a priority, and hopefully able to work through the test cases without complications.
API and CLI
After the introduction of the
sn_messaging crate, where all client message types are now defined, we were working on making the necessary changes in
sn_api and CLI to be all compatible again with
sn_routing messaging. This is now complete and all changes have been merged to the main branches of all these crates. We’ve also made some progress in trying to move the
sn_routing message definitions to the
sn_messaging crate, although this is still an ongoing task with a lower priority than most of the other things we are working on.
We’ve also merged some API improvements made by @Scorch onto the
qjsonrpc crate which makes path parameters generic. Thus we’d like to thank him for his contribution.
BRB - Byzantine Reliable Broadcast
Tuesday marked another milestone as we merged the CI/CD PRs which resulted in automated test coverage, versioning and our first releases of the brb crates on crates.io.
A major improvement since last week’s update that made it into this release is support for generic Actor types via rust traits. This means that the code is now flexible enough to support most/any public key cryptography algorithm or library without further changes to the core libraries. Support for ed25519 is already included but we will be looking at adding BLS support. Even hardware signing devices could be added in the future.
Additional improvements include:
- The brb_node_qp2p CLI has some display fixes and a new retry command in case of any dropped packets.
- logging. All println calls have been changed to appropriate logging functions.
Work in progress:
- test cases are being improved to return results and remove any calls that would panic()
- doc comments are being added to make the code more usable by other developers
Finally, some more great news! Our CRDT specialist, and the big brains behind the brb crates, has agreed to extend his contract by another 3 months to assist with further feature and integration work.
This week we merged another batch of fixes for issues uncovered with the stress test tool. With these in, we seem to be in a much better place. We still have problems with forks (divergences in the section chain) which sometimes happen during very heavy churn. This problem will be solved by integration of the new membership algorithm, work that is already in the planning stage.
We also merged a couple of minor changes. First we made sure nodes never create connections to clients - instead when a node needs to send something to a client, it reuses the connection that the client created previously. If no such connection exists, it reports an error. This by itself doesn’t really fix anything, but it makes certain failures manifest themselves in a more obvious way, which simplifies debugging. In a similar vein, we improved error handling (PR under review) so that errors are now more specific and contain more information, again to simplify debugging.
Finally, we fixed a long standing issue with our continuous integration pipeline where one task would keep consistently failing. Even though it wasn’t critical, it still prevented us from seeing that satisfactory green check mark before merging a PR. Turns out we were just a bit too strict in the way we treat compiler warnings. This fix was also rolled out to several of our other crates.
Safe Network App & UX
Feature Tracker / Screens & Flows
If you recall the UX update in November, we took you through proposed updates to the Safe Network lexicon, introducing a new metaphor of a Safe to replace Account, tokens rather than Safecoin, and deprecating the use of the term Vault.
Well, these changes have started to make their way through to candidate UI designs where they can start to prove themselves in the user experience pixels, rather than just on paper.
If you jump into the Figma file for the Safe Network App designs, you’ll start to see these in action, as well as a few other refinements.
It’s a big ‘ol file though with hundreds of different screens and flows, so it’s not always easy to take in everything that’s going on. So here are a few highlights:
Unlocking Your Safe
Here we see the metaphor of a Safe for the user’s data being introduced—locking and unlocking it—with some visual cues that hopefully re-enforce that too.
You’ll also note that the Action Menu has been promoted to the top app bar, via a traditional hamburger. And that it’s now accessible in the locked state, too. But more on that later.
Working with Tokens
Along with ushering the end of the term of Safecoin, changes to the wallet screen include an overarching Tokens section, a bit of a UI spruce up, and a more multifunction New Transaction action button. It’s a way to kick off a new transaction—skipping some of the slightly longer flow we had previously—and also another entry point to start earning Tokens.
With the deprecation of the term Vault, here we can see an exploration of how we can favour a verb such as Earn to start flows for offering resources to the Network in return for Safe Network Tokens.
This flow can be started from a number of points, including from the Earnings tab on the Token screen, from the home or apps screen, and from the utility itself (as well as from the Action Menu or a typed command.)
A couple of steps to configure things, and we are off and rolling.
The Action Menu
You’ve seen this previewed before, but here’s the Action Menu, so named because you can use it to search and access all functions of the app, as well as directly kick off commands.
It’s now accessible top left of the app, and here you can see it in both locked and unlocked states.
A wee reminder of how commands can be searched and kicked off too. Handy.
Feel free to dig into the Figma file to see all the details on this, with a multitude of individual flows explored and documented.
Feel free to reply below with links to translations of this dev update and moderators will add them here:
Russian ; German ; Spanish ; French ; Bulgarian
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!