Here are some of the main things to highlight since the last dev update:
- @jimcollinson provides another fascinating insight into the progress on the SAFE Network App UX.
- The Symlink support and
SequenceCRDT PRs have been merged into the safe-api codebase.
- A new section has been added to the CLI User Guide describing how the new
safe seqcommands can be used to store data on a
Public Sequencedata type. These new commands will be part of the upcoming release of the SAFE CLI.
- Work continues at pace to get SAFE Client Libs and SAFE vaults AT2 ready, with only a few more missing pieces to the jigsaw.
- We extracted the XOR space module out of Routing into its own separate crate, allowing it to be used by other parts of the project.
Vaults Phase 2
This week we’ve continued working on improvements based on the results from the last testnet. We have delegated the duplication process to the node responsible for holding the new copy of the chunk. This eases the stress from the Elders who were previously responsible for fetching the data and sending it to the new holder.
We have also removed the caching of ahead-of-time responses at Elders. Instead of holding on to these responses and processing them when the request eventually arrives, we use the accumulated BLS signature to validate that the section has approved the request and process it accordingly. These enhancements have already shown a lot of improvement in terms of memory usage and performance, and we are excited to share them with you. But first, there are a few more features that we’d like to add before the next iteration of the Vaults from home testnet.
SAFE Network App UX
We’re pleased to share with you today a whole trove of screens, flows, and documentation.
Much of this you will have seen before, but there have been many hundreds of changes, tweaks, refinements, and iterations over the months.
We’ve dropped it all in one Figma file for you to dig in to—alongside the Feature Tracker — so you can keep abreast of the whole shebang. There are some screens still to drop in, and some still to be fully documented, but it’s a living file that we’ll be publishing week-to-week.
We hope this starts to paint a more detailed picture of some of the initial experience we’re building for Beta, and most likely before.
Please feel free to ask questions about any of it here, or directly in the comments on the Figma file if that’s easier when you are down in the guts of the designs.
Symlink support was merged into the codebase this week after resolving an issue that turned up in testing at the end of last week.
Following on from that, work commenced on an implementation of
glob() that operates in a FilesContainer. This is an experimental feature that enables pattern matching (e.g. *.txt) when resolving a SafeUrl. This could be handy for filtering results when using
safe-cli commands such as
files ls or
A couple of API and CLI commands for the
Sequence CRDT have been added this week and are ready for the upcoming release of CLI. These new commands allow users to store a Public Sequence on the network and append items to it using its XOR-URL. You can take a look at a new section added to the CLI User Guide describing how these new
safe seq commands can be used to store data on a
Public Sequence data type.
As part of the implementation of the
Sequence API and commands, the use of
AppendOnlyData has been removed and now the
NRS Map Containers are stored on the network as
Public Sequence content. This doesn’t affect any of the CLI commands or APIs since it’s just an internal change in the implementation switching from one data type to this new one.
This past week we finalised all the PRs in the full stack for a first iteration of the
Sequence CRDT, while at the same time removing the
As anticipated last week, the last steps to achieve this were all about final clean up of the code in the safe-client-libs and safe-api crates, as well as creating tests for our CI test suite. We’ve been also testing all this E2E using our CLI tests suite with a Baby Fleming section, and we have all our functional tests passing.
We are also investigating an issue we still see on some of the vaults which sometimes don’t return a piece of content. We are still not sure the exact root cause of it.
Some of the next steps now are related to being able to attach an AT2 payment to a CRDT operation and that’s what we are starting to look at now. Another pending task is to transform our current
MutableData data type to a new
Map CRDT, which we believe can be done after we have our first iteration of the
Sequence CRDT released and thoroughly tested by the community.
This week we’ve been fully onto the integration work with AT2 SAFE Client Libs and SAFE Vaults. With
Simulated-Payouts in action as mentioned in the previous update, both modules are now AT2 ready with only a few more missing pieces (choosing efficient routes for payments, refunds, etc.) that are yet to be written, alongside the integration testing. Almost all of the flows that deal with
Money (PUTs/Mutations) have been updated to work with the underlying AT2 mechanism.
We are evaluating options in paying for data, which define, amongst other things, how sections use
safe-transfer. Related to that we’ve progressed with farming. Tests have been written to verify the BFT properties of reward accumulation. Reward accumulation is designed to allow for rewarding a minimum unit of work, as well as leaving it up to upper layers to decide what it considers to be “work”.
Finally, we’ve been making corresponding updates in SAFE Client Libs for these latest changes to the PUT/Mutation process. Once we have these in we’ll be looking to verify everything against the updated vaults.
This week we finally merged the shared state signatures feature that we discussed in previous dev updates. We then addressed another requirement from the vault team - turns out a vault sometimes needs to manipulate the BLS signatures in a way that is not directly supported by the Routing API. We decided that instead of making the API more complex and potentially convoluted, we would let the vault perform these manipulations directly. But to avoid duplicating potentially non-trivial logic between vaults and Routing, we decided we would expose some low-level BLS helpers out of Routing. In particular, we implemented a Generic Signature Accumulator which handles the nitty-gritty details of collecting BLS signature shares, validating them and combining them into a full signature. The PR has been created and is currently being reviewed.
The work on the DKG replacement is progressing along nicely. A key issue when handling DKG around splits has been resolved in this PR - a very high level summary of our new approach to this is that we will now allow new Elders to vote for the new DKG. The PR is currently being reviewed and rebased against the latest fleming branch (because of the recent changes there, this will take a bit of time).
We also extracted the XOR space module out of Routing into its own separate crate, because it’s actually useful for other parts of the project. This crate defines the
XorName structure which is a 256-bit long number serving as a unique identifier of nodes and data on the network. It also defines the XOR metric (distance between two names) which plays an important role in message routing. Finally, it provides the
Prefix structure which forms the core of the disjoint sections concept.
Feel free to send us translations of this dev update and we’ll list them here:
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the SAFE Network together