Here are some of the main things to highlight this week:
- We released SAFE Mobile Browser v0.3.1 and SAFE Mobile Authenticator v0.2.1
- @jimcollinson put together a video on the UX of file and data permissions.
- We finalised an enhancement made to the CLI
files synccommand which adds support for detecting modified files even when their size didn’t change.
- PRs for the network launch tool and CLI have been merged in another step towards a Baby Fleming testnet.
We have contracted an expert in
conflict-free replicated data types to review the SAFE Network and also help to prototype a
byzantine fault-tolerant CRDT solution. This could allow us to efficiently handle highly concurrent data as well as offline working and much more. This also, and perhaps more importantly, will provide further formalisation of our data types with concrete guarantees on correctness moving forward. Much of our data is very close to a fit for CRDT, but not formally so. This exercise will, at minimum provide us with much better clarity on the correctness of data mutations. This work is scheduled to begin March 5, and is expected to take about 8 weeks. More details will be forthcoming as that work proceeds.
Vaults Phase 2
From the observations in internal testnets, we’ve been making a lot of improvements in Routing, PARSEC and SAFE Client Libs which is bringing us closer to the end of our baby Fleming milestone. With the improvements in gossip (more details in the Routing section) we are seeing stabilised CPU and memory usage by the vaults and we’re having passing tests too. We made some improvements in Client libraries to improve speed by skipping the part where we wait for all vaults to respond to GET requests.
The first implementation of the network launch tool (to be able to launch a local Baby Fleming network) was also finished, and successfully tested in all (Linux/macOS/Windows) platforms. This tool covers just the very basic requirements for the upcoming release and it’s already integrated in the CLI, which means there won’t be a need to release it separately, we can make use of it with specific CLI commands (see below for details).
Last week we finalised an enhancement made to the CLI
files sync command which adds support for detecting modified files even when their size didn’t change. This was made possible thanks to the ability to calculate the XOR-URL of local files without needing to upload them to the network, enabling us to do this in a very efficient way. By simply comparing the XOR-URLs of local files with the ones already linked from the target
FilesContainer we can detect which files were modified and therefore re-upload them in a sync up operation.
We were able to finalise and merge the PR to have vault related commands in the CLI, and we also added detailed instructions to our CLI User Guide for how to use them to install a
safe_vault binary, and for launching a local Baby Fleming network, making use of our network launch tool. If you take a look at the User Guide you will already be able to understand how simple it will be to run a local Baby Fleming via the CLI once it’s ready and released.
Since we are in the process of having all PRs and crates ready for our Baby Fleming release, the current state of the
master branch of the
safe-api repo is not compatible with vault v0.20.1, either running it locally, nor with the shared vault. Therefore building CLI/authd from
master branch cannot be used with a single-vault network, and it will be usable only with the Baby Fleming network.
We’ve also started to make the necessary changes in safe-api to be able to support content streaming by allowing to fetch files providing a range of bytes. This will then need to be exposed by the desktop SAFE Browser so any webpage can stream videos or any other type of content in the standard way, i.e. through the HTTP
Labelled Data, Indexing and Token Authorisation
This week we’ve polished off the vast majority of the token implementation . With a final PR under review for the vault, enabling proper verification of Tokens and Caveats in vaults, checking these tokens against stored app credentials, which enables us to detect and reject non-current (previously issued) tokens for a given app.
As we get these final PRs merged to our main token branch, we’ve started planning the next steps for data indexing. This is the process whereby all data a user PUTs to the network will be recorded in their own personal indexes so that it can be found later (and it’s this process which we’ll later be using for managing data via labels too).
Some refactors are being made in the Client Libraries to separate the FFI-specific code from the core logic. This clears the path for the new changes coming in SAFE Client Libs which will speed up development of new features, improvements and refactors.
SAFE Network App
We’ve been hard at work on the UX of file and data permissions, and we thought you’d like a whistle-stop tour of the progress to date. @jimcollinson has put together a video on that very subject:
Let us know what you think on how it’s all shaping up!
SAFE Browser (desktop)
After some classic Windows-CI pain this last week, we’ve decided to take the plunge and convert our Continuous Integration (CI) system to GitHub Actions. This has – somewhat shockingly for CI work – been quite a pleasant affair. With GitHub Actions proving to be powerful, flexible and fast. We’ve finally got the CI system signing all our release products here. (Previously Windows had to be done manually). As this was working nicely we decided to move over tests and automate more of the release process too. All of which seem to be working faster than it was on Travis CI.
Now we’ll be automatically generating alpha/beta releases from just a tag (no human intervention necessary at all!). Which should take a few things off of Stephen’s desk.
The only marginal downer we’ve had in the process was getting macOS end-to-end tests to go on GitHub Actions. This has proved a pain as it appears some user interaction is needed to accept some permissions, and there’s no obvious way around this (if anyone has GHA E2E experience and has a solution for this, let us know!). In the meantime, we’ll keep running macOS e2e tests on Travis CI.
SAFE Browser (mobile)
Today we released version 0.3.1 of the SAFE Mobile Browser
This patch release comes 2 weeks after our v0.3.0 release supporting the Perpetual Web and includes several fixes and improvements based on the feedback we received from that release. For example, we have fixed an authenticate loop on iOS, as well as iOS system-level dark theme overloading issue, and added support logic for audio streaming on Android.
For full instructions on installing the Browser on your mobile device, and on how to connect it to a vault, please see the previous release post on the forum here.
SAFE Authenticator (mobile)
To accompany the new Mobile Browser, today we also released version 0.2.1 of the SAFE Mobile Authenticator
As with the Browser, this Authenticator release also contains fixes and improvements primarily from feedback received in the last couple of weeks. This includes a reworked
Add a vault page and a generic error message on login details not matching.
Remember to have a read of the previous release post on the forum here for full details on the Authenticator and the Browser, and how to install and use them.
As always we encourage as much feedback as you can manage to help us fine-tune these apps.
Routing and quic-p2p
This week we finally finished the work on improving gossip (as mentioned in the previous two dev updates - gossip is the mechanism by which nodes exchange messages to update their internal state in order to reach consensus in a fully decentralized manner). After exploring a couple of dead ends we eventually found an approach that removes the artificial delay between gossip messages while also avoiding flooding the network with too many messages. This resulted in a pretty nice performance improvement in our internal “Baby Fleming” testnets. Additionally, we exposed a couple of parameters which allow us to tweak the network further, possibly allowing further performance gains after more testing.
We are continuing to work on quic-p2p improvements which are shaping up nicely. We discovered some areas for improvement while carefully looking into the bootstrapping logic: in several conditions, there could have been a case where some unneeded connections lingered on and consumed resources, and the echo service logic introduced some needless latencies (i.e., the initial connection to the SAFE Network took more time than necessary). We are reworking these modules now and with all these changes quic-p2p should become even more efficient and lean.
Feel free to send us translations of this Dev Update and we’ll list them here:
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the SAFE Network together