Today, we are expanding the reach of the Test 17 network to trust level 1 forum users. See this topic to download the binaries for Test 17 (the binaries haven’t been updated, so there isn’t any change for trust level 2 forum users).
If you are unable to connect to Test 17, you might have to try again later. We’ve recently merged some changes in Routing (for the rate limiter) to remove the restriction on the number of clients allowed per proxy node, however, we haven’t updated Test 17 with these changes yet so currently there is a maximum number of 200 clients that can connect to the network at a time. This limitation will be removed the next time we update the network.
Note: test accounts are limited as to the amount they can upload to the network. If you hit the upload limit of your account (strange errors, unable to log in, etc) you can request a new one from the @moderators
There are quite a few changes that are coming up (see the individual sections for more info) that should be live soon. These changes are currently being tested with integration and soak tests. We’re hoping the number of times we need to restart and update the backend of the system before Alpha 2 will be minimal, and we’re hoping to get there quite soon 
It is partly for that reason that we are consolidating changes, frequent updates would require more network restarts. This way we hope to minimise the inconvenience for developers, website builders…etc…
Marcin (@marcin) and Andrew (@canndrew) joined us earlier this week - Andrew actually re-joined us (as you may remember) and has already started work in full flow - he started with MAID-2149 and is almost through with it and Marcin is new of course and he’s getting familiar with the code right now.
SAFE Authenticator & API
During the last week, the frontend team was fixing several issues reported for the browser, which includes the safe-app and safe-authenticator plugins, the example apps, as well as working on new functionalities and suggested enhancements.
As anticipated in the previous update, we put a lot of effort into making sure that all objects created by any webapp through the DOM API are released/freed from memory either when the web page is being refreshed, or when its browser tab is closed.
The safe-app plugin runs in Electron’s main process, but the DOM API is exposed to the renderer processes (all separate processes), so when a webapp invokes one of the DOM API functions the renderer is actually sending an IPC message to the main process, and the objects are effectively created/instantiated in the main process by the plugin (when it invokes the underlying safe_app_nodejs functions). This is the only reason why webapps are provided with handles to reference these objects, and it’s the webapp’s responsibility to let the plugin know when they can be freed by invoking the corresponding free functions.
This has been implemented, but given the nature of its impact (as this could be seen as a memory leak even though it’s actually the webapp’s responsibility), we are now performing several tests with different scenarios to make sure this mechanism is working well and stable.
Another task we were working on, which is also finished and going through internal testing, is the toggle to allow browsing of clearnet sites by using the “File” > “Toggle SAFE Browsing” menu action.
Some enhancements were also made to the way the browser interprets URLs. If the URL of a safe:// website is simply safe://<service>.<public Id> (i.e. without the name of the index file), the browser will load the file index.html located in that service container by default.
Several enhancements are being worked out on the Authenticator’s UI/UX as previously mentioned, and a very useful and well requested feature was the possibility of verifying the number of mutations available and allowed for the account. This has been now integrated into the Authenticator UI, and it will be displayed right after you log in with the ability to update/refresh it from there.
The NFS API has been recently enhanced in safe_client_libs, the changes required in both safe_app_nodejs and the safe_app plugin are already merged and going through internal testing.
The changes in the NFS API impacted the web-hosting-manager sample application which has also been updated to be compatible with the new NFS API. With this change, DataMaps are now stored as MutableData instead of being stored by the frontend layer as an ImmutableData chunk.
The NFS API changes had to be updated in the web hosting example app and we are testing the same along with minor updates to present user-friendly error messages. Also, an issue was reported in the testnet feedback: when the Public ID or Service name was too long, it used to overflow. This is now fixed.
All this has been very good progress towards having much more stable applications / examples, and a friendlier experience when interacting with them; however, this also implies that a great amount of effort needs now to be put into integration tests to make sure all the pieces work well together as expected, and even though some minor issues were already discovered, everything is looking much better.
SAFE Client Libs & Vault
This week, we’ve been working on completing the operations recovery feature outlined in MAID-2146 and MAID-2148. For now we’ve discarded more complicated approaches (such as adding atomic transactions to the core network features) and decided to focus on making composite operations robust and stable. To reiterate, the problem we’ve been trying to solve is the graceful handling of composite operations failures in Authenticator (the part of SAFE Client Libs). For instance, if the browser crashes while a user creates their account, an account packet will be created on the network, but the next step, creation of the user’s root directories, won’t be completed, effectively leaving the user’s account in an incomplete state and blocking further usage of apps. It took us quite a bit of time to find and implement a generalised solution for this problem, so we’ve agreed on introducing some basic rules for recovery - e.g., if a user tries to store a Mutable Data instance on the network and it already exists (as a result of a previous failed attempt), we’ll try to add missing permissions and entries instead of just returning an error. The same approach is used for other atomic operations, such as updating Mutable Data entries/permissions. Currently, the pull request that implements this feature is being reviewed and it’s getting close to completion, as we’re planning on adding more tests to make sure that it works correctly, taking all edge cases into account.
In parallel, @canndrew has learned about lots of changes in our codebase and quickly became productive. He’s been working on providing options to request access to arbitrary Mutable Data objects, which is a new feature in Authenticator described in detail in the JIRA task MAID-2149. To put it more succinctly, this feature allows apps to share permissions to change arbitrary Mutable Data objects owned by the same user. It caters to the case when an app has created a new Mutable Data object on the network on behalf of a user, but other apps can’t modify it even if they know its location and encryption keys. For now it’s possible to share permissions only through the access container, which is currently restricted to the default containers (such as “Documents”, “Videos”, etc.) and the apps’ own directories. In particular, because of this we couldn’t share public IDs between apps. This change solves the problem by introducing a new IPC (inter-process communication) request type: ShareMData, which contains information about Mutable Data location and required permissions. This request is sent to the Authenticator, prompting a user to allow or deny it. If it’s been granted, the requesting app’s key is added to the Mutable Data permissions list and the app is then allowed to modify the Mutable Data. This feature is implemented now and the pull request is being tested and reviewed.
Lastly, as a side effect of implementing tests for operations recovery, we’ve added a new testing feature in mock routing (the part of SAFE Client Libs that simulates the real network behaviour in a locally controlled environment). It allows to match specific requests on the lower network layer and override responses, sending predefined values. This is very helpful in simulating network failures and recovery, and possibly it could help us in expanding our test suite even further, covering more complicated error cases.
Routing & Crust
Routing, Crust and Vault now get their own config files with optional developer options. These can be used to change the behavior in local networks for these specific crates. In Crust, we added disable_external_reachability_check in the dev options so that folks can run their local network without having to make sure they are externally reachable. In Routing, one can disable/enable resource-proof with disable_resource_proof, rate-limiter with disable_client_rate_limiter and/or minimum section size the proxy should be a part of before clients are allowed to bootstrap off them via disable_min_section_size_for_client_bootstrap. Routing also gets a new option to specify the group-size in case someone is interested in experimenting with various group sizes in their locally run network. For Vaults, we’ll be adding a dev option that lets users bypass the mutation limit for client accounts.
Rate-limiter has also seen a few tweaks as a consequence to its client requirements in testnets. We have now removed the restriction on the number of clients allowed to bootstrap off a proxy. With this, the rate limiter will now leak (for the leaky bucket algorithm) at a rate of 5MiB/sec. So per sec 5MiB are going to be distributed equally among the current active clients.
Operations recovery is a mammoth task that is being done in safe_client_libs right now. It requires some trivial changes in Routing with regards to better error specifications, so that will get merged too once the safe_client_libs crates have their code ready and tested.
.
