Firstly, sorry for the delay in this week’s update, we had a few things we wanted to complete before sharing this update and I would also like to thank @dirvine for picking up and producing a bumper Dev Update last week. As David highlighted in that update, Crust and Routing are now signed off as MVP ready with Vaults in the final phase of testing and it is all looking good to be signed off as well. So no MVP release as part of today’s update, we need to tidy up a few loose ends before wrapping and tying the bow around about it. However… yes there is a however, if you are super keen to get involved in community Vault testing a little earlier - we are in the position to share Vault (TCP only) binaries that can be run from home (subject to some limitations please read before jumping in, details below). I really want to be 100% clear, this is NOT the MVP… I repeat …not the MVP but it is a Vault running from your own computer and joining the SAFE Network (we are running 100 Vault nodes) - which is in itself extremely cool and another milestone as part of the rolling community testing release cycle.
So how do you do this…
- Download safe_vault binary and unzip files.
- To run a Vault, it is currently necessary to be either directly connected or forward a TCP port in your router and then specify the port you have forwarded within the Crust config file (
nullwith the port you forwarded e.g.
- Execute the binary (from the command line, although not necessary on Windows) - you have now joined a Vault to the SAFE Network
##SAFE Launcher v0.4.2
- SAFE Launcher has also seen a patch release to v0.4.2 which should be used with this iteration of the SAFE Network - download Github release here.
- SAFE Launcher documentation here.
This Vault test comes with some limitations:
- To run a vault, it is currently necessary to be either directly connected or forward a TCP port and specify that within the Crust config file.
- File size will currently be limited to 5Mb
- Vault size will currently be limited to 30 MB (30 chunks)
- Storage per client account is limited to 50 MB (50 chunks).
- Churn handling - replacing connections and relocating data after nodes joined or left the network - is still very rudimentary. It will almost certainly fail if there are hundreds of megabytes of data to relocate, which is why we set a low Vault size limit for now and ignore the one set in the Vault config file.
But too high a churn frequency might well do even more damage to the network; we’re not sure how much - that’s what real-world tests are for!
However, for most of these limitations we already have concrete solutions that we will implement in the coming weeks.
- Zombie data: If there is heavy churn it can happen that a client requests deletion of data while that same data is being relocated. In that case, sometimes the data comes back from the dead after its deletion.
- Known issue - occasionally the client may not receive enough “agreeing” responses from a network operation in order to decide the result. This causes the client to hang. It should just be restarted if this happens.
- There is still a lot of work to be done in the Routing library for it to make the security guarantees needed not only for safecoin but also for preventing anyone from taking over the network entirely. There are known attacks that the network is vulnerable to.
This really is just for folk that are super comfortable in the command line and messing with their routers to port forward, however it delivers the expectations that David outlined in last week’s update … I don’t know you go on holiday for a week and folk start to “convey our feelings and desires” and as none of us want to see David “slaughtered” by the community, so we thought we better share working Vault code for folks to test
This is the next phase of community testing - aimed squarely at techy community members and moves us yet another step forward. For the rest of us that prefer things simple and easy to use, the download and click to install completed MVP will not be far behind.
As shared in the postponed dev update post we published the Roadmap here. We have received a lot of good feedback and as @nicklambert has already stated the front-end guys have taken all the feedback onboard and are implementing improvements.
##Crust - uTP / API : Vinicius / Andrew
MVP (Minimum Viable Product) ready and feature frozen.
Looking beyond the MVP implementation, work has already started on the transport library which will eventually replace parts of Crust’s internals. @vinipsmaker has been working on adding heartbeat / keepalive messages to Crust.
MVP (Minimum Viable Product) ready and feature frozen, resources reallocated to other libraries.
##Vaults : Andreas / Brian / Fraser / Qi / David
After more advanced testing this week of the Vault code (in particular using a mock Crust interface to simplify testing and fault-finding) we’ve made some simplifications to the code.
These may persist in light of how well the testing is now going, but further development of the simplified code will be needed to allow the Vaults to handle all that will be asked of them. An alternative would be to look at working through the complexity that was revealed in the previous code.
Either way, it appears that the simplified code is likely to be robust enough for supporting the upcoming MVP. Current tests using the mock Crust are indicating that a small network can handle things like three Vaults within a close group all going offline simultaneously. The tests aren’t exactly full stress tests, but we are approaching the limits of what we would expect to be realistic scenarios and the Vaults are handling this well.
On the droplet Vault testnet (a 50 node network), testing has been going very well. In a test run earlier this week, Vaults successfully retained all data on 2433 churn events (1211 started and 1232 stopping) with an event taking place every 20 seconds. We are keen to continue gathering and begin sharing more of this type of data as the testing of Vaults expands into the community.
##Client - Launcher : Krishna / Spandan / Shankar
maidsafe_utilities logging was formatted to follow a standard JSON structure which will be sent to the log visualiser server. There is a little bit of work still left in Vaults logging to determine how an ID can be associated to Vaults which can then be used for identifying the Vaults. This ID can be used in the log visualiser server to analyse the logs of a Vault.
A Work In Progress Pull Request is raised for handling the ImmutableData operations. Once in place the library should ensure no Immutable Data chunks of size > 1 MB (currently) are stored on the network. Moreover this PR will help us to take the Low level API RFC discussions forward with snippets to showcase.
We tried a few samples with SSL integration for Launcher using a self-signed certificate, but things don’t seem to work as we would like. The first problem is that we need to accept the untrusted certificate and also the clients (JAVA, nodejs) should add the CA or allow all untrusted certificates which seems to be more hassle. The proxy doesn’t seem to redirect for a self-signed certificate. Thus we have decided to park this for now and will try incorporating other standards that can be added to the Launcher APIs. The changes will be detailed in a separate RFC later this week. Once the RFC is discussed and agreed, the changes detailed in the RFC must be implemented across all the APIs that are exposed so far.
To repeat the message from David last week, MaidSafe resources are now being much more focused into areas that will have a direct impact for App Devs and early SAFE adopters or as he put it “the people that matter”. So now we have working Vault code and an imminent MVP release, even the more reserved amongst us can “convey our feelings and desires” and begin to allow ourselves to get (a little bit) excited.
Thanks as always for your support; here is a link to the weekly transcript.