Hey guys,
We have finished the stabiliser sprint. This essentially was a small and feature light sprint which basically means that no additional features in any crate would be added. The main purpose of this sprint was to address all the TODO points regarding workarounds and designs that may have been done in haste in prior sprints. As a result the crates have much better error handling logic and test cases written for them.
@BenMS and @brian_s have made sure that churn handling was properly integrated into routing. This will allow the vault network to not lose data when other vaults join or leave - the very incidents which trigger churn. We ran 40 stable nodes and one additional node which kept refreshing every 5 seconds to simulate churn, for just over 43 hours. After tens of thousands of iterations, we saw the routing table still populated correctly and the network continuing to be stable. So a great result!
@Peter_Jankuliak and @vinipsmaker have worked and stabilised one important aspect in Crust - it can now connect to peers more than once. Another important thing to note is that Crust is now no longer blocking as async workaround for the blocking Rust-networking library has been put cleverly by the duo. It involves making a false connection initiated by a local socket to a blocking TCP acceptor/listener from a different thread so that it can be terminated gracefully. Hence it is a better programming paradigm and the user of Crust can expect a deterministic and graceful exit on demand.
@anon86652309 and @qi_ma made sure that churn handling, enabling the network to manage users turning their computers off and on, was fixed and properly tested in safe_vault. Also completed was the propagation of PUT error on low-balance to the Clients. So if the safecoin balance is not sufficient then the vaults would not allow the user to mutate the network (i.e., Put something on it). Instead an error would be returned so that the user has a chance to purchase more safecoins and reattempt the PUT.
Finally, the Client modules. Observer pattern for error handling and other miscellaneous async events was coded and stabilised. This allows the user of safe_client to tap into any one of the various categories of events and act upon it if additional actions are wanted over and above what the client module requires. It is much like boost::signals2 where the clean up of uninterested observers is done lazily during the next capture of an event. Also, previously safe_ffi restricted the retrieval of files to just the home directory for the service. That restriction was lifted and the users can now retrieve files from any node in the hierarchy of folders and directories starting from the service home directory.
Please note that the name of safe_client crate is planned to be changed. The new library name will be decided shortly and we will let you guys know when that is complete. We also are planning on making a stable branch for safe_ffi while the master would be the current working branch during development cycles.
To make sure that this sprint hasn’t introduced any regression, we tested the self-authentication example and it works well across 18 nodes, so lots of reason for optimism as we move into the next stage.
With that we now embark on our journey for the next sprint, Rust - 5, which will be a major feature-sprint for a lot of crates. We begin planning this week and will see RFC’s getting raised and discussed. Some of the high level objectives are as follows:
<1> uTP Hole Punching in crust - This will be our crux concept for peer-to-peer networking. Once fully in place it will allow the nodes (clients and vaults) to talk to each other beyond NAT boundaries and thus achieve decentralisation in a true sense in the public network. Community run nodes being part of the droplet network seems right around the corner !!
<2> Connection Management Implementation in Routing
<3> Messaging Infrastructure in safe_vault - This should be quite a fun deliverable, but is also a big one in itself. The messaging APIs for inter-node communication would be put in place. We might see the use of MPID (MaidSafe Public Identity) in this iteration. It would allow chat engines and clients to work on SAFE-Network.
<4> Messaging Infrastructure and Launcher implementation in the Client Modules - Again quite a lot of work might go into this. Launcher is expected to be the only application the users gives their credentials (PIN, Keyword and Password) to. Launcher would then authorise Apps on behalf of the user giving each one of them a separate environment (a directory) to work with. This not only prevents Apps from knowing the user credentials, but also removes the ability for them to tinker with folders/data of other Apps or user’s files and folders. Sharing of data between the Apps will be done in a controlled and monitored manner if so desired.
Please note that we have not yet concluded if all of these items will make it into the sprint, so this is not a final list by any means. We should be in a position to confirm the final list during next week’s update.
Also, it is planned to open GitHub issues and solutions to the community when rust-5 is underway. So anyone in the community interested can solve GitHub issues. If the solutions are satisfactory, the successful contributor will be awarded a bounty, similar to as explained here. We are working out a few details in the planning phase about the github issue task estimation values and indicating the same in issues accordingly. More info about this will be published once we have the answers in place.
I love updates!
Great work, thanks for the update.