MaidSafe Dev Update :safe: 17th May 2016 - TEST 3 - Update (20th May 13:30 BST) Now Complete

###Updated: 20th May 13:30 BST - TEST 3 now complete.
TEST 3 is now complete and the network is being brought down - big thank you to everyone who took part :sunglasses:.

Hi everyone,

Today we are happy to release SAFE Network - TEST 3.

TEST 3 incorporates improvements from the findings from TEST 2 and we really want to see how these code changes behave during this community testing phase and we hope to see the usual enthusiastic response and participation from the SAFE community. The focus for us in this iteration is primarily the performance and behaviour of the new Vaults (and the underlying libraries), when faced with real world networking / computation limitations. We are prioritising Routing traffic and taking aggressive action to drop all other traffic if a node cannot accommodate the network traffic as we expect - THIS WILL RESULT IN NON-ROUTING DATA LOSS - in this test that is exactly what we want to assess. This means that from the Client side we expect to see lost accounts, lost files, etc, but we should always have a Vault network that will scale and continue to accept new connections. So creating web rings (for example) within this iteration is probably not the best way to contribute. Running a Vault and testing out the Launcher and running the Demo Application will help create useful logs to analyse - just do not be surprised if you experience some data loss.

So for example, dropped messages will have error messages along the lines of; Insufficient upstream bandwidth. Dropped {} messages with priority >= {} these will only appear in the log file Node.log and not in the console. Messages prioritised 0 shall never be dropped, however, if messages with a lesser priority > 0 have been queued on a node for more than 60 seconds that node will drop all of these messages. This is why we expect to see some data loss in this iteration, as data messages are given a lower priority.

We are hosting one hundred droplets this time around with the idea being that the less nodes we host, the faster we can get to a situation of community nodes comprising the majority of the network. The varying physical limitations will mean we can confirm fixes, or see issues much faster.

###Get involved!

Please be aware this release will not allow older nodes or clients to connect to the network. You must use the new binaries to connect to the network.

This is a test network and comes with the normal risks of using pre-release software. Please also be aware that this is a test network and will be reset, wiping all data that is stored on it.

Expect unusual Launcher and Demo App behaviour / sudden disconnecting due to data loss - this test phase is focused on testing Crust and Routing

SAFE Vault binaries
SAFE Launcher binaries
SAFE Demonstration Application binaries

How to download and install SAFE Launcher
Using SAFE Launcher
How to download and install the SAFE Demo App
Using the SAFE Demo App

###Where should I report issues?

GitHub is the best place to report issues and bugs. Using GitHub will help us (MaidSafe) manage issues and prioritise work within the Dev team faster.

For SAFE Vault issues / bugs
For SAFE Launcher issues / bugs
For SAFE Demonstration Application issues / bugs

If you are not sure if it is a real bug and have a question, the forums #support category is a great place to post this. As we know this is a friendly and responsive community and someone will help you out.

As always - thanks in advance for participating in TEST 3, it really does help us out a lot.

If this test is successful we shall be looking to implement ACK in Routing (message acknowledgement mechanisms) which was getting tested internally today, but wasn’t quite ready for this release and MIO in Crust, there is more about each of these in the sections below. So in the next test release, we should see a scalable network without any data loss and improved performance.

We also have in the pipeline a way to restart networks while maintaining data persistence, which will help make future version changes a lot less painful for everyone. We have also started work on better accommodating nodes with differing capabilities in terms of bandwidth speeds and other physical limitations, we are calling these asymmetric nodes, so nodes will carry out tasks more suited to their capabilities.

I have made a minor change to the layout of the Dev Update to reflect the recent team changes structurally within MaidSafe the company. So on with the team updates…

##Crust, Core: Andrew, Spandan (tl) & Vinicius.

We are porting Crust to a completely async design. The blocking Crust right now (due to library limitation and other fundamental factors) spawns a huge number of threads. At least two per connection. Now that MIO is stable and available for all platforms, the new Crust will use an event driven MIO loop. This basically means we dramatically reduce the number of threads from close to two hundred when we have a large routing table size to merely one. This will give us better performance overall and much more resilient code. The blocking paradigm involved a lot of timeouts, daemon threads, poor multiplexing and wasted resource, but it was something we had to live until async libraries matured and became supported on all platforms.

Now that we have had a few community releases with existing code, we have clearer visibility regarding the way things work from the point of core MaidSafe libraries in the real world and are now implementing these changes in Crust which we believe will resolve a lot of the shortcomings of blocking design. It also means we need to port a few 3rd party libraries that Crust uses to be async as well e.g. IGD. Once complete we will have a vastly improved code base that does the right thing efficiently.

##Routing, Vaults: Adam, Andreas (tl), Fraser & Qi.

We finished the message priority implementation and are experimenting with message acknowledgement mechanisms that could greatly reduce the load on the network by removing a lot of redundant messages. We also managed to slightly reduce the number of threads in the current Crust implementation and are now running tests with these changes.

We also added a “–first” option to Vault to designate the first node of the network. No other nodes will attempt to start their own network now if they fail to join an existing one. If you wish to start your own network you should manually edit the config file (safe_vault.crust.config) and choose a unique network name "network_name": "test_network", then start the first node with the --first command line argument and the remaining nodes without that. If you do this you are not joining the SAFE Network TEST 3.

Meanwhile, @adam and @qi_ma are helping out with the ongoing effort to port Crust to MIO completely and thereby solve the thread count issue for good.

##Client : Krishna (tl), Ross, Scott & Shankar.

The initial version of the log visualiser was tested on the cloud with scaling enabled and the basic functionality seems to be working fine. There are few changes and improvements suggested by the Routing and Crust devs which will be enhanced later this week.

@scott and @shankar have been working on the website changes that are planned to be rolled out along with the alpha release. Internal tools have been put in place to enable easier testing of web related components, which will make @scott’s life a lot easier.

Since the helper tools have now been set up, we are planning to focus back on the safe_launcher. The safe_launcher doesn’t have a comprehensive test suite for running unit tests, so we are planning to address this issue along with the bugs that has been raised in the issues section of the safe_launcher repository. Setting up the CI for the launcher will be our priority for this week.

Thanks as always for your all of your support, here is a link to the weekly transcript.



Much appreciation to all the hard work!

Things are flying now!


In 2 seconds a routing table with 32 nodes, this is like real improvement!! Woow! Congrats to the devs with this release :thumbsup:



This is huge success. I am a developer too, also crypto user. <3 Your Project


account created :slight_smile: 2 vaults running - since the aim is to to loose connections and to hav insufficient speed let me just set up some nodes at 2 other pcs :open_mouth: :grin:


Same here! Around 5 seconds, it already picked up 32 nodes.

Awesome! Thank you for the hard work!

Now I can run my own mini safenet. Sweet! Does it support IPv6 address yet?


[safe_vault::personas::data_manager] Stats : Client Get requests received 0 ; Data stored - ID 17 - SD 9 - total 1124999 bytes


you bet you will !!! :wink:


Thanks this is the focus right now, the new crust mio loop will really improve this much more. This test though will lose vault data - so websites etc. will probably not work as expected. We are really wanting to get crust and routing tested here.

I hope next test will be even more stable here and then vault data can be properly stabilised as well. Very cool to be able to roll out these tests like this .


great :smiley:

superquick vault connected to the network here as well. Already storing data and getting client get requests :sunny:


2 nodes on a laptop with ubuntu 16.04 LTS, 2 nodes on a pc with ubuntu 14.04 LTS and 4 nodes on a larger pc with ubuntu 16.04 LTS gnome

took seconds to start =) just download and run - perfect as always :heart_eyes:


I7 2600k. Arch linux.

I am running one safe vault. It is taking between 18-40 percent CPU power. Yikes.

1 Like

I ran the DEMO App and managed to upload files. Deleting files didn´t work and now I can´t connect to the network. Is it down?

EDIT: I am stuck with “Initialising” in the Demo APP. I can connect to the page though, so the network is up for me.

EDIT2: Restarting the Demo APP didn´t help, I restarted the Launcher and things work out fine again.

1 Like

works fine here, deleting and adding.

1 Like

hmmm - laptop with i5 - nothing special - ubuntu 16.04 - 2 nodes

the amd-forgotten-whatsoever-8-core is at 4% max (4 vaults)


We expect a lot of this, vault data will not survive this test, unfortunately. Network should stay up though, that’s this test.


That’s what’s going to be solved by porting stuff from threads to async code (MIO).

1 Like

Still running blocking I/O in crust this week. Hopefully next week this won’t be the case

1 Like

So creating web rings (for example) within this iteration is probably not the best way to contribute.

Thanks for the heads up @Ross

There will be no SAFE Webring during testnet 3

That’s great actually… it means I can have more time to extend and test my remoteStorage web app code for SAFEpress instead :slight_smile:

And @Ross, you didn’t think you would get away with not updating us on ARM did you? Tut, tut! Come on, please tell :slight_smile:


On Windows 10/64 it’s doing great. My CPU isn’t going above 3% so far.