After last week’s successful testnet we’re launching a new one, this time allowing nodes to join.
Note: This testnet is now offline. Latest one is here
Last week’s testnet allowed us to track down a bug where we’re not handling registers (directories) that cross a size limit (thanks @stout77 and @neik). We had always imagined that you’d need to pay every X entries for registers, so this issue has just brought that question to the fore. To solve this, we’ve added a fix which simply limits a register to 1024 entries for now, which will give an error when it fails. Down the line we imagine there will be app layer logic to enable linking together registers so that this size limit can be overcome.
We now want to test some other changes we’ve made around messaging, stripping as much as possible out of qp2p, which we believe is causing some issues with dropped communications, and leaving it all to quinn. This is in a testable state now so we’re keen to see what happens.
Once again we have removed split functionality, so we’re operating with a single section, but this time you’ll be able to join with your own nodes if you fancy it. The more testers the merrier, so dive on in, the water’s lovely!
Well we think it’s lovely but we’re well aware that for many of you the water may seem shark-infested or otherwise unwelcoming. We want to understand what the sharks are and how to remove them, so please take our anonymous poll at the end of the update.
General progress
It’s been all hands on testnet, but @oetyng has found some time to scope out systematic tests for two different aspects of our network; data retention in presence of churn and network health in presence of message drops.
Mostafa and @davidrusu continue to make good progress in the world of consensus.
With the first major qp2p update in, @bzee is looking at removing our custom wire format, which may simplify things if we go all in on the bidi
flows.
@jimcollinson is working through some onboarding flows for data, such as where files might be transferred from a centralised service to a user-sovereign one, likely a very common use case.
And the entire team has been experimenting with ChatGTP - it’s not perfect by any means but it actually comes up with some pretty good coding suggestions. Minds have been duly blown.
Another week another testnet
We’ve got a 27 node testnet up. Each node has 50GB of space, and our single big section can go up to 1,000 nodes or more.
Goals
Once again we’re aiming to ensure that we’re not losing data until nodes are full - apart from the known case of registers hitting a limit. Our internal tests have shown data sticking around quite happily as long as we’re not filling up nodes. Full nodes will herald the end of this run, should we get there. We’re continuing to study client->elder->adult timeouts too. We want to see how relevant those values are under load.
Like last time, we’re asking you to avoid larger uploads. We’re not enforcing it this time but please keep them under 10MB, so we can rule out large files being the cause of any errors.
Churn
All info is good info, as far as we’re concerned, but if you want to join as a storage node please leave it running for as long as possible .
The stability of data will be related to the amount of churn that takes place. Right now we have four copies of each data, so we’ll also be seeing how that fares here too.
We’re testing the waters here, though ideally we don’t want this one to be brought down by churn. Data replication bugs (moving data to a new home when one goes down) will be the topic of a future testnet.
Getting involved
We really want to make it as easy as possible for everyone to get involved, but understandably it’s a daunting prospect for some. Below are some detailed instructions for Linux, macOS and Windows. Follow these and it should work, or at least fail in an understandable way (probably router related). Whatever happens it won’t blow up your machine!
Two modes - upload/download and storing
You can help us with this testnet in two ways: (1) by uploading and downloading data; and (2) by offering your device as a storage node. If you’ve not done this before we’d recommend you start with (1) and move on to (2) if you feel inspired. We’ve provided instructions for both.
Uploading/downloading
To upload and download (PUT and GET) data to the test network, you just need to install the safe
binary on your operating system. See the instructions below.
The bad news…
First the bad news. This is bleeding edge stuff and we don’t yet have a nice GUI to help you upload and download data.
And the good news
But using the command line isn’t too hard. Mostly it’s just copy and paste. And for the folder creation/deletion tasks you can always use a GUI file manager, of course.
Linux and macOS
Installation
If you have run safe before you need to delete the old version.
# Delete the existing `~/.safe` directory.
rm -r ~/.safe
# Check that safe has been uninstalled
safe --version # --> should give a 'not found' message
If not it may be installed in /usr/local/bin
:
sudo rm /usr/local/bin/safe
[password]
To keep things simple, we’re now going to recommend running the installer as the root user. It puts the binary at /usr/local/bin
, which is always on PATH
on any Linux/macOS distribution. For future updates the installer will just overwrite it, so you won’t need to clear anything as an additional step.
On Linux:
curl -so- https://raw.githubusercontent.com/maidsafe/safe_network/master/resources/scripts/install.sh | sudo bash
safe networks add main2 https://sn-node.s3.eu-west-2.amazonaws.com/testnet_tool/main2/network-contacts
safe networks switch main2
safe --version # should be 0.68.0
On macOS:
# switch to sudo shell
sudo su
# run install script
curl -so- https://raw.githubusercontent.com/maidsafe/safe_network/master/resources/scripts/install.sh | bash
# return to your own user profile
exit
safe networks add main2 https://sn-node.s3.eu-west-2.amazonaws.com/testnet_tool/main2/network-contacts
safe networks switch main2
safe --version # should be 0.68.0
All subsequent instructions are the same for both operating systems.
Uploading
Uploading a file
# Upload a file in your current directory
safe files put [filename]
e.g. safe files put examplePic.jpg
Uploading a container (a directory plus contents)
Choose a directory (e.g. /home/[yourUserName]/Pictures
) or make a new one and add some files. Then cd to the parent directory (e.g. /home/[yourUserName]/
) and run:
safe files put ./[directory] --recursive
e.g. safe files put ./Pictures --recursive
Downloading
Downloading a file
Use safe cat safe://[address] > filename.txt
to download a file:
safe cat safe://hygoygym7tsj5hhyyykd1aqpw3djxea6om6xku568ahm7hy7gfn6q5gy7xr > coast.jpg
Downloading container (directory plus contents)
safe files get safe://[address?v=version]
# For example this will download the container to your current directory. This is just an example, it's not live data)
safe files get safe://hyryyryikpk16oxay7wa1midtbkibxoec6eg4fsgd853gx5xuhs19ujmgzwnra?v=hc7xjae7f8o96xk9446gzyy13j9z7es47rpqdcu81iap61jdcu6no
Running a storage node
To run a storage node (an adult) you need at least 50GB free disk space plus a router that allows you to connect.
safe node install
safe node bin-version # should be 0.72.2
Now download the testnet config file and switch to that network by running:
safe networks add main https://sn-node.s3.eu-west-2.amazonaws.com/testnet_tool/main/network-contacts && safe networks switch main
Now try to connect to the network with:
RUST_LOG=sn_node safe node join --network-name main
If that gives an error try:
RUST_LOG=sn_node safe node join --network-name main --skip-auto-port-forwarding
If that doesn’t work it may be you can’t join from that machine / router.
If successful the folder ~/.safe/node/local_node
will start to fill up with new folders and chunks.
Windows
Installation
Press the Windows key or click on the Start button and type “Powershell”. Then right click on the Windows Powershell
entry on the menu and select Run as administrator
.
Paste the following text into the terminal by right clicking on the bar at the top of the window, then select edit → paste:
Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://raw.githubusercontent.com/maidsafe/safe_network/main/resources/scripts/install.ps1'))
Now hit enter to run the install. If you have a previously installed binary, you’ll be asked if you want to overwrite it. Say yes. Now exit the session.
Start a new session without administrator privileges and now run safe --version
. It should be at 0.68.0. Then:
safe networks add main2 https://sn-node.s3.eu-west-2.amazonaws.com/testnet_tool/main2/network-contacts
safe networks switch main2
safe --version # should be 0.68.0
Downloading
Downloading files
safe cat safe://address > filename
e.g.
safe cat safe://hygoygym7tsj5hhyyykd1aqpw3djxea6om6xku568ahm7hy7gfn6q5gy7xr > coast.jpg
Downloading container (directory plus contents)
# For example this will download the container to your current directory. This is just an example, it's not live data)
safe files get safe://hyryyryikpk16oxay7wa1midtbkibxoec6eg4fsgd853gx5xuhs19ujmgzwnra?v=hc7xjae7f8o96xk9446gzyy13j9z7es47rpqdcu81iap61jdcu6no
Uploading
Uploading file
# Upload a file in your current directory
safe files put [filename]
e.g.
safe files put examplePic.jpg
Uploading a container (directory plus contents)
Choose a directory (e.g. /home/[yourUserName]/Picture
s) or make a new one and add some files.
Then cd to the parent directory (e.g. C:\Users\[your username]\
) and run:
safe files put .\[directory] --recursive
e.g.
safe files put .\Pictures --recursive
Running a storage node
To run a storage node (an adult) you need at least 50GB free disk space plus a router that allows you to connect.
From your Powershell session, run safe node install
to get the latest version of the node. Then use safe node-bin version
to check you are on 0.72.2.
Download the testnet config file and switch to that network by running:
safe networks add main https://sn-node.s3.eu-west-2.amazonaws.com/testnet_tool/main/network-contacts
safe networks switch main
Now try to connect to the network with:
$env:RUST_LOG = 'sn_node'; safe node join --network-name main
If there is an error, try:
$env:RUST_LOG = 'sn_node'; safe node join --network-name main --skip-auto-port-forwarding
If that doesn’t work it may be you can’t join from that machine / router.
If successful the folder C:\Users\[your username]\.safe\node\local_node
will start to fill up with new folders and chunks.
Test data
To follow
Poll time
We really hope you’ll give this a try. If you get stuck there’s plenty of folks who’ll be more than willing to help out.
In the meantime we’ve a couple of poll questions to help us gauge what we can do to increase involvement.
- I’m not comfortable with the command line
- Too busy to get my head around it
- I’m not at all techie but like seeing other people have a go
- I think I’d need a lot of help and don’t want to be a burden
- Nothing - I’m happy to pitch in when I can
0 voters
- Video walkthroughs
- More regular testnets so I know when they are
- More structured feedback from the team
- GUI so I don’t have to use the command line
- Podcasts / audio
- Clearer goals for each testnet
- Clearer instructions for each testnet
- Something else - please expand below
- Nothing - testing software’s really not my thing!
- Nothing - I’m already involved
0 voters
Useful Links
Feel free to reply below with links to translations of this dev update and moderators will add them here:
Russian ;
German ;
Spanish ;
French;
Bulgarian
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!