[Offline] Another day another testnet

Another day another testnet

We’ve got a 20 node testnet up. Each node has 50GB of space. But joins will be tempered by filling up that space

Goals

Once again we’re aiming to ensure that we’re not losing data until nodes are full - apart from the known case of registers hitting a limit. But we’re not going to allow unbridled churn as we did last testnet.

Like last time, we’re asking you to avoid larger uploads. We’re not enforcing it this time but please keep them under 10MB, so we can rule out large files being the cause of ansafe networks switch public
y errors.

Churn

After the last testnet, we’re back and limiting churn again. The limit is set to allow joins, but only after the first 20 nodes fill up. Which should let us test normal join conditions, but without having to go through a split.

Getting involved

We really want to make it as easy as possible for everyone to get involved, but understandably it’s a daunting prospect for some. Below are some detailed instructions for Linux, macOS and Windows. Follow these and it should work, or at least fail in an understandable way (probably router related). Whatever happens it won’t blow up your machine! :boom:

Two modes - upload/download and storing

You can help us with this testnet in two ways: (1) by uploading and downloading data; and (2) by offering your device as a storage node. If you’ve not done this before we’d recommend you start with (1) and move on to (2) if you feel inspired. We’ve provided instructions for both.

Uploading/downloading

To upload and download (PUT and GET) data to the test network, you just need to install the safe binary on your operating system. See the instructions below.


Linux and macOS

Installation

If you have run safe before you need to delete the old version.

# Delete the existing `~/.safe` directory.
rm -r ~/.safe

# Check that safe has been uninstalled
safe --version # --> should give a 'not found' message

If not it may be installed in /usr/local/bin:

sudo rm /usr/local/bin/safe
[password]

To keep things simple, we’re now going to recommend running the installer as the root user. It puts the binary at /usr/local/bin, which is always on PATH on any Linux/macOS distribution. For future updates the installer will just overwrite it, so you won’t need to clear anything as an additional step.

On Linux:

curl -so- https://raw.githubusercontent.com/maidsafe/safe_network/master/resources/scripts/install.sh | sudo bash

safe --version # should be 0.68.2

safe networks add public https://sn-node.s3.eu-west-2.amazonaws.com/testnet_tool/public/network-contacts
safe networks switch public

On macOS:

# switch to sudo shell
sudo su

# run install script
curl -so- https://raw.githubusercontent.com/maidsafe/safe_network/master/resources/scripts/install.sh | bash

# return to your own user profile
exit

safe --version # should be 0.68.2

safe networks add public https://sn-node.s3.eu-west-2.amazonaws.com/testnet_tool/public/network-contacts
safe networks switch public

All subsequent instructions are the same for both operating systems.

Uploading

Linux upload both

Uploading a file
# Upload a file in your current directory
safe files put [filename]
e.g. safe files put examplePic.jpg

Uploading a container (a directory plus contents)
Choose a directory (e.g. /home/[yourUserName]/Pictures) or make a new one and add some files. Then cd to the parent directory (e.g. /home/[yourUserName]/) and run:
safe files put ./[directory] --recursive
e.g. safe files put ./Pictures --recursive

Downloading

Linux download

Downloading a file
Use safe cat safe://[address] > filename.txt to download a file:

Try running this!

safe cat safe://hygoygym7tsj5hhyyykd1aqpw3djxea6om6xku568ahm7hy7gfn6q5gy7xr  > coast.jpg

Downloading container (directory plus contents)
safe files get safe://[address?v=version]

# For example this will download the container to your current directory. This is just an example, it's not live data)

safe files get safe://hyryyryikpk16oxay7wa1midtbkibxoec6eg4fsgd853gx5xuhs19ujmgzwnra?v=hc7xjae7f8o96xk9446gzyy13j9z7es47rpqdcu81iap61jdcu6no

Running a storage node

To run a storage node (an adult) you need at least 50GB free disk space plus a router that allows you to connect.

safe node install

safe node bin-version # should be 0.72.6

Now download the testnet config file and switch to that network by running:
safe networks add public https://sn-node.s3.eu-west-2.amazonaws.com/testnet_tool/public/network-contacts && safe networks switch public

Now try to connect to the network with:
RUST_LOG=sn_node safe node join --network-name public

If that gives an error try:
RUST_LOG=sn_node safe node join --network-name public --skip-auto-port-forwarding

If that doesn’t work it may be you can’t join from that machine / router.

If successful, the folder ~/.safe/node/local_node will start to fill up with new folders and chunks.


Windows

Installation

Press the Windows key or click on the Start button and type “Powershell”. Then right click on the Windows Powershell entry on the menu and select Run as administrator.

Paste the following text into the terminal by right clicking on the bar at the top of the window, then select edit → paste:

Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://raw.githubusercontent.com/maidsafe/safe_network/main/resources/scripts/install.ps1'))

Now hit enter to run the install. If you have a previously installed binary, you’ll be asked if you want to overwrite it. Say yes. Now exit the session.

Start a new session without administrator privileges and now run safe --version. It should be at 0.68.2.
Then add the public network and switch to it.

safe networks add public https://sn-node.s3.eu-west-2.amazonaws.com/testnet_tool/public/network-contacts
safe networks switch public

Downloading


Downloading files
safe cat safe://address > filename
e.g.
safe cat safe://hygoygym7tsj5hhyyykd1aqpw3djxea6om6xku568ahm7hy7gfn6q5gy7xr > coast.jpg
Try it!

Downloading container (directory plus contents)
# For example this will download the container to your current directory. This is just an example, it's not live data)

safe files get safe://hyryyryikpk16oxay7wa1midtbkibxoec6eg4fsgd853gx5xuhs19ujmgzwnra?v=hc7xjae7f8o96xk9446gzyy13j9z7es47rpqdcu81iap61jdcu6no

Uploading


Uploading file
# Upload a file in your current directory
safe files put [filename]

e.g.
safe files put examplePic.jpg

Uploading a container (directory plus contents)
Choose a directory (e.g. /home/[yourUserName]/Pictures) or make a new one and add some files.

Then cd to the parent directory (e.g. C:\Users\[your username]\) and run:
safe files put .\[directory] --recursive
e.g.
safe files put .\Pictures --recursive

Running a storage node

To run a storage node (an adult) you need at least 50GB free disk space plus a router that allows you to connect.

From your Powershell session, run safe node install to get the latest version of the node. Then use safe node-bin version to check you are on 0.72.6.

Download the testnet config file and switch to that network by running:

safe networks add public https://sn-node.s3.eu-west-2.amazonaws.com/testnet_tool/public/network-contacts
safe networks switch public

Now try to connect to the network with:
$env:RUST_LOG = 'sn_node'; safe node join --network-name public

If there is an error, try:
$env:RUST_LOG = 'sn_node'; safe node join --network-name public --skip-auto-port-forwarding

If that doesn’t work it may be you can’t join from that machine / router.

If successful the folder C:\Users\[your username]\.safe\node\local_node will start to fill up with new folders and chunks.


Test data

This is what we’re using to verify data-storage just now.

safe://hygoygym7tsj5hhyyykd1aqpw3djxea6om6xku568ahm7hy7gfn6q5gy7xr 
safe://hygoygym19bdbzg6jakfzkmacxurpkbuphgsxi6af3aecugfqjfaq66shww
safe://hygoygyq1xiofs8kx11jwkcguzuocd1xa5qp767bjjz39yrh44eo1mjkjch
safe://hygoygypnrqirzo7r5w464mnda8s6sdpatyrco5pda1w8nh5appa3ycb98h
safe://hygoygykajkeropyam6yihj9exdhn5o4b1pbjbeet4bazqo8m47ga3pw8mo 
safe://hygoygyp8c1wy7ijrpnesz9cj7enf4ms9oxhmp5m4r4yw4pjcemgywekf9r
safe://hygoygye13iznm1yxzpxp9seh6j8weh84c5ib3fq9mxozz4h6hb95o3uduo
safe://hygoygycc11b4pmrcb97ro6ro3wxh3cfrgi7ctbiwyya8mkktq1j35ysmzh
safe://hygoygykzkguckqygm985f5jrhcw87wiskbc97bottab47tq6fm9zm85f7r
safe://hygoygyewnaj8o7za3qycckfy95o5j9gex6s3b8jygr9j48aiyzthc36ffo 
safe://hygoygykime3s7rup1ezeiqeqo1zy1o4bqmckewhriun889c64n4umxcyih
safe://hygoygyc98c1ozh1cn9edxeworctuagt465zyybr61m8judfry4a4zycf8c
safe://hygoygyqcq8bift7urnftiqxzn7d877ckwtk85haawg7oigm3bhn3r1dx7e 
safe://hygoygyk7dzmcqxm3q4np87unu6tb9d8qt44fowxgqrhjy1xs6xgt69hade 
safe://hygoygymq98z86j1oagpctmg7pnixz94ckekpgytu1jkb45du8xw49qrr4w 
safe://hygoygypoq7y31uedr7c6q6e3jaxhd94id3r8bwn7od3f79hgdj7xbqibjr 
safe://hygoygykw8iw7wt9o3c7w36hi5xc1coyjdmdfm6dima3y9ehefxipxkwg3w 
safe://hygoygyc67ozi6m9i9o8xq5wti7ysa4g6nodxetmdgw64g5pw1qmucheeac 
safe://hygoygyqsxzn4p6mjx7ggj98pqc4he53xk1c54kw3hq1967n4yga1nbo3iw 
safe://hygoygyxojetcr5553xfyjnfoh3ywge7grs8d7z66e8nf678stecagdkomw 
safe://hygoygyqf4au3yf7ouyi7imu1hty5qrsjppy1mj5uba5tahnxkmocys5g7h 
safe://hygoygyx41a9samc35ep6h7y6zwku5ess3377f8awtuwn44r1pc5w3ragky 
safe://hygoygyej97u6irwmgq1od4s8m6h8epbkgmmwqtghhhy1hbcsntizpjmh3c 
safe://hygoygypn9urw4mhoqfabnke71e8usekjeg8m54t1fkoeigdyr5srgsqoia
32 Likes

First joining nodes now :slight_smile:

10 Likes

Again, this should be done before trying to up/download.

3 Likes

Just edited to add that

4 Likes
willie@gagarin:~/projects/maidsafe/safe_network$ safe files put ~/cock01.jpg 
FilesContainer created at: "safe://hyryyrywc5mzxqytriik7oiws7stww7twsyz1i5duriobidxncqrcj1wxjhnra?v=hkjuuyxyicggackyqjdckgou8ajwdfbhzzehfj7equwpeonsqz3xy"
+---+-------------------------+--------------------------------------------------------------------+
| + | /home/willie/cock01.jpg | safe://hygoygyqzr1t99g38e5orf6o6ts4cja8ck7wo1hnn1rqcte4b19shnpbqeh |
+---+-------------------------+--------------------------------------------------------------------+

the network name is public

Can I make a plea that all future testnets are named with a unique identifier or timestamp?
IMHO public_161222 would make comparing log entries etc a LOT easier and IF we get simultaneous testnets will avoid much confusion.

EDIT: retreived cock01.jpg to one of my Hetzner nodes

safester@ubuntu-2gb-nbg1-1:~$ safe cat   safe://hygoygyqzr1t99g38e5orf6o6ts4cja8ck7wo1hnn1rqcte4b19shnpbqeh  >cock01.jpg
safester@ubuntu-2gb-nbg1-1:~$ ll
total 172
drwxr-x--- 3 safester safester   4096 Dec 16 17:34 ./
drwxr-xr-x 4 root     root       4096 Dec 16 17:06 ../
-rw------- 1 safester safester    462 Dec 16 17:32 .bash_history
-rw-r--r-- 1 safester safester    220 Jan  6  2022 .bash_logout
-rw-r--r-- 1 safester safester   3814 Dec 16 17:32 .bashrc
-rw-r--r-- 1 safester safester      0 Dec 11 04:22 .cloud-locale-test.skip
-rw-rw-r-- 1 safester safester 146127 Dec 16 17:34 cock01.jpg
-rw-r--r-- 1 safester safester    807 Jan  6  2022 .profile
drwxrwxr-x 4 safester safester   4096 Dec 16 17:31 .safe/
-rw-r--r-- 1 safester safester      0 Dec 16 17:10 .sudo_as_admin_successful

lets see how that does as an Adult…

7 Likes

We’re going to put the node version in the nets in future.

We tried to do it with this one actually, but had some problems with the setup and scripts not being able to parse things correctly.

9 Likes

This one looks faster. Test-folder same as in the “wild testnet” - nested folder of 2800 files, all together 580MB

Last time upload took 22 minutes, in this testnet only 12m57,423s.
Download around 5m30s before, now 4m48 on the first test.

13 Likes

Coming to you from the middle of nowhere in Alabama I present to you commitment :joy:

safe cat safe://hygoygyk8kr9xnfmbynkb9yc6gun1k735s3zoy4bk734uw1eohst4bbk8zw > commitment.jpg

8 Likes

safe files cat should be safe cat - I’m always doing that too.

6 Likes

That’s dedication for ye :slight_smile: Thanks Josh

11 Likes

safe cat safe://hygoygyk8kr9xnfmbynkb9yc6gun1k735s3zoy4bk734uw1eohst4bbk8zw > commitment.jpg works better :slight_smile:
I keep doing the same - adding an extraneous “files” to the cat command
– oops beaten to it

5 Likes

Looking good at first pass!

First domain ?
safe://uni2 :slight_smile:

Unclear the reason to suggest sudo… and I don’t like that from previous confusions that spawned across user accounts and symbolic links to other partitions for .safe. So, it seems to work, at least for non-node, as just | bash for the install.sh
curl -so- https://raw.githubusercontent.com/maidsafe/safe_network/master/resources/scripts/install.sh | bash

Noting [nrs create] seems to have switched to [nrs register]
and ?version is active not just integers :+1:

That nrs creation is misbehaving on user error but perhaps that control is where the coinage is active. Still it’s odd that error leaves name then unavaliable as taken, rather than allowing another try at it… perhaps in future that will require add rather than register??

$ safe nrs register uni --link safe://hy8oyryjeza8oaoc9zxshwi9xzjy6gyimg7sxiu1568r3yxiosfsbiir44w?v=0
Error: 
   0: InvalidInput: v param could not be parsed as VersionHash. invalid: '0'

Location:
   sn_cli/src/subcommands/helpers.rs:275

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
$ safe nrs register uni --link safe://hyryyry1w7zxqs1gy43cfye3khj8kfnj6agukudwor7mstmyr1z5ijktuewnra?v=h7irf4oowfgeaq148s6sp1nhpg8ihuoctq16kr3oyc3t5h9swhtpo
Error: 
   0: Could not register topname uni. That name is already taken.
   1: NrsNameAlreadyExists: uni

Location:
   /rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/convert/mod.rs:726

Suggestion: Try the command again with a different name.

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
expand for commandlist

safe
safe cat
safe config
safe config add
safe config add network
safe config clear
safe config remove
safe config remove network
safe dog
safe files
safe files add
safe files get
safe files ls
safe files put
safe files rm
safe files sync
safe files tree
safe keys
safe keys create
safe keys show
safe networks
safe networks add
safe networks check
safe networks remove
safe networks sections
safe networks switch
safe node
safe node bin-version
safe node install
safe node join
safe node killall
safe node run-baby-fleming
safe node update
safe nrs
safe nrs add
safe nrs register
safe nrs remove
safe setup
safe setup completions
safe update
safe wallet
safe wallet balance
safe wallet create
safe wallet deposit
safe wallet reissue
safe xorurl
safe xorurl decode
safe xorurl pk

from safe setup completions | grep '_describe -t commands' | sed "s/ _describe -t commands '//" | sed 's/commands.*//' | sort | grep -v 'help'

2 Likes

My node joined about five minutes ago. No sign of chunks but regular bursts of log messages all of which finish with a handfull of these errors:

[2022-12-16T18:12:39.505322Z ERROR sn_node::comm] Sending message (msg_id: MsgId(9b89..824d)) to 142.93.38.111:35572 (name 38608b(00111000)..) failed, as we've reached maximum retries

I joined without the skip command option/flag as that didn’t work yesterday (probably for other reasons, but I think I was using it in previous testnets) but as the first suggested join command worked I assume it is ok?

EDIT: this is a cloud node.

3 Likes

That’s the old type of format we used to have for versioning, you need the VersionHash obtained when creating your FilesContainer as the version rather than 0,1,2…, e.g.:
safe://hyryyryic19xruuyzfikthcjfhwtfjtujjdyequwkyfb9u4uonrytg5hpscnra?v=hadrie5t9dqb7qonzyw337xj5gakagd5cxjssasqun7z65gtmt8ky

6 Likes

Yes worked for safe://uni2 with that.

The download progress is nice to see; could do with similar for upload progress.

I’ve uploaded a file
available with
safe files get safe://hyryyryu8g8eb5xhzew88w5wm4zd7urok5uzf9mthe3bgy19ngfu9tw8kthnra?v=hp4tbjcpebe8zt1tp97h84ep5hmtj5j9p64geqcwidj1dzkzkw3fo

but cannot figure the use of safe files rm to try that… the error messages perhaps on all commands later need a simple foobar example for encouraging the basics that should be intuitive.

2 Likes

and an extra :+1: for that

1 Like

not sure if something is wrong with my nodes it first said not taking new nodes then started erroring with connect to the endpoint: Address in use

[2022-12-16T18:21:50.792765Z ERROR sn_node::node::bootstrap::join] Network is set to not taking any new joining node, try join later.
[2022-12-16T18:21:50.792939Z ERROR sn_node::comm::link] Error sending out from link... We have 1 open connections to node "143.110.168.239:1200093825016177680".
[2022-12-16T18:21:50.792968Z ERROR sn_node::comm::peer_session] the error on send :Send(ConnectionLost(Closed(Local)))
[2022-12-16T18:21:50.793029Z ERROR sn_node::comm::link] Error sending out from link... We have 1 open connections to node "104.248.167.4:4367893825016945648".
[2022-12-16T18:21:50.793048Z ERROR sn_node::comm::peer_session] the error on send :Send(ConnectionLost(Closed(Local)))
[2022-12-16T18:21:50.793447Z ERROR sn_node::comm::listener] Error pushing msg MsgId(007d..be24) onto internal msg handling channel: SendError(MsgFromPeer { sender: Peer { n>
[2022-12-16T18:21:50.844477Z ERROR sn_node::comm::peer_session] Error when attempting to send to peer. Job will be reenqueued for another attempt after a small timeout
[2022-12-16T18:21:50.844968Z ERROR sn_node::comm::peer_session] Error when attempting to send to peer. Job will be reenqueued for another attempt after a small timeout
[2022-12-16T18:21:50.896684Z ERROR sn_node::comm::peer_session] Error when attempting to send to peer. Job will be reenqueued for another attempt after a small timeout
[2022-12-16T18:21:50.897171Z ERROR sn_node::comm::peer_session] Error when attempting to send to peer. Job will be reenqueued for another attempt after a small timeout
[2022-12-16T18:21:50.948525Z ERROR sn_node::comm::peer_session] Error when attempting to send to peer. Job will be reenqueued for another attempt after a small timeout
[2022-12-16T18:21:50.949098Z ERROR sn_node::comm::peer_session] Error when attempting to send to peer. Job will be reenqueued for another attempt after a small timeout
[2022-12-16T18:21:51.048344Z ERROR sn_node::comm] Sending message (msg_id: MsgId(af0a..6f1d)) to 143.110.168.239:12000 (name 56e8c3(01010110)..) failed, as we've reached ma>
[2022-12-16T18:21:51.048420Z ERROR sn_node::comm] Sending message (msg_id: MsgId(af0a..6f1d)) to 104.248.167.4:43678 (name a1dcf8(10100001)..) failed, as we've reached maxi>
[2022-12-16T18:22:20.794884Z ERROR sn_node] Err(
   0: ^[[91mCannot connect to the endpoint: Address in use (os error 98)^[[0m
   1: ^[[91mAddress in use (os error 98)^[[0m

Location:
   ^[[35m/rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/convert/mod.rs^[[0m:^[[35m726^[[0m

^[[96mSuggestion^[[0m: If this is the first node on the network pass the local address to be used using --first
   Cannot start node. Node log path: /home/ubuntu/.safe/node/local_node

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.)
[2022-12-16T18:22:21.796465Z ERROR sn_node] Err(
   0: ^[[91mCannot connect to the endpoint: Address in use (os error 98)^[[0m
   1: ^[[91mAddress in use (os error 98)^[[0m

Location:
   ^[[35m/rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/convert/mod.rs^[[0m:^[[35m726^[[0m

^[[96mSuggestion^[[0m: If this is the first node on the network pass the local address to be used using --first
   Cannot start node. Node log path: /home/ubuntu/.safe/node/local_node

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.)
[2022-12-16T18:22:22.798890Z ERROR sn_node] Err(
   0: ^[[91mCannot connect to the endpoint: Address in use (os error 98)^[[0m
   1: ^[[91mAddress in use (os error 98)^[[0m

Location:
   ^[[35m/rustc/69f9c33d71c871fc16ac445211281c6e7a340943/library/core/src/convert/mod.rs^[[0m:^[[35m726^[[0m

^[[96mSuggestion^[[0m: If this is the first node on the network pass the local address to be used using --first
   Cannot start node. Node log path: /home/ubuntu/.safe/node/local_node

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.)
[2022-12-16T18:22:23.800805Z ERROR sn_node] Err(
   0: ^[[91mCannot connect to the endpoint: Address in use (os error 98)^[[0m
   1: ^[[91mAddress in use (os error 98)^[[0m

2 Likes

this means that the port is already in use

on debian based systems:

sudo apt install iproute2

To check open ports on your Debian system, issue the following command in the Terminal:

sudo ss -tulpn

more info https://vitux.com/find-open-ports-on-debian/

3 Likes

safe://logos

cat will download a file as expected
safe cat safe://hygoygyx5a1o7kh6ofp5cjp7ujnyipg56e36zr13yy5hq9m4794yksq3uyy > ./safe_wallpaper.jpg

but get is wanting the filescontainer… unclear why the difference

$ safe files get safe://hygoygyx5a1o7kh6ofp5cjp7ujnyipg56e36zr13yy5hq9m4794yksq3uyy
Error: 
   0: You can target files only by providing a FilesContainer with the file's path

Location:
   sn_cli/src/subcommands/files_get.rs:340

and I get AccessDenied when trying variations to move a files - looking to add the wallpaper to the logos

$ safe files add safe://hygoygyx5a1o7kh6ofp5cjp7ujnyipg56e36zr13yy5hq9m4794yksq3uyy safe://logos/safe-wallpaper.jpg
Error: 
   0: AccessDenied: Couldn't write data on Register found at "safe://hyryyryu5pmb5kwbzc735ydbsyu6i18nqwohfp155tt9b569d1b1dihm54enra/safe-wallpaper.jpg"

Location:
   sn_cli/src/subcommands/files.rs:317

but perhaps that is just testnet and ownership limits.

:smile: happy … I can’t seem to break it… will come back tomorrow to see if its still there.

2 Likes