Joshnet [May 4th Testnet 2023 ; Offline]

It took a bit longer than we might have hoped, but here we have our first public testnet of the Safe Network code layered atop libp2p. :tada:

At the moment we have 25 nodes up, with around 1gb of storage available to each of them.

Running things

The Peer Argument

Every node has what we refer to as a Multiaddr, and looks like this:

/ip4/178.128.166.245/udp/36572/quic-v1/p2p/12D3KooWPuaKwn7rFNn4oj1ABDpikVZ7scHs29ZNCHBveLHgCmak

This address includes both the protocol and identifier of the node. We can pass this address to the safe CLI, leading it to contact that node and get a view of the network.

(And this is a real address you can use to contact the joshnet)

Using the Client

To put/get files you’ll need to use the safe client. Which you can grab for your platform from github.

Use is fairly simple at the moment. For example, to upload a directory to the network, it looks like this:

$ safe --peer=/ip4/178.128.166.245/udp/36572/quic-v1/p2p/12D3KooWPuaKwn7rFNn4oj1ABDpikVZ7scHs29ZNCHBveLHgCmak files upload -- <dir>

And to download the same uploaded files inside the directory back:

$ safe --peer=/ip4/178.128.166.245/udp/36572/quic-v1/p2p/12D3KooWPuaKwn7rFNn4oj1ABDpikVZ7scHs29ZNCHBveLHgCmak  files download

Note: Locally, uploaded file addessses are stored and then automatically downloaded with the above command, files will be put under ~/.safe/client/downloaded_files folder.

How to share files and download them?

When you upload you will also eg, see:

Successfully stored file "testnet.exe" to 332f45ae9482a8e03b5d5579f15fd9c8806fe4a2b30da2bc6e5ebb9a98fe4095

This wee hex string allows you to easily share files (without having to have the addresses locally).

safe --peer=/ip4/178.128.166.245/udp/36572/quic-v1/p2p/12D3KooWPuaKwn7rFNn4oj1ABDpikVZ7scHs29ZNCHBveLHgCmak files download -- testnet.exe 332f45ae9482a8e03b5d5579f15fd9c8806fe4a2b30da2bc6e5ebb9a98fe4095

(here you can change the target filename to whatever you want.)

Running a Node

The node contacts an initial peer similarly to the client, by passing it in through --peer. It also needs a directory specified for the data it stores. And it’s a good idea to keep the logs in a directory.

$ safenode --root-dir=/tmp/safenode --log-dir=/tmp/safenode --peer=/ip4/178.128.166.245/udp/36572/quic-v1/p2p/12D3KooWPuaKwn7rFNn4oj1ABDpikVZ7scHs29ZNCHBveLHgCmak

Your mileage may vary with the node. We still don’t have the full nat traversal setup in place. So it’s a try it and see situation. But all nodes should be able to join the network with relative speed (compared to the old setup), if they are capable of being dialled!

Transfers

At the moment, while some limited transfer functionality may well be possible, we’re not focusing on that at all for this testnet. There’s still some work to be done establishing genesis and faucet procedures, so we’ll be looking at that in an upcoming testnet.

Testing

All client cmds will run and output logs to a temporary file. That file location is output in the cmd itself to keep things simple.

Right now, we’re looking purely at chunk storage and longevity, as well as the ability for folk to start nodes. (It’s unclear to us how many folk might actually be able to particiapte as yet).

Data is still stored in node memory, so that may well climb over time (up to its 1gb limit). So how much we can store and how fast this goes will likely depend on folk starting more node (more so than prev testnets where things were perhaps more limited).

So let’s see how long this lasts and hopefully learn something along the way! :muscle: :bowing_man:

48 Likes

First now to read

21 Likes

Amazing! I was kind of hoping this wouldn’t happen this week because I’m mega busy but I’ll try to get at least a client setup and maybe a nice as well.

23 Likes

Fantastic news well done team and ill be along shortley for a play :slight_smile:

15 Likes

In mail inboxes it will be forever at 5th May :slight_smile:
(yes, I’m still here)

19 Likes

Omg!!! Don’t break it… I am coming :partying_face: (in about an hour)

@Southside looking at you :wink:

@maidsafe you rock!!! :heart:

23 Likes

Results in:

d:\SN>safe --peer=/ip4/178.128.166.245/udp/36572/quic-v1/p2p/12D3KooWPuaKwn7rFNn
4oj1ABDpikVZ7scHs29ZNCHBveLHgCmak  files download
Starting logging to directory: "C:\\Users\\Vort\\AppData\\Local\\Temp\\safe-clie
nt.log"
Instantiating a SAFE client...
Trying to download files recorded in uploaded_files folder

c:\Users\Vort\.safe\client\downloaded_files is empty. There should be files, right?

upd. If there will be ever answer, pls ping me.

1 Like

I cannot upload it errors out for me!

./safe.exe --peer=/ip4/178.128.166.245/udp/36572/quic-v1/p2p/12D3KooWPuaKwn7rFNn4oj1ABDpikVZ7scHs29ZNCHBveLHgCmak files upload -- C:\SharedXC2\music-qB-DLs
Starting logging to directory: "C:\\Users\\tseko\\AppData\\Local\\Temp\\safe-client.log"
Instantiating a SAFE client...
Storing file "00.Info.m3u" of 1366 bytes..
Successfully stored file "00.Info.m3u" to aed0a83f3b979d9401d5c68c2f3d0b0ee67bd825c18bb6bed495112f2c33862e
Storing file "00.Info.nfo" of 2377 bytes..
Successfully stored file "00.Info.nfo" to 623abe61f47628d0b50a546d05fa76a762c6c42b91c501a27b6cf62377c5dc46
Storing file "01. Umberto Tozzi - Se non avessi te.flac" of 33975829 bytes..
Did not store file "01. Umberto Tozzi - Se non avessi te.flac" to all nodes in the close group! Network Error Outbound Error.
Storing file "02. Amedeo Minghi - 1950 (Live).flac" of 20435585 bytes..
Did not store file "02. Amedeo Minghi - 1950 (Live).flac" to all nodes in the close group! Network Error Outbound Error.
Storing file "03. Mia Martini - Cumme (Live).flac" of 30821522 bytes..

on windows 11!

5 Likes

Do you get the same with smaller files?

6 Likes

New setup, probably (definitely) going to need some hand holding. Will be on in the next few hours. As @Josh said @Southside dont break it before then :crazy_face:

5 Likes

I’d like to submit Pre-Dev-Update Thread! Yay! :D - #5203 by Southside for the absolute Worst Prediction of The Year :slight_smile:

11 Likes

Yea, this will break or fill up very easily if we shove large files on it. It’s a test for small and few files to check comms/churn etc.

16 Likes

It’s not your worst this year, but yeah, hang your head in shame. :joy:

13 Likes

Awright awright, I heard ye…

13 Likes

I am a bit sceptical of this one as we have only a few nodes. My deep feeling and experience is that kad with less than 2000 nodes is flakey. So this one is worth throwing node’s at for all those with direct / public IP addresses.

14 Likes

@Vort
you will have to execute upload something first to populate that record folder.
Otherwise, you can only download something else that other people shared, for example:

safe files --peer=/ip4/178.128.166.245/udp/36572/quic-v1/p2p/12D3KooWPuaKwn7rFNn4oj1ABDpikVZ7scHs29ZNCHBveLHgCmak  download -- testnet.exe 332f45ae9482a8e03b5d5579f15fd9c8806fe4a2b30da2bc6e5ebb9a98fe4095
10 Likes

@SmoothOperatorGR
According to the printouts, two small files got uploaded successfully.
Currently, the client requires ALL CLOSE_GROUP_SIZE (8) closest nodes to a chunk must reply with ChunkStore(Ok()) response, to consider that chunk to be correctly uploaded.
And any one single node delayed with its resp will be counted as failed and fails the whole file upload.
Considering large files like yours will trigger over 40s chunks to be uploaded at the same time, causing an extra rush point among nodes, there is highly possible one of nodes cann’t complete bunched requests in time. Espeically with flaky connections.

I experienced similiar issue with uploading 30MB file to public testnet, however succeeded in uploading to a local testnet.

We may consider flat the burst of traffic during large file uploading, or loose client checks for future runs.

11 Likes

Upload and download of small files looks fine to me !

./safe --peer=/ip4/178.128.166.245/udp/36572/quic-v1/p2p/12D3KooWPuaKwn7rFNn4oj1ABDpikVZ7scHs29ZNCHBveLHgCmak files  download -- "safe wallpaper.jpeg" 89bf86508e7054781eb0db7cd32a11aa26cd9ea22d300d80a00965e252727ad7
Starting logging to directory: "/var/folders/z_/n_9kwvnn5f11q35z7m18c8sm0000gn/T/safe-client.log"
Instantiating a SAFE client...
Downloading file "safe wallpaper.jpeg" with address 89bf86508e7054781eb0db7cd32a11aa26cd9ea22d300d80a00965e252727ad7
Successfully got file safe wallpaper.jpeg!

:clap:

12 Likes
  1. I was expecting commands with example files, like with previous testnets.
  2. Looks like your example command is wrong, files in the wrong place.
  3. Even if I fix it, still something wrong happens:
Downloading file "testnet.exe" with address 332f45ae9482a8e03b5d5579f15fd9c8806f
e4a2b30da2bc6e5ebb9a98fe4095
Did not get file "testnet.exe" from the network! Network Error Provider record w
as not found locally.

Maybe I’m still doing something wrong…

3 Likes