[Offline] Fleming Testnet v6.2 Release - General & CLI Support

If I remember correctly simultaneous uploads are not supported with latest versions of sn.

I’m not uploading simultaneously but created new keys while the upload continued.

safe keys balance

Why checking for node keys?
Cli data is stored at .safe\cli\credentials

Ignorance!

Things are slow (5 mins for safe cat safe://harmy) but here are some audio files:

time safe cat safe://harmy
Files of FilesContainer (version 0) at "safe://harmy":
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+
| Name                                                       | Type       | Size    | Created              | Modified             | Link                                                               |
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+
| /AlbumArtSmall.jpg                                         | image/jpeg | 3323    | 2021-06-30T10:15:17Z | 2021-06-30T10:15:17Z | safe://hygoygyxomjswyj751bbk4d84537uxnkgbs51xj33mu15fx45agbsyfsh5h |
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+
| /AlbumArt_{4DD65B30-67F2-4E13-9649-F28B3BB1E2EA}_Large.jpg | image/jpeg | 12310   | 2021-06-30T10:15:17Z | 2021-06-30T10:15:17Z | safe://hygoygyerwyj86o7p96pm9nctqsfg16mkn16pcyh5kueep8fa73oqh3bd4e |
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+
| /AlbumArt_{4DD65B30-67F2-4E13-9649-F28B3BB1E2EA}_Small.jpg | image/jpeg | 2944    | 2021-06-30T10:15:17Z | 2021-06-30T10:15:17Z | safe://hygoygyjt1u7mpxzm1jz9zpr188iwq7j88brpqn8mg95mte36s1cwy1tzqa |
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+
| /AlbumArt_{C92AF3FE-4952-4CA2-B8E5-825D02C3371C}_Large.jpg | image/jpeg | 14948   | 2021-06-30T10:15:17Z | 2021-06-30T10:15:17Z | safe://hygoygyej7ta9moyceho3yq4fe97xfcfnk1qmuujbyx3oz45bouef8wh38c |
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+
| /AlbumArt_{C92AF3FE-4952-4CA2-B8E5-825D02C3371C}_Small.jpg | image/jpeg | 3323    | 2021-06-30T10:15:17Z | 2021-06-30T10:15:17Z | safe://hygoygyxomjswyj751bbk4d84537uxnkgbs51xj33mu15fx45agbsyfsh5h |
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+
| /Folder.jpg                                                | image/jpeg | 14948   | 2021-06-30T10:15:17Z | 2021-06-30T10:15:17Z | safe://hygoygyej7ta9moyceho3yq4fe97xfcfnk1qmuujbyx3oz45bouef8wh38c |
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+
| /Thumbs.db                                                 | Raw        | 18944   | 2021-06-30T10:15:17Z | 2021-06-30T10:15:17Z | safe://hyryyyyec36np1whg81apj7nj9tt8gz1koxmgbf9u5dt44gqa1bt1gtet3e |
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+
| /aaCamptown Races (jazz).mp3                               | audio/mpeg | 2604494 | 2021-06-30T10:15:17Z | 2021-06-30T10:15:17Z | safe://hygyy1yqq4ckxp8jgzd9gyenju8c9wqpd9f46xsie9t94fp79x919gawsxy |
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+
| /aaDrink To Me Only With Thine Eyes.mp3                    | audio/mpeg | 951988  | 2021-06-30T10:15:17Z | 2021-06-30T10:15:17Z | safe://hygyy1ymhj5fhubi5qptt9ej5ei5pyxa4uhyo8w7bg47n6usdcpywnek85y |
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+
| /aaMitch Miller - My Darling Clementine.mp3                | audio/mpeg | 3581996 | 2021-06-30T10:15:17Z | 2021-06-30T10:15:17Z | safe://hygyy1ye43of1fd8umspae4yccdq9en8ztguqc1k9dw9ajnewm1b4sjr41y |
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+
| /aaMy Old Kentucky  Home Johny Cash.mp3                    | audio/mpeg | 2671135 | 2021-06-30T10:15:17Z | 2021-06-30T10:15:17Z | safe://hygyy1ym5gbn6z7uy66wnb6qe86j3f61t3ttgaduzzdexw94cum8gxoieka |
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+
| /aaShe'll_Be_Coming_Round_The_Mountain.mp3                 | audio/mpeg | 1314567 | 2021-06-30T10:15:17Z | 2021-06-30T10:15:17Z | safe://hygyy1yezyh1yspfcs89uc1fso3ofygbmbh9djsq54gpn48nbsag6sc4eco |
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+
| /desktop.ini                                               | text/plain | 393     | 2021-06-30T10:15:17Z | 2021-06-30T10:15:17Z | safe://hy8oycycx374sf7no55rbwyrapfssmwofzqyzd1ireoyzkokd1fwmzcwxpr |
+------------------------------------------------------------+------------+---------+----------------------+----------------------+--------------------------------------------------------------------+

real	5m50.459s
user	0m1.524s
sys	0m0.234s
2 Likes

Slow is good, not great, but correct. With a small network then we will find it slow with concentrated activity. This is good, we have many optimisations to make, but the fact it slows and not crash is important. I am happy if this happens, but I know like many that this is us working at worst possible levels of optimisation (i.e. almost none).

Coming down the line

  • coins vanishing (AT2)
  • Message flow simplification and optimisation
  • Get/Put fast flow with current as fallback
  • Dbc integration (possibly with bulletproofs or similar for enhanced privacy)
  • T7
  • Some smaller testnets that hopefully have crushed most bugs.
  • Fleming
22 Likes

I’m seeing an IGD info log when up- or downloading a file, is that normal? I thought it should only come into play when running a node:

[qp2p::endpoint] INFO 2021-06-30T13:24:17.841097170 [/home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/qp2p-0.12.4/src/endpoint.rs:272] IGD request failed: Could not find the gateway device for IGD - IgdSearch(IoError(Custom { kind: TimedOut, error: "search timed out" }))

A simple Hello World text file worked nicely but now I’m also getting timeouts for a 600kb file, might still be going though. There’s a bunch more like these:

[qp2p::connections] WARN 2021-06-30T13:36:04.069795922 [/home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/qp2p-0.12.4/src/connections.rs:221] Failed to read incoming message on uni-stream for peer 209.97.176.100:54768 with error: TimedOut
[qp2p::connections] WARN 2021-06-30T13:36:04.069865017 [/home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/qp2p-0.12.4/src/connections.rs:261] Failed to read incoming message on bi-stream for peer 209.97.176.100:54768 with error: TimedOut
2 Likes

That is also a bug, it won’t affect you but related to the above where we try idg, we need to squish that. @lionel.faber is our :superhero: there so thanks for the report :+1:

2 Likes

thanks, glad to know :smiley:

2 Likes

So how to know now if data loss occurred or network is just slow at the moment?

For example, I am trying to download safe-the-planet.png now. I have no answer from network. What it means?

3 Likes

We have a timeout at the client(which is configurable) that errors out if there’s no response found :slight_smile:

So, network is slowed dramatically, but not crashed. right?

I can guess that if 60 KB file is downloading for 10 minutes, then no response has arrived.
But I do not know if it means that network is broken or slow.

By the way, download command shows this now:

[qp2p::api] ERROR 2021-06-30T14:56:01.079358900 [C:\Users\Vort\.cargo\registry\s
rc\github.com-1ecc6299db9ec823\qp2p-0.12.4\src\api.rs:266] Failed to bootstrap to the network: Connection lost
1 Like

We need to break this down.

For the network and low level libs, they don’t. They don’t have the authority or right to decide when to timeout.

Apps can set such timeouts if they wish, like some browsers report site not found and then it appears. Decentralised network / all networks can do this.

So like speaking to somebody, how do we know they are ignoring us or not hearing us? We wait a reasonable time (Set different for us all) and then give up or retry.

It’s a subtle issue, but a real one, I am strong on no responses where possible as it leads to timeouts and magic numbers.

Very interesting thought experiments, but I can tell you nearly every time we have such timeouts and it seems like a Put failed and test fails, then next test to Get passes :smiley: that sums it up how wrong timeouts are at low levels.

4 Likes

That though, is now an error and should feed back to user.

1 Like

Yep, if the network crashed we’d be seeing errors from the networking layer like connection lost, dropped etc.

Yea that’s a network error, the node might be offline or it could be a glitch. Can you try rerunning them? :slight_smile:

3 Likes

@happybeing is there some special command to start a node so that it logs data for vdash ?

@Vort I built it same as you did with cli and I think my node is functioning as I see chunks arriving in the chunks folder

Where does the IGD issue fall into this mix, is it a priority to solve or not urgent in the big scheme of things right now.

1 Like

You won’t see PUTs yet, only GETs and I’m not sure if you need to modify the logging level so show those yet. Unfortunately you would need to restart your node if that’s the case, but bear with me and I’ll be pushing an update to vdash shortly which you can help test if you like.

@neik Can you run this command (assumes Linux) and tell me if you get any output (copy a line for me to see if you can):

grep "Writing chunk succeeded" ~/.safe/node/local-node/*.log

If that works the version of vdash I just published should work too so give it a try. If there’s no output from the above command I’m afraid you’d need to increase the logging level of your node which would mean restarting - but you probably won’t want to do that because it will have to wait to be accepted again (I believe).

I think the following should work for anyone when starting a node:

RUST_LOG=info,quinn=off safe node join
1 Like

Here’s a 5M audio file with a prize of one forum like for the first person to correctly identify the interviewee (took 3m to upload over mobile b/b):

time safe files put 02\ -\ Why\ wouldn\'t\ you\ have\ been\ good\ in\ mrdical\ practice-.flac 
FilesContainer created at: "safe://hyryyrbcgchksiyz9k9wc5r3pgf1diugayaze3ch54p5st9huoc53j4ajehnra"
+  02 - Why wouldn't you have been good in mrdical practice-.flac  safe://hyryyyyxm9at3q5x4stpwkuexmsx9cmg7m8bs3hp99w7rn7orr9tr7n89mo 

real	3m39.806s
user	0m9.419s
sys	0m4.104s