Piquing my interest but not quite sure what it entails yet. Just safe_core renamed??
It’s node/routing/messaging and client rolled into one repo. Still outputting a sn_node bin, and still exporting client for APIs.
This should (once it’s going, just tidying up after the merging), 1) stop the doubts over what is compatible with what. (this node + this client from sn v0.0.01 will always work together). It also 2) should simplify a lot of work (a messaging change wont require 4 PRs to get it in) and 3) allow more robust testing, as we have client code directly with node, we can run those tests. But also do integration testing when changing messaaging/routing too!
So should be a nice wee change once we have made the rust linter happy!
28 Likes
I’m paralyzed by the anxiety: do we get the testnet this week or not? 
If yes, could it be today? My eye for the Github tealeaves has been quite good before, but now I’m totally lost with the view there…
EDIT: What I see is some action that seems mostly like adjusting some details. That leads me thinking that, either A) everything is so well, that now is a good time to do some housekeeping while making some final adjustments or B) the devil in the details has revealed some nasty flaw in the design and the housekeeping is done on the side of serious conversations.
9 Likes
I know how you feel
2 Likes
Of course it can also be just a more mundane bug hunting going on… But I tend to paint a more dramatic picture… Mind of an overinvested person, I guess 
2 Likes
Things like this and this make me think that cleaning is a way of waiting till more important fixes are done.
Silence from team is a bit disappointing by the way.
But I think nothing too bad is happening. Just usual polishing to perfection.
3 Likes
Yes, I spotted them too, and I agree.
The silence feels loaded because it was preceded with “this is valid testnet”, and “all the hanging issues seems to be solved”, plus the overall optimistic tone of the preceding weeks.
2 Likes
Far too few data points to draw real conclusions but here are some results from a wee test I left running last night.
filesize time
5MB 2m1.468s
10MB 3m43.119s
20MB 6m59.187s
50MB 17m25.503s
100MB 41m34.945s
200MB 110m25.682s
512MB 368m8.248s
I got a couple of errors as well
on the 20 MB file
[sn_client::connections::messaging] ERROR 2021-06-16T02:01:48.558868295 [/home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/sn_client-0.60.0/src/connections/messaging.rs:369] Error sending Query to elder: QuicP2p(Connection(TimedOut))
and on the 512MB test file
Error: NetDataError: Failed to PUT Public Blob: ErrorMessage { source: InvalidOperation("Not enough storage available on the network"), msg_id: 1910fa0b.. }
When I tried to GET that file to check the md5 I get this msg
willie@leonov:/tmp/std-file-upload-tests/gets$ safe files get "safe://hyryyrbmc1jxoexdpbyh7jfwx7e7hgtig1iku887s6qm1foji1t7kdyn94enra"
[sn_client::connections::messaging] ERROR 2021-06-16T13:24:33.310880183 [/home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/sn_client-0.60.0/src/connections/messaging.rs:439] Timeout while waiting for response to client request w/ id d9347521..: Elapsed(())
Error: NetDataError: Failed to GET Public Blob: NoResponse
Now when I try to put a small file I get a similar “out of space” error despite vdash reporting I have used only 1.1GB of 153.7GB
Busy with other stuff (beach beats SAFE) but I will have a hard look at the logs later.
This is on a std run-baby-fleming with the latest versions 0.28.1 and 0.48.1 on a i7 8core with 16GB RAM and plenty free disk space.
7 Likes
Baby Fleming?
I got the same from a network I set up on DO.
4 Likes
Really hoping for it this week! If not this week, I’ll hope for it by end of June 
1 Like
Remind me again, where is the best place to post node logs? Just in case these failures are of any interest.
what command did you start the node with? did you manually set the size?
2 Likes
Just the defaults
safe node run-baby-fleming
then safe keys create --test-coins --for-cli
and then I ran a trivial wee script
#Southside
#
#April 2021
#trivial script to put a series of files to exercise the SAFENetwork T5 testnet
#download standard test files from https://www.thinkbroadband.com/download
#store locally and generate checksums
# upload files and check time elapsed
#upload checksums for each file
#Setup
rm -rf /tmp/std-file-upload-tests
mkdir /tmp/std-file-upload-tests
cd /tmp/std-file-upload-tests
#Get the files calculate checksum and write to SAFE Network
for i in 5MB 10MB 20MB 50MB 100MB 200MB 512MB
do
curl --output $i.zip http://ipv4.download.thinkbroadband.com:8080/$i.zip
ls -l $i.zip
md5sum $i.zip > md5-$i.txt
time safe files put $i.zip
safe files put md5-$i.txt
done
Long story but I only have access to my laptop right now and the dev work for my proposed deduplication-test script is on the desktop. So that very early first stab was all I had access to. My home network is deliberately only accessible from the house and I am stuck 200 miles away until son#2 recovers from COVID. Not the best idea to try talking him through opening up the necessaries over the phone.
2 Likes
Crazy amount of github action! 75% chance it happens tomorrow? 
5 Likes
I’m repeating the tests I ran last night with node v0.49.0
seems MUCH faster so far but again I’m getting an error
[sn_client::connections::messaging] ERROR 2021-06-16T18:11:20.466017776 [/home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/sn_client-0.60.0/src/connections/messaging.rs:439] Timeout while waiting for response to client request w/ id f7bbbb79..: Elapsed(())
FilesContainer created at: "safe://hyryyrbej1i58znroixdaani5hr99qow6pca3j98i5z9xnr6e5ocuwfze5rnra"
E 50MB.zip <NetDataError: Failed to PUT Public Blob: SelfEncryption(Generic("Failed to obtain any response"))>
I will let this test run complete and see if I can GET the files back and compare md5s
teaser so far…
willie@leonov:~/projects/maidsafe/dedup-tester$ ./std-files-upload-test.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 5120k 100 5120k 0 0 6745k 0 --:--:-- --:--:-- --:--:-- 6736k
-rw-rw-r-- 1 willie willie 5242880 Jun 16 18:02 5MB.zip
FilesContainer created at: "safe://hyryyrbjxyjdkwx8qzxbf19sxr5cp3z5fohqeyirdhkn3x419yyjkygoxyynra"
+ 5MB.zip safe://hyfenrym6m19mkdjperroq88n14mkpbemyatwsxehwdrc6rhnqpcemeud3y
real 0m14.686s
user 0m3.698s
sys 0m0.325s
FilesContainer created at: "safe://hyryyrbqtuhgrmeewp85xen8fw9rdin56p8qomt3fb558i673xkckm5shianra"
+ md5-5MB.txt safe://hy8oycyqefc6jwd3kgyxdp1hiut51defj48ftk9pcb85frazdb9bqwig7by
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 10.0M 100 10.0M 0 0 7409k 0 0:00:01 0:00:01 --:--:-- 7409k
-rw-rw-r-- 1 willie willie 10485760 Jun 16 18:03 10MB.zip
FilesContainer created at: "safe://hyryyrbkekb73wpzwp46kwgtmam1d3qm8d8gjmijdchp8wgomowz5obku3onra"
+ 10MB.zip safe://hyfenrykcrjmkk78txsiep9zzh9a6ah5jgccjuo8g9yeicjkimwsp8hc31o
real 0m35.215s
user 0m9.965s
sys 0m0.774s
FilesContainer created at: "safe://hyryyrbmhyentn6x1f96jown6id6jwmbj64i4is38b7x7ibhagpt7rhay5rnra"
+ md5-10MB.txt safe://hy8oycyqmhjxdgx4togmkebugoy7sy9kqf37qqgrruhm9u4jsbfhas8budh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 20.0M 100 20.0M 0 0 7521k 0 0:00:02 0:00:02 --:--:-- 7518k
-rw-rw-r-- 1 willie willie 20971520 Jun 16 18:03 20MB.zip
FilesContainer created at: "safe://hyryyrbeaazfysqbewczwdmgobsrz3dibaatq113irc9m8ej798gucnhptwnra"
+ 20MB.zip safe://hyfenryxon8oeymwyeeau8cgt3nj5u1g9psuzqojthz5n4pmz95jxxg93uh
real 1m29.119s
user 0m24.976s
sys 0m1.471s
FilesContainer created at: "safe://hyryyrbx4muegkc635hie111t7adkkbgzf1uzrcw7q4aogwbxtbgxoqcdrcnra"
+ md5-20MB.txt safe://hy8oycyekpn1h3d7yig9g5j4fr5hxq35e3kjkm9qs48p3zbwhhm4m6na8ye
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 50.0M 100 50.0M 0 0 6857k 0 0:00:07 0:00:07 --:--:-- 8166k
-rw-rw-r-- 1 willie willie 52428800 Jun 16 18:05 50MB.zip
[sn_client::connections::messaging] ERROR 2021-06-16T18:11:20.466017776 [/home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/sn_client-0.60.0/src/connections/messaging.rs:439] Timeout while waiting for response to client request w/ id f7bbbb79..: Elapsed(())
FilesContainer created at: "safe://hyryyrbej1i58znroixdaani5hr99qow6pca3j98i5z9xnr6e5ocuwfze5rnra"
E 50MB.zip <NetDataError: Failed to PUT Public Blob: SelfEncryption(Generic("Failed to obtain any response"))>
real 5m40.328s
user 0m42.405s
sys 0m1.885s
FilesContainer created at: "safe://hyryyrbpgh58gbesjtr6h61uyzx49hd5ntcppss8ph3bzegsirc8yfuz7iynra"
+ md5-50MB.txt safe://hy8oycye9gnrf8xiy1id9eiykbrwfjo5yh7i4my6dn6mzawhu567sstkhzy
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 100M 100 100M 0 0 7752k 0 0:00:13 0:00:13 --:--:-- 7847k
-rw-rw-r-- 1 willie willie 104857600 Jun 16 18:11 100MB.zip
FilesContainer created at: "safe://hyryyrbcngmzmnzu9cqpcjhg6eykg4zxwoekb84rdfw4xngzgm41e1r6yhonra"
+ 100MB.zip safe://hyfenryk5iwnsajkkxnh8gqii7txkg5n394gem8dujpiy51nbagdkae196o
real 15m53.138s
user 3m43.745s
sys 0m7.164s
FilesContainer created at: "safe://hyryyrbpbqwrs9383t3zsq1reyh34dgbqk1o199f14qbpesij6bnk6emsuhnra"
+ md5-100MB.txt safe://hy8oycyq4s1tqzen58p9acefctzqt1k4xqgz9tepyr1dkxzr9q6d9geccoc
7 Likes
wow really fast! I was a bit anxious because it was slow before! but yeah! go brrrrr
3 Likes
My money is on next week. I also said that last week.
5 Likes
tl;dr sn_node 0.49.0 is a LOT faster than 0.48.0
I accidentally aborted the last run of the 512MB file = Ctrl-c instead of Shift+Ctrl-c
so here is a wee table of how they compared.
| filesize | sn_node 0.48.1 | sn_node 0.49.0 | |
|---|---|---|---|
| 5MB | 2m1.468s | 0m14.686s | |
| 10MB | 3m43.119s | 0m35.215s | |
| 20MB | 6m59.187s | 1m29.119s | |
| 50MB | 17m25.503s | 5m40.328s | failed |
| 100MB | 41m34.945s | 15m53.138s | |
| 200MB | 110m25.682s | 61m19.929s | |
| 512MB | 368m8.248s | aborted |
Now to do another run cos the dev ants have released sn_node 0.49.2 …
Also note that as the size of the file to be put increases, the time to complete grows non-linearly.
11 Likes
I might have jumped to the wrong conclusion there… On running the same commands again I get a reasonable increase in memory usage, nothing that looks “dangerous”
Nothing else “heavy” running on this box though, can anyone replicate this?
safe node update
safe node run-baby-fleming
safe keys create --test-coins --for-cli --preload 500000
Also …
Again I see a failure when I try to put the 50MB test file
[sn_client::connections::messaging] ERROR 2021-06-16T23:20:39.732964960 [/home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/sn_client-0.60.0/src/connections/messaging.rs:439] Timeout while waiting for response to client request w/ id ef5585b1..: Elapsed(())
FilesContainer created at: "safe://hyryyrbx7wyra5m49mbe6s8gndm54qr34q7ifptmywr635nkcptswmt3p3cnra"
E 50MB.zip <NetDataError: Failed to PUT Public Blob: SelfEncryption(Generic("Failed to obtain any response"))>
can some one else try
curl --output 50MB.zip http://ipv4.download.thinkbroadband.com:8080/50MB.zip
ls -l 50MB.zip
md5sum 50MB.zip > md5-50MB.txt
time safe files put 50MB.zip
and so far sn_node0.49.2 is very slightly slower than 0.49.0
| filesize | sn_node 0.48.1 | sn_node 0.49.0 | sn_node 0.49.2 |
|---|---|---|---|
| 5MB | 2m1.468s | 0m14.686s | 0m14.785s |
| 10MB | 3m43.119s | 0m35.215s | 0m36.218s |
| 20MB | 6m59.187s | 1m29.119s | 1m39.737s |
| 50MB | 17m25.503s | 5m40.328s | 5m48.750s |
| 100MB | 41m34.945s | 15m53.138s | 16m27.564s |
| 200MB | 110m25.682s | 61m19.929s | |
| 512MB | 368m8.248s | aborted |
6 Likes