5-4/3/2(022) Playground! [Offline once more ]

Aha, well that sounds like nodes are full, without enough waiting in the wings.

Mem usage per node is still low, ~200mb on average. And honestly now that much more put atm on average. Hmmm. Gonna dive into the logs some and see what I can see.

Sounds like it’s reporting full without that being the case, though :thinking: .

I’m unable to cat the

5 Likes

I am still getting this error…

willie@gagarin:~$ curl https://raw.githubusercontent.com/maidsafe/safe_network/master/resources/scripts/install.sh | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  7237  100  7237    0     0  25005      0 --:--:-- --:--:-- --:--:-- 25041
=> Downloading Safe CLI package from 'https://github.com/maidsafe/safe_network/releases/download/0.58.4-0.57.1-0.50.4/sn_cli-0.50.4-x86_64-unknown-linux-musl.tar.gz'...
=> Saving to '/tmp/tmp.E8dnCJrhee/sn_cli-0.50.4-x86_64-unknown-linux-musl.tar.gz'...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   685  100   685    0     0   2255      0 --:--:-- --:--:-- --:--:--  2260
Warning: Failed to create the file 
Warning: /tmp/tmp.E8dnCJrhee/sn_cli-0.50.4-x86_64-unknown-linux-musl.tar.gz: 
Warning: No such file or directory
  0  9.8M    0   852    0     0   1318      0  2:10:10 --:--:--  2:10:10  1318
curl: (23) Failure writing output to destination
=> Unpacking Safe CLI to '/home/willie/.safe/cli'...
tar (child): /tmp/tmp.E8dnCJrhee/sn_cli-0.50.4-x86_64-unknown-linux-musl.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
chmod: cannot access '/home/willie/.safe/cli/safe': No such file or directory
=> Adding statement to '/home/willie/.bashrc' profile to have Safe CLI binary path in the $PATH
=> Profile '/home/willie/.bashrc' already contains a statement to set Safe CLI in the $PATH

I will grab the binaries manually.

1 Like

Same as @Josh

willie@gagarin:~$ time safe files put Desktop/cockerel-southside.jpeg 
FilesContainer created at: "safe://hyryyryyn7e7somp39di84bzyfnbem4mm6un5kfbrqhead8kt1ixt638wfneoeuy?v=hgip1akd6y1eg5cnzxjfqayrb7yyyobuttozu3uwcoggpcnoq1aho"
+---+---------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+
| E | Desktop/cockerel-southside.jpeg | <ClientError: Error received from the network: InsufficientAdults { prefix: Prefix(0), expected: 4, found: 1 } for cmd: MsgId(34d5..94a0)> |
+---+---------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+

real	2m54.901s
user	0m9.381s
sys	0m0.188s
2 Likes

I am still waiting, not yet been accepted.
So “not enough adults” means the network is full and there are not enough new nodes to join… or do I misunderstand.

1 Like

Should do aye.

It doesn’t say anything about this tbh. So the join process could have stalled.

But none of that should mean lost data or slow respones. :thinking: There’s something else afoot there.

3 Likes

Yeh I can see there’s some error in the logic around chunk storage.

➤ Chunk holders of 617e6c(01100001).. are empty adults: {04e332(00000100)..} and full adults: {}

There should be more than one here.

(empty is misleading there. just means non-full).

6 Likes

Just more info as I’m going.

I’ve found where we transition from 4 chunk holders to less, and we have a spate of ProposeOffline there, seemingly due to connectivity check fails.

Which makes me think we’re cleaning those up and failing nodes when we should be retrying.

For the curious:This log file line 24413 is the last instance of 4 chunk holders, and you can see what’s afoot after that.

theissue.log.zip (571.6 KB)


I’ll just be confirming this, and then seeing if my simple fix for node conn retries I added this morning might sort this (that’s pending a merge just now, so not in the current release)


Aha, yup, a failed msg send from nodes will trigger the Cmd::HandlePeerLost, so mayybe that fix will sort it. waiting on the merge there and then will trigger out a new release

9 Likes

A search for empty adults shows this happening every couple of hundred lines or so

dont you need the filecontainer xor url for get?

You mean for @Sascha’s .txt? He posted it.

It was a general question… so yeah FileContainer xor url

usually people post the safe cat ‘file xor url’ > ‘file’

so you cant safe files get if thats the norm

1 Like

Yes, you need it afaik, but as i mentioned i dont use it much. That will change now that I know that I can see download progress.

1 Like

This is what i mean. Feels like I have been living under a rock.

3 Likes

I have a ~700mb folder of random files being uploaded and it is there purring allong for 4 hours… anyone has success with put?

1 Like

Been AFK for an hour or so - Came back in to find Im still “Retrying after 3 mins”.
Also

willie@gagarin:~$ safe files put -r ~/projects/maidsafe/olympus/
FilesContainer created at: "safe://hyryyryynt8eokije8inp7j9bzz71iy4j53oakwyijkbq87be6rk9w3jx6tuyeuy?v=huy3nchwspkx69b6kczf7t613qaqgqkhpk4xrty6hcn76mnff8pny"

|---+-----------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------|
| + | /home/willie/projects/maidsafe/olympus/index.html                                       | safe://hy8oyryyb6zip1tkwgs5ijzksou4khbu5yophgwnuj5afno9kc31npsigntwy                                                                       |
+---+-----------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------+

<ClientError: Error received from the network: InsufficientAdults { prefix: Prefix(0), expected: 4, found: 1 } for cmd: MsgId(…)> for all the actual content though I can cat te index.html itself just fine.

Aye, with every chunk there @JPL .

I’m going to bring this down, as I think it’s voted most of its adults off now. If i get another release through I’ll try and reup and we can see if that’s had any impact.

9 Likes

@chriso would you mind having a look at this when you have a sec? :bowing_man:

2 Likes

I see this consistently on my own desktop.
Also seen it on an AWS instance and a n00b I was helping a couple of weeks back reported similar.

All on Ubuntu20.04.

1 Like

Hmm, I don’t reproduce it initially, although that was on an Ubuntu 20.10 VM. I’ll see if I can get an Ubuntu 20.04 VM setup.

The progress indicator was always there. In fact, it used to be a progress bar rather than just text, but we had to drop the use of that crate.

5 Likes