Same file now with
RUST_LOG=sn_client=trace,qp2p=debug enabled: Trace Log
Same file now with
Perhaps this is a known issue, but putting large files on systems with limited memory available fails with no error message. If the system runs out of memory, it simply kills the put.
In the same vein, there needs to be some way to manage memory so that it isn’t consuming 2x the put file size in memory. It would make it untenable for a lot of people to put any files of a couple gigs or more.
Since upload of 512 MB file takes 6 hours (source), memory limitation is not the biggest issue.
V6 running solidly for me. No errors, just upload put time on larger files seems slower. Also still waiting to node join:
[sn_node] INFO 2021-06-18T08:25:47.628559617-06:00 [src/bin/sn_node.rs:124] The network is not accepting nodes right now. Retrying after 3 minutes
put speeds seem stable:
50Mb: slower at 25:15,
Intel® Celeron(R) CPU J3455 @ 1.50GHz × 4
Ubuntu 20.04.2 LTS
2 hours later and both are downloading nicely.
How long did it take in previous tests for issues to surface? Not going to say “resolved” to avoid jinxing.
Has anyone managed to join? my node is trying but no luck yet.
Aha, i see. Transfer is hitting an (only one…?)elder out of order and stalling things. I guess this is more likely as we do more transfers for larger files (and its one transfer per chunk atm).
I was looking at this a couple weeks ago, before we had some other fixes in. Hadnt seen it since, so didn’t get it in amongst the other work. But i have some code to update the elders in a branch, so will get that in next week. and then we need to resend the original and we should be good (hopefully).
atm, client is still rather dumb. w/r/t Cmd failures. We’ll get there though!
Also worth noting jsut how much of the logs (and so elder work) is writing our transfer history. I imagine this slows down nodes a fair bit (a good whack of the memory issue came down to serialisation). We should be able to relatively easily avoid this I think.
Good thing here is that this is client side issues, not really node stuff just node
Aye, there’s been no attempt at optimisation of such things at all yet. We’ll get there thougH!
So much of an upload is the payment coordination, so I’d hope to see a big boost in perf here as we get DBCs integrated
Yes I think you are correct, however I would take these results with a huge pinch of salt. That spreadsheet was only to compare the speeds I got with various versions of sn_node on a baby-fleming running on one internal machine.
it’s a useful pinch of salt though!
These were single runs with normal browsing etc going on in the background and I suspect mean little unless they were repeated several times and averaged.
Wait for network split.
No luck for me too yet.
this was where data got lost last time right?
Finally found the admin pwd for this router - what ports do I need to forward?? .
Probably in the OP but they dont jump out at me immediately.
You can forward any, you’d just need tp specify them when starting your node, eg
./target/release/sn_node --public-addr <public ip>:<some port> --local-addr <lan IP>:<same port> --hard-coded-contacts '["126.96.36.199:34518"]'
for the curious current node mem load is pretty good. highest is ~280mb, avg is much closer to 100.
(We’ll see with a split if things change wildly)
Seeing the trace for my put, and this comment, it is kind of confusing. Is the network calculating price for each chunk being uploaded? Does this mean that someone could run out of tokens in the middle of uploading a large file and lose all their funds (gaining nothing)?
Why wouldn’t the entire price be checked up front and paid for a single time?
When I try to join, the join process starts OK and once it went through a few cycles with 3 minutes re-trials. After that I have gotten:
[sn_node] ERROR 2021-06-18T17:45:10.510039609+03:00 [src/bin/sn_node.rs:143] Encountered a timeout while trying to join the network. Please try again later.
But does this mean, that my connectivity is okay and I am eligible to run a node?
Well, upload seems completely borked for me now but download is working nicely:
$ time safe cat safe://hyryyyyjhxs65mpjyt9ihttd54b67fap7uq6s3cmbrjb5gswb3anwurkxyy > 1.flac
$ time safe cat safe://hygoyeymnywejbhhx6jkw5nhsd19ggtgxs4q6kwongyeo5jk9ap5x1e1p9h > 1.png
Node is still trying to join, occasionally timing out but no other error messages so far.
Download really impressive Sounds like PUT not even looked at yet. Really interested to see if data maintains after network splits.