Fleming Testnet v6 Release - *OFFLINE*

Same file now with RUST_LOG=sn_client=trace,qp2p=debug enabled: Trace Log

2 Likes

Perhaps this is a known issue, but putting large files on systems with limited memory available fails with no error message. If the system runs out of memory, it simply kills the put.

In the same vein, there needs to be some way to manage memory so that it isn’t consuming 2x the put file size in memory. It would make it untenable for a lot of people to put any files of a couple gigs or more.

2 Likes

Since upload of 512 MB file takes 6 hours (source), memory limitation is not the biggest issue.

2 Likes

V6 running solidly for me. No errors, just upload put time on larger files seems slower. Also still waiting to node join:

[sn_node] INFO 2021-06-18T08:25:47.628559617-06:00 [src/bin/sn_node.rs:124] The network is not accepting nodes right now. Retrying after 3 minutes

put speeds seem stable:

10mb: 1:50
20mb: 4:34
50Mb: slower at 25:15,

System:

3.7 GiB
Intel® Celeron(R) CPU J3455 @ 1.50GHz × 4
Ubuntu 20.04.2 LTS
64-bit

8 Likes

2 hours later and both are downloading nicely.
How long did it take in previous tests for issues to surface? Not going to say “resolved” to avoid jinxing.

Has anyone managed to join? my node is trying but no luck yet.

1 Like

Aha, i see. Transfer is hitting an (only one…?)elder out of order and stalling things. I guess this is more likely as we do more transfers for larger files (and its one transfer per chunk atm).

I was looking at this a couple weeks ago, before we had some other fixes in. Hadnt seen it since, so didn’t get it in amongst the other work. But i have some code to update the elders in a branch, so will get that in next week. and then we need to resend the original and we should be good (hopefully).

atm, client is still rather dumb. w/r/t Cmd failures. We’ll get there though!

thanks @Vort ! @JBildstein (same issue both).


Also worth noting jsut how much of the logs (and so elder work) is writing our transfer history. I imagine this slows down nodes a fair bit (a good whack of the memory issue came down to serialisation). We should be able to relatively easily avoid this I think.

Good thing here is that this is client side issues, not really node stuff just node

12 Likes

Aye, there’s been no attempt at optimisation of such things at all yet. We’ll get there thougH!

2 Likes

So much of an upload is the payment coordination, so I’d hope to see a big boost in perf here as we get DBCs integrated :+1:

11 Likes

Yes I think you are correct, however I would take these results with a huge pinch of salt. That spreadsheet was only to compare the speeds I got with various versions of sn_node on a baby-fleming running on one internal machine.

it’s a useful pinch of salt though!

3 Likes

These were single runs with normal browsing etc going on in the background and I suspect mean little unless they were repeated several times and averaged.

Wait for network split.

No luck for me too yet.

1 Like

this was where data got lost last time right?

help!!!

Finally found the admin pwd for this router - what ports do I need to forward?? .
Probably in the OP but they dont jump out at me immediately.

You can forward any, you’d just need tp specify them when starting your node, eg

./target/release/sn_node --public-addr <public ip>:<some port> --local-addr <lan IP>:<same port> --hard-coded-contacts '["138.68.187.212:34518"]'

4 Likes

for the curious current node mem load is pretty good. highest is ~280mb, avg is much closer to 100.

(We’ll see with a split if things change wildly)

7 Likes

Seeing the trace for my put, and this comment, it is kind of confusing. Is the network calculating price for each chunk being uploaded? Does this mean that someone could run out of tokens in the middle of uploading a large file and lose all their funds (gaining nothing)?

Why wouldn’t the entire price be checked up front and paid for a single time?

1 Like

When I try to join, the join process starts OK and once it went through a few cycles with 3 minutes re-trials. After that I have gotten:

[sn_node] ERROR 2021-06-18T17:45:10.510039609+03:00 [src/bin/sn_node.rs:143] Encountered a timeout while trying to join the network. Please try again later.

… sooner.

But does this mean, that my connectivity is okay and I am eligible to run a node?

1 Like

Well, upload seems completely borked for me now but download is working nicely:

$ time safe cat safe://hyryyyyjhxs65mpjyt9ihttd54b67fap7uq6s3cmbrjb5gswb3anwurkxyy > 1.flac
real 1m39,382s
user 0m53,041s
sys 0m30,060s
size 67.5MiB

$ time safe cat safe://hygoyeymnywejbhhx6jkw5nhsd19ggtgxs4q6kwongyeo5jk9ap5x1e1p9h > 1.png
real 0m2,637s
user 0m0,926s
sys 0m0,450s
size 910KiB

Node is still trying to join, occasionally timing out but no other error messages so far.

1 Like

Download really impressive :wink: Sounds like PUT not even looked at yet. Really interested to see if data maintains after network splits.