Jippy Testnet v5 @bochaco from community member to super
Hanging for me too. Ubuntu 18.04.4 LTS.
sascha@Knut:~$ safe keys create --test-coins --for-cli
The above seems to hang on Linux Mint. I get no output for several minutes.
This is, of course, trivial, but I like consistency, as may some scripts. Upper and lower case T and V should not be used interchangeably.
The developers are just amazing. Thanks for all your efforts. Do we have to call @bochaco Sir from now on. Senior Engineer has a ring to it
C:\Users\Vort>ping 184.108.40.206 Обмен пакетами с 220.127.116.11 по с 32 байтами данных: Ответ от 18.104.22.168: число байт=32 время=68мс TTL=54 Ответ от 22.214.171.124: число байт=32 время=73мс TTL=54 Ответ от 126.96.36.199: число байт=32 время=75мс TTL=54 Ответ от 188.8.131.52: число байт=32 время=76мс TTL=54
I will restart node. Maybe intermittent failure.
I’d say this is hanging too:
safe cat safe://hygoyeyx768nkst7qqjjk8khjm67tr1n4otbctcts5j7dten43zae3ga83y > ~/safe-the-planet.png
Ubuntu 18.04.4 LTS
I just fetched the node logs from the testnet and it looks like we have lost the elders of a particular section due to an OOM issue. As mentioned before, we saw the same issue during internal testing but it didn’t reproduce after some fixes were made. Looks like it has come up again.
I will be looking at the logs now to see if anything specific has triggered this. We might restart this testnet with memory profiling enabled and we will need your help taking down another network so we can study the results.
Wellllll deserved @bochaco !!! great stuff.
Looks like i won’t dive into 5, but will await 5.1, or AE… whatever is next .
Excited to get back into this sooooon.
[qp2p::api] ERROR 2021-05-13T11:30:24.756355000 [/Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/qp2p-0.11.9/src/api.rs:266] Failed to bootstrap to the network: Connection lost
[sn_client::connections::listeners] ERROR 2021-05-13T11:30:24.756836000 [/Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/sn_client-0.54.9/src/connections/listeners.rs:74] Error handling network info message: QuicP2p(BootstrapFailure)
Congratulations @bochaco !!!
As Lionel stated above, we have confirmed that we lost Elders in a particular section due to an Out Of Memory(OOM) issue. This is something that we had seen during internal testing once, as mentioned in the OP here, but we were unable to reproduce after merging some potential fixes.
We are going to take down the current v5 nodes, then relaunch a new version of v5 with memory profiling enabled in our new Digital Ocean nodes.
We will then need your help to hit the new testnet to see if we can reproduce this issue, and hopefully the extra information we get from the memory profiling will help us pinpoint and resolve it.
Thanks for your help everyone, we’ll post back here shortly once the new v5 is online and ready to go.
Congratulations @bochaco! A word to the wise - keep on avoiding management for as long as possible! It is much more fun making stuff, especially when you are as talented as your good self!
I reckon “Jefe” might be suitable for Bochaco. “Patrón” should be reserved for David.
$ safe keys create --test-coins --for-cli
[qp2p::api] ERROR 2021-05-13T06:09:13.713115040 [/home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/qp2p-0.11.9/src/api.rs:266] Failed to bootstrap to the network: Connection lost
Me too. Debian 10.9
Two questions for @maidsafe:
In previous test nets I mentioned 2 problems leading to data loss:
elders didn’t ensure that an adult about to be relocated properly stored the chunks sent by its new section
such an adult was connected to its new elders AFTER these chunks were sent to it.
The consequences was that adults that didn’t explicitly allow the port they used in the firewall were relocated successfully even though they didn’t store chunks of their new sections.
Can you confirm that both problems were corrected?
My second question is related to promotion to elder stage: Correction of previous problems allows adults to be relocated in another section without data loss but doesn’t allow an adult to be promoted elder. For that the node needs to allow the port it uses because a complete stranger must be able to connect to an elder (new node or client).
So, do you allow the port used by each of your elders, or intended elders for the future?
Same for me, took the steps through the setup as usual.
Config looks just fine to me, but I won’t get a response.
I suggest you to wait till v5.1 announce (look at post from StephenC above).
Saw is just now.
And already waiting for the announcement of v5.1
Chance to release AE on v5.1 too? I mean if its ready, why not. v5 never got going so why not go all out
Much better to debug one thing at a time, particularly when you have a tricky bug like this which you think you’ve fixed several times but keeps popping up again.
Once T5 has done it’s work, adding AE for T6 should be quick.