Fleming Testnet v3 Release - * NOW OFFLINE - v4 RELEASED*

Development Releases
307

Not sure if it have any use, but will put it here.
This looks like a hang. No join, no reject.

[sn_node] INFO 2021-04-16T08:32:58.837253300+03:00 [src\bin\sn_node.rs:118] The network is not accepting nodes right now. Retrying after 3 minutes
[sn_node] INFO 2021-04-16T08:35:58.842053100+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\mod.rs:127] 1d121d.. Bootstrapping a new node. 
[sn_node] INFO 2021-04-16T08:36:09.007579300+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\qp2p-0.11.8\src\endpoint.rs:270] IGD request failed: Could not find the gateway device for IGD - IgdSearch(IoError(Custom { kind: TimedOut, error: "search timed out" }))
[sn_node] INFO 2021-04-16T08:36:09.174612700+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\bootstrap.rs:196] Bootstrapping redirected to another set of peers: [165.227.229.59:47925, 165.227.229.103:59359, 165.227.229.131:53334, 178.62.74.247:51223, 165.227.229.104:33663, 188.166.174.88:54107, 165.227.229.129:57822] 
[sn_node] INFO 2021-04-16T08:36:09.225622900+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\bootstrap.rs:196] Bootstrapping redirected to another set of peers: [165.227.229.150:33874, 138.68.130.204:50087, 165.227.229.57:45221, 138.68.136.72:39857, 138.68.146.170:32859, 165.227.225.237:60110, 46.101.61.67:59088] 
[sn_node] INFO 2021-04-16T08:36:09.289635700+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\bootstrap.rs:183] Joining a section (000), key: PublicKey(15cb..fc8e), elders: {067813..: 165.227.229.150:33874, 0755dd..: 138.68.130.204:50087, 093a49..: 165.227.229.57:45221, 0ac8a0..: 138.68.136.72:39857, 1809bc..: 138.68.146.170:32859, 1afa0a..: 165.227.225.237:60110, 1dbaef..: 46.101.61.67:59088} (given by 46.101.61.67:59088) 
[sn_node] INFO 2021-04-16T08:36:09.289635700+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\bootstrap.rs:402] Sending JoinRequest { section_key: PublicKey(15cb..fc8e), relocate_payload: None, resource_proof_response: None } to [165.227.229.150:33874, 138.68.130.204:50087, 165.227.229.57:45221, 138.68.136.72:39857, 138.68.146.170:32859, 165.227.225.237:60110, 46.101.61.67:59088] 
[sn_node] INFO 2021-04-16T08:36:09.444653100+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\bootstrap.rs:402] Sending JoinRequest { section_key: PublicKey(15cb..fc8e), relocate_payload: None, resource_proof_response: Some(4) } to [46.101.61.67:59088] 
[tokio-runtime-worker] ERROR 2021-04-16T08:36:09.654653100+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\qp2p-0.11.8\src\connections.rs:299] Failed reading from a bi-stream for peer 46.101.61.67:34750 with error: Failed to read expected number of message bytes: connection closed: closed by peer: 0
[sn_node] INFO 2021-04-16T08:36:10.155653100+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\bootstrap.rs:402] Sending JoinRequest { section_key: PublicKey(15cb..fc8e), relocate_payload: None, resource_proof_response: Some(2) } to [138.68.130.204:50087] 
[tokio-runtime-worker] ERROR 2021-04-16T08:36:10.506653100+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\qp2p-0.11.8\src\connections.rs:299] Failed reading from a bi-stream for peer 138.68.130.204:55313 with error: Failed to read expected number of message bytes: connection closed: closed by peer: 0
[sn_node] INFO 2021-04-16T08:36:13.844853100+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\bootstrap.rs:402] Sending JoinRequest { section_key: PublicKey(15cb..fc8e), relocate_payload: None, resource_proof_response: Some(7) } to [138.68.136.72:39857] 
[tokio-runtime-worker] ERROR 2021-04-16T08:36:18.719053100+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\qp2p-0.11.8\src\connections.rs:299] Failed reading from a bi-stream for peer 138.68.136.72:37550 with error: Failed to read expected number of message bytes: connection closed: closed by peer: 0

Another attempt:

Summary 
[sn_node] INFO 2021-04-16T08:46:03.331750700+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\mod.rs:127] 2ed420.. Bootstrapping a new node. 
[sn_node] INFO 2021-04-16T08:46:13.494850700+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\qp2p-0.11.8\src\endpoint.rs:270] IGD request failed: Could not find the gateway device for IGD - IgdSearch(IoError(Custom { kind: TimedOut, error: "search timed out" }))
[sn_node] INFO 2021-04-16T08:46:13.763850700+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\bootstrap.rs:196] Bootstrapping redirected to another set of peers: [165.227.226.230:33552, 165.227.229.148:47735, 180.150.10.164:64827, 138.68.169.179:47468, 178.62.122.173:58245, 165.227.229.55:32936, 165.227.232.158:50965] 
[sn_node] INFO 2021-04-16T08:46:13.818850700+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\bootstrap.rs:183] Joining a section (001), key: PublicKey(0194..09fc), elders: {284404..: 165.227.226.230:33552, 29b182..: 165.227.229.148:47735, 2e205e..: 138.68.169.179:47468, 2f9253..: 178.62.122.173:58245, 303156..: 165.227.229.55:32936, 38e213..: 165.227.233.86:56882, 3c01e3..: 165.227.232.158:50965} (given by 165.227.232.158:50965) 
[sn_node] INFO 2021-04-16T08:46:13.818850700+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\bootstrap.rs:402] Sending JoinRequest { section_key: PublicKey(0194..09fc), relocate_payload: None, resource_proof_response: None } to [165.227.226.230:33552, 165.227.229.148:47735, 138.68.169.179:47468, 178.62.122.173:58245, 165.227.229.55:32936, 165.227.233.86:56882, 165.227.232.158:50965] 
[sn_node] INFO 2021-04-16T08:46:43.818539100+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\bootstrap.rs:402] Sending JoinRequest { section_key: PublicKey(0194..09fc), relocate_payload: None, resource_proof_response: Some(2) } to [165.227.232.158:50965] 
[sn_node] INFO 2021-04-16T08:46:43.818539100+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\bootstrap.rs:402] Sending JoinRequest { section_key: PublicKey(0194..09fc), relocate_payload: None, resource_proof_response: Some(1) } to [138.68.169.179:47468] 
[sn_node] INFO 2021-04-16T08:46:43.921559700+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\bootstrap.rs:402] Sending JoinRequest { section_key: PublicKey(0194..09fc), relocate_payload: None, resource_proof_response: Some(2) } to [165.227.226.230:33552] 
[sn_node] INFO 2021-04-16T08:46:43.972569900+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\sn_routing-0.60.3\src\routing\bootstrap.rs:402] Sending JoinRequest { section_key: PublicKey(0194..09fc), relocate_payload: None, resource_proof_response: Some(0) } to [165.227.233.86:56882] 
[tokio-runtime-worker] ERROR 2021-04-16T08:46:44.138603100+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\qp2p-0.11.8\src\connections.rs:299] Failed reading from a bi-stream for peer 165.227.232.158:34184 with error: Failed to read expected number of message bytes: connection closed: closed by peer: 0
[tokio-runtime-worker] ERROR 2021-04-16T08:46:44.184612300+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\qp2p-0.11.8\src\connections.rs:299] Failed reading from a bi-stream for peer 138.68.169.179:40174 with error: Failed to read expected number of message bytes: connection closed: closed by peer: 0
[tokio-runtime-worker] ERROR 2021-04-16T08:46:44.237622900+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\qp2p-0.11.8\src\connections.rs:299] Failed reading from a bi-stream for peer 165.227.226.230:50566 with error: Failed to read expected number of message bytes: connection closed: closed by peer: 0
[tokio-runtime-worker] ERROR 2021-04-16T08:46:44.297634900+03:00 [C:\Users\runneradmin\.cargo\registry\src\github.com-1ecc6299db9ec823\qp2p-0.11.8\src\connections.rs:299] Failed reading from a bi-stream for peer 165.227.233.86:44502 with error: Failed to read expected number of message bytes: connection closed: closed by peer: 0
2 Likes
308

My log shows no activity for 9 hours. How can I tell if it has crasched or is still working? This is the last output.

[sn_node] INFO 2021-04-16T00:01:39.455438208+03:00 [src/event_mapping/mod.rs:44] Handling RoutingEvent: EldersChanged { prefix: Prefix(10), key: PublicKey(0a1f..c0a9), sibling_key: None, elders: {8ff64c.., 91b2f2.., 9666a8.., a8ab5a.., aee52d.., b6fb60.., ba9a09..}, self_status_change: None }
[sn_node] INFO 2021-04-16T00:01:39.455537422+03:00 [src/node/handle.rs:27] Handling NodeDuty: No op.
4 Likes
309

I am new to how Rust works so probably stupid question, where can I set this?

1 Like
310

Interesting! If a non validating node didn’t reject bad data, could that be used as a signal to other elders (to remove the bad node)?

If not, it sounds like an incentive for nodes to take a shortcut and save resources, at the expense of data integrity.

311

The economic incentives can work under normal conditions without removing the need for it to still work without them, for example, as I’m writing this there are reports from China that there has been a coal mine explosion in Xinjiang resulting in a major power outage which has taken approximately 30% of the global Bitcoin hashrate of line in an instant, maybe something like that could happen with a lot of safe network elders being in a single large datacenter somewhere?

1 Like
312

No luck… my node didn’t join over night.
Latest log entry.

sn_node] INFO 2021-04-16T02:10:05.086055200+01:00 [src/bin/sn_node.rs:118] The network is not accepting nodes right now. Retrying after 3 minutes

1 Like
313

Yep it was the same for me

314

What would be the best way or command to stop and restart the node in case it’s hung?

315

With more logging this happens:

1 Like
316

I suspect there isn’t as much new data getting pushed onto the net atm. Maybe try generating some to see if it causes some splits?

The bugs in the network may also be hindering. I’m not sure how much is still alive, but cat not NRS create completed yesterday. So, still very much a playing sandpit atm! :grinning:

4 Likes
317

I node restart with the same name will be considered as rejoin and will got relocated to same section but with half aged once get online agreement from the section.
However, there is chance that the section is closing the section (i.e. rejects joining) already.

Hence, the restart of node may get :
1, relocated with half aged when section is accepting joining node
or
2, rejected if section is not accepting joining node

cc @dirvine

4 Likes
318

How to know it this happening?
I do not see where name is stored in root dir.

So how node can know which name it should use after restart?

1 Like
319

It stored as member_info carried within routing’s cache.

320

I see only reward_public_key and reward_secret_key in root dir.
Is it only for elders? I have removed my root contents for it, so can’t check.

By restart I mean stopping sn_node process and starting it again.

321

After seeing no activity for 12 hours I did

safe node killall
and then
safe node join

Now I’m getting the “not accepting nodes right now” message.

322

you mean the member_info cache? yes it’s only for elder.
and it’s in routing level, and never flushed to disk. It’s a cache containing the list of members of sections (including those left/relocated)

3 Likes
323

If process stopped then “name” is lost too?
So what restart are we talking about then?

324

I killed my perfectly working node yesterday. Today I’m trying to join again, but the process just hangs after 4 retries. No error messages, the process just stops. Claims to be retrying in 3 minutes, but doesn’t.

Ubuntu 18.04.4 LTS

1 Like
325

Look at my post above.
Node emits error mesages, but with default log level they are invisible.

2 Likes
326

How can you tell it’s not retrying?