"FailedExternalReachability" error

I’m surprised to get the “FailedExternalReachability” in my logs even while ports on both sides of my private test-network are open.

I’ve opened port 5483 on my instance on AWS. The port get’s open whenever I start a new network using safe_vault -f.

So the seednode is running on AWS and the port is open. Next step I start a node from home with the same settings and I can actually see the port go open (DMZ host in action, it’s normally closed). The nodes trie to connect to AWS, it actually does but in 4 seconds the nodes stops and I get this in my PC vault log:

Listener started on port 5483.
ERROR 18:41:32.384463600 [crust::main::bootstrap mod.rs:184] Failed to Bootstrap: (FailedExternalReachability) Bootstrapee node could not establish connection to us.
INFO 18:41:32.384463600 [routing::states::bootstrapping bootstrapping.rs:201] Bootstrapping(b0f337..) Failed to bootstrap. Terminating.
DEBUG 18:41:32.384463600 [routing::state_machine state_machine.rs:267] State::Bootstrapping(b0f337..) Terminating state machine

This is weird, as I also checked that my local port did open.

I write this in the “support” category but I actually think it is a bug. I’ve seen this one with several others as well in community networks over the last few days. Anyone a clue? Is there a timer wrongly set in the code somewhere?

2 Likes

Before now I’ve opened ports out and forgotten to fix it to both directions… but I wonder you’re not that daft.

On AWS windows server I 've set the security settings to inbound 5483 on TCP. Output is unlimited afaik. I’ve opened port 5483 on my router as well, I could run a Vault with the Maidsafe 12b. It did give me trouble some days later and then I’ve actiated DMZ host as that automatically opens ports when needed.

1 Like

UDP and TCP?

both in and out – until I check definatively which we need
I just opened them both - sloppy I know but it works for now.

1 Like

It’s all TCP. But I’ve added UDP as well. No luck. Did that worked for you??

1 Like

Yes I am connecting OK with TCP and UDP 5483 open to all both inbound and outbound
I know this can get hardened later.

1 Like

These are my aws settings:

This is inbound

1 Like

wrong, it’s 3 seconds :stuck_out_tongue_winking_eye:

const CHECK_REACHABILITY_TIMEOUT_MS: u64 = 3 * 1000;

could this be the troubler?? maybe double it to 6 seconds??

Sorry been busy failing to connect …

Yes that looks reasonable - to my eyes anyway

Just to confirm, does the reachability test pass if you bump that to 6 seconds? If you need a build with that changed, I can bump it even to 10 seconds for you to try if you tell me which platform you need.

2 Likes

Thanks, I’m willing to learn to build myself and @Seneca already told me how to and is willing to help. he already promised me a build with that value changed so I’m good :+1:. hopefully I can confirm if this helps in the next 24h. Will update here.

6 Likes

IDK if increasing the timeout will solve it as connections usually materialise in a fraction of second when everything’s Ok (but do let us know the findings by increasing it so that we can tweak it too).

What i suspect is happening is due to a windows bug in MIO the stun/echo service does not work. This used to be a panic with mio and we had temporarily forked and modified it to return error instead of crashing. This is solved in Mio-6 (the port to mio-6 pr is ready for merge - but we only want to do that after testing routing, vaults and clients using it against the droplets to ensure there is no regression/hidden-bugs etc.).

So what can be happening is: If you have only Windows seed nodes and you ask them your external address, you will get nothing. This will affect nodes with port forwarding. To confirm you should be observing these:

  1. You have UPnP enabled - you will be able to bootstrap (atleast the reachability check will pass)
  2. You have UPnP disabled, but port forwarded and you use one or more Linux/OSX seed nodes - you will be able to bootstrap (atleast the reachability check will pass)
  3. You have UPnP disabled, but port forwarded and you use only Windows seed nodes - you should fail the reachability test in most cases.

So without even changing code/recompiling if you can see that ^^^ then it’s definitlely due to the mentioned issue and will be addressed in the upcoming versions of crust.

3 Likes

My gut feeling told me it could be timing, but thinking about it later I thought it shouldn’t be possible as the seednode opens the right port when it starts. Only thing that needs to happen after that is starting a vault on my pc, which opens the right port and make a 2-way to connection. So I agree it would be strange if that wasn’t possible in 3 sec. I’m trying to build the vault locally here though, see if I can get it running in the first place. If I succeed I might try to change that setting to 6 sec. after all just to make sure :+1:.

Just did several tests with @Southside @viv and @ustulation. It all ended up in a hangout and we’ve tested several builds for the safe_vault binary. As it turns out, connecting from windows-to-windows gives trouble on the network. Connecting from windows-to-linux isn’t a problem. Linux-to-linux isn’t a problem either.

@Viv made a build with the proposed update to mio-6 and I was able to connect from windows home PC to an AWS windows machine.

I guess it’s case closed, this “FailedExternalReachability” probably won’t show up in TEST 12c. Will make it more easy to start a community network later on as well ;-).

9 Likes

Thanks for the mention but I did nothing of any use or import - its all down to you guys :slight_smile:

1 Like

I had the FailedExternalReachability error also on ubuntu, but fully disabling the firewall solved the problem.

oto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 *:ssh                   *:*                     LISTEN      -               
tcp        0      0 *:45463                 *:*                     LISTEN      1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 ip-213-127-221-70:49634 ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 d54C55C28.access.:41778 ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 s206-116-50-52.bc:51210 ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 ec2-35-156-220-25:50075 ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 ec2-35-156-100-37:51248 ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 d99-199-170-208.b:59523 ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 client-178-16-39-:42240 ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 ec2-35-157-157-22:42246 ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 ip1f13ece4.dynami:44962 ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 213-66-76-227-no9:49383 ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 85.93.19.51:44886       ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:51174 host86-184-57-178.:5483 ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 static.243.181.46:37804 ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 185.red-79-157-57.:8987 ESTABLISHED 1542/safe_vault 
tcp        0     52 ubuntu-512mb-nyc1-0:ssh 202.109.143.106:1172    ESTABLISHED -               
tcp        0      0 ubuntu-512mb-nyc1-0:ssh 81.241.34.151:50764     ESTABLISHED -               
tcp        0      0 ubuntu-512mb-nyc1:45463 50.ip-92-222-36.e:46362 ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 13.80.119.160:52840     ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1-0:ssh 81.241.34.151:50750     ESTABLISHED -               
tcp        0      0 ubuntu-512mb-nyc1:45463 185.16.37.149:59915     ESTABLISHED 1542/safe_vault 
tcp        0      0 ubuntu-512mb-nyc1:45463 108.61.165.170.vu:52865 ESTABLISHED 1542/safe_vault 
tcp6       0      0 [::]:ssh                [::]:*                  LISTEN      -               
udp        0      0 *:5484                  *:*                                 1542/safe_vault
1 Like

Yes, I’ve seen that one today with my pc as well. Adding a rule to the firewall for port 5483 should do the job :+1:.

I added 5483 and 5484 but this didn’t work. I think the 45463 tcp did block it. I only noticed this (random) port was assigned after the vault was up so I left the firewall of for now.

update:
There’s also a tcp 51174 in the middle i see now?

ubuntu-512mb-nyc1:51174 host86-184-57-178.:5483
1 Like

This topic was automatically closed after 60 days. New replies are no longer allowed.