Crust Test - NAT Traversal (concluded)

crust
test

#1

As we mentioned yesterday we’re excited to be taking the next step toward SAFE-Fleming, our next major milestone, and we’d like to take you on this journey with us. We have been making updates to our networking libraries and would once again greatly appreciate your support in running this Crust Test to help us understand how effectively Crust facilitates connections between Peers, even when not directly connected to the Internet.

This tutorial will guide you through the following:

As results arrive and issues are logged new versions of the apps will be released to continue to improve these findings. We’ll let you know and ask you to run again.


Before running the test

There are a few steps that you have to follow before being able to connect to the test network.

  1. If you don’t already have one then you’ll need a SAFE Network Forum account.
  2. Once you have registered the account you’ll need to have reached Basic Trust Level 1.
  3. You’ll need to then register your IP address to get an invite to the test.
  4. Now we’re ready to download the test.

That’s it!


Running the test

The test is a simple application that lets you connect to other peers running the test. To run it all you need to do is:

  • Run the client executable. (MacOS users Read this first).

  • Enter a name - this can be real or made-up (optional). I’ve entered “DGEDDES_MAC”. Pressing the Enter key will allow the client to start looking for peers to connect to.

  • … that’s it - you are connecting to peers and participating in the test!

  • Leave the client running for as long as you are comfortable with. The longer it runs the more peers it will find and connect to.

The application then collects your NAT (router) type, operating system, confirms whether UPnP is supported and then starts to look for other available peers to connect to.

If others are participating in the test at the same time, you will see connection attempts being made, whether these are successful and the type of connection made e.g. TCP or UDP.

The app will notify you when all known peers are connected to, and then will check for new peers every 10 seconds.


The Crust Dashboard

The Crust Dashboard shows you the global results of the test so far and displays the hole-punched peer-to-peer connections that have taken place.

A range of geographic locations, NAT (router) types and operating systems which will be displayed.

Whilst we currently show the global connection attemptes, we plan to add the ability to filter dashboard results to let you see your own connections in isolation.

There are 3 main sections to the dashboard:

  • Filter - controls what results are displayed on the chart and the table
  • Chart - shows the percentage of successful connection attempts for each protocol
  • Table - shows connection attempts in more detail based on the filter
Crust NAT Traversal Dashboard

Please note the dashboard is designed to be viewed only on desktop computers. Viewing on lower resolutions (tablet/mobile) may result in a less than perfect experience.

What’s shown on the table?

The table on the dashboard shows the following content for each attempted connection:

# TCP HP UDP HP NAT Type Operating System Country
Unique identifer Connection attempt result Connection attempt result EDM / EIM / EDM Random Linux / MacOS / Windows Geographic location (derived from IP Address)

What do these terms mean?

Term Description
TCP HP Hole punching via Transmission Control Protocol (TCP)
UDP HP Hole punching via User Datagram Protocol (UDP)
NAT Type Network Address Translation enables a device to connect to the internet (through your router). Types available are: EDM; EIM; EDM Random.
EDM Endpoint-Dependent Mapping (EDM) involves our external address changing depending on the remote endpoint we are talking to. Most routers have a fixed offset they use when changing the address so can be predicted.
EIM Endpoint-Independent Mapping (EIM) allows the device to accept incoming traffic to a mapped public port from ANY external endpoint on the public network.
EDM Random More complex version of EDM where there is no fixed delta (offset) when generating the address.

Duplicate results

If multiple clients are run from the exact same environments (same IP Address, Operating System, NAT Type, TCP HP Result & UDP HP Result) then the results will be considered a duplicate and the results will be ignored (i.e. not recorded). This is also true if the peer_requester and peer_responder have been swapped.

If the clients have any differences in their environments (e.g. NAT Types) then the results will not be considered duplicates and will be recorded on the dashboard.


Privacy Policy

For the test we record the currently active IP address of your login in order to avoid spamming attempts.

When running the test application client you’ll see your end-point info (IP address and port number) and the end-point info of any peers you are directly connected to.

If you would like to learn more then please see the Privacy Policy on safenetwork.tech.

Cookies

The invite server (crusttest.maidsafe.net) creates a single cookie once the user has been authenticated using their Forum account. The cookie contains the following information:

  • Session ID

Giving Feedback

As always, we’re keen to hear from you so if you have any feedback on the client application, dashboard or overall test process please just reply to this post. Also, if anything isn’t as clear to follow as you’d like it to be, or you have run into a problem, then let us know below.


Frequently Asked Questions

How long is the test scheduled to run for?

The duration is not fixed as it depends on the results of the test.

What is the Client Application?

An executable which users download and enables two peers to connect to each other, via the proxy node. The proxy node is the hard-coded contact which all peers bootstrap to, and it also receives the connection attempt results from the peers in the test.

What operating systems are supported?

The client application is supported on the following operating systems:

  • Ubuntu 18.04 x64 (or higher)
  • MacOS 10.13 (or higher)
  • Windows 10 x64

Why do I see ‘“Client” can’t be opened because the identity of the developer cannot be confirmed"’ on MacOS

On MacOS you can expect to see a pop up that warns that the client executable “can’t be opened because the identity of the developer cannot be confirmed”.
mac

The software has in fact been signed but because the client is a Unix executable rather than a packaged Mac app it is flagged by the system. To resolve this:

  • In the Finder, go to where you downloaded and unzipped the client.
  • Control-click the client icon
  • Choose Open from the shortcut menu.

The app is now saved as an exception to your security settings, and you can open it in the future by double-clicking it just as you can any registered app.

To confirm the executable has been signed by MaidSafe you can open a terminal and in the same folder as the executable run:
$ codesign -dvvv client

We provide the checksum for all our zip files so users can be assured that they are approved by MaidSafe.

We’ll be looking to see if we can improve on this as we move forward.

What data is captured by the Client application?

  • IP address and port number
  • NAT type
  • Operating System
  • Protocol method
  • Time to successful connection
  • UPnP support check

NAT type, OS, protocol method along with connection attempt results are sent to the dashboard for display. More granular data, such as successful connection time and UPnP check, is logged for detailed debugging, but is not publicly accessible.

How long should I leave the client running?

The longer the client is running the more peers it can connect with but it’s completely up to you how long you want to run the test for. MaidSafe will have peers running for the duration of the test.

Why do I need an Invite?

By registering your IP address at the start of the process, your IP address is recorded by the hard coded contact (the proxy node). Then when you run the client application, your IP address is matched with that on the proxy and you are authorised to the test. This is how we prevent spamming attempts.

What is the Crust Dashboard?

The Crust Dashboard allows anyone to view the crust connection results in a meaningful way. It does not display all data received link to ‘what data is captured by the client application’

What database is used?

A MongoDB database is used to store the connection results received from the test.

What’s in the ZIP File?

There are 4 files included in the downloaded ZIP file:

  • client: This is the client application that the user runs to connect to others in the test.
  • client.crust.config: This is the main configuration file which defines some essential parameters for the client application including the hard-coded contact.
  • log.toml: This file configures the output content and format of the log file.
  • p2p-config: Additional configuration file which lists some hard coded contacts and hole-punching and rendezvouz options.

An additional 2 files are created when the client runs.

  • client-##########.log: This is the log file which collects all connection activity (such as attempted, successful and failed results).
  • client.bootstrap.cache: This is for future use when we connect with multiple proxies (currently there is only one proxy).

What is my end-point info?

When your node connects directly to another node you can see their end-point info within the console app. The end-point info is:

  • IP Address
  • Port number

Conclusion of Crust Test (v1)
Crust Test v2 (concluded)
Crust Test: Why Encryption Matters
SAFE Network Dev Update - October 18, 2018
#2

First!!! Does it apply to this as it’s kinda nd of an update?


#3

This is great. We need more people testing. So far mostly all Maidsafe team members out there


#5

:slight_smile:

I keep expecting the IT department to contact me one of these days when I run these tests from work. lol


#6

I think you should either make it easier to participate or give some incentives for it, if you want more people in. This is great and I’d like to run it and help, but most people won’t go through all the effort of getting the required forum level and the installation process without an incentive. Something like what NOIA is doing would help: https://medium.com/noia/instructions-to-join-the-testnet-7d46c14127f8


#7

This would cause us all sorts of issues. We have been careful to not identify people to this level.

Sign up on NOIA Network dashboard and complete your KYC application in order to get approved and therefore qualify for the Airdrop.


#8

It looks great :tada:


#9

I think the process was really easy and well explained, excited for more!


#10

I could see more Failure connections than Success connections :thinking:


#11

Works like a charm here(linux).

Running openwrt with no specific settings.

And, is it a good idea to show ips?


#12

@im-tony you probably have a more restrictive router.
Personally I was able to connect with everyone so far:

My router implements NAT with endpoint independent mapping.


#13

I’ll connect with you all in several hours when I’m home! Way to go @maidsafe


#14

Having problems connecting and think it is related to a problem I had before, which is I don’t have an public IP.

From ISP support page:
There is a lack of IPv4 addresses in the world. Therefore, X uses non-public addresses.
in expectation of IPv6, ISP x is gradually transitioning to private IP addresses.

The positive thing is that it is possible to ask for a public IP.
Should I post the text log from crust_test here?


#15

i got 339 successful and 137 failed connections , took me 1 minute to complete the test, fast as it can be…


#16

We are mainly trying to find out how many private ones can we reach. So how many routers can we forge our way through and reach despite people being behind NATs - so all info is good (especially good to know what is the % of NAT Traversal we can do ATM)


#17

My router thinks there is another router in the way because the ISP has given me a private IP instead of public IP adress, so no connection was succesful. :frowning:


#18

crust_test-v0.1.0-linux-x64
=>

./client: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by ./client)
./client: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by ./client)

#19

image

I feel so connected right now!


Let me know if I can show any information that will help with these tests!


#20

i got 339 successful and 137 failed connections

I think you’re looking at the global number of connections from the dashboard (?)


#21

@ustulation What can I do the best with my router to get more connected counts?