Update 11 May, 2023

Massive thanks to everyone who joined, or tried to join, the Joshnet last week. We really appreciate your enthusiasm, and it has helped us proceed towards the next testnet (naming suggestions please!). First, we were delighted it stayed up for so long. This was completely unexpected seeing as it was basically held together with string and rubber bands, so it’s a great demonstration of the robustness of the libp2p foundation that we are building upon.

For the next testnet we want to use disk storage rather than squeezing everything into memory, which will be a more interesting user experience.

During the Joshnet we noticed a couple of issues that need addressing. One was unexpectedly high bandwidth. We suspect this is due to nodes apparently joining from behind a router (NAT), but not actually being detectable by clients, resulting in a lot of messages going nowhere; if so, this possibly having an effect on other nodes in the network. We’ve also found high node memory and CPU in the nodes, which seems to be due to the way in which libp2p handles data republishing. We’re on that one right now.

Finally, we realise the “onboarding experience” was a little rudimentary (to say the least), so thanks to all those who persevered. Rest assured Chris is on the case to make this much smoother for the next testnet, or if we are quick in fixing the other issues, the one after - we are super-keen to relaunch!

General progress

@roland is taking the lead on saving data to disk in readiness for the next testnet. And @Anselme is porting further data types to use the RecordStore setup from libp2p, so that we can do away with more custom code and rely on their automated republishing.

There are a few bugs with NAT traversal as it is implemented in libp2p and Rust. As a first step, we want to be able to auto detect nodes behind NAT so we can take appropriate measures, e.g hole punching. @Bzee has submitted a couple of issues to the rust-libp2p repo and experimented with AutoNAT so that network nodes can detect whether a joining node is behind a NAT.

In relation to this, @aed900 started on the bootup/discovery process and worked with @bzee on the optimal setup in relation to the role of AutoNAT and the Identify protocol. The Identify protocol is what nodes use to communicate about the observed addresses of other nodes.

@bochaco has been digging into testing, specifically a basic test which can spawn tasks to churn nodes, plus create and query registers all concurrently on a testnet.

@Anselme is working on the DBC refactor, while @chriso is still on the OpenSearch stuff for monitoring, while also putting together the ‘safeup’ script for easy automated testnet launches on a variety of environments. Meanwhile @qi_ma has been fixing bugs, smoothing out the data replication flows and working to reduce node memory and CPU usage too.


Useful Links

Feel free to reply below with links to translations of this dev update and moderators will add them here:

:russia: Russian ; :germany: German ; :spain: Spanish ; :france: French; :bulgaria: Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!

54 Likes

First! Since long time…

26 Likes

Second!! Yay!!

22 Likes

Bronze im on the podium

24 Likes

Ant’s full steam ahead!

SouthsideNet for a naming suggestion. Seems he has been putting in a lot of work to get things going with test networks.

18 Likes

Was just about to suggest this he’s always there to help us technophobes too. I’m not a stickler for the rules so just 2 toes will suffice :crazy_face:.

Looking forward to next testnet :clap:t2:

17 Likes

WillieNet surely. :grimacing:

12 Likes

Thanks so much to the entire Maidsafe team for all of your hard work! :horse_racing:

19 Likes

WilleNet or MarkNet both equally deserving!

14 Likes

kind of you @Josh but DimitarNet is way before me, along with many others. I’ve almost hung my keyboard up and am living off previous endeavours.

I mean, I’m literally standing in a field walking my dog as I type this!

Just back from chasing a hare out of sight. She’s not fast enough to catch anything BTW.

19 Likes

VortNet :stuck_out_tongue: :wink:

11 Likes

The list can go on for a while but you are the only person here that has consistently had a usable and working app across a string of tests, among a host of other contributions. Wille is always here to help and dig in, tough one.

16 Likes

I’ll quote @happybeing here

Dimitar goes out and pounds the streets, puts up posters, raises awareness - all I do is pound the keyboard, occaisionally in a constructive, encouraging way but mainly putting folks backs up and raising blood pressure :slight_smile:

18 Likes

haha my cat woke me with a fresh bunny for the snake this morning :slight_smile:

10 Likes

Just as long as there’s no surgery involved I won’t complain either way.

11 Likes

Thx 4 the update Maidsafe devs

Can’t wait for the next TestNet, the last one was really amazing (It’s actually always super exciting to see SN online)

@happybeing nice picture

Keep hacking super ants

13 Likes

Thank you for all the hard work MaidSafe!

16 Likes

Thank you for all the hard work devs.
@qi_ma has his work cut out to reduce memory and CPU usage right now on file up/downloads though…
Last night I noticed a lot lower memory utilisation on uploads but a serious bump on downloads - not a spike cos it didnt come down again after the download was finished.
Right now on an idling network we still have CPU spikes every 10-15 secs

.
This on a 25 node testnet with 32GB RAM. I have no clue yet about the small upticks in memory usage either - Remember there is no actual file transfer in the period above. I expect this will get solved when the libp2p work is completed.
All in all I can only repeat, we had a most encouraging JoshNet and Im just gagging for some of the in-flight PRs to land and we get new code to play with.
Thanks one and all.

18 Likes

Further to above - I restarted the network, uploaded ~200MB of files and then downloaded them
the significant rise in memory only happens after the download and it continues to step up even though there is no further network activity.

14 Likes