Update 22 December, 2022

image

On this, our last update of the year, we’d like to wish you all the very best for Xmas and the New Year, have a great break, enjoy yourselves :beer:, and thanks for all the testing! We’re really pleased to go out on a high with a stable testnet which so far at least is was behaving exactly as we’d hoped [since this was drafted we’ve seen things fall down!]. No doubt saying that will put a curse on it and send it crashing down [yep], but at the moment we’re looking really good. If time permits we’ll put up another one before the end of the year which will allow joins again.

General progress

Mostafa has now finished integration of the consensus protocols we mentioned a few weeks back in the context of membership and handovers. Just testing remains.

@qi_ma has come up with a canny way to separate data replication into batches to avoid the memory spikes that can crash nodes. This seems to be working nicely in our testing.

@chriso has been beavering away on the release process to make sure the testnets go smoothly and @roland is looking into how we can improve observability and tracing with passing logs to the elastic server.

Meanwhile @bzee continues to battle with qp2p and @bochaco is still poking at some edge cases around messaging and connectivity.

Smaller nodes

One issue the previous testnet identified was a bottleneck in replicating data. With large nodes and lots of data, as/when we gain a node or lose a node, the data must be moved around the network. This means we cannot churn faster than we can replicate data. The larger and therefore fewer the nodes, the more this becomes a problem. Internally, we are setting up tests with smaller nodes, say around 1 GB. A node that’s fifty times smaller should have fifty times fewer replication tasks to perform alleviating the bottleneck, thus the load is better spread. This might mean having larger sections (maybe 200 adults per section), or just more sections.

This coupled with the updates to the data replication flow have been looking positive so far.

What would this mean in practice? Well, operators will likely run several nodes rather than one, which would be positive overall as those nodes will be in different address spaces in a section, and when the network grows (it will grow fast), the likelihood is that eventually each node will live in a different section. This means the data will be replicated in 1GB chunks across the network instead of having 50GB replicated in the same section of the network. So, this approach will be more decentralised as well as better at balancing the replication load. There is also likely to be more of a role for elders storing data (currently they don’t) to help in the early stages of a section when we are still busy adding adults.

We are testing this approach internally now with 1GB nodes and if all goes to plan (looking good so far) we should be able to put a public testnet together in the New Year some time. Until then most of the guys are enjoying a well-earned break, although MaidSafe being MaidSafe, there’s always someone, somewhere, tinkering with a new idea or getting stuck into an optimisation. They’re not a put-your-feet-up kind of a crowd.

Take it easy everyone, spend some quality time doing what you enjoy doing. See you on the other side :santa:


Useful Links

Feel free to reply below with links to translations of this dev update and moderators will add them here:

:russia: Russian ; :germany: German ; :spain: Spanish ; :france: French; :bulgaria: Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!

64 Likes

First for once

Wish all the dev team happy holidays and plenty of renewal of spirit during whatever break they have (or don’t have).

Thank you dev team and community members who have put in such hard & tireless work over the past year.

34 Likes

Merry Xmas and Happy New Year

It’s been a good year and a lot of progress… have a well earned rest.

:+1:

28 Likes

Third :slight_smile:

What a year so far and to be having stable(ish) testnets is a fantastic way to round the year off.

all the best wishes to the team for Christmas and let’s look forward to the year ahead :clinking_glasses:

23 Likes

Great end to the year!
Merry Christmas and happy new year to all.

21 Likes

Congrats on all of the clear progress this year and happy holidays.

These two are actually independent factors. As I explain further in the most recent 50Gb testnet thread, you can achieve the same intended goal of large section size with large node sizes as well. Or, I think, preferentially with the network being agnostic to node size. Allow them all, the small as well as the large.

19 Likes

Happy :snowman: holidays and all the best to everyone for 2023 :partying_face:. It can’t get worse again surely?! :crazy_face:

24 Likes

Merry Christmas and Happy New Year to the team. It’s been fun this month with the testnets - let’s keep them going in 2023!

I like the small nodes idea. Opens the door for smaller machines with lower bandwidth and less data. Being able to run lots of nodes seems fine for servers. Easy enough to make a node manager to assist the user in handling multiple nodes.

The churn is something I’ve worried about for a long while. Always seems like it might be a potential problem, but reducing node size appears to fix that. Also like the communication reduction from not having to report node capacity. Of course code simplification is always a win.

All up, seems like wins all around to me.

Hope the team has some good food and relaxation and a great break all around.

Cheers

21 Likes

A snowflake! :safe: … everyone is a unique instance and together a storm.

19 Likes

Thank you @maidsafe for all you efforts, and enjoy the well deserved break!!

:clap::confetti_ball::pancakes::clinking_glasses:

17 Likes

Have a merry xmas team! Relax and enjoy this lovely time of the year.
Looking forward to next year :tada:

16 Likes

Thanks to the whole community. All the debates, all the questions, all the support and all the telling us when you think we are wrong makes this a great place to hang out.

We let you down a lot this year as we hit a ton of bugs, and some took too long to fix. Our issues with qp2p, which still exist, and our debates over consensus, which still persist :slight_smile: And, of course, all that’s happening in the world these last three years has been tough.

This place, though, this place has always felt like home to me. I thank you all for making that so :bowing_man:

43 Likes

Thanks so much to the entire Maidsafe team for all of your hard work! :racehorse:

And Merry Christmas, everyone!

16 Likes

Have a fun-filled festive, folks.
Looking forward to further testing soon.

Thanks to all of the team for all you have given us to play with this year.

14 Likes

Have much appreciation for and tremendous faith in this gritty, little team! :christmas_tree: Merry Christmas to you all and a Happy New Year! Looking forward to the Safe Network’s launch in 2023. :sunglasses: THAT’S right, I said it!!

14 Likes

Well done team on all the hard work :clap:t2: Have a nice Christmas and get some well earned rest :+1:t2:

15 Likes

Looks like SSDs are the only menu item now in order to handle the concurrent read/writes from 1000 nodes per TB disk. Sell your spinning rust plates :smile:
Happy Holidays All!

14 Likes

NO! You did not. Building the Safe network is a mad journey into the mathematical abyss of the great unknown. Personally I’m just here for the ride. Merry Christmas, and Happy New Year! And Judging from the performance of the last test-net, my trusty intuition tells me something feels different, we are getting closer, even if there is still a few hills to climb, cheers!

17 Likes

Thx Maidsafe devs for all your hard work this year

A lot has been done :clap: :clap: :clap:

Super fun to have had TestNets to play with…

Happy holidays to all you super ants that make this fun :ant: colony possible, keep hacking super ants :crazy_face:

16 Likes