Update 10 August, 2023

The next testnet, which should be launched any day now, will look at variable store costs. As a quick refresher, when nodes get full, the price of data storage increases in order to attract more nodes onto the network; conversely, when there is plenty of space on the network, the store cost falls.

Before it stores a chunk of data a client now asks a selection of nodes in the address range (a close group) for a price. The price offered by each node is dependent on how much data a node is currently looking after. As we have seen during the testnets, spare capacity, and therefore price, will fall in a distribution. In the last testnet, while a few nodes filled up completely, most were somewhere between half-full and full.

Bearing in mind that each stored chunk will be replicated across a number of nodes (8 currently) it makes sense for the client not to select the lowest price as that might lead to insufficient nodes keeping the payment and preventing replication, but rather to select a store cost towards the top of the range offered, which will be a price thatā€™s acceptable to the majority of nodes in the group. In summary, the client should seek the lowest price that still guarantees that a majority of the closed group will store the chunk.

Getting this right is going to take a few testnet iterations, but weā€™re excited to finally be able to make a start.

Another thing weā€™re looking at is how to exchange DBCs over the network. At the moment DBCs have to be exchanged out of band - via messaging, email or similar - but @anselme and @dirvine are working through a process whereby the DBCs that are output in a transaction will be made available securely on the network, so the recipient can grab them so long as they know the address and have the right key.

Elsewhere, weā€™re working through a few bugs from the last testnet. There are still some issues with connections, with a few nodes not getting added to other nodesā€™ routing tables properly. Another issue is the speed of data verification which is quite slow. We have now introduced a CLI option to not verify PUTs in order to speed uploads, but this may cause problems with larger files. Waiting for 20 nodes to respond before a client can connect to the network was also quite slow, so weā€™ve reduced that number to 8 nodes which will make things a bit snappier.

On the plus side, payments for storing data were processed pretty quickly, so weā€™re very happy with that.

General progress

@joshuef has been heading up the store cost efforts, and has implemented the majority node price mentioned above. He also raised a PR to cache store costs at the client and use that as a way of ensuring nodeā€™s routing tables are updated, rather than pinging nodes each time. This should speed up PUT requests.

Bugfinder general @Qi_Ma has fixed an error in the pruning logic as well as tracking down the root cause of the doublespend verification we talked about last week and is fixing that too.

Meanwhile, @Chriso has introduced the ability to build custom branches of safenode and deploy for testing, so we can now test unmerged code in our modern, Rusty version of the testnet deployer tool.

@Aed900 continues to chip away at ā€˜nodes from homeā€™. Heā€™s built a prototype relay feature allowing nodes that receive a private autonat status from the network (meaning theyā€™re behind a router or firewall) to communicate with the network using another node as a relay.

On similar territory, @bzee investigated an issue with unroutable nodes in libp2p that was causing problems. It may be due to a recent change in libp2p but more sleuthing is required, and that will require some special debug tooling in the network stack it seems.

And @roland fixed a bug in the testnet code where a bootstrap peer was not being provided when launching the faucet.


Useful Links

Feel free to reply below with links to translations of this dev update and moderators will add them here:

:russia: Russian ; :germany: German ; :spain: Spanish ; :france: French; :bulgaria: Bulgarian

As an open source project, weā€™re always looking for feedback, comments and community contributions - so donā€™t be shy, join in and letā€™s create the Safe Network together!

57 Likes

Thanks for the update!

15 Likes

How does this work in relation to an early network when many nodes are needed despite most perhaps being empty so store cost is low and not attracting nodes.

Is this a issue or is the minimum required to launch within the scope of this community?

What are we talking, 2k, 5k, 10k, more?

The requirement should surely be unique node operators not a handful of guys with 1000 nodes on big machines.

14 Likes

I think the early network will likely be a ton of us enthusiasts and hopefully grows very quickly where we are just a small part of the whole thing.

The economics are interesting and I donā€™t believe we really know yet, but the community will help to debate the various bootstrapping things. One thing may be the foundation launch millions of nodes and publish a ā€œSafeDevā€ fund address or similar. Then switch off nodes every week or similar.

25 Likes

Fourth is the new third!!!
:slight_smile:

#StillAPodiumAndDontCareWhatYouLotSay

14 Likes

In this scenario are you saying that the foundation will use the rewards from having the majority of early nodes to start a development fund?

It could work but may also be seen as centralized and backfire if not only needed briefly

Anyhow exciting times, just happen to be driving with a bunch of new nodes.

17 Likes

And that may be the other way :smiley: :smiley:

12 Likes

Personally Iā€™d like to see a repeat of the last testnet, with fixes, that demonstrates it is rock solid, no memory leaks, no nodes falling over, etc before moving the target. but thatā€™s just me. :wink:

14 Likes

Deeply jealous. I always used to enjoy opening Dell boxes :slight_smile:

8 Likes

Thanks so much to the entire Maidsafe team for all of your hard work! :horse_racing:

15 Likes

DM me an address. :grinning:

4 Likes

Only if you promise that they will be full of the original contents :slight_smile:

#NoDaft

6 Likes

I wouldnā€™t do you like that :rofl:

6 Likes

I can feel the maidsafeā€™s fire!!!

9 Likes

Great progress! Looking forward to seeing how the pricing mechanism works, and of course all bug fixes are important steps.

Nodes from home and a slick UI & itā€™ll all be so tangible for those who donā€™t dabble with VPSā€™ and command line :slight_smile:

The pace of progress is exciting. Keep up the amazing work!

16 Likes

Douse the affected part in cold clean water and be more careful in future!!!

4 Likes

Thx 4 all the hard work Maidsafe devs

Super excited for the next TestNet

Itā€™s actually a brilliant idea if the foundation launched nodes. The tezos foundation did it at the start and it was well received, because most understood the importance.

It would be helpful if Passkey could also be integrated into this somehow.

Would be fun if relays would be like a combination of Tor and VPN

Keep hacking super ants

12 Likes

Oh this raises so many questions and a few ways to game this. Just one gaming method is to change the code on my node to multiply the amount by 10 and then when then average is taken or selecting closer to top price then the node will get a lot more then it should. Also the other nodes will too without even asking for it.

Also is the quoting system in place still? If so then there will always be a disparity of price when quote is done and then executed. Now if time is tiny then maybe very close to zero disparity, but if day or two then larger and if it is 30 days then a lot.

Is the node forced to accept the chunk if the upload is done some time after the quote and price has risen a lot? Also what if 2 of the 8 nodes go offline and another 2 nodes have taken over that address range, will they even accept the chunk since they did not give a price for the quoting?? What if the original 8 were 1/3 full and gave cheap price and then they 2 new ones taking the place of the 2 leaving are 2/3rds full, will they accept the chunk??

It would be good for more details on this.

In my opinion the price for uploading a chunk should not be determined by the node by itself because of too many avenues to game the system. Maybe if it was determined by group averaging of say 20 closest nodes and updated regularly (after x events or 1/3 day delay using timer not time)

13 Likes

:white_check_mark: Stable test networks
:white_check_mark: DBCs
:white_check_mark: Memory issues fading
:white_check_mark: Great community feedback
:gear: Dynamic Payment Pricing
:gear: AutoNAT
:gear: UI
:gear: Safe Browser
:hourglass_flowing_sand: Network Upgradability
:hourglass_flowing_sand: Computational Layer / AI ?

Great work @MaidSafe!

21 Likes