The next testnet, which should be launched any day now, will look at variable store costs. As a quick refresher, when nodes get full, the price of data storage increases in order to attract more nodes onto the network; conversely, when there is plenty of space on the network, the store cost falls.
Before it stores a chunk of data a client now asks a selection of nodes in the address range (a close group) for a price. The price offered by each node is dependent on how much data a node is currently looking after. As we have seen during the testnets, spare capacity, and therefore price, will fall in a distribution. In the last testnet, while a few nodes filled up completely, most were somewhere between half-full and full.
Bearing in mind that each stored chunk will be replicated across a number of nodes (8 currently) it makes sense for the client not to select the lowest price as that might lead to insufficient nodes keeping the payment and preventing replication, but rather to select a store cost towards the top of the range offered, which will be a price that’s acceptable to the majority of nodes in the group. In summary, the client should seek the lowest price that still guarantees that a majority of the closed group will store the chunk.
Getting this right is going to take a few testnet iterations, but we’re excited to finally be able to make a start.
Another thing we’re looking at is how to exchange DBCs over the network. At the moment DBCs have to be exchanged out of band - via messaging, email or similar - but @anselme and @dirvine are working through a process whereby the DBCs that are output in a transaction will be made available securely on the network, so the recipient can grab them so long as they know the address and have the right key.
Elsewhere, we’re working through a few bugs from the last testnet. There are still some issues with connections, with a few nodes not getting added to other nodes’ routing tables properly. Another issue is the speed of data verification which is quite slow. We have now introduced a CLI option to not verify PUTs in order to speed uploads, but this may cause problems with larger files. Waiting for 20 nodes to respond before a client can connect to the network was also quite slow, so we’ve reduced that number to 8 nodes which will make things a bit snappier.
On the plus side, payments for storing data were processed pretty quickly, so we’re very happy with that.
@joshuef has been heading up the store cost efforts, and has implemented the majority node price mentioned above. He also raised a PR to cache store costs at the client and use that as a way of ensuring node’s routing tables are updated, rather than pinging nodes each time. This should speed up PUT requests.
@Aed900 continues to chip away at ‘nodes from home’. He’s built a prototype relay feature allowing nodes that receive a private autonat status from the network (meaning they’re behind a router or firewall) to communicate with the network using another node as a relay.
On similar territory, @bzee investigated an issue with unroutable nodes in
libp2p that was causing problems. It may be due to a recent change in
libp2p but more sleuthing is required, and that will require some special debug tooling in the network stack it seems.
Feel free to reply below with links to translations of this dev update and moderators will add them here:
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!