Thanks to all of you who have mucked in with testing ThePriceIsRightNet. It really is a massive help, and once again you’ve uncovered some unexpected issues.
Indeed, this one has been a bit of a curate’s egg - good in parts.
Starting with the good, all our nodes are still alive with no serious CPU or memory issues. As planned, the store cost has increased as nodes filled up, which is great, although it’s not been massively consistent.
Which brings us to the slightly whiffier bits. The cause of the dreaded “Network Error Could not retrieve the record” is most likely due to bugs in the payment and verification system with clients making insufficient payments in return for storing chunks.
If the payment is insufficient, some nodes will reject the chunk, meaning it fails to replicate across the close group. Repeated attempts also fail because the client has been reusing the initial payment proof again. The payment is still insufficient, so nodes reject the chunks again. The client can’t retrieve the chunks since they were never properly stored. The node price may indeed be right, but the client isn’t calculating it correctly at the moment.
So… we’re working on improving the cost calculation, checking current prices on reuploads, and paying per chunk. This should help ensure sufficient payments to all nodes meaning everyone’s happy.
Elsewhere, client bootstrapping is still sloooow despite reducing the number of nodes from 20 to 8. We have found a bug and have a fix in place there, which is speeding things up nicely in our testing.
Other stinkies are still hanging around. We continue to see nodes ending up with no stored records; there’s still a slow memory leak. These are not showstoppers and we’re investigating them while moving on in other areas. Unfortunately, we still can’t use QUIC as a transport because the
libp2p implementation still wrongly identifies nodes behind NAT as public, and appears to have a larger memory footprint than TCP at the moment.
@Anselme has been looking at streamlining DBCs. The current model is to some extent a hangover from the previous pre-libp2p design which used section keys as a point of reference. These no longer exist and we are now looking at a flatter setup that behaves more like a decentralised ledger, with transactions stored on the network and with nodes tracking unspent coins. More about this in a future update.
@Bzee is also working in this area, including looking at potential workarounds to the current AutoNAT challenges with integrating QUIC.
Feel free to reply below with links to translations of this dev update and moderators will add them here:
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!