For the next testnet we hope to put to bed the memory issues that sank the last one, so that’s been our focus this week.
The pay-one-node branch is looking good, and with a bit more testing will be ready to merge. It should reduce memory usage compared to the previous approach of paying all nodes that store a chunk. As we mentioned last week, ultimately we should be able to offer upload speed and assurance trade-off options in the client.
Paying just one node also simplifies the royalty payments. Compensating five nodes means five separate royalty messages being broadcast for the same chunk, which then need to be deduplicated by the receiving nodes. This is complicated by the fact that these
CashNoteRedemptions are encrypted. Using one node simplifies this a lot. In addition, these gossiped messages no longer need to be encrypted (with no loss of security) so we’re putting that in now.
The exact impact on royalty payments memory use needs to be tested further on a larger testnet, but it looks like some other simple optimisations are possible too. For example, setting each node to subscribe only to relevant messages, not all messages, has seen a big reduction in mem, and we’re looking to stop unnecessary rebroadcasting too.
GossipSub itself, we’ve made verification stricter for testing. Rather than just expecting a minimum number of messages now we check for the exact amount.
Now we’re going to hand over to @anselme to walk us through how payments work.
This is a summary of how we pay for data on the Safe Network as well as an introduction to money transfers in general.
Before we get into data payments, let us first understand how basic transfers work. Everyone on the Safe Network, both clients (wallet apps, Safe CLI, etc) and nodes (data storage) have a public/private keypair. The public key (aka
MainPublicKey) is used to receive tokens. It is equivalent to bitcoin or ethereum addresses. The private key (aka
MainSecretKey) is used to sign transactions and prove ownership of money.
This is what a
MainPublicKey looks like:
You can obtain your own with the command:
safe wallet address
Money in a wallet is stored as
CashNotes. These are like a cheque for a given value. They contain all the necessary information for the owner to spend the value in them, but don’t have value on their own as they are useless without the private key.
When someone wants to send money, they create a Transaction where they spend their own
CashNotes in exchange for new
CashNotes that are owned by the recipient. As those
CashNotes contain secrets that might affect the privacy of the sender and the recipient (the
MainPublicKey), they are never sent or stored on the Network.
Instead, the sender creates very small
CashNoteRedemptions that contain the minimum necessary information that the recipient needs to verify the transaction and re-create their own
CashNote on their side.
CashNoteRedemptions are encrypted and packed into a Transfer which is sent directly to the recipient. The recipient of the Transfer can then decrypt it using their
MainSecretKey, verify and rebuild the
CashNotes, before adding them to their wallet. At that point, the transfer is complete!
You can send a Transfer with:
safe wallet send <amount> <to>
safe wallet send 42 93d01b3c6c0d41c4e50c855c753f906ba478f97a838415e6d74615a4b037a7101e724f935727bbf23d17293ab74a3027
This will print out the encrypted Transfer that can be published or sent directly to the recipient.
They can receive it with:
safe wallet receive <transfer>
Data on the Safe Network is paid for using those same Transfers.
Before data is stored on the Network, it is prepared locally by the user: cut into chunks and encrypted. Each of those chunks have a unique hash value from which we derive their
Nodes on the Network also have a
NetworkAddress. Data is stored on the nodes which have the closest
NetworkAddress to the data itself. This is why we say that data is content-addressable.
By querying the Network, a client can get the closest nodes to a
NetworkAddress. Once we know them, we first perform a
StoreCost query to ask the price for holding data at that location. The price depends on the storage capacity of the nodes, which ultimately depends on demand.
Once we know the
StoreCost we can safely send the data along with the Transfer paying for it. Upon receiving the data, nodes verify the Transfer, take the payment, verify the data itself and store it.
Along with that Transfer, another small Transfer is sent:
NetworkRoyalties, These are sent to a pool and help keep the Network alive. When receiving data payments, nodes make sure that those
NetworkRoyalties are also valid before storing the data.
All of this is done automatically when you upload data using the CLI:
safe files upload some_file
@roland Submitted a PR to resume CLI uploads across multiple runs. This stores chunks locally and moves/removes them once paid/verified. He also refactored the codebase to release RPC client as a binary to fix some testnet deployment errors.
@qi_ma also created a PR to avoid duplicated hash work, making the most of the latest changes to libp2p.
@jimcollinson has been looking at what’s needed for an MVP launch, with a focus on on-ramps and off-ramps for a soft launch.
And @joshuef has been drawing up a priority list of the remaining technical work before we enter beta.
Feel free to reply below with links to translations of this dev update and moderators will add them here:
As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!