Update 23 September, 2021

Today we’ll be looking further at DBCs and ownership. The core network and the DBCs/Mints have been moving forward in parallel, but we are getting close to being able to put them together :crossed_fingers:. There is so much going on at the moment - much of it unfortunately impossible to describe in an overview like this - but in summary it has been a good week for working through some longstanding and knotty problems with stability, connections and testing.

General Progress

@oetyng has been investigating the best way to deal with small files that won’t be self-encrypted because of their size, dubbed Spots to distinguish them from the bigger Blobs (> 3072 bytes) which do go through SE. Spots will include many plain text data files, including DataMaps, so still need to be encrypted unless they are to be public published data.

@yogesh and @joshuef have fixed a bug that was sending a message to all the elders in a section rather than just the selection that are required after an AE update message. This was drastically increasing the back and forth messages on the network causing an amplification of messages (more messages to elders meant more to adults and then more back again). With the fix merged in PR#473 efficiency and throughput of the network has increased, meaning our E2E tests are passing much more consistently.

The introduction of some temporary AE-probing messages is also in the works. It’ll be used by sections to probe other sections in the network periodically, looking for updates by triggering AE flows, and will help the sections stay updated, giving more stability to the network. (This should eventually be deprecated once we have DBC traffic being routed all over the network).

@Yogesh has also fixed a problem with the AE tests, where looping was causing amplification of messaging.

@ChrisO is working away on the CLI and API, updating them to be compatible with the latest versions of the network, and some commands are now working using the genesis key, though this work has yet to make it into main, it should be coming ‘soon’.

@Chris.Connelly and @lionel.faber have been experimenting with removing the connection pool from qp2p. The connection pool keeps connections open after an initial contact, but it is a blunt instrument and something of a black box. We want more control so we can optimise resource usage, and are therefore looking to implement something better in in the safe_network codebase.

Previous memory issues have now mostly been fixed (we’re down to most nodes idling ~80mb and going up to around 250mb under load), but there is one edge case outstanding found by @qi_ma where the log file of a newly relocated node is ballooning in size, presumably because messages are not being received.

DBC Ownership

When designing a DBC system, one decision you need to make is whether or not DBCs will have owners. With ownerless DBCs, anyone who gets their hands on a DBC can spend it, which approximates gold coins quite well, but it has problems of its own.

Since anyone who has a copy of the DBC can spend it, it’s common practice to immediately reissue a DBC once you receive it to ensure someone else doesn’t spends it first.

This reissue involves sending the DBC to a Mint to get it converted into a new DBC of the same value, this DBC churn is wasteful for both the client and network, and worse it weakens our security as sending the DBC to a Mint could result in the DBC being stolen by a malicious Mint node (remember, anyone with a copy of the DBC can spend it and Mints need a copy to execute a reissue).

Owned DBCs resolve these issues by adding the DBC owners public key to the DBC, a Mint will require a prove of ownership before it will reissue an Owned DBC. This proof usually involves the owner signing the reissue transaction.

With owned DBCs, we can store the DBC publicly without fear of theft since only we have the secret keys to spent our DBC’s.

NOTE: We can actually model ownerless DBCs with owned DBCs by packaging up the secret key with a DBC when making a payment.

Privacy and Owned DBCs

To illustrate this section, imagine Sam would like to receive donations in SafeCoin, Sam broadcast their donation public key online for all to see and solicits his followers for any spare coins.

Anyone wishing to donate to Sam would reissue some of their DBC’s to Sam by using Sam’s donation key as the DBC owner and sending the resulting DBCs to Sam.

Now, unfortunately, anyone auditing the Mints reissue logs would be able to find Sam’s transactions by scanning for transactions involving Sam’s donation key. This is not what we want from a privacy conscious monetary system.

We discussed this scenario at length, but with existing cryptography we could not find a way to combine a static donation key and DBCs with one-time-key owners.

Luckily around the same time we were exploring these questions, @mav had been digging into the math behind BLS key-derivation techniques and came up with a really clever technique that allows us to do public-key side key-derivation. The technique lets anyone derive a child key from a well known public key.

For example, in Sam’s case above, suppose Janet wished to donate $10 to Sam. Janet would first derive a new owner key using a random index

owner_index = random_256bits()
owner_pk = sams_donation_pk.derive_child(owner_index)

Then Janet would reissue a $10 DBC with the owner set to owner_pk . This $10 DBC is then sent to Sam along with the owner_index . When Sam goes to spend this donation, he uses the owner_index to derive the owner secret key from the donation secret key:

owner_sk = sams_donation_sk.derive_child(owner_index)

So to recap, we’ve obfuscated the donation public key using a random derivation index. This index is sent to Sam along with the donated DBC so that Sam can derive the owner key when he wishes to spend it. Since the derivation index is never sent to the Mint, the audit logs will not be able to find any references to Sam’s donation key.

The Spentbook and Spend Keys

Reissuing a DBC always involves a write to the Spentbook. The Spentbook is a log of all DBCs that have been spent and the reissue transaction the DBC was spent in, it can be thought of as a large distributed table that maps a DBC to a transaction. Each time a DBC is spent, it is added to the Spentbook. When someone tries to spend the same DBC again, the Mints will notice that it already has an entry in the Spentbook and refuse to sign the reissue.

In earlier iterations of the Safe DBC system, we had the Mints themselves writing the Spentbook entries on each reissue, but recently we’ve moved to a new model where Clients are expected to write to the Spentbook. This change is a significant reduction in load on our Mint nodes as writing to the spentbook and the required ownership proofs verifications were expensive parts of the reissue flow.

To illustrate the change, lets continue from the example above, Sam received a $10 DBC donation and now he’d like to spend it.

First, Sam builds up the transaction he’d like to execute, here he is splitting his $10 into an $8 DBC and a $2 DBC.

let tx = ReissueTransaction {
  inputs: { $10DBC }
  outputs: { $8DBC, $2DBC }

Now that Sam’s specified the transaction, Sam needs to commit the $10 DBC to this transaction by logging it in the Spentbook.

Writing to the Spentbook requires a proof that you are actually the owner of the DBC, the way we do this is to force clients to derive a DBC Spend Key and prove ownership by signing the transaction.

Spend Key

A Spend Key is how DBCs are referred to in the Spentbook. It is derived from the DBC owner using the hash of the DBC.

let dbc = $10DBC;
let spend_key = dbc.owner.derive_child(sha3_256(dbc))

You may be wondering why we don’t use the DBC owner directly? Why derive another key when the owner is supposed to be a one-time-key? Well the Mint does not see the random index used to derive the owner, so we have no guarantee that it is actually a unique key.

The Spentbook requires us to have a unique key for each DBC so we use the hash of the DBC to derive a Spend Key that is guaranteed to be unpredictable. So in all, we have three keys at work here, derived in chain:

well_known_key <-- published widely and associated with a real entity
owner_key <-- derived from the well_known_key using a random index
spend_key <-- derived from the owner_key using the DBC hash

Committing to a Transaction

Sam’s now got a transaction he’d like to commit to and a spend key.

let input_dbc = $10DBC;
let tx = ReissueTransaction {
  inputs: { input_dbc }
  outputs: { $8DBC, $2DBC }

let spend_key = input_dbc.owner.derive_child(sha3_256(input_dbc))

Sam signs the transaction with this spend_key and sends the tx , spend_key , and signature to the network to be written in the Spentbook.

Once the Spentbook has been written, all that’s left to do is to get a Mint signature certifying the transaction is valid. To do this, Sam just sends the tx transaction to the Mint, each Mint node will query the Spentbook and ensure that each transaction input is committed to this same transaction. If that passes, we check the standard reissue constraints, namely the sum of inputs must balance to sum of outputs, and that each input DBC is valid.

Once all checks pass, the Mint signs the reissue and returns the signature to Sam who then aggregates and forms the final output DBCs. Done.

This new reissue flow results in Mint nodes that simply act as validators with a read only view into the Spentbook, this should reduce the resources demands on Elder nodes and give us a speedier network.

Useful Links

Feel free to reply below with links to translations of this dev update and moderators will add them here:

:russia: Russian ; :germany: German ; :spain: Spanish ; :france: French; :bulgaria: Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!


First? I guess

Good work guys, can’t wait to taste the fruit of your hard work :)!


Where is the Spentbook located?


Just to get a message in……


Wow! Holy moly, great job Ian!

Amazing work by the team as usual. Things are really coming along :raised_hands:


This is excellent - A real feeling that everything is falling into place and loose ends are being dealt with and connected to other loose ends.
Well done to everyone involved and a special thank you for the brain power of Ian Coleman yet again :slight_smile:


I was asking the same thing earlier.
Want to know answer too.

It is interesting to know what will prevent attacker from overflowing it.


That is some great progress! Top notch reveal from Ian on derived child keys, more passing test, bug fixes…I feel it.

Thanks for the wonderful update as usual team :pray:


Not first, hoorah!


8th is the new first


Thought it was stored on the network in a register but there was also mention of elders storing it locally. Not sure but those are the most likely scenarios I believe.


and so say all of us…


It’s a data item on the network. So at the address of the DBC and where there are many DBCs on the in side each will contain the same data. So the transaction has to happen in this fashion and cannot be spent in any other way. This allows the idempotency and means if a transaction fails during signing etc. (at re-issue) then the client can just ask again to get the outputs signed.

It’s a nice pattern in these networks to say, here is the transaction I am gonna do when I get consensus and it’s locked in place. Then get consensus (The hard part in a churning network) whenever you feel like it and retry if there are any issues.


Each entry is signed by the “owner” key and contains immutable data. So you search the network for the address and can see it was spent in a specific transaction (or will be)


As it is immutable we can store it as just the transaction set and it can not be changed after written (WORM). As it’s client signed then it’s also like a commitment from the client and that cannot be changed later on. So your transaction is locked by you :smiley:

I like it and think it is unique to us so far, but expect many will copy this, or they should anyway, it removes all sync issues you get from getting consensus plus recording the spend. So we make the client record the spend and then we sign to approve that spend and we can sign the same transaction many many times and it makes no difference.


So the one-time key is valid for multiple signings of the same transaction, presumably?


Yes, well it’s more like this.

  1. We want a DBC to be only able to be spent exactly once.
  2. We want the spend to be signed by the section who can authorise the spend

So the SpendKey is a single use key, it can be spent exactly once. The reason being it is the Address in the network we check to see if the DBC was spent (or will be spent) or not. So the SpendKey signs the transaction set to say, “hey I am the owner of this DBC as you can see” and I have signed this is the way I will spend this DBC(s).

Then the output DBC’s need a sig from the network and that can fail as nodes churn, network connection drops etc. So you say “hey I owned this DBC and I said I would spend it to X so can you sign X”. Where X it is the output DBCs (to make them valid). As X cannot change (As you committed to X in the SpentBook then it’s fine mints sign it whenever they are asked to as X is valid.

So the DBC output X is an amount that is valid to an owner that is valid and it’s fixed in place.

Then when folk spend X it’s the same thing, they say I am making X->Y and write that to the SpentBook and then get sigs on Y.


Not quite understand.
I mean overflowing the storage, not single record.
So if user decides to split his SNT, then merge them, then split again … in a loop - it will result in making more and more records (which will be replicated like chunks were during testnet, right?).
Nothing will prevent it from happening and such data will be stored in network forever.


Great update…