Update 22nd July, 2021

Exactly, yep. This and sequence eventually will be replaced by CRDT types built atop the register. So there will be only one non-chunk type inside the network. But in terms of use there will still be map/sequence + register for end user use :+1: (aka app use)

26 Likes

Finally back in the land of the living and did not fall off any mountains. (Apologies to those who are disappointed :crazy_face:)

One bug that interests me other than data loss is why each successive upload takes longer.
Are there any ideas to why that happens?

14 Likes

Might be solved now but just a thought that requests are not ignored… and for a few iterations the new elder continues to acknowledge it was an adult and transitions through a [new elder]. Perhaps more complex than worthwhile, especially if it’s shifted position and cannot take the same action. I don’t know well enough what tasks each flavour is doing.

4 Likes

I ran @southside’s test on sn_cli 0.33.1 & safe_network 0.9.1 and it looks pretty good to me :slight_smile:, fixed? tagging @Vort

+  /home/josh/tmp/testnet-file-uploads-20210722_2030/100.dat  safe://hyfeynypzd1yizac8ouqpsazc1ntkp83668wqxhtyhqxgu8cioy5533frfw 
	0.14
FilesContainer created at: "safe://hyryyrbj8h1zqm1xwqheqxywqwzta5su1mmer88y4sz1f4cedfaf7s3ncbanra"
+  /home/josh/tmp/testnet-file-uploads-20210722_2030/100.dat  safe://hyfeynyq4o1a45wj46ibcp9ju7es1aw5y83z7fjaeocembodqskfzmirzye 
	0.14
FilesContainer created at: "safe://hyryyrbm7y8b1hsuwn1idt7o86enu9gnxak8hnks1tkssoasucq3538rz6hnra"
+  /home/josh/tmp/testnet-file-uploads-20210722_2030/100.dat  safe://hyfeynypjah5sjbyrbqqz5rpi4tzmccndrowjqeq8xnaeh36t6wy38hmwhc 
	0.17
FilesContainer created at: "safe://hyryyrbkccwpzsj5aoxumkapmh5o4rkfbfaph96eup91upm9wgnqegohmcwnra"
+  /home/josh/tmp/testnet-file-uploads-20210722_2030/100.dat  safe://hyfeynyk3qi445feww4h543h5idouc1h7ff4qbzof5sik9qtdom4nq44pjw 
	0.08
FilesContainer created at: "safe://hyryyrbxp7p9eputgrz86wg8qy6tsd3xjb1fcaxur4n11uhfrxtnd93tfwonra"
+  /home/josh/tmp/testnet-file-uploads-20210722_2030/100.dat  safe://hyfeynyxq79mgr78eq7by53jb7jnfgjcbmjethz8om4s5repbik3i7jk3zh 
	0.13
FilesContainer created at: "safe://hyryyrbjizugdriee7j78xcn71zypwq1ypt8hyhr68k8d9owpxbeyn76jgonra"
+  /home/josh/tmp/testnet-file-uploads-20210722_2030/100.dat  safe://hyfeynyxjkdxykyies99xfejrdiabnri9yprd6bpg7yx3x6n65qwbg3kstw 
	0.15
FilesContainer created at: "safe://hyryyrbm5xghs5ic3b8p9jc1cbtfpwfk4bpfqdzbrtjbek4ozetwtwydbfanra"
+  /home/josh/tmp/testnet-file-uploads-20210722_2030/100.dat  safe://hyfeynykpyaipz3au1wsh336wukb7qqqdjdywxrq8espnt7arnz5awhokac 
	0.14
FilesContainer created at: "safe://hyryyrbjk54p5no686zga7gfeh5wna68r8g1dpayx7nk3xbhn94b6o7yt6ynra"
+  /home/josh/tmp/testnet-file-uploads-20210722_2030/100.dat  safe://hyfeynyk5cb1sj41jiso8dbj9hn8g6swdiqitdfu3dcnmjgfypb7nbortmr 
	0.12
FilesContainer created at: "safe://hyryyrbxerk6hbt4pf6inw6jb8ogdr9uxobmoatrtub81b7u9hpd8ot49rcnra"
+  /home/josh/tmp/testnet-file-uploads-20210722_2030/100.dat  safe://hyfeynyetpn9fwuwcnbk4g3x8z17fs1d31ipijqizrzqjn63kfg7y7hxbwo 
	0.14
FilesContainer created at: "safe://hyryyrbqz5ejto3db1fdqsz43qz6cf8ob7wexmdg7df59k3u5eo6bq1f9oynra"
+  /home/josh/tmp/testnet-file-uploads-20210722_2030/100.dat  safe://hyfeynymg6gajihji5acjm6rxwkrd5cyuxncj766puc171aezz9zhrd53ee 
	0.14
7 Likes

Never heard of Pedersen Commitments (why else would I have? :joy:) but this video explains it really succinctly!

@dirvine isn’t Bill a friend and supporter of the project?

Can’t wait to see some updated UX!
Great to hear these bugs are getting exterminated.

Oh and are blinded values the same as blinded signatures? If not, how do they differ?

6 Likes

The “blinding” referred to in the summary is different from traditional blind signature DBC’s which normally require fixed denominations, eg as described in the “Scrit” whitepaper. What we have recently implemented is based on “Confidential Transactions” used in monero and bitcoin elements sidechain, except using BLS for (multisig) encryption. In this scheme amounts are encrypted by sender/reissuer so that only the recipient (holder(s) of secret key) can view them, regardless of who sees the DBCs. Even the mint nodes never see the amounts. The sum of outputs is proved equal to the sum of inputs with pedersen commitments and a bulletproof proves that the amount is non-negative. This adds a great deal of privacy, however I should point out that it does NOT make the transactions unlinkable by mint nodes in the way that a blinded DBC would. For that, something like ring signatures or perhaps snarks of some type may be indicated. In practice this may not be such an issue because no single mint node sees all the Tx, and also (perhaps) the mint nodes will keep the info private, though we should not rely on them doing so, and auditing / proof of payment without public disclosure of reissues remain open issues. Also though, linking together amountless, single-use-key tx does not provide much information by itself.

Another privacy improvement we are considering is to enforce/require the use of one-time public keys, which is a bit similar to “stealth” addresses but conceptually a bit simpler as it does not require a diffie-helman exchange.

With regards to traditional blinded DBCs, the mint only processes fixed amounts, eg 1, 10, 100, 1000, etc. Like cash. For each amount, the mint has a unique key with which it signs and validates DBCs. So the privacy pool is the total number of DBC issued with that amount, which could actually be quite small for very large amounts. (In Scrit, the privacy pool is even smaller because there is also a concept of epoch/expiry.) The blinded mint is unable to link transactions (reissues) together, however it CAN see/log/publicize the amount of every DBC, along with date/time, maybe even IP in a naive implementation. We feel that the route we are taking offers a stronger privacy path going forward, as we have just achieved strong (information theoretic) amount hiding and “good enough” unlinkability seems possible.

24 Likes

Keep up the good work! :racehorse:

2 Likes

Thanks for @danda. You let me know what I was wondering about while reading about confidential transaction, .

I think the one-time public keys could be one of the good solution for unlinkability features

I think your DBC + confidential transaction + one-time public keys( or something) is a better innovation than DBC + blind signature.

4 Likes

one-time-use keys are good for preventing users from accepting multiple payments to the same public-key which can harm privacy. So yes, very helpful in that sense. However, it is orthogonal to the (un)linkability I’m referring to, which is based on the fact that a given DBC id (hash) appears as an input in one reissue (tx) and an output in another, thus linking the tx together and forming a graph that can be studied/followed by anyone holding a full set of all the reissue data. Whether anyone (even a mint node) would be able to get a full set of that data is still an unresolved question, and the answer may depend on what comes of further discussions regarding auditability of the system and proof of payment (receipts).

To my knowledge this is the first time anyone has combined DBCs and amount-hiding aka “Confidential Transactions”. :slight_smile:

13 Likes

Yeah me too! :smile:

I’m all seriousness, the (re)assembled UI will come a little down the track, as we are neck deep in the masses of work that comes before that: market research, user needs analysis, mucho planning… It’ll come though!

And I have to say, I’m pretty damn excited about the prospects. It can’t be misunderestimated just what things like DBC, CRDT, pre-upload encryption, storage quotes etc bring to the table. Hopefully that’ll largely go unnoticed by the user. But there’s an elegance to it… a Safe (and the network) is now no-longer in the sole preserve of the online space.

15 Likes

I sincerely hope you find a good solution to unlinkability features soon.

I think this as a very important development. Thx. @danda

4 Likes

I think we agree, the past is over

4 Likes

Thx for the update Maidsafe devs.

Can’t wait for another testnet, would be fun if more people participated and if a testnet stayed online to be constantly bombarded.

:rofl: bravo

Keep up the good work super ants

6 Likes

Receipts seem like such a good feature but it should probably be a choice. An either or scenario. And if you choose to keep a receipt on Safe, there is a lack of anonymity in the real world too that I think most would understand. Can’t say receipts or some level of public verification of a tx is a bad thing but the ability to have one should not hinder the other either so I see the difficulty you have to engineer out of this.

Question. Do reissues have to be kept by a mint to be able to perform a future reissue of the same DBC or could a reissue be considered ‘settled’ and forgotten/deleted?

Edit: I think my answer is

I hadn’t thought of the fact you would have to go through this process again but I can’t wait to see how you present the elegance and possibility, Jim. Best of luck and keep those creative juices flowing!

6 Likes

No, I feel we should not (keep the re-issue, so here I agree with your conclusion) and for bad elders that do we might have a slight worry, but I don’t think so. For single use keys, we only need to write a Key Revocation Packet. Lots of internal discussions on this but I an very very keen we have revocation and use a form of it as SpentBook.

Here’s a snippet

Key revocation

Is a mechanism to identify no longer usable keys. The existence of a key in a revocation scheme means the key is no longer trusted and we should not accept any signature from it. In Safe parlance, it carries zero Authority .

Key recycling

Is a mechanism that effectively revokes a key and replaces that key with a new key. In Safe parlance, this means to carry over the Authority .

Key types on Safe

We have

  1. Data keys (a BLS key that Authorises mutations to the Data network).
  2. Node key (an ED25519 key that identifies a node)
  3. DBC key (a BLS key that represents a Dbc “owner”)

Each has differing revocation requirements and we will explain these next.

Detail

Data key revocation

These are always only revoked by the “owner” of the data. However, the choice is to revoke or rotate . Rotation allows the owner to move to another person or simply a new key, in case of possible security issues and users wishing to update keys, i.e. we can carry over the Authority . Revocation means this key can no longer authorise mutations.

Note: This key can be that one given to apps to mutate data and revoking this effectively kills the app for that user.

key == BLSPubKeyName + tag(01) // XorUrl
content ==
enum { Revoke: Signature // of key,
       Rotate: NewBLSPubKey, Signature // of key + new key, 
     }

The Signature in both cases is produced by the key being revoked/rotated. i.e. only the key owner can revoke these.

Node Key Revocation

This is primarily a key rotation mechanism and we insist nodes rotate keys when being relocated, promoted or demoted. This gives us a trace of a node key history from its first Join . As above, but note the network can Kill the key.

key == EDPubKeyName + tag(02) // XorUrl
content ==
enum { Rotate: NewEDPubKey, Signature // of key + new key,
       Kill: (SectionKey, SectionSig), 
     }

Here a node can rotate his key (only once) proving to any network section he has done this exactly once. The network can kill this key in the case of bad behaviour. ( @qi.ma this prevents the Sybil attack on relocation/demote/promote). Note: a new node joining will not have rotated a key and this packet will not exist as there is no previous key. On the first promotion, this packet has to exist to rotate from his initial key.

DBC key Revocation

Here we assume single use keys. This means a key is spent exactly once, although that may mean several DBCs with the same owner being re-issued, that key is then dead. We assume all DBCs have a nonce along with the owner field, this allows owners to derive keys for their DBCs without knowing anything except the nonce and their base secret key.

This packet contains the Hash of all inputs + outputs. This is used to link the spend against a particular set of transactions.

key == BLSPubKeyName + tag(03) // XorUrl
content == Spend: TransactionHash, Signature, // of key + hash,

The client will create the DBC KRP as follows:

FOR EACH input
  CALL hash_extend
END

FOR EACH output
  CALL hash_extend
END

CALL hash_complete
CALL create KRP
CALL store KRP

When this packet is stored then the client can request a re-issue. For many different input “owners” then this packet need stored with the same TransactionHash at the location of each key and signed with the appropriate key.

Mint re-issue

IF sum inputs == sum outputs
  AND EVERY input owner has a `KRP` with the correct TransactionHash
  AND EVERY output key has NO `KRP`                  
  AND EVERY input `parent key` has a `KRP`
THEN sign all output DBCs  
ENDIF
12 Likes

eliminating data loss!!! let’s have the wayback machine copied on here asap!

5 Likes

Maybe dumb questions but

  1. Are Pedersen Commitments considered discrete log contracts? Which I believe lightning network uses. I may be way off base there.

  2. I very curious how batching will work!?

1 Like

They are of a similar niche, but not exactly the same. You can do more than just confirm sum inputs == sum outputs with the partial homographic parts of pederson and more so with more homomorphic libs. So you can start to create a contract language from this. So kinda really :smiley: (btw easy answer is they are an important part and the discrete log issue comes from these commitments when using ECC and not RSA)

For us, self encrypt a ton of data, get store cost then pay for them all in 1 single re-issue. Then upload all data with payment (this is a 10,000 foot view though)

8 Likes

You know, when I heard Buchanan say homomorphic encryption I started wondering if you guys were clearing a path to pave later :smirk:.
My imagination started reeling a bit and I was wondering, if it’s using homomorphic, which should preserve privacy, could this feed into Big Data about the Safe Network economy? How money flows, supply, etc? Maybe I have that wrong about the computed results being anonymized?

The fact you could create a contract language is music to all of our ears and possibly a necessity in this market. Although smart contracting can be janky, dangerous, etc and should use formal verification IMO, it does still seem a necessity.

Thanks for taking the time for these responses @dirvine and @maidsafe team. They are really insightful.

7 Likes