RFC: Data Chains (Datachain)

Since a few days “Data Chains” is an official RFC. This is the topic to discuss all ins and outs of this RFC.

Summary

DataChains are a container that allows large blocks of data to be maintained. These blocks can be validated by a node on a network, close to the data name, to contain valid data that was guaranteed to have been correctly stored onto the network. Code for this RFC is available on github.

Here’s the link to the dev discussion on the dev-forum.

Here’s the link to the RFC on GitHub.

Here’s the link to the first topic on this subject on this forum.

Here’s the link to a blogpost (Metaquestions by @dirvine ) from David.

As this is an official RFC now we might want to talk Data Chains in this topic. Enjoy :thumbsup:

12 Likes

This is a post by @dirvine as posted on the Dev Forum.

The way I see this coming into play is in phases. Here are some example notions, although I believe there is a great deal more that can be achieved. These points are merely to help the reader see a larger picture and perhaps see other immediately beneficial uses.

  1. Data Republish, allow a node to effectively prove a DataBlock is valid on the network. This holds the type and hash of the actual data, so the block can exist without the data, but the data at any time can appear and be known valid.
  1. Network entropy measure - we can measure a blocks lifetime on the network in terms of where it was created (first seen) in terms of which bucket it was in (as the network grows/shrinks buckets or common leading bits change). This measure can be used for many measurements and ranking type mechanisms, such as how many churn events has this node bene in a particular group, or noting a mass increase in a data type (such as safecoin) in particular parts of the network etc. (there is a lot can be pattern analysed).
  1. The ability for ledger based systems (with a small change to SD as currently coded in the data_Chains repo) where an SD item can be Put with a ledger bit set (so not at version 0). This can be used on per type or per transaction basis. So features such as I want to keep a receipt of a safecoin transaction or similar can actually be possible. Same can be used where we want appendable items to persist (force ledger bit set on immutable comments etc.)
  1. Ability to track a nodes history in the network even with changing id’s if it were deemed useful to track a nodes rank over sessions or through time.
  1. The ability for differing size nodes, where not all nodes in a group hold all data. This is a simpler mechanism but likely vital.
  1. Ability on data republish for a network to collapse fully and then reconnect without data loss. This is why archive nodes initially hold more data than they can be asked for. Also helps with software catastrophic failure (such as bad update, but this shouldn’t happen if we implement update validation).
  1. Ability, if required, to force one use only crypto keys? This makes joining more difficult as network grows over time.
  1. With merkle locks (Qi currently adding the detail to the RFC) in place then the Genesis groups can be secured and chains validated form that point.
  1. The possibility of two network (re-)combining from differing genesis blocks. This is interesting and does require rules as to which data will be accepted and how it’s paid for, but could be a considerable addition to decentralised networks, perhaps even in read-only mode to start with. This is for sure a separate phase/rfc group, but very interesting.
  1. Securing group claims across the network. With secured genesis blocks then link chains can be transferred across the network to prove a valid network node in a particular group. This could be granularised based on distance between nodes (xor) and secured checkpoints (the location in the chain where groups split).
6 Likes

Network entropy measure

That’s very cool and I think the word “entropy” is very important.

Like @dirvine, I hold some very strong beliefs about time weakening our actual perception of life. It’s the most simplistic and error-prone way to measure entropy.

So if we can cut away the fat and get down to brass tacks by measuring entropy instead of time, then we are taking a giant step in the right direction.

Bravo guys…bravo.

5 Likes

Have the devs looked into the “Threshold cryptosystem” for concensus? Maybe it’s not practical for SAFE as each node in the network has its own identity. On the other hand it might get used for group_consensus to keep the group_signature small.

In short:

  • 1 node creates a private and public key
  • The private key is split into 10 pieces
  • The pieces are shared by the node and the original full private key is deleted

Now the group can sign things using just 1 public key while no one still has the full private key. Actually quite an interesting concept. Hard to understand if you want to know the details. You know, math and stuff :yum:.

Here’s a nice presentation of the idea for people using several devices. If someone steals you phone for example, they only have a part of the priv. key so they can not takeover all your data.

5 Likes

We’ve looked into threshold crypto a little but so far haven’t been able to find a system that suits our needs. One complication is that the shared keys need to be constantly regenerated as new nodes are added and removed, and sections merge and split. Creating a system that is decentralised enough and fast enough for this kind of use seems like a really huge task (if it’s possible at all). That said, if we did find an appropriate system it would definitely be considered for adoption.

(that’s my take on it anyway from the last time I looked into it)

7 Likes