Limits of the Safe Network Consensus Algorithm

[krnelson]

Recently the RFC 0061 — Safe Network Token Distribution RFC was posted concerning the politics of where and in what proportion tokens will be distributed. I was asked to start a new thread since my off-topic thoughts over there concern what happens before the actual distribution decisions that the RFC covers.

This thread is not concerned with the politics of who and in which percentage tokens are distributed, the RFC is the best place to take those.

At a high level two methods for the creation of SNT have been mentioned (references at end):

Method1:

• 100% of SNT are Pre-Farmed at genesis.
• 30% (upto possibly 100%) Airdropped or equivalent to initial stakeholders, otherwise…
• 70% Pre-farmed SNT are distributed to participants over time in the most secure way possible but from outside, meaning external the Safe Network consensus algorithm.

Method2:

• 30% of SNT are Pre-Farmed and airdropped or equivalent to initial stakeholders
• 70% Are created based on need and as required and the whole process managed by the Safe Network consensus algorithm.

In some surprising news @oetyng has informed us that currently they do not want to see (likely meaning it is not possible for) the Safe Network consensus algorithm to secure the data structures and related rules for SNT data. Meaning Method2:

@oetyng

Reality vs Wishes

The problem in my view is this:

• We don’t want large sums of tokens being controlled by sections.
• We don’t want sections to be able to create tokens.

Why?
Because we don’t know of a secure way to do any of those now - nor do we know when or even if we will learn a way.

On the RFC thread I lead off with this question (repeated below and slightly modified for clarity):

Why can the Safe Network Consensus algorithm be trusted to handle data in a secure way, run a bunch of rules for handling it, but not token data and rules for handling that?

There were a couple of responses on the RFC thread:

@neo pointed out that is was to reduce the attack vector against the SN consensus algorithm (“Section” for short).

A first thought to this not being a reason to discount Method2 completely is that Method1 has as much if not more attack surface than Method2, both political and technical. In addition the Safe Network Consensus algorithm is going to come under maximum attack anyway, regardless.

@JimCollinson informed us that:

@JimCollinson

Additionally, there is also issue of audibility of supply. This becomes much a more complex and difficult problem to solve—and may be impossible while maintaining privacy—should the entire supply not be minted at genesis.

I first read “audibility of supply” to mean “audibility of total supply”, and speculated that it could possibly be solved with the hard limits provided by u64/u128 and well chosen SNT divisibility rules. Those supply limits could not be broken easily without being noticed.

However since that appears too easy and on further reflection what Jim might really be saying is the problem is “audibility of distribution” which appears to have at least two parts:

1. In what percentage and to who is SNT being distributed as covered in the distribution RFC. In Method2 this would always be under decentralised control of the node operators. So this raises the possibility that perhaps politics is playing a role in not wanting Method2 and it is not actually a technical limitation of the SN consensus algorithm.

2. Audibility of the rate at which SNT is created (and possibly destroyed if that even possible with DBCs) by the SN consensus algorithm.

It is not clear is one or both these secondary problems are what the Safe Network Consensus algorithm is unsuitable to handle, or whether it really is “audibility of total supply” and u64/u128 is not a politically viable solution.

@oetyng

Now, welcome community, and solve this for us all; what other ways are there?

…
Something else? Then PUSH for it.

If you don’t, then this is what you’ll get.

I would like to push for decentralised solution meaning getting to a place where the Safe Network consensus algorithm can be trusted with all data types and the rules for handling it. Not doing so sends a very powerful wrong message that will be used to discredit the project by its detractors on launch, not to mention the new attack vectors Method1 opens up to both political and technical as we are getting a glimpse of over on the RFC and related threads.

If Method2 really is definitively impossible then so be it the devs are the ultimate experts (@oetyng, @danda and @bochaco and any others i missed from reading through the updates, sorry). That said It may still be productive to get some more insight on the technical/political realities of why Method2 is not possible for those of us that hold out hope for there still being a chance.

References

RFC 0061 — Safe Network Token Distribution RFC

Proposal for Encrypted Distributed DBC’s (now with poll)

Update 5 May, 2022

@maidsafe

Current thinking is that all SNT will be encoded in a single genesis DBC - one with no inputs.

So, all SNT that will ever be created will be in this one form, like the universe before the big bang. Once the network launches for real, the genesis DBC will be reissued and subdivided and spread across the firmament. The best mechanism for this distribution is what we’re looking into now.

Update 21 October, 2021

@maidsafe

It is also interesting to think about what our total money supply could be. Just as a hypothetical starting point, let’s say we come up with the Genesis DBC amount by taking the current SNT market cap and calculating a Genesis amount such that 1 DBC = 1 USD. I haven’t calculated what the Genesis amount would be by this method, but let’s just pretend it is 50 million. Ok, so we launch with denom One (1*10^0) = ~1 USD and the Genesis DBC = 50 million. cool. We will ignore distribution/farming effects for now, except to say they could temporarily inflate/devalue the currency.

Update 16 September, 2021

@maidsafe

The decentralization functionality can be broken up into a few main areas:

1. MintNode peers within a Section.
2. Sharding: Each Section is responsible for a subset of DBCs.
3. Client Aggregation
4. Distributed Spentbook written by Client

[dirvine]

@krnelson

Why can the Safe Network Consensus algorithm be trusted to handle data in a secure way, run a bunch of rules for handling it, but not token data and rules for handling that?

I think this is a misconception. There are 2 points to think of and see why I say this.

1. Network secures data, but does not securely create data
2. Network secures data by making it permanent

Whereas tokens are either created by the network or their ownership is transferred in an atomic manner.

For the former then we all know Safe and how it works. For the latter then we know with data we can use CRDT/BRB consensus rules etc. but the previous owner is a person who signs an agreement to transfer the data, if they conflict via concurrent access we have fork resolution mechanism to allow the, however a fork when talking money is where doublespend happens.

Hoep this helps

[neo]

You mentioned this in your post, but was thinking this before I read it there.

For method 2

The distribution of the 70% has to be beyond the normal algorithm of taking the input DBC(s) and then distributing them according to the RFC stated proportions.

How do we determine section to section how much should the section be adding to each of the stated proportions. Meaning what if one section has distributed a certain amount but another section 1/2 that amount at a certain point in time. This could be because of the number of chunks stored, updated (append), data blobs updated (account blobs etc), app dev rewards given out, etc.

How does the 70% get evenly distributed amongst the differing rewards, does each section get a set portion of the 70%, then what if the original section splits into 2, so each has 35% to give out, then after a period of time one of the 2 split 4 times and the other 3 times. But unevenly which is to be expected. the 35% down one path ends up with 12 sections and some with around 2% and other 4%, the other path has some at around 8%, 4%, & 2%. Does this now mean that if one ends up storing their chunk in one section they get 4 times extra compared with another section?

This is what one would expect if method 2 is used since there is no network wide consensus on how much is to be given out.

Then for the network to know exactly how much exists is another issue if token is created, since there is no set amount created. Race conditions could see the total SNT exceed the agreed total supply by a small amount, or even many % depending on how timely any audit of network wide supply is.

But if say method 2 is used but instead of creating SNT the whole is created in the genesis section and the “pool” DBC the section has is split evenly between the 2 new sections. At least this way the total SNT across the network is not going to increase or decrease. BUT still have the issue of non-equal extra payments across sections given when slowly distributing the 70% as shown in the example above.

Obvious these things can be worked out, but it is a consideration needed.

My thought is that method 2 can be done without actually having sections create SNT but rather have their “pool” DBC which they keep taking small amounts. Obviously the “pool” DBC is not the same since parent pool DBC —> New “pool” DBC + extra payment DBCs for any normal reward payment.

[krnelson]

Thanks David that helps a lot in understanding the problem/insight into the limitation of the consensus algorithm. IIUR the Safe Network consensus algorithm forking resolution method is acceptable for storing data securely, but not suitable for guaranteeing unique data, like tokens.

Does this the same limitation prevent the following alternative decentralised method from being implemented where DBC is not actually created by sections?

Method2 (B):

• 100% of SNT are Pre-Farmed
• 30% are airdropped or equivalent to initial stakeholders
• 70% Are managed by the Safe Network consensus algorithm, “sharding”

Sharding method outlined under “DBC Mints” in this Update “Sharding: Each Section is responsible for a subset of DBCs.”.

Or perhaps is it related to complexities @neo posted above, or similar?

[oetyng]

@krnelson

Does this the same limitation prevent the following alternative decentralised method from being implemented where DBC is not actually created by sections?

No the limitation doesn’t prevent it, but the alternative you mention falls under first point:

@oetyng

• We don’t want large sums of tokens being controlled by sections.
• We don’t want sections to be able to create tokens.

And that’s mostly because, for a very long time before the network has grown large enough, it’s just too much to be controlled by too few.

[krnelson]

@oetyng

No the limitation doesn’t prevent it

Thanks for the clarification @oetyng.

Just to be absolutely clear as per David´s post above specifying why, this is not technically correct:

@oetyng

• We don’t want sections to be able to create tokens.

The correct word is “Can´t”, in that the Safe Network consensus algorithm can´t do this - and it is not a case of not wanting. Do you agree?

@oetyng

for a very long time before the network has grown large enough, it’s just too much to be controlled by too few.

When you say “too few” will control it I take it you mean too few sections, not individuals.

Under Method1 a few (very brave) people will be centrally responsible for 100% of SNT supply existing outside of the decentralised Safe Network, at least until it can be mostly distributed. This distribution process may take many, many years so the attack surface may be long in time. I understand the loose plan is to have “many” distribution scripts running on servers outside of the Safe Network consensus algorithm but connected to it in order to do the job of distribution. Each one of these scripts under the responsibility of the few brave and trustworthy people selected to do the job. Lets call this group “Script Network Maintainers”.

@mav has documented how astonishingly fast data networks related to Safe have grown over on this thread.

On day 1 the Script Network Maintainers instead of worrying about their distribution scripts and related traditional server security could instead be running/responsible for many standard Safe Network nodes. Given their initial advantage they would be guaranteed to have a high proportion of elder nodes under their control. This number could easily be made to match or exceed the “many” distribution scripts/maintainers considered secure for Method1. Then the community is allowed in. If the Safe network grows at even a fraction of the pace of similar data networks, the amount of sections would grow to dwarf any initial number of distribution scripts/maintainers chosen for Method1.

Considering this, please help us understand how centralised Method1 is considered more secure than Method2(B) ? I am assuming there are other technical and perhaps political factors involved that we are not aware of yet.

[oetyng]

@krnelson

The correct word is “Can´t”, in that the Safe Network consensus algorithm can´t do this - and it is not a case of not wanting. Do you agree?

No, I don’t agree, and it’s quite simple why: that formulation is from another context, and wasn’t applied in this one.

You were asking why the consensus used for data is OK there, but not for tokens. David explained why.

The citation though follows up with an explanation which shows that it is another context:

@oetyng

The problem in my view is this:

• We don’t want large sums of tokens being controlled by sections.
• We don’t want sections to be able to create tokens.

Why?

Because we don’t know of a secure way to do any of those now - nor do we know when or even if we will learn a way.

That is to be read “we don’t want to commit to that design, as we would then be required to solve a problem we do not yet know how to solve”.

---

@krnelson

When you say “too few” will control it I take it you mean too few sections, not individuals.

Too few individuals is what is meant (and is the problem it cooks down to either way).

---

@krnelson

Considering this, please help us understand how centralised Method1 is considered more secure than Method2(B) ?

I personally don’t think there is enough information to show what level of security Method1 would give. What I see with that method is uncountable number of known and unknown risk points during an extended time. It’s hard to asses which of them all would be there in the end, but to me it seems very hard to ensure that there would be anything less than a very large number of them.

All that to say that it’s a quite complex long running arrangement all in all.

Method2 is not a method described by Maidsafe (not recently anyway), as what has been said is that an autonomous distribution is yet not known how to solve.
So the comparison you are trying to make is moot, akin to divide by zero. Once there is something to compare to (any suggested solution) then a comparison can be made.

Though even at that point, I think perhaps someone else than me would have to answer that, considering that I find the security of Method1 very hard to assess (and thus to compare with).

[happybeing]

The comparison is also made difficult because Method1 includes any thing from zero to all of the 70% being distributed immediately by airdrop, the remainder over time (by humans in ways to be defined). I think it’s easier to separate out the immediate airdrop option, even if both might operate in some combination.

These discussions are difficult because the details are not available but even if they were, comparing them will be hard for the reasons Edward gave (complexity).

I think the main question we can reasonably try to answer is whether, if automatic distribution is not available within a short time frame, how much it any of the 70% should be distributed immediately to holders and shareholders?

Immediate distribution is a much easier thing to understand and reason about with regard to security and centralisation. The alternatives offer high degree of uncertainty around both, but the possibility of a much wider distribution which is important.

That’s a hard question, too little good information on one side (whether automatic or Foundation), so I think people will be forced to go with their feelings and beliefs rather than reason, which is not a good thing IMO, especially if this decision is handed over to the community which some have argued for.

[krnelson]

@oetyng

Method2 is not a method described by Maidsafe (not recently anyway), as what has been said is that an autonomous distribution is yet not known how to solve.
So the comparison you are trying to make is moot, akin to divide by zero. Once there is something to compare to (any suggested solution) then a comparison can be made.

Update 16 September, 2021 covers Method2(B) and it was briefly mentioned a few times since then IIRC, but of course we are on fast moving ground. My questions so far are like stabs in the dark to try and get an idea what the problem is and what and where the limitations are. All we could tell previously is that it must be a pretty decent problem to drive centralised Method1 solution to be seriously considered.

Noone outside of the core devs will be able to even begin to think about the problem let alone possible solutions until the problem is documented/defined more publicly. With that in mind I will try to sum up what we have learned so far (very open to corrections and elaborations!):

Method2 and Method2(B) are possible options and there is no limitation of the Safe Network Consensus algorithm preventing them. The problem lies elsewhere…

@oetyng

• We don’t want large sums of tokens being controlled by sections.
• We don’t want sections to be able to create tokens.

Why?

Because we don’t know of a secure way to do any of those now - nor do we know when or even if we will learn a way.

@oetyng

That is to be read “we don’t want to commit to that design, as we would then be required to solve a problem we do not yet know how to solve”.

Going on what your saying so far the problem appears to have two parts:

A)

@oetyng

And that’s mostly because, for a very long time before the network has grown large enough, it’s just too much to be controlled by too few.

B)

@oetyng

what has been said is that an autonomous distribution is yet not known how to solve.

On Problem part A)

I refer to paragraph from my previous post “From Day1”. It would be possible to quantify how many trusted individuals (of the “Script Network Maintainers” group) would be required, operating X amount of Safe Network nodes as elders at network launch, to be more or less equivalent to the security provided by Method1 “Script Network Maintainers” group given initial median growth trajectory of similar data networks. Ignoring Method1 external risk factors of running a centralised server network to do the job of course.
[Edit: Main point being that for any selected number of trusted Script Network Maintainers and the servers they administer for Method1, there likely exists Y trusted Safe Node operators that would do the job to an equally probable level of security (and likely more given its decentralised).]

@oetyng

I personally don’t think there is enough information to show what level of security Method1 would give. What I see with that method is uncountable number of known and unknown risk points during an extended time. It’s hard to asses which of them all would be there in the end, but to me it seems very hard to ensure that there would be anything less than a very large number of them.

There is a good argument to be made that “it’s just too much to be controlled by too few” for Method1 as well. Method1 will have all the known security and related weaknesses that traditional servers have. Recent experience tells us it will be more so given that some absolutely spectacular hacks have been levied against companies operating in this industry, especially wherever they even slightly centralised risk (so called “distributed bridges” and whatnot). Hacks so audacious it leaves many speculating with compelling but inconclusive proof that only attackers with state level resources and very low level network access and reconnaissance could have pulled them off, repeatedly. By audacious think 1000+ different geographically distributed servers maintained by different “Script Network Maintainers” and all of them all being somehow identified and directly targeted. We have already seen worse.

If Method1 is the only way forward I would recommend a tried and battle tested blockchain based solutions to try to mitigate the risk through decentralisation.

On Problem part B) “autonomous distribution is yet not known how to solve”

This appears to be the heart of it.

Given this limited information and to try and feel out the scope of problem part B) to help understand better. Would there still be problem if there was no complicated distribution schedule with percentages going to different groups? Simply paying Node operators for resources provided (either through Method2 or Method2(B)). Or is this a missing the forest divide by zero question again?

[oetyng]

One thing, just noticed this:

@krnelson

Pre-Farmed

This formulation is misleading I’d say.
Naturally it brings to mind “pre-mine”.
However the term “pre-mine” is actually not very suitable for SAFE Network.

(Issued at genesis is more correct I think.)

So, to start with, “mining” for us, is simply staying online and providing resources - no other “mining” exists - and there will be no end to that “mining”. For that reason, “pre-mine” is not a term that applies to SAFE Network the way it does to e.g. block chains with limited caps, where the mining has an end.

Because the former doesn’t limit the potential for future earning, which the latter does.

Another one is that “pre-mine” usually refers to ownership of the tokens at the project members (founders, developers, other staff etc).
There, depending on if going to foundation, or to maid holders and shareholders, that is somewhat to not at all applicable.

[krnelson]

Noted. I will use “Issued at genesis” from now on, thanks.