Vitalik Buterin on MaidSafe's consensus mechanism

Features
1

During the Zapchain AMA that is going on right now the following question came up:

“Hey Vitalik, I heard you say many times that the problem with blockchains
and scalability is that every node has to process every transaction.
MaidSafe has a consensus mechanism based on close groups around nodes,
which allows for scaling. Do you foresee any way something similar could
work for blockchains?”

his answer:

"Maidsafe’s approach is very similar fundamentally to the random-sample
paradigm I describe in my document:

However, I have not yet seen from Maidsafe a good answer for what happens when
one of these “close groups” fails (eg. 5 nodes disappear, 28 of 32 nodes
get bribed, etc). I would consider a system that doesn’t answer that
problem hopelessly fragile, and likely in practice to eventually break
and not be able to recover. Now if Maidsafe does come up with a good
answer to the problem (or if they already have), I will be quite glad to
change my opinion; though I suspect they’ll end up reinventing my
fallback schemes and challenge lottery mechanisms."

the link to the converation: https://www.zapchain.com/a/FRsI5InA2e

maybe he needs to be pointed to the systemdocs… :wink:

11 Likes
2

Yes this is one of the issues we will have when we are so immersed in getting the codebase and launch in a way that is smooth and sleek. We are doing some pretty amazing things in the background right now (visible on github though) which will transform the speed of development, ease of engagement and reduce dramatically the barrier to entry. It means we cannot also be in protracted discussions, regardless of value.

some folk are in a fortunate position it seems to be able to spend inordinate amounts of time in blog’s, reports and investigation. We went through that years back (without the benefit of such a large audience). Every time I for instance engage to explain I lose at least a day and usually closer to a week in mailing list floods.

The best thing is to get the dev work behind us and then show via experiments and measurements how it all works.

The thing most folks do not grasp is that true decentralisation as in an ant colony works like this (in terms of what if 1 ant/group get’s XXX).

  1. A single ant is almost nothing
  2. A single ant can be easily washed away in a raindrop
  3. The kicker is that does not affect the colony
  4. The colony strength is repeatability and following simple rules.

This of it like this, an ant carrying food back to nest, or cleaning nest get’s killed. Does it affect the colony, well yes if you throw a ton of math you can even calculate how much. That does not however kill the colony. Why? This is natures way, wastage etc.

So we build decentralised systems and do not see this essential component. OR so it seems in many cases.

In short, allow your ant to get killed, but (and the big but) do not allow a single ant being killed, compromised or otherwise affect the colony. So in tech talk, if a group get’s killed and your design does not only allow that but use it as a natural step then your design is gonna die.

I very much disagree with math to the n’th degree and the amount of deep deep math investigation with really more complex equations. Nature I am afraid does not work that way but works when you align with it and allow the inevitable to happen and make sure you expect and work with it.

Now this 100% does not mean it’s not great fun (and essential) to investigate what is happening and try and formalise it, we do that. It just means the obvious should be considered and taken care of. Look at our case, say you could kill a group, the network fills that space in immediately!

Say you could control a group, well that is inordinately difficult (say you could control 51% hash power) but theoretically possible (like a beeetle shorting the switch for the nuclear launch :slight_smile: ), buy hey so lets look at it anyway, there is a realm and universe where it will be possible (and we may miss something, i.e. zero day exploit etc.).

So that happens, what is the consequence of controlling a group, you

  1. won’t be able to decrypt data (network nodes cannot)
  2. Cannot create safecoin (needs multiple groups deterministically chained).
  3. Cannot spend safecoin (groups do not have keys capable of signing)
  4. Cannot block access to data (needs again several chained groups)
  5. Cannot change data (it’s mathematically checked via crypto hash)
  6. Cannot alter structured data (groups do not have keys capable of signing)
    …

Anyway lots more, so what can you do. Well vandalism really, potentially cause minor blockage or deny access (perhaps) to some network traffic (which is loosely parallel you cannot get them all). I am sure we all can come up with a ton more, honestly though I am very driven to get released right now (so don’t take this as me ignoring or unable to engage in hugely long deep discussions, we have many many times :wink: ) Also there is the system docs

So seems cool, is it robust, I think so, is it tested, only in lab so not in wild. Is it possible we have something wrong, yes it is and this is why we need code as readable tested and clear as possible.

So a goal is not getting too steeped in any thought experiments (too much, although they are important) and build release and test in the wild. This is what needs happen and the rest can be calculated for as long as we wish. We have folks looking deeper into what we do and it’s far from simple, I am sure we will have more, but the truth is there is an inordinate amount of thought already in SAFE and to try and explain it is almost impossible, so I am driven to show it instead :smiley:

I can almost guarantee though that pretty much any question will have an answer on this forum, our docs, reddit etc. as I know lots of folk will read this and then say “ah what about XXX when YYY does ZZZ etc.” it’s natural to ask, but I really need to concentrate on the dev processes, which are almost completed now, but at wicked pace.

46 Likes
3

Awesome post!

It is a shame that respected community members talk with some confidence about something they have limited knowledge of. I know Vitalik suggested that maidsafe may have solved the problems, but it still spread some inadvertent FUD on the project.

I can’t wait to see safe net go live. I think it will surprise a lot of people.

9 Likes
4

@dirvine
Most people say they want proof for any claim. I agree with you, we should spend more time “showing” rather than explaining. Here’s an example…

The car seller says, “it goes fast!”
The potential buyer asks, “how fast?”
The seller says, “get in the driver’s seat and find out.”

The test-drive answers that and many other questions. And if you record it on video, then upload it for everyone else to see, you would have saved yourself a ton of common FAQs.

Unofficially, this is how I would answer the question. I’m sure you guys have already thought of it.

Question
If (28 out of 32) are needed for consensus, what happens when 5 nodes disappear from a group, leaving only 27?

Answer 1
The SAFE Network is in a constant state of flux, and the moment (less than a second) 1 or more nodes disappears (goes offline), others will take its place. The node replacement is determined by the XOR closeness. I believe this is called self healing. I suspect this requirement (28/32) has to be met before any other authorizations are allowed.

Answer 2 “Theory”
If the SAFE Network suddenly lost 5 nodes in the middle of a (28/32) consensus process and for some unexpected reason, replacements cannot be found quickly. There could be a dynamic rule: 7/8 ratio needed for consensus. So the group size would be as follows: 7/8, 14/16, 21/24, 28/32, 35/40… and so on.

The smallest group initially formed would be (7 out of 8), and naturally grows as big as it can.

If 5 nodes were lost, leaving a group of 27, and none could be replaced in X time… then the dynamic consensus adapts to 21/24. Otherwise, it will keep trying to grow bigger to increase security.

It’s just a theory, for the devs. I have no idea how viable it will be. Answer #1 should already be sufficient.

3 Likes
5

Let’s not, a DOS attack on honest nodes could then reduce the amount of consensus required.

2 Likes
6

You do realize 7/8 is the ratio already in place right? The only difference is the total number of nodes.

7

Isn’t that the reason why your vault is actually like 16 vaults in reality? That means 16 groups. So if 1 group would go down in half (let’s assume a big internet error in Europe) only one of your groups goes down. I have no idea how fast new nodes would join. Nodes need to be checked first etc.

8

Yes, which is a vulnerability. It means that if you have 7 compromised nodes in a full group of 32, you can DOS 24 honest nodes to bring the group down to 8 total, in which you have a 7 node majority.

But this is a non-problem anyway, since if you can connect to the network at all you should always be able to connect with replacements that are slightly further away in closeness to replace any nodes that went offline for whatever reason. Until those connections are made any decision’s completion is simply postponed.

9

As I said, this is just “theory” designed to increase security when able, but flexible to adapt to catastrophic changes if necessary. Your question is relevant as to how long it takes nodes to join. I wish I knew. That could be used to determine if/when the consensus group is reduce to enable a transaction to go through.

If we look into the future, we shouldn’t be surprised how bigger farms will likely create hundreds even thousands of nodes. As the network grows, it makes sense to scale the security group size with it.

@Seneca
I agree, you just pointed out what Vitalik was alluding to. What happens if those honest nodes are pushed away, via DOS or other attack. And the answer should be #1 which states a minimum 28/32 needed before any other authorizations can be done.

I don’t want to bother the devs with a debate. I was hoping to push this as a future thought.

10

I think it is really really hard to push nodes away or to DDoS them. On an ip-level you’re only connected to maybe 2 addresses per group. Every message you sent out using this 2 ip-connections, will be forwarded by them to the other 30 nodes. So if someone tries to DDoS your ip (let’s assume they have it) they can only take out 1 node in the group. The other 31 will be alive. And your attacker has no clue what their ip-addresses are. So I have no idea how to attack a group. I think it almost can’t be done. Now let’s assume I’m the bad guy, so I start to attack these 2 ip-addresses that I’m connected to. The other nodes will see some weird behavior of 2 nodes in the group, so their ranking goes down. Now let’s assume I really go nuclear on these 2 ip-addresses, what happens? I kick down 2 nodes in the group. the other 30 will still be alive and well. And I have no clue what their ip-addresses are… So to kick down a group, take like 12 nodes out at once? I don’t think it’s possible.

3 Likes
11

It’s like smashing the ocean with a rock; eventually the ocean consumes it; “punching holes in the ocean” @dirvine

5 Likes
12

It’s possible that Ethereum could be over engineered. They have to be careful to let the best design emerge rather than to try to predict all possible future scenarios in a top down grand design approach.

It’s not possible to design something like this by trying to forecast every possible exploit. Instead it’s better to design it with emergent properties so that there is an in-built immune system for instance or self repairing mechanism.

So I would say David has the right approach here. As long as SAFE Network is “evolve-able and resilient” by design, then it will be attacked and develop immunity to those kinds of attacks as they happen.

9 Likes
13

A design which solves all possible problems from the start is impossible. There is no design like that.

However you can have a design which is built to evolve and be flexible enough to be extremely resistant to attack.

Ant colonies are a very good model to look at when you’re engineering an emergent architecture. It’s not a good idea to look at any top down over engineered design from the minds of a mathematician or engineer because these designs often try to look at all possible risks and manage them but it’s not possible to control for all risks at once.

If you look at how biology works then a species doesn’t start out immune to all viruses. It develops immunity over time, it develops resistance. The key feature is adaptability of the species, the hive, the colony, the chain, the code, etc.

As long as team SAFE Network focuses on having the ability to easily evolve and adapt then it can resist attack.

9 Likes
14

Is this resolved ? And 20 characters

15

Generalised answer: Basically Yes.

Read up on disjoint groups.

6 Likes
16

I think you really just avoided the questions and felt like you either don’t like the questions or don’t have answers. Vitalik is a well-respected crypto leader and Chief Scientist of Ethereum( the most successful crypto project to date) and its founder, he spent time learning about maidsafe and definitely know what he was talking about, it would be more convincing to take his question head-on, shouldn’t we encourage some healthy conversation, instead of sitting back in the closet and completely shut oneself off the larger crypto community.

17

Who has sat back or shut themselves off? He’s trying to build the basic network, when it exists in the wild it will be more interesting to the wider community.

You need to go read about disjoint groups as neo suggested above to warren.

How was he anything other than respectful and engaging?

Disjoint groups did turn out to be a lot of hard work and took many months.

/raised eyebrow

3 Likes
18

I don’t like the way he just tried to dodge the bullet, why not take the question head-one like a man instead of taking half an hour defending the dodging? If there was already disjoint group discussion, share the link or content, it is also helpful to bring about more discussion and shouldn’t we encourage more interaction with the larger community, instead of accusing " you guys know nothing about the issue, we already discussed". When an ex-employee questioned this project last year, all questions raised were also ignored and dodged. Maidsafe looks like an isolated island in the vast crypto ocean, I don’t think you could attract developer mindshare if always turn yourself inward and so disengaged.

19

This discussion with vitalik was well before there was a disjoint group solution. Kinda hard to refer him to that. Maidsafe got past it though, isn’t that the thing to focus on here? The mighty vitalik made some fair points, David et al did some amazing engineering and showed what was possible.

The ex employee you refer to was not ignored?! He made several long posts with different points, many of which were simply irrelevant because he did not understand how the network had changed from when he worked for maidsafe. His posts were picked apart and his technical arguments found wanting. Did you read the whole thread yourself? I suspect you haven’t or else you would not be bringing it up as a negative for maidsafe; it only reflected poorly on him. He was handled in a polite way considering the content if you ask me.

It just hasn’t launched yet. It will be a bit isolated until at least the MD APIs are sorted and app devs can really do stuff.

Eth was just an ignored $65M market cap shitcoin 2 years ago. Things change quickly, but they won’t change until after the network is further along. You can’t will it to happen sooner, we all just have to wait for nature to take its course here. IF it works it wins, if it fails it fails. There isn’t a lot of middle ground.

8 Likes
20

No the questions from Ben were not ignored they were challenged head on and it was Ben who was dodgy when anyone came up with answers. From any side there is bias. I’ve followed this project for about 3 years or so and try to be as technical and realistic as possible, though I’m a natural optimist I am also realistic. That said, Ben was then working at some new Blockchain startup so who’s to say he didn’t have some kind of reasoning to attack the legitimacy of this project and garner some attention for himself and that company?? Who knows for sure? All I do know is if this project spooks you then maybe don’t be involved in it.

6 Likes