SAFE Network concerns from an old employee

[BIGbtc]

@Nigel

I would love to hear a solution.

You wont hear a solution from this guy, he has no talent in the solution department. His skills lay in the muddywaters department. The solution is coming from Maidsafe and this dude should be ignored.

Lets get this project done, way too much energy being spent trying to defend against guys like him. There will be more, and after that more, and more after that. The complexities of this project make it way too easy to refute any defensive from Maidsafe.

[Philip_Murray]

I welcome his critism as it confirms the project is currently on the right path, i think he just done it in the wrong place, he is obviously very knowledgeable but the code has changed alot since he was working for maidsafe.

[iamamystery]

I would like to see less passive-aggressive responses in this topic.

I’m finding the discussion quite informative. Essentially vtnerd if I understand you correctly you’re simply wondering how the network handles the classic “split-brain” problem.

[Pierce]

My comments were not passive. I made no attempt to dampen my statement. This man is a professional and should by now know how to engage in a straight forward productive technical correspondence. His approach is the reason we’ve gotten to over 120 post with no clear resolution.

Here we are so close to launch and instead of things flowing, these people keep throwing in dams to hamper the fluidity of development. This discussion takes up significant resources. It annoys me to no end when these guesstimators enter the room, throw around inflammatory claims of infeasibility then expect concrete answers when they don’t extend themselves enough to do a little research. It’s arrogant and disrespectful.

If you’re curious ask questions. If you’re properly educated on the matter point out flaws and or make suggestions. This man stands between the two extremes with little apparent desire to move towards the latter. He’s either lazy or foolish. Again not passive. I make no apologies for being aggressive.

[neo]

@vtnerd

The network topology will differ from each client. If two clients write to the same group at the same time, each machine in the group can receive the writes in different order. This cannot be resolved on the client side because both requests were issued at the same time. The remote side has to agree or “come to a consensus” on what happened first. I cannot recall an implementation to handle this situation, or a discussion of an algorithm that would be used to do it. Solving this problem adds lots of complexity. Asking people to never make two writes from different clients seems inappropriate given the security implications outlined above.

The impression I get is that you assume the writes will have to “work”.

Actually if you write to different addresses then there should be no issue.

If you have multiple machines writing to the same address and the group members get the requests out of order (normal) then none (or one) of the requests will succeed and the failed requesters will be sent an error message. If the group gets enough members (eg 28 if group size is 32) to agree on one request then that request will succeed and the others will be returned a error because 4 out of 32 cannot constitute agreement.

So then it is upto the requesters to perform normal error handling.

[digipl]

Hope this commit, coded and finished, ends with some concerns.

https://github.com/maidsafe/safe_vault/pull/545

But, as Andreas Flacker say, this is a rare case and, in distributed computer, always can find more and more rare cases. In the end, the Godzilla Attack become the most likely.

About the possibility of a division of the network I ask myself two questions:

In the nearly seven years of the bitcoin, ever its network was divided?

if not, as I think, what real possibilities are, almost in 2017 with better communications, this division happens in a much more robust network as SAFE?

[Traktion]

@digipl

In the nearly seven years of the bitcoin, ever its network was divided?

Yes, several times. This includes one time where a bug resulted in a ‘bad’ chain becoming dominant before being abandoned by miners.

IIRC, splits actually happen periodically, where the heads of the block chain disagree for a period of time. I am not an expert in this, but it is wrong to think it never occurs.

[digipl]

Is different a fork of a chain that a network division. In a fork the miners choose a different leader to write the block and we have two, o more, possible lasts blocks. In a divided network we have two blockchains, two different group of miners, two different group of nodes and each group is unable to communicate with the other.

[reivanen]

Please don’t do unfounded hype. It is as bad as unfounded criticism. Launch is not “so close” by any definition of language. Beta launch can be seen as “so close” if everything goes well.

[Traktion]

I am not sure there is a material difference in resolution (choose longest chain) either way, but point taken.

However, unless a fork is due to a bug or a malicious attack, transaction propagation errors (e.g. Net split) must surely be the cause. If a transaction reaches half of the network, but not the whole network, then the blockchain heads are going to be different.

Regardless, I agree that it isn’t a critical problem for Bitcoin so far. Moreover, choosing one full chain over another, rather than at the finer grained by group basis, safenet should actually be better equipped to handle this.

[vtnerd]

edit: forgot to answer a question, whoops!

@Traktion

I don’t mean merge the forked groups - I mean in each case of conflict, the best group can be chosen as the source of truth.

To put it another way, we don’t need to take everything from one side of the fork or the other (like the longest chain on a blockchain). Instead, the best group can be chosen in each case, no matter which side of the fork they resided. Therefore, conflicts are likely to be localised to specific data items, as held by groups which have applied conflicting changes.

Unless you can fake a group or force entry of many nodes in to a group (a bigger problem regardless of splits), I don’t see how an attacker can influence the decision over which group is chosen to be the source of truth?

You can currently fake a group running a network with the current implementation, which I mentioned on Metzdowd. David seemed to imply that I misunderstood this, and so I asked where in the code the messages were being authenticated as being from the group. Currently the accumulator is just counting messages. Based on the status updates, I believed this to be an area of development.

Whether the attacker can influence the decision depends on the implementation. If the implementation is simply the group with total closest XOR distance, an attacker could pre-compute keys to be closer. Preventing that would mean ensuring that everyone joined the network with a value provided by the network in a way where no collusion took place. I do not believe this problem to be adequately solved in theory yet (this was also mentioned on Metzdowd). It can be a little difficult to know for certain without seeing an implementation or a very clear description of the algorithms taking place.

@Seneca

… is/should not usually be accepted. Being able to stroll through the DHT like that seems a security risk to me. Other nodes should only share IP addresses of nodes that are close to you, which they can check. This should only be necessary due to churn or node relocation on joining.

This is an interesting thought, but it would alter the logarithmic lookup for the DHT. Digging into TOR hidden services some more could prove useful too. IMO, protecting the IP of some resource in a network intended to deliver quickest delivery is ultimately just going to be scary.

@Traktion

Perhaps there would be a possibility of merging 2 data chains (i.e. transactions made by forked groups), where there were no conflicts too. Data chains should definitely give us a sequence of transactions on both sides of the fork, many of which may not conflict.

How can you “merge” ownership of a single coin owned by two people?

@Traktion

Either way, I would like to reiterate that blockchains have this same problem, but a side of the fork has to be picked (the longest blockchain).

Perhaps an interesting question is whether CPU hashing a fake longer chain is easier than faking/overwhelming a safe net group to ‘simulate’ a split, to change history. My understanding is that this cannot be done without enormous resources on safe net (you need many more nodes than the entire network), but it would be interesting to dwell on.

Yes, blockchains do have this issue. There is no reason to suggest that SAFE requires computing resources to fake a network, at least not IMO.

@Nigel

I think @vtnerd is spending enough time here to warrant a proposal on how to fix the issues he so clearly sees. He seems to understand this at a very deep level and is willing to defend his position that something is broken but what are his thoughts on a solution?? @vtnerd tell me with all of your brilliance, how you would fix the issue you have addressed?

I do not think there is an easy fix. I have thought about this.

@Pierce

If this guy asks for code one more time I’m going to hang myself. He was directed to the code several times already. What does he expect during such a frantic development cycle? For someone to hold his hand and take him to the exact code segment for each of his concerns?

I have read a decent portion of the code. I asked for code when I knew it was highly unlikely to exist.

@neo

If you have multiple machines writing to the same address and the group members get the requests out of order (normal) then none (or one) of the requests will succeed and the failed requesters will be sent an error message. If the group gets enough members (eg 28 if group size is 32) to agree on one request then that request will succeed and the others will be returned a error because 4 out of 32 cannot constitute agreement.

I think the most recent commit clears up when I was talking about.

@digipl

Hope this commit, coded and finished, ends with some concerns.

  github.com/maidsafe/safe_vault

fix/data_manager: fix Put and Post handling in data manager

by afck
on 12:17PM - 05 Sep 16

3 commits
changed 21 files
with 810 additions
and 515 deletions.

But, as Andreas Flacker say, this is a rare case and, in distributed computer, always can find more and more rare cases. In the end, the Godzilla Attack become the most likely.

About the possibility of a division of the network I ask myself two questions:

In the nearly seven years of the bitcoin, ever its network was divided?

if not, as I think, what real possibilities are, almost in 2017 with better communications, this division happens in a much more robust network as SAFE?

This has a different data race, but still one nonetheless.

@Traktion

Yes, several times. This includes one time where a bug resulted in a ‘bad’ chain becoming dominant before being abandoned by miners.

IIRC, splits actually happen periodically, where the heads of the block chain disagree for a period of time. I am not an expert in this, but it is wrong to think it never occurs.

@digipl

Is different a fork of a chain that a network division. In a fork the miners choose a different leader to write the block and we have two, o more, possible lasts blocks. In a divided network we have two blockchains, two different group of miners, two different group of nodes and each group is unable to communicate with the other.

@Traktion

I am not sure there is a material difference in resolution (choose longest chain) either way, but point taken.

However, unless a fork is due to a bug or a malicious attack, transaction propagation errors (e.g. Net split) must surely be the cause. If a transaction reaches half of the network, but not the whole network, then the blockchain heads are going to be different.

Regardless, I agree that it isn’t a critical problem for Bitcoin so far. Moreover, choosing one full chain over another, rather than at the finer grained by group basis, safenet should actually be better equipped to handle this.

You both are correct. When a new block is “mined”, it has to announce it to the entire network. Another miner could find a block roughly at the same time. This is a classic data race - the block announcements are not sent through an atomic broadcast. Each node will “see” a block first based on the network latencies between the two miners. The ~10 minute interval increases the chances that every node on the network sees the same next block. I think the prior cases were of this sort, but I would have to re-read the specifics.

The network could also be partitioned to create multiple valid versions. I cannot recall this happening, it is rare. If it happened, then the some data center probably went offline for some time period, but the likelihood of someone find the next block in such a window was unlikely. And if it did, only a small number of people would know about it. The adversarial case gets a little hairy (attacker “owns” an ISP router, partitioning it, etc.). I am not sure if anyone tried this before, but they would have to mine another block. And the drop in hashrate would be noticeable (tripwire).

[Traktion]

@vtnerd

You can currently fake a group running a network with the current implementation, which I mentioned on Metzdowd. David seemed to imply that I misunderstood this, and so I asked where in the code the messages were being authenticated as being from the group. Currently the accumulator is just counting messages. Based on the status updates, I believed this to be an area of development.

Whether the attacker can influence the decision depends on the implementation. If the implementation is simply the group with total closest XOR distance, an attacker could pre-compute keys to be closer. Preventing that would mean ensuring that everyone joined the network with a value provided by the network in a way where no collusion took place. I do not believe this problem to be adequately solved in theory yet (this was also mentioned on Metzdowd). It can be a little difficult to know for certain without seeing an implementation or a very clear description of the algorithms taking place.

Ok, this is interesting. I would be interested to hear what @dirvine has to say about this.

If it is trivial to create a group, why would there be so much emphasis on preventing malicious nodes from joining a group? I understand that when you attempt to join he network, there is a neotiation that occurs, which places you into a different group (that you cannot choose). This is to prevent attackers from targeting groups which are know to contain specific data.

Naturally, I assume this effort would be relatively worthless if an attacker can just create a group and do with it as they please. In fact I would assume that new groups could only be created by splitting other groups which are considered too large, to form two separate groups. The same process of registering a node on the network could then be reused.

@vtnerd

How can you “merge” ownership of a single coin owned by two people

You can’t - you have to choose one owner or the other. Whatever is used to measure whichever the ‘best’ group is would influence this decision.

For blockchains, the longest chain wins. With safenet, i am not sure what metric is currently used. Maybe the longest data chain would provide a similar decision making mechanism?

@vtnerd

Yes, blockchains do have this issue. There is no reason to suggest that SAFE requires computing resources to fake a network, at least not IMO

Well, this is rather key and relates to my first answer. My understanding is that you cannot fake a group easily, but I admit that I do not understand the specifics of how. I am assuming that is a gap in my knowledge though, considering the efforts made to prevent unauthorised entry into existing groups.

@dirvine, is there something we can see to help the understanding of he new groups are formed and how the network prevents fake groups from being created?

[vtnerd]

@vtnerd

This has a different data race, but still one nonetheless.

Just wanted to clarify, that this patch gets a little crazy with nodes popping in out of a group. The expected vote count can vary from machine to machine. Placing machines under load (DDoS) can affect some of this behavior. I am going to guess that people are considering this attack out-of-scope, etc. Which is fine. The behavior in all cases will take some time to work through, which is the concern.

@Traktion

Well, this is rather key and relates to my first answer. My understanding is that you cannot fake a group easily, but I admit that I do not understand the specifics of how. I am assuming that is a gap in my knowledge though, considering the efforts made to prevent unauthorised entry into existing groups.

@dirvine, is there something we can see to help the understanding of he new groups are formed and how the network prevents fake groups from being created?

The problem is verifying that its the correct group. Prior suggestions were to ensure that the network chose where a node was placed on the network. That would work, except for the case where someone joins and then forges valid messages from a group that isn’t on the network. I have not seen a way for a node to rule out that possibility when receiving messages. Its just really, strange.

[dirvine]

@Traktion

@dirvine, is there something we can see to help the understanding of he new groups are formed and how the network prevents fake groups from being created?

To prevent a fake group attach means not even create a group, fake one on network hops and claim it’s a group. Disjoint groups RFC is your best bet. There are a lot of things to prevent though, fake groups is a huge one and covers a huge area really. In any case we do cover it, but a lot of info.

To disallow the off line address attack then secure name shows the process as now (it is also documented and set in diagrams in routing repository docs)

Of course data chains is a really big help as well. There is a repo and blog posts.

The sentinel part of the language of the network blog post also helps.

The docs are pretty good
http://docs.maidsafe.net/routing/master/routing/index.html Thre are flow diagrams an dsequence diargams for joiing etc.

Also for routing table which is in the routing repo now 0.23 branch.

Also some RFC’s covering beyond joining faking etc.

maidsafe/rfcs/blob/master/text/0019-new-kademlia-routing-logic/0019-new-kademlia-routing-logic.md

maidsafe/rfcs/blob/master/text/0037-disjoint-groups/0037-disjoint-groups.md

maidsafe/rfcs/blob/master/text/0030-secure-node-join/0030-nodes_key_as_name.md

The part we are working on is targeting (i.e. restart continually) words are chosen carefully in these above posts though I think for other purposes. We are saying in all the tests, we have not yet secured the groups and turned down parts etc. as we are testing different things so “per current implementation” can be really misleading. So attack is on design then test code etc. This is all messy and misleading. Outlining how this all works then critiquing it would be helpful instead of backwards,

Now I am again not working but answering huge questions a difficult balance. Perhaps this is the goal of such statements in the OP? I take some comfort in the rest of the team are not stopped working like this or we would be going backwards. Not a dig at you @Traktion questions are great, but we do need to get better ways of getting through them all or some way to index all the papers docs etc. in an easy to find knowledge base.

There is also a paper on vault security in the WWWF and I think on the IEEE as well (not 100% sure its on security though). I am sure a lot more but the search for this perfect answering machine seems to continue

[dirvine]

@vtnerd

That would work, except for the case where someone joins and then forges valid messages from a group that isn’t on the network.

Can you explain how this can work with disjoint groups?

[dirvine]

@vtnerd

You can currently fake a group running a network with the current implementation, which I mentioned on Metzdowd. David seemed to imply that I misunderstood this,

I have asked specifically for you to describe the process here and show us the security issues you state we have. It would be helpful as now it seems to be all about partitions and bitcoin. There is too much movement here to even attempt to get a grasp of the issues you are stating exist in the process I have asked about.

SAFE Network concerns from an old employee - #123 by dirvine

[polpolrene]

@vtnerd

You can currently fake a group running a network with the current implementation, which I mentioned on Metzdowd. David seemed to imply that I misunderstood this, and so I asked where in the code the messages were being authenticated as being from the group.

Have a look at this one. Even if you could fake a group, your group can’t do that much harm. The node you are misleading can’t really do a thing on the network as it’s not the real network. And you can’t get it’s private keys or log in details or coins. What’s the purpose of faking someone in a fake network anyway? And explain to me how your fake group could fool Sentinel.

Sentinel for beginners

In this topic I will try to explain “Sentinel”. There is a nice document about it on the Maidsafe-Github but it get’s quite technical quite fast. At the top it’s stated that Sentinel is about “network consensus, authority and crypto usage”. David did a presentation on Sentinel which can be found here. Let’s start at the beginning. Your client connected you on ip-level to the network using CRUST. That’s the part where you can have connections based on UDP, TCP, etc. It creates holes in your fire…

[vtnerd]

@dirvine

I have asked specifically for you to describe the process here and show us the security issues you state we have. It would be helpful as now it seems to be all about partitions and bitcoin. There is too much movement here to even attempt to get a grasp of the issues you are stating exist in the process I have asked about.

SAFE Network concerns from an old employee - #123 by dirvine

Getting into a disjoint group is a proof-of-work problem, with an easy difficulty. You try hash keys until you get a matching prefix. People generating vanity address for bitcoin are doing something similar already. If you notice, they have some pretty long prefixes. I have personally found a 40-bit public-key prefix for the secpk1 curve on a newish-i3. Took less than 36 hours I think, nothing too crazy. Adding a hash to the step will slow down the guessrate some, but not significantly. You could introduce scrypt or something to slow it down more, but I would advise against that because its still an advantage to someone with more CPU power. You could also increase the prefix length of the disjoint groups, but then you risk having empty groups. Even 2^32 is a pretty large number.

In fact, this is something else strange I have noticed. When the network decides to split a group, there does exist some probability that the group is empty. Seems like that would have to be coordinated or figured out.

Anyway, after generating my prefixes I either join the group legitimately (assuming the network is not enforcing anything else), or I join the network and start sending fake messages as-if I am the disjoint group. More specifically, if I am the proxy-node for a client I send spoofed responses from the non-existent group and never forward the request message.

[dirvine]

@vtnerd

Getting into a disjoint group is a proof-of-work problem, with an easy difficulty.

Eh no it’s not, Can you show where this is the case?

[dirvine]

@vtnerd

In fact, this is something else strange I have noticed. When the network decides to split a group, there does exist some probability that the group is empty. Seems like that would have to be coordinated or figured out.

Maybe define this assertion as well, Why would an empty group split (or even exist)? If it’s empty it’s not a group. If you have seen a bug int he current commits to the 0.23 branch it would be good to know.

I will link this once more. It would be really helpful if you explain the process and point out the issues.

SAFE Network concerns from an old employee

@vtnerd Lee I said this And you replied with this, which looks very much unrelated ? However, In MaidSafe all messages and events are always agreed and designed for out of order delivery, this has always been the case. If you can answer the questions I did ask it will help an awful lot, I asked this to get specific and find any errors or flaws. It is an Engineering approach and should be specific and based on the design and implementation. If you can outline how group consensus works a…