MaidSafe Dev Update - November 29, 2016

@rand_om slaps @bochaco with a trout.

There you go my dear :innocent:

10 Likes

No trout, but anyway: Monty Python, The Fish Slapping Dance - YouTube :wink:

5 Likes

Great to read the update! Seeing all the pieces coming together slowly. Amazing job to the whole team. There more I read about governments all over the world imposing their agendas onto people the more obvious it is how important Maid and bitcoin really is. You guys are carving out a better future for generations to come. Taking back the power, one line of code at a time. Love being part of this. Keep it up, lets keep the fight going.

2 Likes

Truly amazing work as always guys, the revolution is near!

2 Likes

Guessing that if the SafeCoin CEP gets finished and the home vaults come out before the authenticator, then we may see a jump from Alpha 1 straight to Beta? Decentralized and SafeCoin seemed to be the requirements for BetaNet

But if authenticator comes out first then we may see the name “Alpha 2” be utilized

Then again, maybe Troon HQ has scrapped the alpha 1,2,3 etc plan from earlier due to things changing / evolving too fast. Never a bad thing!

Great work! Lots of excitement.
Mobile SAFE apps!

6 Likes

By “group” we just mean a bunch of nodes doing stuff together. Currently we always want groups to be fully connected, to avoid additional difficulties: So any fully connected set of nodes can do stuff as a group.

One of the things they’ll have to do (which is being implemented in the routing crate) so we can securely relay messages is: agree on who are their neighbours—specifically the neighbouring groups’ members—, and cryptographically sign them, so the recipient of a message has an uninterrupted chain of group signature lists proving the authenticity of the sender without having to trust any single node. For that, it is useful if the groups are disjoint and every node is just a member in one such group: it reduces the number of member lists I have to sign and clearly defines who else has to sign those lists. It gives me a clear picture for that: I am only in group A, and the neighbouring groups are only groups B, C and D, so all I have to do is sign the lists of members of B, C and D (and, crucially, everyone in A sees and signs the same picture!), and exchange those signatures with the other members of A.

A completely separate thing that groups do (which is implemented in safe_vault) is managing data: A bunch of nodes store the same piece of data, make sure that they update it the same way if e.g. Post requests come in, and give that data to any new members of the group. For that, it’s perfectly fine to just consider the 8 closest nodes to the data’s name the “group”. So while each node would be a member of exactly one routing-level group (i.e. those are disjoint from each other), it would be a member of potentially several data groups.

Fortunately, the new routing table implementation still ensures that the 8-closest-nodes-type groups are fully connected, so they can continue to be used, too. In fact, each such group is automatically a subgroup of the node’s routing-level group.

Another way to view these group types is this: in a perfectly balanced network with 8 * 2n nodes, e.g. 32 nodes, eight of whose names in binary start with 00, 01, 10 and 11 respectively, the two group notions are exactly the same! Every data group will coincide with one of the four groups belonging to one of the four prefixes 00, 01, 10 and 11, namely the prefix of the data chunk’s name.
If I add a node to that, I will still have exactly four routing-level groups. But one of them now has 9 nodes and several overlapping 8-closest-node type subgroups.

So the notions are only different in less balanced networks or while the network is growing from 8 * 2n to 8 * 2n + 1 nodes.

15 Likes

@AndreasF So a vaults close nodes, are comprised of 8 close nodes. Each of which come from a separate disjoint group? The biggest advantage to using the existing vault algos is avoiding a rewrite that strictly uses the group in YOUR routing table? Just trying to get this clear in my head :smile: are there any disadvantages? Would save som time and resources which is great!

1 Like

Another advantage is that each data chunk will always be spread across exactly 8 nodes (unless currently being replicated). There’s no reason to replicate one chunk 15 times and another one 8 times, just because their routing-level groups currently have that size.

No, they always come from the same routing-level group: Every vault-level group is completely contained in some routing-level group. (And if that routing-level group has exactly 8 nodes, the two kinds of groups are even identical.)

4 Likes

Maybe a stupid question: but that means that each of the 8 nodes has a real copy of that data chunk, or have some nodes only a hash of that chunck as check?

1 Like

They’d each have a data chunk, yes.
(In practice, they’ll get the hash first and then take a few minutes to download the actual chunks, of course.)

5 Likes

It’s really pleasing to see how quickly the newest Devs are able to contribute.

The Network seems to have evolved so much since the crowd sale and yes it’s taken longer than most of us expected…but when it’s born, it feels like it will be fully formed.

16 Likes

So we want Disjoint Groups to have 8 nodes as a minimum at least to satisfy Vaults??

5 Likes

Yes, that’s the current MIN_GROUP_SIZE value: If a group falls below that, it starts merging with its neighbour.

5 Likes

So every data chunk now has 8 total live copies around the network? It used to be 4 right? Is this to be safer against network blackouts / data loss etc?

5 Likes

4 will still be more than enough. At the moment it’s all group members, with data chains this will reduce once more for efficiency. So more for simplicity right now. Saved us maintaining lists of nodes holding data, we just make them all hold it for now.

13 Likes

Some time ago, @polpolrene made the following post, which has had a very great impact on my thinking about network and been a good tool for giving others the flavor of the security and privacy of individual actors’ part in the functioning of SAFE.

There has been considerable evolution in the codebase since then, and significant restructuring of node duties and relationships. Especially, with Disjoint Groups, Node Aging and DataChains, a lot more stability has entered as far as longer-term relationships between nodes goes, as far as I understand things, anyway.

My question is: While all this forward progress has been made on the stability of the network structure and efficiency of data handling is obviously huge strides forward, how does it affect the obscurity of all actors over the long haul?

Not that peeling one or two of the seven or eight encryption and obfiscation layers off would be a deal-breaker, in order to improve network function, but I’m just left with that question, from my Simpleton’s perspective, of course. :grin:

7 Likes

More secure group messages
The current group messages do not provide a high level of protection against attacks on the network, even under the additional assumptions that routes are disjoint and that a quorum of the individual messages need to be intercepted. Assume that an attacker controls 10% of all nodes in the network, and group size is 8 and quorum size is 5. Consider a group message sent via 10 hops.

In that scenario, the chance to intercept any given group request is 71%. With the group-to-group hops proposed here, it will be just 0.43%. Further adjusting the group size and quorum size parameters, this can be reduced to a negligible number, making this kind of attack virtually impossible.

looks like more security on the routing -level. But I really need to update the encryption-layers topic sometime to really get my head around this stuff.

5 Likes

A lot of what is happening aids that obscurity actually. We are closing down holes that would have allowed fake messages etc. In terms of the security layers and IP address scrub etc. there is no change there. This work is mostly about security readiness and reliability. So think of these changes as more efficiently securing the network and reducing the opportunity for bad actors. We are not complete as you know but that is the path we are on.

18 Likes

I wouldn’t be surprised if the coming testnets with Vaults work pretty well when it comes to data retention. Especially when the network is small having 8 nodes storing a chunk shouldn’t be a problem.

8 Likes

Thanks David. I assumed so, but had to ask! P

5 Likes