Update 24 November, 2022

I mean seriously. I see no other way around it. You have to say “We have elections going on, get back in a minute!”

7 Likes

Thx 4 the update Maidsafe devs

Have a good weekend SAFERS :partying_face: :beer:

And keep hacking super ants

10 Likes

The thing is, we can slow some stuff but not all. i.e. things like NODE_LOST need fast agreement, especially if there is a large network change. i.e. China goes offline or something. That is where we bump into the real world, and our perfect math kills us.

It’s like I keep saying nature does not care about much of what we care about. Things keep happening, and we need to keep up or get off. In the case of network membership events, many of them cannot be slowed down.

9 Likes

Fast in what terms? Not clock on the wall time, I think, but it needs to be faster than something else on the network, right? Why can’t everything else then be slowed down? What can fast mean, other than “before this and this and this other operation”?.

Yeah, but nature also doesn’t care about double spends. There are plenty of animals “double spending” by faking the appearance of poisonous ones. :wink:

Really, why not? I want to understand. In the case of China going offline, why can’t the network just say “all data traffic is on pause until we get enough nodes again!” Then the network would just wait for minutes, hours, days, weeks… until that happens - or dies.

Or is it because we might have two networks, one inside China, another one outside, and DBC’s could be spent in both of them? It could be eventually resolved after the great firewall is off again, but goods would have changed owners already, and someone would suddenly lose their money due to network resolving for the other branch? :thinking:

5 Likes

This may help. Imagine we lost nodes at a rate of 1 per second. If we lost, say 30 nodes in a section and consensus took 30 seconds to agree on which nodes are off line in which order (so one Decision every 30 secs) then we have lost all data and the consensus group of elders. So no more network.

i.e. the real world is nodes going offline, by the time we linearise that we have lost so many nodes. Now after we lost one, we should be replacing it.

So we have the network collapsing under our feet and we are saying, it’s OK we have the perfect math for this. In fact we will just let all that happen while we decide who died first. (keep in mind they are all dead, regardless of who went first).

Now in this period, where we “think” we paused the network, it died.

This is my conjecture, no maths we know of can control the real world, It just keeps on keeping on. So we go with it or wait for a wee while to realise the consensus we thought we would have can not now happen as we lost >=1/3 of elders while we waited on deciding on who is dead and in what order they died.

If we had acted in a better way then we may still be alive. There are choices,

  1. Concurrently act on each “death”
  2. Batch deaths and be able to have 1 vote for a number of nodes who died.

None of which are easy, mind you :wink:

12 Likes

Helped a lot, but I still need to probe:

How many nodes there are per section? What I remember vaguely, 30 of them is a lot, and maybe so much, that we could accept death of network in that case? We have to draw a line somewhere and if not at 30, then what would be the number, 50, 70?

(Bit on a tangent: If China or any other big area goes offline, it is quite probable, that at least some of the sections are going to be cut half along the midline. If we have 7 elders, that would mean 3 and 4, and not enough majority, right? But in the case of 9 elders, one of the halves would always have at least 5, and that would work, right?)

I have a bit of trouble with this “nature” nowadays. “Nature” is such a large concept, and thing, that I need some more concrete example how nature can be model for these membership / SAP challenges. I mean, when talking about nature in this context, I could see somehting like “laws of nature” as in physics could be relevant, or some animal or plant behaviour, or microbial life, or… anything.

This is probably really stupid idea, but because I don’t see people as a separate from nature, but a part of it - I wonder does the network have to be perfect? I mean, if it dies, maybe we people want to reconstruct it to the extent we can? It’s not going to be fair and some may win and some lose, but if you look how nature deals with things, that’s in line with that.

Or another thing in nature is differentiation of species, which is basically forking. A population is divided, and the parts don’t interact for a while. After some time, they are not compatible anymore. Could we maybe accept forking, if we could ascertain, that the currencies of different forks are not interchangeable? Now that I think of it, forking is actually one of the most central ways living nature works. And in a way different species have different “currencies”, if we think mating rituals and reproduction that way. No matter how much a peacock spreads it’s train, it’s not going to make any spider sexually interested. So could we just have a forking network, that would keep on forking, but there would be some semi-automatic mechanisms for exchanging the different currencies (and datas) of every fork?

8 Likes

I like you both playing with ideas about nature. We should do this more often, while also realising (as you point out) that we need to be careful taking our thoughts about nature and drawing conclusions about how the network should operate.

One thing I note is that biology seems to use diversity (in species) to maximise the chance of recovery of the biosphere from mass extinction events, and a combination of redundancy and replication (in DNA) to minimise loss of valuable data at the genetic level. These are qualities we would like to have in Safe Network to allow reconstruction of files after mass failures, after mass ‘extinctions’ of nodes.

8 Likes

Right now, up to 2ELDER_SIZE times 2 so when we go to split we need 2ELDER_SIZE on each new section.

What I am keen on is sections running with ELDER_SIZE==9 and Adults == 60 (it’s a sybil defence metric that gives us 60).

Yes it’s more likely we will keep going with ELDER_SIZE==9 :+1:

What I mean is our maths based linearisation of events does not line up with reality. Here is an example

Nodes X and Y crash. We spend 30 secs deciding X crashed THEN 30 seconds deciding Y crashed next. However they were both dead probably before we started ordering them as which crashed first and in our order of say X->Y did that really happen? In any case they are both dead. Multiply this up to 30 odd dead nodes and it makes even less sense.

Neither do I, but I do see maths, time, total order as separate from nature :slight_smile: Maths maybe less so, but zero infinity and prime numbers :man_shrugging:

I don’t think so, but total order is the search for perfection.

I agree.

Forking in our conversation is saying there are 2 sources of absolute truth, which we want to avoid as each source can validate a transaction (doublespend).

The problem is the fork is in our network and in our membership which leaks into our data and currency. So a fork is not a separate network.

I think you may mean the larger problem of a network partition. That is a much bigger story though.

I think that you mean divergance here, similar to forking but going in separate ways. So if a network diverges enough, it can be 2 networks. However, there is one section Tree in our case and all nodes can trace back to that genesis. If we have sections, diverge lots of weird and bad things would happen. A maximum of one partition would be lead back to genesis and most likely none would. So in the China event we discussed then China may be gone and not able to operate, unless we alter the design to allow that a partition can continue from a new genesis they create? (but do they have the data, well som of it, maybe most).

Again I suspect you mean partition. Yes it is possible but it is currently a seriously complex issue to solve. It’s one for after launch for sure.

Right now if we have too much divergence the likelihood is the network breaks.

#TLDR

Forking is where we have more than one set of elders in a section. It can allow doublespends etc.

Partition is where there may be 2 sets of elders but in different networks.

They seem similar in some sense but are 2 separate issues to deal with.

11 Likes

This resonates strongly.
Seems clear to me, such a decision should be allowed to be taken swiftly. Does it create 2 different opinions? Well, eventually another vote can be taken, just to have a consistent view of the network, but if a node got kicked out earlier than it should have…¯_(ツ)_/¯

Another situation where this approach should be taken, in my view, is joining the network.
Say, this node from a DC in China requests to join 10s after I did from my NAS. Somehow, the DC node manages to get in before me, say by 5 minutes. Maybe because of the reduced latency, it managed to get a decision to be admitted before me, tough s*t.
Later the network sees two different versions of its topology: one where I was meant to joyn first, one where I sit in the limbo. You know what, you both nodes are in, shake hands and let’s keep going. If you mess up, I have other ways to tell!

BTW watching Gibson’s Peripheral while thinking of this… doesn’t help, lol

4 Likes

@dirvine @JPL Will we get another primer update this year or would it better to wait for the test network to go live in begin of 2023? Latest primer update was November 2021. To me sounds that the current approach seems to be agreed upon beside that it still needs to get fully implemented.

@JimCollinson Any update on the Swiss foundation?

11 Likes

Updating the Primer would be great but involves many hours of work which we know is in short supply, and is still likely to be invalidated by changes as things become clearer.

I’m holding back making time to update vdash until we’ve had some promising test networks for the same reason.

10 Likes

entirely sensible and exactly what I dont want to hear. :laughing:

12 Likes

I plan to take a look at this in December when I have some time off work. There seems to be quite a bit in flux right now, so it might be a mini update at first with a full update to follow once the team have settled on an approach for consensus and got a stable testnet up and running again.

Too true. It needs a solid block of time to focus. Ideally it should be peeing down outside and dark to deter procrastination too, so November/December are perfect.

12 Likes

Personally it helps me to make as many visuals possible which greatly help to understand the top view of the different moving parts, how they communicate, their roles, potential technology they implement like BFT or VCBC. I would like to dive into the github and try to understand the different parts but I don’t know rust. Anywaysss all work is highly appreciated :slight_smile:

5 Likes

Just the last of the paperwork getting signed now. And the comms to the financial regulator being finalised via there legal team now too.

16 Likes

When you describe your view of fault tolerance, you often describe it in absolute terms (node count). Why not consider it in terms of a byzantine fault tolerance ratio, BFT = bf/N, or crash fault tolerance ratio, CFT = cf/N?

For N=9,bf=2, you only have a BFT of 22.22%. That is a fair distance from the limit of 33.33%. Simply going to N=10 would give you a BFT= 30% … ??? This is significantly better performance, no?

2 Likes

I think the %s here can mislead. The BFT Is just strictly <N/3 so for N==7|8|9 the BFT is 2. If we increase N we have a smaller % of BFT and a bigger % of honest nodes.

So it is kinda like you have these bad guys and you wanna fight them. You have the choice of a team of 5 (N==7) or a team of 7 (N==9) honest guys on your side.

Then the performance issue of getting consensus from a larger group seems more difficult, but I think it’s not so bad actually as we always want 2f+1 votes for agreement So the agreement size is constant as is the BFT size. It means we have the extra valid voters able to help, or do they hinder? I feel they help, but for sure they definitely help us keep the network more live.

6 Likes

Thank you for the heavy work team MaidSafe! I add the translations in the first post :dragon:


Privacy. Security. Freedom

8 Likes