But more telling is a total ignoring of all the other issues
And thats why we test.
Are you worried cloud providers will lose all their data one day? They are using common software supplied by the linux core.
Should we have 10 or 20 significantly different linux cores within each cloud provider?
Of course they do not do that since it would be a total nightmare to maintain and support. My points above would still apply.
This is sounding more - lets divide the network without realising it will cause the network to fail. @urrtag says it better
And we end up with the problems I outlined above. If you want simple redundancy then lets just have the one network and use the archival nodes that are in the plan to be developed rather than wasting that time on working out the way to do the dual network which has the only advantage of redundancy but massive issues of payments for PUTs coordinating the transfer of data between the two and so on. Iâd like to see the network soon and not put back the implementation to change directions now.
Agreed. The above comment was just brainstorming to try to look at the thread in a more positive light. However âŚ
True that a mirrored network doesnât really have any benefits for better redundancy of the data, but it offers redundant consensus or governance and a means for self-optimization. Itâs a another check and balance against malicious nodes and could offer error correction.
My comment above was incomplete. Itâs not really two networks but one with a dual address space. For every section, there is a dual section that mirrors the data. Safecoin would never need to transfer between the two, because you would have a dual Safecoin* there, and the transactions would be mirrored in parallel. Client PUTs go to both networks but either one could serve a GET on that chunk, analogous to a RAID 1 config at network scale on all data, metadata, section membership, and consensus (aka MAID 1?). I agree that it is pointless to consider further until a working network is out in the wild and well established, but wanted to also point out that it might be something that warrants further consideration in the future as a security improvement. Once the base code is working well, I donât think it is as complicated as you believe to add in a mirroring capability. It would be the same vault code, and modified routing code so that some vaults would be assigned to a primary XOR space in primary sections, and others in a dual or conjugate XOR space in dual sections. Is it worth going through this trouble? Maybe not, but maybe yes. Again, to make it clear, I agree with you that it is pointless for any dev to consider at this time.
But at huge cost of equipment, bandwidth, reduction in storage capability, reduction in earning capability and/or increased cost to store. The same redundancy can be achieved with proper network restart (which is a fundamental goal) and adding 1 or 2 or 3 extra copies (depending on minimum copies needed to start with) Think RAID This results in a small decrease of actual storage capability. Remember often there are more copies kept than the minimum anyhow as a node that goes offline for a very short time causes a new copy of the chunks to be kept and when its back online it becomes the one extra than minimum needed.
This needs no change to the code apart from the magic number of minimum copies to have.
And it seems your explanation later hints this is what you were getting around to except you wanted mirrored rather than accepting a higher form of RAID style of protection
The point of my comment was not really about data redundancy, but instead metadata redundancy, keeping consensus safe, detecting malice and opening up the opportunity to evolve or tweak operational settings in the dual network by checking the performance between the mirrors.
I guess a few fundamental questions are :
Is it better to have single section with a total of X nodes working on consensus or N identical sections each with X/N nodes?
Is it possible to use dual sections to better determine which nodes are malicious?
Is it possible to determine if an error has occurred or the section has malfunctioned?
I suppose that it is not feasible with only two mirrors, so N must be odd and N>2. You would need three at a minimum, in the same way that Parsec needs at least a 2/3 majority of benevolent participants.
So this spawns another fundamental question:
Is it more performant, effective, secure, better to have a single section with X nodes participating in consensus and data storage with a chunk redundancy of Y, or N=3 mirrored sections. In the case of three sections, each would have X/3 nodes and Y/3 chunk redundancy.
The added complexity in the mirrored case is that you need check to make sure that all three mirrors are eventually consistent, but this offers error correction. I suppose you could call this methodology Parsec^2 or Parsec-Squared. Itâs a scenario where you would have a hierarchy of two Parsec mechanisms operating. The first level which is the current method for obtaining intra-sectional consistency, and a second level of Parsec that compares the mirrors for super-sectional consistency. I wonder what @pierrechevalier83 would think of something like this? Too complex? No benefit?
Letâs say the cost to run a round of Parsec is C(X). Which is more expensive, C(X) or C(3).C(X/3)? I know, gross oversimpication.
Edit : Assuming C(X)â>O(X.Log(X)) complexity as described here:
Iâm not so sure. You would need to fool 2 of the mirrors simultaneously with certain timing. There is a potential for ongoing error correction due to the second tier parsec votes. If a single section mirror goes bad the other two sections could vote to flush it in the same way that nodes detect malice within a section and terminate a bad node. Alternatively, the extra information from the two section mirrors that are in agreement might be used to aid the offending section mirror so that it could better determine which nodes are malicious and terminate them without needing a full flush of the section mirror. I think itâs more like if fooling a single section costs C then fooling two sections simultaneously would cost at least C^2 not 2C.
But then they need to have different consensus rules, making the mirroring even harder as one operation can be valid in one network but invalid in the other, not even speaking about multiple safecoins (you could have 0 SC in network A and > 0 in network B causing divergent behavior on PUTs. You could force net A to mirror that PUT if it gets accepted by net B but that could break the consensus of net A, as itâs not validated by net A, and make it even more vulnerable to attacks)
Thatâs a good interview. I feel a bit sorry for Linus having to bend himself out of shape. I should imagine (like quite a few good scientists, engineers and techies) heâs somewhere âon the spectrumâ and diplomacy does not come naturally to him. Not sure Iâd like to work with him particulary, but I doubt he deliberately means to be nasty.
I donât think you read the whole brainstorm to the end, where the ideas strayed significantly from the OP. The whole point is that the the consensus rules in each mirror need to be the same, and that an operation needs to be valid in all mirrors, otherwise an error or malice has been detected. The question for mirroring or âsub-sectioningâ comes down to the following:
A second layer of Parsec might have modified rules to confirm that 2/3 of the mirrors agree on what events occurred, ie. a confirmed consensus about the consensus. Maybe call this a meta-consensus?
Not really necessary to consider for launch, but might be a potential optimization later down the road.
Even then one op could be valid in one network and invalid in the other, depending on the order of ops coming out of PARSEC (each PARSEC instance might decide on a different order of events based on the exact timing of each event, the gossiping timing and so on), the structure of the network (you would need to have the same sections with the same nodes in each section on both mirrors), âŚ
The order of events is very important, as itâs the factor that is used to determine which events are valid and which arenât.
say you have a SC account with 1 SC on it, but create two transactions transferring 1 SC from that account each, only the first tx will succeed determined by the PARSEC order
two nodes are updating one dataset concurrently. the current version of the dataset is 1, each node is sending its new dataset with version 2 (current version + 1), because of the same version used by both nodes only one update can succeed, again determined by PARSEC
