Understand if-how Parsec solves this problem, please explain/describe Parsec

[mav]

@g.u.y

I think I’m making good progress to understand the algo currently.

Yeah it does seem like this which is very cool. If you ever felt like writing a layman explanation of PARSEC from what you’ve learned I’m sure many would find it very useful.

@g.u.y

sometimes, all going on is that only ONE node will pass a vote, and the other nodes will simply accept that one vote.
…
Either node would obviously be free to respond to that one vote, “you know what, I have a vote too” - at which point you’d probably “upgrade” to Parsec with its bigger number of rounds & steps immediately.

Sounds like this would be possible. Seems like if events of type one-node-only-passes-one-vote-only (including acceptance) start happening at a rate close to the expected parsec latency then there’s possibly some ‘confusion’ about the benefit but overall it sounds like it would still work.

A simple illustrative example that comes to mind is

• A sends one-node-only-passes-one-vote-only event to B,C,D
• B responds ‘looks good’ to A,C,D in 0.1s then immediately after responding finds they want to create a new event. B broadcasts the second event as a parsec event.
• C for some reason takes 2s to respond to the original event, and so has received both the original and the second event from B before they’ve managed any response. C tells A,B,D to upgrade both events to parsec. Maybe C doesn’t need to upgrade the original event if the second event builds on the original, but more nodes means more chance of conflicts.
• Is there ‘waste’ from the original event and if so is it acceptable? I think it depends on how many nodes there are, how reliable nodes are, how frequent events are, and the in-real-world difference between parsec events and the ‘fast’ vote type of event.

Do you aim for the proposed changes to mainly improve performance or to mainly reduce complexity? Or both? To me it’s not really clear yet what the effect of the change would be for either performance or complexity.

[g.u.y]

@mav Hi Mav glad to talk to youu, the purpose would be improved performance.

Indeed Parsec’s complexity would remain, so this would mean a higher complexity than Parsec.

Wasteful events should not really be a problem.

Will explore this path later, now still remains to ‘Get’ Parsec on detail level.

By the way, do you think the 15,000 locs for Parsec’s reference implementation are really necessary for a reference implementation? Just wondered as 15klocs is quite a bit.

[g.u.y]

Hi @pierrechevalier83 and all, just curious, are you aware of any more Parsec implementation to date aside from https://github.com/maidsafe/parsec/ ?

After meticulous reading of both whitepaper, RFC, the ABA paper for approx 50 hours and now warming up reading the sourcecode, I see no lack of need for additional clarification, so any more resources would make a great difference, thanks

[dirvine]

None we know off, just web search to look, but I cannot see any so far.

[tobbetj]

Have you also seen the YouTube videos with Parsec, on the Maidsafe channel?

[digipl]

@g.u.y

After meticulous reading of both whitepaper, RFC, the ABA paper for approx 50 hours and now warming up reading the sourcecode, I see no lack of need for additional clarification, so any more resources would make a great difference, thanks

You can read the references that appear at the end of the Parsec whitepaper. Especially interesting is the The Swirlds Hashgraph Consensus Algorithm since some ideas of the protocol gossip, within parsec, have been inspired by this whitepaper.

[g.u.y]

@digipl

You can read the references that appear at the end of the Parsec whitepaper.

@digipl so I have read the whole Parsec reference sourcecode now and the algorithm and Parsec’s workings is still totally unclear.

What parts of the Swirlds paper could have any instructive value in studying Parsec?

[g.u.y]

@tobbetj

Have you also seen the YouTube videos with Parsec, on the Maidsafe channel?

@tobbetj Hi Tobias yes I have watched those videos with utmost care also. I’ll follow up in this thread with the additionally clarified unclarity shortly.

[g.u.y]

@dirvine

None we know off, just web search to look, but I cannot see any so far.

Agreed. It could be a community challenge to make an easily-understandable reference implementation in Javascript, such. Anyhow will get back with questions shortly, will start a new thread about the ABA paper and its separate unclarities first.

[g.u.y]

Hi @digipl @anon86652309 @pierrechevalier83 ,

Now posted separate questions for clarification about the ABA paper here ABA paper unclarity and queries, discussion, please help clarify , kindly have a look when you have time.

I reviewed the whole Parsec codebase now, line by line and following code paths.

It’s nice to see logics relating to the ESTablished → AUX → DECided_values mess in https://github.com/maidsafe/parsec/blob/master/src/meta_voting/meta_vote_values.rs .

I think I can summarize the biggest unclarity like this:

After 50 hours of intense study of about two re-reads of each of Parsec paper, documents, video and reference implementation, it’s still perfectly unclear to me WHAT you vote about in Parsec, and HOW the EST->AUX->DEC+coinflip flow helps that consensus vote

If you could help clarify this would be much appreciated. You suggested already that this is mere “unclarity about terms” that I solve by simply looking them up in the RFC, but I assure you it is not so.

I have not been able to apprehend the scope.

I want byzantine consensus, but how does Parsec actually achieve it.

rfcs/gossip_graph.dot.svg at master · maidsafe/rfcs · GitHub is a great case in point as the unclarity is already in place around a_0/a_1/b_0/b_1/c_0/c_1/d_0/d_1:

Firstly how do you represent multiple votes in parsec
The point being, if Alice, Bob, Carol and David are voting about seven different options of grocery shopping lists, then how should this be introduced to the algorithm, so that it eventually gets into the ESTimate value and so on.

The paper and reference implementation hardwiredly discuss the usecase of join/part Network Events, however that is only one very specific application of Parsec, is it not.

Kindly explain to me using how many Gossip Events you would introduce A-Ds preferences about shopping lists 1-7, each of A-D may vote for any number of the seven shopping lists?

Secondly what sense does EST->AUX->DEC+coin make
So now A, B, C and D have voted for which grocery options they’d like to have bought.

So let’s get into the algorithm, here illustrated by the paper figure 3:

All that is unclear relative to my pictured usecase, so let’s kind of start from the beginning at the figure’s bottom:

I understand that the nodes will gossip something to each other. I’m not sure what. I see in the reference code that they send

• Firstly the “initial event”, https://github.com/maidsafe/parsec/blob/b1bfdb790f8010cfec1151a4ff7236d0b6b7de74/src/parsec.rs#L166 , which only basically bootstraps a peer’s participation in the DAG (https://github.com/maidsafe/parsec/blob/b1bfdb790f8010cfec1151a4ff7236d0b6b7de74/src/gossip/event.rs#L172)??

• And secondly the “Genesis Observation” https://github.com/maidsafe/parsec/blob/b1bfdb790f8010cfec1151a4ff7236d0b6b7de74/src/parsec.rs#L170 , here the observation really means cause doesn’t it, meaning what kind of Gossip Event… er. Anyhow, the peer shares which peers IT thinks it has, at start. Why does it share this?

• If I understand the Parsec class right, the next step now is that all the peers are supposed to run vote_for() to shar their grocery list options, that code is here https://github.com/maidsafe/parsec/blob/b1bfdb790f8010cfec1151a4ff7236d0b6b7de74/src/parsec.rs#L284 .
   
  Can you please illustrate how A’s, B’s, C’s and D’s .vote_for() calls should look?
   
  (The only example that exists currently for Parsec makes one single .vote_for() call which is totally undidactic, it’s here https://github.com/maidsafe/parsec/blob/b1bfdb790f8010cfec1151a4ff7236d0b6b7de74/examples/basic.rs#L157 , I don’t see at all how to understand that line especially when not understanding anything else about Parsec’s scope or context in the first place.)

• Next, according to the user’s random timer which should have which interval, the user is supposed to run Parsec’s .create_gossip() (https://github.com/maidsafe/parsec/blob/b1bfdb790f8010cfec1151a4ff7236d0b6b7de74/src/parsec.rs#L316) which is the driver of the consensus process. At respective node’s reception of Request and Response call, the .handle_request() and .handle_response() should be run.

• And then user is supposed to run .poll() to pick up any consensus results, and these are obtained as a vector of “blocks”.
   
  This begs the question, what is a block actually?
   
  The RFC kind of mentions them sweepingly, and mentions there are non-finalized and finalized blocks.
   
  Also, how do votes and blocks correlate with the seven grocery list voting options in the first place.

• Somewhere under the hood, the EST->AUX->DEC juggling is going on, this file https://github.com/maidsafe/parsec/blob/master/src/meta_voting/meta_vote_values.rs#L205 is involved.
  What is the code path actually?

• How does the Parsec class interface the MetaElection class? I see Parsec.process_event invoke MetaElection.new_election() at https://github.com/maidsafe/parsec/blob/b1bfdb790f8010cfec1151a4ff7236d0b6b7de74/src/parsec.rs#L779 , is this the place where Parsec asks consensus to do its job??
   
  The same call (new_election) IS USED to remove already-consensused decisions, via the payload_keys argument.

• Then a contextual question: What is the DKG = Decentralized Key Generation about, is this a Safenet-specific feature inlined in the Parsec code used to generate Safenet keys, or does DKG actually have something to do with Parsec?

If I understand Parsec right then, the grocery list options should sooomehow be translated into sooome kind of Gossip Events, if it’s one per grocery list option peer choice or not I’m not sure of. Anyhow, that-something-thing will be gossiped, and as one node gets a supermajority of votes it will gossip its conclusions to all the others, and when a node has received a supermajority of other nodes’ conclusions, at that point unless you’re in a 50-50 vote, you will start reaching decided values. Is this a correct zoomed-out understanding of Parsec?

The documentiaton and all materials are so exceptionally unclear on this overall purpose level that I need your reassurance.

The subsequent question would be, “now that this is how it is then how is the EST juggling and the AUX value helping, and is there any meaningful documentation about their motivation and workings in any paper, slides or implementation anywhere?”

The whole Blockchain-Consensus phenomenon is still in its early infancy, I am very grateful to you all who participate in this and also while I’ve whined for pages now I acknowledge that your work is better than others’. I hope you can take the time to clarify, please step out of your PhD suit for a second and speak commonsense.

Thanks

[g.u.y]

Hi all, Pierre said he’ll respond to the previous question as-when time allows.

Meanwhile I’ll pass one lift one particular Parsec usecase:

@pierrechevalier83 @anon86652309 @AndreasF and others, could you explain how you set up Parsec to do global order (“serialization”) only?

In other words, voters Alice Bob Carol Dave Eric will at their own discretion add events (for simplicity, a string or a binary blob) to the DAG. Parsec is used only to provide a globally agreed on order of these events.

So basically,

var parsec = new ParsecWithTheGuys(); // All of A B C D E run this, it sets them up.

myGossipEvent = { message: “BINARY BLOB HERE” };
parsec.post(myGossipEvent); // Blocks until Parsec produced a global order

println(myGossipEvent.globalorder); // Will tell at what position in the global order Parsec is. This is such that a supermajority of A B C D E will in this moment agree that this is the one and final global order number of myGossipEvent. Those who don’t agree only don’t agree YET and do so because they lag or are byzantine.

println(parsec.allFinalizedEvents); // Will print out all events that this node knows of as being finalized, with both global order number and content. This also will be produced the same by all of A B C D E, except for if someone is lagging behind in which case this myGossipEvent’s number height will not be reached anyhow

Would you say that this is the primary use of Parsec, or one use of Parsec?

If it’s one use of Parsec, then is this a central or peripheral/unimportant use?

Thanks for your clarifications

[loziniak]

Lots of this topic should probably end up in documentation.

[g.u.y]

Hi Loziniak, right, this is also why I ask these questions here publicly, that hopefully they function as a kind of brainstorm-checklist for anyone who will write Parsec documentation later. Ok I hope @pierrechevalier83 @anon86652309 @AndreasF can have the time to inform what about the last point, soon. As Pierre and others point out, the realities of “consensus science” are a bit different from what they may seem from outside, this is also why I pass some common sense questions like this, even if I may have a maybe-educated-guess already in a given question, having these suppositions confirmed by the experts here is invaluable, even just for such a basic objective as understanding what Parsec is and what problems it’s great at solving.

[Cryptoskeptic]

As long as they aren’t taken off their Dev jobs, I don’t have any issues, so it doesn’t detract from the release schedule. Perhaps during their toilet break

[happybeing]

@pierrechevalier83 and @AndreasF no longer work for Maidsafe, so no worries about distracting them [kidding]. It is a credit to Pierre that he has been answering at all after starting a new job!

[dirvine]

Fraser is also off this month. He is finishing some holidays and will still be around, but will not be working as part of the maidsafe team. Well officially anyway, we will potentially see him and others later on in the game. He still has his seat in the office though and still has work to do with the foundation.

[anon94252342]

Why was this not mentioned in the update? I think its important to communicate if empoyees are leaving…

[neo]

@anon94252342

Why was this not mentioned in the update?

Why is there a requirement to announce ALL holidays by people?

@anon94252342

I think its important to communicate if empoyees are leaving

Who is leaving?

[anon94252342]

@anon86652309 apparently if you read above post from David. No requirement to mention all holidays but if one of the core devs are off for a month , then it would be nice to know right? maybe you dont need to know but others might care more.

[dirvine]

@anon94252342

Why was this not mentioned in the update? I think its important to communicate if empoyees are leaving

Wait until we are at 200 or so then we will have leavers every week almost. Fraser’s a quiet guy and does not need the publicity. He is sticking around and was a friend before joining and after (we worked the lifeboat together) , but he is entitled to try new things. He will and I wish him luck.