Very interesting! So wondering if others got the same sum up take away. There was an epiphany about how the network actually communicates that means the code base can be greatly reduced for improved efficiency (and security)? This is something that is actively being implemented throughout the rest of testnet 2 & 3. And insights of specail coding language will be soon published? All of this is crazy and amazing sounding just want to be sure I understand it all 
Yes, pretty much that’s what I get. As there was a great deal of complexity in the routing code, David was trying to re-factor, or break down the routing code to it’s simplest form. The main issue surrounded different varieties of Consensus groups all interacting and communicating in inter-weaving ways spanning the Network and Vaults. While pondering about the way “classes” in C++ are structurally analogous to different genus, ie totally different animals with different functions/abilities - the epiphany as you put it, was that maybe Consensus groups could be reduced down to their basic “genus” too; this was done and it was found that it can be reduced to just 4 genus. This means code can be reduced/simplified and the way messages are relayed can be done more efficiently. Importantly, it also completely untangles the Vault code from the Network code, which will just simplify things all round.
That’s what I get from it anyway, but I’m not a coder.
Basically, David has just unpicked all the knots out of a massive ball of wool and is now starting to knit with it…no, I don’t mean metaphorically – I just hacked his web-cam……lol
I think you @Nigel and @Al_kafir have summed it up perfectly.
Essentially each kind of “Concensus Class” (or genus) can be:
a) simple (because David has spotted that messages carry the information needed for the routing layer to know what the messages are about and what “role” (class) they are for etc.), and
b) just written once, because the variants (“instances”, or perhaps derived classes) of a Concensus Group class behave the same and so can use the same code.
I really loved the description of the creative process. I think that’s another real gem in this post.
Just to re-factor my reply for non-coders like myself…lol:
. An analogy would be that David opened up a telephone exchange box stuffed full of inter- weaving “consensus Group” wires. His task over the last few months has been trying to explain this spaghetti wiring to everybody, using words and diagrammatically- the system worked, but hard to explain why - even to other engineers. This has hindered community dev involvement as all the explaining takes up time.
He then noticed that there were only 4 types of coloured wires and connections, and that these could be arranged sequentially. This now means he can re-place the spaghetti of wires with just 4 high-speed optic cables arranged more linearly.
The benefits of this are that now the network is more efficient and less can go wrong, and he can draw a much simpler diagram and explain things much more clearly to other telephone engineers. This means more engineers will be able to grasp the wiring system better and so help development and be able to plug in there “apps” easily , thereby speeding the whole process up. This also leads to mathematical/scientific “proofs”, which once people see, will enable them to grasp the concept easier.
To re-factor it further…lol …I’m learnin’…David has found a Worm-Hole across XOR Space-.time
Edit: Love the way we are all translating from the original Scottish mad scientist…lol
Brilliant @Al_Kafir, I think you should do one of those “cartoon being drawn” video explanations using your painting skills - Rolf Harris style (or perhaps not 
). [evil laugh receeds into the distance, punctuated by the startled meow of a very fluffy white cat].
There came one word in my mind after reading the post by @dirvine which I only got for maybe 25% after a first read. the word is KISS. Keep it simple, stupid. Keep it short and simple. Keep it simple & straightforward, you pick one. Easy for me to say while not being a coder working 8 years on a project 
@Seneca helped me out yesterday on the chat to understand the concept a bit more. @Al_Kafir his post is great as well. Cleared it up quite well. Here are some things I think I got from it:
- Maidsafe is a P2P-network where all communication/action is done in groups of 32 nodes. Out of these 32 nodes a user needs 28 nodes to reach consensus. No matter what you do (sending Safecoin to someone, uploading a file to the network, retrieving a video of your dog which you uploaded a week ago) you always have to deal with at least 32 nodes.
- The “Vault” is like the engine of the network, where the XOR-addresses and routing are the locations and highways. In the vault a great number of “persona’s” can be created, all with their own function and address. Each persona works in a group of 32 nodes, and because the persona’s work quite the same from a technical/coding point of view, new ones can be created ans tested quite fast.
- So, one persona can be part of a group which does Proof-of-storage audits on other nodes (is that node really holding chunk ABC?) Another one will just help you to retrieve a chunk from the network etc.
- With the updated vault, new persona’s can be created, explained and tested more easy. Let’s make up a new persona and see what it does:
Folding-persona <-> 32 closest Folding-nodes <-> Problem-pool
As you can see this persona joins a group of 32 close nodes, get’s a number of problems and start to do calculations.
Proof-of-Folding-persona <-> 32 closest POF-persona’s <-> Solution Pool
This persona get’s a problem and solution given by a Folding-persona from the solution Pool and makes sure the calculation get’s done by another Folding-persona to see if the outcome is correct.
This list can go on and on with persona’s to pay people for doing calculations etc. And new persona’s can be added to the network every time a new version of Maidsafe is launched. This happens without “hurting” the older nodes. Because the older nodes (vaults) don’t have a clue which new persona’s are around in updated versions. There can even be “light clients” with a small number of persona’s (like smartphones) and full clients with GPU-power to provide calculations etc.
Yes, my goal is a planetary network that learns. When I first seen computers way back, I thought why not give one to each profession that just asks, Who What Why Where When (less important). I still think of that, but now with the Internet and SAFE as well as projects like opencyc there is a much larger opportunity and when we are launched it will be my toy projects. Allow all devices to communicate semantically, gather the info and share it. I.e. teach one robot CPR and they all know it immediately, goes for any skill.
With the persona type cracked this becomes a much simpler proposition and can be easily mapped out and debated. I think this is pretty exciting, I just need this one last wee push now and we are there. So back to code for me Cheers tho, great to read the feedback, I am delighted it is clear what we can now do.
The opportunity is way to large to bypass so we are implementing this as we speak ( at breakneck speed ) and it’s looking fantastic
It reads like the biggest breakthrough on this project since inception and the inventor is very, very happy…I like that
If you were on-boarding staff to a POD, this post would be prerequisite reading i.e this is the type of thinking that will earn major SAFEcoin from the pool.
I wonder how the rewards get weighted for Core Devs…a breakthrough like this by a POD would be epic and deserving of considerable reward given the flow through effect.
But how is the level of reward decided? By a consensus of humans or maybe the network itself can also provide feedback somehow…
I read somewhere that engineers are the one’s that will save the world, I agree.
I have also be wondering about this with @ioptio and there would seem to be no obvious way to automate. Possibly consensus of network engineers (from any pod) who use a voting app.
Will this recognition of the four types affect transactions and transaction Managers? Are Transactions Managers a type of NAE manager?
If so, will there be a possibility for alternate transaction types that are handled by the core of the network, (I’m hearing that talk by Johann Gevers and the notion of something that is capable of handling “all legal transactions”). Perhaps these alternate transactions would require a certain amount of transaction fee in Safecoin to make them worth the network’s while, but still worth thinking about.
Yes they are.
Yes all that happens now is we add more transaction handlers in place. So we may want to handle
Put<Will>
Put<LegalForm1>
And so on, more interestingly though the Post methods … hold on I missed something
Get (retrieve stuff)
Put (Add to network)
Post(Mutate something you own)
So you could have say a bill of ownership (Put on ) and then altered in a programmatic way (change owner) so
Put<OwnershipBillTypeX>(abc)
and later
```Post<ChangeOwnershipFunctor>(abc)``` which will change ownership say (in this case)
So we could put all the legal types as entities if we wished and the appropriate Post (mutate) functors and the network would handle that. This can be described in the new language we will hopefully publish later today/tomorrow (when @BenMS and Mahmoud agree :-) and then the devs vote )
I would speak with Johann at this stage as he has extensive research done in international laws and the likes. Not governance though as that's different, mostly dispute resolution.
This is an example so you can imagine any type here and functors to be applied. Ultimately we will expose a mechanism for anyone to create types and the network handle it. This is where it will get super interesting and a formal DSL for many things like safecoin, legal forms (if these are actually needed where we are going), computation, AI, search etc. can all take place and the core team become much less responsible for core actions, just the framework that allows this and new mechanisms will need to gather consensus to work. Far from simple, but we just jumped to a few feet of the line.I already heard that this was big, but now I’m seeing what @dirvine meant by it making it easier to build new functions in the network.
If I understand correctly, the core SAFE Network will be able to handle user defined code (eg smart contracts) much more easily, and perhaps even mathematically proven (without the drawbacks of a general purpose smart contract programming language, the turing complete problem). I’m now wondering if the discussions with Vitalek last week were to bring him in on this breakthrough.
That was mostly explaining to a group how consensus works, so from the first post here all through time with the same questions over and over again, 4 hours it took me .
I think Vitalik (a very sharp person who seemed not not require all the basic questions to be answered, but more concerned with learning, very good) and his group will ‘get’ this though and hopefully see the benefits. Now it’s a thing and not a belonging it should help people just adopt it.
Unfortunately the persona types was a 2 month issue for me to wrestle with, done now though.
This is so huge. This makes everything possible.
This is everything you would need to create any kind of property registry, DAO/DAC, any kind of cryptocurrency, governance structure.
You could build an entire legal system, enforceable by the code with such transaction capability.
To everyone who follows this forum, lurkers posters, EVERYONE. This is the most important thing you have seen.
This will change everything.