Code can be found here:
Back when I started this topic back in early November How will Parsec be used?, I had a drive for an idea:
I haven’t continued with the idea since, and I wanted to get further with the idea before I posted anything.
But better to just get the it out there, for anyone to look at, as I might not follow up on it for a while
Parsec for a decentralized event sourced application
So, what I wanted to do, is to use Parsec for a decentralized event sourced application. Any event sourced application actually.
What an event sourced application - following basic
CQRS principles - consists of, is basically different streams of events, produced by a construct called Aggregate, which protects invariants.
Now, the idea was that a binary could be running on machines, forming a network much like how SAFENetwork is formed, but that in addition to this, Parsec is used to reach consensus on actual domain events, which in effect makes the logic network agreed, i.e. a decentralized application.
For more details on EventSourcing and CQRS, see this post SAFE Crossroads Podcast #38, Event Sourcing on the SAFE Network with a detailed explanation of some of the basic concepts.
So, I did what I usually do:
A simple example using extremely simplified mocks.(Bare with the extreme simplification of Parsec, it is meant to just roughly show the principle.)
We have from DDD, EventSourcing and CQRS:
We have from SAFENetwork:
We have the obligatory example:
- Notebook, which is an Aggregate.
And finally, we have a client for this, that combines these parts to a running program.
All this runs in a console application, with an utterly simple end to end interaction.
Naturally, this is such a naïve implementation, that it only serves as a draft for displaying the idea of the concept.
Now, what happens here is that we will run the console app, which sets up a small group of decentralized clients in memory, takes a random client and request a change, which will then be propagated in the group through Parsec, and finally agreed on.
In the Run method of a DecentralizedClient, we see that first of all the client joins the parsec group.
Next, it will enter a loop, where it keeps polling parsec for new events that has been agreed upon by the group.
After receiving any such, they are applied to the local state, and then passed on to the EventRouter, which makes sure any events are dispatched to subscribing ProcessManagers. A ProcessManager is able to issue new cmds as a result of any event it picks up. You can think of it as a state machine.
The cmds that are produced, are then applied to the locally held Aggregate, and if it was a valid request, the change is also requested on all the other nodes in the Parsec group, as well as voted for. (So, again, bare with the extreme simplification of Parsec.)
Now, this is the fun part: what happens here is the very thing that will result in
_parsec.Poll on line 30 in previous image
DecentralizedClient.Run(), yielding a result - if consensus is reached!
_parsec.RequestChange(cmd) on line 53 in previous image, will result in this method being called on all nodes in the group.
When events have been agreed upon, and received through polling Parsec, we will reach line 36 in DecentralizedClient.Run(), which calls the Apply method. This is what actually applies ther events on the local aggregate state, as well as persist these events to a db (probably a SAFENetwork db).
Grasping what this is
So what’s happening here?
Well, we have a binary, a program, and we ask machines to run this and to form a network using Parsec, as to reach a decentralized consensus (network agreed logic) on what the results are from running this program.
An aggregate is identified by its type (
stream name) and id (
stream id), forming the
stream key. In this example it could be
Notebook@fe240945-4a94-e093-b044-989a1a7ecdd5. The XOR address of that unique stream key, will used to localize the group responsible for it, just like data and vaults in SAFENetwork.
And so, a very fine partitioning is achieved, since the growth of data in a typical EventSourced application is preferrably on the number of streams, rather than the number of events in any particular stream - each stream representing an instance of something, with boundaries so defined, that the invariants are protected within that aggregate (remember, aggregate : stream is a 1:1 mapping).
Now, this is for all clever people out there to crunch:
- Data sensitivity everything that happens to an aggregate is open and readable to a node responsible for that stream. How can that be solved?
- Projections. These are by definition a state that is built from events from any stream, which means it possibly needs to subscribe to events from all groups i.e. the entire network. David mentioned secure relay messaging. What options do we have to make this feasible?
- Everything else. Yes, there’s virtually no end to the possible problems. If you want you can fill in here
Not much is planned here. This is, as usual mostly a thought experiment from my side.