I stoped trying to understand a while ago 
Now just a cheerleader and look for the comments to possibly understand whats happening
9 Likes
Personally I think itâs great we have non technical people so engaged in even reading updates. It means more than just tech savvy are wanting this and who better to test it?!
Love the update. Hooray for the great progress @maidsafe makes every week! 
19 Likes
Interesting related page Large Triple Stores.
eg âLUBM 4400K: 1.08 Trillion triples were loaded, inferred and queried. The industry-leading results, and the details of the configuration and best practices are described in the white paper Oracle Spatial and Graph: Benchmarking a Trillion Edges RDF Graphâ
8 Likes
Curious, are CRDTâs and DBC alternative approaches to a Safecoin implementation or meant to be used together to compliment each other? Really hungry to know more details about the relationship between the two.
So CRDT is a data type? That allows for eventual consistency (like PARSEC does) but without the need for PARSEC and therefore avoiding additional performance load? Obviously PARSEC is essential for coming to consensus for basic network operations, building trust/node age, etc but in the instance of Safecoin it may not be needed with the use of CRDT?
13 Likes
I have a request: would it be possible to include a section in the update that would allow non-technical people like me to make a realistical where we are at? Or is it impossible to do that? I understand progress ia made, but I cannot assess it qualitatively, so a section for dummies would be awesome.
I mean, sure, there ia a roadmap, but it gives any clue with regard to time, does it? On the other hand project ressources are highly critical and scarce, so if I understand things correctly the time until completion of a non-beta version should be forseeable.
I remember 2015 when I kindly asked for a roadmap with scales. Now we have a roadmap, but with no scale attached. Or am I looking at the wrong place?
P. S. : donât stone pls
5 Likes
I work in software development and progress around here is hard to follow. More enjoyment can be had from understanding the concepts.
3 Likes
âMacho Manâ Randy Savage is still there if you âimagine itâ. New guy is âOld Dirty Bastardâ.
Cheers have a good weekend and enjoy 
7 Likes
Is there a gh issue or something? i would like to learn how you can use crdts in an permissionless context. bounded counters seem to limit the quota each replica is allowed to transfer without contacting the other replicas, but it could still just ignore that quota without the other replicas noticing it immediately.
is this the crate youâre going to use: Feature Request: Bounded Counter ¡ Issue #61 ¡ rust-crdt/rust-crdt ¡ GitHub ?
3 Likes
Good luck but I think the answer is No. Thatâs why I suggested disconnecting the project boards. It serves little purposethese days
3 Likes
Everyone knows where the project is at and the updates convey progress. The project boards only eat up scarce resources. Perhaps they can be picked up when resources arenât an issue
5 Likes
We can make a CRDT permissioned, but permissioned by either the clients signature or the sections signature. There are a few nice mathematically (now thank god) guaranteed features and even some without tombstones (no infinite growth etc.)
They are a significant aspect of decentralised or distributed computing and since 2009-2011 they have become accepted. These respect the maidsafe designs from way back and allow containers to be ârealâ in terms of âthis is the current answer, check back it might changeâ.
So if you look at dynamo / cassandra back in the day (you will see me talking about them a lot a while back) you can see the speed and efficiency improvements (especially latency).
So we have both an ABFT and many data types, our data types are good but forcing CRDT requirements makes them much better and in a great many cases not requiring ABFT/PARSEC for everything. So self validating on merge.
It really gives the Engineers goals to confirm on each type to give us a nice set of features.
These goals are
Idempotency (replay message/ops as much as you wish)
Associativity (a Ă b) Ă c = a Ă (b Ă c)
Commutativity ( a Ă b == b Ă a ) or (X% of Y == Y% of X)
[edited Cheers @tfa ]
If we ensure the types follow these requirements then concurrent access and updates are handled much better than stopping the world till we get agreement.
They generally work on grow only data, which most of ours is. However, you can use orswat to remove tombstones (the removed items etc.) and bounded counters to allow delete (spend) and add (deposit) in a non-negative manner.
I have mentioned these for a long time but they are new (not as old as maidsafe but interestingly original design and data chains depended on them) and not accepted as possible by many Engineers, mostly because they have never seen them.
Ultimately for us right now it allows us to remove any unknown errors we could have in syncronising writes to data. These save that by swapping sync with the merge of state of operations.
tl;dr More formality to data structures and possibly more. Giving us reduced latency and a very important thing I am keen on, what comes out of ABFT in a multi-part decision (such as add/delete) is consensus valid but not able to be applied, if that makes sense. However, what comes from a merge is valid. On occasion, you can have forks if you have linear requirements and this is more a linked list type structure.
so more formalised structures, not necessarily more work, just more correct work.
21 Likes
Good point. The boards are almost complete for now. lotâs of clean up testing and more testing happening. I will ask @StephenC to get us in line with these and we will try and come up with a way to show folk where we are in the process a little better. Right now itâs that last lap and we are rushing around quickly (but quick maybe a few months, who knows). This is a hard part to project.
Maybe a new testing section in the update, so we can show what we are testing and perhaps same with cleanup (which is vital for decent bug hunting).
23 Likes
(permissionless = as you donât need permission to add your node to the network, everyone can add a node to the network, thus there needs to be a mechanic to catch wrongdoers cause you canât assume everyone is going to be nice)
I think you would need a specialized crdt lib that adds checks to everything and has a failable merge function, so that you can catch wrongdoers when they try to game the system. I think thatâs not the case for GitHub - rust-crdt/rust-crdt: a collection of well-tested, serializable CRDTs for Rust cause it just assumes everyone is playing nice.
Also you need a point at which you can say that an operation was successful say when everyone in the responsible section saw it. But is it really that much faster than parsec? But as you said âmore correct workâ
3 Likes
Exactly, we are at somewhat cross purposes. We are saying the same thing, almost. But what I was meaning was that you cannot be part of the decision group until your node proves itself via node age. Itâs subtle but yes it is fundamentally permission-less, perhaps there is a better way to say/describe this?
Yes, itâs really down to latency. Not all about speed mind you. I think like this, ABFT and family wait on strong consensus by a quorum. CRDT does the same but allows causal order as opposed to agreed order. So this means regardless of when you see an event occur ABFT will tell you it was in another order (potentially). CRDT does similar but does not wait on one decision before the next decision, that is where you get latency improvements.
Apart from latency, there is I believe horses for courses and sometimes you need absolute order and pay that price, but sometimes you need a more naturally defined order and that can be cheaper. I mean cheaper in terms of latency but more expensive in terms o thinking required. I believe though that thinking is not wasted and should also be applied even when you have abft/bft/parsec. So force thinking in a more meaningful way of how you apply operations to mutate stuff. So I see a nice feature of CRDT also helps strong consistency, otherwise, you get consistency on things that might not make sense so much. Itâs hard for me to explain as the years have shown though 
12 Likes
Ahh, yes, ur right, ABFTs/PARSEC need some gossiping rounds before itâs comminting on an event order.
7 Likes
Wow, great update ⌠the network is becoming so advanced that, when itâs released, itâll look like magic! 
11 Likes
Formulas for associativity and commutativity seem to be switched.
But concretely what doesnât that mean ? What are a, b and c ?
3 Likes
So are you big fan of his wrestling? Or do you just like his look?
Yes your right I switched them around. I will edit that.
This is why folk find it hard. Itâs using such laws to create types that can satisfy these. Say a counter as the classic example. 1 + 1 is associative and commutative but not indepotent. so we are counting stuff here. So instead of using integers just use a set and the union is the + operator. The length of the set is the âcountâ. So 1 XorAddress plus another XorAddress is a set of 2 as long as they are unique.
So getting a count of elders in a group/adults in a section now can be made crdt like, so we donât care about replay attacks, order issues etc.
The you can create OR-Sets to handle remove nodes (Observed Remove set) which is just 2 sets where one is add and one is remove. You have a rule such as last write wins/Add wins/Remove wins etc. and build from there.
The trick is with data in this form then itâs guaranteed to merge cleanly, as long as all pre-requsits are satisfied.
That gives us a lot of scope. But more importantly it allows us to look at data types/containers/functions and ask does it follow these requirements (associative etc.).
Making them so does not harm BFT type things, but can make data manipulation much simpler.
concurrency is the big one, so a social network feed/likes in forums and so on. They all work very will with these types of data
So back to what are a b and c. Well really they are operations or advances in the state of something that can happen at different times and possibly repeated, but if each node has such types and has even seen slightly different advances in state then all the nodes containers will merge. This guarantee is important and part of the larger insight. The data you get back from decentralised networks is likely not final and many times it will be transient. You need to know it is valid, but you can never tell if it is current. So you get a âbest answer for nowâ and this is in fact the case with all of the methods, whether BFT or CRDT.
In this last part the method with the lower latency will give the most current (correct) answer, but it can feel like strict order actually would. It is counterintuative in many ways, but thatâs just an interesting point.
Here @tfa There are a bunch of good videos on this subject, but these 2 are pretty good
15 Likes
One paper I came across trying to understand after CRDTs were first mentioned here last week is this one http://nasirifard.de/pubs/fabricCRDT.pdf where they leveraged CRDTs to improve Hyperledger fabricâs latency and error handling. Others might find it helpful.
10 Likes