SAFE Network Dev Update - June 4, 2020

Thx for the update Maidsafe devs.

Bring it on :baby:

In the meanwhile I’ll try out these new toys v0.13.0…

I can’t wait for the day when you super ants can take a break from coding up the network… release it in the wild and the world attacks/enjoys it to make it stronger/better/faster… :wink:

13 Likes

Oh goody something new to try to break with my new shiny hammer.
I had been intending to keep it for a later release but I can’t resist.

Well done to everyone for all their hard work as always.

11 Likes

Great suggestion, and I might implement it, though there are tradeoffs to consider.

Consider that many/most file get operations that a windows user performs will probably not have any symlinks at all, at least when dealing with their own files. It would typically occur when accessing someone else’s public files. So if we do an “are symlinks permitted” check and warn before the GET then we are warning/prompting about something that might not even be a problem and may just confuse the user. If we perform the GET first, then we have a choice: loop through all the files first, before doing anything else to check for presence of symlinks, or just process each one as usual and warn/prompt on first occurrence. The former introduces additional performance overhead. The latter is OK in theory, but the progress indicator doesn’t really play well with prompts, which create visual glitches. (Actually, I notice it has problems on windows in general). Plus, we have non-interactive (not a TTY) mode to consider.

All of that to say that I’m still considering what is the cleanest/simplest/best approach.

It’s really quite frustrating that Windows technically supports symlinks since ages ago but effectively turns them off by default. Hopefully that may change in the future.

14 Likes

Kill your darlings, is what they say when writing. It means cut out all the Unnecessary eloquence, to leave just pure narrative. This makes for a much better story. MAIDSAFE, willing to kill their darlings, proves their professional integrity of the network.

17 Likes

Such an important point needs repeating… the robust grows naturally out of challenge. The opposite error is all to common - small evils for the greater good. Cutting corners and short-cutting does not provide success in the long term. I do wish more people understood how important it is to get the basics right… (apologies) so much of politics :face_vomiting: is exactly this problem. Takes society a long while to understand the basics and projects like this push hard find better solutions for it.

15 Likes

Could this be the play thing?

Great work once again!

8 Likes

Hi, nice update but I am bit confused TBH.

I am not really aware of all the technical details but in this blog post PARSEC looks to be the most viable solution for the Safe Network and the other alternatives are described as “centralised”, “limited”, etc. But now PARSEC is given up. How this will impact the network in term of decentralisation, security, privacy and scalability?

Vlad Zamfir was not totally wrong at the end concerning the hype.

1 Like

Zamfir’s criticisms have nothing to do with Maidsafe’s use or not of Parsec.

Parsec is a, more o less, “classic” Byzantine consensus algorithm, with its own characteristics. The problem for Safe is that this kind of consensus are, computationally, very expensive mainly because it gets a total order that is not necessary for the network.

The solution, in which devs work, is to use CRDT (data types that do not need total order), consensusless transfer with AT2 and verification through BLS signatures.

The basic idea behind everything is less computational cost, because much of the work will be done by the client instead of the network, more network capacity, faster and more TIPS (transactions of information per second).

12 Likes

“Zamfir’s criticisms have nothing to do with Maidsafe’s use or not of Parsec.” I know but he was right by saying it was overhyped. You are yourself describing PARSEC which WAS suppose to be revolutionary as a “more o less, “classic” Byzantine consensus algorithm”.

But why it’s a “more o less, “classic” Byzantine consensus algorithm” today but 2 years ago it was marketed as a new revolution.

And I just want to know how this will impact the network in term of decentralisation, security, privacy and scalability?

Trying to do my part by looking online but I feel like it PARSEC was an important component of the network and I don’t really understand how it can be replaced without repercussions.

The basic idea behind everything is less computational cost, because much of the work will be done by the client instead of the network, more network capacity, faster and more TIPS (transactions of information per second).

So there are absolutely no downsides? I mean I am reading this: https://techcrunch.com/2018/06/02/not-just-another-decentralized-web-whitepaper/ and I see

MaidSafe’s network is not blockchain-based. It’s engineered to function with asynchronous voting of nodes, rather than synchronous voting, which should avoid the bottleneck problems associated with blockchain. But it’s still decentralized. So it needs a consensus mechanism to enable operations and transactions to be carried out autonomously and robustly. That’s where Parsec is intended to slot in.

What Parsec does is it can reach consensus even with malicious nodes. And up to a third of the nodes being malicious is what the maths proofs suggest

And so many other things that PARSEC was suppose to “improve” or “do better” than Blockchain or DAG based techs. I am really confused.

2 Likes

Friend,

PARSEC is better than the others “classic” Byzantine consensus algorithms” but is not better for SAFE.

The problem, as I understand it, is that the theory was put before the practice. And this is a problem only if you believe that without Parsec the same conclusion could have been reached that CRDT and AT2 and BLS could be used instead…

So once again, the important thing is that the “mistake” Parsec showed us the way forward :dragon:

9 Likes

So what are we “losing” in theory without PARSEC? I know there are some benefits to move away from it (described earlier). That’s a legit question, I am not trying to troll or what…

3 Likes

tl;dr We can now very confidently do it faster with less.

Part of the confusion may be the sheer speed at which this field evolves. 2 years ago PARSEC was “revolutionary” - now it’s classic

A crude analogy may be that 40 years ago banks started to use mainframes which was revolutionary, now we use (unsafely) classic mobile banking on our phones.
Technology just moved on. 40 yrs ago we needed room sized machines, now we hold them in our hands. We found better (no sniggering at the back) ways of doing things with improved technology, thats all.

Hopefully someone will come along soon with a cleaner and more precise analogy.

But I can totally see how the speed of the development in the field can confuse.

10 Likes

We lose a lot of info about the exact order in which some operations were carried out. For the purposes of the checks we are doing at this stage we no longer need to know the exact order in which they happened, only the fact that they did happen (and can be proven to have happened)

6 Likes

Thanks really, I start to get it now :slight_smile:

So you mean that between 2018 and now, more suitable solutions have been “discovered” and PARSEC is not the best fit anymore for the Safe Network?

We lose a lot of info about the exact order in which some operations were carried out.

Ok I see, so I can not proof I did A before B, but I can proof I did A?

3 Likes

Yes.

Also had we not climbed the PARSEC hill, we would have been unlikely to find these other faster more efficient ways of doing only the bare minimum to achieve an acceptable result

4 Likes

Yes, I agree. If the event ordering is the only downside then the decision to not use PARSEC is understandable.

2 Likes

David claims his hero is Richard Feynmann but he has also taught the team to think like Colin Chapman, the famous race car designer

also


though my own interpretation of that would be " the whole race plus the victory lap"

7 Likes

It’s more subtle, you can prove A before B when B relies on A, this is th causal order bit. However we don’t care if you do A or B first when they are not related. PARSEC and DAG based consensus (blockchian too) relate everything to everything else
So really these total order things (which I inherently dislike) tend towards coupling everything in one large decide order of everything system.
To me that’s not decentralised or as truly decentralised as order only what’s required, but do so in a provably secure manner and now CRDT is provably consistent so easier to not understand, but explain, if that makes sense. Previously folk seen eventual consistency to mean possibly never consistency and that is so wrong. I see more eyes opening in this direction now.

Our guys are all over this and we should get back to being well in front of the game here, as we could have been a few years back. PARSEC looked great, sounded great and all the rest, a bit like, let’s use time. It suits the less deep thinking and offers the world, but it does so with a nasty sting in the tail. That is the world does not work like that, only our tiny brains cannot see it at times. We need to let go control sometimes and marvel at the sophistication from seemingly complex systems of iterations, just like an ant colony, neural network, or any species interaction with themselves and their environment in nature. There is no overall god algorithm there is only simplicity, randomness and chance.

With CRDT/at2/data chains, we are allowing this to happen and only forcing order where it’s absolutely necessary. As the decentralised field evolves we will see more of this and as CRDT types advance (still early days) we will also see significant self healing increases in knowledge (data) in a truly async manner.

31 Likes

Thanks for taking time to reply. Yes I understand now, I am following the project from “far” since 2015 and I come back in this forum time to time to get some updates. Maybe that’s why I was more confused about this decision but now it makes sense.

I think it is honorable to have taken this decision since PARSEC was lot of work for the team.

15 Likes

Maybe we should have built an ecosystem in which this could have evolved! :smile:

8 Likes