SAFE Network Dev Update - May 28, 2020

AS stated parsec was removed from data almost immediately (after testing it on a network), it still does a small job in routing.

12 Likes

So its not a large change its just significant in that you are deprecating Parsec? but we are still on track?

3 Likes

I must admit I feel somewhat deflated by the Parsec announcement.
Perhaps it is because I don’t fully understand why after all the time and effort spent it is now no longer needed, what has changed to render it redundant.
I get that it is slow and a memory hog but it solved a problem, what will solve that problem now?
What exactly replaces it or is that still an open question?

19 Likes

Got my attention :smile:

Thanks team. Looking good.

I’m feeling a bit relieved that PARSEC is being removed. I think I’d picked up on something about this not really doing the job over successive tidbits without realising, and I get the impression that what has been done recently is cleaner and more streamlined. Fingers crossed that replacing it will be straightforward, but we’ll have to wait and see.

PARSEC is still an awesome piece of work and may find use cases, and remains a credit to all those who worked on it.

I often have to chuck out stuff I’ve put many hours into. This is how development works.

22 Likes

Thanks for all your hard work Maidsafe devs.

:clap: for removing Parsec if it doesn’t play well for now.

Can’t wait to play with the new CLI soon.

Just hang in there super ants… Cheers and keep hacking :stuck_out_tongue:

13 Likes

Good question I hope I can help. PARSEC was this order everything and was worked on extensively. It’s focus for us was pretty much on membership in routing and all other “stuff” it ordered was opaque (unknown to it). Most of that stuff was data and we had to remove parsec from data handling as we could not store files on the testnet. We “fixed” that issue by removing parsec from data.

Then the question is was it required, the answer is, sometimes order is required, but not always and not on every data type. So mostly no order required to say store a bit of data, but to mutate it, then we need some order or similar. This is what CRDT data types does, that is it’s function. So the opaque stuff can look after itself without parsec, in fact it needs to not have parsec as we seen.

Then membership, that is easier to solve actually and we already had solutions for membership without parsec (remember alpha II). The issue we had back then was the team had section merge in mind. This is where section merge back to a larger address space. This was a big issue I had to take up with the team, the network cannot merge backwards, it can only grow forwards. If it merges backwards then you have a more tricky situation to handle (non-linear changes). As we know if it did merge back we have data loss and that is almost guaranteed.

So no merge means relaxed requirements on order of membership events (they become much more crdt aware and always moving forward). So as alphaII grew the network still can with no parsec. However we are formalising that a little bit and if necessary have a crdt for that as well (we paid a CRDT expert to POC that).

Hope that makes sense now @Josh but shout if not.

tl;dr we are looking to depricate something we don’t use in vaults and don’t need in routing.

23 Likes

While I can understand the initial visceral reaction to Parsec being deprecated, I don’t think this is such a massive issue because nothing is being lost (i.e. parsec was doing more than was necessary in an inefficient way).

We have to be able to “kill our babies”, or otherwise put, course correct when a better solution comes along. Perfection is an illusion; once you think you’ve obtained it, you’ll always find a way to make it better. This is why I also disagree that it was bad to publicize parsec. We should be publicizing achievements. You don’t wait until you graduate from university to get a grade; you receive marks along the way.

All in all, I think this is good and honest update. Plus it sounds like testnets are on the way :smile:

26 Likes

We have not changed track in the last few months, this is the track we have been on for a long time now, since we released the first testnet with parsec in it (which we had to re-release without parsec in data handling).

10 Likes

It sounds like the team has been aware that parsec was no longer going to be used, but maybe it wasn’t at all outside the team? The passing comment that it is being removed probably seemed fine, but to outsiders, it sounds like a big deal.

It is a shame it hasn’t solved the problem as hoped, but it sounds like alpha 2 and the recent test nets are proving it isn’t needed now. It sounds like the replacement will be a better solution too. A good result in the end!

17 Likes

Wind is back in my sails :sailboat: thanks @dirvine

Moving on to being excited about vaults from home :tada:

12 Likes

As I understand, as possible as we remove PARSEC(Protocol using consensus to deal ordered data) from routing, the whole performance of network goes up because this reduces the overhead of processing ordered data by asynchronously and It can be used on very small devices by reducing problems such as memory leaks caused by heavy and complicated software.

Is it right direct to understand?

6 Likes

100%. This is also why there were Fleming debates. If you recall Fleming was headed to be a routing only network mostly. So focus on membership changes. That would have been a disaster IMO and I argued for full data handling, even safecoin. I felt that, otherwise we would be misleading folk and mostly ourselves. It did cause a lot of internal debate and strife, but this all shows why Fleming has to be much more than a view of a bit of the network with no function. It has to be a point where we can say “all this works” and the network function is proven now. Lets add upgrades, some clean up and efficiency improvements and call that Maxwell and then just get to beta. No more side tracks, full steam ahead with all functionality, even when it’s brittle.

Cool, @lionel.faber and @ravinderjangra have been running these tests and are currently making vault code much more modular and getting data types sorted with the help of @bochaco and @Joshuef who is also circling the transfers/balances/at2 crate with @oetyng. @adam firing through routing with @qi_ma and @yogesh bridging dkg with routing and also vault logic to. Everyone is doing amazing stuff and looking at testing in a much larger way which without parsec we can do as we can run loads of vaults on single machines with loads of clients to test it and so on. @StephenC has started a charge on that one.

A busy team all around with @JimCollinson killing figma with over 35000 layers and everyone else working hard.

There’s a lot of reasons to be comfy here, we will all understand all the code. No super genius required and no complexity that is not needed. So secure and efficient is the goal.

41 Likes

I wonder there are opportunities for metrics, in testing at least, to understand the balance of adult size and number relative to the adults leaving over time, and the speed the network can replace copies of data.

Priority to certain tasks I wonder has been mentioned before but maybe that is polish for later on.

:+1:

5 Likes

I can’t disagree with most of that statement (you are spot on) except the end. Damage control why? Because PARSEC is faster and better than other existing consensus algos for strict ordering but the team strives for something faster and more efficient for the use case of everyday internet usage?

I can see how someone might think “oh man they made it and they won’t even use it” but that’s not the case. It’s just as they test and see the limitations, and find different, less restrictive solutions, they adapt to what would better suit normal internet activity.

I would think it would still have some kind of use near data chains for like meta section info on Elders or something but idk.

Great progress and looking forward to the upcoming test net @maidsafe! Keep plugging away

Edit: I missed most of the conversation above so my reply probably doesn’t provide much value at this point, lol. Time to scroll up and read.

17 Likes

We should make PARSEC coin and promote the awesome company that created the tech.

8 Likes

Thanks for another great update. Starting to become a bit clearer where the thinking is heading…

I guess one of the things that was nice about PARSEC was that for the majority of the world that didn’t understand it (including me,) it was still fairly clear what it did, and why that role was key - it was the thing that established a consensus and therefore source of truth in the network (at least I think it was!) That’s easy to explain to anyone really, and particularly strikes a chord for anyone who has even a passing familiarity with blockchain type projects. It feels like you know what you’re putting your trust in, even if you don’t really.

However, with the more piecemeal approach, if I was explaining the project to someone, how would I answer things like:

How do I know someone can’t steal my money?
How do I know someone can’t steal or tamper with my data?
How do I know someone can’t print money?
Could someone do those things if they had enough vaults? How many vaults would that be?

Once upon a time I could sort of hazard an answer to those questions, but these days I really wouldn’t know where to begin.

Not questioning the change in approach at all, but just wondering if there is a way of quickly and clearly getting across a sense of how the network secures itself and establishes trust, without having to go too deep into the technicalities.

Hope that makes sense…

10 Likes

THIS. This is the tldr of the last few months of updates. Very exciting!

11 Likes

Makes sense to me, at the time Parsec was meant to be a big deal. As with code and progress goes it is okay to be wrong, it happens and a pivot to CRDT hopefully proves worthwhile in the POCs and productionalizing it. Common problem when you have teams working in isolation and not getting a full vision picture. One of those things where it helps to spend a few extra weeks debating with the full team and seeing the full picture. Is CRDT a new thing in general or were we so focused on parsec at the time as a new cool maidsafe impl that it was just never considered until internally it was realized parsec needs way more work and complexity to be fast enough?

My one ask is we stay humble and no proclamations of full-blown success even if tempting until we are right at a real finish line of a customer facing deliverable. Its exciting to hit a milestone but lets count the chickens up post hatch to see where we are and how healthy the lot is as a whole :stuck_out_tongue: .

15 Likes

This (I.e. additional/appropriate use cases for Parsec) could be something for people (I.e. not the MaidSafe team b/c they’ve got their hands full at the moment) to investigate. Parsec is great work, but it appears to be a jackhammer where a scalpel will do. Maybe others can think of suitable use cases for this “jackhammer”, or when SAFE is said and done, MaidSafe might work on something else that requires it.

9 Likes

More than 50% of the Elders in the section, potentially in the network. There is always a limit to security like this, the numbers are important. Node age means the Elders are aged and proven over time, so not 50% of nodes but 50% of the top 7 nodes in a section. With or without parsec that is the case, however with crdt/at2 we can possibly make this number 50% of all section elders (i.e. you need to take over more than 50% of all sections).

14 Likes