SAFE Network Dev Update - July 16, 2020

This sounds fantastic! :partying_face:

As you know I’ve also been looking at implementing a FUSE drive in Rust, based on Syncer which I now understanding well enough to pull to bits and adapt to the SAFE APIs.

One of the things which I noticed was that it would be much cleaner to implement directories, files, links, symlinks etc as nodes as Syncer does, so the FileTree approach could give us a much more conventional filesystem which would be excellent. The Syncer code seems well thought out and clear, and might well be a good starting point for your own Rust FUSE. I can share my notes if it would help you review the code.

I’d reached the stage where I was ready to have a go at a FUSE mount of the current SAFE FilesContainer, by using about half of the Syncer code, bypassing its disk caching and plugging it directly into a SAFE backend. However, I think I should put this on hold!

I had also started to look at how the SAFE API and core libraries work in order to implement caching for speed, and streaming read/write which are important for many applications and POSIX compliance.

I’ll keep poking around to improve my understanding, but am now wondering if there is much point in me continuing the work I had planned?

I’m happy to collaborate or to take on a sub-project, or equally might go back to some one of my other projects that I set aside when I stumbled on Syncer. I’m new to Rust, but do feel ready to have a go at something reasonably complex, having read an awful lot of Rust code in the last couple of weeks and made lots of notes to understand how it all works.

So @Maidsafe, please have a think and let me know what options seem sensible. I’m quit happy to step aside as I have lots of other stuff to do, but I’m also quite keen to try doing something in Rust while I’m up to speed. So I might have a go at something regardless.

30 Likes

This sounds huge! Also sounds like it would solve one of the problems I was pondering over on the search app thread.

Thanks again for all the hard work everyone. So many steady weeks and then it seems like everything happens all at once!

15 Likes

gogogo team!!

14 Likes

That’s what we always want. Perhaps @danda can get together for a quick brief? Thanks Mark.

20 Likes

Thanks so much to the entire Maidsafe team for all of your hard work. :racehorse:

15 Likes

:tada: indeed! Must have been somewhat nerve wrecking pregnancy with all the covid-19 situations etc. Is this your first one? If so, you are in for a wonderful journey! (My daughter is a bit over a year now.) BTW Netflix has good documentary series about the babys first year.

… and thanks for the team for all the work done! Incredible progress.

16 Likes

I thought you might like it. :slight_smile: me too.

Yes, a goal here is that each FileSystem object is represented uniquely all the way through the API and also in the data storage. This makes update and read operations much more efficient, and the resulting API more flexible.

Sure, if you can send me a pointer, that would be great. I’ve taken a brief look at syncer, as well as surveying various other fuse based systems, rust and beyond. Initially though, the focus will really be on getting the CRDT data type working.

Until we get the Tree CRDT type translated to rust and are able to test it out, this is all just theory. Even then, there could very well be some showstopper. So in that sense, you might want to continue your efforts against the present File API. However, if all goes as we hope it will, that API would be replaced. (Though a thin File API compat layer could possibly live on, not my call.)

That’s great. Ok, so here’s a couple things I could use some assistance with:

  1. Translate Isabelle/HOL formalism to Rust, or even to pseudo-code. There is also a Scala implementation that was machine generated from Isabelle/HOL but it is (a) inefficient and (b) extremely obfuscated, so I think not a good starting point. If you or anyone here can assist with this, it would be amazing.
  2. I’m still trying to decide if its better to use the Fuse low-level (inode based) or high-level (path based) API. Keeping in mind of course that we would like this to work on Windows also. Path based seems easier, but has limited support in rust/unix… ie only fuse-mt. I’d be curious to hear your thoughts on that, and how an inode u64 might be used/mapped if going that route.

In any event, the idea will be that we have a FileSystem API/crate that can be called directly by SAFE apps, and is also used/wrapped by FUSE daemons (for unix/mac/win). So we will need these wrapper daemons, and that may be something you can get started on…

Hopefully the above gives you some ideas. I will share some design docs as things flesh out a bit more.

edit: @happybeing if you’d like to create a new thread for this, we can continue discussions there.

16 Likes

Sounds good @danda, I’m distracted atm but will get back to you. Very exciting :smile:

13 Likes

Brilliant update.
Brilliant test network.

You can clearly see that we put blockchian tech to shame.

In zero time… tonight I

  1. Installed the safenet-cli and connected to the network, didn’t need to sync 100GB blockchain.
  2. I was able to create an account, and fill it with test coins.
  3. I was able to send test coins to another person instantly.
  4. I was able to upload a file to the network, immediately.
  5. I was able to retrieve a file from the safe network, immediately.
  6. I was able to create a website, and link it to the file container containing my file, immediately.

Block chain, bitcoin… feel like stone age technology compared to the experience I had tonight.

:rofl::rofl::rofl::rofl::rofl::rofl::rofl::rofl:

I just want to state for the record books,

Craig ‘Faketoshi’ Wright did not create the SAFENetwork or SAFECoin.

Bitcoin is about to become irrelevant.

36 Likes

9 posts were split to a new topic: What do you think about using a SAFE female sticker?

This is super exciting news!!! Broadcasting the news here: https://twitter.com/sotros25/status/1283914731410149376?s=21

More grease to your elbow, MaidSafe Team!! Also, congrats to Adam! :smile::blush:

9 Likes

Finally, someone who speaks English. Thanks for the explanation. I am so excited to understand what this means

11 Likes

Hello everyone, here you can find a German translation of this weeks dev update. Enjoy

12 Likes

Yes Zoki did well there… it’s easy to forget the level at which what is pitched… nice and clear and relatable. Just transfer of money to me instantly, is so important :grin:

9 Likes

If ordering of parts of the tree is time stamp dependent, how do the devs propose to resolve this on a decentralised system?

2 Likes

Time in crdt talk is not a timestamp it’s confusing as you see clocks being used, but they mean causally ordered. I wish they never used the term clock. We don’t do timestamps and I am really delighted to see this in the crdt world as it’s been a big fight I always have when folk use wall/clock time or try to, it does not exist and has infinite bugs :wink:

13 Likes

They use a pseudo time such as a Lamport timestamp. It’s not the actual time that’s important, but the ability for different participants to arrive at the same eventual ordering whatever the order they receive and apply the edits in.

If you want a great intro to this watch the video linked in the OP. it’s really well explained.

5 Likes

@danda I’ve watched Martin Kleppmann’s video on CRDTs (excellent - not just the tree move stuff, the whole thing is very good) and scanned the paper (Isabelle/HOL looks hard!) and the corresponding repo.

I’ll get back to you on the specifics in you post, but for now have made a dev forum topic (below) which includes my notes from the video. I’ve started wondering how this can work with SAFE so if you have stuff to share on options please do. I’d also like to understand how SAFE is using CRDTs currently, as I think this will help - are there any design docs around this I wonder?! Reading the code is a hard way to learn as the libs create many layers of indirection which I’ve found quite difficult to follow, so any SAFE CRDT implementation docs would be useful to see.

EDIT: do you use Riot (now Element) chat? If so we could set up a dev room for this. Is your id @danda there?

9 Likes

Thank you for the heavy work team MaidSafe! Together we are going to the future of humanity!
:thankyou:

I add the translation into Bulgarian in the first post :dragon:

6 Likes

I miss a summary in this update. As always, great work, thank you guys!

2 Likes