SAFE Network Dev Update - July 16, 2020

[danda]

@happybeing

This sounds fantastic!

I thought you might like it. me too.

One of the things which I noticed was that it would be much cleaner to implement directories, files, links, symlinks etc as nodes as Syncer does, so the FileTree approach could give us a much more conventional filesystem which would be excellent.

Yes, a goal here is that each FileSystem object is represented uniquely all the way through the API and also in the data storage. This makes update and read operations much more efficient, and the resulting API more flexible.

The Syncer code seems well thought out and clear, and might well be a good starting point for your own Rust FUSE. I can share my notes if it would help you review the code.

Sure, if you can send me a pointer, that would be great. I’ve taken a brief look at syncer, as well as surveying various other fuse based systems, rust and beyond. Initially though, the focus will really be on getting the CRDT data type working.

I’d reached the stage where I was ready to have a go at a FUSE mount of the current SAFE FilesContainer, by using about half of the Syncer code, bypassing its disk caching and plugging it directly into a SAFE backend. However, I think I should put this on hold!

I had also started to look at how the SAFE API and core libraries work in order to implement caching for speed, and streaming read/write which are important for many applications and POSIX compliance.

I’ll keep poking around to improve my understanding, but am now wondering if there is much point in me continuing the work I had planned?

Until we get the Tree CRDT type translated to rust and are able to test it out, this is all just theory. Even then, there could very well be some showstopper. So in that sense, you might want to continue your efforts against the present File API. However, if all goes as we hope it will, that API would be replaced. (Though a thin File API compat layer could possibly live on, not my call.)

I’m happy to collaborate or to take on a sub-project, or equally might go back to some one of my other projects that I set aside when I stumbled on Syncer. I’m new to Rust, but do feel ready to have a go at something reasonably complex, having read an awful lot of Rust code in the last couple of weeks and made lots of notes to understand how it all works.

That’s great. Ok, so here’s a couple things I could use some assistance with:

1. Translate Isabelle/HOL formalism to Rust, or even to pseudo-code. There is also a Scala implementation that was machine generated from Isabelle/HOL but it is (a) inefficient and (b) extremely obfuscated, so I think not a good starting point. If you or anyone here can assist with this, it would be amazing.
2. I’m still trying to decide if its better to use the Fuse low-level (inode based) or high-level (path based) API. Keeping in mind of course that we would like this to work on Windows also. Path based seems easier, but has limited support in rust/unix… ie only fuse-mt. I’d be curious to hear your thoughts on that, and how an inode u64 might be used/mapped if going that route.

In any event, the idea will be that we have a FileSystem API/crate that can be called directly by SAFE apps, and is also used/wrapped by FUSE daemons (for unix/mac/win). So we will need these wrapper daemons, and that may be something you can get started on…

So @Maidsafe, please have a think and let me know what options seem sensible. I’m quit happy to step aside as I have lots of other stuff to do, but I’m also quite keen to try doing something in Rust while I’m up to speed. So I might have a go at something regardless.

Hopefully the above gives you some ideas. I will share some design docs as things flesh out a bit more.

edit: @happybeing if you’d like to create a new thread for this, we can continue discussions there.