Safe Network Dev Update - February 18, 2021

Here’s a link to the tweet supporting this week’s dev update, so you can like, retweet, and comment!

17 Likes

I agree.

I came across this crdt gem the other day: Data Laced with History: Causal Trees & Operational CRDTs

CRDTs pointed to a paradigm shift in the network’s role for document collaboration

This means that all the computation can be pushed to the edges of the network, transforming the tangle of devices, protocols, and connections at the center of it all into a dumb, hot-swappable “transport cloud”. As long as a document finds its way from device A to device B, sync will succeed. In this world, data has primacy and everything else has to orbit around it!

Could this be the chance to finally break free from the shackles of cloud computing? It always felt like such an affront that our data had to snake through a tangle of corporate servers in order to reach the devices right next to us. We used to happily share files across applications and even operating systems, and now everything was funneled through these monolithic black boxes. What happened? How did we let computing become so darn undemocratic ? It had gotten so bad that we actually expected our content and workflows to regularly vanish as companies folded or got themselves acquired! Our digital assets—some our most valuable property—were entirely under management of outside, disinterested parties.

21 Likes

How many full-time devs are working on the project nowadays?

4 Likes

The team is 14 in all at the moment with 1 Exec admin, but even that position is super important. Sharon saves us all loads of time and has our back. Jim on design, Stephen all over the place keeping the ship together.

23 Likes

Yes, it’s a big step in the right direction for sure. To me off line first, then add in things like threshold_sigs/encrypt and we have some powerful tools. If we add BRB (Byzantine Reliable Broadcast - like AT2) when we require co-ordinated order we have a decent set of tools. For CRDT so far, Seq, Map, Tree etc. give us a decent start to make it easy for devs.

What we are looking to do is restrict CRDT’s to metadata only (pointers and descriptors) to protect them from holding actual data. That may change but for now it makes sense. So all data will be self encrypted (we need to force that better). This keeps the CRDT’s and state small as well.

[EDIT co-ordinated order meaning prevent doublespend. So some data types we can deterministically resolve conflicts (forks), but currency etc. we have to avoid them and BRB allows that]

15 Likes

I will try out the updated sn_api today and see if I can get a local network running. Will try to connect the Telegram: Contact @SafeNetworkWallet_bot again for demo purposes of the Safe Network CLI.

14 Likes

Some high level areas to be worked on:

  • Complete initial sn_fs/brb integration.
  • integration of BRB into safe network.
  • Make brb_membership to brb link 1:n instead of 1:1. ie support multiple brb datatypes per voting group.
  • Storing file content as XorData, in a CRDT compatible way.
  • ability to work offline, then come online and sync to/from network.
  • Sharing/Policies/ACLs
  • separate sn_fs and fuse, to make a filesystem API usable on any OS (without mount)
  • mount capability on windows, macos, etc. beyond linux.

Also, somewhere in there, we may need to move to using file-system level ops, rather than crdt_tree ops. Ie, we would send “mkdir” across the network rather than equivalent opmove(s).

17 Likes

4ace6n

9 Likes

Something that should be up in the OP but I forgot when noting down bits for the update :man_facepalming: :

@bzee has been firing in PRs getting sn_nodejs up to speed with the latest API changes, which is needed for the browser to be udpated. As well as taking a look into the limitations of the current lib, and seeing opportunities to improve things!.

Thanks @bzee! :tada: and apologies for missing your contributions in the OP.

(@maidsafe can we get that added upstairs please :bowing_man: )

24 Likes

David. This must be the first time ever in the many years i have followed the project you have given a timeline. Now i believe we are close. Dont tickle me pls :joy:

Thank you :pray: :relaxed:

7 Likes

We all do now, we are using the correct design now, it’s a lot simpler and much more robust. The only niggle/worry will be write speed of data, but we will see how that goes. Write speed has never really worried me too much though, it can be slow.

25 Likes

Write speed sounds like something that can be optimised in time. If that is the sort of state we are at, where we are concerned about performance of writes, it sounds like we’re in a very good place indeed! :grinning:

Compared to blockchain write speeds, it will be stratospheric anyway!

17 Likes

Actually write speeds for a mostly read storage device can be worked around very efficiently. They do it now with rotating media by having a large cache between the interface and the media. It is common enough to have a sizeable cache on the drive and the operating system to have one or more GB being cached in otherwise unused main memory.

These work so well that there is little noticeable delay for mostly read media. One of my drives with a large cache will noticeably slow down its write speed (like 1/2 speed) after about 2 to 8GB and that is due to the cache.

13 Likes

I just tested the sn_cli again with the new sn_authd and sn_node and they work a lot better. Previously I’ve been stuck with no success at uploading file to my local test-network but now it works!! Only thing I couldn’t get to work is to see my local test-network site on the Safe Browser.

The prior means we can hook up a Telegram SafeNetwork Demo again! Once the shared-section is online it will be even better.

16 Likes

Thx for all your hard work Maidsafe devs.

Maybe nodes staking SAFE and paying it up as penalty could also be an option.

Always code to see code reduction :clap: :clap: :clap:

@bzee @mav @scorch :vulcan_salute:

Hopefully the Brave browser will be able to easily integrate the SAFE Network like IPFS.

Keep hacking super ants :crazy_face:

10 Likes

Yeah, seems to me fetch speed is the key. Here, Safe really will be competing with what most people consider a “speedy” clearnet. If most pages don’t load almost instantaneously on Safe we are in trouble. This needs to be true from the beginning imo.

4 Likes

Simple caching should mean that most commonly used pages (most popular sites, listings etc) should come down really fast. There will always be a latency effect, but if sites less often used are tolerably snappy and video is fast enough to stream, then that will probably be good enough. Video was streamable from the last test net, so read speeds hopefully shouldn’t be too much of a problem, but latency for casual ‘surfing’ may take a bit of work.

3 Likes

Is this his first mention on the forum? I ve been wondering when that will happen because it was very much out in the open on his github for ages.

Nice that he can now be openly mentioned though!

8 Likes

It should indeed. We have removed that cache for now though (just for simplicity). Plus we want to see speed without it, to begin with.

12 Likes

Something I’m running into right now made me think of this. Does this imply that rewards is working (at least nominally) and that it’ll need to be refactored once the new messaging is implemented? Or more like rewards doesn’t work fully quite yet, but we can’t make it work until messaging has been refactored?

Maybe this belongs in its own thread, but it’s also possible that somebody knows off the top of their head. I’m currently working on a pet project to embed a remote procedure call interface into the node, and one of the features that was requested was resetting the reward key on the fly.

To accomplish that, basically I set the NodeInfo.reward_key to the new key locally, then call Node::process_while_any(RegisterWallet(new_key)). But, it seems that when I try to process the operation for RegisterWallet after setting the new key, the other network nodes are rejecting the change. Logs are claiming “invalid operation”. Seems the other nodes in the network believe the RewardsState of the node I’m trying to update is stuck in AwaitingRelocation (in the rewards module).

Am I barking up the wrong tree in my approach to setting the key, or is this just because this part of the rewards flow needs to be ironed out by upstream? Just trying to figure out how much I should bang my head against the keyboard to try and make it work :laughing:

8 Likes