SAFE Network Dev Update - August 20, 2020

[Nigel]

@dirvine

Back-burner for now

Understandable. Hope it stays in the queue though

[Nigel]

@maidsafe

We will also have a push in-house to document the message API that exists in Vaults. This represents the most fundamental API and so using only industry recognised formats and types in this area will allow folks to build vault clients in any language, or indeed from any device capable of sending and receiving messages.

I might be reading this wrong but it makes me think mobile vaults… like say a vault written in SWIFT (iOS) or Android. I imagine this makes the possibilities almost endless barring the device can pass resource proofs. Please correct me if I am wrong.

[neo]

@Nigel

Android. I imagine this makes the possibilities almost endless barring the device can pass resource proofs. Please correct me if I am wrong.

Think of all the firesticks and similar that could have a 256GB SD card inserted and be a 24/7 vault. Get them to earn their way more than just streaming

[drehb]

@Nigel

makes me think mobile vaults

Interestingly it seems that you can actually use rust libraries in Android apps.

[Toivo]

Great update again!

@maidsafe

So with these (rather sweeping) changes, we have been able to reduce the SCL codebase by something like 18,000 lines of code in the last few weeks.

To put things in perspective for us no-coders, how many lines there are left in the SCL codebase, roughly speaking?

Will PARSEC be completely removed, or will it still be used for some operations? Are there situations, where you need strict order of events etc?

Also, how do you see the situation now in relation to OOM issues with previous testnets? Too early to say?

[JoeSmithJr]

@dirvine

Now the data elements themselves contain all of the keys, signatures and permissions. So it’s much better and much simpler to grok. You look at the data, see who can do what (permissions check) then look at the operation supplied (logic check, i.e. is it crdt correct) and then apply the op .

Seeing how the permission system is under restructuring, let me suggest: How about not storing permissions at all? Store just the owner (who has full access) and switch to capability-based authorization, for example macaroons: when a request comes in, verify if it’s accompanied by a certificate that inherits the appropriate rights from the owner.

This would also future proof the permission system, as it could be extended at will without having to worry about incompatibilities with existing data, and so on.

As an additional benefit, it would give access things impossible with ACLs but easy with capabilities, such as secure delegation of permissions to apps.

[joshuef]

@Toivo

To put things in perspective for us no-coders, how many lines there are left in the SCL codebase, roughly speaking?

In the SCL/src folder (actual code), we now have ~7800 lines of code.

[joshuef]

Wee side note: This isn’t a permission overhaul per se. The data has always for a while now had a permission matrix of public keys to do validation for a request (write/read/modify perms). We’ve just remove another layer that involved specific network requests and functionality that in the end could have been circumvented with some tailored node code anyway.

So with the new code base, final arbiter is the data itself.

---

@JoeSmithJr we were approaching this with the previous token ideas.

@JoeSmithJr

Store just the owner (who has full access)

I don’t think “one owner” will be possible.

Right now, for revocation of a token, you’d have to be updating the data / changing ownership (to invalidate the token). And I don’t think there’s a way around this…

And in that sense, just providing a keypair to an app allows us to act in the same fashion (the keypair would have to be present on the data , and can be removed from the data to revoke access).

But at the base level I think we’d need this setup (though happy to hear other ideas on this!)

Bearer tokens atop that is gravy, I think. And definitely something we’ll be picking up again and looking into for finer grained permissions etc.

[Toivo]

@joshuef

@Toivo

To put things in perspective for us no-coders, how many lines there are left in the SCL codebase, roughly speaking?

In the SCL/src folder (actual code), we now have ~7800 lines of code.

Whoa! I was thinking like “OK, maybe 18000 lines is like 10% reduction… that’s significant!”

Sounds like refactoring really pays off.

[dirvine]

@Toivo

Will PARSEC be completely removed,

Yes

@Toivo

Are there situations, where you need strict order of events etc?

Not that we have found. The casual order seems to cover this very well. However, in CRDT land, there is a lot of effort in selecting a “timestamp” (read version or whatever, never wall clock time) to prunde log history. This is an agreed point in history that is common across all replicas. This is a much better approach, but as ever we are at the leading edge here and working with some great ideas, a good thing for us is we have a great test pool for ideas. All in all total strict agreed order is not required for anything in life, in my opinion, so we don’t need it.

@Toivo

Also, how do you see the situation now in relation to OOM issues with previous testnets? Too early to say?

Too early, parsec still there, but we are doing real end-to-end (e2e) testing now in scl, vaults and now routing. We should find those errors earlier now.

[JoeSmithJr]

@joshuef

Right now, for revocation of a token, you’d have to be updating the data / changing ownership (to invalidate the token). And I don’t think there’s a way around this…

There could be a caveat like “as long as this piece of other data is set as x” (or similar) and then changing that piece of data could invalidate that token and any token descending from it.

@joshuef

And in that sense, just providing a keypair to an app allows us to act in the same fashion (the keypair would have to be present on the data , and can be removed from the data to revoke access).

With bearer tokens with caveats, one owner (public key) stored on the object is enough though. Full access is granted on all requests (inlcuding change of owner key) when signed by this key. All other tokens are just (potentially restricted) descendants from there.

[Toivo]

Thanks a lot, I really appreciate you taking time to give this general overview.

[joshuef]

but if you issue a token, and then want to revoke it, you’re down to having to change the owner key, or?

“as long as this piece of other data is set as x”

Or doing this… which I’m not sure what the difference is to just removing a key from that data?

[JoeSmithJr]

@joshuef

if you issue a token, and then want to revoke it, you’re down to having to change the owner key, or?

I issue a token, add a caveat “as long as xyz is set as 1” and then I change xyz to 0 when I want to revoke the token.

@joshuef

Or doing this… which I’m not sure what the difference is to just removing a key?

It’s just showing that revocation can be easily done within the specific authorization framework (bearer tokens with caveats) that also has other benefits.

[dirvine]

@JoeSmithJr

I issue a token, add a caveat “as long as xyz is set as 1” and then I change xyz to 0 when I want to revoke the token.

Are you seeing the token in this case as a data item on the network? If not the issue perhaps will be “when” does 1 → 0 i.e. what event links this to the permission (causal order again)

[joshuef]

Okay understood .

Yep, I’m with you on the benefits of tokens in general

[JoeSmithJr]

@dirvine

Are you seeing the token in this case as a data item on the network?

It’s just a small blob of data attached to the request, so it doesn’t matter where it’s coming from.

As for the “xyz” part, it would be something on the network, yes. Some code would run on the nodes checking for validity. It doesn’t need to be hard coded right from the get go: if only the owner is stored with the data and authorization is done by validating bearer tokens, it becomes extensible, for example plugin based (and if a token is not understood by any of the plugins installed, it’s rejected).

@joshuef

Yep, I’m with you on the benefits of tokens in general

The “xyz == 1” thing would, for example, allow multiple tokens to be depend on a single piece of data: one token for read access, another for write access, both revoked changing a single value. That’s just the minimal example for how it’s more efficient than storing the keys directly.

As for complexity of implementation: at first, it’s enough to implement a “restrict to read-only” caveat since the “change the key” way of invalidation applies to bearer tokens as well.

[dirvine]

Yes, interesting. We need to consider though the semi-lattice/crdt area here. The work in our data types and permissions is ongoing, but the most difficult part is sync the when permissions change wrt to the current version/order etc. of the data. So such tokens would need to follow the same patters AFAIK.

i.e. The revocation or change of permissions needs to identify the point in the order the change happened. Think of it like this, you have a “make read only” token and it’s applied to data just before or just after I try to write.

If we keep in mind that change permission operation can be applied at any time even a year from now, or immediately. i.e. the crdt pattern means we can work offline and when back from the African jungle after a few years, we can apply the operations we did offline and they must work.

I am adding this to make sure we don’t try adding the extra wee bit the turns into a huge issue. I hope the background helps (timers/time and all that does not help at all here, it’s all no order or causal order at most and we need to work in that arena).

[JoeSmithJr]

@dirvine

Think of it like this, you have a “make read only” token

There’s no such token though. “Read-only” is no longer the property of the data but the property of the token.

Data has one property: owner (there are no permissions attached to the data). So, access is granted not by verifying if the data has the right permissions attached to it but by verifying the request’s claim that it has authorization descending from the owner of the data.

@dirvine

i.e. The revocation or change of permissions needs to identify the point in the order the change happened.

You’d still have to verify if a token was invalidated just before though. Or maybe not: CRDTs aren’t about giving the “right” decision but about giving an eventually globally consistent decision regardless of timing and conflict. In other words, it’s fine if a write is applied a millisecond after the permission was “supposed” to be revoked as long as, in the end, everybody converges on that same state and acknowledge the write.

[dirvine]

@JoeSmithJr

There’s no such token though. “Read-only” is no longer the property of the data but the property of the token.

This is what I mean though, it has to sync in a semi-lattice manner. i.e. to allow

@JoeSmithJr

(You’d still have to verify if a token was invalidated just before though. At the same time, CRDTs aren’t about giving the “right” decision but about giving an eventually globally consistent decision regardless of timing and conflict.)

to happen.

This part of the design would be the most easily overlooked, but the most critical part of any design actually. When it’s part of the data it is still very difficult to do, but when it’s separated out then I am not sure there are any potential solutions, yet.

I am not knocking the idea, but hopefully prompting with a “Oh great, an unsolved issue” as this is where Engineering magic happens.