[RFC] Labelled Data, Indexing and Token Authorisation

If that’s the use case you want, then a consensus time or trusted time duration is exactly what you are asking for.

I should probably leave this to smarter guys than me, but just so I can understand :slight_smile:… do you really need a consensus on time for this? If I am the permission holder, and can withdraw those permissions at any time I like anyway, it is surely only my view of the duration that should matter? Two accounts do not need to be in sync, or start their clocks at the same time, or agree on what GMT is, for that, right?

1 Like

Based on my understanding…

Not if one leaves at least a single client device logged in. You can then be the time master.

If all your devices are logged out, you need the network to do this for you via some kind of time/duration consensus, ie parsec blocks.

1 Like

NTP still exists and that is typically correct :wink:

1 Like

Which would be the ClientHandler(s).

As long as enough think it’s still within the time period (and otherwise valid), the request would be fulfilled.

The reality is this sort of caveat won’t be offering precise accuracy. Similar to how revoking a token or app’s access isn’t guaranteed to be instantaneous.

ClientHandlers all need to get the relevant data to assess a token’s validity. Their local time could be one facet. Other network data (if the publicKey is still valid, eg,) may be required too.

This should be more than accurate enough for purpose.

2 Likes

Can you elaborate the limitations that prevent usage of standard macaroons?

2 Likes

The validation is built assuming the validator is the issuer. And so requires the secret key used to sign the macaroon. Which would mean any client handler would be able to mint their own macaroons. :frowning:

There’s a paper (away from comp for a few days so don’t have the link, though it’s from the forum…somewhere ) over paper proposing bls macaroons, but they themselves are more complex and don’t permit attenuation (and also there’s no standard impl).

3 Likes

I agree, macaroons cannot be used without this!

Just to be sure I understand correctly:

  • The issuer is the owner of the labels applied to data
  • He cannot validate macaroons, simply because he isn’t always online to validate the chain of caveats

But I am not sure about the latter point: For labels created for data sharing, would it be really unpractical to enable sharing only while the owning user is online?

In this case macaroons show a difficulty because the owner’s daemon would have to be polled via the address of his account in XOR space, but I think they could still be viable:

  • This would make sense security wise, because the chain of caveats is directly verified by the owner, without secret key delegation
  • This could be compared to a web site in clear net that becomes inaccessible when its authorization service is out
  • For labels applied on private data, offline owners wouldn’t create a breach of perpetual web
  • The rules are:
    • public data are always accessible
    • private data have restrictions to be accessible by other users: create labels (or other caveats) and stay online
  • The user himself doesn’t need to remain online, he only needs to have a daemon running somewhere.

A simple use case example with macaroons in these conditions:

  • I create a label L on some data
  • I create a macaroon. The location field is initialized with my account id (needed for communication to my client manager)
  • I create a first caveat to allow reading of L data
  • I don’t find this is restricted enough so I add a second caveat that only allows reading for 2 hours
  • For an even more restricted macaroon, I also create a third caveat that only allows reading by a specific account id
  • I send the final macaroon with the 3 caveats to the user owning this account
  • When this user wants to get one of these data, his client manager must pass this macaroon to my client manager which, in turn, will ask my running daemon to validate the macaroon
  • My client manager will then return the validity response to his client manager. Response will be invalid if 2 hours have elapsed, label of data isn’t L or account id isn’t the right one.

The daemon will be able to do validation using the PC time and a set of input data, in my example:

  • Requested data id (the set of applied label names is derived from it)
  • Requester id (for restriction to a specific user)

I would like your comments on this proposal.

Also, how do you intend to implement such a use case in your proposal? I must confess it isn’t clear to me and a practical example would be welcomed.

1 Like

Away just now, @tfa. I’ll try and back to you proper in the next days :+1:

7 Likes

There is no hurry. Enjoy your holidays.

7 Likes

The issuer most likely will be the Authenticator. For us, this likely means SNAPP, or the safe cli

The ideal is that they do not need to be online to validate the macaroons. But yes, if using the actual macaroon libs, the issuing authority will need to validate them and be communicable.

It’s possible to do. It may provide a bad user experience and is most definitely sub-optimal IMHO.

Something to remember here is that this isn’t only about sharing data, but all access to data. Your applications can be given permissions based on labels. Or other users.

Requiring the need the need to check in with an Authenticator a) creates more networks messages for each and every request and b) creates a bottleneck around that authenticator (to a point where we move back towards having the Authenticator as the gatekeeper of the network, which has previously proven to be problematic due to bottlenecks.


Macaroons are neat because they are tokens that can be attenuated. But they require centralized validation. Which is find for that use case as you’re, in the end, always asking returning the token to the same place.

For us, our token will be given to apps who will be talking directly to the network (to avoid the Authenticator bottleneck), which can check validity only if it has the secret key for macaroons.

For this reason we’re looking at asymmetric keypairs. With the Authenticator minting the token, with a SecretKey, to be stored for reuse. But which can be validated by the network using the PublicKey.

This gets us very close to decentralized macaroons. (And once we have fleshed out some attentuation ideas, there).


Good Q. I’ll try and flesh this out here, and update OP once this seems clear enough :+1:

Something we touched upon above, but which needs to be expanded on is the use of BLS Key-Shares. These allow each label to have a keypair, and a key-share to be delegated to apps for part of the validation against label PKs stores on data,

First, in general:

  • An app X wants to create label L and apply to some data.
  • App requests Authenticator for permission.
  • Label L doesn’t exist, so a keypair is created for it in the account.
  • KeyShares are provided to app X once permission is granted.
  • Auth permission is granted, within a timelimit and a Token is minted via a BLS key-pair sign. The token sez “He who bears this, can manage label L, for two hours from < now >”
  • Token is returned to app X
  • PublicKey is stored in the account AccessContainer for retrieval later.
  • SecretKey is stored by Authenticator

When X wants to access data, it must:

  • Make a GET request, and pass along the token.
  • ClientHandlers validates that the token matches the request type (ie has PUT permission if a PUT requests`
  • ClientHandlers validates that the token with a PublicKey which has been stored in the AccessContainer.
  • ClientHandlers validates that things are still within the specified timeframe
  • ClientHandlers agree the request is valid and send to DataHandler
  • DataHandler checks the token’s labels against those on the data. If a PublicKey stored in the data’s Permissions array validates the KeyShare stored in the token, the request is valid and the GET is done :+1:

Sharing data

The OP proposal has a flaw in that the suggested Shared Token only allows you to access data where the original owner minted the token. It does not give flexibility for delegating responsibility to another account eg.

As such, our fallback system which is used for label key management, is generating of BLS Key-Shares.

Eg: Each label will have a master key-pair, used to generate a key-share for each app, this key share can then provided to the data handler for validation against the permission’s PublicKey set. This is touched on ion the OP, but will need to be expanded.

So we can provide another user with a keyshare for our label, and that account can then create / manage it’s own tokens, which will provide key-shares to apps for signing requests.

ClientHandlers will treat these tokens as above. Just instead of the account/authenticator having access to the original KeyPair for a given label, it was provided by another account.


Hope that answers some of your Qs, @tfa ? Let me know if that raises more, thougH! Good to be thrashing this out, especially after a good few days away from the idea :slight_smile:

4 Likes

Been thinking on this some…

I think perhaps our CH validation step is one that is specific to an app caveat. Only with that caveat in a token, the CH validates against a stored PK as we have it here.

Without that, the CH does not validate, and this is considered a LabelToken (ie for one label only), which would be a token that is signed by a LabelKeyShare… This basically gets ‘passed through’ (so long as any other caveats are valid) to DH, where the DH will attempt to validate this LabelTokenwith stored PublicKeyShares.

If any KeyShare on the data can validate the token, then any other caveats to be validated at DH are validated, and request is returned…

This still gets us account/app revocation via PK checks etc. Still gets us Token validation at some stage.

Tampering with an app’s token (ie, remove app caveat): is passed through to data. As PK used to sign doesn’t exist here… request is invalid.

Apps cannot access Label secret keys, so forgery there should be impossible…

Not sure if I’m missing something here… thoughts/qs @lionel.faber @bochaco?

2 Likes

It’s still not clear to me but at my age mind is not sharp anymore.

Just few remarks though:

Not each request, only requests for private data not owned by user. For private data owned by current user (which I guess will be most of private data the user is accessing) the macaroon possessed by the application will be checked locally by the authenticator (running on the same PC).

The usage of macaroons I am proposing is a complete rewrite of the way that permissions are given to applications. The applications are not given keys anymore and data is owned directly by user. Instead, applications are given macaroons that allow them to read or write the user data and validation is done locally by authenticator.

Macaroons are more features rich. In the OP “macaroon-like” expression is mentioned but what is proposed seems more limited. With real macaroons any number of conditions could be validated:

  • standard ones implemented by the authenticator on date, requester id, data id, label name, …

  • dynamic ones implemented by the owner on any kind of data: balance of a wallet, data from a weather API, random function (why not!), … For these specific conditions, the authenticator would need to be forked, or a plugin system would need to be developed for anyone to be able to add his own conditions.

Also, macaroons are more secure, and I have never been at ease with app keys which seem too much power given to applications: How can we be sure that an app key is not stored outside the network by an application and then the application uses it when the user is not connected anymore? They are regular applications, not sandboxed, written in any language and can do anything they want.

This refers to a past implementation where the authenticator was proxying all network requests. To avoid this, we can imagine that the application remains the program directly interfaced to the network and authenticator only validates and signs the requests.

The workflow would be:

  • The application asks the authenticator to validate the request with the macaroon.

  • The authenticator checks the macaroon with its associated secret key and, if it is valid, signs the request with the account private key.

  • The application sends the signed request to the client manager, which will accept it and relay it to data manager (as usual)

Only in the case of shared private data the request with the macaroon will transit in the network. Suppose A is the owner and B is a user granted access to some data of A. When B wants to access some of these data the workflow is following:

  • Client manager of B sends the request and the macaroon to client manager of A
  • Client manager of A sends them to a daemon running on A PC (I suppose here that the daemon is not the authenticator itself but an application)
  • The daemon transmits them to the authenticator
  • The authenticator validates them and signs the request
  • The daemon sends the signed transaction to client manager of A
  • The signed request is sent directly to data manager from client manager of A but with a response to be sent to client manager of B.

Note: The first 2 steps could be also implemented at the application level by an out of band direct communication between application on B and daemon on A.

3 Likes

In your proposal for rewriting the auth setup to use macaroons, all requests would have to do this due to the need for the issuer to validate the macaroon.

Macaroons are feature rich in some sense, but limited in others (ie: the need for possessing the SK to validate the macaroon). The proposal covers the latter while not severely impacting the former. We can still implement caveats in a similar fashion to macaroons. Though I’m not sure how you imagine 3rd party validations to work within the context of the network?

Are you meaning that macaroons are more secure than giving apps keys?

If that is the case, then so is the proposal in the OP as the setup for apps would be the same (provide app with token to present with requests). Though I am not sure how giving a token, proper macaroon-lib using or that proposed above is any different to giving keys to an app. You still have no guarantees over its storage…

But this is still a bottleneck. Every request must have auth information. Which means every single request going first to the network, and then coming back to the running authenticator, from each client handler…

That authenticator’s connection / hardware is a bottleneck, and more network requests are made, ie: to/from the authenticator to confirm the validity of the token (which in itself provides another avenue of attack… can we trust that authenticator clock? should we?)

Or in your proposal

Here you have the authenticator upfront validating something before sending on the request to the network. Which is exactly the old setup, with the auth as a bottleneck/relay on requests from apps.


The OP attempts to get around this limit of macaroons with the BLS keypair setup, and provides a tangible method for doing this, while still enabling us to setup macaroon like ‘caveat’ and validation, as well as having scope for attenuation by apps and the like too.

2 Likes

The big difference is that the user must be connected to validate the macaroon. And with a proper HMI in the authenticator (like a scrolling box), he could even see if validation requests match commands issued in the application. With app keys a malicious application could act while the user is off line.

No, there is no such bottleneck. In my proposal the application asks the local authenticator to validate the macaroon and sign the request and then sends the signed request to the client managers, like current implementation (I suppose). There is no supplementary traffic on the network (in the general case).

@joshuef let me refresh my memory with some of these use cases and I will then comment.

@tfa, I don’t think the flow you propose is really what you’d want:

  • the bottlenecks you are introducing are due to each request would need to go through user A’s authenticator, and/or also user B’s authenticator if it’s shared data
  • plus you’ll need one or both users to be online to authorise each single PUT to the network, remember very likely each request would go to different sections to fetch/mutate different chunks of data
  • you not only need users to be online but also would be exhausting the user/s with auth allow/deny popups/confirmations
  • the complexities you are introducing with those requests going back and forth between authenticators, daemons, the network, and users, it really becomes unmanageable from my point of view
  • you seem to be only thinking of some use cases, how about IoT apps, you do want them to work when you are offline. Or how about some data intensive apps, you certainly don’t want to alow/deny every single request.

When you give an app permissions you are trusting the app, and letting the network to enforce those permissions each time it sends a request, regardless which form of token/keys you give the app to identify itself and its granted permissions. I don’t see myself checking every single request to see if the app is doing what I expect/allow it to do in terms of perms.

I think that even if you/someone would prefer such a flow, it can still be achieved by an app which was designed to be a gateway for all requests, but that shouldn’t be how the network manages permissions/tokens/macaroons. If there is such an app which acts as a proxy and gateway to verify each single request coming from an app, then that’s up to users deciding to use such app.

4 Likes

But there is a need for a local authenticator instance running. Which, as @bochaco notes, means many IOT implementations or other apps running on limited hardware may not work.

@bochaco, i don’t think in @tfa’s proposal there’d be a need to manually validate all requests w/ pop ups etc. But still, we’d be routing all traffic through one app. Which, locally may or may not be a bottleneck depending on your hardware / traffic load.


@tfa you mentioned above that the flow was still not clear, and I forgot to reply to it, apologies!

Can you be more specific about what parts are unclear in the flow in my earlier post? Would love to get us all on the same page here, as I think the OP gets us the macaroon behaviour you’re after while solving the SK validation problem.

2 Likes

ah ok, I’m sorry I misunderstood that, but then why would you route reqs trough such app if it’s not to be able to intervene, then such app is doing the same OP proposes the network (vault/client-handler) to do (but just enforcing the user to be online)?

For constrained environment where it isn’t possible to run an authenticator in parallel to validate macaroons and sign requests, there is still the possibility to give the application its own direct credentials to the network.

Well, this is possible in theory, but how easy is it in practice? For example, is it possible to develop an application using the ffi but without an authenticator? I mean, connected directly to the network with its own credentials but still benefiting from high level functions provided by ffi.

  • Absolute security when user is offline

  • Possibility to visualize the application usage of resources like rate of requests and just a big red button to stop everything immediately if something looks suddenly suspicious (with a simple caveat that controls the state of this button)

  • Programmatic control thanks to the richer set of caveats provided by macaroons. An unlimited set is possible because caveats can be programmed in an ad hoc way client side (I mean not programmed in core). I already mentioned several possibilities, but I would like to mention another one: the application could be provided a safekey with safecoins, the authenticator could refill it periodically, and a caveat could be defined that validates the rate at which the application consumes its balance.

3 Likes

I see what you are imagining now, sounds good, but I’d say that’s just an app to be created and separate from main/core logic, in fact such gateway app can just be authorised by the authenticator once, as any other app, and do all that with macaroon attenuations and so on.

3 Likes