Deletable Data, and Secure uses for Structured Data

If you put up data and remove your own datamap, it’s as good as deleted from your perspective. It means your tie with that data is gone, unless you keep versions (which is an issue for actually deleting data). So it’s not straightforward as it seems. If the data is actually truly encrypted then there should be no difference between the existence of a chunk or not. Otherwise were all in trouble :slight_smile:

When we say not deletable we don’t mean per user, that’s cool if you remove your reference to it, we only mean network does not delete it. Like if you copy a file then delete it, it may exist in the ether somewhere, you cannot tell, but you have no reference to it any more. We should perhaps make that more clear actually in the wiki, perhaps?

[EDIT] This does mean though even if you delete your reference the network does not refund you, if it did that then it would have to keep a tie to what you have and that’s not good for security. So the network charges for Put and does not refund a delete, from a users perspective. So yes, you pay once and it’s a forever stored chunk, but the flip side is no refunds for data that cannot be tied to you. Note deletable data will be tied to you (so the network can recognise you own it), there is a risk of exposure there, unless we also do an obfuscation step as well, so again more complexity, that’s why I hesitate to rush into that for now.

12 Likes

Now I am confused. Previously, I thought that whatever data (encrypted chunks) that are PUT on the network stay on the network forever. If you delete the datamap for those chunks, they still stay on the network, but they have “no owner”. Now, when you talked about deletable data above, I thought you meant that we actually would be able to delete the chunks from the network. But you just meant that we will be able to delete the reference to the data. Is this correct?

Exactly. Later there will be deletable chunks, but there is a lot to consider, especially security and privacy with that.

If data can be traced to you then it can be deleted (we are working on a method where it cannot be traced to you, but it is very complex and not for right now), if that makes sense. So we don’t want to do that.

There is a mechanism now where you can create throw away id’s to publish data anonymously, so great, but it cannot be deleted. Public data cannot be deleted, You will in the future have private data you can delete. I think of data as a thought, if you share it then it cannot be unshared, or forgotten, at least by force. So the network works like that.

11 Likes

Because right now the data is traced to the identity that originated this data, you can make many accounts and store data, and reduce the surface of targeting a specific account or behavior, via a new alias when putting data each time. Then data is linked to the account that put it.

If even the account that put it could be detached from the control of the datamap, then it is totally untraceable.

Yet data shared publicly, where the keys are given to public, so the datamap is known to all. Even if you delete it’s reference from yourself, the originator. Others still have access to the knowledge of that datamap. So it can still be accessed by others.

3 Likes

Does this mean Structured Data is not available for some time yet, or are you talking about something else (that can be deleted)?

Yes structured Data is there and delivered in this sprint (we are tidying up like mad). It’s very limited so you could not store data there really (you can edit/delete these though). So data (content) is immutable data (chunks) and these have no owner and should not be traceable to any id.

1 Like

Perhaps we should say data-metadata and data-content to be more clear?

2 Likes

You mean limited to 100K of content, or in some other way?

Yes, but it will be charged as though it was 1Mb so very expensive. It’s also a way to link data to you, so we need to discourage folk trying to store all content there as it would be dangerous. Encrypted metadata will be fine though.

chunks are the real content, it really makes a lot of sense then.

6 Likes

BTW, is SD how you do datamaps?

Or is there some other reason you want it available early (e.g. to help with messaging or something)?

No, well yes, sort of. It allows us to implement a mechanism for that, but in a very secure way not tied to id’s.

It allows app devs to create many ways and systems of linking data of all kinds (soon computations). This means for instance we could have a rdf type, owl type, etc. semantic data structures. We can also create a SD type for particular languages or groups of devices to communicate information (like medical instruments) an much more.

I wish I was up the top of the stack to show some of this stuff, but still down in the depths in implementation code in the core. Really SD will progress a little Spandan has a great rfc to make it collision proof. I hope to progress it to a chain-able type, then at that stage any computation or contract etc. is possible. The whole network can become extremely powerful and people can try some amazing things.

So SD is a really massive step in allowing many degrees of freedom to app devs, not just store/share/message but create complex data types and link these in many ways to create a huge brain if you like. I will really try to get time to write some of what I feel can be done with it all as soon as I get a minute. It is very exciting though. For instance safecoin farming will be a SD chain, but needs coded into vaults to enforce the chain, I want that limitation to vanish and others be able to create chains of logic/linked data etc.

6 Likes

Thanks David, these responses are very helpful to me trying to get a sense of what to use when and how, because much of it remains pretty mysterious even though I’ve been following as best I can for a long while now. I’m still not sure if I understood your earlier suggestions for SD types for blogs, comments etc. I think you were just indicating the ability to create linked semantic data structures, but there was some question in my mind that you were suggesting more than that - ways to enumerate for example, that might make up for the lack of server side DB type indexing - but in the end I concluded probably just my wishful thinking! :slight_smile:

1 Like

I do :wink: I will try and explain.

Create an SD type for your blog comment (say 999)
Create a blog (say type 900)
For your blog name xxxx you will find it at hash(900+xxxxx)
Comments go to `hash(999 + xxxxx + 1) and so on (from last comment)

I will get into more details soon, the guys will likely code an example to let folks see how to do this.
There will be improvements via linking (when we get there) to allow append-able data owned by the blog owner, to be able to remove comments) and more. For now this is a baby step to fully enumerating and indexing data.

6 Likes

Ok, so I wasn’t imagining dream code, you are (of course) building the dream mother of all noBackend networks. Time to buy more Safecoin :wink:

This means that SAFEpress and similar projects really need to stay away from the backend stuff until we can see how this part of the network works. It sounds like it will be well worth it. I’ll have to have a think about what we can work on in light of this!

It is very exciting, thanks again for elaborating.

5 Likes

This is bitter sweet. I feel my app project is at a standstill until the backend stuff is finalized, but the flip side is things should go fast once it is.

2 Likes

The initial SD is there in this sprint, so you can do what I said now (or when we give the installers). The next sprint (a short one) will stabilise this and probably give more examples. We should be able to do that for sure over the next few weeks.

6 Likes

A rather obvious point but from an end user perspective it’d be good to have the delete button available for private data, without having to nessecarily understand things like data maps. Could make people feel uneasy otherwise, even if there is perhaps no reason to.

There is :slight_smile: Also when we put out drive (it’s a FUSE type filesystem then it respects POSIX as far as the OS’s allow). So users will not know if they are on a local drive or a SAFE drive :slight_smile: Should be no difference for users or apps. So users will get delete etc. they wont see refunds, but they probably wont even be looking for them, it’s us inquisitive folks that do though :smiley:

4 Likes

Double pass encryption. The chunk is encrypted 2 times and saved the same way it is currently now. But maybe an extra field to the data map for decryption. So when one want to delete his file, the owner encrypt the data one time instead and send it to the data manager with the other hash/key to encrypt the second pass. Now the Data Manager verify the result with the chunk that is stored. If it’s equal then it can be deleted.

Data is basically obliterated and scattered across the globe. If the “key” to reassemble is deleted I think would be honest to say it has been securely deleted.

@happybeing How does this compare to current ways that are used to securely delete data? (Wipe disk, overwrite etc.)