Abusive scenario about de-duplication

@dirvine

locally you don’t need to put more than once but on the network you will

and

or storing your own copy there is no charge

What does this mean? What else would you store but your own copy?

1 Like

Better to think like this, For any data you store from outside SAFE then you pay. Any data you store/copy from within SAFE is free.

3 Likes

Is that because you are just copying the datamap rather than a physical copying by copying contents

What does this mean? That you can’t GET individual chunks?

Yes, otherwise (and we did implement this to try), the ClientManagers need to keep copies of names you have stored to check payment (huge state and security issues as well).

This way though it’s very good as it maintains the no search allowed rules. It is again the thing not yet used enough, there is never a reason to search on the network, so if somebody asks for something not there, then they are trying to hack and will be disconnected.

4 Likes

Only if they exist, so you cannot try to Get chunks that may exist, they must exist.

3 Likes

Thanks for clearing that up.

As you can see these threads are useful, they clear up things for everyone.

7 Likes

No problem. I have never made enough of this facet, but it’s very powerful. Imagine trying to guess login details when with every wrong guess you are disconnected, etc. There is more to it, but the weird network behaviour is that at the data level in routing you always know the data exists, it must. So any type of search at that level is banned.

It’s unrelated to searches in upper layers (for indexing data we are fine, then search indexes). So does not mean we cannot have search engines, just they will search over known data. I hope that makes sense. I realise I am not explaining well here, but in the middle of crust/routing API issues so will come back no problem and go into it more deeply. It is it’s own thread I think though and very useful.

4 Likes

So if I have a file (from outside SAFE) that I know is likely already on SAFE (publicly), I have the choice between finding it’s datamap on SAFE (via a search index for example), or paying SafeCoin for “uploading” it? Seems a bit odd from the user perspective, but from a security point it makes sense.

It does make sense, the explanation is fine!

2 Likes

if i understand correctly, it is not odd
i think it’s like hitchhike, when you jump into someone’s car, arrive at destination, pay him a beer, and then wonder why the heck did you give him the beer, because he had the same destination as you, and he was already traveling to the destination by himself. but the answer is that YOU needed the ride, that’s why you paid. you pay because you need the datamap

1 Like

I like it :smile:

I agree these discussions add to my understanding too.

2 Likes

But, if you use two accounts? One ask for the chunk and if disconnected the other know the chunk don’t exist. An alternative client can make that very easily and you never pay for data already uploaded.

Yes I agree, but it’s a lot of work, a security issue, it makes things very difficult. So you could create a client, do a get, disconnect etc. So a hassle, then all you would achieve in the end is not paying for something already paid for anyway. Doing this for each file / chunk you want to try and store will be really annoying. If trying to say locate a possible user account then there is a space of 2^512 to cover, it’s a lot of disconnections for sure. So there is a gap, but it’s one of those, if somebody wants to try that they will upload at factors slower than if you paid, then to have to pay you need a registered account etc. so doable but …

I should add we don’t need to tell you that you are disconnected, the network can keep you connected and bin all requests etc. So a lot can be done, one idea is to always give you an encrypted “rick-roll” :slight_smile: There is a lot of things that can be done here, but at least we know what is happening.

4 Likes

Does this count for SD’s as well? If so, it wouldn’t be possible to use forward-referencing SD chains, where the next SD is posted with as identifier the hash of the previous SD. When trying to GET the entire chain one would get disconnected after trying to GET the non-existing SD after the last one.

That would be quite a pity, since we need forward referencing if we want to create truly decentralized linked-lists for decentralised discussion threads and such.

At the moment it will only be immutable data, SD is a slightly different mechanism and needs to be able to be free of this rule for the time being.

1 Like

Thanks for the fast reply! So eventually it will be forbidden to GET non-existing SD’s?

1 Like

I feel it will be settable whether these are allowed to be searched as some for sure will resist search (by returning dummy / random data every attempt). Things like login or session packets will likely follow this patter. So range based search protection.

@Seneca, SD’s can be updated, so link lists are easy without worrying about referencing non-existent ones. When you add one, simply update the null forward link on the previous ones to point to it and set null as pointer to next.

Can’t do that if the previous one isn’t yours. :wink:

Ah, I see

Could blogs and the like use a multi owned SD then the blog APP can do the linking yet the poster still owns the SD as well. But of course that is app specific.

The other choice is that the APP “buys one SD in advance”, then the poster fills the previously bought one on the end of the chain ( or the poster’s post is copied into it to keep things atomic ) and the APP then buys another to link as the last in the chain. And it is then waiting to be filled later on.

The bought in advance SD can have its ownership transferred to the poster.

BUT of course if the posters own the SDs and they form a link list then what happens if the owner of one of those SDs decides to re purpose the SD and thus the chain is broken.

Maybe you would be better off with a system that uses SDs to point to the content and link those SDs which remain under the ownership of the APP. Each SD holds many pointers (and its hash to detect modification of content in the linked to content/SD) And link these pointer SDs together to form you link list of sorts.

Did that make sense?