Appendable Data discussion

appendable-data
immutable-data
mutable-data

#201

Interesting point to be made with how to deal with trying to keep public posts/pics/etc that are meant to be ‘private’ between two people by using accountability, which is applicable in some scenarios to building apps with the Appendable Data structure. The Snapchat app used to disable snapshots on the device till people were bypassing it with jail broken devices, now apparently they just tell you which user(s) took a screenshot which is honestly a good way of keeping people accountable (maybe still a hack?). Everyone opts in and if something leaks you know who to point your finger at which puts people in control.

I agree some warnings or education on the part of the app would be extremely beneficial. I personally like wizard like setups or tutorials that inform you of app functionality (or implications here) on first use.

I also think that searching through the appended data will end up being super easy like a time machine of data. Maybe via an app or the browser. You go back in one appended data’s timeline individually or have all other visible posts roll back to where you are in one particular appended data timeline for full context. Do it by WebID with any extra details like who owned it at the time etc.

I think there’s a good chance those who aren’t posting public anonymously and have nasty things to say might learn quick that they should speak their minds anonymously or be held accountable. Maybe folk will end up with split personalities at the end of it all :joy:


#202

Been out of the disccussion for a couple of days and a lot has happened here. I started replying to some points but as I scrolled down and down some of it was answered and some was just made moot. There was a comment by David that struck me though:

Doesn’t it though? Knowing this about the forum I’m now editing in a separate text editor and that definitely raises the bar for replies. You can of course just dismiss me as being paranoid … and maybe I am … regardless of my decisions in particular here, there is an assumption (a generalization) being made about people and behaviours.

IMO this is a common problem with humanity - we often tend to assume that other people think and act as we do … sometimes this appears to be true for some things in particular, but as a general principle the assumption is a judgement without evidence and hence IMO a bad judgement.

In this instance, we are at least talking about self-censorship. There are ways around the problem in this instance and perhaps there will be on the live network too … but let’s not make another assumption to fix the problem of the previous one.

The philosophical problems being raised and discussed herein are important, but will not be solved by us … we can only make pragmatic judgements and move forward (or not move forward at all).

This is why I personally support the network having a diversity of ways to acheive things - even though it may appear inefficient on the surface. Pragmatically, one method will most likely be implemeted before the other in any case … so I’m really just arguing in favor of the development of all possible ways that can be reasonably implemented.

One reason the existing Internet has been so popular is the lack of constraints - it’s openness - the freedom to build whatever you want (even a Safe Network). That diversity of possibilities opened the floodgates to where we are today.

Okay, all that said I’d just like to restate What I was hoping for when this thread started – to see some clear pro’s and con’s develop for the data-type methods which are competing for the developers time and energy. I think @neo has been attempting this, however for me the whole of the discussion is rather disparate and I have a difficult time connecting the dots - perhaps I’m in the same boat as @happybeing:

Cheers


#203

Perhaps because the dots are also moving, even if small amounts :slight_smile:


#204

It’s just that everyone fully aware is so very far from the reality. We cannot assume that while making some easy to use apps.

Edit:

For example a friend of mine was working with mentally handicapped people and told me an interesting example: One guy is not able to understand numbers at all. Still he is somehow capable of using smartphone. His staircase has code lock, so he has shot a video of my friend pushing the right code, and he can imitate that after watching the video, and get in. How clever!

Now, being mentally handicapped he is also prone to do all kinds of mistakes all the time, for example sharing that video somewhere. It is good it can be deleted from Facebook for example. Ok, it is somewhere in their databases, and maybe someone has made a note of that code. But it is still better to be able to limit the publicity.

I think this is one key point: even if public vs. private is at some fundamental level 1/0 type of question, in practice it is not. In everyday situations the question is more about the ease of access, than public / private as a sharp distinction. And that’s why the ablity to delete public data is valid, even though it does not absolutely guarantee anything.

It is also good to remember that half of the population has less than average IQ. And those that have very low IQ do have some serious troubles understanding what they are actually doing in many situations, that seem trivial to people with higher IQ. I wouldn’t say SAFE is really safe for them.


#205

Well people are already paying for it when they make the change anyhow, so the question is rather moot


#206

I know getting to launch ASAP is important, and that it has already been suggested that mutability might come later, so I’m not trying to break in open doors.

I’ll elaborate some on the topic here, and I’m looking at it based on what I think humanity will need and adopt, rather than what technology would be great, because I think that is the root of the question, not the technology. The necessary technology will be possible to achieve when we are clear about what properties the system shall have.

The viewpoint I am taking here now does not necessarily reflect my ultimate opinion, but I will take on this hat now and plow through it because I find it useful to lay it out (side by side with other viewpoints). That doesn’t mean I make argumeents I don’t believe in, but I do not necessarily consider them to be the ultimate truth (as well as being simplified and extrapolated to make a point).

What I will argue for is this:

Data that has not been uploaded to the network as public, should always have the possibility to be mutated. Only the public data should be immutable.


Appendable data is not something I oppose, on the contrary I think it is a very logical solution. Moreover, erasing data that [intentionally] has been made public is not possible to fulfil while maintaining the properties desired for the network.

Imagine you are a group of dissidents working together, you have access to very sensitive material that you upload, encrypted, to the network.
One of the members gets caught, and while the others are certain there will be torture and an eventual compromise of the keys, there is simply no way to re-encrypt the data, since the old data is always there.
That makes the network a pretty bad option, for a use case that was supposed to be among the most(?) important ones.

As I see it the argument put forth against mutability is (among others) this:

  • If any one else than the owner(s) of a key, has [at some point] gotten the key, the data is now supposed to be regarded as “public”, and for that reason it would be destabilizing to allow mutation on it.

I think this is an incomplete line of thought. If the group of dissidents is increased with a new member, also given access, has the data suddenly become “public”? Where is the limit on number of people and relationship to the original data/owner for it to become public? The reality is that there is no such clear definition.

If you write something in the sand, and let everyone see it, people are not going to be fooled to think that what you wrote in the sand is persistent in any way.
If we have a data type which is commonly known to not be persistent, to be sand, then no-one will be surprised when the waves erase it.
We don’t go around worrying that people should be writing important historical data or patents or book keeping on the beaches, do we? Why would we worry when we transfer that into the digital world?

Yes, maybe it makes it technically easier to develop now, solves problems with caching, performance, CRDT this and that, but I would say those are not important things. They are important, but not in the same ballpark of importance. They are implementation details.

Yes, this could be the only network where everything is permanent. But is that a higher goal than everything else? The purity of the manifested idea, over the actual utility? It seems like such an imbalanced notion.
I mean to say that the goal is not that all data [that we want secure] should be permanent. The goals are - among things - that there should be secure access for everyone, that you should be the sole owner of your data.

I think we have started to tilt into that the network is the owner of your data.

When your data is not only yours, then you do not have the “right” to modify it. BUT, when is it not only yours? That is not a question that can be solved by technology, so we only have a couple of blunt options:

  • Some seem to argue that it is the moment you upload it to the network.
  • I would say that the only rational thing, is that it is the moment someone else has access to it AND acts on that by copying the data. By copying, they MAKE IT theirs.

We don’t want the situation that the data is immediately not ours when we upload it to the network. If you cannot control it, it is not yours. A fundamental rule has been broken. It is a pseudo-ownership.

If you receive keys that decrypt some data on the network, you cannot be sure that the data is permanent, and if you truly consider yourself as an owner of the data (now that you have been given access to it), then you need to copy the data - in fact that is the action required to manifest your opinion that you are also an owner of the data. And the only way you can ensure, that the data is there for at least as long as you want, is to copy it and upload with your own encryption. The only way to ensure that the data is there forever, is to publish them unencrypted, or (maybe slightly less sure) to forget the private key, and publish the address + public key.

This moves the point of no return from uploading the data to when you share the [keys to the] data. This makes the definition of ownership a dynamic case by case process including the humans being exposed to the data, instead of a static decision taken by the designers of the network here and now (take a moment to reflect what that might mean in a bigger picture…). If you consider yourself an owner when you see it, then you need to act on it, and make it yours by copying. (It could even be built into apps that they immediately copy everything it reads, so you automatically take ownership of what has been shared with you - and… what ever of it you didn’t really want … you can delete afterwards, if you happen to care to. Less dedup you might say, but hey, is the goal to save disk space, or to be aligned with human needs?)
I.e. as long as only you have the keys (and you still haven’t been caught and tortured), you know that this data can be made inaccessible for ever. As soon as you share the key, you no longer know this for certain. We need to trust people to know that they won’t be scribbling their checks, their will, and the cure for cancer, in the sand on the beaches. And we need to be understand that in this life there is a need for death as well as for life. And some things must die. Or be deleted.

Why is it necessary that we cut the power of our own data into half, by forbidding us the right to manage it? Don’t look so much to the technology, look to the actual human need. The network with beautiful pure clean perfect logic, but not serving human needs, that is not a useful network - it is just … a perfect implementation of some useless idea (in the broader sense of 100% world wide adoption, not for specific use cases).

The actual goals, the actual use cases, what are they, and what do they require? Is it shown that everything permanent is an absolute requirement for them?

I do not believe for a second that the network cannot fulfill the goals when some well specified part of it is mutable. It might be harder to get it to work. But since when did we start take the easy way instead of the necessary way?


#207

Exactly. Public vs private is in reality very far from boolean logic. If I publish something on FB and remove it seconds later, I am 99.99% sure nobody noticed it and will ever notice it. Until I make some crime like murder nobody will ever be interested in reading my deleted posts hidden somewhere in FB db. If I publish something on my web and delete it minutes, even hours later, than there is 99.99%+ probability there is 0 copies of it. And if I delete it after few days, there is some probability it is somewhere in google search DB, and very low probability in some internet archive, but nobody will try to find it since he does not know it exists. But if I do something in my private bedroom, with no windows and no electronic devices, there is quite a big probability that someone from my family, or my neighbors is trying to monitor my activity. I feel much more private hidden in the crowd of infinite useless public data than in private apartment. For me, delete button is absolute must. Yea, dirvine and admins can check all my typos and edits, but others can’t;) And that is really valuable, since my English is poor:)


#208

I thought everyone could, I have no admin here, just a regular user?


#209

Lol, I will have to use another editor for this:)


#210

I don’t know if everyone can, but I can. It might depend on trust level, but it doesn’t require admin rights AFAIK.

A point to note is that edits shortly after you post are not recorded. So correcting minor typos in the initial post escape!


#211

Trying to make a clear distinction between the ideal (ignoring technical difficulties), priorities (like making sure the public/immutable/perpetual is working properly before looking at private/mutable data) etc. is helpful.

Also: on Discourse forums moderators can see more than normal users, like posts that are deleted.


#212

Yes, this is a multi layered branching.

What are the goals?
What is the definition and scope of them?
The definition and scope of the goals will be very important, as to get to a common understanding of what is then a realistic path (technically/idealistically), and what the actual options are.
And as there might not be a full consensus immediately on what the definition and scope are, the possible paths can be in a super-position (unless there is some higher level path that can allow the different paths).
Also, might be that some interpretations give that not all of the goals are 100% attainable together, there might be conflicts between them.

This nature of things get unearthed at points of discussion like this.


#213

I fully expect that if the previous versions of public data are kept in the network itself, there will be easy way to see them. This might create problems of someone accidentally mixing a previous version instead of current one and there may be only slight difference in these versions. Like webshop, where a product or price is a bit different. It is not the end of the world, but it creates hassle.


#214

The previous version of the data is currently stored in almost any database. In fact the tendency is to use, more and more, immutable systems such as Event Sourcing.

http://www.odbms.org/2015/10/the-rise-of-immutable-data-stores/


#215

This is true, in some sense, and I am for one almost only writing event sourced applications.
But I don’t think the tendency can be taken as proof that 100% immutability is desirable (not implying that’s necessarily what you meant).
You would never be able to re-encrypt anything if it was actually so. Well, you could, but it would be pointless, since old keys would still give anyone the old data.

And that is just one example of how immutability is not a silver bullet.

I might add that the event sourced applications are almost never using event sourcing only, but some other projection storage as well, for efficient querying. That storage indeed would be troublesome with immutability…

On the other hand, it is often possible to run in-memory projections. But the larger the streams, the beefier machines needed to process the streams in reasonable times, snapshotting not always applicable.
It would be limiting, but maybe an OK evolution of things.


#216

Isn’t this just the same as “If I publish something on SAFE and then update it with a new version seconds later, I’m 99.99% sure nobody noticed it and will ever notice it…”?

I am also one of those still concerned (or at least wondering and trying to be as objective as possible) about not being able to delete private data, but more from maybe a technical perspective (or ecological?) wondering WTH we want to waste resources to keep/hold something that I’m not interested in keeping it, and that nobody will have access to, ever, specially if I simply throw away the encryption keys (so here I’m assuming that’s good enough to prevent access to anyone, including myself, and even in a distant future there will be no way to break that encryption with some alien tech. Will I care if aliens share that tech after I die?).

On the other hand I know there is no way to be sure that something private and encrypted is effectively deleted from the network when I requested to, so I do get that, at most, the network could create an incentive for vaults to remove what has been flagged to be removable. Perhaps the incentive is that if they remove something flagged to be removed by a user, such vault has more space to store something that it’s effectively being used and more likely to receive a GET request for, but this will also need some different type of farming mechanism where more used data generates higher rate of earnings to farmers (which from my understanding it’s not the current plan for farming rates and lotery). So there are clearly technical challenges here, but it sounds like they can still be figured out. So from the perspective of technical challenges, I’d agree on saying that perhaps we don’t need that for v1 of SAFE, but just consider and/or decide if it can be upgraded so users can flag private data (i.e. encrypted and not shared) as “deletable” and figure out a way to incentivise vaults to be more “green” (I also wonder if such a flag could be bad as it may imply it’s likely more interesting data to try to break the encryption for?).


#217

I was thinking about same vulnerability, but with “hard reset button” of your autorization aplication you could lost access to your all data forever before torture. Or even better you will lose access, but somebody else will get new keys. Anyway I think, that 3 letters agenies etc. have some other options to get all information they want.


#218

Another option might be that for such a situation, no single person has the whole key. In that case they might still feel comfortable with uploading the data, even though they know they will never be able to avoid getting that price on their heads by that - which they will likely have even if they all try hard to forget their part of the key. And the data will always be susceptible to the rubber hose attack. With mutability, the rubber hose attack could be protected against by deleting, or an increased risk (by leaks) could be restored by re-encrypting.

So the need for “revoking” keys is not exactly circumvented by the partial keys, more like, post-poned.


#219

Hey, I really want to share these two poems from Polish Nobel prize winning poet Wislawa Szymborska. I think it is self evident why the first one, Discovery, but the second one On Death, Without Exaggeration, might need some justification. I feel that on some level it is against the first one, but most of all it is just a very beautiful poem that I cannot help but share.

I feel and hope that these poems help to open the scope of these questions in a new way.

Discovery

I believe in the great discovery.
I believe in the man who will make the discovery.
I believe in the fear of the man who will make the discovery.

I believe in his face going white,
His queasiness, his upper lip drenched in cold sweat.

I believe in the burning of his notes,
burning them into ashes,
burning them to the last scrap.

I believe in the scattering of numbers,
scattering them without regret.

I believe in the man’s haste,
in the precision of his movements,
in his free will.

I believe in the shattering of tablets,
the pouring out of liquids,
the extinguishing of rays.

I am convinced this will end well,
that it will not be too late,
that it will take place without witnesses.

I’m sure no one will find out what happened,
not the wife, not the wall,
not even the bird that might squeal in its song.

I believe in the refusal to take part.
I believe in the ruined career.
I believe in the wasted years of work.
I believe in the secret taken to the grave.

These words soar for me beyond all rules
without seeking support from actual examples.
My faith is strong, blind, and without foundation.
.
On Death, Without Exaggeration

It can’t take a joke,
find a star, make a bridge.
It knows nothing about weaving, mining, farming,
building ships, or baking cakes.
In our planning for tomorrow,
it has the final word,
which is always beside the point.

It can’t even get the things done
that are part of its trade:
dig a grave,
make a coffin,
clean up after itself.

Preoccupied with killing,
it does the job awkwardly,
without system or skill.
As though each of us were its first kill.

Oh, it has its triumphs,
but look at its countless defeats,
missed blows,
and repeat attempts!

Sometimes it isn’t strong enough
to swat a fly from the air.
Many are the caterpillars
that have outcrawled it.

All those bulbs, pods,
tentacles, fins, tracheae,
nuptial plumage, and winter fur
show that it has fallen behind
with its halfhearted work.

Ill will won’t help
and even our lending a hand with wars and coups d’etat
is so far not enough.

Hearts beat inside eggs.
Babies’ skeletons grow.
Seeds, hard at work, sprout their first tiny pair of leaves
and sometimes even tall trees fall away.

Whoever claims that it’s omnipotent
is himself living proof
that it’s not.

There’s no life
that couldn’t be immortal
if only for a moment.

Death
always arrives by that very moment too late.

In vain it tugs at the knob
of the invisible door.
As far as you’ve come
can’t be undone.


#220

Toivo, I really liked them, thanks for that.

And also a very interesting way of broadening a discussion.

The last one, the last lines made me chuckle.

Death
always arrives by that very moment too late.

In vain it tugs at the knob
of the invisible door.
As far as you’ve come
can’t be undone.

It’s like a defiant statement: “Oh death, are you here. Well you are too late, because I have already lived and let live. That can’t be undone, so do what you’ve come to do.”

The first one made me think of the nuclear bomb.