The essence of SAFE apps

Hi guys,

earlier today I was having a discussion with the front end guys in the office, and our little brainstorm gave me an idea that I think is worth sharing and getting your comments on. I’ll keep it short as I need to get back to work/sleep.

Obviously a SAFE app can be the conservative equivalent of a normal program; with your data securely stored on the network: ie storing your data under your MAID account and the application needs your permission to access your data.

We have Drive to reconstruct a (cross platform) file-system for your files, but at it’s heart the SAFE network does not have a filesystem. It only has recursive datamaps that point to hundreds, thousands of chunks that make up your data in a MAID account. Those chunks live in XOR space and the datamaps are just big indexes to help you find them again.

Private data only allows you to do so much, so public data (under MPID) and privately shared data (under MSID) are where a SAFE application really starts outperforming what a traditional application could never so easily do: a user does not need to make a ‘private copy’ of data in order to have it usable. In some sense the SAFE network combines all hard drives into a single giant virtual hard drive used by everyone simultaneously.

Take a simplified example of a contact address book. On your MAID account you do not need to store your contacts. rather you could (minimally) only store the public IDs of the contacts you want to have, and the actual contact information is retrieved from the MPID (or MSID) contact details. Your contact changes some details, and they are automatically reflected in all contactbooks that have him.

Of course, the simple contact book is not the interesting point, I just use it to illustrate my general point: what is a SAFE app? what is different from any type of application we had before ?

A SAFE app is a filter. A SAFE app is a piece of code that indexes / maps / relates different public, shared, or private pieces of information. From all the chunks that are collectively stored on the full XOR network, an app builds a specialised type of datamap that pulls together pieces of information to make sense out of them.

Importantly this pushes strongly for open source data types: different applications with similar or overlapping purposes should be able to collaborate. Afterall, they want to read public data from countless different owners and those owners might have written that public data with a different app.

Secondly, realising that this inherent structure of the SAFE network should be reflected in the way SAFE apps are designed, helps us all (community included) to prevent repeating our old mistakes: if we build the SAFE network and then populate it with apps that try to fence data in private MAID accounts, it’s both more expensive (every user has to pay more safecoin), and it would miss out one of the essential goals of MaidSafe. That is to make apps/services secondary to the data. We currently have an internet filled with services that try to claims as much of your data as possible. We should not repeat that mistake on the SAFE network !

So food for thought!

Cheers Ben

19 Likes

I agree.

Your data is still your data, regardless of which APP created it. Saving money (Safecoin) on storage is also a nice bonus.

This raises some interesting implications.

How much better performance will we really get out of storing the existing filetypes on the safe network? Is the real value added going to be in developing new file types, for the standard document, spreadsheet, presentation, video, audio files etc.

How much work would it take to create a space which allowed for secure realtime collaborative editing?

4 Likes

Simple example but I really want this. Thx for the good read, it’s eye opening.

Especially this.

3 Likes

This has been posted before on here but your first post brought it to mind again @BenMS . Good stuff sir.

2 Likes

In some ways it does come close to TBL vision of the semantic web. I personally think the future of the web will have a more fundamental restructuring - to be built on MaidSafe as rebuilt internet - and the semantic web can be a consequence of that restructuring.

4 Likes

Since I knew MaidSafe, I always dream that the SAFE network, may be the beginning of a global distributed operating system. A new [Plan 9] 1 in which all computing resources of the world could be exchanged by anyone. And where the pyramid scheme became collaborative systems all placed on the same level.

And I think that here, I am not alone with this dream.

8 Likes

I wonder if it would be possible to develop an app that only uses the users’ own data. That would be pretty neat. I as a developer would gladly let the users own ALL data, including meta data. Of course, one tricky thing would be to store for example database indexes. It would be unfair to let only a few users have the burden to store lots of extra app data that has to do with the application as a whole rather than the users’ own private data.

2 Likes

Even though this is a relatively simple example, I personally think that this (intentional or not) one of the best examples you could’ve chose and here is why (and I’ve made this point before)

We should use the profile of the user as the basis for an app, any app really. And depending on the app, different information can be accessed that which will be defined in my profile, not in the app and access is given to the app from within my profile.

So let’s start with a minium profile that has a unique ID on the network and has nothing else in it. You should be able to add all kinds of information to your profile:

Nicknames (as many as you want), every nick that is defnied will be used for a different app.

  • hillbicks is used for maidsafe and github
  • foobar is used for that forum about BMWs you like
  • fiftyshadesofgrey_lover is for that BDSM Forum :stuck_out_tongue:
  • and so on

Realname you only need that one once and give access to all the apps that should have it.
Messaging define the apps that should have access to it (e.g people that are able to message you) Depending on the app, a different nickname is used, when definied above, otherwise the ID.
telephone Same as messaging, give access to the apps that need it.
address Same as messaging, give access to the apps that need it.
location Same as messaging, give access to the apps that need it.

I think you get the idea.

Next you can define groups of people and give groups the same access to your information as you give it to apps, but these permissions override the permission apps get, because you might have one or two people from the BMW forum that can have your address, but not everyone on there in general.

Now, if you think of an app like Facebook. Everything you share, you can share with a specific group of people that is definied in your profile. So that dark joke you want to tell your friend won’t piss of your grandma or boss. The same goes for flickr, instagram or vine, you name it.

Last but not least, you should be able to define what app can share its data with another app, e.g. share your spotify playlist with your facebook group called friends, because your boss shouldn’t know that you like Death Metal.

I really think having the profile manage the permission and give access to all apps from there, can make all the difference. Facebook is not as popular with young people anymore because their parents are on it. You won’t share the latest party pic on instagram because your boss might see it. I think this would enhance the user experience immensely and underlines the core idea of Maidsafe. All the information that you share with the individual apps or groups are still yours and if you think that an app behaves in a malicious way, delete the permission to access your profile and all the information about you is gone from that app.

Again, this might’ve been a simple example but I really think it should be at the core of the plattform.

8 Likes

I agree, because when you think about it, the “linked” contact list is actually the basis for a social network. From there you only need to add features like wall, tweets, share, like, etc. But at a very core, a social network is just that, a list of contacts that can communicate with each other. Interesting stuff…

EDIT: Thinking more about it: the Safe social network will most likely not be a single app, but a collection of app that uses a common contact list. Each app would add a layer of data and filtering. So you and me will both be on the same social network but the actual feature of it will depends of the app we chose to use.

You can visualize it like a venn diagram. The contact list being the cross section and each app would be a specific section adding a layer of complexity over it.

EDIT2: Actually the whole Safe App ecosystem can be seen as one big map of overlapping venn diagrams where data connects everything together. Fascinating.

5 Likes

They get paid when they store, and they get paid when someone reads them. And they also get paid every time that data is updated. What is unfair?

Every app that works with private data (e.g. static blog) will use only user’s own data.
You mean drives (or vaults), I suppose. That isn’t possible and should not be possible.

1 Like

Hmm… That’s good. I didn’t think of that.

Yes, but I meant meta data such as database information for the app itself, not the users’ private data. But as you wrote, the users get payed for storing data! So that will be fair. One practical problem may be where to store large amounts of app meta data. The app data will probably have to be stored separately in the app’s own account.

That is one of several use case scenarios.
If you look at my comment above, that is why I mentioned a static blog. It is self contained.
If you have a dynamic app that needs a DB, then it depends on details:

  • in some cases you will be able to store data on SAFE, download both the DB and data (data partially or fully) and store changes back once you close the app.
  • in other cases it won’t be possible, so yes, you could not have data on the SAFE network

But for static and single user DB apps, it’s quite likely that it will be possible to download the DB and upload it back once you’re done. If your app crashes or drops Internet connection before it’s done uploading changed DB, you may lose updates or partially upload the DB or data and end up with corrupt data. Cool, now that I know that, I won’t run such apps that way until there’s a better way (e.g. save 2 or 3 versions, and delete the oldest version only when the newest is opened without and error, possibly keep transaction log locally, etc.)
I also won’t run it on other people’s hardware and when I’m traveling.

Several months after the launch the behavior of popular apps will be well understood and new versions will have smarter ways to deal with problems.

I think it will be possible for apps to store data on SAFE and access it directly, without having to first download it locally. Of course performance will be an issue, at least in the beginning. A single database write for example will likely be tediously slow, lol, but I believe it can be done and with performance that is acceptable.

One point I want to bring to the discussion is that on a decentralised network, an app will also be decentralised (at least the naturally compatible apps will be). Even an app that relies heavily on sharing information will have to do so by not relying on a central app-database maintained by the developers.

Of course, it is possible to set-up such a centralised app-database, but it’s not the natural way of building a SAFE app. Rather a SAFE app needs to be build such that is ‘nomadic’ (thanks @Ross for introducing the word today in a discussion) in XOR space. Regardless on which MAID account it is activated, it starts/continues weaving a web of overlapping links with other information it can access from that MAID users perspective.

For example, a social network might be fragmented and disconnected if all the current users fall into several disconnected groups. After all if these groups of people are disconnected in real life, why would their online-social network be connected (behind the scenes, as on the Facebook servers). Only when they explicitly reach out to connect would the private data of both users now reflect that they know the other and want to be connected.

6 Likes

I just cried a little, at how beautiful & exciting this is

3 Likes

If your app uses a mySQL DB, how exactly are you going to do that?

The SAFE network is basically a distributed file system, and MySQL uses ordinary files to store the data. So in theory it should be possible to run MySQL directly on SAFE, except in practice it will be horribly slow in the beginning. It will be better to use a NoSQL solution in the beginning I guess.

1 Like

Not only would it be horribly slow, but also horribly expensive, because every single transaction would cost one PUT.
So it won’t work.
The vast majority of popular apps use SQL DB and cannot use NOSQL, so until they’re rewritten (or like I said, downloaded), static apps and downloaded apps will comprise the great majority of personal apps.
Download-use-upload makes all new transactions get committed in one PUT, so it’s going to be the cheapest and fastest way to work with existing single-user apps.

@janitor :

Not only would it be horribly slow, but also horribly expensive, because every single transaction would cost one PUT.

I think you are making a wrong assumption here. In which case this statement is false:

So it won’t work

It may well be too slow for many applications (a CMS for example), but I believe @Anders is correct when he says:

I think it will be possible for apps to store data on SAFE and access it directly, without having to first download it locally.

This is because you can access files on SAFE Drive without the entire file needing to be downloaded.

So it will be interesting to see how a MySQL database stored on SAFE (as a file) and accessed using the SAFE Drive will perform. It may well be good enough for some tasks.

I agree that NoSQL is the way to go, but I’d also like to see someone do some tests on MySQL.