Question about Storage on the SAFE Network

Hi Guys,

I just found out about SAFE last week. I’ve been doing tons of reading to understand it and tried the alpha network. I’m really impressed.

Anyway, I have a question regarding storage bloat. My understanding (maybe i’m completely wrong) is that users pay to buy storage with Safecoin. The storage bought and used is permanently stored on the network. This seems completely unsustainable and extremely wasteful (money, resources and environment). Wouldn’t the network storage quickly become bloated with files that are rarely and never used? I get the reward for being a farmer goes up when less than 30% of free storage is detected to encourage more storage, but I don’t think that would hold up if the SAFE Network became very popular. Shouldn’t the Safecoins pay fora combo of storage space and time to live?

Thanks!

1 Like

In the long run it’s not a problem due to maximal growth economics at scale. When I raised this question myself, I forgot about the value in extra “mud” sloshing around the network for improving data obfuscation. Even with this in mind, dirvine talked about automatic ways to identify dead data in the early days (look for the google tech talk presentation). There are other solutions to this in the form of archive nodes, temp data markers, and giving the user incentives to remove unwanted data. This topic has been talked about at length with regard to safecoin but it is applicable to data in general. Other threads in the forum have considered this from time to time. Keeping everything allows for a lot of simplicity, and a lot of benefits at the same time.

3 Likes

Can you explain “maximal growth economics at scale”? Sounds very “buzzwordy”. Is it saying that as the network grows, more storage will become available and therefore we shouldn’t worry about bloat? If true, that honestly doesn’t give me a warm and fuzzy feeling. Again, seems very inefficient.

The idea of receiving Safecoin for taking data off the network would alleviate some of the bloat, but it still seems inefficient to me. I can still see tons of wasteful storage being used because people forgot, abandoned the network, or even died.

Why is there such a strong opposition to paying for the storage in space and lifetime? Are we worried this will hamper the network’s growth and popularity? I feel the free market and supply and demand should make the price of Safecoin fair, so you shouldn’t have to worry about this.

Not buzzwords, just a sentence trying to compress as much information into as little words as possible.

To be inefficient there must be a definition of waste. Lost or unwanted data is not waste if it serves a double purpose, and keys that are lost by their owners might eventually be found. Granted, sometimes you can have too much of a good thing, so it’s about maintaining a low relative proportion of “junk DNA” to hot data at any instant. You can either have the network try to find ways to find truly “abandoned data” that could be removed, which is impossible to define, and would be nearly impossible to actually remove in any event without excessive complication due to deduplication; or just grow a little bit larger to absorb and minimize the effects of this rarely accessed cold data.

Following a pro growth regiment is much simpler and more robust from multiple perspectives. Safecoin farming economics make gratuitous uploads of worthless unique data expensive for an adversary, since they are essentially paying for more storage to come online to counteract their attack. Incentives to clean out real garbage would be used by well-intentioned players, and simple features that allow one to use lots of resources at lower/ ultra-low cost by allowing the user/owner to indicate data as unique and temporary will go a long way towards dealing with actual unwanted data.

When I first started looking into SAFE I was of a similar frame of mind as you are now. This mostly comes from one’s normal desire to not be wasteful with precious resources. After taking some time to read more, mediate on different design implementations, and view all of the SAFE philosophy and project objectives, I changed my mind or ‘saw the light’ so they say. I think most people starting out with a similar mindset will also, eventually. Cheers.

4 Likes

I’d suggest scanning through previous threads on this. Much has been debated and no point repeating it all here. I am sure the mods will merge this with it in due course anyway.

2 Likes

Indeed - search for ‘permanent storage’ and you’ll find enough to keep you going for weeks @edog305 :wink:

2 Likes

To add a simplistic partial answer is that history (and research shows future tends) shows us that the two major requirements for having “forever”/persistent data is increasing at a rate that exceeds making this sustainable.

  • data storage sizes
    • unit drive storage is increasing at approx 10 times every 5 years
    • unit production is increasing (at least doubling every 5 years and maybe much more)
    • unit pricing dropping even through capacity is increasing
    • basically what you stored 5 years ago on one drive will take 10% of a new drive
    • basically what the world stored 5 years ago takes so much less than 5% of world’s storage now
  • Bandwidth
    • unit backbone links have been doubling in speed in approx 15-18 months for decades.
    • Number of Backbone Links increasing year by year
    • consumer links limited by political, commercial and user perception. Complex actions at play. Some homes in the world have 1Gbit/sec or higher links and some have 1Mbit/sec or lower links
    • Basically bandwidth increases to meet demand.

From these two major factors determining the viability of “forever”/persistent data we can see in a simplistic fashion that it is quite viable to do.

So for a constant number of nodes what was stored 5 years ago will be taking much less than 5% of the current storage space for those nodes. When archive nodes (run by ordinary users) are added with permanent backup storage units the ability of the network to shift little used data to them comes available.

One other factor is that as the network grows so will the proportion of world’s storage move over to SAFE storage. So not only will data take up much less than 5% of space on a per node basis, with the network increasing year by year that data stored on SAFE 5 years ago will be taking a lot less than 5% of the current storage space on SAFE

As has been said, the rate of access to most older data decreases as time goes on and this means that bandwidth required for old data decreases. But this does not mean the value of that data decreases. Lost data or never used data is rendered impossible to detect because SAFE design makes it impossible for the even the network to know the data. To overcome this there may be introduced a way for the uploader to mark the chunk as temporary and then the network knows.


Finally something I didn’t see touched on is that we are now coming into the era where “Data is Valuable” This means that data in and of itself is valuable. Your family photos that are only stored on digital media, the last Will and Testament of your Father, the school results, and so on

A lot of this valuable data will only be accessed occasionally and to be kept Secure for a long time. This ability has to be open to Everyone. And SAFE will provide a safe way to store and Access this information without unwanted people getting access to it.

Basically a Secure Access For Everyone network

2 Likes