Permanent data storage + lots of nodes falling away

What will happen if SAFE gets wildly successful at some point and people are going to upload a massive amount of data, but at a later stage we get a large dip in user interest somehow (whatever the reason). Such a large dip that the combined storage of vaults is not enough to store all the data anymore (since data storage is permanent (?) this will only increase). What will happen in such a scenario?

2 Likes

This should be averted by the network drastically increasing farming rewards as free space becomes scarcer. If that fails, it’s probably game over. But I think that mechanism will only fail if SafeCoin is considered worthless, or if there’s a huge totalitarian crackdown on SAFE in many countries.

11 Likes

Yeah, like he said.

If that happens, it will be MUCH easier to get SafeCoin, so if you just host a vault then you can get CRAZY RICH :slight_smile:

So I think all the incentives are in the right place for success here.

4 Likes

The SAFE network create a space, call sacrificial, who can be used to balance the network if required. About a third of the network is used to control these spikes or prevent someone, with great resources, try to attack the network or speculate with the safecoin value.

This sacrificial space also serves to control the farming rewards.

5 Likes

I’m sorry. I’ve never heard anything like this. Perhaps it’s just the way it’s expressed, but could you direct me to a source on this?

2 Likes

That is the purpose of sacrificial data, we don’t need it but it acts as a weather vane for network supply, way too much means all this is stored, as it decreases to backup levels only we increase farming reward. So the network globally can balance. Sacrificial chunks are a pure measure and should never be used to retrieve data.

6 Likes

[After thought: Well, can’t get much closer to the source that that, David.]

Okay, I’m starting to feel like I’ve been missing something obvious, but I have. So where can I go to find out about sacrificial data. I’ve been through the wiki a bunch of times, and all over the forum. But I’ve somehow missed that vein of data. I’m not asking for explanation at this point (certainly wouldn’t refuse it, though) but I’m sure there must be something written somewhere?

Maybe it’s not finished / documented yet.

Lots of the breakthroughs here aren’t even finished being coded yet, so we shouldn’t expect to see their documentation yet.

We just have to be patient and let the team do their thing for a bit longer :slight_smile:

It will be worth it!

1 Like

Possibly not discussed publicly enough, but this featured in this sprint (actually last sprint put initial parts in place). Main thing is that it is there now and the purpose is a measurement, it allows the balance of the network to happen, It is perhaps described more here https://github.com/maidsafe/rfcs/blob/master/agreed/0005-balance_network_resources.md So a lot of complexity of algorithms removed by a much simpler approach (I search for simple everywhere). [edit the link is the next step, the sacrificial chunks are in place so not documented clearly]

5 Likes

Answering this RFC David give more information.

Ah this is where the network calculates in real time that
cost. Otherwise it’s not going to be able to give the most efficient
price. It requires a lot of explanation, here is a massively simplified
view of what happens and what’s being tested.

Data is stored in 3 locations (ignore replication etc.) Original Backup and Sacrificial

Sacrificial Data is the one that gets removed to store other types

Sacrificial is also the network storing 50% more copies than it needs (very healthy)

As each sacrifical is stored the Farming Rate doubles (twice as hard to get a safecoin)

As each Sacrifical cannot be stored or is removed farming rate halves (farming more profitable)

Farming rate is then used to calculate cost of storage So a user will
pay something close to the 5th root of farming rate per chunk initially
and then this also dynamically will be altered with population counts
(linked to deduplication rates) (this is to be tested)

So this means the network calculates cost on network health and in real time
which is very much designed to provide resources at the minimum price
people (farmers) will provide those resources. Another push is farmer
efficiency so people provide unused resources (computer already on share
unused b/w/ hdd and cpu) which should drive costs of resource purchase
to an absolute minimum.

External safecoin pricing is separate but linked. If farmers get $X
per hour they may wish to farm, if it goes to 1/X they may choose not
to, the network will rebalance automatically regardless of external cost
of safecoin. All it discovers is the farming rate required to get more
resources, it does not know why that is the rate but calculates based on
the rate required to attract/deter farmers whatever that may be.

8 Likes

Interesting, thanks for all the answers.

Lots of incentives and methods in place for a scenario like this not to happen.

When considering a discontinuity/black swan somehow however (e.g. as Seneca pointed out already a government crackdown, a global law that makes it illegal to use the SAFE network, they might not be able to trace who uses it but it might scare people in such a way that 90% of the nodes will disconnect. This would destroy the network where you would need the network the most in such times).

Will the network be able to recover from that? If there is no storage space to PUT data will storing data be simply infinitely expensive? If the network is full and we have 4 copies of a chunk but one node of it falls away and we have only three chunks, will it continue functioning in a proper way without being able to make a new 4th copy? If data is really lost for a while (or just one chunk) will it say, can’t find the chunk now, try again later? Will the network be able to make room for new data despite of the fact that it doesn’t have enough room for older data (by leaving it as it is and not making new copies for example if it hasn’t been GET for a longer period of time)?

1 Like

This really just WONT happen, it’s economics coming to save the day.

If it gets even close to this, then the amount someone can earn by hosting a vault will be extremely high, so it’s basically 100% guaranteed people will set up vaults to earn that money.

Shoot, I would! So just count on me doing it if that happens lol :stuck_out_tongue:

1 Like

The network is still in development and you propose extreme situations such as 90% of its members disappear suddenly.

Of course you can play with What if … situation all you want, but I think it’s smart for now fix the probable and leave the improbable for the future.

2 Likes

Totally agree, never said it should be taken care of immediately.

That said, there are a lot of situations in history where we could have wished that people would have played with what if situations a bit more. Black swan events can and do happen. So good to think about.

1 Like

@Dirk83 please don’t be out off. We need people who ask questions, and who wonder about ways that things could fail. It’s hard to respond to them all once you’ve been convinced yourself is all!

5 Likes

Great to see this taking shape! Elegant and simple, yet perfectly suitable.

I particularly like the way that the network lets the fiat price of Safecoin implicitly affect farming rates. Data is valuable, so what better asset to back a currency with?

While same may argue that energy is the ultimate raw asset, data access includes energy costs. Data is also the most precious product we produce from energy; it is the spine of our civilization.

Ofc, even more importantly, it will ensure the cost of the network is optimal and reliable. Brilliant!

3 Likes

It’s really great, and I think it will give SafeCoin enough escape velocity to become the store of value. The store of value, because I’ve come to believe that it’s a function that’s naturally monopolized. Other crypto-currencies on networks with unique features will be used for sure, but people will only buy those coins when they intend to use those features in the near future, and not buy them as a store of value. They’ll inherently be more volatile and likely less secure than the biggest independent coin.

Fiat currencies don’t demonstrate this behaviour as much, because their usage is enforced by law. In the crypto world you see this happening though. Bitcoin dominates by orders of magnitude. Sometimes new innovative coins come along, they rise, peak after a while, and then significantly go down again. They don’t achieve escape velocity. They don’t beat Bitcoin’s market cap. I think a userbase counted in millions is required to achieve that. I think SAFE will see that scale of adoption.

I don’t mean to say that we as a community are in competition with the Bitcoin community. I think many of us belong to both. But the respective coins themselves are in competition, for the position of the store of value. Better than gold.

3 Likes

Anyway, back to the topic at hand, I think the biggest risk in such a situation as described in the OP is the network not having enough reserves (un-issued SafeCoin) to sustain extraordinarily high farming rates for an extended period.

This is not a risk in the early days, but if in the future 99.99% (or even more) of the coins are in circulation, and there is some black swan event causing a significant drop popularity, it is quite likely that network income (from PUT’s) will also drop significantly. Then the network only has a very small reserve for farming rewards, a reserve that is barely replenished. If there are no SafeCoins left to farm, the network is essentially insolvent and will fail catastrophically.

I think the network algorithms should aim to always keep a solid reserve of un-issued SafeCoins. It gives the algorithmic monetary policy the flexibility it needs to be effective in turbulent times.

1 Like

@Seneca! If I’m correct, when someone buy storage, the safe network recycle a small percentage of coins especially for that case.

It recycles all the coins, none go directly to a farmer. They only go to farmers over time as they successfully serve data requests. So yes, there’s a constant income (from the perspective of the network itself) of coins, but also a constant outflow (farming rewards). The balance between income and outcome can and will vary, depending on the conditions of the network, but over time the vast majority of coins will come into simultaneous circulation. My point is that the algorithms should anticipate hard times, when a lot more SafeCoin flows out than in, and should thus aim to keep sufficient reserves.