We’ve had questions (and attempts to answer them) about “best” ways to organize storage on which to place vaults.
At the moment there’s a mini rush going on related to the other well-known decentralized storage project because they’ve started running their Alpha 2 (I wouldn’t call it Beta 1) network. They call it Test B.
One thing that caught my attention was repeated discussion about “best” media, RAID levels and so on. While it is normal that people are concerned about that, and similar questions were asked on this forum, I wanted to put together my thoughts regarding RAID levels.
First, despite the “rock bottom” cheap capacity peers can provide, there is no escape from the laws of economics. Some (let’s not name names, except Warren’s) have declared the end of artificial resource scarcity, but what we’re witnessing is the same old economizing which confirms my earlier expectations that we can’t ever stop economizing (unsurprising, really, except to socialists). We can, and won’t be able to afford to uneconomically use resources on a large scale.
Second, some expect ridiculously low storage prices that would be brought about by creative individuals serving data off cheap desktop drives without “bloated” enterprise storage hardware.
But instead the uncertainty (of whether a miner will be online tomorrow) that is a consequence of decentralization results in the need to replicate data multiple times, and - this is important - punishments have to be meted out for extended or permanent downtime, as well as for data corruption (where data is destroyed/corrupted on a per-contract basis). While on the network the default level the replication provides sufficient protection, on the individual level you can’t benefit from it as a farmer.
If you lose a drive together with vaults that live on it, the network will survive but your reputation (and wallet) won’t. If you have just 1 HDD, you have no choice but to go unprotected, but if you’re a serious small and medium farmer (“SMF”), you’ll have 4 or more disk drives. What then?
You can set them up in a non-RAID-ed configuration (4 separate disks and with a separate file system on each). Normally that means that a failure of any single drive doesn’t impact the rest. But with these systems the network must still punish you individually, so if you happen to be running these four drives even if the other 3 survive, your reputation will suffer and perhaps significantly so (that is, it may turn out to be cheaper to RAID5 them, than to lose 1 disk out of 4, because the associated reputation risk may be too high relative to the cost of 1 HDD).
Or you can setup 4 accounts (one per each drive) to prevent that, but you can’t in advance tell which is worse - to suffer a 100% reputation wipe-out of 1 account, or partial data loss of 1/4th of your data (the first scenario, above). As I commented last year, we won’t be able to know which approach is better until the way algos work is finalized, and even then there will be different versions of “better” depending where you live, how much you pay for power, HDDs, etc.
My current conclusion is that I want to use RAID1 (for mixed personal + mining use) or RAID5 (for mining only), but because of the uncertainty (I don’t know the way system will punish and reward farmers) I did the opposite - I prepared a farming rig with a single disk.
Once we have more details on how the system works I think SMF’s will use some sort of RAID protection (probably RAID5 or RAID6, because RAID1 is quite inefficient). But, as I argued in here, this will make it harder for SMFs to compete.
Because the network has to work with untrusted farmers and because individual participants want to protect their reputation, data efficiency will be very low (multiple copies on RAID5/RAID6 or RAID1)
It will probably make economic sense for casual farmers to engage in “unprotected farming” (this is where I am starting, although I didn’t see myself doing this)
I think we’ll see a lot of rigs with 2 disks and one identity per disk drive. That way it’ll be easier to split risk and even engage in “merged farming”.
At the moment I guesstimate the risk of farmer centralization is very low; even specialization will be hard!