Network Performance and Vault Profitability

Nice to see studies like this, looking at the popularity between various versions of the same video content.

I’d think the performance of downloading/ streaming files will be great no matter where the chunks are stored. It’s the rate of adoption that get’s me thinking, the uptake of Safe, whereby folks think that farming is a good idea…when they run the executable for the first time …the message to educate the user in the client is important, the type of thing you’d normally run an eye movement, clcik behaviour trial for a GUI etc

Industries such as porn, who hold the masters and like what they see with Safe…how are they going to transition operations I wonder…are they going to promote the idea of the customer downloading SafeBrowser and cranking up a vault to potentialy recoup some of the pro membership costs. Or maybe they spawn a million docker instances and get more tickets in the lottery.

Same with movie, music studios, dropbox etc …they see the potential, how do they crank up operations. It’s the fascinating part of the equation for me…how will the Safenetwork gain momuntum, bottom up home connections or top down corporate driven demand…anyway it goes, there’s a revolution coming :sunny:

4 Likes

Interesting to see what the Bitorrent creator is up to with his Chia Network. Proof of Space, Chia Farming, conversations with David :thinking:

In talking to hard drive manufacturers, the biggest optimization they point out is potentially eliminating the cooling fan. Proof of space otherwise uses almost all of the other core requirements of generalized storage. You can potentially use older and slower seek times or bus technologies, but to get the read density needed, you have to use state of the art read and write heads. The core issue is that the demand for cheaper mass storage that is almost hot storage but a bit slower is very high already. Spinning disk has mostly become that solution as large storage buyers and even desktop computers have used spinning disk to help scale out SSD.

For storage costs there are two price points that matter. The best price to store a terabyte online is Wasabi at $5/Tb/month. The cheapest drive we could find on June 4, 2018 at Amazon was $22.00 per terabyte for a Seagate 4TB drive before taxes and shipping (if any — thanks Prime!) Since Wasabi gets more expensive after about 18 weeks it’s pretty clear that ownership of the spinning drive is going to be the cheapest farming platform currently.

The most interesting alternative strategy that we’ve seen is write once blu-ray optical disc robots. A storage medium being limited to writing exactly once is less of a blocker for general purpose use than you might think, because that’s the usage pattern of a large fraction of all commercial storage and higher layer logic can be added to make such a thing look like a normal file system which happens to not free up space when you delete things and continues to lose empty space even when overwriting. Some ex-Facebook folks leveraged the near hot storage they’d created at Facebook to start a blu-ray robot storage company that Sony ended up acquiring. However it looks like Sony shelved the business shortly after acquiring it which doesn’t bode well for the technology. This is a potential optimization direction for pretty quick look up, write once optimization on proofs of space. But it is trading off the mass market scale of spinning disks for a new market of novel optical disk robots. They have to be quick to move through all the discs you’d want to read in the minute you have to find your best proof of space.

There is some truth to the general assertion that mass success of Chia could encourage storage innovations. The problem is that there isn’t (yet) an order of magnitude better way to store information. That puts the technology speculation out in the land of pure scientific discoveries. We are hard pressed to come up with a huge storage innovation that isn’t equally or more valuable to the buyers of the 585 million hard drives and SSDs expected to sell in 2018.

There are a few optimizations in response to Chia’s success that we might see in storage technology. Letting a farm co-exist underneath a RAID that’s otherwise in use for other valuable purposes could unlock a lot of unused storage space, especially if the farm is beneath the RAID’s redundancy. We do expect to see custom ASICs added to some drives (and maybe all of them) to make it easy to seed a farm. But those are technologies for allowing the spare capacity on existing storage to be more expediently used for farming, not exclusively for farming. It is also important to note that the speed that per terabyte prices are falling has slowed storage industry wide.

The primary attacks on proofs of space are algorithmic ones which use extra computational resources to get more rewards out of a fixed amount of space. Resisting those sorts of attacks is a major development focus for us and the subject of our Asiacrypt paper which we’ve since made further improvements upon. We’re also going to have a best implementation competition with real cash rewards in the future after we’ve published a specification. We feel confident that such efforts will quickly result in a plateau of rewards for a modest amount of computation and that any better trade-offs will require an infeasible cost prohibitive amount of computational resources.

8 Likes

Always interesting to see people researching and thinking in real world numbers like this. I hadn’t even considered people might hire space in a data centre to run a vault. I was more thinking of data centres themselves (gradually) switching over to SAFE.

What I see in those figures is that it might in theory be worth buying a $22 drive and earning SAFE coins at somewhere in the region of $5 a month, but is that enough for people to actually do it? Will they give up 10GB for 5 cents a month? On the other hand for a company to do this at these prices on a large scale is obviously a genuine business proposition, because people are already doing it.

I do really hope dirvine and neo are right about the home vault having the advantage, but I’m still struggling to see it in practice.

Either way, for me the question that is still important is: How can we distinguish an individual human owning a vault at home from a virtual machine owned alongside thousands of others in a datacentre or wherever, and how can we give that individual more of an advantage?

One fairly leftfield idea that I like is to make farming into a game that is essentially an elaborate Captcha. eg. A farmer is occasionally prompted to feed the animals and water the crops. Obviously something like this wouldn’t be proof of unique human, but it may not need to be - all we’re trying to do is further slow down the factory farmers, in tandem with all the other measures for Sybil resistance. Something like this also raises the idea of attracting farmers with the offer of fun, something virtual machines struggle to enjoy, as far as I know!

I suppose a more obvious way is to dilute any factory farmers by doing everything possible to get as many individuals as possible choosing to run vaults. To this end, I like the idea of making running vaults and creating content front and centre of the User Experience, rather than surfing the web. It sounds a small thing, but I think it could make a lot of difference, and in a sense, it’s this potential for interaction that social media thrives on. Personally I don’t see a problem with being quite aggressive about this, bundling a vault with the browser, requiring an account to have a vault linked to it etc. but I’m aware this isn’t part of the plan at the moment.

Not sure they’re much use in themselves, but just putting these thoughts out there in the hopes of triggering some better ideas from smarter people than myself! As with the more fundamental security measures, I feel like a lot of little things adding up could really make a difference to slowing down the big guys.

I’m not too worried about the big guys, as long as the small guy has an oppurtunity to compete. A big guy could run his big setup at home or in the datacenter…he has money or skills or access etc and good luck to him.

The talk around costs for the little guy in the datacenter is for individual pricing. It does not take into account the massive scale potential of a combined Safe user base (given an easy option)

The little guy might have some disposable income to allocate to some ‘cloud’ instances, but does not posses the skills or wherewithal or will to make it happen for himself.

So give him the option to become a Farmer:

1/ Create a local farm of x size to (a) Support the network (you turn off the computer at night = low chance of Safecoin) (b) Safecoin Farming (keep computer on 24/7 = higher chance of Safecoin)

2/ Create a ‘cloud’ farm (maintenence free, higher chance of Safecoin) - Choose an approved Farming partner (Provisioning API)

1(a) could be flagged as unreliable, maybe that helps with overheads? but certainly keeps expectations in check…even if the provisioning is 100% identical

By providing a no brainer approach to ‘cloud’ vaults and the resultant economy of scale, maybe the little guy gets to compete ok. Maybe the guy who will cure cancer from the desert, logs in to Safe at the sandune cafe and gets to provision an instance from a voucher he recieved through an Aid program or he has some of those mobile phone credits used in certain countries (the unbanked)…it could make all the difference to kickstarting his life with a vibrant Safecoin price and all done without owning a computer.

…for everyone

4 Likes

Yeah, that’s interesting. As I say, I hadn’t really thought so much about collectivised farming. In an ideal world it seems a bit backwards in the sense that it’s another layer of abstraction on top of the network. I’d also worry that it’s swinging back round towards a partly centralised model. Who ultimately has control over the vaults? Without enough brakes, these things can move very quickly, as we’ve seen with Google and Bitcoin.

I should have made clear I was re-purposing those figures to make a different comparison. Perhaps I was doing that unfairly, but I’d be interested to see some real world estimations of how much a farmer might earn. In the long term, how much more can storage pay out than what it costs, without causing unsustainable inflation? But that’s a question for a different thread…

2 Likes

Hm. Do you think someone would setup a datacenter server/VM just to run a fault? Of course that wouldn’t make sense financially. I will probably run a fault on my underused web server (VM), which will me cost nothing extra but runs 24/7 in opposite to my home machine. And I’ll still need that server in the forseeable future (i.e. while SAFEnet didn’t yet replace the web).

The other thing I didn’t see mentioned in the context of Sibyl attacks is influence of malicious parties that don’t care about money – it might not make sense financially to run a lot of nodes/faults, but if some state agency would try to gain influence on the network, they have enough money and time. They could run a lot of the oldest and most reliable nodes, as they run a lot of Tor nodes and whatever.

Yes, I have, for example, prepared 3 servers that I will put in different collocations. There may be no financial sense, but do not forget that not everything is money. A successful network will change the future of the world, and no one will take his money into the grave …

2 Likes

You won’t be the only one. But the discussion is mostly about profitability – or influence.

The SAFE Network needs to prevent big data centers from grabbing all safecoins. My prediction is that huge tech companies will use the SAFE network as a common platform for their 5G and IoT, and that’s trillion dollar markets. And then the tech giants want to run their own farming nodes for reasons such as ensuring network stability but also to earn safecoins so that they themselves can use the network.

Small farmers may be unable to compete with big data centers in terms of bandwidth and latency unless the SAFE network is designed to balance the farming reward so that small farmers also can earn safecoins.

Latency will be tricky, but altruistic home nodes in well connected regions will be able to compete. I will be running SAFE off an xeon with NVMe drive with a custom router on a gigabit clear channel near a fiber optic pipe. Happy to run at a loss. Makes no sense to buy or pay for such hardware to be provisioned, but ‘spare’ beats economies of scale a lot of the time and I think SAFE is banking on that.

2 Likes

Farmers with small bandwidth and high latency will still be useful for the network I think, because each client has many TCP/IP/UDP connections to the SAFE network I imagine. And some farmers will be faster and some slower, but that can be balanced so that the faster farmers deliver more data chunks than the slower farmers.

For example when a user downloads a video from the SAFE network, a higher percentage of the data chunks for that video can come from fast farmers than for the slower farmers. So all farmers will earn safecoins and it’s just that the faster farmers will earn more. The total bottleneck in the example will often be the user’s connection to the internet I think, so with some slow farmers that simply means more simultaneous connections, and from the user’s perspective the performance is the same as when all farmers are fast.

I do not think that for big data centers will be interesting SAFE network, when they can only estimate profit and reward can change from month to month, while you need keep servers 24h/365 online to increase your reward.
So if SAFE network will have double net income of their current business they will be willing to move with small part of their capacity.

My idea is that big tech companies will benefit greatly from some common platform to run their IoT and 5G businesses on and the SAFE network is something many of the companies would trust more, than say a platform developed by a competitor. For example Google wouldn’t trust Huawei’s infrastructure and vice versa, but they can both trust the SAFE network. And the big revenue comes from services running on top of the platform, and with solutions from different companies that can integrate seamlessly with each other they will increase their total revenue.

Running thousands of fast farming nodes is a drop in the bucket for the tech giants, and that’s something to be prepared for I think so that the big data centers don’t grab all the safecoins.