Some quick numbers based on youtube usage

5.000.000.000 hours of video watch a day
500.000 uploaded hours of video a day
or 10.000x video hours watched per day than hours uploaded.

gets are 1000x the puts cause each MB is in 1000 chunks so one put for 1MB upload but 1000 gets when that 1MB is requested

so we got 10.000 requests x1000 gets per request 10.000.000 gets per put

so with youtube’s consumtion there should be enough payment of the 10.000.000 gets for each put

200W/h consumption of a desktop pc

an average cost for kWh in the world is about 20c of the dollar and thats in UK.

0.20/1000 = 0.0002 → x 200 → 0.04 of dollar per hour electricity

720MB for an hour of full hd video means that per day we got 350.000.000MB or 360TB per day of storage

lets focus on one day. 360TB of storage needed, lets say each vault got 100gb of space so its 360.000gb / 100gb = 3.600 vaults

10.000.000 gets / 3.600 vaults = 2.777 gets per vault per hour so 2.777 = 0.04 of dollar x 3600 = 144 dollars electricity cost per hour uploaded

around 720MB for an hour of fullhd video so

144/720 = 0,20 dollars for a MB upload just to pay for the electricity. in the youtube scenario. for a GB its 200 dollars, thats for the youtube only scenario quick calculation of input and output for a day.

6 Likes

The price model count that user already have PC which run 24/7 or do not care much about electricity cost, or if does, than have low power consumption device which might be just 20W/h with single HDD. So than it should be just 2 cents for a MB upload.

Than high speed connection of each vault also reduce this price.

Edit:
with 1PUT = 10 000GETS would be 0,002 cents
with 100Mbps of each vault would be 0,00003 cents

3 Likes

Don’t forget that any file is also stored several times, I think the current plan was 8, so 2,880 terrabytes for those 500k hours of video.

3 Likes

Interesting, I have had thoughts in my head that “get” maybe should cost 1/10 of a “put”, compared with those numbers you show with 10x watch hours per upload.

I know “get’s” is supposed to be free but can’t let the thought about everything should have a cost “there is no free lunch” escape me. Farmers perform work for delivering get requests and a cost of “get” request might also give a lower "put"price.

Are there mote people than me having similar thoughts?

In my head there is also numbers like $0.5/GB put, $0.05/GB get, don’t know if that would be reasonable or not, just feelings. :slightly_smiling_face:

3 Likes

I think that the people who download information de facto generate payment for farmers, because they get no money when they record the data. There is really nothing free …

2 Likes

That is true for old PCs and some todays gaming rigs and high performance workstations. Normal PC is today around 60-80W.

And there is ton of low power computers available. For example I have small server with 32TB storage and power consumption <60 W in medium load. Raspberry Pi with connected HDD will be around 10W, with SSD even less.

3 Likes

Than the SafeNetwork would lost one of the main benefit. If you think that to store some random file on SafeNetwork is too expensive, than store just ones which are rare or expensive to lost. In general it should be more expensive than store on your own RAID storage, but accesible from every place connected to network forever.

5 Likes

1.-The size of the chunk doesn’t change the ratio between PUTs and GETs. If a video is seen, on average, 10,000 times that will be the final ratio between PUTs and GETs.

2.- You don’t count the cache effect as a small percentage of videos have hundreds of millions of viewers while others have few or none. And in these popular videos the cache substantially modifies the number of GETs a farmer will receive.

3.- 200W consumption to manage 100gb. is, in these days, ridiculous.

4.-Codecs improve year by year and 720MB, for a standard 1080p video, is too much.
imagen

5.-An amateur farmer will use his (paid) computer while using it for other things. He uses his (paid) internet connection and the spare space of his (paid) hard disk. So the real cost of being a farmer is, basically, zero.

6.-A professional farmer will ensure that his equipments are optimised and that the costs of both, purchase and use, are as low as possible.

8 Likes

I thought one 1mb was split into 1000 chunks… Mind memory I read that makes you remember things that are not true… There is a documentary about that.

Ok so disregard the x1000 gets but the 2.777 would simply be 2 or 3 gets per vault so if you even got cheapest pc at 10w its 0.002 dollars x 3600 = 7.2 dollars per hour so 7.2/720 is 0.01 dollars per MB? So 10 dollars per GB ok that is only for the electricity cost.

And I calc electricity cause people dont have pc open 24/7 most people shut down pc at night amd most people use their pc for a few hours a day not even all day

Later I will try make a model about how that progress every day to infinity

3 Likes

The 1080p on Youtube is about 2,3Mbps as MP4 H264 and 2,5Mbps as WEBM VP9, but most popular youtubers alredy use 2k, 4k or 8k. Than WEBM VP9 codec has bitrate:
2k ~ 8,6Mbps
4k ~ 17,3Mbps
8k ~ 21,2Mbps
And if they want keep highest quality like H.265 it is way more. So the quality of content is qoing to follow common speed of bandwidth and with ongoing fiber optics adoption, LTE 5gen…it will push uploaders to use high quality videos only with bigger average bitrate than today even with next generation codecs.
https://www.tutorialguidacomefare.com/test-video-quality-720p-1080p-1440p-2160p-max-bitrate-which-compresses-youtube/

2 Likes

As you know, GETs are paid for by PUTs, so it isn’t free. It is just paid for indirectly.

It isn’t dissimilar to Google pricing free apps, such as YouTube, using adverts. The consumer isn’t directly paying for anything and there is no guarantee that there will be sufficient advert revenue, but it seems to work out.

With PUTs paying for GETs, we can at least see that there is a connection between uploading something and it being downloaded. It will be interesting to see how it pans out.

4 Likes

That is why your estimate is off base. “The YouTube only scenario” is an edge case of extreme read:write ratio. The diversity of other online activities will balance things. Some applications will write lot’s of chunks and never read them again for 10 years or never read them, implying an extreme Put:Get ratio.

Intuitively, its hard to ignore the fact that there will likely be more Gets than Puts even when considering all online activities. This is from the simple observation that it is easier to consume than produce content. I suspect that the ratio will be close to what ISPs provide for up vs down bandwidth, since they have likely already examined the economics of it. In USA I believe that ratio is usually 10:1 unless you have fiber or a business class connection. Averaging over all connections might push the GET:PUT ratio closer to 4:1 presuming a pareto 80:20 rule with a twist. The ubiquitous ability for anyone to passively earn Ptp rewards might also affect these ratios.

4 Likes

I love that you’ve done this exercise, but the results are not correct.

Is there a source for these figures?

Just a typo of 350 should be 360 but in numbers and calculations typos are sorta critical… in this particular case it doesn’t affect the result but do be careful.

What you’re saying is 3600 new vaults per day would be needed to allow 360TB/day of upload. The way this figure is used later on is incorrect.

This is important to clarify because 3600 is not the network size, but that’s how the figure is used later.

The calculation also implies that existing vaults are full so uploads must be handled by new vaults, which I think was not your intention.

When calculating GETs per vault it must account for total network size.

A few things.

10M gets is not right, since there’s message routing, so each GET from client-to-vault must do log2(total_vaults) hops. Every hop incurs the work of a GET. 10M chunks served to clients would be much more than 10M GET requests overall.

The units don’t match. 10M GETs comes from ‘gets per put’, 3600 vaults is ‘new vaults per day’ and the result of 2777 is ‘gets per vault per hour’. The numbers are incorrect in the first place, but the units are also a real mess.

10M ‘gets per put’ is used to calculate an hourly rate but the 10M GETs are spread out over a long time, not just one hour.

3600 ‘new vaults per day’ is incorrectly used as a substitute for the total network size.

It’s a handy calculation to do but the numbers and units are wrong, and with no source to back up the initial claim this is a really misleading result.

This figure is incorrect, and is obviously incorrect based on existing storage expectations. To put it in some real world context, Amazon Simple Storage Solution is 2.3c per GB for PUTs and 1c per GB for GETs (source). So where is the 10,000x factor coming from that results in 200 dollars per GB? Even accounting for electricity costs, multiple redundancies, work for routing, there’s no way they accumulate to 10,000x more. Maybe I’m wrong on this? Would be interested to know if people think storage on SAFE would reflect existing norms or not.

I can’t say I have a good method for doing the calcs otherwise I’d put it here. It gets really difficult and based on too many assumptions. Would be interested to see you take a second stab at this.

These people will not be vaults.

100 GB per vault (your assumption)… then reset that once per day by turning off and on… no way vault operators will accept that mount of churn and load on their network connection, nor will the network accept that amount of work.

I’m sorry if this post comes across as negative; this really is a great exercise to run through and I appreciate you putting in the effort to do so. Please do keep working at it, I think it’s worth pursuing despite the initial errors.

I think the biggest improvement can be made when calculating the total work in each hour. The dollar per work is already much closer to being correct so is affecting the result less than the total work numbers.

13 Likes

Some of this is a repeat of @mav but it should give some other information/analysis

I agree with @mav, need a source for this.

As I said before to you, the chunk size is one MB and stored as a single 1 MB chunk. Not in fragments.

Also needed is the rates for very popular and trending videos. This is an extremely important metric due to caching in the network.

Lets assume some old analysis that 35-50% of youtube downloads is for the trending and popular videos. Thus caching is operating for this percentage

Average hops is log2(total sections) NOTE: sections because each hop is back through one node of each section in the return path. Even if it is nodes instead of sections then multiply the amount by 1.5

Lets use 40% as the popular or trending video traffic. The average hops for caching is nominally 1/4 average hops since caching occurs closer to the receiving node.

For this youtube example we HAVE to assume safe is world wide and nodes are on the order of 100 million nodes. ~5% of the world’s internet users are running vaults. At approx 100 nodes per section makes this is 1 million sections

log2(1,000,000) = 20
60% of gets have 20 hops
40% of gets have 5 hops
This is an average of 14 hops per get request

Each hop is similar to the work needed for a get.

so we get 10,000 request x 1 get per request ==> 10,000 gets per put

Now taking into account caching and hops this becomes
14 * 10000 gets * hops ==> 14,000 hops (hops equiv to gets with no hops)

Thats nice. BUT the user is already using the computer so the electricity cost using your figures is ZERO

But a better measure is trying to determine the incremental cost for the drive and modem while transmitting. This is in terms of 2 to 5 watts MAX

==> max of 5/1000 KWH/H * 0.20/KWH * 24 hours ==> 0.024 max per day
==> min of 2/1000 KWH/H * 0.20/KWH * 24 hours ==> 0.0096 min per day

There are no metrics for %age on mobiles and many mobiles use lower resolution since the phone’s quota is restricted. This more than compensates for higher resolutions as most videos are viewed on phones now a days during people’s idle time.

Lets just use your estimate. Even though the hours viewed per day seems high
5 GHr viewed/day ==> 3,600,000,000,000 MB viewed/day ==> 3,600,000,000,000 Chunks/day ==> 3.6 Terra Chunks/day ==> 50.4 Terra Hops
500KHr uploaded/day ==> 360,000,000 MB uploaded ==> 360 M Chunks/day ==> 5.04 G Hops/day

For storage there is a significant amount of deduplication because there is a significant number of copyright free duplicate videos uploaded to youtube. As this figure is unknown we cannot account for it except to know the final figures will be noticeably smaller.

Considering that any computer built in the last 12 months has at least 1TB hard drive. Even laptops are 1TB or more. So 100GB as a typical vault is very much an underestimate. We can safely use 500GB as the average vault for 2019 and more from 2020, 2021

The world wide network which the whole scenario is based on has a minimum of 100 million vaults at an average of 500GB gives us 50,000,000,000 GB ==> 50 million TB

360TB/day is 2,880TB/day stored since 8 copies of each chunk
This represents 0.00576% of the total storage needed per day
OR 2.1% of the total storage for 1 whole year of youtube upload.

At that rate there is no significant number of new vaults required per day

That does not provide meaningful information since new data is not being directed only to any new vaults

Uploads are “randomly” spread across all vaults

Now lets consider the 100 million vaults and the incremental cost to run the vaults.
100,000,000 nodes * 0.024 $/node/day ==> 2.4 million dollars world wide per day

If we attribute ALL this cost to youtube then we can work out a cost for operation. This is not really valid since the network is also being used for a lot more than youtube.

(5,000,000,000 + 500,000) * 720 chunks/hr * 14 hops/chunk * 24 hr ==> approx 1209 T hops/day

Cost per hop (if all cost is due to youtube)
$2,400,000 / 1,209,720,960,000,000 chunks ==> $0.000000002/chunk

Storage is
500,000 hr/day * 720 chunks/hr * 8 copies ==> approx 2.88 G chunks/day

Cost per chunk PUT is
$2,400,000 / 2,880,000,000 chunks ==> $0.00083 per chunk PUT (incremental cost) If the network was ONLY used for youtube

IF ALL computers were used ONLY for vaults and using your figures then the cost is 40 times that (200/5) which is $0.033 / chunk incremental cost.

What if we had to buy new drives to store this uploaded data
Amount of data per day
500,000 hours/day * 720 MB ==> 360 TB per day upload
360 TB * 8 copies ==> 2,880 TB stored

at $50 for each 1TB drive ==> $144,000 worldwide/day

For 100,000,000 nodes this means the incremental cost IF NEW Drives were bought purely for youtube storage
$144,000 / 100,000,000 ==> $0.00144 per day per vault

BUT BUT the vaults are using drive space that already exists and is unused so the actual cost to store the data is ZERO

5 Likes

As an addendum

If the network has 100 times the uploads and traffic since the web is way more than youtube then the above running costs per MB of youtube traffic drops 100 times

$0.0000083 per Chunk PUT

Cost for storage if ONLY new drives are bought to store chunks remains the same.
BUT of course using spare resources means the real cost is ZERO for storage.

1 Like

This is 100% true and can be seen on the noia network. They are giving 200 token a month (4.42$) for a node to be online 24/5. There are almost 2000 nodes online for this money:

It is not enough to pay the electricity bill… But I run my server anyway so this is a free money for me… and it looks the same for another 2000 people.

6 Likes

It is this kind of dashboard that would be useful to manage a vault. I wonder if it is typical of the average GET/PUT ratio mentioned earlier in the thread.

Edit:
I didn’t pay attention to MB / GB units, so the ratio is 0.4489/37.66

3 Likes

Is the dash board showing 37.66 GB uploaded to the storage or uploaded to the network from the storage?

1 Like

here is the number of hours uploaded and viewed a day The amazing amounts of data we create in a single day - YouTube

3 Likes

37.66 GB uploaded to the network. There are only 19.84 mb used on the node…

But this is a test network, so no real traffic for now.

2 Likes