Network speed? Network data structures? Forever storage economics? Challenge-response authentication?

I recently I gave Safe project presentation at Paralelni Polis at Prague (in Czech) Maidsafe - Distribuovaná pamět civilizace - Google Slides and I got a few of question that I wasn’t able to answer. I am sure these were addressed somewhere already. Thank you for pointing me there.

SAFE network
(1) How fast the network will be, mainly in terms of latency? Would it be usable for ordinary browsing? Are there any reliable tests or predictions? My fear is that the network will soon be overwhelmed by the protocol communication.
(2) Is there any schematic documentation of Safe logic network structure? Meaning what are the main data structures saved in DHTs, that the network relies on: node PKI - client data and datamaps - safecoin - data chunks - …

SAFE economics
(3) User can store the data forever without need to pay per time or giving (forever) equivalent amount of disk space hile the network itself seems to be unable to forget even if resources are going down. This seems to be a design glitch, isn’t it?
(4) Since users has to pay for storage only once, this seems to contradict the fact that farmers are paid based on resource/time.

Self-authentication
(5) Client authentication is based on static key+password+PIN right now. Will it be possible to authenticate based on challenge-response? This would greatly enhance client side security, mainly when private keys would be stored on hardware like TREZOR and would never leave the device.

4 Likes

(3) yes indeed this seems strange at first …
but as resources are going down new PUTs are becoming more expensive and farming reward goes up as long as it takes for farming to become profitable again.
(Uhm - you could say freshly stored data is paying for false predictions of storage costs for old data)

(4) this is true - storage doesn’t cost the amount necessary for “storing” but it will cost as much as farmers estimate their costs for storing it long-term … yes there will be large amounts of data trash floating around inside the network but time has shown storage gets cheaper and cheaper (and the same file being uploaded more than once will reference the same source inside the network [data de-duplication i think is the keyword here] and therefore will not use more storage space) and so “old data plumming everything” should not be an issue

1 Like

Hehe, welcome to the club. I discovered this just recently because I simply couldn’t believe it (so I thought I had misunderstood what I read).

It doesn’t “contradict” anything - it is clearly stated that farmers get paid for resources when the resource is being accessed, but not at any other time.
It’s just hard to believe that’s how it will work. It doesn’t make economic sense (which IMO is going to turn out into a big problem as soon as real Safecoins start being used).

to be honest i don’t really understand the intention of your question - yes you could write different login-schemes mapping to very complex key+password+pin’s even without users recognizing.
No Data leaves your device unencrypted anyway - so this is something optional (and yes i’m very sure those login schemes will appear / will be used)

there are many ideas floating about making login more secure/easier:

2 Likes

(1) The number of hops is O(log(n)) but, as the data is stored in several places, you have always the possibility to find a node closer to your XOR situation. The O(log(n)) is the worst situation.

Some number…
10.000 Users → 14 Hops Maximum
100.000 Users → 17 Hops Maximum
1.000.000 Users → 20 Hops Maximum
10.000.000 Users → 24 Hops Maximum
100.000.000 Users → 27 Hops Maximum
1.000.000.000 Users → 30 Hops Maximum

(2) Read that:

(3)(4)This is discussed many times in the forum. And, as you can read, my opinion is totally contrary to @janitor .

The stored data follows, the last decades, an exponential growth. While that continues, and nothing indicates will not, the new data will always be the majority. The storage cost, of old data, decreases very rapidly to become, in a few years, irrelevant.

2 Likes

But farmers need to make an investment today, knowing the value of it will become irrelevant just 2-3 years from now.
You think they’ll invest anyway, I think they won’t.

1 Like

You’re talking about pro farmers here, and we don’t know if they will be needed or how significant they will be. If they are needed, the network will pay enough to attract them, if not, farming rewards will be lower because ordinary farmers are providing the storage.

Data storage costs and bandwidth costs decrease exponentially over time. Let’s assume for this example that the costs halves every 2 years, and that the price to store data for the coming 2 years is X. We get the following figures:

2 years cost: 1X
4 years cost: 1.5X
6 years cost: 1.75X
8 years cost: 1.875X
10 years cost: 1.9375X
12 years cost: 1.96875X

If you contine this list you’ll see that it never reaches 2X. So the network will on average charge 2X for uploading data to the network, and will over it’s life course reward that same amount to farmers for hosting that data.

This means that on average, the rewards earned by farmers for hosting that particular data will be:

Years 0-2 rewards: 1X
Years 2-4 rewards: 0.5X
Years 4-6 rewards: 0.25X
Years 6-8 rewards: 0.125X

Edit: This is a “perfect” example, naturally it won’t go like that in real life. It will all vary due to variations in supply and demand and technological progress. But overall, this type of equilibrium will exist.

2 Likes

What does that even mean?
There’s cost of purchase and cost of operation.
What you’re showing looks like cost of new capacity, which doesn’t play role in one’s ROI. I don’t care how much a new 8 TB HDD costs, I care how much I’ll make using the 4 TB HDD I bought this month.

The ~2X cost above is the cost paid by people who buy storage.
The 2X cost charged by the network is not what goes to the farmers, but what buys the next batch of storage.

Rewards go to everyone (regardless of when they added capacity), not to people who added capacity this year.
Sure, disk capacity goes up while cost goes down, but the way this is modeled is too simple and incorrect.
This example shows how one over years gets a 190% return on his investment in HDD, which is impossible (if it were possible, no one would invest in anything but SAFE farming).

This way of modeling makes more sense to me:

  • Buy 8TB HDD for $200 (it is slightly more expensive)
  • SAFE sells your space to the uploaders for $400. You (the farmer) don’t get anything at this time.
  • Let’s assume you get a 2% monthly hit rate in Year 1, and 1 and 0.75% in years 2 & 3, respectively earning 40 GB * 12 = 480 GB in year 1, 240 in year 2, and 180 GB in year 3, or 900 GB for which you get paid less than it cost you to buy 8000 GB (that much should be obvious).
  • In this case I think you’d make closer to $30-40 for farming, rather than $380 (Seneca)

X is the cost for 2 years of storage, which includes write-off costs and operational costs for that period of time. If the life time of your HDD is 2 years exactly, then you can’t run it longer than that and thus only make 1X.

So, where did that remaining $360-370 in SafeCoin go to? What you’re saying here could only be true if the network would be contracting it’s SafeCoin supply at a massive rate, absorbing many SafeCoins and not re-issuing them to farmers.

Ad (1) I know about O(log(n)) feature of Kademlia routing (XOR). My question was more about real-world situation when you have high latencies due to variable quality of nodes PLUS communication overhang due to distributed nature of the network. What one can be worried about is a web page loading more than 15s etc. There will be a testnet hopefully soon, so we will get a glimpse of it.

I wouldn’t be surprised by a 2, perhaps even 3 seconds latency for non-popular content. There are many factors though, so it’s a hard guess. When you send a GET, you actually send three GETS, each to a different datamanager group. Each one of those then sends the GET to two vaults that actually have the chunk.

So there are actually 3 * 2 GETs, but we only have to wait for the fastest of them all. So it’s O(2(log(n)), but the odds of a worst case scenario are rather low.

I think high latency vaults won’t be profitable on SAFE, so hopefully we won’t see many of them.

@Seneca overtook me. :rocket:

Keep in mind too that the different managers, which transfers the data flow through the XOR space, are nodes with high rank and to have a high rank you need to be faster. There are a natural selection of better nodes to manage the network,

weird i thought safenet was building a network that has extreme stability using the extra resources that hundreds of millions of people around the world have and don’t actually use.

I didn’t realize that the economical model was purely based on some random guy wanting to buy a load of hard drives to profit from doing absolutely jack shit in work.

1 Like

Storing lots of chunks and some churning will make it have same effect. On average same rewards with little variance.

You can attach a value to storing 1GB permanently. It is like a discounted cash flow calculation in the valuation of a company. If costs never go to zero you can make the calculation for whatever time frame you consider permanent (eg projection of time that network will exist).

Farming rewards denominated in safecoins might go up but if the long term existence of the network is in danger (eg by a better alternative or a fork) their value in dollars will not resemble permanent data storage. Permanent data storage for everything long term is not sustainable. Garbage collectors and pay per time frame need to be integrated at some point.

Well, maybe the remainder got destroyed?
It’s not impossible to invest $400 and get $40 (or indeed nothing) in return?

If your hit rate is less than 50% of capacity a year, even if SAFE is selling GETs at 200% of the cost, you can’t break even. In order to get 4TB of GETs per year (8TB HDD size), you need to serve 11K 1MB chunks a day in GET requests. I’ve been claiming all along that nobody will get that many. 50% a year is around 4% a month.

In my view, for a while it’s going to be expanding, but it would be driven by people wanting to store, and not read, uploaded data. Then once people figure out their ROI is crap, the capacity will start contracting. Whether there’s a balance, I don’t know but I suspect that the economic model (including rewards) will have to be significantly altered.

I know Yamanaka and you claim to have considered this closely (he even created a model). My views are based on simple free market logic and I can’t say I’ve seen anything reassuring yet.

Yes I know it’s sad and old-fashioned - people investing capital and labor (required to operate farming rigs) to earn a living.

I, too, look forward to Fedcoin which would allow us to support ourselves by simply attaching to the Fed Faucet.

it’s not sad or old fashioned its just flawed massively .

You are creating a network where the majority of people on the network are connecting and giving up resources for a small amount of value in return, they can then either sell it , or use it themselves in the network .

you are saying you don’t understand the economics because it doesn’t directly cater to the profit chasers who are inevitably the ones causing such instability and fluctuations to the network…

you use the unused resources of tens/hundreds of millions of people, you have more space than the whole internet combined , or you can cater to a small percent of profit chasers , which will come and go if the price is right for them, causing huge fluctuation in the networks ability.

you are not expecting to offer huge profits in ROI on hard drives purchased because the vast majority of people will be using unused hard drive space that costs next to nothing to give up because you are already using the device as it is

1 Like

Shiny new large capacity HDDs are dropping from the sky and network connectivity isn’t an issue either because it’s simply a basic human right, so it should be provided for free… by someone.

That won’t get anyone to buy a HDD they don’t need or upgrade their network so that they can actually provide useful service. This kind of approach is going to create mayhem on the network. Without much incentive to stay online and/or provide useful service, network churn will be enormous. Running out of free capacity? Just delete couple of vaults!

1 Like

If i believed that bitcoin and their inflated unstable POW blockchain method was ideal that i might agree with some of what you are saying ,
but the profit chasers economically are the ones driving instability in crypto , causing difficulty on the network to spike, dumping coins and causing declines in value at a constant rate. cuasing centralization of the network (huge spikes in difficulty when say for example…a whole mining operation burns down) affecting the network directly and slowing it down.

I know that harddrive space/SAFE and POW blockchain are completely different things and these will translate slightly differently but also similar in ways economically to what we will experience here compared to other crypto currencies.

I suspect that the thousands of things that are possible once the safenet is up and running and fully functional for a while will likely be an incentive for people to stay online and provide useful services, also the small amount of value you earn from your own hard drive allowing you to store and create your own things on the network would also give much incentive. Making the creation of websites much easier than it ever has before

there is like 10 different features and other things mentioned throughout this forum that mention why trying to profit farm maidsafe will be considerably more difficult if not down right pointless , The intention to put to use the space no one uses that equals more than the entire internet.

this is exactly one of them , deleting a couple of vaults would lead you into losing your reputation in your group, take a long amount of time to get that reputation back, while slowly having to wait till your vault reaches a decent level of files to earn you enough money from requests. all the while , once it gets filled back up and you find out that actually your vault was earning more value from more requests before you deleted the vaults cos the data you were storing was more in demand