Best Safe Node hardware

Guys! Thanks for answers. But I still have questions :smile:

janitor, it’s something different from what dyamanaka wrote. in this case I realy don’t care to have more vaults. As for me I don’t think I need 0.25 four times in different pockets. If it is only reason, of course.
or I don’t know something important to undestand the whole situation?

It is said on the site that CPU is matters.
But I noticed that some users are talking about atoms and odroids. which perfomans is low

I still don’t understand. What is the role of storage and cpu?

Is there any formulas or explanation what is the share of cpu performance in getting ‘donuts’ for farming?

1 Like

It is said on the site that CPU is matters.

Where? What is said?

For vaults there’s little crypto, so the processing load is not so crucial. User machines will be doing more crypto, so there CPU matters more.

Research by reading the posts on this thread, and there’s a recent one by @ned14 somewhere (easiest to find it via reddit - someone asked this question there which I answered, and then added in Niall’s more detailed and informed response too). David Irvine has added that it is best to split large disks into multiple vaults.

The are no equations to study, just get ready for testnet 3 and we’ll start to see then.

1 Like

here http://maidsafe.net/farmers

Each node (computer) will be given a ranking by the network, this ranking will be based on the attributes of the node. For example, a node that provides a lot of extra storage, has a powerful CPU and lots of spare bandwidth on a computer that is always switched on will be given a good ranking by the network. The network will give a lower ranking and subsequently store less data on a node that provides less storage, with a slow CPU and limited bandwidth on a computer that is switched on intermittently.

Thanks, that doesn’t indicate the relative importance of these factors. David has clarified that CPU is not likely a hinderence because the processing is not onerous. In fact it’s not a factor in itself, but how quickly nodes respond that matters, so CPU is one element which contributes to response time, and availability will also be very important.

EDIT
@alexkon see these links:

2 Likes

Very true and its a consistent issue I always try and ensure the design follows and that is ultimate inclusion, so amortising the safecoin farming across as many nodes as possible and it is very hard and there are issues. Larger nodes will likely win more in the end when archive nodes are used (nodes that can store large containers of older chunks as a single looked after entity). In the short term the requests will be so random its best to have many actors in the game.

The network will decide dynamically on cpu, b/w, space, on line time etc. there are so many variables and they will change continually beyond our control (but following our rules) as they should, but quantity is always a winner.

7 Likes

I just said that despite @warren’s pronouncements about the end of “artificial” scarcity, you won’t be able to create four 40TB vaults out of one 40TB system and thereby get something out of nothing. Unscarcify that!

@dyamanaka said you might get more requests from 4 * 10 TB than 1 * 40TB
That may or may not be true (for example, we don’t know what will be the maximum value of “concurrent requests served”), but what I said is undeniable true (you can’t make 4 * 40 TB out of 1 * 40 TB). If you managed to do that that’d be a security issue with MaidSafe.

To add another detail about @dyamanaka’s comment (I didn’t want to comment on it at first because I think it’s too speculative) - if it’s true that the protocol will be forgiving with regard to the performance, then there’s no reason why a vault couldn’t serve 8 chunks at once. A good vault can serve each of the 8 chunks faster than a Raspberry can serve a single chunk, but if the devs decide to favor centralization then they may introduce artificial limit of say 2 concurrent chunks to force you to have 4 vaults of 10TB or 8 vaults of 5 TB than one vault of 40TB.
Performance-wise that would be unnecessary because it doesn’t make sense to expect that 8 Docker instances on the same box would work faster than that same physical box without the Docker overhead. It would improve the stability and redundancy of the network, but it wouldn’t impact how much capacity you have at your disposal (my original claim).

I mentioned in here* that I don’t think ping will be a big factor. If it becomes, the slowest guy will never ever get any requests. There are 4 copies of everything. If the best guy sometimes happens to be caught in the middle of serving a request (I think that’s very unlikely - it takes less than 1s to serve a 1MB request - does anyone think some vaults will get 100K requests per day? No way!), the request would go to the next guy. The fourth guy would never get anything.
So MaidSafe will try to decentralize and be very tolerant of poor performance.

1 Like

Has it ever been discussed to give the slow repliers a (smaller) reward as well? After all, they are contributing to the ecosystem by keeping copies of files. Even though they don’t serve them fastest of all, the principle of providing storage capacity that adds to the decentralization of SAFE is more important than that extra bit of responsiveness. It seems unfair to me that slower systems (whatever the reason may be) risk almost never getting a reward. If it’s a common problem those farmers would stop farming, which would reduce the total capacity of the SAFE network.

3 Likes

That is not what I meant. We are getting lines crossed here.
If a farmer has 40TB of data chunks available, they will “likely” receive more GET requests than a farmer who is hosting 10TB of data chunks. Yes, it’s speculative because the 10TB farmer might have more popular chunks, resulting in more requests.

If you consider real life factors like: trends, viral events, peak activity, etc… This is a common event where masses of people do the same things at the same time.

I’m not worried about a vault serving 8 chunks per second. I’m talking about a vault trying to serve chunks per second that exceed their bandwith. This causes them to be “busy” or delayed when responding. If you have a very large vault and low bandwith, this is likely to happen at some point. If it’s not an issue, then we would not need opportunistic caching.

From my understanding, the Network rewards better performance. It is clearly stated the fastest of the 4 chunk replies gets the Safecoin farm attempt. This is good because we want a fast Network.

I have not heard any discussions about paying 2nd, 3rd, and 4th place replies. Adding to that idea, if each Safecoin farm attempt is a % chance. Then we could make a payment structure like this…

Vault Replies (Fastest to Slowest)
1st Place Vault (1% chance to farm Safecoin)
2nd Place Vault (0.5% chance to farm Safecoin)
3rd Place Vault (0.25% chance to farm Safecoin)
4th Place Vault (0.125% chance to farm Safecoin)

This is just some ideas. Equal pay for equal work?

2 Likes

Something like that, yeah. The incentive to be the fastest is obviously still there, and at the same time we won’t be alienating slower farms from the network.

I think it is possible that there are more than four copies online on the network right? Or does the network delete chunks if there are more than four? If not, maybe it’s better to give the fastest the 1%, and all later ones 0.25% chance?

1 Like

Yes, there should eventually be more than 4 copies, due to caching popular data chunks. Caching distributes the workload from overwhelming demand. This leads to a whole other debate about compensating vaults for caching.

Yeah, the fastest should get the 1% and the rest get 0.25%. Just to be clear, we aren’t setting the percentages in stone. It’s just for illustration.

I didn’t have caching in mind when I said that. I was thinking about a vault going offline (3 copies left), a churn event (4 copies again), and then the vault coming back online (5 copies?).

You’re right I just re-read that part of your post again.

Ah-ah-ah, I thought of that too! Seriously, I considered that and concluded that any closely time-spaced requests are likely to be cached by intermediate nodes. That of course “depends” (like everything else), but just consider the likelihood of receiving 60 unique requests per minute - that would be enormous and IMO is highly unlikely.

Well it can’t “exceed” their bandwidth because even the crappiest disk is faster than the average network.
Even if you get 10 requests, you can serve them within 1 second using regular HDD. But your network most likely won’t be able to deliver them within 1 second, so what will happen those requests will be delivered at the speed that network can deliver. And I again claim that this will never be seen in real life because you will never get that many requests.
1 request per second means 86400 requests per day. Never… Let alone 5 requests per second.

In the topic about the risk of farmer centralization I said that would be the best approach in a for-profit environment, but I don’t believe MaidSafe will have the luxury to allow that, otherwise the 3rd and 4th guy will never get any GETs.

That’s another thing I claimed in my arguments against centralization: I think MaidSafe will seek ways to be “inclusive” (to use that terrible, but popular word) so economic priorities will have to be considered along with the project’s values (decentralization, etc.).
That’s why I think proposals like yours will be adopted. Such ideas will not result in the best service at the lowest cost, but there’s no other way to stop farmer centralization so I think some sort of policy (or “political”) tuning will be necessary and that will make our Monsantos less competitive and discourage pro-farming and centralization. On the one hand they won’t be allowed to thrive, but on the other they will be welcomed because they’ll contribute good bandwidth and QoS, which will result in frequent tuning of algos and increase their risk, so it’s going to be tough.

2 Likes

I tried to answer that question here.

Yeah, the fastest should get the 1% and the rest get 0.25%. Just to be clear, we aren’t setting the percentages in stone. It’s just for illustration.

Is this being actively considered for implementation?

I recall previously David song that this was being catered for through the ranking algorithm. By making higher ranked vaults earn at higher levels, but not that much higher, so as to allow everyone to participate.

We aren’t at that stage yet, so it’s just ideas being thrown around by the community.
When TestNet3 goes live, this thread will be useful.

1 Like

Even then you’d be surprised how unimportant CPU is.

Yesterday I had to stake into the heart my vision of the evolution of the internal Maidsafe code base. I had been advocating a design where we offload crypto, hashing and UDP packet reassembly into the GPU via OpenCL, but it turns out my assumptions were out of date. CPUs have caught up significantly with GPUs at crypto and hashing, and after doing some empirical testing this past week I came to the conclusion that a CPU-only design - which is far less disruptive - would be competitive with an OpenCL design on existing CPUs, and exceed it on newer CPUs with dedicated offload hardware.

Due to the loss of the GPU offload, I also had to scratch this week my ideas for arm’s lengthing user data handling such that they never become readable by any Maidsafe code - this would make exploiting Maidsafe code to steal user data very hard. Unfortunate, but in the end better to realise the design mistake now rather than waste tons of effort on it.

Niall

5 Likes

Any guesstimates on the number of vaults that could be handled per core?

I guess this is interdependent with other components, but I’m guessing it’s an order of magnitude above 1:1

I’d really like to know some estimates of the optimum on this as well.

David Irvine has said that they’d been testing them with up to vaults 64 on one machine running Ubuntu, but I doubt that’s with a lot of data being stored or transferred at once, etc. I’m betting that most machines could handle more than one, in any case.

1 Like

That’s all testnet 3 playtime guys!

3 Likes

In another thread I mentioned that I think that “vault hit rate” won’t be nearly as high as you (and @dyamanaka) expect.
If it’s less than 1% (which I think it should be), you’d get less than 1 request per second on a box with 64 vaults. Makes sense to me.

1 Like