Latency Keeps Centralization Alive


#1

I was thinking that latency will keep centralization alive more than we might like. Centralization of course means info enclosure, sponsorship and resultant lack of transparency. In short it helps keep corruption and inequity alive through scarcity, bottlenecks and the corruption that rides the toll roads it engenders.

If we take a city the size of LA and suppose we can get the mesh hops down to .5ms additional latency per hope at .5 miles for an average hop its possibly about 30ms latency across its diameter or in the worst case. So that amount of latency seems plausible for things like gaming, but its not going to create the kind of distributed computer that can replace a data center for many tasks. You need compute nodes to be as close together as possible for that kind of thing and that implies centralization. It doesn’t mean we can’t provide a good quality distribution system political speech or most content forms save for possibly for more demanding gaming which under centralization may become quite popular with render farm games distributed through tech like PCell.

In the longer run out beyond classical and relativistic physics approaches there is the possibility of instantaneous communication which would break this centralization and compute bottleneck on distributed nodes. But it would also possibly bring us a world with such radical changes as to be quickly unrecognizable. In the interim, hypothetically a decade out there is the possibility of cellphones running open source software and hardware where the hardware may be 400x more powerful than we have today. Think of 2 tflops cpu 100 tflops gpu 1 tbyte ram 500 tbytes of SSD. If everyone has the equivalent of a big server in their pocket the evils a tiny bit of server activity in a distributed computing mesh across a city where everyone owns the software and hard that make it up are potentially reduced.

In the mean time, latency even within a data center and the need within big compute systems for a tight proximity fabric of components means that augmented reality, Google Glass and Street View and render farm games will run off more centralized stuff like PCell with its centralized toll road back channel and its asymmetric upload. That type of asymmetry in some ways is not all bad, even if it doesn’t work for mesh. We are used to it with our human sensory IO where we take in a lot more info through our senses than we are aware of putting out in our communications. And when we do need output we’ve already processed and compacted it a bit. Still if climbing the mountain of search implies stuff like Watson Debater it also means centralization. Although that phone 10 years from now looks like it could handle debater.


#2

@Warren,

This is something that I am concerned about too. The current project I have in mind for MaidSafe is DB intensive and for one of its possible applications at least, requires fast, interactive, real-time responses. @dirvine argues that performance might actually improve with MaidSafe because of the parallel processing involved in recovering data from different chunks at the same time - but we will see I guess . .

What I was wondering is: In a transitional state, where I have a conventional web server set up but keep the DB on MaidSafe - if I set up my own “data centre” with a lot of storage for MaidSafe, is it possible to ensure that at least ONE of the copies of chunks is always in my own “data centre”? If that were possible, then I would only have the “within data centre” latency issues as usual ie it seems like I would have the best of both worlds?

Phil.


#3

@philip_rhoades that is a fascinating question. Even if the Maidsafe system forbade one of the four copies residing on your server what would prevent you from setting up a mirroring where events were driven from the side of your server with your server retaining a 5th or possibly original copy and the network mirroring. That type of situation might or might not grant you credit for sharing your resources with the network but maybe you could still partition server resources that weren’t being used for your original or 5th copy to the MaidSafe system- if any of that makes any sense and assuming I’ve understood the question or maybe I’ve repeated it (?) It will be interesting to see how David or the crew respond. Sorry, as is probably apparent I come mainly from distant speculation and we need a pro to attempt an answer to your professional question and not just any pro.


#4

@Warren,

That might work, and it would be even more of a transitional state, but the main reason for using the MaidSafe environment for the DB is to leverage the built-in security - if I have the original copy still on the old Internet, I lose the value of the storage on MaidSafe (except for maybe as a backup - which is still valuable of course). There are a few people who could help with this analysis but they are all flat out getting ready for the launch so serious discussion will have to wait till sometime after that I guess . .


#5

Knowledge or control over where data is stored would I think be a fundamental breach in the security of the network, so while I don’t actually know how things are done I’m 99.99% confident that no one can use SAFE to store data and control where the chunks end up.


#6

@happybeing,

Ah yes . . hmmm . . I don’t know enough about the algorithms to know how the choices are made about where to store the chunks - I presume there is some trade-off between network distance for security and efficiency of locally running apps?


#7

Essentially its really important that no-one knows where their chunks go (i.e. on which physical machine) and that no machine owner knows the physical identity of a machine that supplies a chunk they store.

There is some leaking of physical identity (IP address) at the fringe of the network, but this is stripped away at the first hop. If this were not the case, the network would be vulnerable.

I don’t understand the algorithms either, but have listened to David’s explanations enough times to be confident about these points.


#8

Do you mind posting a little about that idea, or providing a link to where you mentioned it previously. I am trying to get an idea of use cases for optimising ease-of-use and performance in this API.


#9

@vtnerd,

It is a little early yet for putting up stuff for general consumption but I will forward you some of the private messages with some more info them. It may be that for some reason(s), technical or otherwise, the MaidSafe plan won’t work out - but we will start to get a better idea once the launch is over and people have time to look at other things - although my suspicion is that things will get EVEN BUSIER after the launch for the main dev teams . .

Regards,
Phil.


#10

What does a mesh need? Power, a computer of some kind, and some way for computers to talk to one another. So explain to me why rigging up tiny computers every so often is so complicated? I mean in a large city cell phones will probably do it and if they don’t well just get a bunch of cheap computers and hook them up everywhere. Power them with solar and load them up with the appropriate software. Why is combatting centralization so complicated?


#11

Real-world distance is no factor in the distribution of chunks, at least not in the algorithm. It could be that the market will cause some centralization, since the farmer who can deliver data fastest gets the SafeCoin reward. Hence farming is probably most profitable in areas where there are a lot of other SAFE users, like big cities. If you live in the middle of nowhere with a bad latency to the rest of the world, you have some disadvantage when farming. Arguably bandwidth may in practice be more important than latency though.


#12

I was hoping it might be possible with forseeable tech to route centralization completely with a mesh of phones running something like Safe. But no distributed computer will be an across the board replacement for massively parallel computers that have components as closely spaced as possible with big power budgets. A wireless distribruted mesh of end user hardware may change the world but until phones themselves have the power of a data center we are stuck to an extent with data centers as a centralizing force. If we can get unfettered q communication we can break that, but short of this, c is the limit and a centralizer.