MaidSafe chops files into blocks and stores each block on a minimum of 4 nodes.
As a file can contain thousands of blocks depending on the file- and block-size that means theoretically thousands of nodes can store blocks of just one particular (large) file?
I can image that as the network starts with a limited amount of nodes, a lot of blocks will be stored at the same nodes, meaning that for instance 10% of a file could end up on one machine and that percentage decreases as the amount of nodes increases, correct?
So if the network DOES grow to hundreds or thousands of nodes that means that unique blocks are stored on an increasing amount of unique nodes.
My question is; will there not be a problem with the amount of TCP connections that need to be opened to fetch all these unique blocks from all these machines. How does this work? Will my machine keep the connections open? Does the software open and close the sessions rapidly for each block?
Non of the above scenarios make sense to me as they will either hit OS ulimits for tcp maxcon or be very slow and/or expensive in terms of RAM and CPU.
I am probably making a wrong assumption somewhere but maybe someone can explain
We don’t use tcp, although looking at dual protocol NAT traversal so may introduce it sometime. None of the rest is really important or different from today’s web, perhaps even less so. The network is recursive you may be thinking DHT and iterative which is normal. We don’t do that either
When I visit a site, I fetch a file from one destination address, it’s a bit different fetching thousands of blocks from thousands of unique dst ip’s. I was just wondering if that would not be a problem.
If you used osx with tcp and normal dht then yes you may hit a limit fast, we should never have more than 64 connections. Even at that we currently multiplex down to 10 sockets. All this is undergoing test now. So think connect to a group do everything recursive, it does not matter if there are 10 hundred or 10 billion nodes. You will be OK.
We could than talk of hops and so on and on but it’s really been considered a lot and discussed in the forum lots to if you are really interested in it from that perspective.
I believe you don’t make a direct connection with the vaults hosting the chunks you are requesting. You request them of your close group, and they’ll route the GET request through the XOR address space and when the chunks come back they also come through your close group to you. This may seem inefficient, but it’s part of the anonymity and resilience of the network.
If you view source of this page, you’ll see there’s (probably) at least 100 files (icons, JavaScript’s, fonts, etc.), so you’ll open a number of connections.
For a better approximation of behavior on a DHT network use system tools and your torrent client stats. You can have 50 torrents each with 50 files and it works fine. You could have easily checked that before posting.
