Big Data: Moving large amounts of data into SAFE

People,

One of my projects will involve moving hundreds and then thousands of TBs into SAFE in one go - I have started to wonder how that is going to work. I just found this:

It seems I will at least need a fibre connection and I am going to have to work out how 1,000 TB can sensibly be split up - I don’t know how many files or DB records that 1,000 TB will make up yet.

Phil.

5 Likes

Be gentle at first. :wink:

4 Likes

I suppose you’ll be bandwidth constrained on the point of upload.

Google Fiber is being offered in more and more cities. The service gives you 1,000 MegaBits of bandwidth. Most people have like 50 MBs.

Not likely to be available in my little Australian country town , ,

Well, first that’s going to require a bus load of safecoin to get it all up there. Be sure you budget for that.
Second, my understanding is that Australia has some mediocre Internet as a whole. Maybe you could find a data center and get some type of arrangement to rent some space. Another option would be to buy several 5tb drives and ship them out to people with some big pipes. I’m sure there are plenty of people on the forum that would be willing to do it. I have unlimited upload capability on my isp, but have a 5mbit speed cap for now. Might try to up it when safe goes live.

2 Likes

I am on @dyamanaka’s farmer group and we have talked about setting up some stuff - this might be another incentive to set up our thing where we do have access to a big pipe . . good thinking! Thanks.

Yes, the upload speed causes a bottleneck, which is very low in most areas.

Mass upload could be a service offered by the farming group if they are interested. We need to get more people with access to fiber.

  • Breaking up a very large 1000TB file into (200 x 5TB drives) and have multiple uploaders should work.
  • File compression software like (7zip) also reduces the original file size. If it’s video, you can use (Handbrake).

I love this @philip_rhoades idea. I have unlimited/unrestricted 10Gbps up and down connections (really an unlimited number of connections as well) available. If there is a nitch market for this I will include this feature to the farming plans being developed. I love an idea you read that makes immediate sense and is simple.

I will try to get you a cost per GB or Tb to you quickly so you can add that to a physical shipping plan.

6 Likes

Anyone out there want to build the SAFE APP that allows company x to plug in the 50TB SAFEball case they receive in the mail (From us)—> create their SAFE network vault-----> transfer their 50TB of data they want to the SAFEball (which will be self encrypted)----->ship it to our “data farm tractor service”----->we plug it into our 10Gbps upload to the SAFE network.

That way we never have access to their information and they only pay one time to have their private data forever. Those two features will make us instantly better than any cloud service out there. The vault is already built, the self encryption is already built…really it seems to be just some minor adjustments to make it dummy proof plug in play.

4 Likes

I considered the possibility of distributed upload service, but you’d need trusted parties to make sure they will not just sell your data and/or the media, and also I am not sure how you can upload to my account.

Too many problems to solve, it seems.

I don’t see the value - if they can use your box they must have the connectivity.

So can’t they just read a page of instructions that walks them through: create SAFEnetwork account, purchase from you an amount of SAFEcoin which you send to their account, upload their data with a trivial public app (or just using normal tools with their SAFEDrive?).

What’s the box for?

Reselling the bandwidth, the ultimate commodity. Not a good biz to be in.

Let me know when you launch - I’d love to order your box… and not return it.