Massive scale maidsafe farm

There is a bunch of video faqs being published now ( @ioptio and @frabrunelle did them with @Shona editing), that will help then lets jump on skype or some other snooping network and let everyone hear :smiley:

4 Likes

Feel free to edit the planning doc with any tasks you feel are important, I’ve a couple of hours before some supporters arrive to meet with me;
So after the weekend will be the best time/even sunday evening; I work ultra late and as much as I can handle so time differences are not much factors here.

1 Like

I added some stuff there for you to delete/edit/play around with. Just a quick splurge so no worries it should not be kept, but may help with some ideas at this early stage of collecting thoughts.

1 Like

I think we’re still on topic: the title says “Massive scale maidsafe farm”
What a great way to get there, by telling everyone about it :smiley:

there = Massive scale maidsafe farm;

2 Likes

It would be sort of like a 51% attack, though closer to a 76% attack in the SAFE network. There would possibly be very minor data loss at something less than 76%, but not significant, I think.The difference being that a 51% attack would seek to gain some control, whereas the 76% attack would be purely distruptive, as no degree of control could be gained.

The main interest here to me would be to see how the network adapted to such a massive churn event. I’m guessing that it might slow the network some, but it should recover. Multiple such events in succession could mess things up, I’m guessing, but the neat thing is that the the network ANT workers would just keep on doing their simple tasks as presented to them, till it stablized. Then whatever the condition, it would move forward, loss or not.

Anyway, that’s just my non-developer takeaway from all I’ve been able to glean, but it’s an interesting image.

3 Likes

As an interjection to this thread, I think it would be a good thing to note post 4 of this thread MaidSafe Korea: Updates and Questions - #3 by happybeing where David Irvine lays out his hopes regarding the distribution of the network and farming, especially as regards big rigs.

2 Likes

haha dont worry about that. Building a massive storage farm is a whole different game then running scrypt mining rigs in your basement :wink:

I’m still not convinced on how I would not get an unfair advantage over home farmers.
Peering in a datacenter will get me much lower responsetimes over average Joe with his 4 yr old crappy sweex nat router.

I’m not sure how maidsafe will handle this, but using thin provisioning on my backend storage server will also fool the network into thinking I have petabytes and petabytes of storage I don’t really have.

What might be even worse, if I can get this thing to compile on Solaris I can run 128 hardware threads + zoning to fire up thousands of maidsafe instances on a $1000 Sun T5240. All with the competitive advantage of being peered to a number of fast backbones.

(All this is just theoretical btw, not planning to make a setup like this, I think it’s a better investment to just buy maidsafes and storj’s and wait till the adult industry finds out about cheap storage)

1 Like

That’s performance advantage. Read the two posts @fergish kindly shared, which explains why it might not be to your financial advantage.

I think what I don’t understand yet is how the network might punish temporary farmers. I’m sure even with just the geeks, freaks and volunteers together the network will be enormous, but there has to be some kind of incentive for Joe to leave his computer up and running 24/7. I mean if I can plug a couple of usb hd’s into my dd-wrt router I’d be fine with it, but to leave my computer on all the time (and without rebooting) I would need to earn some kind of incentive. (Especially windows users with the beloved “Important updates have installed and your computer will reboot in 15 minutes whether you like it or not” will have a disadvantage).

I’m also not sure what happens to the maidsafe data on my drive when I have to reboot. Will there be a kind of hashing to see which data is still valid or will all just be gone (including reputation) ?

Power is expensive where I live. Running a simple, not too fancy computer 24/7 costs me 50 euro’s per year.
Other regions will be cheaper but their internetconnection will probably suck compared to mine.

Apologies if this is all covered in the whitepaper, but unfortunately I’m not smart enough to understand half of it.

Depends on what you mean by “temporary farmers”. You won’t be penalized much if any for rebooting or other short-term offline conditions.

If you are online for a while and get data stored in your vault, then go offline for an extended period, your vault will be downranked and become less able to earn safecoin, while being always available and responsive will give your vault a higher ranking and earn you more safecoin.

The earning of safecoin IS the incentive to leave the computer on. My understanding of the vault software is that it will run at a low resource level. No reason to keep a monitor on while not in use, etc.

See Best Safe Node hardware for more data than you can probably use, but a pretty good coverage of some possibilities.

If your desktop is a powerhog, that’s another matter. I’m setting up a couple nodes that consist of low-power Intel Atom processors in netbook and thin client configurations that use very little energy. Others are using Odroid, or even raspberry pi. These can be gotten cheaply if you want to run a dedicated farmer with a low resource footprint.

If your desktop computer is what you want to use for only when you are online, or doing other things (image editing, gaming, web surfing, etc.) you can run a vault on it to access the SAFE network, and turn it off when you’re not using it. It just won’t earn much safecoin in that instance. But maybe it will still earn some, even then.

Hope that helps.

It helps a bit, thank you :smile:

Still wondering if my $1000 128 hardware threaded sparc beast is cheaper then 100’s or 1000’s of raspberries or o-droids delivering similar performance… But I could never compete with integration in dd-wrt and/or open-wrt *HINT HINT DEAR DEVS :wink: … The network could scale up to bizarre proportions in short time if their userbase decides to earn some coins and plug in a spare usb disk to their already running and paid for router.

The reason why I keep going on about this is because I see massive potential both for maidsafe and storj and I’m still trying to wrap my head around most of the concepts.

The one thing every IT manager worldwide has in common is that they wake up with storage costs nightmares 2-3 times a week and crying themselves to sleep again. The last place I worked at grew from 70 tb of managed storage to 700tb in just 3 years. Storage is a growth limiting factor nowadays because setting up reliable storage isn’t that easy. The price of a TB of enterprise grade storage can easily be $1000-$2000 per terabyte per year if you calculate the san switches, sfp’s, support contracts, licences, downtime, cables, hba’s, people costs and so on and on… The need for storage grows way faster than the size of harddrives (I believe only 20% growth in size per year on average for enterprise class drives).

Now I’m not expecting to see for example netflix ditch amazon anytime soon. But some teenager somewhere will at some point code a competing delivery platform with just $100 investment. Nowadays you’re just out of luck if you need lots of data for your idea…

1 Like

I’ve said before, and David Irvine has agreed, that rigs such as yours will not hurt the network, and if they work economically for you then there’s no downside. It could even be very beneficial to the network. As long as enterprize participants are stable actors, they’ll benefit from the arrangement. They won’t hoover up undue proportions of safecoin, however. They’ll find that it is a good mutual relationship.

1 Like

Hey @Erwin … kickin up some dust and scouring the forum for participants looking at farming. Appears you had some monster rig in mind. Heads up fast forward 2 years to 2016 - check out the current state of affairs.

3 posts were merged into an existing topic: MaidSafe Dev Update – October 4, 2016 – TEST 10

Looking back a couple years the enthusiasm for farming seemed high. A simple strategy to re-engage old interest is efficient and effective. While it may remind some how long its been, it will also remind them how far its come. This can be applied to many other threads.

i plan on using my 8 core desktops. 4tb hgst drives that arent cheap but are very reliable, and some icybox 10 bay,usb 3, for overflow drives.

4 Likes

Slightly curious as to the effects of node rank when we look at large amounts of volatile memory and how the vaults handle the use of this memory rather than archiving information right away?

Overall would it be good to have a boatload of RAM and will it be used efficiently by vaults? Personally im considering running drives in RAID on a FTTH connection. If loads of RAM would help for caching on the network then il get a load of it also…

2 Likes

My understanding is from reading the info about caching is that each node will have upto a certain amount of cache. So I think its better to be running multiple nodes (vaults) so that you can maximise the amount of memory used for caching

3 Likes

@BIGbtc Sorry I stepped out for a while :slight_smile:

Currently caught up in other things so I don’t really have the time to get a farm going. Also I started work at one of the largest san storage vendors in the world so I guess my boss wouldn’t appreciate my conflict of interests :wink:

1 Like