Which OS for farming?


#1

Hello!

I just exchanged for a Mac Mini that I intend to use for farming when the time comes. I was considering partitioning a couple 20gb partitions to allow for a few different OSs. I was surprised to discover that it would only allow me to create a 2nd partition, but I believe I can still do what I intended, & I’m looking into how to do this now… when it occurred to me that I might as well ask in case this is already known. Is there one OS that would be particularly advantageous? One of the reasons I decided to pick up this Mac Mini is the Thunderbolt port. I’d like to maintain the use of it to add a drive at some point, assuming it will allow for the fastest transfer rate.

Thanks!


#2

Maidsafe algo shouldn’t favor any particular OS, it would be awful if it did.

What the devs build on (OS X?) should be safe & easy (especially if non-OS X can’t make best use of Thunderbolt?), and I assume Ubuntu 14.04 x64 as well, but I’m saying this just because you disclosed you have OS X.

I am not giving any significance to the choice of OS, I’ll install whatever is easiest and cheapest.


#3

Keep an eye on FreeBSD, Niall who is a cored dev here is very keen on it ;):



#4

Thank you for the tip. I will educated myself further on the OS.


#5

FreeBSD, does that still suffer from crashing when you unplug a USB device? :slight_smile:


#6
  1. Shouldn’t the write performance be unimportant? That is, if you hit the jackpot and get a block to save, it doesn’t matter how fast you do it. So far I have not seen any mention of the importance of the write performance.
  2. Caching on vaults will be of little importance. On vaults, cache hit rates will be extremely low - virtually zero. And whatever actually gets accessed multiple times will be cached on caching nodes, and not on vaults.

I am convinced that manageability and simplicity will be play a relatively larger role.


#7

Indeed I am. Last month or two I’ve built out scripts for a home NAT network simulator where it will create N “home computers” behind M “home ADSL modems” on FreeBSD though the mass creation of appropriately configured BSD jails. This could be used as part of automated functional testing of the SAFE network as it would be routed from home users as currently we have no facility for doing that.

The same thing could have been built using OpenVZ on Linux, but then you need a custom kernel, and it’s still a 2.6.34 Linux kernel at that. Plus veth config on OpenVZ is a black art.

Only if it’s a USB3 device :smile:

No, seriously that’s long fixed now. If you use the PC-BSD distro (http://www.pcbsd.org/) I think you’ll be very pleasantly surprised. Video cards now “just work”. It’s the only OS which isn’t slow on a magnetic hard drive too (if you add a SSD based ARC cache and throw RAM at ZFS). It’s noticeably snappier than Linux to use on the same hardware. Hardware driver availability is still a big problem, but even stuff like my Logitech webcam or my USB 11n wifi works straight out the box. Older, popular devices tend to be very well supported, newer devices (e.g. my Intel 7260 wifi) might be very popular but no drivers yet (though they’re coming).

For me the only thing in the way of it replacing Linux for me is the lack of good power management. FreeBSD does have power management if the hardware is ACPI compliant, it’s just a lot isn’t. Plus anyone with a Radeon graphics card can’t suspend the PC :frowning: and I like suspending my work each night.

Depends if we fsync written chunks or not. Currently we don’t.

You’re right my earlier statements were made pre caching nodes.

Also, of late ZFS on Linux has become fashionable. Taints the Linux kernel so the purists will never like it, but a lot of distros don’t care. I can see a future where ZFS on Linux will be the most popular ZFS config.

Niall


#8

I feel the same - whatever works is fine. I thought one of those free NAS projects has it integrated on Linux.

What I meant by “manageability and simplicity” is if one could use the same OS as they already have (e.g. OS X on OS X, or maybe Ubuntu Docker containers on Ubuntu) that would be attractive to a lot of hobby farmers.

I run bitcoin servers at home and have couple of other things going on and it’s a pain in the ass to manage (free space, security, etc.) and maintain (updates, upgrades) that stuff. If one spends 1 hour per week on managing his infra and makes less than $100 a month from farming then once you deduct the cost of h/w and electricity it’s almost like working for a minimum wage. In that situation key is to cut down on monitoring and maintenance, hence the need for simplicity.

Docker is impressive, although it has certain overhead. However from what I’ve read here it seems the devs are determined to not reward only the best performers so it’s possible that Docker’s overhead won’t matter much.


#9

I’d certainly like to give FreeBSD a try on a dedicated server if we had some straight forward instructions for configuration and containerization.


#10

So one could try out FreeNAS if it’s just for farming. It’s pretty easy to set up http://www.freenas.org/


#11

Yeah, that’s the one.
You’d have to be careful, though, with FreeNAS updates. I don’t know how they work (it’s been a while since I used it), but if it has dependencies that conflict with MaidSafe, you might need to be extra careful as you upgrade the both.
Or you could (more complications, slightly more overhead, more expensive, easier to break…) create iSCSI targets on FreeNAS (they’d “live” on ZFS) and run MaidSafe on an iSCSI client such as Raspberry Pi or a similar weakling.


#12

I know docker is not available for FreeBSD (and I am not an FreeBSD expert at all) but could one run the maidsafe core components in a container or FreeBSD jail http://www.freebsd.org/doc/handbook/jails.html ?

That way you can run it on FreeNAS and even if the underlying packages get updated, it won’t intervere with MaidSafe as all dependencies would be in the container.

Right?


#13

The fact that you have to think about goes to show that it’s not a non-brainer approach (which should be concerning if your goal is simplicity).

Now, related to the actual question, jails are different and from my recent experience with FreeBSD some packages are really outdated and it requires a fair amount of monkeying around to install new software on it. I eventually managed to do it, but not being a BSD expert and not wanting to become one, I decided to use Ubuntu 14.04.

Someone may find the appeal of ZFS (or whatever other features of BSD) worth it, others won’t. My preference is to take the Linux penalty hit (assuming it exists) and spend saved time on other things. If you do a lot of farming, a 5% gain can get you a free rig after each 19th rig you buy. I know I won’t have more than 1 or 2, so …


#14

I read you can run ZFS on Ubuntu. Wouldn’t that give you the best of both world?


#15

What’s very cool as well is that Synology is now going to ship with docker installed and thus addons will be build as docker containers. Making a maidsafe image and put it in their appstore should be trivial. https://www.synology.com/en-us/dsm/5.2beta/

Synology DSM is Linux and defaults to ext4, the new Atom power devides with AES-NI intrsuctionset should be a good device for farming since it’s on by default and most have some free space to share.

Synology has a huge install base and thus it could be a very good starting point to get maidsafe into the homes.


#16

It likely would. One of my concerns is that I’d need multiple disks to take advantage of its s/w RAID implementation and possibly SSD to take advantage of other nice features.

My basic logic here is if my standard Ubuntu setup is 5% slower than ZFS, I’ll still do just as well as long as MaidSafe round-robins PUT and GET requests to all vaults that aren’t total crap. If this assumption is wrong, then ZFS won’t help you, because in that case only high end setups (which may involve ZFS, of course) strategically colocated in fast, low-latency datacenters would farm SAFE (I wrote about this reasoning in Feasibility of datacenter farming (and the risk of farmer centralization)). That would be a whole new world and you could basically retire your homegrown rig at that point.
That’s why I intend to stick to the KISS principle (say, Ubuntu 14.04 containers on Ubuntu 14.04, few non-RAID-ed HDDs) until I am proven wrong.

Could be.
I have one of those small NAS boxes and those are quite useless (ARM CPU and 512 MB RAM - pretty useless), so as long as it’s got at least 1-2 GB of RAM and a better processor, that could work.


#17

Any updates to this at this point? Might FreeBSD still be potentially advantageous?

I was wondering, could a distros of FreeBSD be made specifically for farming? Anything to make things simpler. :smiley:


#18

Linux, of course.

Not only is it the most performant OS out there, it also does something to further the common good of humanity, not shareholders.


#19

Synology could be a great point of access since most people who own a Synology have hundreds of GB free space and the device is running anyway so to them it would be a win-only situation. However packages which are published in the App Store of Synology aren´t opensource anymore or are they?


#20

Anyone familiar with Qubes OS? I’m intrigued, and was wondering if it could be used for farming, and if so, if there would be a benefit to it.