What machine configurations will be best for farming


#1

@chrisfostertv said:

Why? because it seems if were going to share big hard drives, it may be more efficient (and profitable) to slice up the drive and assign a vault to each slice of that space.

I’ve been wondering about this, can you elaborate your thinking? Perhaps best over in a farming setup topic, but if you just want to reply to this I’ll move it over.


Barebones linux image for dedicated farmer nodes
#2

Assigning partitions/volumes to vaults


Docker containers ala SAFEcontainers

ZeroVM could also be considered for multiple vaults assigned to one HDD


Introduction to ZeroVM

What is ZeroVM?

ZeroVM is an open source virtualization technology that is based on the Chromium Native Client (NaCl) project. ZeroVM creates a secure and isolated execution environment which can run a single thread or application.

ZeroVM is designed to be lightweight, portable, and can easily be embedded inside of existing storage systems. This functionality allows developers to push their application to their data instead of having to pull their data to their application, as is the case with more traditional architectures.

More Details

While ZeroVM itself provides only raw execution environment, the ZeroVM Run Time (ZRT) includes a port of glibc and an in-memory file system.

This provides a C99 compliant environment supporting most of the POSIX syscall API. ZeroVM doesn’t expose any non C99 or non POSIX API.

All ZeroVM magic is handled transparently to the application. True to POSIX/UNIX traditions, all IO to and from ZeroVM is modeled as files. Input data is presented to application as STDIN, log as STDERR and output as STDOUT.

Communication channels with peer ZeroVM instances are also presented as files. The rest of the visible file system is all transient and memory-backed in the current implementation.

The standard C99 library and a major part of POSIX are available; however, there are some behavioral deviations from what would be expected as a “normal” implementation.

For example, since ZeroVM is deterministic, time functions always return zero. (We assume this is within the C99 standard.)

This could be interpreted by the application as if it is running on an infinitely fast computer. Threading is cooperative (handled automatically) and deterministic, hence all thread synchronization primitives are NOPs.

Developing applications for ZeroVM requires use of the provided cross-compilation GNU toolchain.

Docs


#3

It’s not that straightforward. "It depends."
As David said it depends on many factors.

Personally I wouldn’t do it simply because it’s complicated and it’s completely unclear whether in any given case it’s better to have X than Y. Even for the same h/w configuration it will depend on the bandwidth and reward schema (partitioning will increase response time; it will happen differently for each individual configuration and network connection and we have no clue how badly latency will be penalized).


#4

Interesting idea. More ID, more chance to be other node’s group member.


#5

Eventually the idea is to have SAFE running in every nook and cranny of available resource.

Can we leverage free resources that are on offer.

Just think right now, most ISP’s give a small slice that is offered as a website…100mb or something silly…I’ll throw a vault on there if possible.

What about all that free drive space thats on offer out there. Some services offer connection via an API. It would be nice if a SAFE client had bindings into a bunch of these services…so you’d use your local ram/cpu to put and get to those freebie resources.


#6

Not sure if I’m off topic…

I’ve been looking at the Intel Edison; it is a $50 embedded system running a 500 MHz Intel Atom processor.

It can be hooked up to an external hard drive over USB 2.0, powerful enough to run MaidSafe, and it’ll use very little electricity.


#7

No Ethernet, no Sata …but looks like it runs 64bit linux, so will run Docker

Edison Forum



I’m partial to the MinnowBoard MAX-Single Core at this stage @ $99



#8

Nice. The MinnowBoard seems like a promising low-cost dedicated platform.

Yea, the Edison is missing a few things. No Ethernet, but it does have WiFi. No SATA, but external hard drive over USB 2.0 is still an option. Separate development board means additional cost.


#9

Also look at the Odroid series from http://hardkernel.com.

I have a couple of Odroid-U3 ($65 with 2G RAM) with big USB disks ready to go so it’s easy to get SAFE running. They provide Ubuntu 14.04 on eMMC (eMMC’s are up to 64G, so you could run a very low power vault without a disk). There’s also a really nice little case, and lots of other add-ons, and a decent forum online to get help (as well as folk here).

If you want SATA there are higher spec models, but you pay a bit more.

I’m not convinced vaults need a lot of CPU or disk bandwidth, and power consumption is also important.

Also consider cost, because to remain viable for farming, hardware may need to be replaced at regular intervals or old vaults will lose out to faster newly purchased hardware.


#10

The Odroid seems to use a Samsung Exynos processor, which I believe is based on the ARM architecture. To get MaidSafe to run on the ARM architecture, some extra work may be required; while the Intel Atom processor is x86 compliant.


#11

You are OK we work fine on arm :slight_smile: We compile for it and run CI tests etc. on a bunch of these. Mark uses one solar powered as well :smiley:

@Viv is also doing some cross compiling for arm for mobile API’s at the moment. It may lead to a pretty big change but donkey work in our serialisation scheme. If so its fine as it wont take long (and can be done in parallel with launch) and may potentially reduce codebase by another 30% (a very good thing). We will see soon.