Cross Compiling MaidSafe Rust Code for ARM

These instructions show how to use Linux (Debian in this case) to build the SAFE Network Rust code to run on ARMv7 processor, such as the Odroid-U3. This involves cross-compiling on the Linux/Intel machine, and transferring the results to the Odroid afterwards.

Pre-requisites

This assumes you’ve already been building the Rust compiler from source (Nightly), and using it to build the MaidSafe Rust libraries on Linux/Intel, so if not you need to do that first. This post about building CRUST that has everything in it you’ll need.

Build Libsodium For ARM

You’ll already have libsodium, but will now need a version compiled for ARM. Below I assume libsodium is in ~/src/libsodium-1.0.3. You’ll also need a cross compiling C compiler such as gcc or clang, plus the cross compiler tools for your compiler.

I have gcc-4.9. To add the cross compiler tools, I did:

sudo apt-get install gcc-arm-linux-gnueabihf

Static or Dynamic Linking?

If you link libsodium statically, you end up with only one file, the executable, to install and run on your target device. Its a bit bigger if you do this, but saves you having to install both the executable and the library needed for dynamic linking. Below are two sets of instructions, 1) Static Linked libsodium and 2) Dynamically Linked libsodium. This is either/or, choose which you want and just do that one.

1) Static Linked libsodium

To build libsodium you need to tell it to use the ARM version of the compiler by setting environment variable CC, and then running ‘configure’:

mkdir ~/src/libsodium-1.0.3/target/arm-unknown-linux-gnueabihf cd ~/src/libsodium-1.0.3/target/arm-unknown-linux-gnueabihf export CC=arm-linux-gnueabihf-gcc-4.9 ../../configure --enable-static LDFLAGS=-static --build=x86_64-unknown-linux-gnu --host=arm-unknown-linux-gnueabihf --target=arm-unknown-linux-gnueabihf

[*Note: I wasn’t able to get libsodium to take notice of the --host directive, which is supposedly all that is needed see issue #274 comment. Instead I set the compiler directly with “export CC=…”, but left the “-host” etc directives I’d been trying in place. Setting CC is not ideal because I haven’t specfied any compiler flags, and I believe it makes the --host, --build, and --target directives redundant. I’ve left them in because this is what I used and it did work! You might like to try without specifying the compiler, and using only “–host=arm-linux-gnueabihf” which is supposed to be what libsodium needs. However, if you do get this working, you may need to use the same triplet (i.e. “arm-linux-gnueabihf”), instead of “arm-unknown-linux-gnueabihf” as I’ve used elsewhere in these instructions. So when you build the rust libraries and maidsafe code for ARM, substituting “–target arm-linux-gnueabihf” instead of “–target arm-unknown-linux-gnueabihf”, and similarly when you create symbolic links etc otherwise things might not match up and you’ll get errors somewhere. I ended up using “arm-unknown-linux-gnueabihf” because that’s now the convention for configuration names, and is what was given in the examples I found for compiling Rust for ARM, and “arm-linux-gnueabihf” wasn’t working in libsodium anyway.]

Build libsodium for ARM:

make v=1

The “v=1” is not needed but tells make to be “verbose” and so is more informative. If make succeeds, you will now have created static library as “libsodium.a”. For Rust to find and link with it, create a symbolic link from /usr/local/lib to the file. Like this:

sudo ln -s $HOME/src/libsodium-1.0.3/target/arm-unknown-linux-gnueabihf/src/libsodium/.libs/libsodium.a /usr/local/lib/libsodium.a

2) Dynamically Linked libsodium

To build libsodium you need to tell it to use the ARM version of the compiler by setting environment variable CC, and then running ‘configure’:

mkdir ~/src/libsodium-1.0.3/target/arm-unknown-linux-gnueabihf cd ~/src/libsodium-1.0.3/target/arm-unknown-linux-gnueabihf export CC=arm-linux-gnueabihf-gcc-4.9 ../../configure --build=x86_64-unknown-linux-gnu --host=arm-unknown-linux-gnueabihf --target=arm-unknown-linux-gnueabihf

[*See the note under 1) Static Linked libsodium (above) regarding --build, --host and --target directives.]

Build libsodium for ARM and install it in /usr/lib:

make v=1 sudo make install sudo ln -s $HOME/src/libsodium-1.0.3/target/arm-unknown-linux-gnueabihf/src/libsodium/.libs/libsodium.so /usr/lib/arm-linux-gnueabihf/libsodium.so sudo ln -s $HOME/src/libsodium-1.0.3/target/arm-unknown-linux-gnueabihf/src/libsodium/.libs/libsodium.so.13 /usr/lib/arm-linux-gnueabihf/libsodium.so.13

At this point you’ve build libsodium, either as a dynamic or a static library, and created symbolic links so the rust compiler can find it.

Before you can compile with Rust for ARM you will also need to build the std Rust library crates for ARM because these are not shipped with the compiler, as follows.

Get/update rust nightly from github so you can build the std crates for ARM

cd ~/src git clone https://github.com/rust-lang/rust.git cd rust

[For information only: the instructions below are adapted from: https://github.com/japaric/ruststrap/blob/master/1-how-to-cross-compile.md. See also: GitHub - japaric-archived/ruststrap: [SUPERSEDED] by https://github.com/warricksothr/RustBuild]

This next bit takes a while to compile, so if you have 8GB or more RAM on your build machine try “make -j4” instead of “make” in what follows, as this will speed compilation by using multiple CPU cores:

cd ~/src/rust ./configure --target=arm-unknown-linux-gnueabihf,x86_64-unknown-linux-gnu make sudo make install

Now install your ARM Rust libraries with the rest of the Rust compiler libraries:

sudo ln -s $HOME/src/rust/arm-unknown-linux-gnueabihf /usr/lib/rustlib/arm-unknown-linux-gnueabihf

At this point everything is done. The rest of this post explains how to cross compile some test code and try it out on an ARM device, and then how to build a maidsafe example and try that out as well.

Try Cross Compiling with rustc

You should now have a rustc installation that can cross compile for ARM, so let’s test it out.

Create a test file “hello.rs” containing:

pub fn main() { println!("Hello, world!"); }

Tell rustc the name of the cross-compiler (which must be in your PATH), and compile it:

$ rustc -C linker=arm-linux-gnueabihf-gcc-4.9 --target=arm-unknown-linux-gnueabihf hello.rs

Check it produced an ARM binary

$ file hello hello: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=8ecf390faf2b65d29d3292a3cb78d9ada788d6c0, not stripped

Looking good :smile:

Check it works on your ARM device:

$ scp hello me@arm:~ $ ssh me@arm ./hello Hello, world!

Cross Compiling with Cargo

Now let’s build with cargo, but we can specify a different linker for each target architecture using ~/.cargo/config

Create ~/.cargo/config containing

[target.arm-unknown-linux-gnueabihf] linker = "arm-linux-gnueabihf-gcc-4.9"

Test with Cargo

$ cargo new --bin hello $ cd hello $ cargo build --target=arm-unknown-linux-gnueabihf $ file target/arm-unknown-linux-gnueabihf/debug/hello target/arm-unknown-linux-gnueabihf/release/hello: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, BuildID[sha1]=8088b28a2818b4a2c55c3638b71d2f805833e17f, not stripped

SUCCESS!

Ok, so we seem to have everything working to build for ARM with Rust. Finally we are ready to build the SAFE Network code for ARM.

Building A MaidSafe Example

This will work in your existing MaidSafe directories, but this example clones the code from github for a fresh start:

mkdir ~/src/maidsafe-rust cd ~/src/maidsafe-rust git clone https://github.com/maidsafe/self_encryption cd self_encryption cargo build --target=arm-unknown-linux-gnueabihf

That should build the library (in ~/src/maidsafe-rust/self_encryption/target/arm-unknown-linux-gnueabihf/debug).

Now build an example - something we can run on the ARM device.

cargo build --target=arm-unknown-linux-gnueabihf --example basic_encryptor

Finally try the example out on your ARM device (e.g. Odroid-U3):

Test MaidSafe Example on Odroid / ARM

Still working on the Linux/Intel build machine. In the following example, the Odroid is on the local network at IP address 192.168.43.46, and the user name for it is “odroid”.

Copy libsodium to Odroid…

scp /usr/lib/arm-linux-gnueabihf/libsodium.so.13 odroid@192.168.43.46:~

Copy basic_encryptor example to Odroid…

scp ~/src/maidsafe-rust/self_encryption/target/target/arm-unknown-linux-gnueabihf/debug/examples/basic_encryptor odroid@192.168.43.46:~

Login to Odroid…

ssh odroid@192.168.43.46

Install libsodium…

sudo cp libsodium.so.13 /usr/lib

Create test file, encrypt it, decrypt it, and compare…

man ls > ls.txt ./basic_encryptor -e ls.txt ./basic_encryptor -d data_map blah.txt diff ls.txt blah.txt

If you list the directory you’ll find the data_map file and a directory called chunk_store_test. The directory will contain the encrypted chunks of “ls.txt” and the data_map contains the self_encryption map needed to decrypt them.

ls.txt and blah.txt should be identical, so diff will not produce any output.

17 Likes

Thanks very much!

You’re doing a great thing here, explaining it to us!

2 Likes

Brilliant Mark,the only thing perhaps to add is that if you compile libsodium statically then you do not need to copy .so files around with your apps (static compile will produce a .a lib). It will link statically into the app as well. Then you have a single app that contains everything you need. ( ./configure --enable-static LDFLAGS=-static --build=x86_64-unknown-linux-gnu --host=arm-unknown-linux-gnueabihf --target=arm-unknown-linux-gnueabihf )

2 Likes

Thanks David, I have some more testing and tweaking to do with cross-compiling libsodium thanks to some help from Frank, it’s maintainer: I’ve messed up the --host etc flags which is why I ended up setting CC.

I’m also not sure if I need any extra instructions to help people get gcc set up as I was doing other stuff before I tried this route, but if people hit problems I can probably figure out what is needed from my notes.

Thanks for the static linking tip. I’ll try both changes out and update the post. You do not know how amazed and chuffed I was when this ran on Odroid. :smile: My first cross-compilation!

4 Likes

Would be very interesting to build the SAFE Network Rust code to run on MIPS processor too.

In this way we could integrate in OpenWRT and be able to use routers as SAFE nodes.

A simple and inexpensive way to extend the SAFE network.

For now I found this.

4 Likes

@dirvine @anon86652309 Cross compiling for ARM: I got static linking working and have updated the OP but am now trying to cross compile maidsafe_vault and maidsafe_client without success :frowning:

I wonder if you have any clues about this as I’m stuck. I can still build the basic_encryptor example (release or debug) for ARM.

With maidsafe_client or maidsafe_vault, if I try this I get the same error, in one case with libcrypto, the other libtime:

$ cargo build --release --target=arm-unknown-linux-gnueabihf --example rest_api 
   Compiling maidsafe_client v0.0.1 (file:///home/mrh/src/maidsafe-rust/maidsafe_client)
error: linking with `arm-linux-gnueabihf-gcc-4.9` failed: exit code: 1
note: "arm-linux-gnueabihf-gcc-4.9" "-Wl,--as-needed" "-L" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/examples/rest_api.o" "-o" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/examples/rest_api" "-Wl,--whole-archive" "-l" "morestack" "-Wl,--no-whole-archive" "-Wl,--gc-sections" "-pie" "-Wl,-O1" "-nodefaultlibs" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/libmaidsafe_client.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libmaidsafe_types-e022f498f2e0be44.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libself_encryption-86be9d88599c4863.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libcrypto-093a8ad0ae39b61a.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/librouting-438939270f2da06f.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libmessage_filter-c4297ec9773f359a.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libaccumulator-4ea207c6b4eb723c.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/liblru_time_cache-7fa82329a30ad63b.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libsodiumoxide-75eb99c13a8dec07.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/liblibsodium_sys-230d4350cd007c4b.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libcrust-c59c517caf52ab64.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libcbor-0bd941ffbb5daf77.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libbyteorder-399c175f6a7726ac.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libutp-d5f3579e915156bb.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libnum-7ad397ad0b46ae38.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/librand-de6cdb9e4fd93d55.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libtime-10c3f659b0cce9f1.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/librustc_serialize-c1e8163a38ed3d54.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libitertools-c093e3b3d8671961.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/liblog-8a6aba167994951e.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libasynchronous-5f0cce4e19357a9a.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libnum_cpus-16707c6acca9fe91.rlib" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/liblibc-ef5cbad4ef5c7a1e.rlib" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/libstd-7e44814b.rlib" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/libcollections-7e44814b.rlib" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/librustc_unicode-7e44814b.rlib" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/librand-7e44814b.rlib" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/liballoc-7e44814b.rlib" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/liblibc-7e44814b.rlib" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib/libcore-7e44814b.rlib" "-L" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release" "-L" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps" "-L" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/build/rust-crypto-093a8ad0ae39b61a/out" "-L" "/home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/build/time-10c3f659b0cce9f1/out" "-L" "/usr/local/lib/rustlib/arm-unknown-linux-gnueabihf/lib" "-L" "/home/mrh/src/maidsafe-rust/maidsafe_client/.rust/lib/arm-unknown-linux-gnueabihf" "-L" "/home/mrh/src/maidsafe-rust/maidsafe_client/lib/arm-unknown-linux-gnueabihf" "-Wl,-Bstatic" "-Wl,-Bdynamic" "-l" "sodium" "-l" "rt" "-l" "c" "-l" "m" "-l" "dl" "-l" "pthread" "-l" "rt" "-l" "gcc_s" "-l" "pthread" "-l" "c" "-l" "m" "-l" "compiler-rt"
note: /home/mrh/src/maidsafe-rust/maidsafe_client/target/arm-unknown-linux-gnueabihf/release/deps/libcrypto-093a8ad0ae39b61a.rlib: error adding symbols: File format not recognized
collect2: error: ld returned 1 exit status

error: aborting due to previous error
Could not compile `maidsafe_client`.

ASIDE: I tried cutting and pasting to run the “rustc” command as shown with --verbose, and removing libcrypto from it but strangely this had no effect at all.

I’m guessing either the libcrypto-093a8ad0ae39b61a.rlib file is the wrong architecture, or the wrong ld is being called, but everything suggests they are ok. I’m not sure how to check the architecture of an .rlib file as “file” just reports “current ar archive”.

I’m wondering if there’s something about certain crates that gets in the way of them cross compiling, but it seems doubtful, especially as compiling the basic_encryptor example is not that different, yet works.

On a side note Mark, rust now has musl capabilities. So we will be able to cross compile fully static (i.e. 100% static) so this is gonna become very much simpler now.

For this error rust-crypto is now using asm code which makes a lot of platforms fail (just no asm there yet). We are factoring that out.

1 Like

Ah, OK, how about libtime?

See this for all the changes (you can click through to musl). Rust 1.1 stable, the Community Subteam, and RustCamp | Rust Blog

The binaries will be 100% static with no dependencies at all :slight_smile: It’s a big achievement the rust team have done there. It means you can compile on the latest version of linux and run code on the oldest version you can find. That is pretty big.

4 Likes

Thanks David, I see you sent me the instructions for building rustc to support musl:

If you guys are going to hit this very soon I’ll wait as I’m off on hols at the end of the week, but I won’t let that stop me participating in the first networks! :smile: So if building myself is feasible and the only way to participate I’ll definitely try to do that.

Thanks again for responding.

2 Likes

I think we will be doing this Mark, it makes perfect sense as it will save atone of time in packaging for linux. So enjoy and leave this one to us. We will shout when it’s done.

3 Likes

Okayyy :-). Expect pestering to resume on the API then lol :wink:

3 Likes

Hi, I decided to have a go at this. A year has past since this post and the tools are a little more streamlined now, mainly that you don’t need to recompile Rust. I learned quite a bit. I have now produced a cross-compiled ARM binary of safe_vault. It contains a static libsodium and dynamic links to the glibc library, which is as portable as it gets for ARM on Rust, with current methods. At least I think that’s what it is… ldd does not correctly report the dynamically linking for cross-compiled binaries while they are on the host system - they need to be on the guest system to get that information. It was certainly built with a static libsodium library however. The “file” command does report that it is dynamically linked, so I’m reasonably sure that it is what I expect it to be.

I believe it is a drop-in replacement for the ARM7 Maidsafe release.

I don’t have access to ARM hardware at present, and creating a virtual machine under Qemu on Debian almost got me to the point of testing it but not quite, and I’ve run out of time.

So I would be grateful for someone to download it and play with it. Since the seed vault of community1 is currently participating in Testnet4, that isn’t available to test on. You might try attaching this safe_vault to Testnet4 or the stable droplet network. I’m sure you all will work something out. :slight_smile: Please let me know how you get on.

It is here: http://91.121.173.204/armv7/safe_vault

Incidentally, an x86_64 (aka amd64) Linux version, suitable for any 64-bit Linux computer with an Intel/AMD CPU, is here: http://91.121.173.204/amd64/safe_vault That is 100% static linked.

I will shortly put up a number of these, being automatically built every night from the current repo. Next will be builds for 32-bit Linux, Windows and OSX. So people will be able to run the bleeding edge build appropriate for their systems, if they so choose.

4 Likes