Deterministic builds

This is a test to see if building safe_vault on different linux distros is deterministic or not.

The builds are not identical, which is not what I hoped but at least we know.

The c compiler is the variable that isn’t controlled well enough. Unfortunately this is also an extremely complex component so going into the details of my exploration in this post is not going to be helpful! (Some leftover notes about it appear at the end of the post).

Good news is no more openssl dependency so that simplifies things quite a bit.

Also good news is the build is repeatable, ie repeating these steps produces the same result every time.

Gotta say, xorurls are going to be a damn blessing. Truly universal and consistent management for remote dependencies is going to be pretty epic (as guix and nix are beginning to show).


Build on AWS Ubuntu AMI

# install rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.profile
rustup target add x86_64-unknown-linux-musl

# install gcc so musl-gcc can be built from source
# this is the variable that needs better control
# but building repeatable copies of gcc from source is no easy task
# This installs gcc
# gcc --version
# gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
sudo apt-get update
sudo apt-get install build-essential

# install musl-gcc
wget https://musl.libc.org/releases/musl-1.2.0.tar.gz
tar -xvzf musl-1.2.0.tar.gz
cd musl-1.2.0/
./configure
sudo make install
export PATH=$PATH:/usr/local/musl/bin

# install and build safe_vault
cd ~
git clone https://github.com/maidsafe/safe_vault.git --depth=1
cd ~/safe_vault/
cargo build --release --target x86_64-unknown-linux-musl
sha256sum target/x86_64-unknown-linux-musl/release/safe_vault
28b09022a93054541ce4fc889ef14370f0e566bcada79ad297a4f08a8854b2d4

Build on AWS Linux 2 AMI

# install rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source ~/.profile
rustup target add x86_64-unknown-linux-musl

# install gcc so musl-gcc can be built from source
# this is the variable that needs better control
# but building repeatable copies of gcc from source is no easy task
# This installs gcc
# gcc --version
# gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-6)
sudo yum groupinstall "Development Tools"

# install musl-gcc
wget https://musl.libc.org/releases/musl-1.2.0.tar.gz
tar -xzf musl-1.2.0.tar.gz
cd musl-1.2.0/
./configure
sudo make install
export PATH=$PATH:/usr/local/musl/bin

# install and build safe_vault
cd ~
sudo yum install git
git clone https://github.com/maidsafe/safe_vault.git
cd ~/safe_vault/
cargo build --release --target x86_64-unknown-linux-musl
sha256sum target/x86_64-unknown-linux-musl/release/safe_vault
d5507ccc901df0ec4e5c8f4d23322750625e2fd4e9aaf78d564978ca6a19aebb

Notes on installing gcc from source (which depends on having an existing gcc installation)

I had hoped to build gcc-7.5.0 from any existing version of gcc, then use gcc-7.5.0 to build musl-gcc to give a consistent copy of musl-gcc, but it did not. The original prebuilt gcc matters.

# instructions for AWS Linux 2 AMI
sudo yum install gcc gcc-c++
wget https://ftp.gnu.org/gnu/gcc/gcc-7.5.0/gcc-7.5.0.tar.gz
tar xzf gcc-7.5.0.tar.gz
cd gcc-7.5.0
./contrib/download_prerequisites
cd ..
mkdir objdir
cd objdir
$PWD/../gcc-7.5.0/configure --prefix=$HOME/GCC-7.5.0 --enable-languages=c,c++ --disable-multilib
make -j 2
make install
# gcc --version will still show the prebuilt version
export PATH=$HOME/GCC-7.5.0/bin:$PATH
# gcc --version should now show 7.5.0
8 Likes