Basic self_encryptor example DIY instructions

Development
104

This is the exact reason we want to distribute apps and examples as we go. It will al become clear as you see these deficiencies disappear as we release more., We are trying to create the perfect storm and this is a great start. I think very quickly we will all see this and it is compelling.

3 Likes
105

Awesome. Much more success with this than with crust. Encrypted, decrypted, got my file back. So painless.

The only confusion for me, was that I ran the encoder inside of the self-encryption repo folder, and I didn’t twig that chunk_store_test and data_map weren’t just standard issue files in the repo. So I spent a few minutes failing to find the bloody map :expressionless: Heh. But after I realised, everything works perfectly.

I am impressed.

5 Likes
106

.1. If SAFE works perfectly, then I wondered there are check and balances. What is to prevent someone compiling altered source without those checks, in a way that they does damage to data quality; if there is any trust, then perhaps that can be tampered with? Specifically, I wonder that chunks need to be not trusted by default.

.2. If you append to a chunk and try to decrypt, it seems to hang:

echo “QQ” >> somechunk

Decryption hangs after giving error
thread ‘’ panicked at ‘called Result::unwrap() on an Err value: InvalidLength’, /home/rustbuild/src/rust-buildbot/slave/beta-dist-rustc-linux/build/src/libcore/result.rs:729
thread ‘’ panicked at ‘called Result::unwrap() on an Err value: RecvError’, /home/rustbuild/src/rust-buildbot/slave/beta-dist-rustc-linux/build/src/libcore/result.rs:729
thread ‘’ panicked at ‘called Result::unwrap() on an Err value: RecvError’, /home/rustbuild/src/rust-buildbot/slave/beta-dist-rustc-linux/build/src/libcore/result.rs:729

That unsuccessful attempt then also creates a 0 byte file where perhaps that should not exist, if it cannot be decrypted.

.3. I note that no chunk is created in ./chunk_store_test where file is small; the data_map seems to be a chunk in itself. It’s unclear to me, how large a file needs to be before it starts creating chunks that are not data_map.

.4. A corrupted chunk, still sees a file decrypted.

I don’t know, if a chuck is corrupted, what prevents its being propagated and potentially usurping the original. I wonder the data_map might need a hash of the chunk to be sure of its integrity?

So, for example, a simple replace of one random character (w) with another (v):
sed ‘s/w/v/g’ ./chunk_store_test/copyofchunk > ./chunk_store_test/originalchunkfilename
corrupts that one chunk.

Decryption then completes, forming a corrupted file. I did this with two image files and as might be expected the result is hard to predict; one file was a valid .jpg with lost image, and another was a malformed .jpg that couldn’t be displayed.

1 Like
107

Data on the SAFE Network at http://systemdocs.maidsafe.net/content/en/how_it_works/vault.html should help here.

A 3Kb file will start producing chunks, min 1Kb chunk size and min 3 chunks.

For files, other parts of system can determine whether the content is valid before attempting decryption, e.g. checking the hash of the content of a chunk equals it’s name. However, now that self encryption is in it’s own module if those checks fail internally the content should remain unaltered, so what your seeing, above and in .2. is a bug. It looks like it might be in as the second todo item in the SE ReadMe for the 0.1.3 iteration, if it’s not that I’ll add it in, thanks for taking the time to point it out.

2 Likes
108

First attempt with ubuntu 15.04

mranderson@neo:~/safe/self_encryption$ cargo run --example basic_encryptor -- -e /home/mranderson/safe/matrix.jpeg
    Updating registry `https://github.com/rust-lang/crates.io-index`
 Downloading rustc-serialize v0.3.14
 Downloading regex-syntax v0.1.1
 Downloading cbor v0.3.6
 Downloading num_cpus v0.2.6
 Downloading regex v0.1.33
 Downloading tempdir v0.3.4
 Downloading asynchronous v0.4.5
 Downloading docopt v0.6.64
 Downloading strsim v0.3.0
 Downloading time v0.1.25
 Downloading libc v0.1.8
 Downloading rust-crypto v0.2.31
 Downloading byteorder v0.3.10
 Downloading rand v0.3.8
 Downloading gcc v0.3.5
   Compiling gcc v0.3.5
   Compiling strsim v0.3.0
   Compiling byteorder v0.3.10
   Compiling regex-syntax v0.1.1
   Compiling rustc-serialize v0.3.14
   Compiling libc v0.1.8
   Compiling num_cpus v0.2.6
   Compiling rand v0.3.8
   Compiling asynchronous v0.4.5
   Compiling rust-crypto v0.2.31
   Compiling time v0.1.25
   Compiling tempdir v0.3.4
   Compiling regex v0.1.33
   Compiling cbor v0.3.6
   Compiling docopt v0.6.64
   Compiling self_encryption v0.1.3 (file:///home/mranderson/safe/self_encryption)
     Running `target/debug/examples/basic_encryptor -e /home/mranderson/safe/matrix.jpeg`
thread '<main>' panicked at 'couldn't open /home/mranderson/safe/matrix.jpeg', examples/basic_encryptor.rs:131
An unknown error occurred

so then I took out the “.jpeg” then

mranderson@neo:~/safe/self_encryption$ cargo run --example basic_encryptor -- -e /home/mranderson/safe/matrix
     Running `target/debug/examples/basic_encryptor -e /home/mranderson/safe/matrix`
chunk  written
mranderson@neo:~/safe/self_encryption$

SUCCESS!

then I deleted the file matrix.jpeg

then ran:

mranderson@neo:~/safe/self_encryption$ cargo run --example basic_encryptor -- -d /home/mranderson/safe/self_encryption/data_map /home/mranderson/safe/matrix
     Running `target/debug/examples/basic_encryptor -d /home/mranderson/safe/self_encryption/data_map /home/mranderson/safe/matrix`
! chunk_store_test AlreadyExists
chunk  written

**

Then the file reappeared :grinning: !!!

**

2 Likes
109

Hi,

I am trying the self_encryption example, but I am getting a ‘linking to ‘cc’ failed error’. Any idea what this could be?

Thanks, Dirk

491×974 219 KB

110

I see the same issue in this topic, gcc is installed, could be 64 bit version, will try to figure out if that works.

111

do you have lib sodium installed? follow the scripts in the .yaml file to install it. Should do the trick

112

Thanks, that does the trick, working now. Can’t seem to find my chunks though… they should be in the directory I run the statement from?

113

No, they are in a subdirectory

1 Like
114

chunk_store_test?

it gives a lot of random data when I do a file listing.

829×908 499 KB

115

Thems chunks! The names are their hashes.

2 Likes
116

I can apprently encrypt OK, Decrypting is less fun.
Latest git pull from all libs and last nights rust reported ready to roll

willie@gagarin:~/projects/MaidSafe/self_encryption$ locate chunk_store_test
willie@gagarin:~/projects/MaidSafe/self_encryption$ cargo run --verbose --example basic_encryptor – -e /home/willie/Desktop/WindsorDavies.jpg
Fresh byteorder v0.3.10
Fresh regex-syntax v0.1.2
Fresh rustc-serialize v0.3.15
Fresh strsim v0.3.0
Fresh libc v0.1.8
Fresh regex v0.1.33
Fresh cbor v0.3.7
Fresh num_cpus v0.2.6
Fresh rand v0.3.8
Fresh libsodium-sys v0.0.5
Fresh docopt v0.6.66
Fresh asynchronous v0.4.5
Fresh tempdir v0.3.4
Fresh sodiumoxide v0.0.5
Fresh self_encryption v0.1.4 (file:///home/willie/projects/MaidSafe/self_encryption)
Running target/debug/examples/basic_encryptor -e /home/willie/Desktop/WindsorDavies.jpg
chunk written
chunk written
chunk written
chunk written
willie@gagarin:~/projects/MaidSafe/self_encryption$ cargo run --verbose --example basic_encryptor – -d /home/willie/Desktop/WindsorDavies.jpg /home/willie/Desktop/OhDearHowSad.jpg
Fresh strsim v0.3.0
Fresh rustc-serialize v0.3.15
Fresh regex-syntax v0.1.2
Fresh libc v0.1.8
Fresh byteorder v0.3.10
Fresh regex v0.1.33
Fresh num_cpus v0.2.6
Fresh rand v0.3.8
Fresh libsodium-sys v0.0.5
Fresh cbor v0.3.7
Fresh docopt v0.6.66
Fresh asynchronous v0.4.5
Fresh tempdir v0.3.4
Fresh sodiumoxide v0.0.5
Fresh self_encryption v0.1.4 (file:///home/willie/projects/MaidSafe/self_encryption)
Running target/debug/examples/basic_encryptor -d /home/willie/Desktop/WindsorDavies.jpg /home/willie/Desktop/OhDearHowSad.jpg
! chunk_store_test AlreadyExists
thread ‘’ panicked at ‘called Result::unwrap() on an Err value: Decode(Other(“Expected Unicode string or variant map, but got Break”))’, …/src/libcore/result.rs:731
Process didn’t exit successfully: target/debug/examples/basic_encryptor -d /home/willie/Desktop/WindsorDavies.jpg /home/willie/Desktop/OhDearHowSad.jpg (exit code: 101)
willie@gagarin:~/projects/MaidSafe/self_encryption$

I have raised this as an issue in

Should this be the correct place to raise concerns and keep this forum for more general discussion?

1 Like
117

Yes, you can raise any issues you encounter with libraries/examples at the corresponding GitHub repository. Thanks for reporting this one. Ross has added a comment on GitHub for the resolution. We’ll clarify the documentation and update the example to make it fail more gracefully in future.

118

working fine now I know about the data_map.
I will raise any other problems in GitHub

thanks :smile:

1 Like
119

Wow, impressive feature this is!

I am playing with the libraries and I succeeded encrypting and decrypting an mp3 file following the instructions :smile:

I hoped that @BenMS would add in his OP that you also need to install libsodium before you do anything.
The instructions are from another thread here :

" 5.) install libsodium if you have not yet done so

Unfortunally, there are not libsodium packages for Ubuntu. So, you should download, build and install libsodium as follows:

Download the newer tar archive from https://download.libsodium.org/libsodium/releases/17 (you can also get the git repository from https://github.com/jedisct1/libsodium6).

Follow the ritual:

$ tar xzf libsodium-1.0.3.tar.gz
$ cd libsodium-1.0.3/
$ ./configure
$ make 
$ make check 
$ sudo make install

Once you have installed you have to reload/rebuild the GNU linkers. A simple sudo ldconfig is enough:

$ sudo ldconfig

Now run

$./configure

and it should generate the makefile. (I’m assuming you have at least build-essentials package installed)."

2 Likes
120

I’ve also needed to run

"sudo ldconfig "

Before libsodium would link into the program just after install.

2 Likes
121

I’ve ran "sudo ldconfig " also but forgot about it :stuck_out_tongue:
Now I edited my post so whoever follows the instructions does not get stuck :smile:

thx

1 Like
122

At the moment, I have things “working” under MinGW_w64 but the 32-bit version. I’ve tested routing and this project so far, but I had to tweak a few things I think because I’m running Windows 7 32-bit. To fix the assembly compile issues I set my CFLAGS env variable as “-march=native” which I think says make it compatible with my current hardware. I had to copy a pre-built libsodium.a into /bin/i686-pc-windows-gnu/ of the project for it to link. I tweaked a little bit of source which was casting to size_t for a memory allocation, when the data types were both u32 under 32-bit. Is it possible this could be re-written to evaluate the types and only cast when different? I’m a little new to rust so I’m not sure.

I’m going to try a fresh install of MinGW_w64 with 64-bit gcc, and see if I can get it working while running the 32-bit OS.

123

Hi All ! I am also trying to build the self_encryption module on windows 10 64bit.

I was stuck on the libdodium installation until i saw @Tom_Carlson’s post in this topic:

download https://download.libsodium.org/libsodium/releases/libsodium-1.0.2.tar.gz3
cd crust (or cd routing)
mkdir bin
cd bin
mkdir x86_64-pc-windows-gnu
copy from the libsodium-1.0.2.tar.gz archive, only the libsodium.a file into the new bin/x86_64-pc-windows-gnu directory just made

I was not able to find the libsodium.a file, so instead ended up DLing the prebuilt version libsodium-1.0.2-msvc.zip, and putting the dll libsodium-1.0.2-msvc.zip\x64\Release\v120\dynamic\libsodium.dll in the folder self_encryption\bin\x86_64-pc-windows-gnu.

This apparently solved my compiling issues, but running the example sends out an error.

PS C:\Users\jean-\Documents\GitHub\self_encryption> cargo run --verbose --example basic_encryptor -- -e C:\AA_project.pdf
       Fresh strsim v0.3.0
       Fresh byteorder v0.3.10
       Fresh rustc-serialize v0.3.15
       Fresh libc v0.1.8
       Fresh regex-syntax v0.1.2
       Fresh cbor v0.3.7
       Fresh memchr v0.1.3
       Fresh libsodium-sys v0.0.5
       Fresh rand v0.3.8
       Fresh num_cpus v0.2.6
       Fresh aho-corasick v0.2.1
       Fresh sodiumoxide v0.0.5
       Fresh tempdir v0.3.4
       Fresh asynchronous v0.4.5
       Fresh regex v0.1.38
       Fresh docopt v0.6.67
       Fresh self_encryption v0.2.0 (file:///C:/Users/jean-/Documents/GitHub/self_encryption)
     Running `target\debug\examples\basic_encryptor.exe -e C:\AA_project.pdf`
Process didn't exit successfully: `target\debug\examples\basic_encryptor.exe -e C:\AA_project.pdf` (exit code: -1073741515)

Looking with Google, the error code seems to be of a missing dll, so I guess the error indeed comes from the way I dealt with libsodium.
It could also be gcc, but to my understanding things seem ok on this side:

PS C:\Users\jean-\Documents\GitHub\self_encryption> gcc -v
Using built-in specs.
COLLECT_GCC=C:\TDM-GCC-64\bin\gcc.exe
COLLECT_LTO_WRAPPER=C:/TDM-GCC-64/bin/../libexec/gcc/x86_64-w64-mingw32/5.1.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../../../src/gcc-5.1.0/configure --build=x86_64-w64-mingw32 --enable-targets=all --enable-languages=ada,c,c++,fortran
,lto,objc,obj-c++ --enable-libgomp --enable-lto --enable-graphite --enable-cxx-flags=-DWINPTHREAD_STATIC --disable-build-with-cxx --di
sable-build-poststage1-with-cxx --enable-libstdcxx-debug --enable-threads=posix --enable-version-specific-runtime-libs --enable-fully-
dynamic-string --enable-libstdcxx-threads --enable-libstdcxx-time --with-gnu-ld --disable-werror --disable-nls --disable-win32-registr
y --prefix=/mingw64tdm --with-local-prefix=/mingw64tdm --with-pkgversion=tdm64-1 --with-bugurl=http://tdm-gcc.tdragon.net/bugs
Thread model: posix
gcc version 5.1.0 (tdm64-1)

Would someone have an idea about where my mistakes are :smile: ?