Excellent Mark, we had not setup CI on arm yet (travis heading that way) so was curious. The command is cargo test I just added a -tested to show what this meant (my bad).
1 Like
The command is cargo test I just added a -tested to show what this meant (my bad)
yes that was confusing 
1 Like
Ok, Odroid took a while and did a lot of stuff, then this:
make[2]: Leaving directory `/home/odroid/rust/rust/arm-unknown-linux-gnueabihf/llvm/bindings' llvm[1]: ***** Completed Release Build make[1]: Leaving directory `/home/odroid/rust/rust/arm-unknown-linux-gnueabihf/llvm' fetch: arm-unknown-linux-gnueabihf/stage0/bin/rustc Traceback (most recent call last): File "/home/odroid/rust/rust/src/etc/get-snapshot.py", line 78, in <module> main(sys.argv) File "/home/odroid/rust/rust/src/etc/get-snapshot.py", line 60, in main snap = determine_curr_snapshot(triple) File "/home/odroid/rust/rust/src/etc/snapshot.py", line 298, in determine_curr_snapshot (platform, rev)) Exception: no snapshot file found for platform linux-arm, rev 7b95b5c make: *** [arm-unknown-linux-gnueabihf/stage0/bin/rustc] Error 1
I’ll leave it now but wanted to feed back!
EDIT: Maybe this is the issue - lack of an arm “snapshot”!
Since the Rust compiler is written in Rust, it must be built by a
precompiled “snapshot” version of itself (made in an earlier state of
development). As such, source builds require a connection to the Internet, to
fetch snapshots, and an OS that can execute the available snapshot binaries.
Snapshot binaries are currently built and tested on several platforms:
Windows (7, 8, Server 2008 R2), x86 and x86-64 (64-bit support added in Rust 0.12.0)Linux (2.6.18 or later, various distributions), x86 and x86-64OSX 10.7 (Lion) or greater, x86 and x86-64
You may find that other platforms work, but these are our officially
supported build environments that are most likely to work.
Rust currently needs about 1.5 GiB of RAM to build without swapping; if it hits
swap, it will take a very long time to build.
Source: GitHub - rust-lang/rust: Empowering everyone to build reliable and efficient software.
3 Likes
Could be there is a lot of upheaval just now with that. Arm is a priority though so I am not worried. One of the Rust devs is gonna get back with a Pull request to help with benchmarks in the Beta channel I think so you may get the bench with beta. That’s better.
You can also do cargo doc and then look in target/doc/self_encryption/` and see the docs. A nice feature is they get compiled so any code there for examples etc. is checked as correct. The rust guys have really picked the best of a lot of languages and tools here. Think it’s only the beginning and I am glad we choose to take this brave/crazy step as it really will open up the network to many more developers.
4 Likes
Where did these numbers come from? I just ran cloc on the C++ code base, and you have to include the third party libs, and all the helper scripts to get close to this number. I’m not sure thats a fair comparison, considering the C++ code includes a crypto library (crypto++), a networking library (ASIO), and the unit test framework (google test). Excluding the srd/third_party_libs yields 68303 lines of C/C++ code (which includes headers), and my pretty large changes to the NFS are in that count too.
The rust codebase appears to be 8836 lines (if you include crust). Although I know of at least a few features that got removed (DataBuffer), and likely others. So I’m not exactly sure what this tells us honestly, other than the C/C++ header system is archaic and increases code count (along with compile times). Things like DataBuffer weren’t perfect (possible design issues), but the C++ version was using it heavily to reduce network traffic.
EDIT:
This was actually the branch where I dropped the dead code in NFS, and re-wrote the testing disk-storage utility in substantially less code (didn’t bother with async). So it should be less than your count, but not 430,000 lines less.
2 Likes
Language files blank comment code
-------------------------------------------------------------------------------
C++ 352 11221 9452 48107
C/C++ Header 309 8373 6915 25601
Bourne Shell 191 1345 454 5585
Javascript 7 101 140 2807
CMake 13 300 502 1289
C 1 40 35 906
Python 2 13 2 111
CSS 1 20 13 74
HTML 1 1 1 42
DOS Batch 1 0 1 14
make 1 4 6 5
-------------------------------------------------------------------------------
SUM: 879 21418 17521 84541
-------------------------------------------------------------------------------
~/Devel/MaidSafe-dirvine/src [eggs_variant] $ cd -
/home/dirvine/Devel/Rust
~/Devel/Rust [master] $ cloc *
2571 text files.
2078 unique files.
3471 files ignored.
http://cloc.sourceforge.net v 1.60 T=5.73 s (344.6 files/s, 48085.1 lines/s)
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
HTML 1729 32918 10230 217719
Rust 69 1442 1611 7727
Javascript 142 159 126 1062
Bourne Shell 2 195 82 917
CSS 2 90 21 545
YAML 14 25 2 406
D 15 30 0 30
-------------------------------------------------------------------------------
SUM: 1973 34859 12072 228406
-------------------------------------------------------------------------------
The numbers come from running cloc here, if you include third party c++ then it’s well over 2 million in there alone, so this did not include third party code (I deleted the third_party directory to run this actually just in case). This was run on 8th April at around 14:00 showing c++ for our libs alone at 73708 not counting launcher etc. [my rust was up to date, but c++ was not updated for several weeks] This is my copy of the code on my computer right now. This is with the 98% reduction in vault code from testnet1, the vault lib was huge.
So anyone can run this and go back in our history you will see our codebase alone over 1 million lines, boost is over 2 million lines and we have not needed boost like additions for rust as it has the features we need in it’s core lib mainly. Things like io are just there so nice.
Great thing about git is this is all there for the counting. For instance the databuffer is not required as there is a memory mapped io now available and buffered read/write streams, but that alone is a good example of the point being made.
[Edit: for the avoidance of doubt I have created a tar.gz of our src dir alone (so 100% our code) in c++ and anyone who wishes I will email it to them, may be nice for posterity as well 
Saves getting from git, but git is ultimate holder of history. ]
8 Likes
Wanting to play with Rust I’ve installed Sublime Text 3 and set it up for Rust - but it wasn’t (isn’t?) auto-detecting rust source (e.g. editing self_encription/src/encryption.rs). When I switched file type from Plain Text to Rust the syntax highlighting sprang into life.
I have also set up vim with Rust features but they also seem not to be there so maybe the same issue.
@dirvine Is there some trick to getting these files auto-detected as Rust source with ST3 and vim?
I use vim so have set up racer for that (line 24 here)
You need to compile racer and point it to a checkout of the rust lang git clone git@github.com:rust-lang/rust.git So just point to this in your .vimrc file, like the link.
Sublime is pretty similar I think. I also know one of the guys is grabbing a bunch of these and putting them together in an eclipse plugin so that may help. I am not sure how far he has got (Krishna, one of the guys in India, phenomenal at connecting stuff like this)
1 Like
Maidsafe’s C++ codebase can likely be shrunken too. The line-count should primarily be a factor of how well abstractions/concepts were put into code. Better designs allow more code re-use, and less duplication. Development over time tends to generate cruft as a better implementation becomes more obvious later. I believe the fashionable word is “technical debt”.
I would expect C++ to have more lines of code, but its not a practical difference if the abstractions are similar. For instance, achieving the trait style concepts for type-erasure in C++ (or even C) is possible, but requires more lines of code. The C++ is still strongly typed too.
What frustrates me about the line count debate is that its both revealing and misleading. At Virginia Tech, I remember some professors promoting research into how to reduce the number of lines of code a programmer wrote, since the metrics showed a strong causal relationship with bugs. They pushed for more/better libraries to reduce the line count. To me a more accurate description would be the number of abstractions in a program. The libraries can’t do everything (which is what the profs seemed to hope for); a quality product augments the libraries by providing useful abstractions that can be composed internally. I’ve seen it done the wrong way (and done it the wrong way, many times), which usually results in copy-and-pasted boilerplate or difficulties when trying to compose into higher order procedures. This is never diserable due to the strains on testing and maintenance.
Even in the new Rust codebase I saw unnecessary duplication. The original Maid object suggested copying and pasting several lines to create keys, and then provide it to the creation function. It was a trivial example, but a good one anyway - the function should’ve done that common task (like a default constructor). I don’t want to be critical of whoever wrote that, because its often a symptom of working too quickly.
When a project does reach nearly 500,000 lines of code, especially a strongly-typed one (hopefully there aren’t many dynamically typed ones that size), it becomes increasingly difficult to make changes. Rust isn’t going to change the boredom of updating a never ending stack of code because you realize a design for a widely used abstraction was terrible for some new feature desired.
Just remember that the Rust IO library is marked as experimental. It would be extremely unfortunate if a working implementation existed in Rust, and was even fielded (best test environment), only to have the API completely changed. I think an important goal is to avoid re-writing critical code paths, not continually re-write them. New code is always the most likely to contain a bug, and its transitive property from OS to your application.
The DataBuffer was technically never needed, but maximizing virtual memory takes some work too. I don’t think its a completely “free lunch” as it were. IMO, you have to carefully write to page aligned boundaries to maximize the benefit, otherwise frequently accessed data keeps multiple pages active. In the past, I used an anonymous mmap on *nix and VirtualAlloc in Windows to guarantee page aligned boundaries, but that might be pushing things too far here. You can just ignore the first x bytes, and the last x bytes with a memory allocation - which still takes a bit of care.
1 Like
Being out of touch I blindly followed links and ended up with Vundle / rust.vim. Have racer in ST3 but not tried with vim yet. Here’s my entire vimrc:
set nocompatible " be iMproved, required
filetype off " required
set rtp+=~/.vim/bundle/Vundle.vim
call vundle#begin()
" Rust
Plugin 'wting/rust.vim'
" let Vundle manage Vundle, required
Plugin 'gmarik/Vundle.vim'
" Plugins
Plugin 'tpope/vim-fugitive'
" Color schemes
" All of your Plugins must be added before the following line
call vundle#end() " required
filetype plugin indent on " required
Readability: using double quote as comment delimiter… null points.
May as well stick with ST3 for now. There’s an IDE (Solid Oak) based on libnvim (from neovim) but it won’t build libnvim (or neovim) because it’s not finding libtool (which is installed but not on the path). Strange. :-/
This looks OK Mark I think. Feel free to grab my complete vimrc if it helps, it has setup for c/c++/latex/markdown/rust/nim in case you get more experimental 
1 Like
The guys in the team seem to like this a lot Mark you may get best mileage out of it. Perhaps @Viv can help you out. I know he had to help a couple of folks to get racer doing completion.
Sorry about the noise then, Google turns up older documents. The problem is that a few of the functions are marked as unstable, and Crust is currently using one of them. So my confusion started with that.
Asynchronous IO would still be nice, because multiple kernel threads don’t match the behavior exactly. I’m guessing they want to figure out how to do async + user threads (tasks), because thats where boost/ASIO are headed too. Unfortunately, I don’t think C++ or Rust can make it as transparent as Go! did, due to the language designs. Although I remember reading a C++ proposal by Microsoft that brought the behavior really close. My guess is that Rust steals some ideas from C# and Go!, which is basically what the Microsoft proposal seemed to do.
This is a really nice writeup of concurrency support in Rust and makes some attempt at comparisons with go as well.
I seem to have completion working (and the syntax highlighting - once I manually tell ST3 that its a Rust file). So far so good but will need some basic things like keyboard shortcuts and general howtos. Would be good to sit and watch the guys working to pick these things up. Oh for the days of the old computer room filled with terminals and a PDP11/34. I learned so fast by being nosey
@Viv if there is a trick to getting ST3 to detect Rust files automatically, and if you have some simple how-to for common tasks etc. they would be welcome, but don’t divert yourself. I’m very low priority and able to do what I need now.
@dirvine, I’m also cloning your vimrc/VAM setup for comparisson - thanks for that.
EDIT:
python --version says 2.7.9 but starting vim says…
UltiSnips requires py >= 2.6 or any py3
YouCompleteMe unavailable: requires Vim compiled with Python 2.x support
Looks like vim syntax highlighting is working but no-completion obvs.
1 Like
I went micro vax way with vms where you could spawn from the report writer as any user you wanted, oh security has come a long way
1 Like
Earlier I was remembering paying over £50 each of my own money - a lot more than fifty quid is worth today - for some 8Kbyte static RAM chips. 1980 I think. Wipes tear.
1 Like
There is no comparison to Go!. Everything is a user-space thread, and so asynchronous IO is done completely without the programmers knowledge. Absolutely brilliant for the intended usage environment.
Rust exposes kernel threads, which was probably a requirement for its intended use cases. However, I think this makes any asynchronous (event driven) behavior with the OS less transparent, and more like C++. The message passing in the stdlib is mainly convenience (no 3rd party libs necessary), but something that can be done in most other languages (possibly all). It does push the idea of the actor model a bit more since its a core feature.
The shared stack frame thing can definitely be done in C++, but isn’t checked by the compiler as thoroughly. This type-checking for threads could be a huge advantage for Rust programs. If used competently, I see only positives to this feature (right now). Its a matter of seeing what happens when/if someone does break it - you now have to fight the compilers additional type checking AND invalid code. I think Rob Pike hates C++ const for this reason, but I could never find an elaboration of his “C++ const fiasco” comment in a mailing list post. Since I like C++ const, even with programmers incorrectly throwing around const_cast at times, I suspect I would be willing to fight the compiler for the thread checking.
I meant this seemed related somewhat since it actually mentions go as having same motivation (and rust had green threads and still has but found them lacking in way, although you can still use them. !! So here is the relevant part. Anyway I need to get some code done.
Message passing
Now that we've covered the basic ownership story in Rust, let's see what it means for concurrency.
Concurrent programming comes in many styles, but a particularly simple one is message passing, where threads or actors communicate by sending each other messages. Proponents of the style emphasize the way that it ties together sharing and communication:
**Do not communicate by sharing memory; instead, share memory by communicating.
--Effective Go**
Rust's ownership makes it easy to turn that advice into a compiler-checked rule. Consider the following channel API (channels in Rust's standard library are a bit different):