MaidSafe Dev Update :safe: 12th May 2015

Hello,
As this is my first write-up here, I’ll just take a brief moment to introduce myself before jumping to the interesting part. My name’s Spandan Sharma, which bears little semblance to my username. I have been working for MaidSafe remotely from India and until recently I had been working as the front-end dev. As we have migrated to Rust, I am now making contributions to both the core and the front-end code :).

So with that out of the way, on with the update. We ended the second Rust sprint last week in which we had set out to create a simple working of Maidsafe DHT network (a very simple one is done, but we hope by today or tomorrow to have a better example for you to play with). It was an important and busy sprint with a feature complete CRUST module (for the planned DHT working model) being one of the major achievements. More work was carried out in Routing with some glitches being sorted out by the dev team, with a few devoting their weekends to make changes. Crust and Routing now look to be API compliant with some minor tidy up to be done there (we expose too much so will tidy this up). As you know by now clicking on the API documentation link on the repositories will show you the documented (and versioned via semver) API.

Sentinel (the module that confirms group based consensus on messages on the network) also received some TLC with further considerations being given to it’s design prior to implementation. Sentinel was also abstracted out as generic library and it now enjoys its own crate. The implementation that can now be seen has parametric polymorphism and can be pulled in from its own separate crate.

David has also migrated the code-base for all MaidSafe libraries in Rust from his repository to a newly created MaidSafe repository for Rust. From Github’s point of view there requires no change, contributors will now have to however modify the tracking upstream pointer in their local devices to point to the new location.

Apart from the main sprint, the previous week also saw a few equally important tasks being
carried out in tandem. For instance, exposing Rust code in the simplest maintainable way to other languages (with the purpose that ultimately the MaidSafe API will be available to the developers in a variety of languages), a POC setup for Highfive and nightly Jenkins build for maidsafe_vault. Highfive picks up reviewers for each pull request automatically and assigns it to them.

In a brief interlude from dev tasks, Ben delivered the keynote speech at the Amsterdam Bitcoin meetup, one of the largest in Europe. Ben’s 45 minute talk was well received and was attended by a few community members. While Ben had a good one in Amsterdam, Nick had no less in Las Vegas and just in case you missed it he published a trip report here.

Getting back to the libraries, with a view to continuing to decentralise responsibility of the underlying code throughout the core team, we have decided to implement specific library maintainers, essentially direct points of contact for specific library related queries. There will be two maintainers for almost all libraries, a primary maintainer and a secondary maintainer. The secondary will become active if the Primary is not available, here’s the distribution so far (Primary followed by secondary):

CRUST: Prakash, Niall
Routing: Ben , Peter
Vaults: Qi , @ustulation
Client: @ustulation , @Krishna_Kumar
Drive: Mahmoud, Brian
Self Encrypt: Brian, Qi
Sentinel: Peter, Ben
Types: Mahmoud
LRU, Accumulator and Message Filter: Prakash

This weeks focus lib is CRUST (connections in Rust) https://github.com/maidsafe/crust. This is an exciting library for any p2p project and we hope other projects will make use of it. Initially this uses TCP as the first protocol with Reliable UDP currently being added. This library has the ability to manage and maintain connections within any P2P network and remembers connections previously seen (and directly addressable, i.e. NAT traversed). At the moment it requires compiled via Rust nightly builds as it uses a Weak pointer like structure which is not yet cut over to beta.

To see it in action (for those who like to play) then it’s the usual (add in git clone/build/test instructions) and in the target/debug/examples folder you will find some example uses of the lib. There are a few files and to begin with you can run a simple receiver / sender pair. If you run the receiver and then in another terminal run the sender you will see data transfer (OK boring). For a more interesting example consider rust_node, this shows the ability for nodes to connect automatically to each other and create small self-connecting networks. You can run this on your own machine or several machines on the same network.

So start a node by typing ./crust_node and you will see it start a beacon listener. This means it’s listening for any new nodes starting and will answer them with it’s endpoints. These new nodes will then connect to this one. To see some data send/receive examples type crust_node -s 1000 and see the node connect, find the network and continually send / receive data. Set up several nodes and see them all interact.

This is important and as the README on the repo will show it is a great foundation for a self discovering network to start up. We then add further functionality via routing to create a neat secured DHT, but that is for next week :slight_smile: and as we drive towards launch everyone will see all the pieces come together and why each is important.

CRUST still has a few to do’s, as can be seen from the README, but none that will affect the API.

Anyhow, I hope you enjoy playing with this example, the start of an autonomous network is coming to your computer right now, it’s going to be enlightening :slight_smile:

So reflecting back, good progress overall and this week we turn our attention back to planning the next sprint, more details to follow on that next week. It’ll hopefully make an interesting future update :slight_smile: !!

28 Likes

If you see the repo’s you will see exactly each modules progress. Basically though in laymans terms Crust network is A OK now, routing being tested today and if the vaults plug into this as neatly as we imagine, the testnet will be back up. We are much less focussed on this as we are in progressing to feature complete. The progress is beyond normal as this has been a complete rewrite in a new Language. So from my perspective moving to this iterative process means giving really measurable components on a weekly/bi weekly basis and never regressing (which is working).

Like rushing though a forest searching for the river we are moving very fast, but accurate timescales are near impossible. For each sprint we are achieving a lot (more than normal). Next sprint being planned right now, but may take us to feature complete or very close to it.

Testnet3/beta now merges and each iteration just adds more. It would not be possible to do the whole network in 2 sprints so if we get feature complete in next sprint or even a further one then we are in great shape as it’s all testing and that means everyone. This week we also are automating nightly installers and the highfive reviewing tool.

The question now is when can we get installers and code in your hands and what features will be there and we will know that this week I hope.

24 Likes

Can’t wait to hear about installers! Been watching the progress on github and recently I’ve seen more progress than ever before. Congratulations to the team for being so effective and building such transformative tech! Cheers

6 Likes

That’s great you are so close; however, I think it is worth the time to remove the outdated road-map at this point and at least put something simple in it’s place to demonstrate the recent progress. It does not need to be fancy or detailed, but SOMETHING to show progress was made and that testnet 3 is merged with Beta. For us long time believers and rather large stakeholders, the simple 10 minute change to the roadmap website will at least help the price from being further diminished. Not everyone who buys/sells MAID is savvy enough to go to GitHub and understand what it means. Please consider this…

7 Likes

Why? Let them do some homework if they want to invest, and btw, these are not stocks. These are proxy-coins that you can use when the network goes live to get the real SAFEcoin. If people don’t believe in it, they should sell. Maybe I’ll buy some more :wink:

4 Likes

Your concerns are noted and reviewing the roadmap is one of the actions we had for this week. I agree that we should continue to try and provide a snap shot position of our progress to interested parties. You should see the amendments being reflected during the course of next week.

For your awareness this is not a 10 minute task though, moving to Rust changed the landscape significantly (in a good way) so we will be looking at the whole roadmap again rather than just ticking off a couple of items. I hope this makes sense.

6 Likes

Totally understandable. What do you think are the odds of achieving a working SafeCoin implementation this summer?

I remain at a loss for words with the unbelievable progress within just the last month. Amazing work! I’m excited for testnet3!

2 Likes

I think that there are still certain benchmarks that are necessary and important, and that I dont think can be avoided one way or another.

First there is a Public Test Net, defined as a point where people from around the world can join an existing network and show how the network behaves at a global scale.

Second there is Test Safe Coin - where a variant of Safecoin is implemented, and people test its functions, particularly the farming function the PUT function and the transaction function.

Finally there is actual safecoin - which people can get in exchange for maidsafecoins and use to PUT data that will never be erased.

I don’t think it matters so much what these are called, and perhaps the first two can done at the same time or nearly so. But these are the significant benchmarks as I see it.

And I think that for publicity reasons it would be good to have names and some kind of event, maybe an email blast associated with them.

5 Likes

Thanks Nick, I understand it may take more than 10 minutes. You guys are the greatest!!!

3 Likes

@culexevilman

https://travis-ci.org/maidsafe/crust

2 Likes

Is there a way to see progress on github.com/maidsafe besides watching commits pile up?

I found this recently https://coveralls.io/r/maidsafe?page=1 but not sure of the accuracy. Can anybody say anything about it’s accuracy?

It is as accurate as the kcov tool. A few projects use this now, but we have some concerns internally about the figures. I have noted it does report coverage decrease when a PR has added code but no tests etc. So it seems to be reasonable, but no coverage tool is 100% really as it depends how they report the coverage (i.e. like asserts etc.).

3 Likes