MaidSafe Dev Update :safe: 3rd August 2015

Pretty much every week is a new, mad, hectic adventure in MaidSafe, but some are that little bit more intense than others and they seem to happen just when it is my turn for the Dev Update. So this might be a long one …

In our continued drive to ensure it is crystal clear that the SAFE Network belongs to everyone, not a project belonging to and controlled by MaidSafe the organisation, we made a simple (not for me I did the tedious part) change to the library names. We renamed all the libraries that had a prefix maidsafe_ to safe_ so we now have safe_client, safe_dns, safe_nfs and safe_vault . This decoupling from MaidSafe may not be that significant to this excellent and quickly growing community who already “own”, contribute and understand that the SAFE Network belongs to no-one, however it should help communicate this as the network grows.

As avid followers of the current RUST-3 sprint dashboard may have noticed, last week was not a good week progress-wise. In fact as you can see from the burndown chart we pretty much flat-lined; this was due primarily to issues we hit in routing, which in turn blocked progress in other dependent libraries. The decision was taken to extend the current sprint by one week to ensure we deliver a robust and stable routing library at the end of this sprint with reduced code complexity.

So routing is now undergoing a slimming down exercise; we moved the responsibility for securing messages down from safe_vault to routing several months ago and in this sprint we defined the fundamental data types also at routing. It is now time to push some responsibility up to the higher layers and have routing become a library true to the word. To do this we are exposing the interface to routing through an event channel (similar to the way crust exposes events to routing). As a result safe_vault and safe_client will take on a bigger responsibility in handling the content messages and how these messages flow through the network based on that content.

While this is happening and in order to keep momentum moving forward we are temporarily disconnecting safe_vault and safe_client from routing and have created a mock routing layer for these libraries to allow work to continue. The mock APIs will match the real interfaces so when the routing work is complete it will be an easy switch over from the mock network to the real live network; the idea being that no-one is stalled and we still deliver our sprint objectives, albeit one week later than expected.

The extra week allows time to complete and polish the safe_vault installer and the examples bundle. Good progress has been made on these and they are on track to be delivered as desktop installers on our core supported operating systems (Windows 7 onward 32 & 64 bit, OS X and Linux potentially using MUSL) at the end of this sprint. Work is also ongoing to integrate MUSL with the benefit being that “An application can really be deployed as a single binary file and run on any machine with the appropriate instruction set architecture and Linux kernel”. MUSL is a bit of an installer extra which may or may not make it into the final sprint installer deliverable. We will now focus on ARM / Linux as a priority for both CI and deliverable installers for ARM (hopefully hard and soft floating point variants).

From a lower level point of view, work continues in crust adding UDP hole-punching and uTP integration features; these are also making good progress with a couple of implementation options being explored to integrate these into the crust library. A lot of developer effort is also going into self_encryption and memory_map to remove the restricted file size limitations. We anticipate that all of this work will give us a functional and stable network by the end of the sprint.

With the additional time it is also planned to throw a desktop app on top of the safe_dns crate, which will also tie into the browser plugin. The client guys are working on a desktop app that will allow users to upload content to the network and consume it from a browser. Yes a UI! This may be limited just to a subset of browsers to begin with perhaps even only to Firefox in this sprint, but even still this will be a pretty nice deliverable.

@ioptio also did a great job presenting the story of our transition to Rust to the Rustcamp at the weekend and by all accounts it was very well received. Developers were especially motivated by the ease with which front-end developers transitioned to core development work and this picture did not seem isolated just to us, but appeared to be a general theme across the entire event. Video of @ioptio and all the other talks will appear online soon. Paige left for Europe last night and will be attending a few conferences including BattleMesh 8 and Chaos Communication Camp 15 .

The observant amongst you (pretty much all of you :smiley: ) will also notice that we are trying to grow the development team with some pretty senior hires that will introduce even more depth to our technical team. We are looking for a Senior Network Engineer and a System Architect . If you know of anyone fitting the profile please let them know about these opportunities.

On top of all this a lot of CI infrastructure work has taken place, for example transitioning OS X CI to Travis CI - thanks to the super supportive guys over at Travis - and the MaidSafe Foundation folks have also been building and preparing for a mini fab lab event … did I already mention the words ‘mad’ and ‘hectic’ :slight_smile:

As per tradition here is the link to the meeting transcript: https://docs.google.com/document/d/1t613QCcxniNV6TuQRkEW7DrxjwBCi1f5p0Xjw7At7yk/edit?usp=sharing - thanks @justine_mclevy

35 Likes

Wow @Ross that’s a hell of an update, and some really great news that’s getting me very excited. I wasn’t expecting you to get to the browser plugin, but to have an App with UI, using safe_dns in Firefox will be incredible. However…

:kissing_heart: :kissing_heart: :kissing_heart: :kissing_heart: :kissing_heart: :kissing_heart: :kissing_heart:
:kissing_heart: :kissing_heart: :kissing_heart: :kissing_heart: :kissing_heart: :kissing_heart: :kissing_heart:

Its all a bit too much in one update! Well done and thanks to everyone for all your hard work.

21 Likes

I figured you would like that one :smiley:

9 Likes

I guess it could be a payoff for getting stuck into SAFEpress and not pestering you guys for a while. It must be a whole month I’ve left you all to it. It can’t last though… as soon as I’m feeling better there is a significant risk of some questions. Sorry man :wink:

2 Likes

Plan is get Crust solid and feature complete, then routing (hope this week and next) then vaults and we all pile into client side (the whole team so prepare for mayhem there), so you will get hounded like mad. I am also investigating partnering up Engineers with projects if we can get a breath. It will help us see how folk use the API and make sure we provide enough and not too much in there (big API’s are bad and confuse folks, small simple effective is the target here).

17 Likes

Okay a couple questions.

Can we farm safecoins with this vault/client?

What with all this discussion of APIs and app development is there any clear documentation I could read up on how exactly they work? And what code language would you recommend I learn to write an app?

2 Likes

Would you consider using servo as a main UI for the safe-apps?

2 Likes

For starters :thumbsup: Thanks for all the progress made so far on this. And especially on the progress and releases targeting app developers. Am also celebrating like @happybeing :tada::tada::tada::tada: :dancers::dancers::dancers::dancers::dancers:

Can we have this be documented out and have a wiki on how this first app is developed. i.e the current code it contains and the purpose it plays out to have the code. It would be nice to now the different components that are required to actually make the app run.

3 Likes

Of course, we will document all of this. I’m not sure where it will reside right now, it will definetley make it to our developer focussed website, but I’m not exactly sure when that will launch yet. We will keep you updated through this forum/blog.

6 Likes

This current release will not enable farming safecoins, that will come in a future release. We will document how we have created early apps (I’m not sure where yet, but we will keep you updated) and this will help you gain a better understanding of the API. I think the available app languages and which you should use has been discussed in an earlier thread. I seem to remember Python was one that many people thought was quite approachable for those dipping their toe in the water with app development.

4 Likes

What’s being promised here appears to be a web app, so it will work in a browser with the SAFE plugin, and will be written in HTML/CSS/Javascript (like SAFEpress).

So the process of building this kind of app is fairly straightforward. The tricky part is that you’ll be talking to SAFE network rather than a web server (which will be the same for all SAFE apps).

It is though likely that this early plugin will offer a subset of the SAFE API, but we’ll have to wait and see how much they expose.

EDIT: The diagram in this SAFEpress post shows how I believe web apps will work on SAFE Network.

7 Likes

I have a question to the developers that probably has been answered several times, even once in this thread already, but I was not able to really understand it, so I am going to rephrase the question: When can we expect (I know it is never certain) to have a most basic global testnet running (without safecoin/spam protection, without sophisticated hole punching, i.e. still with configuring ports and such, without a sophisticated GUI/installers)?
Is it possible/running already (I saw on the forum people discussing it)? I imagine some kind of command line installer, so that all the die-hard fans of SAFE here on the forum can just install it, get in contact to each other, somehow form a global network, create accounts and somehow start to upload files to it, maybe just in the command line? I am aware that without safecoin, there is the risk the system would run out of storage space, but maybe people would be so decent to upload only small files? Or will you wait for safecoin to be implemented to allow some kind of private testnet?

4 Likes

It’s coming! At the end of this sprint we should have a network where we can all test, upload files, download files etc. Dev-Bundle 1 will provide all of this.

5 Likes

Thanks! This is indeed good news!

3 Likes

What happens after Dev bundle 1? I mean at some point you’ll need to impliment safecoin. When that happens will you wipe all the data on the network and start over or will the data stay intact and we’ll just get an upgrade of some kind? What will happen when “Dev Bundle 2” comes out?

According to the road map, safecoin will be implemented in Dev Bundle #3. I am going to assume it will operate in testnet until official release date.

2 Likes

@nicklambert For the available app languages is there a repo responsible for porting the language as well as the bindings. If so the repo can be exposed and if well documented other developers can tweak it to suite the programming language they feel comfortable with.

3 Likes

Good question @mwaa. Once we have got the API the way we want it we will get in ported to enable it to be accessed by mobile devices. We will get this documented as we go. We will be looking for community feedback on the quality of our documentation we produced, it’s for you guys after all.

7 Likes

Do I understand correctly that vaults will be running locally (ie maidsafe internally) but that clients can connect to this vault network from all over the world (ie by anyone) for dev bundle 1?

2 Likes

I hope someone else jumps in here on this but that’s what I’m getting from it saying LAN (local area network) for dev bundle 1. I also imagine that there is a size limit to PUT data because of this and also the fact that compression algorithms haven’t yet been added (dev bundle 2)

1 Like