MaidSafe Dev Update :safe: 11th August 2015

Hi All,

As mentioned in last week’s update an additional week was required to retrofit an event based channel in routing as the main communication mechanism with the upper layers, safe_vault and safe_client. Between us, anyone working on other parts of the system were probably relieved to get a bit of breathing space, allowing time to better integrate and test their work, so we shouldn’t treat the routing guys too harshly over the delay,… that being said, feel free to interpret that as a royal we. The positive is that the routing API has converged to a point where consensus has been reached allowing us to tick it off as finalised. With the API settled, and some of the responsibility for network security transferred to higher libraries, work on safe_vault and safe_client can now continue unabated.

The relevant changes, published a few days ago, are accessible from crates.io. The remaining tasks for routing during this sprint involve the completion of the implementation with respect to those changes.

Compression has been introduced in self encryption to enable more efficient use of storage space, coupled with a potential reduction in cost on a per chunk basis. A proposal has also been put forward in an RFC to increase file size handling to a temporary maximum of 1Gb using memory maps. Unrestricted file sizes will be the next logical improvement after that, intended to form a significant part of a future sprint. On a related point, the sodiumoxide crate has been updated, including some PR’s from our own fork of the library, allowing us to revert to using that as an independently maintained library in our projects. Thanks to the maintainer for merging those PR’s, it frees us from repetitive application of boilerplate to meet our requirements.

Crust is coming along nicely with the integration of rust-utp, in time this will enable the community to contribute to the network with their own vaults. Completion and running of the tests is all that remains to be done here. Hole-punching is still being actively pursued, initially as a POC example, with proper integration with crust coming after that challenge reaches fruition.

A lot of recent effort has gone into enabling Musl with Rust. Realisation of this goal will give us dependency free binaries, on Linux based systems, to package into the installers. In brief, what that means is, any code linked to a binary as well as between libraries will be done statically. By implication, once installed, the binary runs as a self contained process, thus removing the requirement to package additional files or apply manual updates to your system. The main benefit to be gained from this is a single installer that caters to the various flavours of Linux, Ubuntu, Fedora, etc., over a variety of hardware architectures x86, x64, AMD, Intel, etc. There’s a Musl Wikipedia article for those who may be interested.

Achieving this goal is proving to be less straightforward than one might expect though, and as last week’s update hinted, we’ll have to see how this pans out during the rest of this week. In either case, work has continued on preparing the installers for the currently supported systems scheduled for release at end of the sprint. Again, as a reminder from last week’s update, the packaged installer will include the safe_vault binary, and will be accompanied by the examples bundle, providing ample opportunity to give the network it’s first serious workout.

A weekly QA catchup has now been penciled in, with artistic license, for Friday afternoons, the first having taken place last week. The intention is for the QA team to provide status reports to maintainers covering all aspects of agreed policy. This will allow us to: provide standardisation throughout the codebase, ensure issues and PR’s are being properly addressed and in a timely fashion, make regression free examples available across the libraries and insist that the all the t’s are crossed and the i’s dotted… With everyone, QA, community, developers, you involved, we should observe best practice across the board. Testing is an area to receive particular attention in the early phases with a matrix covering supported compiler releases, similar to that already included in the memory map fork, release mode only testing at the CI’s, and the inclusion of ARM CI facilities and testing.

As promised last week, work has continued on the desktop app, including drag-n-drop capabilities, showcasing safe_dns with authenticated upload to the network. The browser plugin is also shaping up and will be made available to view any uploaded web content. To reiterate, at the moment this acts in conjunction with the mocked routing the guys have been using to test their code. Collaboration amongst the developers has contributed to keeping the mock API in tight conformity with the actual API so the transition to using a full routing network should be seamless.

For the technically minded, development of the desktop app required writing the ffi bindings, an example of which can be found here, to a c language interface that’s exposed to SWIG. The plugin will use js-ctypes to call back into the c interface for content recovery. Pretty simple really.

Now for a demo, in the following screenshot you can see at the start where Firefox is being run followed by a few network actions getting logged via the plugin.

This one shows safe_nfs lib usage from Python, highlighting that error code propagation is being accurately reflected through the binding layer. Python is not a requirement for the desktop app/browser plugin but the guys decided to give it a quick try over the weekend.

Anyway, enough of the development views of what lies under the covers. The design team have been doing some really nice work on the UI in parallel to this, as a preview the current version will look like,

However, we suspect that most of you will be more excited for the next version which is probably going to look like,

As usual the weekly transcript is available for your perusal.

Regards,
Brian

54 Likes

Reading this gave me a headache cos im not really tech sawy haha!

Screenshot looks good though, is this the design we will have when we open the SAFE client/app or is this just for SAFE uploader ?

Something trendy is the background pic on user’s profil like facebook, even openbazaar has it.http://openbazaar.github.io/OpenBazaar-Design/
People like to create their own little virtual space lol
I think it’s not easy to create a nice UI for decentralized software, that’s why the more pics the better ( my opinion)

3 Likes

The design is specific to the uploader for now. I’m sure the design team will have something in mind by way of customisation in future releases, although, at the moment I’m not aware of what that might be.

What will the installers enable as far as connectivity to other machines, over what sort of lines?

Is this sort of the beginning of people (or at least network nodes) being able to connect globally?

Amazing update @brian_s! I personally love getting all the details, Maidsafe team is great with full disclosure also. You’re right about people being excited for mobile too :smile: simply can’t wait to play around and upload and retrieve some data!

6 Likes

What I’m getting is that this first release will be in house network that is globally accessible or least storage is in house; hence the 1GB limit. then sounds like mock routing gets swapped out for the real routing that is actively being worked on for the remainder of this sprint so we can all connect globally

3 Likes

Yeah, I know we’re not going full blown. I’m trying to get a handle on the user experience, in relation to the network. I got some data that they’re using digital ocean servers for the time being. That sort of keeps it “in house” but lets everybody play on a network that is starting to approximate the end product, but not yet there. I think that’s it anyway.

4 Likes

That is the short term goal, start putting this in peoples hands. Initially apps may be static binaries, but installers for vaults etc. This part will move quickly though.

10 Likes

@Brian_s

I think it would help to explain what you mean by the uploader and relate it to the screengrabs because although I can guess what is going on here it isn’t clear, and I think people will be at a loss to understand how you go from uploading files to seeing those screengrabs.

Crazy work guys! Im very exited for build my own website or project on Safe Network!

6 Likes

Ah yes, the uploader is a desktop app to select a service (from the DNS system, such as website) and a public name. Then you will be able to upload to this location, in this case a website. So a sample of an app to create static sites for now. Just a demo really of a decentralised web / html network.

As the API’s stabalise further then we will put out a lot of these examples. There will also be an examples zip/tarball to allow everyone to just run our examples without needing rust etc.

@Krishna_Kumar will walk you through that for sure.

15 Likes

@fergish technically non devs have been globally connecting nodes for about a month :wink:

Continuing the discussion from Running a publicly available crust node within a private LAN:

A huge thank you! to everyone who is obviously putting a lot of “blood, sweat, and tears” into building the world’s future network. You are an inspiration for all and stories will continue to be told about your special team for a very long time.

At least in my community the word is spreading fast and there is much anticipation.

13 Likes

Very exciting. I keep sharpening my rust skills, hoping to catch up down the road… but now python bindings!! And I ask myself why oh why must I be out of town this weekend when it sounds like this release will arrive. Thanks for keeping the rapid progress up!

1 Like

@happybeing The desktop application is like a File transfer application to the Safe Network.

The usage of desktop app would be,

  1. Select the service name and public name (www.happybeing.com), www is the service and happybeing.com is the public name
  2. Drag and drop the files to be served for the service(www) under the public name(happybeing.com).
  3. The desktop application would create the public folder and the dns mapping record in the Safe Network.

Then we expect the Firefox Add-on for the Safe network to be installed on the Firefox browser (Will detail the installation instructions once we are about to wrap it up). Once the Add-on is installed, just hitting safe:www.happybeing.com on the address bar of the browser would serve the page from the Safe Network.

We are also planning to provide a template section along with the drag and drop feature, i.e, drag and drop your own files or use the existing template (to help non technical users).

@Scott has illustrated the the desktop application walk through,

use an existing template feature is missing in the above image. Above is the initial version of design, later we thought of adding use an existing template feature for making it easier to try for users with no html background. So they can edit the template from the app and save/publish to the network.

Hope this gives some clarity :smile:

22 Likes

Not only does this explanation give some clarity; it also brings an unprecedented quantity of excitement :joy:

14 Likes

That sounds pretty bloody rad, @Krishna_Kumar. Do you have file views as well, something like an FTP client?

Is there anything I can pull on git to checkout the FF plugin? I’m keen to see what sort of API is available to us in the browser.

1 Like

Since this is the first application/example out of the Rust boundary, we have lot of challenges to resolve. So we are putting out a simple example, without much App functionality. We can certainly evolve the application with each iteration. We also have few restrictions from the API level as of now, but all those would be resolved in the upcoming weeks. This is just the first step towards porting the APIs.

I am not sure about what API you are meaning in this context(Safe/Firefox). The Safe Add-on does not expose any API, it merely intercepts the safe: scheme and renders the content.

1 Like

Aha, @Krishna_Kumar I was thinking about how to save something to the network, for example PUT a file upload etc?

I guess that’s something that’s out of scope of the FF plugin itself then at this stage?

You are right @joshuef, the uploading part is done from the Desktop Application at this stage.

@Krishna_Kumar
Can your public name consist of special characters, foreign languages characters, space and tab?
for instance public name : !ɔċ لعَ贼德…www
Have you tried stuff to break the SAFE dns? Like:
www…www…www.
safe:safe:safe:safe::

Will there also be an visual indicator that I’m visiting a SAFE website? I think you would know if your visiting a SAFE website when you log into the SAFE Network with your SAFE creds. But what if I unknowingly downloaded a malware Firefox Add-on for the SAFE network and I go to safe:www.happybeing.com without logging in? Not everything is what it seems on the current internet.

Will the Add-on be available through Mozilla or directly from Maidsafe?

If NSA/Attackers can change http: to safe: and serve content on the current internet, would it be detectable, if your not logged into the SAFE Network?

2 Likes