For this week, the focus shifts to the upper layers (safe_client, safe_vault) to integrate with the actual Routing crate and testing of the same.
@ross couldn’t hold himself back and wait for the upper layers. He has started to run his own tests against the routing crate within the LAN and will be extending his test to the droplets in the upcoming days. As @brian_s advised in last weeks post, the Rust-utp integration with Crust is now complete and testing these new features will keep @ross very busy this week.
Just to summarize where we are after the rust-3 sprint:
- Unified structured data implementation (multiple modules)
- Name Service integration (dns crate)
- UPnP integration in Crust
- Memory map & compression in Self-Encryption enabling larger file handling
- Examples (Firefox Add-on and Desktop application)
A diverse set of functionality has been added to the network, giving it a clear direction and shape towards a stable, scalable and secure network. Though this is just the initial implementation, it would definitely benefit even further with the sprints to follow and backed by the technical debt sprints.
As for the client libraries, we needed to implement RFC’s (Unified structured data, Name service) as part of the sprint objective and with the sprint extension we also decided to set up a UI example showcasing the dns crate functionality. This certainly presented us (@ustulation and myself) with a few hurdles (setting the mime/type for a file as part of the file metadata itself) and has also given us a few ideas for the sprints to come. Porting the rust libraries to other languages has been a challenging task to say the least . There were several hurdles to overcome and there are still a few to be addressed to improve the process here (Using the SWIG port for nodejs and other platforms). However, with our current time limitation and considering this is our first crack at porting rust libraries, @ustulation has made the task look easy :D. We’ve currently exposed a handful of low level API’s through the FFI bindings (a mechanism which enables a program written in one programming language to call routines or make use of services written in another) which caters to the requirement of the dns crate’s example.
The Desktop application and the Firefox add-on are now close to completion. I’d guess elaborating a little on the progress with the examples so far would be helpful for the curious app devs here :). The overall idea was to create a desktop application using FFI and the Firefox Add-on using js-ctypes.
We decided to create the desktop application using nw-js formerly called node-webkit. The first week was pretty much the UI and behaviour integration for the application. All went smoothly and we were able to cross compile to all three major desktop platforms (OSX, Linux and Windows).
The Firefox addon was also developed to intercept the
safe: protocol scheme and serve the content from the SAFE network. This add-on was extended from the previous proof of concept which we did few months back. The dev tools for Add-on used by Firefox changed from what we had used previously. Now, we use jpm for the Add-on development. The scope of the add-on for this sprint is limited to intercepting the
safe: scheme and serving the content from the network.
We also did a quick analysis on chrome’s feasibility to support custom protocols. But unfortunately they don’t seem to support custom protocol handling through their SDK. Moreover, the Chrome SDK restricts serving the content from the plugin after intercepting the request. We looked at TOR’s implementation and found that the requests from Chrome are redirected to a proxy server via the add-on, which definitely is not an ideal approach for us. We decided to park Chrome add-on for now as we wouldn’t have been able to deliver a suitable solution within the sprint deadline, however, the Firefox implementation is very much a success.
Last week we started to integrate the FFI bindings with the desktop application and a new set of issues started to spring up. We had to follow a different build approach for building the native npm modules using
nw-gyp. Even after building the native modules with
nw-gyp, we were not able to use it in Windows after packaging the application. Building the native modules wasn’t straightforward either. After a long struggle we decided to move to electron formerly atom-shell over the weekend and the application is now ported to electron successfully. Electron seems to provide better support for native module integration as well.
While everyone is busy integrating routing to the upper layers, I would be fixing the desktop application and the Firefox add-on for this week.
Progress with installers has been good with both the apt repository and yum repository for linux being setup for the Vault binary. Once the vault library has integrated with the refactored routing API’s we’ll be using the same installers to set the network up on droplets. Installers for other platforms have also been coming along well and more info on this should be made available through this week as we get to testing the vault binary. Work in the installers also identified a few edge cases in terms of cache file storage location that various modules were relying on. These have now been finalised and is currently getting worked on being implemented and tested by the guys in the crust module.
Here is the weekly dev transcript