MaidSafe Dev Update - August 3, 2017 - Test 18

Today, we are pleased to be releasing Test 18 for all forum users who are at least trust level 1 :tada:

This is a new network, so trying to access data from Test 17 won’t work (you need to create a new account). We are planning to take Test 17 offline this weekend (both the internal test and Test 17 itself).

Here are the main changes compared to Test 17:

  • The storage limit has been doubled from 500 operations (store and amend) to 1000 per account.
  • Add new FFI function to get the account info from the network (number of available/used mutations)
  • Implement operations recovery for account creation, app authentication and app revocation.
  • Modify the client Rate Limiter paradigm to not put an upper threshold on number of clients with unique IPs to bootstrap off a proxy.

It’s also now much easier to run your own local network (because of the new developer configuration options that have been added to Routing, Crust and Vault). See this Dev Forum topic for more info.

Earlier this week, we started to document how to use the mock, local and actual network (for developers who want to build SAFE Network applications and websites). See this Dev Forum topic to read our first draft.

For Test 18, we are intending to have some front-end updates next week to introduce an option for apps to request access to arbitrary Mutable Data objects (see this JIRA task for more details). The network itself shouldn’t require an update (unless something unforeseen occurs of course).

Based on the feedback from the community, we are redesigning the UI for Web Hosting Manager app. You can see the new design in this Dev Forum topic. Please do share your feedback on the latest design.

SAFE Browser v0.4.0

To connect to Test 18, you need to use SAFE Browser v0.4.0.

Accounts can be created using the authenticator bundled with the SAFE Browser. To open it, click on the Authenticator icon in the toolbar of the browser.

This testnet is limited to forum users who are at least trust level 1. See this topic for more information.

To obtain an invitation token, trust level 1 forum users can use this website, which can also be accessed via the SAFE Browser itself when creating an account.

Each client account is limited to 1000 operations. It’s worth noting that this number is just a temporary placeholder and is shared between all the apps on a given account.

Please be aware that we might need to restart this test network, wiping all data that is stored on it. We will announce this in a dev update (when possible) prior to taking it down.

Download

Changelog

  • Freeing of safe_app object handles. Object handles created by the applications will be automatically freed on closing the tab or even on refresh.
  • Toggle safe browsing. Allow browsing of clearnet sites by toggling from File > SAFE Browsing Enabled
  • Index file to be read automatically. Providing the <service>.<publicid> as the URL loads the index.html file of the service.
  • The number of store/modify operations completed for the account is displayed in the Authenticator.
  • Enhancements to the NFS API are now being exposed to the DOM layer for web applications.
  • safe: protocol URLs will be opened in the SAFE Browser when opened from external links.
  • localhost: protocol is now supported. In order to open a URL like localhost:<port>, the URL must be converted to localhost://<anything>:<port>, where <anything> can be any characters but not empty, for example, localhost://p:3003/.
  • Several UI enhancements and minor fixes have been introduced, like ordering the application list, reducing the header size, loading overlay has rounded edges.
  • Issues with the building process for mock and real network have been resolved.
  • Compatible with Test 18.

Known limitation

  • History and bookmarks are not saved in the network right now.

Example applications

Known limitation

  • Public IDs created via examples are not exchangeable between the apps.
    • This is the error reported with the Access Denied / Requested Entry not found error. This issue is related to the permissions of the service/mail mutable data. The root folder with the default folders are designed for easier sharing and collaboration between apps. Applications can request access to these while the authenticator manages the app access levels. In the case of public IDs, the public ID is added as a key into the _publicNames container. A corresponding MutableData is created by the app and the permissions for the same are set by the app. This MutableData is then added as the value to the public ID entry. Now when the second app tries to add a service to the MutableData created by the first app, it doesn’t have the permissions to do so as this was not created by the authenticator. To resolve this, we are going to introduce an option for apps to request access to arbitrary Mutable Data objects (see this JIRA task for more details).

Web Hosting Manager v0.1.2

When uploading files using Web Hosting Manager, the maximum file size is 25 MB per file.

Download

Changelog

  • Present more user-friendly error messages.
  • Long public ID and service name are truncated and shown on the UI.
  • Internal use of the NFS API has been adapted to recent changes in safe_client_libs master branch. The NFS API was using ImmutableData to store the files and now the API is updated to save DataMap pointers. For each file, a pointer to the data map is stored in the MutableData entry representing that file. The actual data map is stored in the network either encrypted or plaintext depending on the privacy level specified.
  • Window resizing is now disabled.
  • Compatible with Test 18.

SAFE Mail Tutorial v0.2.2

Download

Changelog

  • Compatible with Test 18.

SAFE Web API Playground

A new tool called SAFE Web API Playground is now being formally maintained in MaidSafe’s safe_examples repository, which allows any developer to learn and experiment with the DOM API in a friendly and interactive way.

It is meant to help people aiming at creating websites/webapps that need to access the SAFE Network DOM API, which is available on the global window object in the SAFE Browser.

Instead of having to upload your site to the network every time you want to test some code, you can simply use this tool to experiment and get a deeper understanding of how each API works.

In order to use it, follow the instructions found in its README, and either upload it as a web app using the Web Hosting Manager application or just run it locally and access it from the SAFE Browser using the now supported localhost protocol (see above for details).

It is also available on the test network at safe://webapi.learn/, however, it is much faster when accessed locally.

Repository

Example HTML files

An examples.html file has been published with the safe_examples release. This file showcases some basic use cases of the DOM API, like authorising an application, creating MutableData objects, as well as reading and iterating over MutableData entries. You can upload this file using the web_hosting_manager application and publish it as a website, and then run any of the three examples available from the SAFE Browser.

Another file called leaking.html has been also published with the safe_examples release. This file can be used for testing the browser capability of freeing safe_app object handles. If you click on one of the example applications it will create a client and authorise the app but it won’t free them, causing a leak in clients. After running any of these examples more than twice you will start getting a connection error due to the limit in the number of clients being reached. Refreshing the page will automatically free all leaked objects allowing you to run any of the examples again.

Support

If you need help with anything related to SAFE Browser, Web Hosting Manager or SAFE Mail Tutorial, please use the #support category of this forum.

Where should I report issues?

If you know which component a bug is associated with, feel free to report it on GitHub in the corresponding repo.


SAFE Authenticator & API

  • Upgrade safe_client_libs to latest in master branch.
  • Support the new NFS API from safe_client_libs. The safe_app_nodejs library was internally using ImmutableData to store the files, but this is now being handled directly by safe_client_libs.
  • Allow the logPath function in the API to receive a filename parameter to automatically generate the log path for it. The path will be the same as the path of the safe_core lib log file.
  • The appToken parameter in the DOM API functions was renamed to appHandle to make it more consistent with what it really is.

SAFE Client Libs

The pull request for operations recovery in safe_authenticator has been merged now, but this road doesn’t end here and we are planning on several upcoming improvements and fixes for this feature - all of them required to be sure that we have covered most of the known edge cases and this feature works as intended. One of the possible limitations is the concurrent app revocation: for instance, if a user wants to revoke apps on their PC and a mobile device at the same time, it’s guaranteed by the design that nothing wrong should happen, but we want to verify it and we’ll be improving our test suite. Also, we already have a plan to improve, optimise, and simplify some things related to how apps authentication and revocation work, so that’s what we’ll be working on for the rest of this week. However, as these changes are mostly just nice-to-haves and the main functionality is implemented & tested, we’re releasing new versions of the SAFE Client Libs crates with what we already have.

The team also made a few additional changes. @marcin has improved the test suite for NFS, both on the safe_authenticator and safe_core sides. @canndrew has finished his work on the new feature to share Mutable Data objects between apps through the Authenticator, and his pull request will be merged soon. @adam has updated the repository to use the recent stable Rust version (1.19).

We’ve also been involved in debugging an interesting bug that was reported by the front-end team as we weren’t sure of its origin (on the Rust side or on the front-end). As it turned out, it was neither: we’ve traced it down to node-ffi (the library that we use to communicate between JavaScript/Node.js and Rust) and discovered that this library blocks the execution thread while waiting for a callback to return a result. This comes into a conflict with our rule about non-blocking front-end callback functions and this behaviour caused SAFE Browser freezing in certain circumstances. We’ve tracked down & fixed these particular cases but the problem still remains and soon we’ll be discussing how to fix it properly.

Routing & Vault

The Local Network support via dev options is now fully integrated and tested in both Routing and Vault. There were a couple of fixes regarding rate-limiter. In this commit, we make sure to send enough information to identify and clear the message from the acknowledgement manager in case we get the rate-limit exceeded error. When this error happens, the whole message is forgotten by Routing and the upper libraries are informed, which then have the responsibility to handle it the way they want. Safe_core for instance is going to retry the operation after a wait so that it doesn’t keep getting the same error back. This PR changes GETs to PUTs (mutations) as all it is testing is whether the request reaches the correct number of nodes at the correct destination. GETs are charged for the worst case as we don’t know in advance how much traffic it is going to cause. Changing to mutations allows the test clients to invoke the RPC pertinent to the test without getting an interference from the rate-limiter.

For vaults, we now get the group size and quorum from Routing where it can even be tweaked now that Routing has a config file. The rest of the changes in vaults were to integrate with the Routing changes.

78 Likes

Thanks so much for the update! :slight_smile: :slight_smile:

6 Likes

Yay… test 18! What do I win? :rofl: :tada:

15 Likes

ALL of the above :smiley: :smiley:

25 Likes

Great update, some nice info added as well…

nice

safe://polpolrene/
safe://elvis
safe://rene
safe://davidspeaks
safe://tetris

17 Likes

Test 18???!!!

Woooo.

7 Likes

HOSTING COMMUNITY VIDEOCHAT Q&A HERE!

9 Likes

click here to see safedemo live on testnet 18
puts

It fills up fast, only uploaded a website

8 Likes

Test 18. Hiep hoi!

safe://bit.miffed

4 Likes

Another test. Great to see :slight_smile:

What is/are the implications of this.

EDIT: Finished my Breakfast Coffee and realise this is just one client per IP Addr per proxy

5 Likes

For those interested in cloning the web API playground, use this repo instead: https://github.com/hunterlester/safe_examples/tree/master/safe_web_api_playground

It contains some important updates not yet merged into originally linked repo.

11 Likes

Can anyone (@Joseph_Meagher?) explain what has changed here? I’d have thought pointing to immutable data was achieved by pointing to the data map for the immutable data, so I’m confused! Nothing new there :wink:

6 Likes

just speaking from a developing standpoint the difference between two versions is you can now
directly use handles to retrieve an NFS file for reading ,writing without having to go over to immutable data functions vs previous api the best i could get when requesting a file was the datamap but when i tried to give that to the immutable data it just kept on failing no matter what i did.

10 Likes

All great news and just want to comment on how well the team is working together! You all are really becoming a finely tuned engine and flowing very well. Very cool everyone!

13 Likes

Can’t wait to get trust level 1 and start playing around.
It’s so exciting knowing we are getting closer every day towards safe and decentralized internet.

Kudos to dev team for all their great work. These updates are super helpful.

:clap:

14 Likes

Welcome to the forum. Just read a few more topics and your there :slight_smile:

There certainly are exciting times ahead for this project.

6 Likes

thank You

I should have done it a while ago. I have been lurking around this forum for about a year. Better late than never I guess.
:joy:

Indeed

4 Likes

I’m testing on a win x64 cheap tablet.

safe://home.sam

You can play some games.

My Account Status: 993/1000 a day…

3 Likes

what after test 18? Alpha 2 ? or test 19…

3 Likes

Got new good Internet speed weeks ago. Wanted to try uploading something.
Started trying half an hour ago or less. Win7.
Got the Browser. Got the trust activation code. Registered a Secret and Password.
But it’s stuck in an infinite buffer.
Same happens when I try to view safe:// sites. (It also says it wants to open in another program, allow/deny.)
Tried restarting browser. Same thing.

2 Likes