Thoughts on Safe App Dev + This Week In Frames

Thoughts on Safe Development

After investing a non-trivial effort in developing an app for the SafeNET (check us out on github at: GitHub - Frames-Proj/frames). I have a few thoughts on the Safe dev process.

Protocol Oriented Development

The lack of ability to write any sort of centralized logic might seem intimidating when you first start trying to think about how to design an app for the SafeNET, but once you realize that your job as a developer is to design a protocol rather than write a server, things become easier.

There are some things we are used to on the clearnet that become difficult in protocol-oriented development (such as the ability to have very fined grained control over what your users can and can’t do). Some things seem to be impossible (such as maintaining an efficient search index). As a protocol designer you have to think about attack scenarios first and foremost, taking into consideration the inevitable shenanigans of malicious users.

One attack scenario we thought a lot about while developing Frames was the possibility that a user would decide to unpublish his or her data. I think any app that goes live on the SafeNET should publish its protocol spec before going through a beta period on a testnet of some sort where the developers ask the community to break the protocol. Even better would be to publish a library that already knows how to talk the protocol.

Tooling

App devs could definitely use more and better tools. This is to be expected for a project still in its childhood like the SafeNET. The mock_routing crate is great for developing against a consistent interface, but it does have a few drawbacks. The most obvious is that you are not working with a real network. To solve this some sort of safe-in-a-box style solution would be good to have. Another issue with mock_routing is that as far as I know it only supports a single user. Testing the interaction between multiple users is vital for protocol development, so this feature should definitely be added at some point.

The concept of the safe_launcher, to provide a central gateway for access control on the safe_net, is good, but I’m glad that the community decided to switch to the authenticator. The first issue was performance. Frames makes dozens of requests a second when it is fetching a screen full of un-cached videos for the Discover or Me page. The launcher is slow at the best of times but when I opened the log tab to watch the requests stream by I was only seeing about one request per second go through. The second issue is the lack of a headless alternative. Frames only builds the code during continuous integration (automatic testing run on every PR) even though we have lots of tests that we would love to be running. In order to do this we would have needed a version of the safe_launcher that did not have a GUI and automatically authenticated.

I talk about the launcher in part to try to make sure people understand what the pain points are for developing real apps. The authenticator cuts out a whole layer of unneeded REST calls, and gets my access to the network a comfortable distance away from a UI thread, so it looks like performance is moving in the right direction. I’m a bit more worried about the lack of a headless alternative for developing against mock (or safe-in-a-box). I could obviously fork off (and that is what I will eventually have to do if this does not get any traction), but I would rather that a headless authenticator/mock network is maintained by the core team.

It has been pointed out to me that it is possible to just generate an auth token in the GUI and store it in a config file, but I don’t want to do that for a couple of reasons.

The most important reason is that I don’t want to store access tokens in the clear. The authenticator acts as an awesome password manager, and I don’t want users to have to worry about their tokens getting stolen. The more we can reduce the attack surface of user’s interactions with the network, the better. I know that there are good arguments for storing tokens for IoT applications, but there is a security trade off that I don’t think is worth while for apps.

The second reason is continuous integration. I’m not sure how pregenerating an auth token will allow me to run travis tests because each test will stand up a new mock (or safe-in-a-box) environment. The least important reason is just a personal preference for command line applications.

This week in Frames

I promised I would include updates on what I’ve been doing on Frames during the week whenever I made these posts. I noticed that GitHub - Frames-Proj/mock-safe-launcher: A pinned version of the Safe Launcher for use testing Frames had suffered a little bitrot, so I fixed that. If any of you were having trouble getting it to work, you should have better luck now.

I’ve been working to get a mock-authenticator built, but am blocked on a build bug from upstream. I have been working on Frames mostly on the train when I don’t have access to the Internet, which is fine for app dev, but tougher for debugging build bugs, so my forward momentum has slowed a bit. I’m going to try to get some sort of mock-routing solution up and running so I can dive into API migration, but it may take a few weeks. One possibility I want to look into is to fork safe_client_libs and alter it to auto-authenticate when built against mock (or more likely add a new feature flag).

27 Likes

Excellent and reasonable critiques Ethan, I think @dirvine, @Viv, @anon86652309, @bochaco, @ustulation would appreciate these.

6 Likes

Great feedback for the team!

7 Likes

Hi @edapa, thanks a lot for all this, we really appreciate it, we need this type of feedback from community to start coming.
I just want to reply to couple of points quickly now before I’m able to go deeper into your feedback.

This was solved, so you should be able to build it with mock without problems now.

if you build the safe_client_libs with --features="testing use-mock-routing" you can then use the following from the safe_app_nodejs to get a SAFEApp instance already connected to the mocked network: app.auth.loginForTest(permissions) https://github.com/maidsafe/safe_app_nodejs/blob/master/src/api/auth.js#L258

Note you need to set NODE_ENV=development

7 Likes

That’s pretty impressive Ethan @edapa . I’m pleased to see you guys jump in the front seat here. Inspirational. Great work.

3 Likes

Have you looked at these threads?

And this post by dirvine

Search though from a decentralised perspective can make use of lucene type indexing, with the indexes using using Immutable Data, this has to be arranged in a manner where it can append information as more links and data is found.As data can be referenced via a datamap hash then the end data will always be found (like way back machine inbuilt to the network).

So the research part if you like will be fitting this in a type of binary tree/Directed Graph where leaves can be updated to reflect new information that fits that part of the object (graph).

So very possible and a few whiteboard sessions could product a crude first attempt at this. If done correctly and without doubt in the open then I think we can have a decent, albeit crude search.

More interesting though will be the use of deep learning algorithms to provide more than search, so more like wolfram type question engines with larger data sets. I spoke briefly with Wolfram and I am sure there is a very long conversation to be had there, with the new data structures available in SAFE as well as their capabilities in domain specific languages in this field.

2 Likes

Created a feature request task in JIRA for allowing concurrent modifications. With the APIs and other functional features getting stable @nbaksalyar or @ustulation would be able to support this soon.

7 Likes

I care less about concurrent writes and more about being able to test with two different user accounts (though obviously testing for race conditions is really important as well). Basically I want to be able to test permission and access control properties of the app in an automated way.

1 Like

That looks awesome. Thanks a ton! Is the testing mode documented in the authenticator/client-libs README? If not it would be a great thing to include (I can open a PR though I might be a little slow to do so).

4 Likes

For posterity here is a perma link to the function you are talking about: safe_app_nodejs/auth.js at 572ccfb85bb0c5dc533b673c19a041849b96bc95 · maidsafe-archive/safe_app_nodejs · GitHub

3 Likes

Any chance of an update @edapa no pressure :wink:

2 Likes

@edapa enter apps into our monthly $125 safe apps competition!!

I have not really looked at it for a while after I ran into issues with getting started with the authenticator API. I may try to get back into it when the semester starts up again.

5 Likes