MaidSafe Dev Update - August 17, 2017

This week the team have and continue to work through a number of JIRA tasks that either fix issues raised by the community in previous tests, or that add features. This might potentially continue into the following week too. Lots of details about that below. Backend code is more or less at a frozen state in terms of required features for Alpha 2 (Authenticator) and frontend doesn’t look too far away from the same too.

We have also spent more time planning development in recent days to spread some of the engineering responsibilities around the team as we have a couple of guys either out at the moment (Adam is on vacation and Bart is relocating to Hong Kong for a few months) or, as in @Krishna_Kumar’s case, preparing to get married.

Congratulations from all the team here Krishna, we wish you and your soon to be wife (Rennee) a long and very happy future together, although she is potentially in for a tough time as she competes for attention with your laptop which may have to be surgically removed for the big day :smile:

Meetups

You may have noticed from the forum that the SAFE Network San Francisco meetup is becoming more active of late. @whiteoutmashups is helping this effort along with @hunterlester who have organised two events in the past few weeks, with the most recent being at 42, a tuition free university. @whiteoutmashups has made quite a few posts about the event which you can see in this topic.

Being at the epicentre of the software world and with a huge number of developers, San Francisco is one of many areas where we would like to see more engagement, and these events represent some early steps as we start a more focussed effort on spreading the word about the SAFE Network. We are now working to maximise the effectiveness of these events and have incorporated both internal feedback and feedback from members of the community. Over the next few events, we’ll be improving the way in which the network is presented, as well as improving the AV so that the content can be made available for those unable to make it to events.

Community-driven content

Continuing in the marketing vein, we would like to point out for those who have missed it, the effort being put in by a few long-standing members of the community including @polpolrene, @Krekc, @JPL and others who are starting a collaborative effort to pull together a new SAFE Network white paper/primer that brings together the most up to date content in one place. This comes on the back of a survey conducted by @JPL that gauges the SAFE communities attitudes toward the way the existing information is presented.

Marketing

David also took to his own personal blog a couple of times during this past week to write about The Impossible Network and to clarify the differences between the SAFE Network and alternative data/storage networks that incorporate blockchain technology. The fact that the SAFE Network completely removes intermediaries and humans from the management of our data through the network’s algorithms is a huge differentiator and the benefits of that is something we will be focussing more on in future marketing communications. We are also now starting to recruit 2 digital marketers to be based at our HQ in Scotland to help promote the network and drive engagement with the technology and the company.

SAFE Authenticator & API

The focus for this week has been to complete the tasks that were added to JIRA. The new IPC for requesting access to arbitrary MutableData objects has been integrated with the authenticator and also with both example applications (Web Hosting Manager and SAFE Mail Tutorial). We have tested the initial integration with the apps and authenticator. An application can set the Metadata for the MutableData in case it wants to share it with other applications as explained in this comment. We don’t force the devs to set the metadata. If the app is not setting the metadata and another application wants to access the same, then there is no useful information to be presented to the user for aiding the decision making on whether to grant access or not. Hence, we decided to provide an option to look up for the applications that have been granted access for that MutableData. With this feature, the user can see the list of applications that have been granted access before and then decide on whether or not to grant access to the requesting application. The API to get the list of apps that are granted permission for the MutableData is being actively developed by the core team and we will integrate the same with the authenticator.

Another major improvement on the browser this week was to enable the error codes being passed back from the APIs on failure cases. We were not able to pass complex objects across the DOM APIs and thus we were only passing the error message previously. @joshuef has resolved this issue and now we can pass the error codes back to the DOM API layer.

The API changes in safe_app_nodejs related to the new IPC for MutableData are also exposed from the DOM APIs. The DOM APIs are now up to date and in sync with the Node.js APIs.

The upload experience of files in the Web Hosting Manager app is improved. The progress bar used to update only when the file upload was completed and when a large file was uploaded, the progress bar would stay at zero percentage until the file is completely uploaded. The user gets an impression that the app has hung, while it was still uploading the file behind the scenes. Now we have fixed it by using the NFS API to write in smaller chunks to the network and the progress bar reflects the status immediately.

We have made good progress this week and the pending task list looks much smaller compared to where we were last week :smile:

SAFE Client Libs

We’ve been continuing to work on tasks that were mentioned in last week’s update, particularly improvements on the authentication side in the safe_core part of the libraries. We have finished with the refactoring and optimisation of the account packet structure and the account registration flow, which is now only the Authenticator’s responsibility (safe_core is concerned only about filling and putting the account packet to the network). These changes helped us to simplify and decouple multiple modules in the libraries.

@marcin is continuing to work on improving the new ShareMData IPC request, basing on the changes by @adam that have been merged last week. After some discussions with the front-end team, we have concluded that the mutable data sharing flow needs to be changed a bit: previously, we didn’t support sharing of Mutable Data objects without metadata because it wasn’t clear what we should show in the UI in this case - a user couldn’t tell what the exact purpose of the requested Mutable Data is just by looking at its name and a type tag. Now, we decided to implement a new function in the Authenticator API to return a list of all apps that have access to a certain Mutable Data - this way a user can infer its purpose by looking at the list of apps that have permissions to access it. It’s outlined in more detail in the JIRA task.

There were some other small improvements to IPC requests: we’ve revised and unified the APIs, making sure that there are no discrepancies in FFI structs. For example, we have changed the Permissions representation from an array of enums to a simple structure because using arrays is more complicated from the FFI point of view. There are other upcoming improvements in the safe_app public API that should fill some gaps: while we have a function, mdata_list_user_permissions, to get a set of permissions that an app has listed in a Mutable Data object, we didn’t have a way to actually read or check these permissions - this will be fixed soon. If you’re interested in more details, see this JIRA task.

Routing & Vault

The final PR for rate limiter which brings in the feature of soft capacity is now merged. We patched a bug in Routing which showed up in the soak tests after many iterations. It’s when two nodes want to connect to each other, prepare connection info request and send it to each other but are blocked from connecting to each other by the mock-crust test. The code in peer manager prepares connection info as a response to both connection info request and connection info response. Thus after the nodes realised that they could not direct connect and started to tunnel to each other which also failed for the first tunnel selected as this was also a part of the test’s simulation logic, one of the nodes got a connection info response from the other. This made its peer manager change state from searching-for-tunnel to preparing-connection-info and hand it over to node.rs. Unfortunately, this operation in node.rs was a no-op when called from handle_connection_info_response. Thus there was a disconnect between peer manager, which thought Routing was asking Crust to prepare connection info, and node.rs which ignored this and never contacted Crust. To fix this, peer manager is now patched to not change its state while searching for tunnel if it gets a connection info response. Eventually, other potential tunnel nodes would be tried and the connection would succeed. This fix is now merged. Soak tests with the latest changeset have been running for a few days now and are looking good.

A Vault PR to integrate with the latest Routing is now up and waiting to be reviewed. It has the necessary refactor mandated by changes in Routing which no longer raises proxy rate limit exceeded errors (as it deals with them internally), the details of which were stated in the previous dev update.

72 Likes

Let me Read it now…

5 Likes

Great news about the digital marketing folks. Just what the project needs at this stage.

13 Likes

very happy to hear that! :slight_smile:

18 Likes

Alpha 2 is gonna be sweet! Have to get someone working on mobile again though. Regardless, great stuff!

7 Likes

Nice too see a start of an shift in focus.

4 Likes

Yes please. There is so much good stuff happening but that is what I hope to hear about every week.

5 Likes

The Browser… :upside_down_face:

Ever watch ‘Stranger Things’ where they have the ‘upside down’ ? I’m sure they use the Beaker browser in that world :smile:

I live in Chrome and when I open this beaker based browser a major disconnect happens with my brain.

Do other browsers use that left side toolbar thing?

Chrome wraps up all the essential stuff up in the top right hand corner…nice and unobtrusive.

My suggestion…make the Browser look and Function like the most widely used browsers out there.

It’s my #1 disappointment with this project and it’s something so trivial, that I feel embarrassed to admit it’s the reason I don’t even bother with the Testnets (and I’ve been here since day 1)…workflow killer.

7 Likes

Awesome marketing blog posts, keep that stuff coming! Thanks for the update :tada:

1 Like

Good Work team. Alpha2 soon.

3 Likes

I feel like a chump. Congrats to @Krishna_Kumar! Also big thanks to @polpolrene, @Krekc, and @JPL for your contributions with the safe primer (white paper). The maidsafe team and this community rock. @chrisfostertv I think many browsers make use of the left side including opera if I remember correctly, its standard for minimizing and exit on Mac as well. I think the reasoning behind it is File, Edit, View, Help tabs are always on the top left. Like you said work flow :wink:

13 Likes

Technically, Beaker was a good choice, launcher integration is sweet…but the ability to switch GUI components around to suit the workflow of all users…should be on the launch roadmap.

2 Likes

Congratulations and best wishes @Krishna_Kumar , hope it all goes well on the day! And thanks to everyone who took time to fill out the survey.

7 Likes

I feel you @chrisfostertv!

Beaker was good to get going. But it’s not ideal, certainly.

We’ll be looking to improve the browser. Fret not!

16 Likes

Morning All,

Lots of work going on in the Frontend but not too much going on in vaults, is that because they are a finished or a nearly finished item? I thought it was basic data storage that seemed the tricky part of this project but now there seems to be a more advanced type of storage and ways of interacting with PUT data so that was maybe telling me vaults are done?

I was cut off from running vaults at home when we needed the port forwarding tweek on the router and haven’t seen much more from them since then. Are the way home routers are set going to be a problem and all need port forwarding?

Just feels like there has been a lot of scope creep for the front end which is fine but isn’t the killer app the backend?
Many Thanks
Al

These are different teams, the frontend team has grown quicker as a part of the hiring process. The backend teams are just as busy but on backend issues that cannot really fit in updates too easily. There we have secured gossip proposals, data chains part 2 and the data chains part 1 simulations → production code tracks. Also the integration of utp back into crust (we wrote it from scratch as existing implementations were sub optimal for us). So it’s a lot of tings that would make the updates too techie really and bring up zillions of questions those guys don’t have time to answer. Then the safe core/client rust work is happening in the middle layer if you like. This is a lot of ensuring front end API’s work as they should and refactoring where needed to get the API’s easy and hopefully error free.

hth

28 Likes

Thanks for the reply David, sounds like still some way off testing vaults from home, hope to see it soon though.

Many Thanks
Al

6 Likes

Thanks Maidsafe devs for another great update as always.

:clap: :stuck_out_tongue:

4 Likes

David,
I am just so thankful to you and your team for the hardwork. The fact that you taje time to read each of our message and like them show so much of your commitments and respinsibility. Keep going and stay strong!! You and your team can do it!!:raising_hand_woman::pray:

13 Likes

Glad to be helping where I can :slight_smile:

Never thought I’d see my name in an update!

4 Likes