MaidSafe Dev Update - July 20, 2017

Last week, we opened Test 17 to Trust Level 2 forum users. The focus for this past week has been on fixing any bugs that have been raised by the community in tandem with finalising the amendments to the Routing design (discussed below). Quite a few of the reported issues have been resolved in their respective projects, so we should be having some new binaries soon (early next week) with the fixes/changes. We have debated in house about the value of these restricted tests and think they provide many benefits. The forum users feedback has been incredibly good and very focussed. It seems there is in fact a greater degree of ownership and it is helping us for sure. It also allows our Engineers to engage more freely as well, so for now it is likely we will continue with these forum users only tests, for at least a while. During this test we have seen 1600 sessions from roughly 100 users storing around 3 GB of data across circa 39,000 chunks of data (2/3rd Immutable and 1/3rd Mutable), and the network has behaved as anticipated, which is very positive given the level of change between this and Test 16.

So short update intro, as we are iterating feedback and updates at the moment. We hope everyone is as happy with progress there as we are.

SAFE Authenticator & API

The frontend team keeps working on fixing the issues reported for Test 17, not only those reported on the forum or on GitHub, but also some issues we find with our internal testing activities.

The following issues were fixed:

  • SAFE Browser issue #73: “index.html not automatically read or added”
  • Web Hosting Manager issue #201: “Update app name”
  • Web Hosting Manager issue #213: “Insufficient account balance for this operation?”.
  • SAFE Browser on OSX didn’t start by double-clicking the binary because the locale folder was not available.
  • An issue with the creation of dmg and zip files when building on macOS was fixed as well.

The following issues are not reproducible on our end and we will be requesting more info and/or help:

  • Web Hosting Manager issue #212: “Upload directory doesn’t always work”
  • Web Hosting Manager issue #210: “Video file corrupts service”

We may not be that responsive to all the requests and questions we receive, but we certainly try to keep up with them and keep track of all the issues reported to make sure they all get eventually resolved. So we would like to encourage people to keep reporting all the issues they encounter as it’s very valuable.

We also continue with some parallel development tasks. @hunterlester has been working on adapting safe_app_nodejs to some recent enhancements to the NFS emulation API implemented in the safe_app library (see this JIRA task for more info).

@joshuef is working on some issues related to URI scheme, re-enabling the toggle safemode functionality in the browser for optionally accessing the clearnet, as well as having the browser store bookmarks on the SAFE Network within the browser’s own container.

@srini has been ramping up by testing the sample apps, studying the API, and helping with reproducing some of the issues reported, and @shona is working on some UI design enhancements for the Web Hosting Manager application.

There is also an ongoing effort to have the browser automatically free all safe_app objects allocated by a web app using the DOM API, either when its tab is closed or when the page is refreshed. This should solve several problems people are experiencing at the moment with reaching the limit of number of clients. We are aiming to solve this by early next week as we discovered it’s a bit tricky to find a way to uniquely identify the tab/page that is effectively initialising a SAFE App instance, as it can be done from a non-active tab.

SAFE Client Libs & Vault

The team has been busy working on the design and proof-of-concepts for some of the upcoming changes (including the operation recovery). There is not a lot of news on this part, but we’ve decided that some of the possible approaches won’t work and discarded them, and now we’re steadily progressing towards viable solutions. Apart from that, we’ve been working on minor improvements on the Vaults and SAFE Client Libs side. We have exposed new API functions to get the user’s account information: how many mutations has been used, and how many are left. This will help to display this info in the SAFE Browser’s user interface. This PR is being reviewed now.

Pull requests to add new developer configuration options have been raised for Crust, Routing and Vault repositories and they will be merged soon. These options allow running local vaults more easily, not requiring to recompile multiple libraries.

Continuing with fixing and testing the issues that have been discovered in the last week, @Qi_Ma has added a regression test to make sure that the issue related to the problem with group_refresh messages arriving out-of-order won’t be occuring again.

Routing & Crust

Work on Data Chains is ongoing. The results from the Ewok simulations indicate that Option B is a feasible design and we are now planning on implementing it in Routing. Apart from the details of the integration into Routing, there is still one point left to be polished, and that is the block proving algorithm (a way for nodes to request proofs for blocks they don’t have yet). Its current form seems to be working well in most situations, but there are a few edge cases related to the structure of the chain that we would like to handle better.

64 Likes

This is my post, there are many like it but this one is first!

Solid progress again this week… very good :smiley:

15 Likes

Not so much news this week but still great!

6 Likes

Great update. Can you offer some hints on “a while” ? You’re waiting to see something change? Something not change? Or want to run it for another 3 weeks?

1 Like

I’m very happy with the new DOM API for web apps - finally got time to play and have my apps authorising hooray! Next is figuring out how to use containers and NFS which Gabriel had been helping me with, so thanks to @bochaco and the team for what is looking very slick and powerful already.

I can see why Maidsafe feel that invited testing is quicker and more efficient, but would be disappointed if we can’t have both! Maybe two strands? Or if we can’t do that, can we extend invitations to people based on recommendations, as there are people who aren’t in this forum who I certainly trust and would like to be able to play and could give valuable feedback. Aral Balkan for example, but I see value in anyone really - if we can ensure the devs are getting focused feedback and feel able to leave the wider community to other forum members.

Whatever the issues here - I hope we can allow more people to play, not least so those putting websites and example apps can share then widely, and not just with each other. I want us to have our cake and eat it, and I’m not talking Brexit! :cake:

22 Likes

Holy crap storm! To have come at the time of the rust rewrite to being close to implementing Data Chains nearly brings tears to my eyes :cry:

This feeling is not unlike that of watching a loved one pushing through great adversity to then finally reach a point of both greatness and emotional relief. I’m very proud of this team and happy to have discovered this project at such a nail biting stage.

Standing from the sidelines not being able to jump in and help in the fight has been stressful to say the least, but at least the finals are approaching. :relieved: From that point there’s only refinement. PhD anyone? :smiley:

I’m still at the edge of my seat, but hopefully not for to much longer. Otherwise I might get a brain tumor or a heart attack at the very least. I need to see this through before I die though. Fingers crossed.

Stay strong team :muscle:

I’m rooting for ya! :tada:

18 Likes

Great update, as always a lot of great work in just 1 week.

I agree on this one Mark. If Alpha 2 is several weeks away we should include the rest of the forum as well. Not only to find even more issues but just to make people familiar with uploading websites and play and stuff. I know it’s probably some extra work on MaidSafe’s side (easy enough for us to want over 4K members on that testnet :wink: ) but a lot of people are very curious and some even never tested one of the old networks.

Keep up the great work!

13 Likes

Thanks so much for all of the hard work!

5 Likes

As others mentioned, this indeed sounds like Alpha2 is at least some more weeks away. As an active tester I’m more than happy with the progress but wonder what the main blocking issues are in order to reach A2. Is there a rough list, or do you just want to continue testing a little while until a higher level of confidence is reached?

One minor (but very helpful) improvement would be to being able to upload the same file twice (overwrite it). I know the WHM is just a dev example but currently it’s the only easy way to upload files so this would increase my testing productivity! :smiley:

3 Likes

wow that’s a killer feature !

glad to hear

4 Likes

Just to be clear (@happybeing as well) we have been discussing that alpha II would be all forum users, but maintain the must be in the forum link for these tests. I think at least until we have data chains part 1 & II in place. In saying that Alpha III will be routing nodes from home, so that’s all open to everyone, but limited as it wont handle our data types just yet. We may work out a use case for that, but that’s the secured autonomous network (the “impossible” thing we are doing :slight_smile: ) so it should allow plenty of external validation, investigation, hacking etc. at that level.

Never the less the current thinking in house anyway is stick with forum members through alpha II and maybe even beyond (remembering alpha III is everyone but ignoring that). So not stick with level 2 users but open out to all forum registered users? It’s not concrete but an idea we would like to get feedback on. It does seem more focused and real that way and also means we can use the forum(s) for support and feedback all in one place.

19 Likes

Wowhow am I just seeing this now?

Finishing groceries, then I’ll be back and can do a call if anyone wants

1 Like

I can be (one of the few, apparently) who can be behind that position (to be clear it sounds like we’re saying Alpha 2 and the test nets before Alpha 3) because Alpha 2 is a development test Network for devs etc. And it is that way because we (the community) asked for that.

So MaidSafe listened to us and changed some plans to make that happen. And this is a pretty logical consequence of all that.

Just my thoughts. I support it

9 Likes

I agree. If anyone’s keen to get early access, it’s not difficult to sign up to the forum, and it makes a lot of sense to leverage and build this strong & positive community for testing.

Restricting the network seems like an easy decision to make when unrestricted access to vulnerable test networks makes them easy to disturb for those who want to cause mischief.

Great update, and I’m particularly pleased to hear that the direction for Data chains is becoming very clear & implementation is soon to begin!

Keep up the fantastic work :slight_smile:

12 Likes

if anyone wants to do a call I’m home so let me know,

I know that @Joseph_Meagher wants to in about 2 hours. but let me know, am back

1 Like

I’m wondering. Routing and Data Chains being as complex as they will there be fairly simple way to upgrade the entire network with back incompatible versions of the two systems? Would it still be possible to do a complete disaster republish with an incompatible set of old data chains.

If in the off chance the upgrade process began network wide little before the global disaster occured, would these modified vaults allow or be allowed to republish or even route data? Has this edge case been considered? Or is not a problem with the current design?

2 Likes

Why?

If you insist on going down the path of compromise, then please ensure there is a safe only version for those of us that believe in a clean break from the clearnet…otherwise…sorry but this stinks!

3 Likes

Thanks for explaining more about testing thoughts. I’m still not sure about it, I want to share this stuff! And saying download this stuff is more straightforward than sign up to the forum and then, this is how you get an invite etc. so I’m not going to share as widely, but maybe that’s a benefit :wink:

What is the biggest benefit for you guys? Is it still about trust, or now mainly about quality of feedback, or?

Can you elaborate - does this mean home vaults first just doing routing, no storage? Then vaults with storage being alpha 4 perhaps?

3 Likes

Thanks team for all of those hard work, expected will see the next alpha release roll out

4 Likes

Another great update, thank you Maidsafe team and keep up the good work super ants
:stuck_out_tongue:

2 Likes