MaidSafe Dev Update :safe: 16th November 2015

BUGS!!! I should just end the update here, as bugs are challenging the team, both software bugs and health related bugs. For most of the already depleted team (bloody bugs) time is being spent identifying and squashing Routing bugs as well as improving error handling to make these less likely and easier to resolve in the future. This is not the kind of Dev Update we like to deliver, nor the type you probably want to read, however it is what has been happening in the last week. So we all have to open wide and take our medicine….

75% of the Engineers are working in Routing, identifying issues and creating a roadmap to get to a successful implementation. These issues are taking longer to resolve than previous bugs as between RUST-3 and RUST-5 some code has been uncovered that will not allow for a stable network to be implemented and this is currently being addressed. The work is happening in a branch called cleanup for those interested in closely following progress.

While debugging, the Devs have identified that RUST channels are being overused and making debugging issues particularly difficult, so we are removing / reducing our usage of RUST channels. Our focus is also centered around reducing accumulator (message and refresh) code; stripping down and simplifying things and enforcing the use of the Accumulator crate when possible. The routing table functionality is also being extracted from Routing and will now find a new home in a crate / library of its own with its own tests.

A lot of effort has gone / is going into improving error handling, so no future errors should fall under the radar and then be ignored. When all of this work is combined and completed it shall allow a functional TCP network to be implemented with the UDP integration to follow. Some medicine tastes horrible to take but ultimately it makes us better and stronger; the guys have been working flat out and the focus from the team this week has been to KICK OUT THE BUGS! As you will appreciate, Routing is one of the fundamental network layers so we have to be absolutely sure this is rock solid before we start the roll out.

One library that has taken the prescribed spoonfuls in the past couple of weeks is Crust and it is ready to be benchmarked, specifically throughput benchmarking / testing. With MIO being supported by RUST-UTP soon, we are also keen to compare and publish our findings. So more to follow here. Work on the Crust RFC is also now complete. The new website is making good progress as well, with all tasks now in JIRA, making following progress much easier, and an advanced draft will be reviewed by everyone in Troon next week.

In other news, Andrew will be in the Troon office from Monday (23rd) for a few weeks from Australia, to enjoy our pleasant Scottish winter, my advice: bring a warm coat. This will mean that from next week both Routing maintainers will be in the Troon office.

So, a frustrating week for us and for you, but on the plus side we have the brightest and most experienced team members giving this their sole focus. The issues we are experiencing are implementation issues and we are very confident that the underlying design is correct. @Viv and @dirvine are managing the work carefully to ensure that the team understand all the issues prior to putting the appropriate fixes in place and we will keep you updated with progress. We are close to providing the deliverables of RUST-5, but there is just a little more work to be done.

46 Likes

Thanks Ross for giving it to us straight. Nothing is more defeating that to find out later things have been sugar coated.
Maidsafe has always been honest and open and that is why you have earned our trust. Keep the honest communication
flowing. Best of Luck

30 Likes

Awesome work take your time dont rush

11 Likes

We’re on it, sorry folks but bugs are bugs, no way around that. Unfortunately routing has allowed some issues to go unnoticed via mistakes in the Error handling and also silencing compiler checks that would have identified some of these small issues. It won’t happen again as we will ensure there are commit hooks to catch such behaviour. As I said last week, the crust issues really prevented routing from testing properly and the panic causes these kinds of slip ups, it’s understandable in a way, but unfortunate for sure.

To be fair the Qa guys were reporting this and there was a continuous reminder to fix, but it was not done. It will be done now though and made solid. Plenty of eyes on and deciphering some of the logic to make use of a safe fast language. The speed recently had not helped but this is minor work to sort it all out, you will see the progress and it wont be slow either :smiley:

You will see a lot of commits to the cleanup branch to make logic more obvious and to handle errors as rust expects them to be handled. Not a worry it will all be made right fast. I am just delighted this is Rust as that prevents really bad code errors and you do need to go a good bit to stop it complaining at you. So thankful for that, indeed.

At the end of the day it is my job to catch such issues and ensure they do not happen, so it’s on me to accept the responsibility and also to resolve the issues efficiently. Glad to have a good team to make that happen so that is great. As we move forward the average capability of the team grows and will do as we continue the recruitment drive (not simple to find hens teeth :wink: ). Thanks again though this will help us reduce the code again a little, removing the small amount of complexity that just was allowed to slip in.

We will make sure the app devs can continue with their app testing and building while we get this part resolved, which will help. Few late nights never hurt anyone :wink:

31 Likes

Thanks for your hard work Maidsafe team, you make my Tuesdays feel like a birthday every week. :stuck_out_tongue:

P.S. I also incredibly love how you guys always give the news as it is. Even if it’s bad, it’s good, because we know our goal.

14 Likes

Thanks for the update! Keep up the good work, lads!

6 Likes

That sounds good, thanks for the update and all the hard work. Best wishes also to those under the weather (literally and metaphorically). My thanks and best wishes to all at MaidSafe… so near :smile:

13 Likes

+1

I could/should have said this. :slight_smile:

4 Likes

True that. At the speed you’ve all been going the past sprints, some nasty bugs were inevitable at some point. Still, it sounds like the fighting spirit is still there, so you’ll be fine. In a few days/weeks you’ll wrap this up with a great deal of satisfaction! Squashing bugs feels great after it’s done!

9 Likes

funnily this is my favorite dev update so far. because it puts all the past (and future) ones
in perspective. now i know that good news really means good here and is not just empty talk. should have been obvious anyways but can be hard to tell for people who are not looking at the code itself. keep it up guys!

8 Likes

Congratulations guys.

1 Like

Keep up the good work. Bugs are always a pain, but it feels so good when they are fixed.

Had a Boss in my R&D days and when we were fixing the project, he would find the solution to a bug and triumphantly announce that he found the last bug and fix it. These bug fix periods saw him do this 3 times a day for a week :smile:

8 Likes

This update to me is one of the most valuable communications I have read to date (Have been following since April '14). How people/organizations deal with set-backs/failures added with the insurmountable pressure you must feel, I don’t even want to imagine, shows what people are really MAID of :wink:

Thank you for showing us how to build a company/vision/network the way it is supposed to be done. I am behind you guys and gals with everything I have.

It just so happens I am an expert with human bugs (Finally a chance to use my craft for good :relieved:) and you are welcome to message me with any questions or concerns you may have. I will do everything in my power to optimize your super computers.

16 Likes

Yep we had one of those in MaidSafe as well for a wee while :wink: amazing what you can ignore after a short time. Takes all sorts, but better without some I think :smiley:

7 Likes

Do you mean that it was using nightly or it would create an unstable network? (and if so, what is it about the code that would make the network unstable?)

keep up the great work guys! all the community is with you!
there’s no bunny without bugs :smile:

3 Likes

That’s not a bug, that’s a feature!

3 Likes

Haha, unfortunately some bugs are bugs! But I’m sure that we’ll see plenty of features before the v1.0 release!

5 Likes

Great work folks. Just keep going. We’ll see the result whenever it’s ready. In the meantime we can ask more folks to have their personal safe:website ready. We start filling the network the moment it’s there ;-).

5 Likes

Hey @smacz no I wasn’t referring to it using nightly, I did mean it was creating an unstable network, with behaviour we would not expect to see. For example I was aware of a node connecting issue (now resolved), I am not at the coal-face on this one - so for specifics best to check out > https://github.com/maidsafe/routing/commits/cleanup

6 Likes