DevUpdate: issue threads

Development
1

Hi all,
Hi MaidSafe devs,

You guys are the heart of the MaidSafe community, and keeping up to date with the progress of development at MaidSafe from ‘the outside’ is not easy. As you can see from the update transcripts, several topics are being worked on at any single time. I was sort of picked as the involuntary volunteer to keep the Dev Updates going, and I’ve been pretty bad at it so far; maybe partially because every issue deserves a development thread on its own here on the forum.

I invite all devs to join with their story on a development thread, successes and set backs. Current issues off the top of my head can be account transfer, routing_v2, drive_v2, RUDP_v2 and RUDP_v1, NFS API, CRUX, Kernel Allocator, port to NDK, Example Apps, installers, Jenkins CI, and so many others.

Already from the length of this list it is clear I am not the suited person to report on so many different diverse issues, most beyond my expertise! What I can do, is stand as a translator and help moderate a non-expert discussion on these topics: what is the motivation, what is the aim, what is the implementation plan, where are we with it today?

With the start of maidsafe.org, around the same time, much of the technical discussions also moved to the internal mailing list away from the public mailing list. The value of the private mailing list is that it eases brainstorming discussions. On a public statement we tend to triple check our claims :wink:

Of course we have recently invested (and keep investing) a lot of effort in the system docs, there are public design documents and implementation proposals on GitHub, and most of all the GitHub wiki and the open-source code are all out there.

I would argue though that a low-entry place for discussions on current development issues is missing. So why not do that here ?; maidsafe.org deserves to be overflowing with development talk, collected links to more technical documents and discussions, open to questions for people to learn about our current work and to enable co-development.

Please share your thoughts here (both community and dev), and I will try to kick it off with a first topic soon !

Cheers Ben

3 Likes
2

A good idea and I am not opposed to posting some of the internal list topics, although some are very very deep in both c++11 and also routing/vault/crypto stuff so may lead to a ton of “what does that mean” questions. In any case this is a good move as the more open the better.

1 Like
3

A good discussion I know of is how to utilise the internal workings of MaidSafe in order to allow peer to peer connections with apps.

There is easily a sizable community of alpha testers, and also some plenty of dev. Unmasking the way to implement an app for the SAFE Network is ideal.

At least from this stepping place derivatives can easily and quickly be created and deployed and a development community can get rolling with concepts that will likely see launch.

2 Likes
4

I’ll be honest in saying that the community reaction to me asking for stakeholder feedback on RUDP v2 did not fill me with positive vibes to ever talking about internal technical debates in public again, unless very necessary (RUDP v2 would qualify, we have a very different design now to before the stakeholder feedback).

That said, no news is usually interpreted as bad news, and I have noticed participation on these forums has dropped off of late. Not sure how to proceed with that. On the one hand I’d like to be more transparent, on the other I would fear that more transparency would scare people because they don’t understand what we’re talking about.

Hell, most of our most intractable debates internally are mostly domain expert X trying to convey understanding into domain expert Y. It can take over a week, or even more. Even a very, very simple thing like kernel memory allocation has befuddled some extremely bright people, not because of the general principle which we all get, but rather how nitty gritty detail A fits with nitty gritty detail B. As an analogy, it’s like a brain surgeon specialising in the amygdala trying to confer understanding, within a week, to an expert in the hypothalmus. Sure, it’s all the same general specialised field, but you’d be amazed how hard it can be to grok a near neighbour specialism when your head is full of your specialism.

You’re talking a level of understanding that 99% of those attending the elite global engineering conferences would struggle with. Even on the Boost mailing lists, I would estimate maybe just five engineers are competent to discuss zero copy networking on all the major platforms, and even then our experience is often out of date because things have moved on since we last had experience. I would worry about too much transparency in this, it could make mountains out of mole hills when it’s really nothing like as big a debate or problem as it seems. To be honest, most of these big debates are really a consequence of lack of empirical data, and it’s about generating the steam to go write throw away code to verify assumptions before one wastes significant resources on a bad design based on wrong assumptions.

Niall

3 Likes
5

I personally don’t need too much… @dirvine comes in and clears up queries around the place…

2 Likes
6

@BenMS, thanks for putting this forward. I, and I’m sure most others, would really like to feel more up-to-date and find some aspects to engage more. There’s the urge to get behind an push but it’s hard to find the bumper.

But @dirvine and @ned14 are right about not bridging the technical data very much. These are debates that few of us can have useful inpup on, and it wouldn’t be very productive to the forward motion.

What I’m thinking would help is a bit more of a PR (in the good senses of the term) approach to how things are going, mainly with a sense toward reassuring the greater team that progress is being made and how real it’s looking to wrap things to the level where the broader team can get more directly involved with the meat and potatoes.

If it’s looking like testnet 3 and beta launch will be later than expected, we’d really like to know. If the dominos are falling pretty much as expected or better, even better. Some snippets about the sort of things being tangled with and what’s been solved would help us wake surfers stay in tow.

A short weekly fireside chat by @dirvine (or someone else with a high level overview) about how things are progressing, without getting too technical would be very good for moral.

5 Likes
7

Do it!!

I’d LOVE to hear the nitty-gritty!!

It will inspire me to learn code and help out :smiley:

8

Thanx for the update. I really like the analogy by @ned14 about the brain surgeon. I think that the devs should focus almost 100% on the project. It’s great to see people like @dirvine giving reply to questions about “how this stuff works” because even the system docs don’t provide all the answers. It’s also great to see people like @erick giving presentations on the inner workings of Maidsafe. But please keep up the good work as devs. Don’t put to much time into explaining all the details about kernel memory allocation etc. If people wanna learn about that they should search for it on the internet. When Safenet is up and running there’s gonna be enough time for anybody to ask about anything. And for devs to get some sleep :wink:

6 Likes
9

+1 @polpolrene, fully agree!

1 Like
10

If someone asks me how SAFE is going, I would have said the same thing at any point in the last six months which is “fine”. They then ask about whether deadlines will be met, and again my identical answer these last six months is “I have no idea, but my parts are generally on schedule”.

I’m not sure how useful any of that feedback is to the general community though.

Niall

11

The community don’t need the detail, what is most frustrating is having an impression of where the project is, that week after week turns out to have been optimistic. “X is coming soon” is understandably vague, but leads us to imagine something overly optimistic. “Y will happen next week” has been a regular refrain that has become meaningless.

As an ex developer I understand these difficulties, but many don’t, and I think it is very discouraging to observers, and it puts a damper on our efforts to interest and enthuse others, because we can’t know what to say about progress. For me anyway.

I’d suggest a focus on providing regular high level progress updates (i.e new roadmap) and avoid saying “next week” or “six weeks” unless you can almost gaurantee to hit that. Please don’t let the new roadmap sit there not updated! :slight_smile:

6 Likes
12

I agree very much. Don’t waste too much time getting too detailed. I imagine that with programming, frequently one has to go 1 step back in order to get 2 ahead. That kind of stuff is hard to understand from the outside and best left to the devs. I for my part am having a little party every time i see something crossed of the checklist (and a big party when moving to the next testnet :slight_smile: )

3 Likes
13

What I find interesting is how what is happening now compares to what was happening pre-crowd sale and prior.

David had the inspiration many years ago and spent something like 10% of his time coding and 90% trying to raising capital to hire other devs. This was a time before the convenient tools like Github, crowd funding and before Bitcoin was a twinkle in crypto daddy’s eye.

I’m guessing that were going to end up with a way better network than if it was fully funded and completed, say even 5 years ago.

To me, time to completion is not an issue…I fully trust David Irvine’s vision and his leadership with the Maidsafe team.

If there’s a voice that is missing on this forum, I would nominate Nick Lambert. Nick is out there promoting but he aint in here enough, talking us through the big picture.

Come on down Nick :wink:

4 Likes
14

I’m not talking about inviting feedback on the technical details, etc. That would be a waste and slow development, something no one wants. I’m talking about a regular something to help keep “the rest of us” in tow.

1 Like
15

I’d like to hear regular, short status updates on the roadmap objectives. ETA’s aren’t necessary, merely a line or two about whether things are going well or whether there’s a problem and how serious it is.

2 Likes
16

As much as I would like technical updates I also think it would be counter productive to progress. I think @happybeing and @fergish clearly spoke my sentiments. I think the dev update list showing who was assigned to what was a great update but clear and regular status updates def help moral. @ned14 I hope you aren’t too discouraged by less talk on the forums Most of us here are not prospectors and personally I never shut up about maidsafe. I’m quite good at giving laymen explanations of how the network functions and how it achieves its core ideals, but have been less “active” on the forum. That said I think you all are an amazing team accomplishing an amazing task that takes time! Best of luck to all of you. -Nigel

4 Likes
17

Wow, thank you guys for a lot of valuable responses, that includes Niall. I do think that a high-level overview from those dev that want to take the time can be valuable. It should really be short, but it would help both internal and external to stay up-to-date. Much like teaching class, it does take a lot of effort to explain something short and clearly, even when you’re the expert and working on it - especially then maybe :), it’s hard to leave out details.

18

I personally believe that until the SAFE network starts generating revenues for MaidSafe the priority for the team should be to focus on getting the engineering right. And the way for the wider community to obtain more detailed feedback about the dev process is to go beyond being passively reassured that progress is indeed happening and use that feedback to contribute back something that is both valuable to MaidSafe and the wider community. It needs to become a two- (or multi-!) way conversation for the time investment in explaining what is going on to pay off.

On our side, as we understand more and more how everything works, we are producing lectures that will make it easier for new comers to get up to speed. The time spent by devs explaining how stuff works is therefore paid back in not having to explain it again in the future and having more meaningful discussions with other community members.

As @polpolrene did here, you can pick one question that bothers you and publish it on the forum. I suggest to push further and publish subsequent posts about relevant parts of the SystemDocs that apply to your question but does not fully answer it and why, look at the relevant academic literature on the topic and publish the links found, look at the current MaidSafe source code and find the relevant parts and publish the links, and tie it back to the current tasks being worked on with links to Jira tasks. That way it will provide full context for others to join in on discussions afterwards. That process can be fully crowdsourced since MaidSafe is operating in a fairly transparent way. And I’ll be super happy to participate in it since it also helps me with the background research I need to do for the lectures.

And you do not need to do this all by yourself, you can do the part you feel comfortable doing within your time constraints. If enough people participate we will be able the share the load and quickly map the entire space. I’ll be more than happy in coaching interested people on how to do it.

Therefore, the way to have more and better feedback is to change it from a distraction to an investment for the person doing it.

4 Likes
19

Cheers Chris, yes I am nearly 100% in the code now (first time in many years), which is great, a lot to do fast. Everything is looking really much better by the day, I wish we had time, but we don’t, but we will have stability and future capable codebase that it very low level and detailed. So for that I am grateful to everyone, the team is really understanding and appreciate I want some last minute changes and alterations. Its only I was too far away from it, but not any more :slight_smile:

I feel much more comfortable with the new roadmap, short dev updates and people being able to get right into github and Jira, so all is open. Rather than timescales and promises, if we trod on through the roadmap, checking bits of every week, people will see real time progress much better. I am desperate though to get the client apps API out now as this is critical I feel. It allows us to figure out best practices in the core without affecting people developing, which will be all of us in it together at last.

Cheers again folks, it’s great to have backers who understand what we are building and how, amazing and incredibly helpful.

8 Likes
20

The roadmap on maidsafe.net is great by the way. There are things on the checklist that were previously unknown to me!