Devhub Trending on Hacker News

Yeah, I’m getting the same warning too…

6 Likes

A bit of a more technical question around how the minimum threshold of replicated data is maintained:

One thing as a long time follower and in the MaidSafe forum would be an active thread where community members can ask technical questions and get technical answers when the team has time to dive in and give feedback. Re-reading this bit I think my question would be: How does SAFE Network, which is decentralized and anonymous for its users, know how many nodes are currently running with its replicated data without a crazy amount of polling and network congestion for every chunk of data. Is the X nodes have this data checked at the time of request and confirmed on all nodes? If so that would be really slow but reliable to ensure if a 1-7 nodes have lost the data that that data gets replicated elsewhere. Also risking that old data that hasn’t been fetched in awhile could have already lost their potential 8 node replication. Now get even more chatty where all storage nodes constantly poll each other for their known chucks they posses, that polling at scale sounds like it would become a real chatty bottleneck to have to basically crontab that seeking functionality to ensure availability of data at all times. Would be good to have a thread in the SAFE Network forum where programmers talk with the team and grill on the nitty gritty to see if these aspects have been solved and can be linked to in the source code for study.

reply

7 Likes

This is exactly the aim of the Safe Network App, which we are working hard on.

One thing with having the streamlined team now is that updating and generally wrangling the websites has taken a bit of a backseat. It’s defo something we’ll need to turn our attention to when we round out the a major release.

Yeah, I don’t really agree with this, and obvs it’s been disussed a lot over the years. @happybeing gave a nice reply on that though.

7 Likes

I’ve just dropped in a wee reply too.

9 Likes

I’m getting an empty page when I click the link.

I think the above is an important point. Bad “marketing” can do a lot of damage.

3 Likes

Is it just me or is the comment system on HN incredibly hard to follow?

3 Likes

#MeToo – troo

1 Like

Thank you! I though I was just a complete lonely idiot.

1 Like

I guess the mods deleted it. It was a link to the primer

1 Like

Yep, I think this project needs an SLD: Service Level Design. Which should be documented by the lead engineers and David.

This should be a diagram that shows all components of the safe network with lines and info on how they relate. I have seen similar stuff put out by MaidSafe in the past but some seem incomplete or are now totally out of date with newer methodologies.

Once that is in place, each component of the SLD needs to be broken out and have technical documentation on how each component accomplishes its job within the network. Basically specs that if another coder came along and had no access to anything they could build out a SAFE Network clone from the specs. It would help reveal what has been solved vs still in progress, just put question marks around a portion of needed logic that hasn’t been solved or accounted for yet. The SLD and specs will also be helpful to maintain as solutions are found inadequate, strike throughs can be made appropriately and updated documentation for newer implementations can be put within them to show the progression of design. Ex: I think we thought parsec could be leveraged to validate certain things that it ended up being too slow or inefficient for, and we are switching to CRDT for those.

The SLD and Specs in Github or somewhere would give those in the technical community the ability to raise issues or questions on the design and hopefully lead to good iterations in open dialog on how things could be improved or where components might be flat our wrong or have bad assumptions etc and need revisions.

And the community Primer is not that, that is way too high level and provides 0 insight to how exactly all the cool stuff mentioned in it gets accomplished in code. Making an SLD and quality specs for something as big as this network encompasses could take a full month or two at this point even, but its honestly something that should have been done 6-7 years ago before folks touch code to iterate on and constantly look back to for progress and whats changed(through markdown revisions etc. in Github history).

11 Likes

All this is very true - however asking David to stop work and put everyone concerned onto this for 6-8 weeks just isn’t going to happen. Well I don’t think so anyway.

BUT it does sound like a very worthwhile, even essential piece of work. Could we as a commnity do as much of the grunt work on this as possible and let the devs fill in the bits that collectively we cannot fill in ourselves? I know this would divert dev resources but not to the extent that asking for it ALL to be done by the devs would.
Tell me I’m daft or discuss :slight_smile:

Here is a possible starting point. https://github.com/maidsafe/safe_client_libs/wiki#crate-graph

10 Likes

I don’t know who in the community would consider themselves close to as technically proficient at this as the MaidSafe team themselves. But as a start I would think folks like mav(I see him run lots of tests with the libs that are available), happybeing(number of poc projects and been around a long time), neo(forum admin, generally gives high level answers to some of the technical q’s around here), tfa(has ran a few community test-nets, mentions some bugs etc), shane(from his experience testing the client facing components) likely understand various aspects pretty well, Sotros may also have a bit of untapped knowledge as I am sure the community meet ups from long past have probably have lead to more technical discussions(again unsure if the discussions then were different implementations to how things are being attempted now). And those few people would be a start but even then I really don’t think they have the knowledge to actually spec out the components to the extent it needs to be that a independent engineer could develop SAFE as MaidSafe has intended it to be initially.

I agree we don’t really wanna take away dev firepower right now for folks that are committing code to the various components in github. But I also believe there is still a certain degree of flying blind and trying to rehash and fix things without a good concise trail of revisions of the where we have been to where we are now to what is left from a technical perspective.

5 Likes

Yeah thats pretty much the list of suspects I was volunteering for the work needed. I don’t think I myself could add much that would not be done more competently by those mentioned. But we can all try and add our bit, even if it was only proofreading others contributions perhaps.

We haven’t heard much from @Shane recently, I hope he is OK.

3 Likes

Do you think me preparing a VirtualBox image with safe-cli and browser all pre-installed would be a waste of time and effort then? I don’t want to duplicate work and muddy the waters.

2 Likes

I understand there is value in documentation such as this but personally I think it isn’t a priority given we have access to both weekly summaries and the code repositories.

What this means is that all that information is available, but not in an easily accessible form. The are many people other than Maidsafe who are capable of spending time creating the modular summary you seek, but I don’t think it should be allowed to distract the core developers. By going through the weekly summaries, asking questions here, and getting to know the repositories, somebody with moderate technical skills who is motivated to do this could produce it IMO.

Who do you think would make use of a modular summary, and for what benefit to themselves and also to the project?

I ask because I’m already agreeing there are benefits, but am not thinking they justify any distraction at this time. Later, definitely

6 Likes

SLD: Systems Analysts and Engineers, Tech Specs: Engineers. Specifically ones that want to see the big picture and then an easy path to every nook and cranny around its design without specifically reading reading source code line by line. If the spec sucks or has complications in a premise then no need to even comb existing code, enables them to quickly figure out where big problems are in design. It enables 3rd party developers and open source community members to quickly and effectively evaluate the project to determine if SAFE is vaporwear or golden even at the spec “sudo code talk” conceptual level or has legs to stand on as a potential full fledged product when fully written in src. At the end of every dev update they mention wanting community involvement, but I forsee 0 involvement for the most part around code or challenging existing design until these pieces are put into place.

Only thing the community can discuss right now are frontend apps and high level backend concepts(which I see many folk doing which is good at least) because we don’t have solid documentation around how the whole network will be built and function right now.

I believe the SLD and Specs will reveal all that is currently unknown, which I still foresee a good bit of, (which isn’t a problem either imo, stating this probject “the research is done its all hammer out code time” is never going to be the case, even up past v1 mainly because waterfall isn’t a thing anymore in modern software design, iteration and enhancment are key but getting a good first version that does the basics somewhat well is critical). Its better we all realistically know where gaps are and everyone works together to find the simplest yet effective solutions to the problems.

9 Likes

Thanks. I don’t see significant enough benefits here to distract the core team, do you?

I think most engineers who want to assess this would do as well to generate that understanding themselves - a summary only speeds up the early stages of this. I agree a summary would help the see more quickly if the system is vaporware, but we know that’s not going to be the outcome so your idea would be that it might encourage some to dig in and contribute.

Obviously that’s desirable but not more than getting the network to MVP asap. Some will see the progress and may dig in because of that, or for other reasons.

Same with the other benefits really. TLDR: I see benefits but nice to have rather than crucial at this stage unless the team have resources to spare from MVP tasks.

BTW They were doing some work of this kind not that long ago - the Wiki on GitHub - so we know they are aware of it and I expect will do more when the time is right.

3 Likes

Code is ultimate expression of the full specification. Good code is relatively self documenting, is always up to date (not stale) and defines precisely how the system behaves.

A high level language abstracts us sufficiently from what the computer is doing, without losing any precision; it just simplifies the interface, at the expense of outright performance.

A specification, on the other hand, is interpreted by engineers with details added to fill the gaps. If it is too high level, it isn’t useful and is of little use to engineers. If it is too low level, it is duplicating effort that the engineers will have to do either way.

Agile has become established for good reason. Those rigarous processes for creating specifications are grossly inefficient and are not needed in all but the most mission critical systems. If used elsewhere projects would slow to a crawl; it is too big an overhead.

This is why agile was born. This is why test driven development has become popular. Giving engineers the power to interpret a small set of requirements, write the code, while testing it in the process, provides three things - a very high level set of requirements (stories), code which is sturdy and not brittle (testable code is modular code) and engineering freedom and satisfaction.

Good developers will slot into a team and will learn what they need to know, when they need to know it. They don’t need to understand the whole system in either great or fine details - they just need to understand the module they are working on and how it impacts on the others (see point above about module, testable, code).

Ofc, there may be many modules with complex interplay between many of them, but the challenge of creating a good architecture is to keep these to a minimum and articulate function clearly. I strongly believe this is always what Maidsafe have concentrated on, with a steady approach of refactoring and reducing complexity with each iteration; indeed, it is the only sustainable way to approach complex systems, IMO.

Let’s not suggest diverting the team into unproductive avenues and claim it is the only way to bring developers in. Developers will contribute where they can and when they can, but probably only when it appeals to them to do so, IMO.

6 Likes

So how do we respond to those on DevHub?

“Use the source”? “RTF not-really-existing-yet M”?

not a challenge in any way, just value your opinion…

2 Likes

31 posts were split to a new topic: Mapping SAFE repository dependencies