Update 03 February, 2022

Not all heroes wear capes: some of them carry laptops. There’s a palpable sense of relief in the air as Andrew James has finally worked his notice and rode in to take over the financial duties and admin, leaving a grateful @dirvine and @JimCollinson free to do what they do best. As well as Andrew’s timely arrival (more from him below) this week we also look at the main strands of the project, the ground we’ve covered and the road yet to travel.

General progress

@yogesh has wrapped up with PR#986-Liveness tracker fixes(how we tell if a node is still functional whilst kicking unresponsive nodes) and PR#976-Active Data Replication that proactively replicates data when a node starts showing bad behaviour. Both of the changes improve and maintain data consistency and availability in the network.

@davidrusu has merged the code simplification work done by @anselme which we talked about last week, so we’re now sharing the consensus code between the handover and the membership algorithms. Next stage, after a bit of bug bashing, will be to integrate the work @bochacho is doing on handling joining and leaving nodes. All looking positive so far. :crossed_fingers:

And in DBC-land, @danda continues to move forward with Ring CTs. The good news this week is that after a couple of detours it looks as though ownerless DBCs and spentbook transaction validation will be possible, with the current design giving added security as mints from other sections validate spentbook entries. Far too much to explain in a short paragraph but we should be able to bring a full description in due course, once the hard graft is done.

Welcome Andrew!

Hi everyone, I’m @andrew.james and as noted above I’m joining the team as the Head of Finance. I started my career as an auditor (the bane of every business once a year) and then moved to industry within the whisky business and then insurance before landing here at MaidSafe. I’m excited to be joining the team and sure I am going to have a busy year ahead of me.

Progress report

Section membership
This is what allows elders to keep control over the adults in their section. It was implemented when it was worked out that just using anti-entropy to handle new nodes joining could lead to ‘split’ brain among the elders under some circumstances.
Progress: Most of the code is written for nodes joining and we are working through the workflow when nodes leave. The ‘supermajority over supermajority’ voting rounds is also nearly there with the final key aggregation being worked on.
Todo: Coding the nodes leaving scenario, testing, integration, selecting elders and adults to be relocated on a split.

Data handling
Getting section membership sorted is necessary for proper data handling, redistributing chunks when an adult goes offline, and gracefully handling splits.
Progress: Registers have been moved to adults. Much of the work going on here now is bug squashing, trying to find out why chunks go missing, etc. Improved query handling at elders.
Todo: Caching on elders

Messages and memory spikes
Aside from inter-elder communications, most messaging is handled by anti-entropy (we reply with missing information so a node can update and retry its message if needs be, eliminating the need to cache messages), and qp2p. We have made great progress in eliminating excess messages here, but some bugs remain, particularly when a new node is contacting elders and asking to join. The latest community testnet threw up a few examples.
Progress: section membership messaging is done bar the testing.
Todo: optimising AE, squashing bugs that are creating memory spikes in testing.

Liveness testing
Regularly testing that nodes are online and taking action if they are not.
Progress: bug testing and implementation.
Todo: Not a lot, it’s pretty much done.

Name Resolution System and domain names
NRS maps are multimaps (registers under the hood) that contain the mapping of all the subnames to Urls for a specific topname. Each topname has its own multimap.
Progress: We are currently testing and improving the API of NRS as well as fixing issues along the way.
Todo: We’re deciding what to store as entries in the NRS map. The current (key - value) mapping is “subname” → Url but for a better UX it might be helpful to store the topname along with the subname as key in the map, like: “subname.topname” → Url.
The reason for that is that if someone fetched an NRS map’s xorurl (the xorurl for the register where the mapping is), that person would have no information on the “topname” of the map they’re querying, just the mapping of subnames.

For example on Google, they would get
“maps” → Url1
“docs” → Url2

So we’re discussing changing it to
“google.maps” → Url1
“google.docs” → Url2

DBCs
The basic design for DBCs is finished, including Ring CTs, along with a mock spentbook that validates transactions, one-time keys, mint nodes trusting spentbook pubkeys, and shared validation logic between mint node and DBC.
Progress: most of the design work is done, tests are passing
Todo: Consider implications, possible attack vectors; integrate DBCs into the network

Payment to farmers
Adults and elders will be paid for their services by the network. There are some designs on the table, but they await the implementation of DBCs.
Progress: Basic design ideas
Todo: implementation and testing when other elements allow

Users paying for storage
Similarly, we have some designs in place but await DBCs. Current thinking is for clients to use DBCs to pay for storage upfront, with the elders quoting a price to sign the names of those chunks, making them valid network data. Mutable data will need to be treated differently.
Progress: design ideas
Todo: implementation, testing, UX testing.

Front end / UX
We have revealed some wireframes and designs for authentication / UX over the last few months, but obviously, those are the icing on the cake, and the cake’s not baked yet.
Progress: basic look and feel, user authentication, lost credentials strategies, and other designs are done
Todo: implementation when other elements ready, user testing

Governance

Heather is looking at governance from three angles: Project governance, MaidSafe’s relationship with the project and governance of the token.
Progress: So far she has found no red flags - which is obviously a positive - but there are a few gaps and vagaries that will need addressing.
Todo: Create a quarterly plan and work out what to tackle in which order.

Documentation
The CLI User Guide has been updated and NRS docs are now undergoing similar treatment.
Progress: New CLI docs are here
Todo: NRS docs, DBCs, etc


Useful Links

Feel free to reply below with links to translations of this dev update and moderators will add them here:

:russia: Russian ; :germany: German ; :spain: Spanish ; :france: French; :bulgaria: Bulgarian

As an open source project, we’re always looking for feedback, comments and community contributions - so don’t be shy, join in and let’s create the Safe Network together!

69 Likes

first yey!
now to read

18 Likes

Second happy days :slight_smile: now to read!

16 Likes

Nice update! Thank you.

16 Likes

Thanks all. Moving forward nicely.

13 Likes

Great update team. I like the detailed progress report layout and to-do’s. Gives a nice overview of where things are at and what’s to come. Welcome Andrew!

17 Likes

Thanks for the update and welcome to Andrew. Looks like Heather hit the ground running, I hope Andrew can too and then we can get best value out of @dirvine and @JimCollinson

Looking forward to testing some of this new code as soon as that is practicable. THanks as always to the entire team for their hard work and dedication.

14 Likes

Very helpful update thanks for gathering all that info and setting it out succinctly.

Welcome Andrew and good luck, it’s great to see the team growing again and Heather already contributing.

It feels like the year is already off to a good start, and great that we’ve got regular testnets to play with - from all sides! Thank you @Josh and @folaht, plus those making scripts and offering help, coming from the community giving more people confidence to jump in and try things out. It’s all inspiring and helped me find time to get vdash working again, to help anyone see what’s going on when running a node.

Here we go :ant:

21 Likes

Nice update for the non tech savvy like myself. Good luck Andrew and good to see you back to what you guys do best @dirvine and @JimCollinson

14 Likes

Not sure if this is new format for every new weekly update, but I like it. Thank you and keep going what you do.

17 Likes

Thanks so much to the entire Maidsafe team for all of your hard work! :racehorse:

15 Likes

Love the ‘progress’ and ‘todo’ elements. Really gives a clear picture. Great work team!

17 Likes

Nice update! I also like the format, clear and understandable for someone like me ;).

Not posting very often but thanks to the entire team!

14 Likes

I like this update format. We should do one of these every month!

11 Likes

I prefer them weekly TBH

7 Likes

I haven’t been following updates in code for a long time and I was surprised to see that file container version isn’t a number anymore but a kind of hash instead. What’s the reason for this?

Note that safe cat commands in this document still output numbered versions, which is not consistent with the new version scheme.

5 Likes

Apologies for the inconsitencies in the documentation. A full update is a work in progress.

Possibly someone like @bochaco would be best placed to explain the new versioning system. I’ll include an explanation in the documentation at some point.

14 Likes

Thx @danda & @maidsafe Team. You are working so hard!

11 Likes

As versions (tips of tree) can basically fork then versions make less sense. If we go from
1-2
1-2a
1-2b
etc. then in a concurrent system, it won’t work as 2b does not necessarily know 2a happened or is happening. So using hashes takes away the requirement to synchronise version numbers and allows us to have concurrent updates.

15 Likes