Explanation sourced direct from the code

I was frustrated by a lot of conflicting information from ‘plain language’ explanations of how the network functions.

So I started reading the code and creating a plain language explanation of my own with references back to the source code for each point.

This is the result to answer my first question about MaidSafe
This has been moved to a branch of the safe-network-explained github pages.
The page will be gradually developed, but since the safe network source code is moving quickly the page will require a lot of maintenance, so it’s a low priority right now.

My questions:

  • is my reading of the code for this topic correct? (no point creating incorrect documentation!)
  • is this a useful form of documentation, and who would the likely audience be?
  • what other features of the network could benefit from this style of documentation (I’m currently working on how vault names are allocated which is an essential part of the security of MaidSafe)
  • will it be too onerous to maintain? I know the network design and code undergoes a fair bit of change (which led to the problem of legacy information in the first place). I have a script that watches each line of source code referenced in the document, and any changes to the code can be updated without having to manually watch every document and line of code in person.
15 Likes

This the consensus group that validates the data (currently 8 but eventually 32). Not the data storage.

There is a RFC being coded that states that there will be 6 copies in 3 address groupings (“permanent”, “backup”, “sacrificial”)

More information here
https://github.com/maidsafe/rfcs/blob/master/active/0023-immutable-data-type-naming/0023-immutable-data-type-naming.md

2 Likes

I think it is a VERY useful form of documentation, I would be part of the audience. I am trying to develop safe applications geared towards secure communication and publishing of sensible content, and having a clear understanding of what happens under the hood is a crucial need. I suppose anyone willing to develop work that touches closely to the safe core, like for instance a command line launcher ( who wants to make this ? :slight_smile: ) would greatly benefit of the documentation work that you propose.

I would love to have such documentation on the client / launcher API - I suppose the MAID guys are going to write it once things stablilize, though.

this leads me to :

I would be cautious here, not to involve too much work in very early stages. I see the code moving really fast these last weeks. I suppose things will settle down in the coming weeks, though, as we approach alpha state.
Great idea to have a script watching for code changes !

3 Likes

Hello @mav, thank you for your research! I’m glad to see others get interested in this technology.

There are just so many moving parts at this point that it’s hard to get a handle on everything about the network - chunk redundancy included. As @neo mentioned, there’s an RFC that may clarify several points that you brought up, but that’s not really documentation, now is it.

In all four sections of the wiki that I have outlined (App Builder, Core Developer, End User, Farmer) a fundamental understanding of the Network’s architecture is key. If I could put everything that I have floating around in my brain about the Network down into the wiki, not only would it be completely disorganized, but also incomplete.

This is a symptom of the Network not being released at the moment, however I believe that we have a better option than sitting around and doing nothing. Members such as yourself are hungry for knowledge, and eager to have a complete synopsis of the Network to share, to study, to curate. This is exactly what I’m doing by creating a newer version of the wiki (at safenetwork.branchable.com).

The most complex part of this is keeping links up to date and I see that you’ve done plenty of research. If you would like to keep current on the development of the information on the wiki, I would advise you to join @wiki (our forum PM group messaging tag) and chime in if we forget or overlook an aspect that you have educated yourself in. We are just getting started - focusing on formatting and templates and categorization, but I’m very excited at where we’re headed.

2 Likes

@neo, thanks for the clarification; I had a feeling something wasn’t quite right with my understanding and you pinpointed it exactly.

@nice, I agree the code is moving really fast and until it settles down the work maintaining this would be more than the benefit it produces. Perhaps in the future it will be a good project, but for now it’s too unstable.

@smacz, as I progress on this project I’m finding the same thing as you - I find I’m writing disorganized and incomplete content. I think that’s a good reason to keep the work private for now, rather than publicize it and add to the confusion and noise that legacy documentation already creates. Reminds me of the xkcd about standards.

Having spent more time reading the wiki, rfcs, and readmes I think I’ll put this idea on the backburner for now. The concept is a good one, but the value:maintenance ratio is a bit low for me to pursue it.

I’ll definitely look into contributing to the wiki, but in the meantime will keep growing my knowledge and trying to create useful explanations from what I learn.

Here’s another explanation I worked on, but will be the last one I post until I pick this up later (possibly much later).

Attacking the network via the vault naming mechanism

An attacker that wants to modify or control data will try to minimize the distance between their vaults to gain control of consensus, so it’s important to understand how they can (or can’t) do that.

The vault name is the only variable in calculating distance to other vaults to form a close group, so this is where the attacker will focus their efforts. The name for a vault is generated by the following steps (outlined in fairly plain language in this long comment):

  • The vault chooses a name for itself. See L124
  • The network sends this new name back to the vault. Source code
  • The vault must use this new name. Source code. Note the new node will not be added to the network unless it uses the new name. Source code

The third point is what makes it so difficult for an attacker to gain a foothold. It’s extremely difficult to predict the final name the network will issue, which ultimately means it’s extremely difficult to form a group of close nodes on the network.

The following topics are ones I planned to work on (I’m sure it’s not a complete list). The topics are ordered in roughly the lifecycle of the network

  • Network bootstrapping
  • Joining the network
    • Initial connection (vault and client)
    • Vault name allocation
    • Client registration
    • Client authentication
    • Obtaining personas from the network
  • Storing data
    • Client upload and self-encryption
    • Public vs Private data
    • Mutable vs Immutable data
    • Coordination between personas
    • Redundancy
    • Vault consensus
    • Churn
    • Handling client change-of-details
  • Retrieving data
  • Safecoin
    • Issued by authority
    • Proof of resource
    • New coin name / farm attempt
    • Client balance
    • Privacy and anonymity
    • Transfer
    • Recycling
    • Monetary policy
    • Converting from presale to safecoin
    • Race conditions and propogation conflicts
  • Attacks
    • Brute force logins
    • Manipulating vault naming to override consensus
    • Ranking algorithm
    • Bribing or hacking nodes to control close groups
    • OS environment interacting with safe code
    • Modifying the supplier of the software
      • downloaded files
      • verification signatures
2 Likes

If you don’t mind, I’m stealing those bulletpoints for my own project list for the wiki. Very well done.

Yes I agree that it should be low-key, even through the beta release as up till then the architecture of the Network will doubtless be constantly changing. However, you do realize that you have a uniquely comprehensive grasp on the issues raised by the Network, right? I’m very imporessed. In the future, I would be honored to have your input on future initiatives if you are ever so inclined.

Go for it.

Yeah I am aware of how little most people comprehend about the network. I want to change that :smile:

4 Likes

Such a resource greatly reduces duplication of effort, because any new developer on SAFE is going to have to build his own mental model of it, abstracted as little as he can stand, if none already exist.

2 Likes

This form of referencing the code, would be ideal for definitive answers to FAQs. Good idea!

This now has a branch on the safe-network-explained github pages.
The page will be gradually developed in the implementation-explained branch of the repository, but since the safe network source code is moving quickly the page will require a lot of maintenance, and is a low priority for the time being.

Glad to see interest in this sort of documentation. It’s really interesting to reach at an understanding of the project from the roots rather than the high-level.

4 Likes