Are Erasure Codes (Storj) better than Replication for the SAFE network?

Gamification on SAFE is much better than SIA. The following things exert strong pressure not to exit:

  • to enter the network you are waiting for n-amount of time
  • to get money - you’re waiting to become an adult
  • if you are of the first participants on the network you are more likely to get parts of valuable popular things (all movies and music created to date will be uploaded in the first few months without a shadow of doubt)

So I’m sure there are many people like me who have very well considered their strategy to get the most out of the newly created SafeCoin…

Edit: I had a few SIA nodes and a Obelisk ASIC SC1 miner and I can guarantee you that SIA is a sad story compared to SAFE…

4 Likes

Child nodes are also farming

It is infant nodes that do not farm.

4 Likes

I agree with what you are saying around the technical incentives to keep a node up from farmer perspective, they are nice and all and help somewhat. I want to drop that for a minute and consider what would happen on the network when say in a given week 30-40% of the nodes churned(where one node was considered good but perma left, and a different person decided to join the network which essentially maintains total network capacity never changed). I would like the network to be somewhat resilient enough to actually be smart and say I have detected ____ nodes that have this bit of data, this is ___ less that I had before and under the ___ threshold safe considers good. Obviously this level of reconciliation can’t happen globally all the time or the network otherwise it would be too bogged down with trying to maintain data replication at all times. Or maybe by design the network can’t really figure out how many nodes share a piece of data, in which case just hoping that just the incentives in place to attempt to keep farmers using the network will be sufficient is a false premise I think. I bet in the wild that would come back to bite you during a crypto winter or if people are not really making enough “income” from being a farmer. IMO there has to be a way that even if we went to 50% of the farmers safe network saw, the network checked on replication and still tries to move files around accordingly so the farmers that remain can take the extra burden from nodes that left. Finding a balance around how often that has to occur I think is the key to a quality decentralized storage network(one aspect of how safe can be utilized).

3 Likes

This concept helps, but my understanding is that sections are fluid in nature too where nodes may enter and leave a section as well. Will nodes that enter an existing section have data pushed on them that is shared among other section nodes if those older nodes leave the section or cut off then? And when a node moves from one section to another does the section they just joined push the data shared amongst the section onto the new node too or will the node that left the prior section drop its old data(and how does this work with safecoin because I thought puts are directly what cause storage growth, what accounts for re-replicating existing data since thats outside the scope of a direct put action and more of a network reconcile feature)? As long as maidsafe feels really good about existing design lets see how it plays out :slight_smile: . I could sit and ask questions about this network all day long probably heh because every high level feature has so many edge cases and underlying requirements and things that must be satisfied to be stable and well engineered.

2 Likes

As indicated by @Antifragile, the invariant that is maintained in the network is that each data chunk is stored in 8 nodes. It is easier to reason by referring to this invariant.

These nodes are the 8 nearest nodes to the address of the chunk and they necessarily belong to the same section (this a property of XOR distant). When a node leaves a section, it is not expected to manage its old chunks anymore. Copies of chunks it used to manage is added to remaining nodes in the section.

This is the reason why when a section falls below 8 nodes it is merged with its neighbor: keep a minimum of 8 nodes in a section that so that it can maintain this invariant locally in the section.

When a node joins a section, it receives some chunks, but not all the chunks managed by the section, only those that are near the new node (when the node is among the 8 nearest nodes of the chunk).

5 Likes

These comments in the source code clarify the intended behaviour for chunk redundancy (which is the responsibility of the Data Manager Persona of the vault).

Worth reading the whole lot but here’s some snippets

[Data Managers are] responsible for chunks whose names are close to it in the network address space.

[Data Managers that have recently joined or relocated] will also try to retrieve any chunks for which it is responsible from any close peer which is not currently busy

Chunk replication to this Vault continues repeatedly until it holds all chunks for which it is responsible.

Note these comments were last updated 3 years ago (July 2016) so still reference the old Structured Data which is now replaced by Mutable Data (and possibly in the future replaced again by Appendable Data), but the mechanisms described by the comments seem to still be accurate.

I think ‘the incentive structure’ is perhaps a more accurate / stronger phrase than ‘gamification’. And the incentives are the key for any permissionless network to succeed so you’ve definitely picked the right point to focus on :slight_smile:

My understanding of the age categories were infant, adult, elder (outlined in the first post of the dev topic on Data Chains - Deeper Dive) - no child category.

To add some links for those that want to dive deeper, in the code this is defined as min_section_size. See the declaration in routing and the usage in vaults for determining which vaults are part of the close group.

7 Likes

They have been mentioned, maybe incorrectly by David, but David saw the child node as a very important part

2 Likes

My bad I have done that in the past child == infant to me really. Age takes over so we will have younger Adults as younger elders but only age determines that.

4 Likes

I don’t know how SIA works or who it’s audience is, so I might be off-base here - perhaps SIA has the same target audiences/users and it’s still not enough… but I imagine that on Safe, the big uploaders will themselves be providing a lot of storage to the network - you wouldn’t go through the trouble as a business to provide a lot of content only to lose it down the track - you’d work to insure that there will be enough storage.

I wonder what kind of tools will be available to get metrics on the network’s various capacities once it’s going. Anyone have any ideas on that? Being such a private anonymous system, I think some metrics might be difficult to ascertain.

2 Likes

Let’s hope we can find a copy of this somewhere! That’s not how the Safe Network does it. The network always knows which chunk is stored by whom, so when one too many of those vaults goes offline, it just creates a new copy.

5 Likes