Lets nail this down. We need a solid breakdown of how SAFE works. Freenet peeps want to know

If you are the closest node to a piece of data, you’re the one that’s storing one of the chunks. Every few seconds, other nodes will ask you to do a little hash puzzle where you need to prove that you have the chunk. They’ll just sent you a few bits that need to be added to the chunk and provide them the hash of this puzzle. You can’t just delete unpopular files in the hope to make more money. Because when you just delete a chunk, you would fail the puzzle. And get deranked. The network will constantly check if you indeed are storing the chunks that you are close to.

When you go offline, they’ll notice as well. and now there’s only 3 chunks left, so they pick a new Vault that’s the closest to the data and he will store the chunks. Vaults get rewarded in Safecoin when do actually get a request to deliver a Chunk and they do. So people pay one time to PUT data to the network, after that the data will move over time over an x-number of Vaults. No one needs to pay again for that, it gives all these Vaults a chance to Farm Safecoin when that Chunk is requested.

Nothing. But you can’t pick your own group. Lets say we have 100K users in the beginning, all in groups with 32 nodes. That’s like 3125 groups. And because those groups are full with already 32 nodes, not a chance to target them. So you need to wait for a group with a spot left, which will probably be a group that’s forming with new users at the same time you try to connect. So the chance of you and 30 other friends to take over a group is quite small. You’re all connecting to the network, but you need to wait for a group to “pick you up” and to provide you an address. You can’t pick your own. So you get the opportunity from a group to join, and it will only last for a short time. You friends will get the opportunity from another group.

Just a nitpick, but as I understand it if there are 100K users there are 100K groups. A group is defined from the perspective of the subject node. If your and my vault are “close” we are in each other’s manager group, but both our manager groups will have members that the other manager group hasn’t got.

3 Likes

Actually, it means you become part of the manager group of the data chunk. You don’t store it yourself, but you pick two (random) vaults from the network and tell them to store the chunk for you. Your vault maintains a keep-alive connection with those two vaults and will forward any GETs for that chunk to those two vaults. If one goes off-line you’ll pick another random vault to store the chunk.

There are actually three manager groups for every data chunk like this, one defined by the hash(chunk), anotherr by the hash(hash(chunk)) and then another by the hash(hash(hash(chunk))). When a client sends a GET to the network, that GET is send to all three manager groups, and thus to 3*2=6 vaults (or at least to 4, I’m ignoring the possibility of sacrificial chunks not being stored here).

3 Likes

Sorry to jump in here , but another difference to mention would be data access. The more data is requested on the SAFE network, the quicker that data is available. No more DDOS attacks!

2 Likes

I agree, but that’s the same for Freenet. The more a website is requested, the faster it will become. No DDOS possible on that network as well.

2 Likes

Does the manager send this challenge to all storage nodes at the same time? Does the manager node also have the data to know what the none+data hash is, or does it rely on all the storage nodes returning the same hash and punishing those with a different hash?

From what I’m seeing here and other threads, the network seems extremely chatty with all these messages being sent back and forth just to keep track of everything. From this other thread https://forum.autonomi.community/t/vault-routing-burden/5663 it looks like the network will require significant bandwidth, much more so than an average home user can expect their internet connection to handle. What is the ratio of management type communications between nodes vs a user just normally requesting a piece of data?

2 Likes

That is a good point with the bandwidth that Im very interested in. Could I for example stream a video in HD format on a 100mbps line? If it cant handle a large throughput it may never be able to be that “Internet 2.0” for the average person.

1 Like

I highly recommend that you read the link in the post above, as it addresses that exact issue. In short, it is quite feasible and there are several propositions on how that is to be done.

That particular thread goes indepth on some other issues pertaining to streaming that you may not be aware of and how those can be solved as well.

However, the concern is mainly with the vaults’ ability to handle the network traffic, the client should only be required to handle the inbound stream of that particular media, much like the server-client architecture today.

1 Like

XOR space addressing is haaaard. Have you any recommendations for reading material on this subject (besides those published by MaidSafe)?

1 Like

I’ve used Freenet a couple of times and I couldn’t handle how slow it was - that’s my personal problem and I’m sure it’s not how everyone feels.

When I first heard about MAIDSAFE, I focused in on the MAID part - it’s like a RAID disk, or more accurately like bit torrent - pulling pieces of your file from multiple locations in parallel and not in serial fashion from one source.

Will it be able to stream HD video? Who knows … we’ll find out. personally I’m more worried about latency than streaming - i.e. I’m more concerned about the speed of hosting websites on SAFE. But in any case, I think it will be much faster than Freenet and I2P and even Tor.

IMO, speed is king for a lot of internet users and the speed and security of SAFE are it’s greatest attraction for me.

3 Likes

You can find more information about the SAFE dataflow in the vault docs:

https://github.com/maidsafe/safe_vault/tree/master/docs

especially 1, 2 and 3.

Of course the network, as global, must take care of everything, but this “global” is the sum of small parts. And this small parts, in a completely distributed environment subject to all kinds of behaviours, must follow the laws of the network. Laws that are simple, clear and enforceable. Mathematical laws that ensures the survival of data while being blind to its content and ownership.

4 Likes

Yes! You answered your own question here. Regarding the chatty overhead it will be interesting to see and we’re right on the point of being able to see for real, because at the end of this sprint everyone will be able to join one network and store/access data on it. That’s end of next week, or maybe two… or my guess three, as they are integrating a lot of bits all at once in this sprint.

4 Likes

This one might help you:

1 Like

XOR space addressing

Maidsafe took kademlia concept and improvised it. CJDNS did the same thing as well. First read the kademlia seneca provided for you. Then expand your knowledge on cjdns as well, so that you will see a lot of similarities between of two, and have a great understanding how xor space addressing works.

Cjdns implements an encrypted IPv6 network using public key cryptography for network address allocation and a distributed hash table for routing.

This means that every IP address comes with public key, and private key.

  • maidsaid xor address is the same as public key address. Except the fact that IP are scrubbed after first hop.
  • Maidsafe routing is the same as cjdns with slight modification

here’s the cjdns whitepaper.

The “address space distance” between any two given addresses is defined as the
of the result of the two addresses XOR’d against one another, rotated 64 bits,
then interpreted as a big endian integer. The so called “XOR metric” was
pioneered in the work on Kademlia DHT system and is used to forward a packet
to someone who probably knows the whole route to the destination. The 64 bit
rotation of the result is used to improve performance where the first bits of
the address is fixed to avoid collisions in the IPv6 space.

The Splice operation takes a Route Label from pointA to pointB and concatenates
it with a Label from pointB to pointC yielding a Route Label for a route from
pointA to pointC.

2 Likes

This my second draft of the SAFEnet overview. Please check it out and let me know what you think. The graphic will follow this. Anyone with PowerPoint is welcomed to help. Thank you for your time! :smile:

@nicklambert, @19eddyjohn75, @Ross, @happybeing, @jreighley, @BenMS, @frabrunelle, @anon86652309, @janitor, @zankfrappa, @smacz, @4M8B, @atleticofa, @jm5, @riddim, @fergish, @betterthantrav

=================================

–THE NON-FREE DILEMMA–

While Freenet is both free to use and store data, SAFE’s relatively high transmissions speeds and superior default anonymity balances the two if not putting it in a favorable position.

Those who cannot pay for safecoin have the option of easily farming for them. It could be argued that in the time it takes for a Freenet user to properly integrate themselves into the Freenet network, at least 1 safecoin will be earned in SAFEnet. It could further be argued that in the time it takes to upload 1 to 3 gigabytes of data to Freenet (which can take days), another handful of safecoin could be earned.

Considering that 1 safecoin will afford the TRULY anonymous user at least 5 gigabytes (this is VERY conservative) of PERMANENT storage space on SAFE, It would be fair to say that there is little preventing 95 percent of people (with access to a computing device) from storing their valued data on SAFE. On top of that, upload speeds are only limited by a users connection speed. Oh, and did I mention that EVERYONE is perfectly anonymous by default. A bold claim backed by great math.

Read below to get an idea of how it all works. Welcome to SAFE…

=================================

–BROWSING THE NETWORK–

All public data on the SAFE network is free to browse and download. As content on the SAFE network grows in popularity, access speeds will adjust accordingly.

=================================

–PUBLIC vs. PRIVATE DATA–

There are two forms user data on the SAFE network. Private data is cryptographically obscured and sealed data that is only known and accessible to those who have the keys to access it. Public data on the other hand is data stored on the network that is visible and accessible to anyone who browses the SAFE network, much in the same way the current internet exposes its data. It can be searched, downloaded, or viewed with little more than your desire to do so.

This does not mean that data vaults owners can distinguish the difference between the two forms of data on their machine. Data managers encapsulate all data with further encryption to prevent this.

==================================

–SHARING DATA–

Sharing data between users will be greatly improved over almost every existing file-sharing solution. With SAFE, a user in most cases requires no more than one or two clicks to share their data with others. By having the option of instantly sharing the index/map of your files both privately and publicly, much of the effort normally used to share files will be almost entirely eliminated.

=================================

–STREAMLINED APP DEVELOPMENT & SECURE HANDLING–

SAFE greatly eases the process of app development by having the launcher module handle routing, self encryption, and authentication. This reduces attack vectors/surface by keeping apps from ever knowing your login credentials. This also means app developers can concentrate on making their dream a reality without having to worry about the complications of encryption and routing in this cutting edge network. SAFE approaches barriers with great forethought :slight_smile:

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

=====================================================================================
-----------------------------------------SAFE NETWORK COMPONENTS------------------------------------------

–SAFECOIN–

Safecoin is a digital coin that is created and protected by the SAFE Network and stays on the SAFE Network. Safecoins will only come into existence once the network is launched. Wallets will actually ask the network what their balance is. The network maintains only current owner ID and previous owner ID. The value of safecoin is based on the amount of storage space available on the network. More space means less value/cost.

Distribution:

Safecoin is slowly fed into the network over time by the network itself. Once certain conditions are met (i.e node population increase, safecoin scarcity, etc), the network will slowly introduce new new safecoin until a total of roughly 4.3 billion are in circulation. It is estimated that it will take about 20 years for this to occur. This prevents wealthy entities from buying all safecoin at once in an attempt to disable new content storage.

More info here: https://safenetwork.wiki/en/FAQ#What_is_safecoin.3F

=================================

–LAUNCHER–

The launcher is a software module (installed on any compatible computing device) that sits between the client and the network. Its basic purpose is to package the information provided by the client into something the network can understand. It also ensures that any data provided to the network conforms to SAFE’s stringent protocol.

Authentication:

There are many uses for personal accounts on SAFEnet. Most notably, the ability to store private data on the network.

Upon creation, only your files are encrypted with your username and password. Your keys are never stored on the network or your machine. They don’t even get sent to the network. This avoids theft.

When you desire to access your files or personal apps, the network uses your username and password to find a map of your files. Once found, your client uses your key to decrypt the requested files the moment they arrive at your machine.

Network drive:

Users with accounts will be provided a personal network drive that will allow them to store as much data as they can afford via safecoin. This drive is similar in functionality to other cloud storage services. The key differences are:

1.) Your data is encrypted by a key only you have.

2.) Your data is spread throughout the world with no companies in control of it.

3.) You only pay for it once, and it will remain in the network for as long as you want.

================================

–SELF ENCRYPTION–

Self encryption is the process by which data is encrypted using its own hash and user credentials as entropy. Files that are split use the hash of its counter parts to further layer the encryption.

================================

–VAULTS–

Data vaults are devoted portions of a users hard drive that store chunks of encrypted data uploaded to the SAFE network. Vaults also allow for anyone with free hard drive space to earn safecoin by providing that unused space to the network.

Data redundancy:

Data is self encrypted before upload and split into 1MB chunks. Each chunk is again self encrypted and stored with a minimum of 4 redundant pieces spread throughout the network.

Farming:

Whenever a user accesses a chunk stored on your node, you are rewarded with safecoin.

Reputation system:

The reputation of your vault depends on the stability of your vault. Your reputation is reset the moment it goes offline and all of the data it stores becomes invalid unless a massive outage occurs. Fortunately, a decent reputation is quickly regained.

================================

–CLIENT–

The client is a software module (installed on any compatible computing device) that sits between the user, the vault, and the launcher. A hub if you will. It communicates with the network via the launcher, monitors and controls a users’ vaults, and receives input from the user to send/retrieve resources on the network.

================================

–CLOSE GROUP–

The close group quorum is a set of determanistically chosen nodes that must reach a 28/32 consensus before several things can be done on the network. These responsibilities include regulating safecoin transactions, data/message routing, and XOR ID allocation (SAFE’s anonymous IP equivalent).

================================

–DATA MANAGER–

Similar to close groups, data managers are determaniscally chosen vault groups quorums responsible for handling data requests. They ensure that data is routed properly throughout the network by verifying the flow of data from node to node by ensuring that malicious behavior is rejected.

================================

–TRANSACTION MANAGERS–

Again similar to both close groups and data managers, Transaction managers verify and assign ownership of the cash like currency called Safecoin. No block-chain is used. Instead these managers keep track of the previous and current owner of a safecoin. This speeds transactions and leaves almost no trace. Enabling a digital form of cash exchange.

================================

–XOR USE IN THE NETWORK–

XOR is used for two very important functions in SAFEnet.

The first is it’s traditional use of file splitting and rejoining. SAFE uses the XOR algorithm to break all files into 1MB chunks before storing them on the network. When a user requests data from the network, it is then rejoined with all of its other pieces. All files under 1MB are padded to reach the 1MB minimum file size.

The second use of XOR in SAFE is for node identification which strongly impacts routing. Each machine on the network is assigned an ID by their close group based on the XOR address space algorithm. After the first hop, your IP is no longer used for communications. Together with groups and managers, XOR allows for IP-less communication systems without the hassle of ID collisions by allowing for a near infinite address space.

================================

–DNS FREE ROUTING–

In SAFE, routing is handled by the data manger and close group quorum based modules that together with XOR ID’s allow for seamless communication between nodes on the network. The data managers are designed to (among other things) keep track of the location of data on the network. Close groups on the other hand are responsible for keeping track of XOR ID’s and their associated node (i.e other SAFE users). A beautifully secure and anonymous alternative to the flawed DNS system.

================================

–E-MAIL/MESSAGING–

There are several propositions being considered. The goal is to make it cheap to communicate with others without allowing for a bombardment of messages to be sent at once. Don’t worry though, a solid plan is nearly fully formed. Read on to get an idea.

Email:

Current design will require a user to buy an e-mail outbox (inboxes are free). Outboxes will be limited to being able to send a few thousand of e-mails to minimize spam/malicious use of network resources. A delivery confirmation system will force a user to wait until their messages have been received and opened before being able to send more e-mails.

Think of it as spending points. Every e-mail you send uses one point. When the recipient accepts your e-mail, you get the point back. Alternatively, if the recipient is taking too long to receive the e-mail, you can cancel the e-mail and get your point back. A minimum 24 hour wait period might be enforced to further slow spam bots.

Messaging:

I suspect that messaging will work much in the same way as e-mail. Messages that are explicitly responded to will yield the sender a returned point. Basically tit for tat. This will make trolling and spamming impractical.

================================

–SOLVED ATTACKS–

Below you will read of the attacks SAFE is design to be invulnerable to. Some of the most difficult network security problems have been solved. Of course, you be the judge.

Birthday paradox/Sybil Attacks:

The SAFE Network requires all requests be processed by at least two groups of Vaults. The Client passes a request to it’s DataManagers, who verify the request based on the Client’s signature. The request is then passed to a deterministically selected group of other Vaults which also verify the request based on its signature. By deterministically selecting the second group of DataManagers, this attack no longer holds true for the SAFE Network, since it is not possible for the attacker to gain control over a Vault by simply surrounding it.

To circumvent this, the attacker would require the ability to surround specific Vaults in the SAFE Network. This cannot be achieved, as it would require being able to effectively generate different values which, when hashed with SHA-512, result in close hashes around one particular point.

More here: https://safenetwork.wiki/en/FAQ#Attacks_on_the_SAFE_Network

ISP attack:

In this attack an ISP creates a simulated or fake network. In this case the Client or Vault will not be able to connect to the bootstrap Vaults listed in the cache file. This file contains the IP port and public key of bootstrap Vaults. Without the public key of any bootstrap Vaults, the connect request will fail. The SAFE Network Client then detects this is not a valid network.

Alternately,

An ISP’s machine pretends to be the user’s machine and asks for the bootstrap information on the users behalf. The ISP could request the information and pass it back to the user. That information is encrypted to the user, so an ISP cannot really access it (unless they can break RSA 4096). If the closest Vaults are fake they will not be able to process the connect request, which is encrypted to each end. So again this attack fails.

This attack is only applicable for SAFE Network Clients that have not yet logged in. On receipt of the login packet, the SAFE Client encrypts a close Vault request to the bootstrap Vault. Again this is returned encrypted, so the attack is circumvented.

Denial of service:

By eliminating DNS and replacing it with quorum based routing and data redundancy, denial of service attacks are no longer possible.

================================

–WHAT THE TRIPLE TRIANGLE LOGO REPRESENTS–

The logo consists of three interwoven triangles each representing a different core design goal.

These are their meanings:

1.True data security:

SAFE guarantees the security and privacy of your data on the the network. By now you can see all the many steps SAFE takes to ensure TRUE data security. It is a near perfect solution. There is currently nothing as intuitive or as safe as SAFE!

2. Autonomous network:

SAFE is self regulating and adaptive. Avoiding hacking attempts and malicious behavior by quorum based decision making. Creation of unbiased management groups using XOR address space algorithms and deterministic selection. Load balancing and message flow optimization by adaptive data caching/relocation. Network flood and spam protection using the integrated safecoin token system. Eliminating data snooping via layered encryption and defeating impersonation attacks by using asymmetric cryptographic key authentication. All of this to keep you safe without human intervention. Even going as far as to tame that gross monster called Sybil.

3. Self authentication:

Your access passwords/credentials are never sent into the network. Your data and account details are hashed then encrypted using your credentials. When you want your data or access to your account, the network just looks for data that matches your hash. Your data map (index of your files) is then given to you. That’s it! This logs you in without having to give the network your access codes. Simple and perfectly secure. Welcome to the SAFE network!

================================

–PATENTS–

Several defensive patents were received by CEO and co-founder David Irvine. These were subsequently transferred to a non profit humanitarian organization. This ensures that others cannot abuse the systems developed by Maidsafe in an attempt to hinder communicative freedom provided by the technologies employed in networks Like SAFE and any future forks.

================================

You can help by responding to each point separately so that I can easily organize this information. This not just about how Freenet compares to SAFE but the ease of technical orientation for newcomers. A succinct overview to wet the palate and encourage further exploration. Help will be appreciated. :smile:

5 Likes

–TRANSACTION MANAGERS–

is out of date. If you grab it from wiki, that’s old.

2 Likes

Which would be cheaper to pay someone else to read and tell me what it says in my language?

(more) On topic: I myself am more worried about ironing down how exactly the system will be updated than anything else honestly. For instance, let’s say it’s discovered that XOR address code (?) has been completely cracked and the system needs to be converted to XOR address 2. How would the network go about changing this?

One idea would be to setup a separate network with all the new fixes and optimizations. Then allow for a one time migration of our vault data. Crude but possible. Down time would be low.

What’s the current plan on how to roll out network updates?

1 Like

Absolutely amazing! @Tonda
…but I would try be a little bit more modest in the first paragraph … sounds a little bit arrogant and slightly like freenet-bashing to my ears (i don’t think it is intended that way … but freenet-people could be offended by this) … especially a little bit funny because freenet is working and maidsafe isn’t online yet and hasn’t proven to do what it promises :wink:

Here you could refer to the term “opportunistic caching” (at least i think it is the technical term of the technology behind it)

If i share a file with my mega account I make right-klick->share->via link->sent this link to a friend … so not that different I think …
… Again I’d be a little bit more modest here :wink:

actually I wouldn’t say the network does care at all about the “value” of the coin in our world … value is determined by the people who use the coin … the network simply decides “hey … there is sooo much storage space … probably I’m paying too much farming reward! let’s reduce it!” … or “wow there isn’t enough storage space - I’ll pay more farming reward from now on!”
[and you have 2x the word “new” in a row in the “Distribution”-paragraph - you probably want to delete one]

About the Launcher the official Version: (just wanted to mention it - I don’t want to imply your version is better or worse - just 2 different variants to say it I think ; )

  • What is the App Launcher?
    The App Launcher is an application that enables users to be create their SAFE account and login into other applications on the SAFE Network without passing your login credentials (pin, username and password) to anyone else. It can be thought of as the single point of contact with the network and keeps you in control of your data at all times.
    (I stole it from the FAQ-task at [MAID-1470] - JIRA https://maidsafe.atlassian.net/secure/attachment/12007/FAQ_content.txt )

If I remember right it is encrypted with password+pin or username+password+pin … I’m pretty sure a pin is involved too but I’m not 100% sure about the exact details …

  1. If you loose your login data all your data is lost forever … But I remember darkly about a proposal to recover data if a group of selected (by you) friends agree that you are you and only lost your login-data … though I’m not sure if anything like this is planned … sounds like a security issue to me …

how about including the link to this video https://www.youtube.com/watch?v=Jnvwv4z17b4

…soooooo i’m tired and have to go to bed - I’ll read the rest tomorrow :wink:

2 Likes