Last week, we released Test 15. Our initial plan was that if Test 15 proved to be stable, it would become Alpha 2. Test 15 was stable for the first few days and we have been very pleased with the results. However, after a few days Test 15 experienced an issue related to client account spam. It appears that someone ran a script that created lots of new accounts and stored lots of data, to the point where most vaults are running at 50% of their capacity, and it no longer became possible for the network to store any new data (hence the
NetworkFull error). Currently, vaults will only go up to 50% of their capacity before rejecting new client data. The remaining is left over to accommodate for churned data being relocated to existing vaults. As things turned out, this situation has been quite useful for us in understanding how the network reacts when full, as this has not been something we have publicly tested before.
As most of you know, it’s quite easy to create new client accounts using SAFE Launcher: you just need to provide an “account secret” and an “account password”. And since test safecoin is not implemented yet, vaults are coded to give 500 PUTs to each new client account (a PUT is a request to store a single chunk on the network). This makes it easy for community members to participate in test networks. On the other hand, it also makes it easy for an attacker to spam the network. We would prefer to wait for test safecoin to be implemented, but at least this spam attack showed us that the
NetworkFull error works as expected
As @fraser said in this post, the main takeaway is that we need to accommodate for such scripts being run in the upcoming testnets. Earlier this week, we discussed various potential options that could help solve this issue. We want to implement a temporary solution that’s not too complicated and that we can easily remove later on when we start implementing test safecoin (see this dev update for more info on our short-term roadmap).
We are currently implementing a solution called invitation-based accounts. Here’s how it will work:
Open SAFE Launcher, go to “Create Account”.
Before asking for an “account secret” and an “account password”, Launcher will ask for an “invitation”. Here’s an example of how an invitation might look like:
You can either enter an invitation token (if you have one) or click on the “Claim an invitation” button.
After clicking on the claim button, you will see an OAuth dialog with a few options (“Sign in with Google”, “Sign in with GitHub”, “Sign in with Twitter”…etc…). The options haven’t been fully decided yet and might change over time.
After choosing one of the options, you’ll need to log in and grant the required permissions (the OAuth dialog will only ask for basic info). There will be a server (controlled by MaidSafe) with a database that keeps track of which OAuth users have received an invitation. The server will only give you an invitation token the first time you log in using a specific option (e.g. your Google account). If you try again with the same account, it won’t work.
If the “Claim invitation” process was successful, Launcher will automatically fill the invitation token you obtained from the server into the invitation box. You can then continue the account creation process as normal.
As a fallback option, we will also distribute invitation tokens to forum mods. Users who prefer to not sign in using OAuth could go to the SAFE Network Forum and ask mods for an invitation token and then copy-paste it into the invitation box. This could also be useful for people (e.g. app developers) who want to have multiple client accounts. They could simply contact the mods and ask for a couple of invitations.
The invitation tokens will be generated by MaidSafe (in batches). Each invitation will be represented by a structured data with type tag 8. The name of the structured data will be the hash of the invitation token.
The way it will work is that when you try to create a client account, the MaidManagers will expect an invitation token along with the normal information. They will hash the invitation token and try to update a structured data from that location in the network to version
1. If the update operation fails, it means that the invitation has already been claimed. The vault config will be accepting a new entry for storing a public signing key. MaidManagers will have a rule to only allow PUT requests for structured data of type tag 8 if it’s signed by the public key contained in their vault config.
Please keep in mind that this solution is intentionally small so that we can get back to the normal day-to-day work It’s going to take us a few days to finish implementing and testing this. Once it’s ready, we will release Test 16 (which should become Alpha 2 should testing go as anticipated).
SAFE Authenticator & API
With continued testing, we identified a couple more bugs with the Authenticator. We are holding back on a simple tutorial that shows how to build the SAFE Browser and the SAFE Hosting Manager example app, and also a simple tutorial that shows how to use the new DOM APIs until the root issue has been solved in safe_client_libs and then tested again from the front-end side thoroughly. The issue is with the MutableData delete functionality. Once you delete an entry from a MutableData, fetching the same MutableData again was throwing an error. Similarly, after you authorise an app and then revoke it, if the same app tries to authorise again, an error was returned. This is being addressed by @nbaksalyar.
We have created an initial wireframe for the developer website. This is only a first cut wireframe which is yet to be discussed within the team, and after couple of iterations, we will be happy to share the plans with community.
@krishna_kumar has been working on a POC for a Java API using the jnr-ffi library. An Android developer is joining our team next week. He will be able to make use of this POC to come up with a full blown safe_app_java library for building desktop applications using Java.
SAFE Client Libs & Crust
Carl has progressed with his uTP implementation and now has a partially working solution here. It does not yet have a full-fledged congestion control (it’s one of the things he will be working on next). However, there is an example which shows how to use it in simple cases and the API looks pretty similar to TCP which is what we would have expected. It also allows multiplexing of uTP streams over the same local socket, which might come in handy in future.
Many crates have been updated to use Serde serialisation instead of rustc-serialize (which is flagged to be deprecated since procedural macros are now stabilised). Crust benefits from this because in the past we needed our own custom SocketAddress and IpAddress wrappers since the standard ones were not serialisable. With serde, this is no longer the case and thus crust code will see some good clean-up in getting rid of manually coded wrappers (leading to better and less confusing code).
safe_core now supports the new invitation-based scheme in combination with the vaults. It also supports the creation of a specialised seeded client that the vaults will recognise as special. It is for populating the invitations on the network so that future clients can be enforced to be creatable only if invited. A binary in the examples folder is almost complete which will allow various invitation-related tasks. Once the chosen seed is given, it will output a public key which will be made known to the vaults via the vault config. Then account creation with the same seed can happen, which the vaults will allow without checking for an invitation. This seeded client will also be allowed unrestricted PUTs to allow it generating invitation tokens as required.
Routing & Vault
We’re working on implementing an invitation-based account creation in safe_vault, as mentioned in the introduction.
In routing, we resolved a few issues with section merges and tunnels, and at the same time started investigating the introduction of section version numbers, which would be a first step towards data chains.
From Test 15, we’ve also identified a bug thanks to @rupert which seems to occur at times when a tunnel connection is promoted to a direction connection causing a state inconsistency between the peer manager and the routing table. We’ve got a patch in place for this and are working to confirm it via tests before bringing it into the master branch.