MaidSafe Dev Update :safe: 29 July 2015

As Ben mentioned in his last week’s update, one of this sprint’s biggest objectives was to implement Unified Structured Data into routing. One of the reasons we refer to it as ‘biggest’ is because it changes how upper layers interact with routing and so it also changes routing’s API. This means that we were not able to think in terms of individual modules anymore, but had to put our heads together with guys from Vault and Client to make sure we’re on the same page.

But there have been other big changes, less visible to the outside world though still very significant, from both a security and code reduction perspective. Previously, our messages looked something like this (header, (body, signature)). Now they now look more like this ((header, body), signature). Blink and you may miss this change, essentially the header is now part of what is signed.

Although it may not seem so, this change is a huge one. Previously, we knew we could not trust the header so our algorithms had to account for this fact. These changes caused us a little code depth which we knew we’d need to deal with sooner or later. On the other hand, last weeks refactor changed this and we can now happily report that all messages are now fully cryptographically signed.

Another big objective of this sprint was to plug Sentinel into Routing. As a reminder, here is a short description of what Sentinel is all about: In a decentralised network such as SAFE we cannot trust individual nodes, because anyone who knows how to use our protocol can become one. Whom we can trust though, is a set of nodes which are picked by the network if they’re all telling us the same thing. We call such set of nodes a ‘close group’. Now, imagine that such a close group sends us a message. What we see internally is a set of messages each from an individual node. It is then the purpose of Sentinel to ‘group’ such messages and validate that they really came from nodes selected by the network, then present the result to the upper layers.

So back to the objective, the plugging itself is not done yet, but a big chunk of refactoring was done for it to be possible. Just to sketch the idea, imagine again that a group entity sends us a messages, that is, we receive a set of messages, we had to identify which pieces of each such message are not changing and only make those part of the group message (we came to call these pieces ‘message invariants’). With this done, we will now be able to quickly move forward and resolve two other issues as well, namely code reduction and switching to a channel based interface as opposed to dynamic dispatch interface [1].

As far as the sprint and other libraries go, since week two, we’ve managed to stay ahead of our progress estimates. CRUST library concluded its work for this sprint and we are working to resolve a final issue for the OSX platform. DNS crate has also concluded its work yesterday and we have verified everything works as expected against the mock routing layer, so a very positive sprint all round. Testing against the live routing nodes should commence in the next few days.

And to be complete, here is also the update from Justine.

[1] Dynamic dispatch interface has the disadvantage that it is single threaded, blocking routing code until the dispatched code finishes. With the channel interface, we just insert a message into the channel and continue right away thus allowing for better parallelisation, enabling multiple actions to happen in tandem. (This is a nice introduction to this approach).

40 Likes

The second I type this:

You decide to post the dev update, good timing!

2 Likes

Nice, glad to hear everything is going as planned. I appreciate the easy technical explanations in this piece, well done.

4 Likes

What exact are these headers? Is it like the Chunk is the body and the header is added by the close group for routing?

1 Like

:), sorry for the delay

@polpolrene, the structure of the header has changed as well, one could say that now the header consists of these fields:

1.) destination address
2.) source address
3.) original message before forwarding (used for signature checking when response is sent back)
4.) message id (to tell messages that came from the same source apart)
5.) source authority

The source and destination addresses are tricky, not only do they contain the address of a node in the XOR space, they also contain an information where the message should be “relayed” if the real destination is a node outside of the XOR space.

8 Likes

Okay couple questions. Do we have installers for Linux and Windows yet? (And Mac too, I’ve got friends with Mac lol) And where do I get said installers if we do in fact have them? Also what’s this learning Rust outreach program I’m reading about on the update here? Also at what point will we have installers for Android (and IOS)?

1 Like

Nice to see everything coming along smoothly :thumbsup:

Would it be possible to have a bit more context/information about this field in the header.

What nodes are outside XOR space?

Thank you for the dev update too

1 Like

for instance a message heading for neo@gmail.com?

[quote=“reivanen, post:10, topic:4580, full:true”]
for instance a message heading for neo@gmail.com
[/quote]So is the original message "neo@gmail.com" ?

Not much of an original message???

It says “original message” not “original message heading”

Installers are being worked on at the moment (sorry I know you have been told this before) and we are looking to deliver these at the end of the sprint. This is getting a lot of focus internally as this will make running the network much more inclusive which is what we are all about after all.

I think you are taking this from the action items against @ioptio in the Google doc? These are actually 2 separate issues. Paige is teaching herself Rust and is also working on some developer outreach items.

I’m not sure on this point yet. Current priority is completing the sprint and getting desktop installers released. Mobile is going to be really important for us so we will get to this, I’m just not sure when yet.

4 Likes

Okay so in a few weeks at the end of the sprint we’ll have installers. Awesome. Thanks. I’m just kind of excited and want my maidsafe. :smile:

1 Like

Yeah no probs, me too. It should be less than that, the sprint is due to finish Fri 7th of August

4 Likes

Okay this might sound like a stupid question but once we HAVE our installers what exactly will we be able to install with them? I mean will we have a client and some kind of interface? Do they come with Lifestuff? Do we just have raw maidsafe and have to use a commandline to farm safecoin? What are we working with here?

Currently the plan is to have installers for the vault binary itself and also a separate installer for the examples(this would have examples from the various rust modules). @Fraser is setting up the JIRA tasks for the same. Do note work is currently ongoing for these ones and you should be able to track these via JIRA too. This filter should show you the tasks as they’re getting added.

Also worth noting that this is not “all the installers” we’re going to be releasing. These will evolve as more apps/features become relevant as part of the sprint work.

5 Likes

I’m on Fedora / Win 7, I despise Windows 8 and after reading about Win 10 there’s no chance in hell I’d use it. Is there a Win 7 version of SAFE?

Definitely :smile: . Well considering all the binaries we’ve got right now are console based, there really shouldn’t be much effort in supporting everything above Win-7.

2 Likes

No problem, so the messages can be - in a sense - divided in two categories: requests and responses. When e.g. you send a request to the network, it travels through different nodes until it finally reaches it’s destination, let’s call it node B. If everything is OK, the node B sends you back a response which again travels through different nodes till it reaches you. Now we have two problems:

a) what if someone other than the node B tries to send you a response? This one is solved with the sentinel.
b) how do you know that a response you receive is something that you requested? The current solution to this problem we have is that the response comes back together with the signed request, and since we were the ones who signed it, we can easily make sure we were the requesters.

For example clients. Clients are those who don’t store data nor control the network, that is, they just query data or request the network (nodes inside the XOR space) to store data in it.

Another example are Nodes that did not find their place in the network yet. That is, when a node starts, it needs to connect to other nodes that are as close to it as possible using the XOR distance (only then it is considered to be “inside” the XOR space), but before it does so, it needs to find its peers. This searching for peers is done with help from the nodes in the XOR space.

You’re welcome :), let me know if you have more questions or if you’d like me to further elaborate.

4 Likes

So the original message is the request. I gather then it is small. Is that correct?

I understand the request-response issues, having written some protocols myself in decades past (-x

Silly me, I should have realised it is the clients. Interesting about the nodes joining network.

Thanks again.