MaidSafe Dev Update - June 21, 2018

Here are some of the main things to highlight this week:

  • This week saw an interview with @dugcampbell published by CryptoIncome. There were also a couple of articles published by Crypto Core Media.
  • @hunterlester has been fully focused on enhancements for the Peruse browser, and we would like to encourage the community to try it and report any issues encountered since that will be very valuable and helpful to us. You can download Peruse v0.6.0 from here.
  • This week has seen the bulk of the Milestone 1 work for Parsec ticked off. The production code has been merged, some initial integration tests written and an example should be getting merged very soon. All being well, the kinks should be worked out over the next few days, and we can move on to Milestone 2.
  • We are happy to see community interest in netsim. We wrote some tutorials and examples to make it easier to get started.

Marketing

The SAFE Academy / Learn

Over the past few months, we’ve mentioned our work in building an Academy to provide new educational material that will help both newcomers and existing community members learn a little more about the Network. As you may recall, we intended to build this as a standalone website. However, once the decision was made to move the new website work in-house, our thinking has evolved.

Rather than maintaining a number of separate websites, with the risk of calls-to-action being confusing across each and creating additional challenges for tracking metrics and navigational complexity…etc…we’ve changed tack slightly. We’ll now roll this content directly into a new ‘learning’ section of the SAFE Network website that is currently approaching completion.

The net result is that this work has been added to the backlog for website tasks. After much debate, we’ve decided that there’s no benefit to be gained from delaying the website launch for this alone. Therefore, the ‘Learn’ section of the new website will be released at some point after the new website is live. Obviously, it’s disappointing to have to push this back a little, but when balanced with the feedback on the Getting Started video last week (specifically production value/audio etc), we felt that it’s best not to rush things out before they are properly integrated with the wider design and feel etc.

We have different skills across the team compared to the start of this project and it makes sense to use these moving forwards, even if we can’t access them immediately due to current workloads. Hopefully, this makes sense to all of you who want, as we do, to ensure that we’re putting out the most professional content possible. A further update will therefore follow on this work after the new website is live.

Press

This week saw an interview with @dugcampbell published by CryptoIncome. There were also a couple of articles published by Crypto Core Media following an interview and we understand that this will lead to a few follow-up articles also.

Community

Finally, a couple of interesting things from the community this week. First off, kudos to @Nerd1 who’s forked the SAFE-CMS project in order to breathe some new life into it. I’m sure he’d welcome everyone’s support in getting this project moving forwards once again. Second, @Sotros25 has now fixed on a date for the SAFE Network: Chicago meetup (Sat 21st July) - you can sign up here.

User Experience & Website Design

This week has seen the last of the major website layout challenges tackled, and rapid development of adaptive layouts. Props to @shankar for embracing the design exuberance, even though some jaw clenching was still audible via Google Hangouts. Comp images and patterns also became production ready assets: not only can @shona deadlift the sun, she also knows how to tame an SVG. Mighty teamwork!

So the project rolls on, and we’re within touching distance of the finish line. I know what you are thinking: Why finish it when you’re enjoying yourselves so much? Well, it’s a living breathing thing we’re making here; always ready to adapt, change, improve, grow. So it’ll never really end—such is life at the pixelface—but we’re almost ready to let it loose.

Maybe it’s best to think of us steadying ourselves in the blocks, fingers on the start line, just waiting for the crowd to hush down so we can hear the starting gun.

But as the code reviews come in thick and fast we’ve started to notice a strange perceptual phenomenon. Margins and padding become rhythm, breakpoints become key changes, classes an unfolding harmony…

At first it seemed an inconvenience, then a curse, but with each git merge the richer and more useful the melody became.

And what a gift for you all; as I’m delighted to be able to share a sneak-peak of the website design through some clever neurological reverse engineering.

All you’ll need is some noise-cancelling headphones and a jailbroken Oculus Rift. Just go to Settings > Advanced > CSSynesthesia, then play these tracks in order:

Here’s the homepage. (Oh the colour! That grid!)

And the ‘How It Works’ page (take your time, absorb):

And this should answer a few FAQs:

So what do you think? As always, we’d be delighted to get your feedback in the comments.

SAFE Browser, Authenticator & API

@hunterlester has been fully focused on enhancements for the Peruse browser, not only fixing bugs as they are found, but also adding new features like the ability to show a notification upon a network disconnection event, allowing the user to reconnect. We are also trying to display all the authorisation request information in the popup, similar to the way the existing SAFE Browser does. We intensified our internal testing activities for Peruse in the last few days, and we would like to encourage the community to try it and report any issues encountered since that will be very valuable and helpful to us.

As we mentioned in last week’s update, you can download Peruse v0.6.0 from here, where you can also see some known issues.

@oetyng reported an issue with the C# API bindings on the Dev Forum. It was helpful to identify the issue based on the information that was provided. @ravinderjangra and @AshwinKumar were able to reproduce the issue after overridden default configuration. The issue was fixed with help from @nbaksalyar. The pull request for fixing the issue is in review.

The old JNI issue with the NFS API is fixed and the tests for the APIs are passing locally. system_uri APIs are needed for registering and opening the URIs for IPC with the authenticator. @lionel.faber is now integrating the JNI layer for the system_uri APIs. He will be able to integrate the APIs with a desktop application once the system_uri APIs are ready.

@bochaco and @joshuef have been spending most of their time in scoping LDP/SOLID integration. The LDP integration will help us design the data as structured and scalable. It will also help in improving the collaboration between apps/users.

SAFE Client Libs

This week we’ve been honing and improving the essential SAFE Crypto implementation: we’ve made it more complete by implementing correct nonces and the mock version of the library for testing purposes (the mock version turns off the real crypto functions for much improved performance).

The SAFE Crypto library provides a basic abstract way for our other crates to use cryptographic functions: instead of relying directly on rust_sodium, they now can use similar functions from safe_crypto. The beauty of it is that in the future we can change the cryptography libraries and algorithms in a more straightforward way. Instead of switching all crypto functions in all crates, we can switch them in the small scope of SAFE Crypto. Currently, it relies on the same battle-tested rust_sodium library, but eventually we’re targeting to switch to a combination of Dalek (for signing, signatures verification, and ECDH implementation) and Miscreant (for symmetric encryption). What we’ve discovered so far is that the Rust Miscreant crate requires to run a nightly version of the Rust compiler, which are unstable and generally not recommended for production use.

And as almost every of our crates will rely on safe_crypto, they’ll need to be compiled with a nightly compiler too, consequently. So, to prevent this, our goal is to be able to build the Miscreant/Dalek version of SAFE Crypto with a common stable Rust compiler. We’ll be waiting for some preconditions to meet (e.g. Rust 1.27 stabilising SIMD instructions on the x86/x64 platform, scheduled to be released today) and we’ll be working with the Dalek/Miscreant authors and the community to help with meeting this goal.

We’re considering to make some API changes in SAFE Core to discourage creating a safe_core::Client object with user credentials directly from an app written in Rust. We are exploring the option of moving this struct’s constructors out of the struct implementation itself: MAID-2718. This would allow us to export Client from safe_app without also exporting its constructors, which would be separate functions in the safe_core::client module. While these functions will still be available by linking directly to safe_core, they will not be accessible from safe_app which is the API we provide and encourage using for external projects. Our end goal is to make it possible to build a complete application in Rust with only a safe_app dependency. We’re currently investigating other approaches to solve this problem, but we’re close to the conclusion. We also made some minor API updates including updating the safe_core::nfs::file_helper::delete function to take its parameters by value for consistency with the other functions in the module. We also are removing the obsolete MDataAction struct.

Routing

This week has seen the bulk of the Milestone 1 work for Parsec ticked off. The production code has been merged, some initial integration tests written and an example should be getting merged very soon. The tests and example are all capable of outputting a graph showing the history of the gossip and the deduced meta votes. This ability to visualise the gossip graph is not only useful when trying to understand the Parsec algorithm, but is also a huge benefit when debugging. All being well, the kinks should be worked out over the next few days, and we can move on to Milestone 2.

In anticipation of Milestone 2, preparations are well underway for a paper detailing how we intend to handle network mutability (nodes joining and leaving) in the context of Parsec.

Crust

We finally fixed the uTP congestion control bug that caused connections to reset when faced with high load. Each uTP peer has an incoming data buffer up to 64 kB. Peers routinely exchange their receive buffer sizes. When this size is very small, the other end is not supposed to send any more data. The problem was that this receive buffer was ignored, hence congestion control was not effective. That’s fixed now and uTP tests pass on all platforms including Windows.

Crust is undergoing major encryption changes. The integration of safe_crypto is in progress and planned to be finished next week. In the meantime, we slightly changed the peer bootstrap process. Crust bootstrap involves 3 stages:

  • service discovery on LAN
  • bootstrap off hard-coded contacts
  • bootstrap off cached contacts

Until now all of those methods were attempted in parallel. To avoid abusing hard-coded contacts we modified bootstrap to try these stages sequentially: service discovery on LAN, then cached, then hard-coded contacts.

We are happy to see community interest in netsim. We wrote some tutorials and examples to make it easier to get started. As requested by some developers from OSS community, we changed GPL to MIT/BSD license. We did the same for the p2p crate. Hope that helps to maximise use of our libraries :slight_smile:

72 Likes

FIRST!!! Love you all :stuck_out_tongue:

11 Likes

Waiting for 5 hours to post first. Noooooo…

Much better.

Excellent update as always. But why do I feel Sarah should be a writer? Anyone else?

10 Likes

Third… time to read now.

4 Likes

Pardon me while I let you know that I’m gonna read it right after letting you know that I’m perhaps too hypnotized to read it.

8 Likes

I think I need a whisky to help me recover from the website preview…

I’m glad to head the learn section will not be separate. Integrating the ‘learn’ section should make it easier for anyone to dip in & start increasing their knowledge of the network.

Great work all - sounds like things are ticking along nicely :slight_smile:

16 Likes

Ok this is another one of those paragraphs which just totally doesn’t make sense and needs clarification. I’m trying to follow the logic here and keep getting lost.

4 Likes

That sounds really good ! Thanks for the update, keep brewing :smile:

6 Likes

That’s great! Maybe throw in a link to the roadmap or something to give it context as to where we’re at.

Ok I have two questions here:

  1. Hard coded contacts is second on the list. If it’s meant to prevent pressure wouldn’t it be the last option? Or are the cached options a last resort if hard coded contacts fail?

  2. Does “LAN discovery” include internet connections? Last I checked LAN is your local area network.

I don’t even know what netsim is really, nevermind whether I should be happy about it or not. I assume in order to use any of these tutorials one has to learn node.js?

2 Likes

Good job guys! amazing

6 Likes

Quality over quantity! And thank you @maidsafe for listening to your community and taking action! Of course sometimes we might not know best on technical choices but when we can give our subjective yet somewhat educated opinions on something front facing, its great the team can be so humble. :slight_smile:

Awesome update! This always the best part of my week. Is that sad?? :sweat_smile: I don’t think so! Muahahaha :smiling_imp:

19 Likes

Haha, what? I am tired and had a vacation beer, so can be that, but I don’t get the videos and all that. Nice and groovy though (Four Tet-Hands) :smile:

Aanyway. About the PR for the delegates GC fix. Isn’t it supposed to go to safe_app_csharp? (Instead of safe_client_libs. Btw what is this resources folder used for there?) Or am I missing some workflow here?

Also, @nbaksalyar , don’t forget to update the MockBindings!

5 Likes

:hugs::hugs::hugs::stuck_out_tongue::stuck_out_tongue::stuck_out_tongue::heart_eyes::heart_eyes::heart_eyes:
This sends chills. Finish line…

3 Likes

This is exactly what Maidsafe has implemented: hard coded contact list is now the last option, this is specified in the last sentence you have quoted!

5 Likes

mm

Enjoyed, hope Mr Draper (MadMen) is not being lead astray.:sunglasses::heart_eyes:
and no doubt i’ll get article 13 (or is it 11) burnt at the stake for posting a Don gif.

3 Likes

It’s a matter of debate, exactly where we are. As David points out recently, undue expectations only heaps pressure on devs, so in the absence of project and gaant schedules, I prefer to think of it as rounding the home term with the finish line in sight, and that’s where sprinter’s legs will make the difference

7 Likes

Thanks so much for all of your hard work! I’m glad a learning section is being added!

3 Likes

Also, I think that the “finish line” comment is directed specifically at the website redesign.

Nice music selection, I’m assuming that’s @JimCollinson 's choices.

7 Likes

Great minds, love great music :+1:t2::wink:

4 Likes