SAFE Network Dev Update - June 13, 2019

Summary

Here are some of the main things to highlight this week:

  • We now have a POC version for a safecoin wallet command-line interface against a mocked vault. See the CLI User Guide we published for more info on how to try it out.
  • We significantly progressed with the test Safecoin and new data types in SAFE Client Libs.
  • We took a few early steps towards enabling BLS keys in Routing by merging a pull request that adds Distributed Key Generation to Parsec.

Marketing

This week, we’re giving a big shout out to @oetyng who has set up not one, not two, but three new Meetups in Sweden! Stockholm, Gothenburg and Malmo now have their own SAFE Network groups. As ever, if you’ve got a meetup account, please do join each of the groups and help @oetyng to get the groups off to a flying start!

@dugcampbell was down at the CogX Web3 Conference in a very wet London at the start of the week (great to meet @Bridge and @Kafkalee in person!) and also on BBC Radio Scotland this morning opining on the new Coinbase Card. And we also sent out a new Newsletter to subscribers today - another wee content experiment, give us a shout if you liked or hated it in your inbox. Check out our latest tweetstorm and, in the coming week, if you’re out and about in Edinburgh on Wednesday, we’d urge you to head along to the Scottish Blockchain Meetup where we’re sponsoring Andreas M Antonopoulos’ talk. We’ll likely have quite a few of the team in attendance so give us a shout if you’re heading along and we’ll make sure we take enough swag along…

Project Overview

We are delighted to announce our latest project deliverable on our way to Fleming: the safecoin wallet command-line interface (CLI) - see the SAFE CLI section below for more details. It shows the basics, including (test) Safecoin and starts us on the path for full integration of this into SAFE Client Libs (coming soon!).

These deliverables will allow developers and interested parties to see the interface that will run in the Fleming release. As stated we are looking at Fleming to really bring something special (no promises, but let’s whisper it since we’re amongst friends… how does full vaults from home sound to you all…? :wink: ).

To accompany the GitHub boards, we’re also providing you with a high-level overview of the project Gantt chart:

It shows the large chunks of work that are currently in flight, each with their dependencies and sub-projects. From this, you’ll see the path to Fleming involves the release of:

  • Wallet CLI (done)
  • SAFE Client Libs integrated with Wallet CLI (soon)
  • Vaults phase 1 (Vault connected to SAFE Client Libs to allow full vault functional testing)
  • Vaults phase 2 & 3 (a mini local vault network, just for you)
  • Parallel to Vaults, we have the work happening in Routing (quic-p2p integration, BLS etc…). And once these are ready, we’ll move onto bringing Vaults and Routing together (in other words, it decentralises the vaults).

Then folks, we are in Fleming release territory…

We plan to share some more deliverables along the way, each setting out the functionality that the released network will use.

So how come we’re doing this now? Well, it’s all down to the project plan bedding in and (more importantly) a well-motivated and focused team (some too focused and working all nighters, we’re looking at you @nbaksalyar). Yup, that’s right. There’s a definite air of confidence brewing and we will continue to build on that as we execute the remaining deliverables.

SAFE CLI

We’re happy to be able to give something that hopefully lets folks start to get their hands dirty with the SAFE CLI.

We have some initial POC work against a mocked vault (integration with the actual safe_client_lib API and mock vault will be following promptly). What does this mean? Well, it means you can dig into the SAFE CLI and start playing around with Key's, keypair's and Wallet's commands. More detail on all that can be found in the CLI User Guide, which takes the user through all the necessary steps to authorise the CLI application using the safe_auth CLI, as well as explaining what these new concepts are, and how to interact with the SAFE Network using the already available CLI commands. We highly recommend thoroughly reading the CLI User Guide before trying to run any CLI command.

Thus, as mentioned above, the CLI now creates a local file named mock_data.txt, but hopefully this lets folks see how things are shaping up and can help us get useful feedback on usability/naming, etc. Some parts where feedback and opinion would be greatly appreciated are around usage flow, argument order and naming.

It’s worth noting some of the current limitations in this POC version. There’s no default Wallet for an account for example, which means the payment source always needs to be specified as part of the commands. This should hopefully get smoothed out later with source becoming truly optional as it should be.

Here are some samples of what’s possible right now. The following will create a Key on the Network, and preload it with test safecoins, as well as provide a nice human readable output:

$ safe_cli keys create --test-coins --preload 15.342 --pretty
New Key created at: "safe://bbkulcbnrmdzhdkrfb6zbbf7fisbdn7ggztdvgcxueyq2iys272koaplks"
Key pair generated:
pk="b62c1e4e3544a1f64212fca89046df98d998ea615e84c4348c4b5fd29c07ad52a970539df819e31990c1edf09b882e61"
sk="c4cc596d7321a3054d397beff82fe64f49c3896a07a349d31f29574ac9f56965"

Once we have a Key with some test coins, it’s possible to check its balance:

$ safe_cli keys balance safe://bbkulcbnrmdzhdkrfb6zbbf7fisbdn7ggztdvgcxueyq2iys272koaplks --pretty
Key's current balance: 15.342

Many more examples are explained in the CLI User Guide.

Please share any issues or questions in the CLI thread where we can all help each other, and/or report any bugs you encounter directly in our CLI GitHub repository.

Mock interface

As a result of a joint effort from @fraser, @jean, @lionel.faber, @marcin, @nbaksalyar, @ustulation, and @Yogeshwar, we made significant progress with the test Safecoin and the new data types implementation in SAFE Client Libs. We implemented the basic functions defined in the Safecoin RFC: creating new balances, coin transfers, and transaction detail requests. There were some other internal changes in SAFE Client Libs which should eventually help us to support multiple ownership for data and wallets through the BLS encryption.

Over the next week or so, we will be rolling out the FFI APIs which would allow app developers to use all new features. We are also planning for new refactoring and simplification work: Clients will be decoupled from Routing, connecting to Vaults directly through quic-p2p instead. This will allow us to prototype Vaults more quickly and move forward with important tasks without being blocked by Routing. Eventually, all pieces should fall into one place and Vaults will be the main (and only) consumer of Routing.

Vaults

Progress on Vaults has continued this week: we are creating a version of Vaults that doesn’t rely on the Routing crate. It communicates directly with clients over quic-p2p. Without Routing, it doesn’t allow the creation of a decentralized network, but it can be used to model a network with a single Vault; or eventually a fixed number of Vaults. This will help us make sure we can establish the vault/clients API and allow the frontend to continue progressing at a fast pace while progress on Routing continues in parallel. This removes a long-standing bottleneck on Routing for experimenting with new data types, safecoin and such features.

SAFE Browser

This week we’ve implemented some improvements regarding auto-updates, and the notification flow surrounding that. We’ve also been looking at how we’re publishing both prod and dev versions of the browser, which has added some complexity to the auto-update feature (as this isn’t supported via GitHub), so we’re looking into what other providers might be best for this setup.

Reliable Message Delivery

We are continuing work on the Reliable Message Delivery aspect of Routing. This follows the Reliable Message Delivery RFC that was recently published. The outcome of this work will be a routing network where messages virtually always make it to their destination; although the authenticity of the messages won’t be verified and the system won’t be resilient to spam in the first place. The first of these limitations will be addressed soon after when we implement the Secure Message Delivery RFC.

What we gain from this work is a significant simplification of the Routing code base, which will help progress other aspects of the upcoming work in the Routing crate and a sound foundation for Secure Message Delivery.

Integration of quic-p2p into Routing

More simplifications to the Routing crate are coming by taking advantage of the new API for our network layer: quic-p2p. We can now “fire and forget”: send a message and expect that it will be delivered or, in case of failure, come back to us so we can resend it. This removes a layer of complexity from Routing where we had to keep track of who was online at any given time. We can now be more loose with this requirement and simply accept a peer as responsive if the message we send them doesn’t bounce back to us. The work on that track has made good progress this week and a PR should be raised soon with the bulk of the initial changes.

BLS cryptography

We took a few early steps towards enabling BLS keys in Routing by merging a pull request that adds Distributed Key Generation to Parsec. Most of this code comes from our earlier effort towards implementing a common coin, but we are cherry picking a subset of this work that will be relevant to Routing in the short term.

We will want to switch to BLS in Routing as it will allow a much more efficient way of verifying messages through Secure Message Delivery. Depending on the rate of progress of different tracks, and on whether it would ease development, we may integrate BLS in Routing before or after implementing Secure Message Delivery.

Community contributions to Routing

We would like to thank @d1vyank on GitHub who has actively participated in the development of the Routing crate this week. They identified multiple opportunities for refactoring in Routing, based on FIXME and TODO comments, and raised as many pull requests to address the issues. This was really useful and we were happy to merge four of their PRs in the last few weeks, including two this week.

We would also like to shout out to @oetyng for some more contributions on our BLS RFC thread this week, and for every other forum member who’s been involved in the various RFC discussions in the last few weeks. :bow:

101 Likes

Last but not for long :smiley:

54 Likes

No mercy this time, we are all first!! :smiley:

30 Likes

Just checking as I’m boarding a plane. Thought I’d snatched first by pure coincidence. Dang it!

31 Likes

Thanks for sharing weekly updates, and nice to see you @dugcampbell :slight_smile:

23 Likes

Fiiiiiiirrrrrrrrrsssssssssss…

Oh.

28 Likes

Damn… today SAFE is over 9000!!! :smiley: I feel the power in the air even at the other end of the continent… Gogogo team MaidSafe!

22 Likes

Wow, just wow…

25 Likes

Perhaps @oetyng would like his/her own “SAFE Ambassador to Sweden” title. Or maybe some other customized title of his/her choosing.

As always, many thanks to the entire team for all of the hard work you do! And congratulations to David Irvine for being first to respond. You are a leader in more ways than one.

27 Likes

Well that’s a mega update and some! Fantastic all round and great stuff to see SAFE meetups sprouting up - very cool @oetyng!

30 Likes

I’m basically speechless. The progress is absolutely astounding and I can’t wait to have a vault running from home again. Without invitation tokens! Let’s prove this tech and keep marching on!

Test nets coming soon!!! :tada:

32 Likes

Blown away. You folk are on :fire::fire::fire:

30 Likes

Great update! Awesome initiative, @oetyng!!

As it should!! Between the ramped up marketing activities and the increasing clarity on development, everything feels so much more palpable. The leap from research to development has been made, and it’s electric! Excellent work, MaidSafe :smile:

28 Likes

pretty impressive update! i’m knocked out cold :upside_down_face: need to read again when i recover :ok_hand:

24 Likes

This reminds me I must have had one extremely brief moment of a dream on someone giving a presentation on malicious nodes and bypassing them via PARSEC. That’s a first; SAFENet dream.
The dream is real.

Sometimes I feel like these developers must have extremely deep senses of insight/reward, whenever being successful in developing or suddenly understanding great connections and then communicating them among their peers. And that isn’t really seen to the general public, unless they consider/ponder all these details of what could possibly be happening together among the developers. Mammoth job!

19 Likes

Glad to hear about some all nighters :stuck_out_tongue: , makes you feel aliveeee as a dev. The best time to code is that like 11pm/23 hr and grind all the way till 3-4am with 0 distractions and no one to ping or bug you in the darkness(as long as management lets you sleep in till 9:30 or 10 so your not totally dead next day). That’s the kinda nights where you can crank out like 1000+ lines of decent code. My managers all say don’t do it and #workLifeBalance but when you wanna really get something done I consider it one of the only effective ways for me to get things done when I really wanna grind away on a codebase. Play a song on loop in the background and melt into your monitor for a solid number of hours.

Exciting update, looking forward to seeing the fruits of yalls labor! Keep grinding and if you need late night code grind music suggestions feel free to dm me :smile: .

26 Likes

Bloody hell! You guys are banging out the code now! Fabulous update and deeply impressive!

SAFE CLI, test safecoin, vaults from home! Stunning! Keep up the great work guys! :partying_face:

26 Likes

Sounds so amazing I’m speechless… another killer update team!

25 Likes

Nice to see so much work being done and concrete plans for implementation. Looks like we’re approaching something big. Good job!

18 Likes

This, this… Mwah…! :heart_eyes::hugs::kissing_closed_eyes:

Mind-blowing update all together.

About the meetups:
Since the groups were published yesterday about 100 people have joined!
Was not expecting so many to join at once and without any pushing of it :smile:
I have found myself with more time on my hands, and I’d really like to get together with people and discuss SAFENetwork and how to develop against the APIs etc.
I’ve actually been thinking about this for several months (only one group though). The reason it became three is because I don’t mind taking a little road trip and there was obviously blank spots on the map. And the three biggest cities here are all within reach for me.

It feels like this is coinciding quite well with all that is happening and coming to fruition now, it’s a good time to start bringing in people.
I’m really looking forward to meeting all these new people, of whom many probably didn’t know of SAFENetwork before.
Now I have some planning and work to do to get material for the meetups. Since there’s probably a wide span of background and previous knowledge, I thought maybe a thorough but high level presentation of the network would be meeting #1. From there on would be deeper.

Big thanks to @Sotros25 who helped out with experience, advice and material.

45 Likes