Proposal for an inspectable, easily deployable, and API-compatible implementation of the SAFE network using JavaScript and WebRTC

I spent most of the last week reading the source code of the current C++ implementation, especially the Routing and RUDP parts. The event-based nature of the code and the concurrency-handling parts make it hard for me to follow the core logic of the algorithms. That will probably come with usage and experience. I recon though, that the two main advantages of the current C++ implementation are its ultimate performance potential, and the security of depending on the least amount of trusted code.

On the other hand, I find it hard to inspect the code to convince myself of the correctness of the algorithms, and the running system to get an intuition of its behaviour during execution. Since I am trying to produce a presentation that will introduce the core ideas of the system, its would seem useful to have an implementation of the same algorithms that might run in a web page using event-based JavaScript and WebRTC for peer-to-peer communication (which would replace rudp). The implementation might compromise on (1) security by leaking some meta-data information about connections between peers, (2) performance by running everything in a single-event loop in JavaScript, and (3) have left-over ā€œcentralizedā€ components that are required to established connections between peers, but would otherwise implement the exact same algorithms. That would probably simplify the implementation and make it easier to inspect the dynamic behaviour of nodes, by leveraging all the JavaScript frameworks for visualization (such as D3.js).

In addition, by aiming for full API-compatibility with the C++ +JavaScript bridge implementation, that would probably allow Web Application Developers to start developing their apps faster, before the ā€˜realā€™ network is live.

I am thinking of reimplementing the Routing library as a first step, and maintaining compatibility with the Routing API, as defined in ā€˜routing_api.hā€™. That could be used to develop a first chat application. All information that would need to be persisted would be stored on the server that is used to established the peer connections. Needless to say, that would be highly insecure so no important user data should be stored.

One major advantage of that implementation, would be to demo the changes in the state of the network as nodes are added, removed, and communicate with one another. The second major advantage would be to make it a lot easier to deploy for web app developers, since a single library would need to be added to their one-page web application. As long as the API is fully compatible, it would mean they would simply have to change the library being used when the real network will be up.

Let me know what you guys think of such a project. I would like to make sure it is seen as a complementary implementation and not a hostile ā€œforkā€.

4 Likes

The way I read this, is that this project would be for (1) Showcasing the network to an audience in a graphical manner (2) Act as a staging ground for developers to observe the interaction between network and app.

Is that around the mark?

Is it correct to say, that the project is not intended to go into the wild?

Yes

I intend it to be open source and freely accessible. In that sense, it would go into the wild. But I would make a clear disclaimer about its inherent lack of security and make it clear that the ā€˜realā€™ implementation is going to be the one produced by MaidSafe.

I think you should go for it erick, to visualize a SAFE network in motion would an awesome promotional tool.

Look forward to the Dev inputā€¦

Agree 100%

@erick Iā€™d definitely pay attention to this project also;

Thatā€™s fantastic, a dynamic visual representation would go a long way in educating people how the network works. I wonder though, how big of an undertaking is that, this seems massive!

Also, for peer-to-peer networking over WebRTC you might want to have a look at OpenPeer. I havenā€™t used it and Iā€™m not sure how far along they are in their implementation but itā€™s worth checking out.

So this would be like a sandbox implementation?

I am moving in that direction slowly but surely, the amount of time I dedicated to the implementation is lacking behind what I put on the lecture series and solving funding issues.

In the last days, I have been reading the awesome doctoral thesis by Cosmin Ionel Arad, Programming Model and Protocol for Reconfigurable Distributed Systems which introduces tools to tame the complexity of such an undertaking. Iā€™ll work on porting Kompics (the Java framework that implements the ideas in the thesis) to JavaScript, and modelling the entire architecture of the distributed system part of the SAFE network using the notation of the thesis. Some of that will eventually makes it way into the lectures series.

Eventually, once I get enough going, it would be great to have support from the community to test and debug the core algorithms in JavaScript.

1 Like

Not sure what you mean by that, but if you mean that you should not depend on the JS version for critical data than yes!

I meant an implementation where you could see the effects and implications of trying new stuff, without it affecting anything real. A place where you observe the load created by different applications in a way that you couldnā€™t on the safe network per se.

In that case, yes it is going to be a sandbox ;-).

Sounds like a good plan, Iā€™ll keep an eye on this and see how I can help when it gets there. Meanwhile, Iā€™ve found something interesting to read, thx for the link.

On a side note, I contemplated for a moment how to send the pdf to my phone. Email? Search on google the name of the thesis? Copy url by hand? This is gonna be so straightforward with Maidsafe, keep up the good work, the world doesnā€™t realize yet just how much it wants it :slight_smile: