Safenetwork Git Portal Discussion

Failing on yarn dev for me after following all the steps above

ERROR in ./src/App.svelte
Module not found: Error: Can't resolve './wasi-example_bg_wasi_test' in '/media/user/remote/SAFE/svelte-wasi-with-rust/src'
 @ ./src/App.svelte 187:34-71
 @ ./src/main.js
 @ multi ./src/main.js
ℹ 「wdm」: Failed to compile.

Looks like you missed this one: Safenetwork Git Portal Discussion - #141 by happybeing

2 Likes

2 Likes

Where does the {content} come from in the output? Just searching for the word ‘content’ doesn’t find anything. Is that something Svelty?

Appreciate very much these exciting updates @happybeing. I have written on a list somewhere “look into a few of the languages Mark seems excited about”, largely because of snippets such as the above, I read it out loud and was like, hold on sorry, sveltewasmerjswasi!? :smile:

4 Likes

Where are you seeing that?

EDIT: is this what you’re looking for? Misunderstood the question at first. It is indeed ‘svelty’. The link shows where the variable ‘output’ is ‘bound’ in the HTML. So whenever you assign a string to ‘output’ it will appear in the HTML at the point in the link. So let output = "" starts it off as an empty string, and the results of Rust writing to stdout (e.g. with println!()) are assigned to it here.

You should see evidence of the tests in two places: at the bottom of the web page under “stdout” and in the browser console. It’s not very user friendly, but just about ok for developers.

Me too @JayBird. This was gobbledigook to me until a few weeks ago and it’s still only the basics that I understand.

I’m really surprised I was able to get this working because I know so little. Even Wednesday I didn’t think it was possible and then I tried something and it worked much to my surprise, and it was quite a simple solution in the end. I couldn’t have done it without the encouragement, and help from two guys who are involved with Wasmer - one of the runtimes used to run Web Assembly (WASM).

WASI is like a systems API. With this you can write code in any language that complies for WASI and because it is WASM/WASI it will run on any device with a runtime (like NodeJS).

What I’ve done is get this working with wasm-bindgen and WasmerJS (a WASI runtime for the browser). You could already do this without wasm-bindgen but it’s much easier to develop things now. So people want this but I think I’m the first do it!

The reason that’s useful, and people were keen to help us that wasm-bindgen makes it much easier to handle calls from JavaScript to Rust WASM and vice versa. It’s almost as easily as calling the same language. Without that you have a lot of work to converting parameters from one language to the other and back. Boring work - and I don’t like boring work!

Anytime you (out anyone else) is interested in what excites me about something, please ask. Sometimes it’s just learning new stuff to find out about it, other times it is because it’s creating new possibilities.

I think WASM/WASI may be the next NodeJS, but faster, more compact and not tied to one language (JavaScript). As with Rust a couple of years ago, it is early days so not many people realise that yet or are working with it. That’s always the exciting sweet spot for me. Jump in! :surfing_man:t4:

3 Likes

In App.svelte.

<h2>Content from WASI</h2>
		<h3>stdout:</h3>
		<p>{content}</p>

I am familiar with react, so I was looking for a prop called content but I guess it’s a different kettle of fish

You mean {output} (or are you looking at a different branch?) See my edit above. The variable name isn’t special BTW.

EDIT: @JPL I’ve added more detail to my reply above.

1 Like

OK thanks. I’m at work now but I’ll have a look again later.

1 Like

So are the main benefits speed of loading a page? I’ve been kind of passively following along not fully realizing the potential benefit. Does this mean if you have and existing JS/ReactJS/Typescript app that you could use this bindgen to compile it to WASM for a better/faster web experience on the Safe Network where loading times and latency might be an issue??

Big thank you to @Southside, @JPL and anyone else who had a go for trying this out. It’s incredibly helpful, picked out one bug and helped me fix errors in the documentation so more people should find this useful.

I’ve since made a few tweaks and will announce the thing on social media this week as I start building the Git Portal Prototype using this and the work from the Proof-of-Concept.

Those interested can follow the next phase at Git Portal Prototype. Ideally also the extension of the proof of concept to allow creation and editing issues and comments, and adding this to the UI, but I’m not sure who will do that as I need to press on.

Opportunities to Contribute

There’s plenty to do if anyone would like to jump in with front-end (I can help you learn Svelte), applying Rust git libraries, JavaScript git modules (for stuff we don’t yet have in Rust), and writing a git-bug compatible library in Rust to support github like ‘issues’.

I’d also welcome help setting down coding guidelines and implementing support for tests, documentation and error handling. @StephenC do MaidSafe have anything written down on those areas than you can share?

I’ll keep plugging away and eventually may get something useful, but it’s a lot for one person.

And of course trying things out helps a lot as noted.

Thank you!

11 Likes

The benefits include small code size so yes the app will load fast, but that’s not such a big deal for you as your bundler probably does a good enough job with JavaScript.

The main reason to use WASM in a Web app will be to make use of a different language such as Rust, for speed, code quality and security. Again, not such a big issue for you I think. I’d suggest switching to Svelte before considering WASM for a conventional app.

I think the benefits I listed are more important for the Git Portal, but it would have been possible to build this in Svelte plus JS. I went straight to Wasm so that I could use existing code written in Go (git-bug primarily, but I also benefited from go-git and go-billy) which saved me months of work getting to a proof of concept. Go adds 12MB to the WASM though (in Rust my WASM file is <200k) so I wasn’t willing to build on that foundation.

Having got there it makes sense to switch from Go to Rust target than reimplement git-bug in JavaScript.

For most web apps you either won’t need this or could use WASM to reimplement any parts that are running too slowly. If you do, don’t follow my route (WASI/WasmerJS) unless you find you really need WASI. You can just use WASM, and that’s well catered for by wasm-pack.

I’m using WASI for very particular reasons that are unlikely to apply elsewhere. WASI isn’t really intended for use in the browser, but is a way to create apps that run on multiple platforms (like NodeJS) so most of the WASI tooling, documentation and examples are about that use case.

2 Likes

I’d like to help out and also have a go at Svelte. I should warn you I’m not the quickest though so put something padded on your desk to avoid concussion. Do you have any suggestions for starting materials?

4 Likes

I’ll wear a helmet.

That would depend on your learning style. Take a look at the tutorial, docs and examples (REPLs) at https://svelte.dev. There’s a lot there but if you are familiar with React not much to unlearn - most things I think are simpler and easy to pick up when you need (e.g. by searching the examples there).

It’s probably worth trying something starting from the official svelte template or directly in the REPLs by modifying things you find there (and if you sign up you can save REPLs and come back to them). I think a lot of people begin ideas as a REPL and then copy the files (manually) into a project. If you want to we could start a topic which others can follow, or if you prefer have a PM thread.

Does that help?

2 Likes

Thanks, I’ll have a look at those and take it from there.

2 Likes

Our contributing to the Safe Network doc maybe touches on a couple of bits that could be of interest. Many of our repos also have issue templates set up, which for us are pretty generic.

Git hooks can be useful as long as they’re not too invasive. We use in sn_api here.

For testing we don’t have formal documentation any more. The switch to first Jenkins, then GitHub Actions, now CD via GitHub Actions, plus our repos morphing so much over the years, and a small team, make it a relatively new process that we’re always refining, and relatively straight forward to manage. I can definitely recommend GHA once you have tests set up - it’s a no-brainer IMO.
Been using GHA for a while now & managed to build up some knowledge, albeit from a more or less purely Rust perspective, so I’m happy to help out wherever I can in setting something up there - DM me if I can help.

Documentation & error handling - these are areas that I think we have a way to go to improve on, we just haven’t found that right formula that we are really happy with. You will have no doubt read the updates where we’re often mentioning improving the docs, and recently there has been a focus on error handling. I’d say we’re still experimenting here. The refactoring work undertaken over the past year has helped simplify things a lot for the current team so hopefully we’ll be in a better place with these two soon. We don’t have anything written down that I can pass over, but if you have any specific questions then fire them over and I’ll let you know our experience on it.

9 Likes

Thanks @StephenC I hadn’t thought of looking at this so will do. I’ll look out for things and share anything useful that I find or set up.

4 Likes

Mini Git Portal update - commits pushed today:

  • Setup test framework for API using uvu and puppeteer
  • Add documentation generation using JSDoc with docdash template

See: GitHub - happybeing/p2p-git-portal-wasi: Peer-to-peer Git Portal in Svelte, Rust/WASI on WasmerJS

11 Likes

Found this in my recent searching and thought you might be interested to see it. -

Had to add this quote " Finally I made the step to read the rust book and boy was it worth it. I mean, Rust is the Dark Soul of modern programming languages, not because it is intrinsically difficult, but because how unforgiving the compiler is."

3 Likes

Mini Git Portal Update (4th Feb 2021)

Three really nice things have been added or enhanced:

  • automated tests
  • continuous integration on github
  • improve generation of documentation (for internal APIs)
  • all the above updated in the README

Development and Production Testing

I mentioned previously that I had set up a test framework, but now I’ve supercharged it. What I had was the basics, but now I have it firing up two servers so the tests can run fully automatically.

One server is for the app, which can either be run on a new build (for production testing) or using webpack-dev-middleware for hot reloading during development. :tada:

The second server is a trivial githttp server stolen from wasm-git (which I worked on recently - and had a PR accepted! - while evaluating options for Git Portal). That means I can create more complex internal tests without using a proxy if I want, though the latter will also be an option during development.

It alls sounds a bit geeky to non-developers, but its a big plus for the project and I’m chuffed because I’ve never done any of those things before. All I have to do now is write the tests! And it will make it much easier for others to jump in and help out.

Continuous Integration on GitHub

Encouraged by @StephenC (thanks for offering to help but I managed with a little trial and error) the production tests mentioned above are now run automatically by GitHub whenever code is pushed to main or a PR is submitted. And all tests currently pass!

I’m chuffed as it’s the first time I’ve set up continuous integration on one of my projects and a useful feature that makes it much easier to collaborate with others…so more PRs please!

Improved Documentation Generation (for internal APIs)

To help folks contribute and make the project easier to maintain I wanted the APIs used by the front-end to communicate with the application-logic to be documented as well as tested, and had a basic set-up in place (from the previous update).

But to make it useable I had to do some more tweaking and ended up writing my first JSDoc plugin. It’s simple, but also a new skill that means I can do much more with this when needed.

The main change this time is to shorten the symbol names. Previously JSDoc was prefixing everything with “export.” which was making the menus unusable, and generally polluting the documentation with useless information.

I also learned a bit more about how to use JSDoc and will at some point be writing some guidelines to make it easier for others to jump in and help so…

Help Welcome!

I’m only one person, and a slow one at that :grinning:, trying to create a GitHub alternative and even though there are lots of economies doing this on Safe Network (see below), I won’t be able to complete this all on my own.

So if you want to contribute in any way, don’t be shy. Testing always helps, following the README and finding things that don’t work is invaluable and of course there is code to write in Rust or JavaScript, Svelte stuff to do, front-end UI to envision and build, new post-GitHub features to imagine and much more. All are welcome and I’ll help you skill up wherever you fancy trying something new.

And it is all built on exciting new technology such as [cough] Safe, Rust/WASM, WasmerJS/WASI, my old favourite Svelte and the fancy automated testing and CI I described above.

This is fun, and an exciting valuable project to work on.

18 Likes