Phantom: a publishing tool enabling users of the SAFE Network to easily manage websites

Evening all, I wasn’t expecting to have another update so soon, but I couldn’t sleep tonight so I smashed through some work instead.

At the moment, most of these calls are mocked, so the safe://phantom website won’t be functional - I’m waiting on a couple of very minor updates in the next week from the SAFE development team to unblock me (and anyone else doing SAFE browser-based development), so while what you’re seeing has all been coded in entirety, the underlying SAFE network libraries are all using mock data for my own testing purposes.

New features

There is now a posts listing page, accessible after selecting an NRS to edit:

Creating a new post takes you to the post edit page, where basic markdown is now supported, including a live updating preview (very similar to the safenetforum editor, only bigger). I still need to add support for anchors, blockquotes, code blocks and images. I’m actually quite proud of the markdown -> HTML formatter so far, I’m just very happy with how so little code is accomplishing so much. Normally, when I write raw Javascript it ends up a horrible mess, so I guess it was a good night for me. :stuck_out_tongue:

At the moment, the save post / publish post buttons do nothing, but all the Markdown -> HTML work has been done, so making those work will be minor, I just need to do some more mocking work first to enable it.

And here is the post listing page if you click the back button:

Design decisions

I’ve decided that posts will be stored as raw markdown, with a small amount of HTML issuing a REL canonical, at their respective post URL, for instance:

  • Accessing /#/post/draft-post will show you the webpage with all theme styling, javascript, etc.
  • Crawlers can simply load just /post/draft-post which will contain the text-only markdown version of the page, but the canonical URL will point to: /#/post/draft-post

This means that despite it being a single page app, crawlers shouldn’t have any issue indexing the content of the website.

I mentioned in the last update that I was targeting a 180kb download for the entire Phantom application (all pages, all CSS, all Javascript, etc), and at the moment I’m doing pretty well:

post4

Compressed, I am totalling 67KB, + 44KB of font files, so 111KB, uncompressed I am totalling around 230KB. I’ve managed to get it so small by being absolutely ruthless with dependencies, at the moment I have no dependencies which aren’t required for VueJS to function (for instance, writing my own Markdown parser as tersely as possible, writing my own SAFE network libraries, my own Promise library, my own cache library - with the position of caring more about code-size than ease of development).

I don’t see any reason why I shouldn’t be able to keep the compressed (what is sent over the network) size under 150KB and the uncompressed size under 260KB for the final project.

27 Likes

:clap::clap::clap: looks great Shane! Will this be or is this mobile friendly? Would be fun to take it for a test drive on the mobile browser when it’s good and ready. :smile:

11 Likes

@Nigel it will be, but isn’t quite yet. I haven’t had time to sort out the SAFE browser on my phone yet, so it’ll be a few more days before I have a chance to test my media queries.

9 Likes

I’m really looking forward to all of your work.

What about the other dude named @AndyAlban? He kinda disappeared just like you did. Does he plan on rejoining and helping you again?

3 Likes

Andy is actually doing really well for himself these days - I went to a comedy show with him a couple of days ago.

He was my protégé earlier in his career and I gave him his first job in the industry. When my company was sold and I started contracting, I took him on again, and then when I joined Bunting as the Head of Technology, I hired him a third time as a software engineer.

He’s working at https://yozu.co.uk/ these days as a Ruby software engineer, with a bit of DevSecOps sprinkled in. We still talk about life, work and code almost every day - he’s going to be the best man at my wedding in Spring 2022.

As for whether he’ll join in, it’s unlikely, he’s just taken on a spare time contracting project so he’s swamped, but it’s open source so there’s nothing stopping him from contributing in the future.

12 Likes

Hi all, I know we only got an update a few days ago, but I didn’t want to leave anyone expecting something this Wednesday waiting.

I’ve made some good progress on the underlying structures used by the app in the last few days, although this hasn’t manifested as that many visual changes.

Screenshots

I’ve added support for images to the Markdown editor, and support for anchors. Of course, this also means support for anchors containing images. :slight_smile: I still need to add code blocks and blockquotes, these aren’t that difficult, I just haven’t got around to it yet.

I plan to add support for image uploads in the next few days (as opposed to just hardcoding images) so you should be able to create and manage images from within the post edit page.

I’ve removed the “Publish post” button from the post editing page, in favour of having a “publish drafts” option on the post listing page. It seemed a cleaner / easier way to manage published website NRS versions.

Here you can see the post listing page. When you save a draft, it updates the FilesContainer. You are then presented with a “Publish drafts” button on this page which when pressed will update the websites version. At the moment, that doesn’t happen - because I need to add code to generate the blog home page (the post listing page). Expect this in the next update.

As I said in the last version, post URLs are indexable by search engines and are stored as markdown wrapped in HTML, with a corresponding canonical link. This gives you the flexible (and cheap) deployments of a single page app, with the searchability of a pregenerated website (what is known in the business as server-side-rendering).

I’ve been asked a few times about making the work responsive. The long and short of it is that I’m waiting on some API updates from the Maidsafe team before the project is in a deployable state where I’m comfortable testing it. As soon as I’m able to deploy and test it to an acceptable level, I’ll look at installing the Mobile browser and doing the responsiveness changes.

Background work

  • In the last week, I finalised RFC-0060: Enable creating and modifying of FilesContainers within the SAFE browser environment, adding an extra option (support for HTTP Content-Type mime-types and charsets when updating Raw buffers to FilesContainers) and put it forward for review.
  • I (re)discovered and reported a bug in the core API’s file synching code which was causing files not to be updated if the new size of a file was the same as the old size of the file.
  • I spent a lot of time messing around with the core API, looking for edge cases where I could eek out more usage, or areas where the API left something to be desired. Expect more topics / bug reports to come of this, I prefer to create a topic when I’m sure something is a bug (mostly to save face), so this takes me quite a while to fiddle.

In short, the next update should contain something akin to the following:

  • Enhanced Github flavoured Markdown support
  • Generation of blog post listing pages
  • The beginning of an inline image uploader
  • Thoughts and idealogies around the support of themes, including the support of a subset of the Wordpress event hooks for familiarity, namely Actions and Filters, where there is a translation directly from server-side-wordpress to client-side-SAFE.
33 Likes

This is an awesome project and initiative. Please keep up the great work. I’m excited to get to use this.

13 Likes

Your a friggen rockstar. SAFE is lucky to have someone tinkering around with it right now as much as you have even in this short window. Looking forward to Phantoms release in due time!

15 Likes

Hi @Shane,

can you provide a MAID address for donations? :dragon:

9 Likes

I’ve been thinking about this since I woke up this morning, and I’ve come to the conclusion that I don’t and won’t have a MAID address for donations. I don’t like the idea of taking value out of the community when I haven’t provided any tangible value as of yet. Don’t get me wrong, it would be so easy to take money from everyone and parade around begging for donations to feed a lavish lifestyle, but that’s not me: I write code because I enjoy it, as soon as I take money it will begin to feel transactional.

In the future this stance might change, depending on which project I’m working on and what my financial situation is, but at least while I’m working on Phantom I have no intention of taking donations to support development. The only support I need is the :heart: of the community. That being said: thank you for the offer, it’s greatly appreciated.

30 Likes

I understand this perspective @Shane and at the same time think it is ok to accept or ask for donations - people don’t have to donate. I think it’s important to be clear about the nature of any ‘transaction’ but it can be as simple as: “if you like what I do and want to show your appreciation through a donation I appreciate that very much, but accept on the understanding that it doesn’t come with any strings.”

My understanding of care work is that it is poorly paid, so it seems reasonable you might find a bit of extra income helpful.

Anyway, it’s great to see you building again and useful to be able learn from such an experienced developer.

10 Likes

Edit: Whoops, didn’t realise the weekly update would be at the same time, sorry for stealing some of your thunder!

Hey all, these updates seem to be coming thick and fast at the moment.

Editor

The editor now has support for the following markdown:

  • *** horizontal rules
  • ___ horizontal rules
  • --- horizontal rules
  • Inline code blocks using single backticks
  • Multiline code spans using three backticks, as well as support for extracting the language you’re targeting.

You can see here the javascript language being extracted from the multiline code span as a CSS class you can target or extract to pass in to a syntax highlighter. There’s no support for any syntax highlighting out of the box due to the size it would add to the bundle, but with the upcoming theme system it will be very easy to add a syntax highlighter.

Theme system

I’ve been doing a lot of thinking and tinkering with the theme system over the past few days and I’ve settled on an idea I like. It’s worth mentioning: this is a first draft, work in progress, and is very likely to change based on feedback or code complexity.

For most users, they will simply go to the theme page, and either click on one they like, or import a theme from a SAFE network URL they found. This allows people to make and share themes easily, and for users to import an entire theme with just a single URL.

Theme manifests

The manifest describes the theme, gives it a displayable name, a banner image to show alongside the name, and tells the theme compiler about the assets the theme contains.

screen2

Theme templates

Themes are built using Vue.js, but don’t worry - I’ve abstracted away all of the complexity and left just a couple concerns to deal with. Theme template files contain the basic HTML for the webpage, as well as some text/x-template script tags which can be used for telling the Vue application what HTML it should use for each section.

… And that’s it. It’s that simple to create a theme. (obviously, you’d have to spend time styling it and what not, but that’s to be expected.) A theme author can create the manifest, publish the files which the manifest points to on the SAFE network and the Phantom theme compiler takes care of the rest. It inlines all of the assets and creates a hash-routing based single page Vue app.

Here is a work-in-progress example of a compiled website, ready to be deployed to the SAFE network:

At the moment, the only CSS in the theme is a red background, but all of the code is there and all I have left to do is create the “bundle” containing the information about the posts and generate a Vue single-page-app route for each post and it’s done.

The final size of a website deployed using Phantom is about 120KB (uncompressed!) and it contains all the code needed to bootstrap the application, handle routing and inject each post / generate a post listing page:

screen3

Like I’ve said, this is a working proof-of-concept and it’s a bit rough around the edges, but the long and short of it is:

  • The Phantom app will contain a handful of basic themes
  • End-users can select a theme by clicking a single button
  • End-users can import a theme created by another author by clicking a button, inputting a URL and clicking “install”
  • Theme authors can create and share themes easily and quickly, with next-to-zero knowledge of how Vue works internally.

All this functionality will be available within the SAFE network as soon as RFC0060 has been accepted / implemented, as without it there’s no way for web apps to create FilesContainers directly in the SAFE network. So at the moment, all of the SAFE calls are being mocked, but I’m fairly (99%) confident as soon as that RFC is implemented all I should need to do is deploy and it should work out of the box.

22 Likes

The last update was really rather developer focused, so I thought I would follow it up with a simple explanation for how themes work from a users’ perspective. You click the one you want and it automatically works. That’s it:

If you want to use a theme made by someone else (or yourself) you just click on Import theme and pop the SAFE URL in the modal which opens, and the theme appears in the list above.

My ultimate aim is:

  • To make it as easy as possible for people to install themes, whilst:
  • Making it as easy as possible to share themes with others, whilst:
  • Making the process of creating a custom theme as simple as possible.

If I can hit all three of those, we are on to a winner in terms of user experience. I will, of course, be shipping with several themes, since they’re so quick to throw together.

18 Likes

Huge props to your productivity and drive to make this so lightweight and inline with SAFE ideals. Great having you back Shane :slightly_smiling_face:

13 Likes

Hey all, again, post updates seem to be coming thick and fast. I’ve really missed writing code, to be honest. I’m away this weekend for a party so I likely won’t be giving any updates again until Monday.

Deploying posts

On my post listing page, I can see two posts I’ve created previously, I select the top one for editing:

Using Markdown, I add a picture of a cat to the top of the post, then I save the post. It’s state is changed from Published to Draft. I can click on the Publish drafts button to deploy the posts and create a new NRS version:

Once I click it, it builds up the posts in to a bundle, as well as the Vue single page app needed for the posts to be displayed. It creates a bundle which is 124KB in size. This bundle contains every. single. line. of. code. required. It’s literally the only thing your users have to download, and as a single page app, they won’t trigger any more network requests.

At the moment, the SAFE network doesn’t GZIP content, but I want to stress that the 124KB is uncompressed. If the SAFE network ever adds GZIP download support, the bundle size will be reduced to around 70KB.

screen6

If I navigate to my NRS within the SAFE network, I’m now presented with my blog. It’s that easy. (The styling here is just for my testing, I’ll make the themes look a lot nicer than this):

If I click on the top post, It displays the post content:

Going forward

We’re now at a point where (barring the implementation of RFC0060 by Maidsafe for the API calls I need) we have a fully working app, which is very rough around the edges.

I still need to do the following:

  • Add the option to add arbitrary pages, similar to how wordpress has “posts” and “pages”
  • Add and style several basic themes
  • Add some more error handling so it’s less fragile when things go wrong
  • Add “custom fields” to themes, so theme publishers can offer configuration options to users
  • Start to think about a plugin system, and how we can add dynamic content in the future (Comment systems, Ecommerce platforms, that kind of thing)

I’m going to take the weekend to think about how the custom fields will work, and how I want to integrate dynamic content.

34 Likes

Fantastic work, Shane.
Take the weekend off and go to a party :slight_smile:

7 Likes

Enjoy the party Shane, amazing work!!

5 Likes

Hi all. Before I start I want to clarify that this update is geared towards future theme developers, as such, if you’re just going to be a normal user (someone who simply uses Phantom for writing and sharing blog posts), this probably won’t be relevant to you.

Theme.json and configuration

I’ve began work on adding the concept of “custom fields” to the theme.json files so that theme developers can add extra options for users to select from. I’m aiming for “powerful but easy”, so there will likely be some compromises here in the future, but it should GET THE JOB DONE.

Below you can see a (valid) theme.json file, from the Banana theme I’m using for testing:

It has the following REQUIRED data:

  1. The name of the theme
  2. A basic description to show the users on the themes page
  3. A square banner image used to “fancy up” your theme on the themes listing page
  4. A template which points to the basic HTML of your theme
  5. An ordered array of scripts which will be imported in your theme.
  6. An ordered array of stylesheets which will be imported in your theme.

It has the following OPTIONAL data:

  • An array of configuration options you can display to the user.

If the config field is present, then the following data becomes REQUIRED:

  • The name of the configuration option
  • A basic description for the configuration option
  • A count (containing either single or multi) which enables your users to input data multiple times if enabled (for instance, we have here a menu-items config where the user can input multiple menu items)
  • An array of fields containing:
    • The name of the field
    • A basic description for the field
    • A HTML5 input type for the field (supported types are: text|email|date|number|tel|password|time|file)

Theme file linter

To make the work of creating / testing themes easier, I’ve created a simple linting tool which codifies these requirements.

For instance, if you try to import a theme.json file which has an invalid config field type, you will be presented with the following error:

Once your theme has been linted, it will be presented alongside the default themes, with the same “1 click install” as the other themes have:

Linting code

I don’t normally link directly to blocks of code, but I’m going to link the theme linter code right here

I’ve gone to the effort of being incredibly verbose and procedural when writing this, so yes, I’m aware it’s very repetitive, and I’m aware it could be simplified, but I figure that minimising the amount of magic happening here will make it as easy as possible for developers in the future to debug issues with their themes.

At the moment, the config fields don’t do anything - I need to add the Theme configuration page which will ingest them, spit out input fields, ingest the inputted data, and spit out something easy to query within the theme template. Expect to have an update with the code to manage that within the next few days.

18 Likes

@Shane i’m not sure how you envisage adapting other themes. Wordpress has a concept of ‘child themes’, were you can only provide the parts you want to override.

There’s a benefit over just forking in that you’d get upstream changes automatically. Could work quite easily with xorurl of a ‘parent’ theme in the config perhaps. That’s just one idea anywho, wanted to throw it out there incase it impacts anything you’re on the now.

10 Likes

I like the idea and it wouldn’t be hard to compose something like that, let me have a tinker and get back to you with a working concept.

12 Likes