Commoditising Hardware - An idea for an operating system on the SAFE Network

This idea is entirely separate from MaidSafe as a company. If I work on this, it’ll purely be in my free time, entirely separate from my actual job.

Well, the keynote address first thing this morning from @dirvine sparked a few ideas in me. The mention of the commoditisation of hardware set a few gears in motion in that thinker of mine, so this is a thread to get everything out and on paper into the community.

I see from searching the forum that this has already been thought about, but I’m now documenting less so much the idea, and more the execution.

SafeOS Stage 1
For the first stage of this project, I envision making a replacement to existing software. SafeOS will be a Linux distribution, and will work like one.

Ultimately, this boils down to the login manager. By replacing this with a version of the Authenticator which then logs into SAFE Network, one can authenticate on every computer running SafeOS, simply by having a SAFE account.

The login manager should also give people the option to sign up to the SAFE Network as well, for those just starting out. If this ends up pervading the world (hey, engineers dream too you know), it’s not really a sensible supposition to believe a user will already have an account.

Taking this idea a step further. We already know that vaults can be used via NFS. This allows then the syncing of data everywhere - have the authenticator not only log in to the operating system itself, but also setting up the user /home to use the users vaults.

Additionally, the next step would then be to sync to another user vault, storing the users “installed” software and configurations. This would then be placed onto the system path, via manipulation of environment variables, symlinking, or some other how. The specifics of that don’t really matter at this stage.

I’d like to be able to demo stage 1 at SAFE DevCon 2019. I can’t guarantee that, but I’m gonna work my socks off to damn well try, and if I don’t manage it, I’ll still be talking about it.

SafeOS Stage 2
Stage 2 will build upon this. Application distribution will start to become more and more integrated with SAFE Network as time goes on.

One thing I can easily see, for example, is stock versions of existing software going live on the network, in an unencrypted, immutable store. This would work for both system software, and for user software. Either can be pinned, or left free to update, and a mutable store would exist, globally readable again, to contain a list of every version of the software, in order, with a link to the respective immutable store.

Assuming the version of a particular package is not pinned, the upgrade algorithm would work like this: for the operating system packages, the OS would detect an upgrade from the mutable store, download and install it, and in the case of the kernel, use hotswapping to update itself.

In the case of user software, I’d start to replace that central vault with a vault containing only configuration data, and a store somewhere detailing what packages, and what versions. These are then each independently mapped via NFS, as before. This reduces duplication on the network, and makes things a bit simpler.

SafeOS Stage 3
This one is relatively easy. Tighter integration of the SAFE browser into the OS, and greater usage of the desktop applications using SAFE.

Additionally, the OS shall now start to move away from the standard POSIX tools. There was talk today about starting to replace some of the GNU CoreUtils with equivalents that work with the SAFE Network. It’s worth raising here that they’d also need to work with ordinary files, in the event of things like external disks etc.

By integrating these tools, and potentially some new tools in line with the CoreUtils ethos, one can then begin to compose scripts to automate various jobs on the SAFE Network.

SafeOS Stage 4
At this point, I’d firstly like to start seriously paring down the size of the OS. Ideally, it should fit easily on a thumb drive as a bootable image, with plenty of space left over.

Additionally, we’d need to set up an installer at this point. The installer will connect to SAFE Network, and provide a way by which users may create an account on the network - fairly important for initial setup. This is, of course, optional, if users already have an account.

The installer should also install the necessary software for the machine to actually act as a node on the SAFE Network. This should not be done when in portable boot mode, however on a static install, it should, under the account of the machine owner. It would perhaps be best if this were provided as an option, as opposed to being required.

Possible future ideas
I can easily see this expanded further. An idea was floated today, by whom I forget and sincerely apologise, that the OS file system layer could be extended to offer various views into the data, as opposed to just a tree structure. For instance, “Give me all the photos taken in Spain in 2017”, would return a folder containing said photos. Windows Vista attempted something similar, to somewhat limited success.

@anon41664782 raised the idea of basing this on Redox as opposed to Linux. I personally think Linux makes more sense initially, due to the stability and maturity compared to Redox, but Redox makes for a good candidate as an alternate version. This will almost certainly be kernel independent, so SafeOS Redox would be a good second candidate, similar to Debian Hurd in nature.

@povilasb and @pierrechevalier83 both raised the possibility of basing the project on NixOS. Doing some light reading on NixOS and how it fits together, it seems to have a rather elegant concept of packages and configurations done through single files, which the package manager, Nix, then handles in terms of installation. I can easily see a case for this - users each have a configuration stored in an MD they own, and the login manager then installs this config upon login, removing it upon logout. This seems rather a lot more elegant than storing binaries and associated configurations somewhere.

@piluso floated the idea of basing SafeOS on Genode. Genode is an operating system framework based on components, which actually have SAFE in their roadmap, meaning this could get potentially submitted upstream, saving us (possibly only me :stuck_out_tongue:) from the cost of administration, while the modular architecture and framework nature of the project meaning a separate SafeOS based on Genode wouldn’t fragment the community. Genode can run multiple kernels via L4, including Linux and FreeBSD, meaning compatibility with existing software is high. The nature of Genode makes it good for security, which is nicely in line with the goals of SAFE itself.

Disclaimer
Not all of the above ideas are mine. Some are, some partly are, and others were proposed by various people I spoke to today. Thank you to everyone who I spoke to who inspired more ideas, more collaboration, and more development on this. I forget who mentioned what, and I’m really sorry for that, I spoke to a lot of people today.

If anyone else has any ideas for how this could be extended further, what should be integrated in, etc, or just wants to work with me, please please please reply. I’m really excited to work on this, and would absolutely love to work with the community, both with ideas, and with building it.

29 Likes

I’m actually a contributor on Redox as well, I worked on their kernel over summer getting part of the ACPI specification written.

The way I envision this, this should all be kernel independent. I think you’re right, there should be a Redox variant, officially supported, similarly to how Debian also has Debian Hurd. However, Redox is still in very early days, and there’s a lot not yet supported. Their network stack is still very young, for example, and doesn’t yet implement WiFi.

Therefore, I think it makes more sense to start with Linux as the core, for now, and possibly work with Redox to get them to a stage at which this can be ported over.

Very good thinking though, thanks for the idea. At some point that will probably go on the project roadmap going forwards. Whether or not it turns out to be stage 5, or something later, will depend on a number of factors, but certainly I’d love to see it happen.

4 Likes

David was originally talking about microkernels for an SafeOS…is that your thinking here?

I think that’s perhaps thought of at too low a level. I don’t really think this needs to be kernel specific. I could see some argument if, for example, one were able to run processes on SAFE Network, and I recall hearing chatter that may be possible in the future. Not sure if that was just a rumour.

Certainly though I’m a big proponant of the microkernel model, and if and when SafeOS Redox is developed, Redox is microkernel based, so by extension SafeOS would then also be.

2 Likes

So, the idea is basically to decouple the hardware from the users files and software? that offers some powerfull implications for user safety, never again will you be able to loose work due to hardware failure. I would be interested in if the idea of one day bringing processes themselves on to the network would be feasible? such that a user with inferior hardware is able to run a more complex process in a shorter time by utilising the nodes on the network to distribute the work?

2 Likes

Yeah thats what David said at the Devcon, emphasizing that Safe is not an Internet replacement. How do you meme that, weve been largely talking Safe in the light of new internet etc…but this is much more scifi, decoupling data from machine even though the data is on the machines.

I still reckon data centers are going to play a big role seeding this network.

2 Likes

This is really interesting point! So how to meme it? How to understand it? What analogies we have?

Like language is decoupled from human bodies, even though it needs humans to exist.

1 Like

Wouldn’t it be simplest just to call it a cloud operating system, since people already often have some of their files decoupled and already use that language?

Anyway, great idea, this is probably the thing I would most like to see built for the safe network.

We could call it a decentralised operating system, or DOS :joy:

15 Likes

OR decentralised disk operating system - DDOS :slight_smile:

11 Likes

Or even MS-DOS

20 char

16 Likes

Just wondering how these ideas never came up in the rebranding thread…

3 Likes

It is good to see this thread being posted. I think that a multi phase approach is exactly right. Even having you home directory as a SAFENetwork mount would give immediate value and would be very cool. Others would likely dive in to help after they have something to play with too.

3 Likes

Hahaha good one. You could probably get away with it too, by only using the full name yourself and letting the users naturally call it DOS for short.

2 Likes

I always thought that the best OS framework to build a SafeOS was Genode OS, and they are aware of the SafeNetwork:
https://genode.org/about/challenges

They launched their general purpose OS “Sculpt”:
https://genode.org/news/sculpt-for-early-adopters

When the main selling points of the SafeNetwork are privacy and security, I think a system like Genode would be the perfect marriage.

Here is the last talk of the team:

8 Likes

Another option I’ve heard floated by both povilas and Pierre here at maidsafe was NixOS. I’m not yet very clued up on either, other than knowing they exist, so I’ll need to do more reading first.

Thanks for posting the video @piluso, I just watched that Genode talk all the way through - that guy was very courageous. For those interested, to show how Genode works he starts with a laptop with a bare-bones static Genode (ROM) system - almost no drivers - and step-by-step configures it using VIM to edit XML configs and add the various drivers all the way to mounting a disk it can boot the custom config from!

I can imagine we could have Genode on SAFE by just writing two or three drivers:

  1. connection and authentication (maybe two drivers)
  2. virtual file system (mounts SAFE NFS as a virtual drive)

You would boot the static-Genode, it would offer to boot from SAFE. You log-into your account and give it access to your Genode drive, it mounts that and boots the soft part of the system from it (config, drivers, window manger, package manager and applications).

Those guys even have a roadmap :wink:

Whatever route you take @connor good luck. Its great to see you ready to work on this. :slight_smile:

8 Likes

Thank you all for the words of encouragement and wonderful ideas, it’s great that everyone’s so enthusiastic about my idea.

I’ve done some preliminary digging into both NixOS and Genode while on my train back from Stansted Airport, which I’ve integrated above. Both seem like really nifty ideas as bases for SafeOS. I’ll dig a little deeper into both when I have more time to, and integrate all of the information into that ever-growing OP up there.

If I get any details wrong, by the way, and this is especially in reference to NixOS and Geode, please don’t hesitate to correct me. This is all technology I’ve not really toyed with, and I’m learning on the fly, so it’s probable I’ll miss some crucial detail.

5 Likes

The c2 wiki has some interesting articles on operating systems (and many other things). Perhaps some useful ideas there, or at least some fun articles, here’s a couple of them that leads into strange rabbit holes

http://wiki.c2.com/?LanguagesAreOperatingSystems
http://wiki.c2.com/?OperatingSystem
http://wiki.c2.com/?PurelyFunctionalOperatingSystem
http://wiki.c2.com/?KillerOperatingSystem
http://wiki.c2.com/?PerfectSystem

1 Like

I think Linux as a starting point is the correct thing to do as well, @anon41664782. I’ll not rule out the other options though, and will definitely make a note of them.

As I’ve said before, this shouldn’t need to be kernel dependent. By that logic, one can integrate this with other bases later on. Perhaps post stable, I’m not sure yet :stuck_out_tongue: