SAFE CLI - High Level Design Document

Summary

The SAFE CLI (Command Line Interface) provides all the tools necessary to interact with the SAFE Network, including storing and browsing data of any kind, following links that are contained in the data and using their addresses on the network.

Conventions

  • The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.

Motivation

A CLI is a bare-bones implementation and abstraction of user facing APIs. It is largely platform and system agnostic and can provide a basis, via Rust library interfaces it would be founded upon, for many other languages to interact with the SAFE Network in a clean and useful manner. Here, we want to provide tools for developers to easily work with the SAFE Network, combining these commands to perform complex network tasks, while leaving the commands themselves simple and easy to follow, masking as much of the underlying network complexity as possible.

The end goal of this CLI is to provide SAFE end users with an abstraction which allows them to read data objects using their addresses (XOR names) as a frame of reference. Users should be able to browse and retrieve, mutate or add new versions to data by following links. As well as manage accounts, key pairs and safecoin wallets & transactions.

Thus using the CLI users have access to any type of operation that can be made on SAFE Network data, allowing them to also use it for automated scripts in a piped chain of commands. All without the overhead of GUIs and their platform dependent complexities, which should allow for a CLI to be implemented in a robust, well tested fashion much faster.

Assumptions

This document doesn’t cover how data is represented on the network here, though RDF is assumed. Equally, it doesn’t include details about supporting RDF data management, either from the CLI or from the API being exposed, though these may come later (to be decided as we have APIs exposed from SCL).

Detailed design

This document describes the proposed $ safe CLI by going through several end-user use cases and showing how the CLI can be used for each of them. This is not meant to be exhaustive but to give enough examples to demonstrate the type of operations to be supported by the CLI app and its API.

The API layer which is to be implemented for this CLI app SHALL also be publicly exposed from this crate, thus any Rust client app can make use of the same type of operations that the CLI exposes, with the exact same type of abstractions to manage the Network data. The SAFE CLI is simply a thin layer on top of this API providing the command line UI.

All commands need a target location (the XOR name) of the object the current operation will be applied to. E.g. if the target location is the XOR name of a Files Container ‘A’, then executing a command to add mypath/subpath/myfile.txt file will result in “mutating” the ‘A’ container to add a link to the location of the newly uploaded myfile.txt file, the link added to ‘A’ container will be named mypath/subpath/myfile.txt.

An additional interactive mode of operation MAY be provided which allows the user to run commands within a shell where the target location will be kept in memory during that session. It can be set and changed by the user in an analogous way as how the current directory is kept in any console session. In the interactive shell the --target argument becomes optional (see below for details of --target).

When executing any command that is creating/mutating data on the network, there is a cost incurred. Such a cost can be paid with a safecoin Wallet. Before we dive into how safecoin wallets are managed and used further below in the Wallet section, let’s just assume there is a default safecoin Wallet set in the SAFE account with enough balance to pay for the operations performed by the exemplified commands.

Synopsis

$ safe <command> <subcommand> [options] [parameters]

If the CLI is executed with no arguments it will then run the $ safe CLI interactive shell:

$ safe
Welcome to SAFE CLI interactive shell!
Type "help" for more information about supported commands.
Type "exit" to exit this shell. Enjoy it!

 safe >>

Refer to the CLI Interactive Shell section further below for more details.

Global options

--target
Specifies the target location where the current command will be applied to. This is optional and it behaves according to the context as follows:

  • When not provided as argument in a command, then the CLI will take the content read from stdin as the target location for the current command. This is what allows the CLI to be used in a piped chain of commands. If the stdin is empty an error is thrown.
  • If it’s not provided within the interactive shell, it will use the current target location set in the session, otherwise an error will be thrown.

--root
Special alias for --target Root. In any command where the target is meant to be the Root Container, --root can be used instead of something like --target safe://<Root Container XOR-URL> which is also valid as long as the XOR-URL of the Root Container is already known by the user.

--version
Displays version and release information

--help
Provides contextual help on commands and subcommands

--output <type>
Sets the format for the output, default is plain text which is specially useful for piped chain of commands. Another type which shows tables with human readable messages SHALL be also supported, but other formats MAY be also supported like JSON, YAML, CSV, etc.

-h
Alias to --output human-readable. This is set as the default in the interactive shell.

--verbose
Increases logging verbosity

--query <query>
Allows to apply filters to the output, perhaps using http://jmespath.org or SPARQL

--dry-run
To test the command without effectively applying the operation

Login / Authorise

$ safe auth

This command simply sends an authorisation request to the Authenticator available, e.g. the safe_auth CLI daemon, and it then stores the authorisation response (credentials) at ~/.safe/credentials, or if the user prefers, in a environment variable. Any subsequent CLI command will read the ~/.safe/credentials file, or environment variable, to obtain the credentials and connect to the network for the corresponding operation.

In the case of invoking this command within the CLI interactive shell, the authorisation response (credentials) will be kept in memory only for the current session, and any other command executed within that session will be able to get the credentials from memory.

Root and Named Containers

It is herein also proposed to remove the hard-coded list of default containers, i.e. the default containers SHALL NOT be automatically created when an account is created, not even the Root Container. The user is free to create them as needed on any account with the corresponding commands/API. To illustrate, this is to allow users to not spend safecoins on creating these containers on an account which may be used only for storing safecoins (just as an example).

Thus, the CLI allows the user to create the Root Container which is automatically linked from the SAFE account packet, or any number of Named Containers which are linked from another container with a custom name provided by the user.

By default containers will create unpublished AppendOnlyData. Users will be able to opt in to create (unpublished) MutableData via a --non-versioned command argument. Likewise, by default files will be created as unpublished ImmutableData.

They will also be able to --publish their unpublished data, this will warn the user of the cost to copy data to a new AppendOnlyData or ImmutableData.

$ safe container

Get help for safe container subcommands:

$ safe container --help

Create the Root Container:

$ safe container create --root -h
Root Container successfully created at: safe://<Root Container XOR-URL>

We can create a Root’s child Container named music:

$ safe container create --root --name music -h
'music' Container created at: safe://<'music' Container XOR-URL>

We can create a Root’s child Container named photos and publish that thus:

$ safe container create --root --name photos --publish -h
'photos' Container created and published at: safe://<'photos' Container XOR-URL>

We can create a Root’s child non-versioned (aka: MutableData) Container named my-secret-db thus:

$ safe container create --root --name my-secret-db --non-versioned -h
Non-versioned 'my-secret-db' Container created at: safe://<'music' Container XOR-URL>

We can also create a Container named my-first-album as a child of music Container. Note we are not passing -h in this example to simply get the new XOR-URL (this command could be piped with any other one which expects a XOR-URL in its stdin):

$ safe container create --name my-first-album --target safe://<'music' Container XOR-URL>
safe://<'my-first-album' Container XOR-URL>

Instead of creating a container as a child of another, we can create it first with no name and at a random location on the network, and then add it as a child to the Root Container, e.g. with name pictures:

$ safe container create -h
Container created at: safe://<XOR-URL>

$ safe container add --root --name pictures --link safe://<XOR-URL> -h
Container linked from Root Container with name 'pictures' to safe://<XOR-URL>

Note we are inferring the link type from the object being linked, and that’s used for the predicate in the graph being generated in the parent Container. E.g. if the link provided refers to a file the container add command would add a graph which predicate is a safe:File instead of safe:Container (there could be different subtypes as well, e.g. published vs unpublished File/Container).

If the target is an un published non-versioned Container (i.e. a MutableData), we can also edit a specific entry:

$ safe container edit --target safe://<MutableData XOR-URL> --key key1 --value new-value

Reading content

$ safe cat

$ safe cat --help

Fetching any object from the network and streaming out the content, eventually this may need to support different serialisation formats if the content is an RDF doc:

$ safe cat safe://<XOR-URL or PNS-URL>

List the links (children) contained in the Root Container:

$ safe cat --root -h
+-------------------------------------------------------------------------+
| Children of: Root                                                       |
+--------------------+-------------+--------------------------------------+
| Name               | Predicate/s | Link                                 |
+--------------------+-------------+--------------------------------------+
| 'music'            | Container   | <XOR-URL of 'music' Container>       |
+--------------------+-------------+--------------------------------------+
| 'pictures'         | Container   | <XOR-URL of 'pictures' Container>    |
+--------------------+-------------+--------------------------------------+

But we can also list the links (children) contained in the ‘music’ Container:

$ safe cat safe://<'music' Container XOR-URL> -h
+-------------------------------------------------------------------------+
| Children of: safe://<'music' Container XOR-URL>                         |
+--------------------+-------------+--------------------------------------+
| Name               | Predicate/s | Link                                 |
+--------------------+-------------+--------------------------------------+
| 'my-first-album'   | Container   | <'my-first-album' Container XOR-URL> |
+--------------------+-------------+--------------------------------------+

If the URL targets a (unpublished) non-versioned Container (i.e. a MutableData), it’s possible to fetch specific entries (we can also support SPARQL here):

$ safe cat safe://<MutableData Container XOR-URL> --key key1 -h
+---------------------------------------------------------+
| Entries of: safe://<MutableData XOR-URL>                |
+----------------+---------+------------------------------+
| Key            | Version | Value                        |
+----------------+---------+------------------------------+
| 'key1'         |       3 | 'just a value string'        |
+----------------+---------+------------------------------+

And if the URL targets a published (therefore versioned) Container, it’s possible to fetch an specific version of it (which by default would be fetching the latest version):

$ safe cat safe://<published Container XOR-URL> --version 234 -h
+--------------------------------------------------------------+
| Children of: safe://<published Container XOR-URL>            |
+--------------------+-------------+---------------------------+
| Name               | Predicate/s | Link                      |
+--------------------+-------------+---------------------------+
| 'an-old-item'      | Container   | <some old XOR-URL>        |
+--------------------+-------------+---------------------------+

Files & Files Containers

$ safe files put

Upload all files and subfolders found within the ./to-upload/ local directory, recursively, onto a Files Container on the Network obtaining the XOR-URL of the newly created container (we obtain the XOR-URL because we don’t pass -h):

$ safe files put --source ./to-upload/ --recursive
safe://<XOR-URL>

Files (ImmutableData) will also default to unpublished in the first instance and will require use of a --publish flag to make them publicly accessible.

$ safe files sync

Given the popularity of rsync command, it’s proposed to have the SAFE CLI subcommand for uploading files and folders to support a subset of the functionality provided by rsync. The reasoning being that users knowing how to use rsync can easily start using the CLI and the SAFE Network, potentially making it also easy to integrate existing automated systems which are currently making use of rsync, or perhaps integrating eventually having the rsync command to use the CLI as the gateway for the safe:// protocol, similar to how it supports others like protocols like ssh, e.g. users in the future could potentially be able to do $ rsync ./myfoler/ safe://mypublicname/myfolder/.

In addition to this, rsync seems to cover most/all of the use cases for copying files and directories across networks. The subset of features supported can be gradually expanded with more features. It’s worth noting though, that opposed to rsync the CLI requires to explicitly define which are the source and target arguments.

Sync up all files and subfolders found within the ./to-upload/ local directory, recursively, with a Files Container on the Network (let’s pass -h to get a nice description of the files synced instead of the targeted XOR-URL):

$ safe files sync --source ./to-upload/ --target safe://<XOR-URL> --recursive -h
+ index.html             safe://<'index.html' file XOR-URL>
+ myfolder/notes.txt     safe://<'notes.txt' file XOR-URL>
+ img.jpeg               safe://<'img.jpeg' file XOR-URL>

Sent 14.71M bytes, 3.27M bytes/sec, 0.000023 safecoins spent

The + sign means that the files were added as new items into the target container.

Sync up all files and subfolders found within the ./other-to-upload/ local directory, recursively, and delete those files which are found at the target Container that are not part of the uploaded set of files, e.g. let’s assume that we only have a file named img.jpeg within ./other-to-upload/:

$ safe files sync --source ./other-to-upload/ --target safe://<XOR-URL> --recursive --delete -h
- index.html     
- myfolder/notes.txt
* img.jpeg               safe://<'img.jpeg' file XOR-URL>

Sent 6.12M bytes, 2.87M bytes/sec, 0.000008 safecoins spent

The - and * signs mean that the files were removed and updated respectively.

$ safe files add

Add the local ./folder/myfile.txt file to a Container:

$ safe files add --source ./folder/myfile.txt --target safe://<XOR-URL> -h
+ myfile.txt             safe://<'myfile.txt' file XOR-URL>

Sent 587K bytes, 5.9M bytes/sec, 0.0000003 safecoins spent

Add the local ./folder/myotherfile.txt file to a Container but with name published-file.txt:

$ safe files add --source ./folder/myotherfile.txt --target safe://<XOR-URL> --name published-file.txt -h
+ published-file.txt     safe://<'published-file.txt' file XOR-URL>

Sent 342K bytes, 5.2M bytes/sec, 0.0000005 safecoins spent

Add a file which was already uploaded to a Container with name linked-file.txt:

$ safe files add --link safe://<XOR-URL> --target safe://<XOR-URL> --name linked-file.txt -h
+ linked-file.txt        safe://<'linked-file.txt' file XOR-URL>

Sent 0 bytes, 0.0000005 safecoins spent

Again, we can use cat command to see the content of a Files Container:

$ safe cat safe://<Files Container XOR-URL> -h
+---------------------------------------------------------------------------------+
| Files of: safe://<Files Container XOR-URL>                                      |
+----------------------+-------------+--------------------------------------------+
| Name                 | Predicate/s | Link                                       |
+----------------------+-------------+--------------------------------------------+
| 'img.jpeg'           | FileItem    | safe://<'img.jpeg' file XOR-URL>           |
+----------------------+-------------+--------------------------------------------+
| 'myfile.txt'         | FileItem    | safe://<'myfile.txt' file XOR-URL>         |
+----------------------+-------------+--------------------------------------------+
| 'published-file.txt' | FileItem    | safe://<'published-file.txt' file XOR-URL> |
+----------------------+-------------+--------------------------------------------+
| 'linked-file.txt'    | FileItem    | <'img.jpeg' file XOR-URL>                  |
+----------------------+-------------+--------------------------------------------+

Or indeed, the contents of a given file:

$ safe cat safe://<Files Container XOR-URL>/myfile.txt
hello world

Public Names

Get help for pns subcommands:

$ safe pns --help

$ safe pns create

Create the Resolvable map for several PNS domains:

$ safe pns create --name mypublicname mypublicname onemorename
mypublicname            safe://<1st XOR-URL>
secondpublicname        safe://<2nd XOR-URL>
onemorename             safe://<3rd XOR-URL>

$ safe pns add

Add mysubname as a sub name to the mypublicname public name and automatically creates a container for the data belonging to the new subname:

$ safe pns add --name mysubname --target mypublicname -h
New sub name 'mysubname' added to 'mypublicname' and linked container created successfully at: safe://<Files Container XOR-URL>

Or alternatively we can add myothersubname as a sub name to the mypublicname public name, but linking it to a container which was previously created:

$ safe pns add --name mysubname --target mypublicname --link safe://<Files Container XOR-URL> -h
New sub name 'myothersubname' added to 'mypublicname'

We can also set what’s the link to follow by default when a client is resolving a PNS URL with no sub name, e.g. safe://mypublicname, in such a case we can set to resolve to a specific location:

$ safe pns add --default --link safe://<XOR-URL or PNS-URL> --target mypublicname -h
Default sub name set to resolve to: safe://<XOR-URL or PNS-URL>

We can also take the value for the --link argument from stdin if we provide a --target argument (otherwise, if no --target is provided then the value from stdin is by default used as the --target value). Let’s do that, and let’s add such a link as a sub name with a name but also as the default link:

$ echo safe://<XOR-URL> | safe pns add --default --name anothersubaname --target mypublicname -h
New sub name 'anothersubname' added to 'mypublicname', and set as the default too, to resolve to: safe://<XOR-URL>

Thus in the example above, the same location will be resolved when fetching either safe://mypublicname or safe://anothersubname.mypublicname.

$ safe pns remove

If needed we can also remove an existing sub name from a public name:

$ safe pns remove --name mysubname --target mypublicname

Or, perhaps we need to remove the default sub name from a public name:

$ safe pns remove --default --target mypublicname

Piping commands

$ safe pns | safe files

When the --target is not provided, the command will assume the target location to be read from the stdin, this allows the user to pipe commands. In some commands like safe container or safe pns, if the --target argument is provided but not the --link then the stdin will be used as the value for the --link.

A nice example of this is by looking at how we can chain commands to upload all the local files of a website, publish it at safe://mypublicname on the SAFE Network, and finally fetch its content using the PNS-URL:

$ safe pns create --name mypublicname && safe files put --source ./my-website/ --recursive | safe pns add --default --target mypublicname && safe cat safe://mypublicname
<html>
<h1>
BOOM!
</h1>
</html>

Let’s assume we have three music files in the local ./music-files/ directory, we can upload them onto the ‘music’ Named Container that is linked from the Root Container as follows:

$ safe container follow --root --name music | safe files sync --source ./music-files/ -h | wc -l
3

CLI Interactive Shell

$ safe

Set current target location on the network to be the Root Container:

 safe >> locate --root

Obviously, we can set the current target location using a XOR-URL or a PNS-URL:

 safe >> locate safe://<XOR-URL or PNS-URL>

The --locate is a global argument supported in the interactive shell and only valid with commands which create new objects on the Network. It requests the CLI to automatically set the current target location in the session to be the location of the new object being created by the command. E.g. the following command will create the Root Container and set the current target location to be the XOR name of the Root container just created:

 safe >> container create --root --locate
Root Container successfully created at: safe://<Root Container XOR-URL>
Current target location was set to: safe://<Root Container XOR-URL>
 safe >>

Change the current target location by following a link named music that is found in current target location:

 safe >> container follow --name music

Keys

Key management allows users to create new sign/encryption key pairs that can be used for different type of operations, like choosing which sign key to use for uploading files (and therefore paying for the storage used), or signing a message posted on some social application when an Key is linked with a public profile (e.g. a WebID/SAFE-ID), or even for encrypting messages that are privately sent to another party so it can verify the authenticity of the sender.

Users can record Keys in a Container, having friendly names to refer to them, but they can also be created as throw away keys which are not linked from any container or other data on the network.

Note that even that the key pair is generated by the CLI, Keys don’t hold the secret key but just the public key, and optionally can have a safecoin balance attached to it, thus Keys can also be used for safecoin transactions (see the wallet section below for more details).

$ safe keys

Get help for keys subcommands:

$ safe keys --help

Create a new Key linking it from a Container, the name of the link in this case would be the public key itself. Note the key pair is generated locally by the CLI, and the secret key is not stored as part of the Key, or anywhere on the network:

$ safe keys create --target safe://<a Container XOR-URL> -h
New Key created at: safe://<Key XOR-URL>, and linked from safe://<a Container XOR-URL>.
Key pair generated is: <key pair info>

You can also optionally use nicknames for easy reference when creating Keys:

$ safe keys create --name business-time --target safe://<a Container XOR-URL> -h
New Key created at: safe://<Key XOR-URL>, and linked with name 'business-time' from safe://<a Container XOR-URL>.
Key pair generated is: <key pair info>

Or create throw away Key:

$ safe keys create --anon -h
New Key created at: safe://<Key XOR-URL>. This was not linked from any container.
Key pair generated is: <key pair info>

We can create an asymmetric key pair and create a Key to enable a friend to set up a SAFE account with it by preloading it with safecoins, assuming the default Wallet linked from the SAFE account has enough balance to transfer the desired preload amount:

$ safe keys create --preload 55 --anon -h
New Key created at safe://<Key XOR-URL>, preloaded with 55 safecoins, which can be used for account creation/generation
Key pair generated is: <key pair info>
55 safecoins spent

An anonymous Key that was previously created can also be linked from a Container, using its public key or a nickname for easy reference. Let’s add an existing Key using a nickname to an existing Container:

$ safe keys add --name mykeys --target safe://<a Container XOR-URL> --link safe://<Key XOR-URL> -h
Key at safe://<Key XOR-URL> was linked with name 'mykeys' from safe://<a Container XOR-URL>

It is also possible that a friend wants you to create a Key for him but having him to provide you with the public key for it, rather than you generating the asymmetric key pair, in this case we can simply create the Key passing the public key to be used:

$ safe keys create --pk <BLS pk> --preload 3 -h
safe://<Key XOR-URL> was created for public key <BLS pk>, preloaded with 3 safecoins, which can be used for account creation/generation
No key pair was generated.
3 safecoins spent

Combining with other commands

It may be that you want files created to be owned by a different Key than the default obtained from the account, and perhaps even pay with a different wallet than the default Wallet set in the account, you can do that by using the --owner flag to specify the key for ownership of the data, and --wallet for specifying a specific wallet to pay for the costs:

$ safe files add --source ./folder/myfile.txt --target safe://<XOR-URL> --owner safe://somekey --wallet safe://somewallet -h
+ myfile.txt             safe://<'myfile.txt' file XOR-URL>

Sent 587K bytes, 5.9M bytes/sec, 0.0000003 safecoins spent by safe://somewallet
Files uploaded are owned by safe://somekey

Wallet

A Wallet is a specific type of Container holding a set of spendable safecoin balances. A Wallet effectively contains links to Keys which have safecoin balances attached to them, but the Wallet also can store the secret keys needed to spend them. Wallets are stored encrypted and only accessible to the owner by default.

$ safe wallet

Get help for wallet subcommands:

$ safe wallet --help

Let’s create a Wallet:

$ safe wallet create -h
Wallet created at safe://<Wallet XOR-URL>

Check your balance:

$ safe wallet balance --target safe://<some wallet XOR-URL> -h
safe://<some wallet XOR-URL> owns 99 safecoins

Let’s add a new spendable balance to the Wallet with a friendly name we can later use to explicitly refer to it in a safecoin transaction. The following command would link an existing Key to a Wallet:

$ safe wallet add --name weekend-leisure --target safe://<Wallet XOR-URL> --link safe://<Key XOR-URL>
Enter secret key corresponding to public key at safe://<Key XOR-URL>:
New spendable balance generated with name 'weekend-leisure' in wallet located at safe://<Wallet XOR-URL>

Let’s transfer safecoins from Wallet using any available spendable balance to an specific named spendable balance of another Wallet:

$ safe wallet transfer --amount 12 --from safe://<wallet XOR-URL> --to safe://<other wallet XOR-URL>/weekend-leisure

Note that a Key can also be used as the source (--from) or destination (--to) of a safecoin transfer, but since Keys don’t hold the secret key to spend the balance, when they are used as the source of a transfer the CLI will prompt the user to enter the secret key to submit the transaction:

$ safe wallet transfer --amount 55 --from safe://<Key XOR-URL> --to safe://<destination Wallet or Key XOR-URL>
Enter the BLS-sk to spend from safe://<Key XOR-URL>:
55 safecoins transferred to safe://<destination Wallet or Key XOR-URL>

Sweeping all available safecoins:

$ safe wallet sweep --from safe://<Wallet or Key XOR-URL> --to safe://<Wallet or Key XOR-URL>

Generate transaction target for a specific amount of safecoins. The payee needs to only click the link to generate a transaction to confirm for the correct amount:

$ safe wallet receive --amount 12.23324 --on safe://<wallet or key XOR-URL> --description "Art Coffe Bar"
safe://<wallet or key XOR-URL>?amount=12.23324&desc=Art%20Coffe%20Bar

The generated URL can be opened by a Wallet-compatible application, like the SAFE Browser, which can render an invoice including a QR code and payment description.

SAFE Ids (WebIDs)

This manages a SAFE-Id profile container. All SAFE-Ids are stored there unless otherwise specified.

SAFE-Ids are based upon the WebID spec, but may have other changes (beyond being a safe:// URL, which means they cannot strictly be WebIDs by the specification).

$ safe safeid

Get help for safeid subcommands:

$ safe safeid --help

Create a SafeId:

$ safe safeid create

# return safe://<SAFE-ID XOR-URL>

Create a SafeId with name:

$ safe safeid create --name josh

# return safe://<SAFE-ID XOR-URL>

Various standard SAFE-Id fields can be easily populated in this flag based manner, eg: --name, --email (Do we want this on safe? probably), --nickname, --surname, --website … (more?)

Update a SafeId field:

$ safe safeid update --name gabriel --target safe://<SAFE-ID XOR-URL>

# return safe://<SAFE-ID XOR-URL>

This can be piped to other commands to link to a PNS name e.g.

Drawbacks

None identified so far.

Alternatives

Getting rid of the predefined and hard-coded list of Default Containers (as detailed in Root and Named Containers section) can be considered optional, and an additional parameter can be supported which allows the user to target each of the default containers by name. However, unless there are strong arguments to automatically create these Containers by default when creating an account, it’s strongly RECOMMENDED to effectively remove them.

Unresolved questions

  • There needs to be more details (to be added later) covering aspects of encryption keys and how encrypted content is to be handled.
42 Likes

Maybe not for in this RFC and to be added by someone else later, but autocompletion would be handy. Not only that you get all the static start arguments, but also if the autocomplete options come from the SAFE Network.

For example safe cat safe://first_part/<TAB>
or safe ls safe://first_part/<TAB>
Here you would get a list of all ‘subdirs’ and ‘subfiles’ (without something like FUSE in between) to choose from.

An example of such a dynamic autocompletion: ssh user@<TAB>

https://www.joshmcguigan.com/blog/shell-completions-pure-rust/

I’ve not worked with it much yet, but I assume that Bash autocompletion with Windows 10 is also possible with Windows Subsytem for Linux (WSL).

Also some short alias for safe:// would be handy, like s/ for example.

12 Likes

:+1: , and 20 characters

3 Likes

Wow, very thorough well thought out and comprehensive. A lot to take in, so only a handful of thoughts from me atm [hears sigh of relief :wink:]

I can’t see people typing in xor URIs, so with or without the safe:// prefix is moot I think. But I don’t recall how we differentiate an XOR URI from one using PNS, so perhaps to differentiate we could just use safe:// with PNS URIs? I think if completion can be used to select XOR URIs (in particular) it would help a lot, although I’m not sure this is feasible (because how would you choose which one you want?).

Re safe auth, I think it didn’t address ways to provide stored password/secret for headless operation - which seems a major use case.

If I understand, you will be writing a Rust API specifically to support this CLI, and I assume exposing it to languages such as JavaScript and Python? So scripts and programs could be created in these and other languages as an alternative to using the SAFE App APIs.

Compatibility. Before embarking on implementation, I’d hope to see the underlying operations tied down in a definitive way. For example will this be compatible with SAFE NFS, and if so, then can the behaviour of say uploading a directory tree be specified so that the same operation in WHM and CLI and SAFE Drive etc can all result in the same data structure (ie in the containers created and the entries in them). Also (yes this area is a bit of a bugbear for me :slight_smile: ) can we have rules or at least recommendations for how to choose between creating a directory as a path within a key, versus creating it as a container? If we will get this, please can you speculate on when (in terms of priorities or task scheduling rather than dates)?! It seems a priority wrt finalising APIs and their implementations, as well as the APIs using them, and will help app developers.

Great stuff. That is quite a post, so congratulations on a thorough understandable job. :clap:

10 Likes

That’s the ideal eventually. The CLI is hopefully the prototype for a bit of a streamlined API, removing some of the lower level steps we have to do now. So eventually everything CLI should be possible with similar commands in all languages + the DOM.

The same operations wherever they are should result in the same data setup, yep.

That’s harder to say as both NFS and WHM precede RDF on the network. What’s likely is that we’ll work towards the same sort of NFS concept, but it will be an RDF schema and not just a SAFE setup we assume. But whatever we arrive at, should be compatible across the board (if not for using the same APIs, for the fact that it’s RDF).

Can you clarify what you mean here @happybeing?

I think this might be covered by the PNS RFC, but I’m not entirely sure what you’re after.

6 Likes

I really like this idea, I think we should try to get it in there eventually when the CLI is stable enough

I agree, with or without is not that important, in fact we can have the CLI to accept both, although I’d prefer to save the “without safe://” to be for raw XOR names, developers will be using raw XOR names quite often when testing their apps, so using the XOR name instead of a XOR/PNS URL could be also supported.

The resolver (fetch) function assumes it’s a XOR-URL if it doesn’t have a subname (you know safe://subname.pubname), and it tries to decode it as such (with CID at the moment), if that fails it then fall backs to assume is a public name and try to fetch it.

7 Likes

What about a potentential

safe cp -r <pns_url> <local path>
(or safe cp <local_path> <pns_url>)

Can that be for example:
safe cp -r safe://draw/test_site /home/draw/

5 Likes

Reading your reply @joshuef I think my question may be superfluous.

Are you saying the underlying Rust CLI will replace the APIs we are currently using, rather than provide an alternative alongside the SAFE App API. And that SAFE NFS may be supported, but on top of the new CLI Rust primitives?

Or…? Can you give an overview of the different APIs and how they are likely to change / not change with the addition of SAFE CLI? I think it will help to understand the overall plan.

At the moment I’m not sure how this fits together, what the plans are for RDF etc. the existing APIs, etc. I think you guys must have an overall picture of how this might all come together even if it is not definite, but that is not clear from the bits and pieces that are public.

3 Likes

yeah, it could be, we were trying to have all those type of operations with safe sync, copying the same form of rsync, so if rsync doesn’t have one to copy from remote to local we have to have something like cp, I don’t know if it has it or not. I think if we stick to rsync cmds we can have users to use this without learning too much and we could eventually even create an adapter for rsync ?? (thinking out loud, not sure if it’s possible)

4 Likes

That would be nearly exactly the usual terminal syntax - I like the idea a lot :slightly_smiling_face:

Easy and intuitive to just grab the data you uploaded through a different source :blush:

7 Likes

Yes rsync or scp (2 chars less) is a more logical name, but on the other hand the command already begins with ‘safe’. Also an alias is quickly made.
With adapter for rsync do you mean expanding the existing rsync command to support safe://…, something like '-e rsh (see quote from man rsync below) or something else?

-e, --rsh=COMMAND
This option allows you to choose an alternative remote shell program to use for communication between the local and remote copies of rsync. Typically, rsync is configured to use ssh by default, but you may prefer to use rsh on a local network

3 Likes

This should provide an alternative to SCL API, and one step further up in the stack abstracting concepts, so if you upload a folder with safe put, it ideally creates a Files Container using an RDF schema that any other Files Container API (including NFS API) can read and comply with. Now, will the NFS API be there in SCL once we have this Files Container API in the CLI, not sure, I personally (this is very personal not sure what @joshuef @nbaksalyar or others think) wouldn’t maintain it, but if there are benefits in keeping that API I guess it could be maintained and make it follow the same RDF schema internally (the NFS API wouldn’t need to change in such a case but just its internals).

4 Likes

Yes, if we find out it’s worthy and makes sense. I imagine that if we have that, existing backend systems could migrate easily??

3 Likes

You also have wget, curl (and probably others) that could give inspiration how the arguments could work.

These cmdline tools are written in c. So starting with good c bindings and a couple of rather easy examples.

3 Likes

Not least, git! :slight_smile:

7 Likes

yes, it’s worth mentioning we were indeed looking at these guys, as well as AWS and Azure CLIs (as per auth we also looked at how npm and cargo publish work, etc.), but obviously we will be shaping things out more as we move forward in the implementation. And just a comment from me as a community/SAFE follower, I’m as excited as you (or more, I challenge you!) about this :blush:

9 Likes

Great to see the CLI progressing along. This will be very useful.

4 Likes

Can someone suggest to me how to vocalise PNS in my head, because honestly I just keep reading penis and it’s pretty ridiculous (I’m sure I’m not the only one with this problem). Pee En Ess pronunciation is not gonna happen, it’s too easy to combine the sounds together (and I’m too lazy to enunciate it that way in my head).

11 Likes

There was a discussion over in the dev forum about it and I expected it to change, but alias I think we still have some schoolyard thinking and it remained as penis, when will we get VJ (Cartmans term for vagina)

3 Likes

I’m glad I’m not the only one :joy: I was kind of hoping we weren’t stuck with it just because I can’t imagine everyone in the world referring to what used to be DNS as PNS and snickering to themselves. I personally liked Decentralized Naming System but understand the change. Def follow that dev forum thread there were a few good alternative suggestions.

Edit: I see you have it hyperlinked in your post. I think Josh is just trolling us :wink:

On a more serious note, the SAFE CLI high level design is a brilliant effort. Great work @maidsafe

6 Likes