Safe dog metadata

What is the intent of safe dog?.. metadata obviously but is that for all files or just the FileContainer??

I had in mind it listed all files within a domain but today looking more closely it seems to be centered on FilesContainer and at odds with safe tree and safe files ls

for example, dog gives

$ safe dog safe://hello
Native data type: PublishedSeqAppendOnlyData
Version: 0
Type tag: 1100
XOR name: 0x5f9078f885083b7022102f7b6047fa1f29aab7573d63f6b6676b10ee581a16f7
XOR-URL: safe://hnyynyix3y68aowrdshbnnyzzsan89ex1ukizkh6s87isc7itb51ademxqbnc?v=0

Resolved using NRS Map:
PublicName: "hello"
Container XOR-URL: safe://hnyydyw3utxujj7ecmh3aofrapuxopb1f8krezb8wrum3fm4m1ebdud33rbqh
Native data type: PublishedSeqAppendOnlyData
Type tag: 1500
XOR name: 0x3338be694f50c5f338814986cdf0686453a888b84f424d792af4b9202398f392
Version: 3
+-------------------+----------------------+----------------------+--------------------------------------------------------------------------+
| NRS name/subname  | Created              | Modified             | Link                                                                     |
+-------------------+----------------------+----------------------+--------------------------------------------------------------------------+
| hello             | 2020-04-01T18:28:58Z | 2020-04-01T18:28:58Z | safe://hnyynyix3y68aowrdshbnnyzzsan89ex1ukizkh6s87isc7itb51ademxqbnc?v=0 |
+-------------------+----------------------+----------------------+--------------------------------------------------------------------------+
| hello/style.hello | 2020-04-01T16:11:47Z | 2020-04-01T16:11:47Z | safe://hbhyyyd446wnju9gouch43qfaj1b9eoss4wadnm8yffdnihwu3iuipsd7r        |
+-------------------+----------------------+----------------------+--------------------------------------------------------------------------+

safe tree is nice as simple as

$ safe files tree safe://hello
safe://hello
├── favicon.ico
├── index.html
└── style.css

safe files ls then more useful detail

$ safe files ls safe://hello
Files of FilesContainer (version 0) at "safe://hello":
Files: 3   Size: 12866   Total Files: 3   Total Size: 12866
SIZE   CREATED               MODIFIED              NAME 
10134  2020-04-01T18:28:09Z  2020-04-01T18:28:09Z  favicon.ico 
2417   2020-04-01T18:28:09Z  2020-04-01T18:28:09Z  index.html 
315    2020-04-01T18:28:09Z  2020-04-01T18:28:09Z  style.css 

what is the hello/style.hello that dog found above?

safe dog itself gives error querying that directly as

$ safe dog safe://hello/style.hello
[2020-04-04T15:12:37Z ERROR safe] safe-cli error: [Error] ContentError - No data found for path "/style.hello/" on the FilesContainer at "safe://hnyynyix3y68aowrdshbnnyzzsan89ex1ukizkh6s87isc7itb51ademxqbnc/style.hello?v=0"

?

The commands mentioned in your post are:

  1. safe dog safe://hello

  2. safe files tree safe://hello

  3. safe files ls safe://hello

  4. safe dog safe://hello/style.hello

Looking at the results, it seems that the network has been switched between commands 1 and 2 because the set of files is not the same in the results.

File style.hello doesn’t exist anymore since command 2 and you expect it in command 4. Can you try safe dog safe://hello/index.html instead?

EDIT:
In a local network I get valid results for a safe dog command on a file:

$ ./safe dog safe://test/index.html
Native data type: ImmutableData (published)
XOR name: 0x05a108cd84235e4846c3b9961ec6bfd4c0a1d53ef5659aab7ee3a0f7e8760d69
XOR-URL: safe://hbhybynbpbbdgaee46jbdc8qcsd5dm9igyw8ku77mfukiz7a7y69w8cdmj
Media type: text/html

Resolved using NRS Map:
PublicName: "test"
Container XOR-URL: safe://hnyydyw5xyknabqan31b8fkpyrd4nydtwpausi3urhtxqob4fk78n6siaybqh
Native data type: PublishedSeqAppendOnlyData
Type tag: 1500
XOR name: 0x36f028580bb02cc8272a9a020f4200e346e276ae664e45ee80745574e2f5ab80
Version: 0
+------------------+----------------------+----------------------+--------------------------------------------------------------------------+
| NRS name/subname | Created              | Modified             | Link                                                                     |
+------------------+----------------------+----------------------+--------------------------------------------------------------------------+
| test             | 2020-04-04T15:58:34Z | 2020-04-04T15:58:34Z | safe://hnyynysefhyh4re7qeropok3fdqa3u3mkxicqoun8nfcrf74yc5j8i4x6kbnc?v=0 |
+------------------+----------------------+----------------------+--------------------------------------------------------------------------+```
1 Like
$ safe dog safe://hello/index.html
Native data type: ImmutableData (published)
XOR name: 0x5bfd426c1d782401a6cceb7c82b1334ff8688b7b330a5e948e7d42f5e6240419
XOR-URL: safe://hbhybyns97ejsb46bryguc345hokaugu9apnfzscakm4keh9kn6zuneby3
Media type: text/html

Resolved using NRS Map:
PublicName: "hello"
Container XOR-URL: safe://hnyydyw3utxujj7ecmh3aofrapuxopb1f8krezb8wrum3fm4m1ebdud33rbqh
Native data type: PublishedSeqAppendOnlyData
Type tag: 1500
XOR name: 0x3338be694f50c5f338814986cdf0686453a888b84f424d792af4b9202398f392
Version: 3
+-------------------+----------------------+----------------------+--------------------------------------------------------------------------+
| NRS name/subname  | Created              | Modified             | Link                                                                     |
+-------------------+----------------------+----------------------+--------------------------------------------------------------------------+
| hello             | 2020-04-01T18:28:58Z | 2020-04-01T18:28:58Z | safe://hnyynyix3y68aowrdshbnnyzzsan89ex1ukizkh6s87isc7itb51ademxqbnc?v=0 |
+-------------------+----------------------+----------------------+--------------------------------------------------------------------------+
| hello/style.hello | 2020-04-01T16:11:47Z | 2020-04-01T16:11:47Z | safe://hbhyyyd446wnju9gouch43qfaj1b9eoss4wadnm8yffdnihwu3iuipsd7r        |
+-------------------+----------------------+----------------------+--------------------------------------------------------------------------+

Just noting that I was confusing dog for cat

This gives what I’d expected

$ safe cat safe://hello
Files of FilesContainer (version 0) at "safe://hello":
+--------------+-------+----------------------+----------------------+-------------------------------------------------------------------+
| Name         | Size  | Created              | Modified             | Link                                                              |
+--------------+-------+----------------------+----------------------+-------------------------------------------------------------------+
| /favicon.ico | 10134 | 2020-04-01T18:28:09Z | 2020-04-01T18:28:09Z | safe://hbwybyddc49peadeqqbpfn75wmitabsoiwsdw7j9spcytb4k5crin6refu |
+--------------+-------+----------------------+----------------------+-------------------------------------------------------------------+
| /index.html  | 2417  | 2020-04-01T18:28:09Z | 2020-04-01T18:28:09Z | safe://hbhybyns97ejsb46bryguc345hokaugu9apnfzscakm4keh9kn6zuneby3 |
+--------------+-------+----------------------+----------------------+-------------------------------------------------------------------+
| /style.css   | 315   | 2020-04-01T18:28:09Z | 2020-04-01T18:28:09Z | safe://hbhyyyd446wnju9gouch43qfaj1b9eoss4wadnm8yffdnihwu3iuipsd7r |
+--------------+-------+----------------------+----------------------+-------------------------------------------------------------------+

but the query remains why dog suggests hello/style.hello

1 Like

Now I understand what has happened: you have created a “hello/style.hello” sub-name. Can you check in your history if you have issued a command like ./safe nrs add hello/style.hello ...?

EDIT:
There is an ambiguity in command safe dog safe://hello/style.hello because it can be:

  • dumping hello/style.hello sub-name
  • dumping style.hello file in file container pointed by hello sub-name

The latter is chosen by dog command because the default sub-name is probably hello. The problem is that style.hello file doesn’t exist, hence the error message: No data found for path “/style.hello/” (which is clear finally).

1 Like

grep bash_history finds
safe nrs add hello/style.css

hello/style.hello is not meaningful for what I would have done… but its not important perhaps.

==============

related to safe dog metadata
I sync (I wonder) but dog does not update version, despite the sync being to the FilesContainer

$ safe files sync ./test/ safe://hnyynyst1mit6obtnepskhuaagir8u9a9ng8neouf4o6uuuuyrr7z6f7zsbnc
FilesContainer synced up (version 1): "safe://hnyynyst1mit6obtnepskhuaagir8u9a9ng8neouf4o6uuuuyrr7z6f7zsbnc?v=1"
*  ./test/index.html  safe://hbhybynsq6r8r45drxqk1acd3u3boff4mwq6u193qq1anpx3o6dcqbtyc7 
safe@water:~/SAFE$ safe cat safe://test99
Files of FilesContainer (version 0) at "safe://test99":
+-------------+------+----------------------+----------------------+-------------------------------------------------------------------+
| Name        | Size | Created              | Modified             | Link                                                              |
+-------------+------+----------------------+----------------------+-------------------------------------------------------------------+
| /index.html | 128  | 2020-04-04T17:13:29Z | 2020-04-04T17:13:29Z | safe://hbhybydkzs1fm6zkmdcom39rx1qdgtxmej1r9j5oqb16fg8r7yaegre8ia |
+-------------+------+----------------------+----------------------+-------------------------------------------------------------------+

and the same for dog showing Version: 0

Yet, safe dog safe://hello shows Version: 3

I wonder if somewhere there is a simple abc of this - what changes versions when, that I’ve not read. Is it only FilesContainers that have Version; what does safe cat safe://hello show (version 0)? what is the difference between “nrs add” and “sync” for affecting metadata?

The problem with this above is that safe://test99 hasn’t updated with sync like that. So, I was expected it would have prompted the v=0 v=1 toggle in the browser. Using the full FilesContainer url works but why did it not follow on to affect the domain associated… must I refresh the domain each sync and is this deliberate to avoid switching websites part way through perhaps??

So,

safe://hnyynyst1mit6obtnepskhuaagir8u9a9ng8neouf4o6uuuuyrr7z6f7zsbnc/?v=0
safe://hnyynyst1mit6obtnepskhuaagir8u9a9ng8neouf4o6uuuuyrr7z6f7zsbnc/?v=1

show this difference but safe://test99 still as v=0

tldr; this could be intended behaviour but it’s not obvious.

No, both file containers and NRS map have a version number. Their evolutions are not necessarily synchronized: an NRS map at version 4 can point to a file container at version 3 or 5 for example.

  • A safe files sync command increments the version of the file container

  • A safe nrs add command increments the version of the NRS map

Note: To avoid issuing 2 commands each time a file is modified there is an option of safe files sync command to update both the file container and the NRS map (--update-nrs).

6 Likes

related:

NRS names can link to paths, which files tree and files ls presently ignore, but dog takes into account. I filed a couple of related issues this morning. see:

https://github.com/maidsafe/safe-api/issues/510
https://github.com/maidsafe/safe-api/issues/509

3 Likes

NRS links to future file containers can also be created (I mean with a link version greater than current file container version). We can also do both simultaneously: link to a future file container with a path that doesn’t exist (yet).

I think these operations should be allowed to allow use cases not yet foreseen, like uncovering a secret in the future at an address known in advance.

I don’t think that allowing a missing path needs a flag in the CLI, it only needs a future version in the link

  • Version is current or lower than current => forbid missing path (because this is necessarily a user error)

  • Version is greater than current => allow missing path

Maybe a flag could be added to allow a future version in the link, because use case is not common and is most probably a user error. I propose --future-version. When this flag is passed then user must pass a link to a future version and missing paths are allowed.

Also, a --past-version could be added to explicitly allow a link to a previous version. Same here, this is probably a user error and a flag is needed to unlock the feature.

1 Like

This needs an ABC guide… it’s a bit confusing atm to be sure what is updating version of what when. It might be simple but the options need setting out more clearly?

There is no guarantee that the path will be created in the future version. So if we make a link:

safe://mylink → safe://xor…/non/existent/path?v=1

and ?v=1 does not exist yet. And then we modify the link target somehow in a way that creates ?v=1 but without creating /non/existent/path, then safe://mylink is a broken link. Plus, it was a broken link before ?v=1 existed.

Personally, I do not see value in this concept of “future” nrs names.

But then I suppose it depends if we go with a model of “allow broken links” or not.

NRS is just a pointer… I don’t see why null pointers should not be useful. :thinking: control for control’s sake is usually error. By default all options that are possible should be available, as maximizing the options increases opportunity for creativity.

There is a risk related to someone spawning multiple nrs with
nrs create insertvaluablename --link safe://null?v=0

but I would hope the cost of creating nrs might prohibit too much of that… or a least for a set of obvious, such as existing unsafe domains… but there are not many of those… 1 million perhaps.

Alt defense is limiting the number of nrs per account… and making the cost of new account the cost but focus on the nrs itself perhaps better.

1 Like

In general I’d prefer when you give the user freedom, when an app/system is trying to be too smart sometimes it’s annoying for experienced users since they cannot do what they want, so I personally prefer warnings rather than preventing users from doing something they may find value in doing, like in the case a user knows what he is doing when adding a non-existing URL link to an NRS name.

2 Likes

One simple case, is that the user is taking actions in an ordered way, that makes sense to them; setting up the NRS ahead of time and shortly thereafter making the content… call it error or opportunity to do it different.

Without good reason to not have broken links, they should remain an option. Also, broken links tempts dungeons and dragons :smiley:

2 Likes

A fine line here between developer convenience(or call it good planning) and domain sitting.

The alt it to have a template page with detail of how to buy… same problem.

The solution is to scale the cost for popular names… or just block those for a time while some auction bidding occurs.

Could do that across the board… and where there is no interest after a period, then allow it to register. Where there is interest from multiple parties, extend period until no new interest or some registration cost is pushed high enough that those putting their deposits forward withdraw and resolve a “winner”.

Some would call that an opportunity for marketing but I’m not sure.

1 Like

So to me, all of this boils down to the question: Will Safe Network allow broken links to be created or not?

Has this been debated/decided in the past? I don’t know…

On the plus side, it gives users/devs more flexibility, and is very similar to how the www works, so people are used to it.

On the flip side, we are building a new ecosystem and have an opportunity to correct mistakes made in the www. Broken links have been a serious problem. Here, we have the concept of immutable data. If we enforce a simple rule that all nrs links must point to existing data at time of creation, then any links pointing to immutable data will work forever, and can potentially be marked as an immutable link.

Links to mutable data are certainly more transient and could become broken just by the data being deleted, so there could be a good argument to relax the requirement a bit there, though I personally don’t see a need yet.

2 Likes

This won’t be possible if we are talking about public data, which is the perm web.

You cannot really enforce it, all we can and are doing is having our apps to store this data using the corresponding formats, but anyone can still insert broken links if they want to (by hacking some of the apps) or if they use some other CLI/client app which does allow it, so there is a chance of broken links still existing on the network once many apps exist out there, and in fact our apps will need to be able to handle such cases gracefully.
We’ll have the same scenario with any data inserted which doesn’t follow the conventions/formats, moreover, some people may use other conventions and formats so we will have to handle them gracefully, at least, if we don’t support them.

1 Like

So I’m actually kind of on the fence here. I guess we have two questions:

  1. Can we technically prevent people from creating broken links?
  2. If 1 is yes, do we want to prevent them?

If the answer to 2 is no, we don’t need to worry about 1.
if yes, then we should think about it.

There are also considerations for url’s being used as RDF/semantic web identifiers.

This is actually a very old debate, going all the way back to xanadu. a couple links:

And there is a concept of a persistent URL even in the www, eg:

Given that SAFE Network is trying to build a more persistent web, it may behoove us to put more thought/consideration into this.

archive.org might be interested to weigh in, for example.

just thinking out loud…

1 Like

Obviously so, but also is it worth the trouble?
… where’s the motivation.

I wonder 2 is a simple no. There is no added value in preventing others doing xyz. The whole basis of Freedom, is opportunity… to learn if nothing else.