When is a xorurl not a xorurl?

Given safe urls and safe xorurl share an overlapping namespace, I wondered about creating a roadblock to a file that doesn’t exist by creating a nrs at the xorurl of the other file but I’m not clear what happened and if it worked… perhaps someone else can try @Southside

============

roadblock

So, getting the xorurl for a random file that does not exist on the network:
step 1of3

$ safe xorurl ./thisisatest 
No credentials found for CLI, connecting with read-only access...
1 file/s processed:
./thisisatest  safe://hbyyyyddhpd545uynusc16893e84bw4yd3zf5okjph13eu155yy4ng38jt 

For curiousity decoded:

$ safe xorurl decode safe://hbyyyyddhpd545uynusc16893e84bw4yd3zf5okjph13eu155yy4ng38jt
Information decoded from XOR-URL: safe://hbyyyyddhpd545uynusc16893e84bw4yd3zf5okjph13eu155yy4ng38jt
Xorname: 8f8d1ef5b98053b325e3ff283e834d0079b9770525bc9651396f6006846c9d31
Type tag: 0
Native data type: PublishedImmutableData
Path: none
Sub names: []
Content version: latest

and cat with it not being uploaded

$ safe cat safe://hbyyyyddhpd545uynusc16893e84bw4yd3zf5okjph13eu155yy4ng38jt
No credentials found for CLI, connecting with read-only access...
[2020-04-12T10:04:51Z ERROR safe] safe-cli error: [Error] NetDataError - Failed to GET Published ImmutableData: CoreError(Data error -> Requested data not found - CoreError::DataError -> NoSuchData)

and dog

$ safe dog safe://hbyyyyddhpd545uynusc16893e84bw4yd3zf5okjph13eu155yy4ng38jt
No credentials found for CLI, connecting with read-only access...
Native data type: ImmutableData (published)
XOR name: 0x8f8d1ef5b98053b325e3ff283e834d0079b9770525bc9651396f6006846c9d31
XOR-URL: safe://hbyyyyddhpd545uynusc16893e84bw4yd3zf5okjph13eu155yy4ng38jt
Media type: Unknown

=====================

step 2of3
So, now I upload a roadblock other file and create an nrs that is the xorurl of the file that is thisisatest which is still not on the network.

$ safe files put ./roadblock 
FilesContainer created at: "safe://hnyynys9jwm9onjudcgomf3knc5h8aokqrdpqqfcts9b58i3jzsp1weo6rbnc"
+  ./roadblock  safe://hbyyyynw7fi7ko8u3wtai6eekko3m5y6u9ud7w1i4ajycmwfwu3wwpnn81 

and then create an nrs at the xorurl of the other file that has not been uploaded.
step 3of3

$ safe nrs create hbyyyyddhpd545uynusc16893e84bw4yd3zf5okjph13eu155yy4ng38jt --link safe://hbyyyynw7fi7ko8u3wtai6eekko3m5y6u9ud7w1i4ajycmwfwu3wwpnn81?v=0
New NRS Map for "safe://hbyyyyddhpd545uynusc16893e84bw4yd3zf5okjph13eu155yy4ng38jt" created at: "safe://hnyydyw5ijn4mqddua5h7hs97p637ew5thnmeigaob9bjii9jszpu1u3kybqh"
+  hbyyyyddhpd545uynusc16893e84bw4yd3zf5okjph13eu155yy4ng38jt  safe://hbyyyynw7fi7ko8u3wtai6eekko3m5y6u9ud7w1i4ajycmwfwu3wwpnn81?v=0 

=====================
step 4of3 shouldn’t work??
but it seemed that later I could do upload of the original file

safe files put ./thisisatest 
FilesContainer created at: "safe://hnyynyzdhb8gdoihqff3dpdpu6si44iqcoi71t5qfyymoduia4cju8z9znbnc"
+  ./thisisatest  safe://hbyyyyddhpd545uynusc16893e84bw4yd3zf5okjph13eu155yy4ng38jt 

=====================

offtopic below:

The reason I couldn’t make sense of this above, was I got distracted by that the desktop browser appears to download content and I’d left the roadblock as an empty file.

bug or feature - desktop browser?

It seems that the desktop browser will download as content the file only on the back of what looks like a xorurl… so, for a zerofile and it seems for text files, it opens the download folder!..

for example:
try safe://hbyyyynw7fi7ko8u3wtai6eekko3m5y6u9ud7w1i4ajycmwfwu3wwpnn81
which is a null content.

and safe://testk2 null
and safe://testk3 not null
still do the same… likely on the back of mediatype

Equally a surprise is safe://hello/index.html for which the xorurl doesn’t action the associate .css file
safe://hbhybyns97ejsb46bryguc345hokaugu9apnfzscakm4keh9kn6zuneby3
which is perhaps odd for the browser??.. but perhaps is a feature worth keeping.

bug or feature - mobile browser?

try again in mobile browser safe://hbyyyynw7fi7ko8u3wtai6eekko3m5y6u9ud7w1i4ajycmwfwu3wwpnn81

Still, odd perhaps the mobile browser doesn’t do the same with that null file and instead suggests “Error Page not found No such data exists.”… and I don’t know if that is an error where url does exist… it should show null?


bug safe cat

I’m also wondering that there’s something wrong with the way that safe cat works

These are different outputs where there surely should be consistency.
I wonder that safe cat is doing what safe files ls should be.
safe cat perhaps should be only for downloading and safe files ls ( :thinking:== safe ls) only for listing.

for example try
safe cat safe://testk3
safe cat safe://hello

The first dumps the content, which is fine.
The second does a listing output because it happens there is nothing exactly at that location but there is content subordinate to it.

Edit: I’ve just logged this because thinking about it, it seems clear that cat=concatenate is being confused with ls=listing https://github.com/maidsafe/safe-api/issues/516

3 Likes

Step 1

Steps 2-4

In the browser safe://hbyyyydiajyh8oy7rddbzxggt4rrhmydebuboog8uogbjfa4tagissr5dr gives me a download option

So Im seeing the same as you so far…

When that downloads… save it…
and is that the content of thisisSouthside or is it roadblock?

1 Like

thisisSouthside

As uploaded…
willie@gagarin:~/projects/maidsafe$ cat thisisSouthside xor-url testing as requested by @davidpbrown

As downloaded…

willie@gagarin:~/projects/maidsafe$ cat ~/Downloads/safe-xorurl-test.txt 
xor-url testing as requested by @davidpbrown

So, much for the roadblock… I don’t know to be sure but that seems odd that roadblock was being put to the same location xorurl. Needs an expert eye perhaps to check that is not a problem.

1 Like

Dunno if this tells us anymore?

willie@gagarin:~/projects/maidsafe$ safe cat $ROADBLOCK Files of FilesContainer (version 0) at "safe://hnyynyz9xjk3ebhmq9zf4oae5e4yadqrjyemtg9tqmzikie3uumnuejqi1bnc": +------------+------+----------------------+----------------------+-------------------------------------------------------------------+ | Name | Size | Created | Modified | Link | +------------+------+----------------------+----------------------+-------------------------------------------------------------------+ | /roadblock | 30 | 2020-04-12T14:29:43Z | 2020-04-12T14:29:43Z | safe://hbyyyydmqbb6uwng57tj4sw4xp95mrynhm9a9sja17qcn3qx9afti6ngeh | +------------+------+----------------------+----------------------+-------------------------------------------------------------------+ willie@gagarin:~/projects/maidsafe$ safe files ls $ROADBLOCK Files of FilesContainer (version 0) at "safe://hnyynyz9xjk3ebhmq9zf4oae5e4yadqrjyemtg9tqmzikie3uumnuejqi1bnc": Files: 1 Size: 30 Total Files: 1 Total Size: 30 SIZE CREATED MODIFIED NAME 30 2020-04-12T14:29:43Z 2020-04-12T14:29:43Z roadblock willie@gagarin:~/projects/maidsafe$ safe cat safe://hbyyyydmqbb6uwng57tj4sw4xp95mrynhm9a9sja17qcn3qx9afti6ngeh all sorts of stuff in the way

as a reminder…
export ROADBLOCK=safe://hnyynyz9xjk3ebhmq9zf4oae5e4yadqrjyemtg9tqmzikie3uumnuejqi1bnc

It’s the xorurl ending r5dr that’s the focus.
That is the xorurl for file ./thisisSouthside AND the location as safe://url attributed to the roadblock.

So, the question is what kind of a clash is a xorurl == safe://url
because it seems that the nrs create of roadblock gets bumped out the way, when the ./thisisSouthside was uploaded.

Location safe://…r5dr is either the safe url=[./thisisSouthside] or it is the xorurl=[nrs created for roadblock]… atm from above it outputs as ./thisisSouthside

I’m expecting midway between that before ./thisisSouthside was uploaded that address was roadblock??.. perhaps we missed a step checking that.

1 Like

Just about to have my tea now, I’ll run through it with you later and we’ll check exactly what is at each address step by step

Meantime…

`willie@gagarin:~/projects/maidsafe$ safe cat $TESTFILE
xor-url testing as requested by @davidpbrown

willie@gagarin:~/projects/maidsafe$ safe dog $TESTFILE
Native data type: ImmutableData (published)
XOR name: 0xd709070f007483186ef31a3a2138b00d019861031e7030525c6a38356d626c64
XOR-URL: safe://hbyyyydiajyh8oy7rddbzxggt4rrhmydebuboog8uogbjfa4tagissr5dr
Media type: Unknown

willie@gagarin:~/projects/maidsafe$ safe cat safe://hbyyyydiajyh8oy7rddbzxggt4rrhmydebuboog8uogbjfa4tagissr5dr
xor-url testing as requested by @davidpbrown
`

@davidpbrown OK trying again step by step.

Create a file, get the safe XORURL for that file and then run xorurl decode. Set the output of the xorurl command as an env variable to avoid confusion with long strings later on
At this point this file is ONLY on the local PC.

willie@gagarin:~/projects/maidsafe$ touch dubious.txt
willie@gagarin:~/projects/maidsafe$ echo "Dubious XORURL goings-on" > dubious.txt
willie@gagarin:~/projects/maidsafe$ cat dubious.txt 
Dubious XORURL goings-on
willie@gagarin:~/projects/maidsafe$ safe xorurl ./dubious.txt
1 file/s processed:
./dubious.txt  safe://hbhydynwmfsobb57edw93y4botmmqac5zpbncsbsdh6betwmia8hp48cd5 
willie@gagarin:~/projects/maidsafe$ export DUBIOUS_XORURL=hbhydynwmfsobb57edw93y4botmmqac5zpbncsbsdh6betwmia8hp48cd5
willie@gagarin:~/projects/maidsafe$ echo $DUBIOUS_XORURL
hbhydynwmfsobb57edw93y4botmmqac5zpbncsbsdh6betwmia8hp48cd5
willie@gagarin:~/projects/maidsafe$ safe xorurl decode safe://$DUBIOUS_XORURL
Information decoded from XOR-URL: safe://hbhydynwmfsobb57edw93y4botmmqac5zpbncsbsdh6betwmia8hp48cd5
Xorname: 5165b4021df503a7f20d06115add866eed089960d87cf0511a2eb83f1ba3b07b
Type tag: 0
Native data type: PublishedImmutableData
Path: none
Sub names: []
Content version: latest
willie@gagarin:~/projects/maidsafe$

and to prove it is not on the shared-section …

willie@gagarin:~/projects/maidsafe$ safe cat safe://$DUBIOUS_XORURL
[2020-04-12T20:23:58Z ERROR safe] safe-cli error: [Error] NetDataError - Failed to GET Published ImmutableData: CoreError(Data error -> Requested data not found - CoreError::DataError -> NoSuchData)

and trying that safe:// address in the browser returns a blank page as expected ( though a 404 equivalent would be nice - hint err err - is that even possible?)

Step 2
Re-using the roadblock file from earlier today and putting that to the network…

willie@gagarin:~/projects/maidsafe$ cat roadblock 
all sorts of stuff in the way
willie@gagarin:~/projects/maidsafe$ safe files put ./roadblock
FilesContainer created at: "safe://hnyynyzru8k98q35nyt6yukc7h8qne1qpz6mwa5wtua7rkxprgtyof1obabnc"
+  ./roadblock  safe://hbyyyydmqbb6uwng57tj4sw4xp95mrynhm9a9sja17qcn3qx9afti6ngeh

Step 3
Make an NRS entry for roadblock using the xorurl for dubious.txt.

willie@gagarin:~/projects/maidsafe$ export ROADBLOCK_XORURL=safe://hbyyyydmqbb6uwng57tj4sw4xp95mrynhm9a9sja17qcn3qx9afti6ngeh
willie@gagarin:~/projects/maidsafe$ safe nrs create $DUBIOUS_XORURL --link $ROADBLOCK_XORURL
New NRS Map for "safe://hbhydynwmfsobb57edw93y4botmmqac5zpbncsbsdh6betwmia8hp48cd5" created at: "safe://hnyydysif8fmmcjmqa3d5us466uxuesxor79q64esocsfdutgt5q5kexpebqh"
+  hbhydynwmfsobb57edw93y4botmmqac5zpbncsbsdh6betwmia8hp48cd5  safe://hbyyyydmqbb6uwng57tj4sw4xp95mrynhm9a9sja17qcn3qx9afti6ngeh

Now catting that shows the roadblock text
willie@gagarin:~/projects/maidsafe$ safe cat safe://hnyydysif8fmmcjmqa3d5us466uxuesxor79q64esocsfdutgt5q5kexpebqh all sorts of stuff in the way

Step 4
Upload the original file - which has not the PC until now
willie@gagarin:~/projects/maidsafe$ safe files put ./dubious.txt
> FilesContainer created at: “safe://hnyynyiz141wmkxxe6h6n49j849bzjpyoxw6bo4qawba3di4b5z9uxqfsabnc”
> + ./dubious.txt safe://hbhydynwmfsobb57edw93y4botmmqac5zpbncsbsdh6betwmia8hp48cd5

then cat these outputs
willie@gagarin:~/projects/maidsafe$ safe cat safe://hnyynyiz141wmkxxe6h6n49j849bzjpyoxw6bo4qawba3di4b5z9uxqfsabnc
Files of FilesContainer (version 0) at “safe://hnyynyiz141wmkxxe6h6n49j849bzjpyoxw6bo4qawba3di4b5z9uxqfsabnc”:
±-------------±-----±---------------------±---------------------±------------------------------------------------------------------+
| Name | Size | Created | Modified | Link |
±-------------±-----±---------------------±---------------------±------------------------------------------------------------------+
| /dubious.txt | 25 | 2020-04-12T21:57:20Z | 2020-04-12T21:57:20Z | safe://hbhydynwmfsobb57edw93y4botmmqac5zpbncsbsdh6betwmia8hp48cd5 |
±-------------±-----±---------------------±---------------------±------------------------------------------------------------------+
willie@gagarin:~/projects/maidsafe$ safe cat safe://hbhydynwmfsobb57edw93y4botmmqac5zpbncsbsdh6betwmia8hp48cd5
Dubious XORURL goings-on

and in the browser…

Is this the way it’s meant to work or have we just confused ourselves?

EDIT: These are the commands I used for the above

echo "Dubious XORURL goings-on" > dubious.txt
cat dubious.txt
safe xorurl ./dubious.txt
export DUBIOUS_XORURL=hbhydynwmfsobb57edw93y4botmmqac5zpbncsbsdh6betwmia8hp48cd5
echo $DUBIOUS_XORURL
safe xorurl decode safe://$DUBIOUS_XORURL
safe cat safe://$DUBIOUS_XORURL
safe dog safe://$DUBIOUS_XORURL
cat roadblock
safe files put ./roadblock
export ROADBLOCK_XORURL=safe://hbyyyydmqbb6uwng57tj4sw4xp95mrynhm9a9sja17qcn3qx9afti6ngeh
safe nrs create $DUBIOUS_XORURL --link $ROADBLOCK_XORURL
cat safe://hnyydysif8fmmcjmqa3d5us466uxuesxor79q64esocsfdutgt5q5kexpebqh
safe cat safe://hnyydysif8fmmcjmqa3d5us466uxuesxor79q64esocsfdutgt5q5kexpebqh
safe cat safe://hbyyyydmqbb6uwng57tj4sw4xp95mrynhm9a9sja17qcn3qx9afti6ngeh
safe files put ./dubious.txt
safe cat safe://hnyynyiz141wmkxxe6h6n49j849bzjpyoxw6bo4qawba3di4b5z9uxqfsabnc
safe cat safe://hbhydynwmfsobb57edw93y4botmmqac5zpbncsbsdh6betwmia8hp48cd5

erm… I can’t make sense of this just now.
So, I tried again… and the upload of a roadblock to a domain that is a xorurl seems to fail… I say seems because I not clear to be sure what is occuring… it suggested that it was created as a NRS.
???.. this seems simple

== prep ===============

$ safe xorurl ./roadblock 
1 file/s processed:
./roadblock  safe://hbyyyyn3xs3nbpc5ocif7g3hq6fmakmijtxemqjurh5b3fc3wm89cx3q4u 
$ safe xorurl ./newfile 
1 file/s processed:
./newfile  safe://hbyyyydhpshh9hfwo56isriq46omuh3wecdojx4nsmpa1ydxf3yersumnk


intent to create NRS at safe://hbyyyydhpshh9hfwo56isriq46omuh3wecdojx4nsmpa1ydxf3yersumnk
before newfile is uploaded

which is 
safe nrs create safe://hbyyyydhpshh9hfwo56isriq46omuh3wecdojx4nsmpa1ydxf3yersumnk --link \1?v=0

with \1 whatever roadblock is.. which we know ahead of time will be safe://hbyyyyn3xs3nbpc5ocif7g3hq6fmakmijtxemqjurh5b3fc3wm89cx3q4u

=>
safe nrs create safe://hbyyyydhpshh9hfwo56isriq46omuh3wecdojx4nsmpa1ydxf3yersumnk --link safe://hbyyyyn3xs3nbpc5ocif7g3hq6fmakmijtxemqjurh5b3fc3wm89cx3q4u?v=0

== work ===============

So, first step uploading roadblock as
$ safe files put ./roadblock
FilesContainer created at: "safe://hnyynysg4kkn68psjd9qeqemqkjhepq5ue4gsujussg6xw9gq8wqtubxpkbnc"
+  ./roadblock  safe://hbyyyyn3xs3nbpc5ocif7g3hq6fmakmijtxemqjurh5b3fc3wm89cx3q4u 

then put in place
$ safe nrs create safe://hbyyyydhpshh9hfwo56isriq46omuh3wecdojx4nsmpa1ydxf3yersumnk --link safe://hbyyyyn3xs3nbpc5ocif7g3hq6fmakmijtxemqjurh5b3fc3wm89cx3q4u?v=0
New NRS Map for "safe://hbyyyydhpshh9hfwo56isriq46omuh3wecdojx4nsmpa1ydxf3yersumnk" created at: "safe://hnyydyiqiitnmb7arbbhdjmit355jwihs5rny9ritqkacoaib4f413ymgrbqh"
+  safe://hbyyyydhpshh9hfwo56isriq46omuh3wecdojx4nsmpa1ydxf3yersumnk  safe://hbyyyyn3xs3nbpc5ocif7g3hq6fmakmijtxemqjurh5b3fc3wm89cx3q4u?v=0

testing the roadblock, should find a file at this new domain…
but gives an ERROR?

$ safe cat safe://hbyyyydhpshh9hfwo56isriq46omuh3wecdojx4nsmpa1ydxf3yersumnk
[2020-04-13T11:43:51Z ERROR safe] safe-cli error: [Error] NetDataError - Failed to GET Published ImmutableData: CoreError(Data error -> Requested data not found - CoreError::DataError -> NoSuchData)

$ safe dog safe://hbyyyydhpshh9hfwo56isriq46omuh3wecdojx4nsmpa1ydxf3yersumnk
Native data type: ImmutableData (published)
XOR name: 0xf1b6e73fc2d21bf56c4abb5e82e7ccd10c1c12fd0acb6e2401bcb9020969ac4a
XOR-URL: safe://hbyyyydhpshh9hfwo56isriq46omuh3wecdojx4nsmpa1ydxf3yersumnk
Media type: Unknown

and that is before ever trying to upload newfile.

1 Like

I’ll get back to you when I can, deep in trying to diagnose a failed starter motor right now.
Had to apply the Ebay solution to the starter motor that wont…
safe-api got updated yesterday, dunno if we should try this process again or see what comes out of the other discussion on NRS update. I suspect we are looking at a moving target …

Indeed it does, let me try to explain it.

When you create a NRS name, the NRS container is stored at the sha3(<string you provided>). This means that even if you use a valid XOR URL string as the string to create an NRS name, that location won’t be the same as the actual file.

Also, when we process a URL, we first attempt to decode it as XOR-URL:

  • if it’s valid we proceed to fetch the content and we’ll simply fail if the content is not found at the decoded location
  • If it couldn’t be decoded as a valid XOR-URL we fallback to try it as an NRS-URL and try to fetch the content from sha3<url's host>

This is all intentional, so people cannot shadow someone else’s immutable data by creating an NRS name using the same XOR-URL string, moreover, because of this people should be discouraged to use NRS names which happen to be valid XOR URLs.

Now let me try to show you what I mean with some commands similar to yours:

$ cat testdata/subfolder/subexists.md 
hello from a subfolder!

$ safe xorurl testdata/subfolder/subexists.md 
1 file/s processed:
testdata/subfolder/subexists.md  safe://hbhynyn9uodh1ju5uzyti3gmmtwburrssd89rcwcy3rzofdpypwomrzzte 

$ safe files put testdata/test.md 
FilesContainer created at: "safe://hnyynywqokprdwt4ytnumf4hqe783tyq15otscu977k5jqozoxoeq83xgabnc"
+  testdata/test.md  safe://hbhynydpan7d94mwp1bun3mxfnrfrui131an7ihu11wsn8dkr8odab9qwn 

$ safe nrs create hbhynyn9uodh1ju5uzyti3gmmtwburrssd89rcwcy3rzofdpypwomrzzte --link safe://hnyynywqokprdwt4ytnumf4hqe783tyq15otscu977k5jqozoxoeq83xgabnc?v=0
New NRS Map for "safe://hbhynyn9uodh1ju5uzyti3gmmtwburrssd89rcwcy3rzofdpypwomrzzte" created at: "safe://hnyydyzicyja3zmwd3bkmrrzk5itw4gxazqt6db35ghmemg1jfkeya4pmhbqh"
+  hbhynyn9uodh1ju5uzyti3gmmtwburrssd89rcwcy3rzofdpypwomrzzte  safe://hnyynywqokprdwt4ytnumf4hqe783tyq15otscu977k5jqozoxoeq83xgabnc?v=0 

$ safe cat safe://hbhynyn9uodh1ju5uzyti3gmmtwburrssd89rcwcy3rzofdpypwomrzzte
[2020-04-13T19:27:43Z ERROR safe] safe-cli error: [Error] NetDataError - Failed to GET Published ImmutableData: CoreError(Data error -> Requested data not found - CoreError::DataError -> NoSuchData)

$ safe files put testdata/subfolder/subexists.md 
FilesContainer created at: "safe://hnyynyseyymino7cdwu8o4ksgp8s1enhxoauwqen377x5gpimbzy5rf8qgbnc"
+  testdata/subfolder/subexists.md  safe://hbhynyn9uodh1ju5uzyti3gmmtwburrssd89rcwcy3rzofdpypwomrzzte 

$ safe cat safe://hbhynyn9uodh1ju5uzyti3gmmtwburrssd89rcwcy3rzofdpypwomrzzte
hello from a subfolder!
6 Likes