Pre-Dev-Update Thread! Yay! :D

b1aad5406cefaa2c798d6969e0dc46ed

10 Likes

Reward Flow is coming !

13 Likes

Hey @Scorch could you shed a little light on your api asynchronous server example?! :nerd_face:

3 Likes

Really encouraging to see @Scorch, @mav, @latch getting so involved in the Maidsafe repos. Community engagement FTW. :clap::clap::clap: thank you

17 Likes

That is reallly goooood!!

4 Likes

Yea sure. It’s nothing crazy right now but it’s a start. I’ll offer some background on why it’s relevant and where it could go , and then I’ll say what I actually did. There’s a thread here if your interested on following the progress.

TLDR:

It’d be cool if we humans could issue commands to make a running node do interesting things (that don’t involve dying). I proposed a proof-of-concept that might solve that, and maybe help in improving existing code… maybe… :slight_smile:

Background:

The idea was kind of born of the fact that most of the tools/commands for doing things with nodes are pretty course-grained right now. For example, you can run or kill a node from the cli, but you can’t do something more complicated like collect statistics. That’s because there isn’t a good interface yet for when you’re looking from the outside, into the node. If nodes were houses on a street, it would be like not having doors or mailboxes on the houses. They do their thing while they run, but there’s not much you can do to communicate with whoever’s inside.

The commands we do have operate on the node itself or on the environment surrounding it. In the case of kill, It doesn’t work with the node (i.e. telling it to shutdown nicely), it operates on the node process (i.e. kill just tells your operating to go out axe the process at whatever stage of running it’s in). In the house analogy, it’d be like knocking down the house to tell your tenants the lease is up.

Despite everything, this approach works reasonably well for the most part, but it has some obvious drawbacks. One is that the node needs to be prepared to die at any moment, because there’s no clear path for things like shutting down, so designing error handling becomes trickier. Beyond that, it would be cool to collect things like logs from the node, or collect usage statistics for improvement, fetch debug information, etc.

This is something that’s been talked about before, but hasn’t ever been fully addressed to my knowledge.

What the Async Server Example does

Given that as an ultimate goal, the program I wrote is a proof-of-concept mostly. The idea is to use the same library that the authenticator currently uses (called qjsonrpc, which is in the sn_api repo) to build a mailbox of sorts for the node to accept incoming commands. The node and the authenticator are different enough, though, that it’s not a copy-paste job to make a node accept commands like the authenticator.

The async server example does two things. First, it provides, well, an example usage of qjsonrpc. In general, that could be handy to other developers looking to use it one day. It also might help to highlight some areas for refactor and enhancement in the qjsonrpc library itself (actually, that’s what my little contribution last week was about). Secondly, it shows how you might build a scheme that the node could use to accept commands, in a manner similar to authd, without requiring massive modification to the existing node’s code structure.

Where to Next

Well, technically this hasn’t even been merged yet and it’s just a PoC, so I’m talking hypothetically, assuming the patch isn’t rejected outright :laughing: Next though I want to take some of the learnings from creating this little example program and maybe use it to propose some further enhancements to qjsonrpc. If the dev team comes to the conclusion that this PoC might be a good model for the node to accept commands, then maybe work could commence on implementing it in the node.

On a related note, there are some conceptually similar things between commanding authd and a node process, so I’m particularly interested in maybe making the command interface generic enough that it could be used for both the authenticator and the node. It’d be cool because then really only one implementation would need to be maintained, and would cut down on existing code in the authenticator (as well as any future implementers of the system), which is rarely a bad thing in software.

One step at a time though. For now, I really just wrote a PoC.

23 Likes

Seems like a good addition to NRS to prevent some phishing attacks or spoofing? @mav?

Invalidate nrs names with troublesome characters by iancoleman · Pull Request #699 · maidsafe/sn_api · GitHub

7 Likes

Looks like a simple subset 30 of the standard confusables, which is nice to see.

I wonder at some later time there could be option in NRS versioning to see user choice of how those resolve. The default perhaps assuming what is wanted… minor wrinkle aesthetic… so long as permanence is retained for future resolving what was… if domain name ownership can be passed to new owners they might jave different interest than default but file under ocd for now. :grin:

5 Likes

Still NRS is basically a APP level data and this restriction would be in the applications. So user choice can be built into the applications like the browser

3 Likes

Yes that’s easier… one point of reference for each confusion and apps get accomodate interpretation if they want.

1 Like

I would expect that there would be a reference data object/file (for people to look up) with the standard table in it and libraries for various languages to process the name. Applications can then change the standard behaviour as they see fit.

It is not a core network issue, which I guess I want to ensure anyone reading realised. But an application layer issue.

1 Like

The change in PR #699 is mostly about characters that should never reasonably appear.

It started with just the space char (See #696).

I tried creating nrs with a non-breaking space to see if it would work, and it worked.

Then I tried with tab char, it worked.

I tried with newline, it worked.

So this needed exclusion for all char::is_whitespace, no big deal, easy to do.

Then we also have control characters, easy to exclude using char::is_control and there’s no reason to have control characters in any normal url, so that was obvious to include.

But then there’s also zero width space which is neither whitespace nor a control character. So we need some special exceptions. There’s 30 of those that are pretty obviously never going to be part of any url. These were manually selected from the unicode character database.

This is the point the PR gets to. It has the ‘obviously not gonna be in a url’ chars.

But we can (and probably will need to) go further.

There’s the character U+00C0 Capital Letter A with grave À vs U+0041 + U+0300 Latin Capital A + Combining grave accent À - looks identical but hashes to different xorurls. But both are linguistically valid and useful.

There’s also homoglyphs, U+006F Latin Small Letter o vs U+03BF Greek Small Letter omincron ο - looks identical, but hashes to different xorurls. But both are linguistically valid and useful.

Some questions are

  • how are people entering these variations?
  • how do we display these variations to people?
  • how do we convert the unicode to bytes?
    • percent encode?
    • punycode?
    • upper or lowercase?
    • utf-8?
    • utf-16?
    • something else?
    • do we use unicode normalization? which one?
  • how do we convert these bytes to an xorurl? (this is already solved)
  • is it possible to convert to and fro each representation?
  • what are the risks and benefits of allowing confusables / homoglyphs / symbols / emoji etc? I always remind myself “I can only read English so would never be able to use Kanji urls, please do not limit your software to Kanji only”, then flip the two character sets to see that ascii is not adequate.

There’s a lot written elsewhere on this topic. This post is a bit of an intro to see that nrs beyond A-Za-z0-9 has some tricky situations. It could easily be 10x longer…

16 Likes

@joshuef it looks as though the push to adapt the new lexicon into the code is underway. There is the recent change from Money → Token but there is still mention of “safecoin” and MAX_COINS_VALUE etc.

// The maximum amount of safecoin that can be represented by a single Token
const MAX_COINS_VALUE: u64 = (u32::max_value() as u64 + 1) * COIN_TO_RAW_CONVERSION - 1;

Do all of these need to be changed? Would seem like a tedious and hefty overhaul.

What made me wonder in the first place is it would seem reasonable to change “safecoin” to “Safe Network Token” along with the transition from money to token.

4 Likes

These are v quick changes with the rust toolset. Basically in vscode or similar select symbol and rename. So we’re OK renaming symbols, small pita is it needs done in several libs at once, but your looking at an hour max work to do that.

3 Likes

as @dirvine said. Shouldn’t be too much bother. And where’s there’s no link it can be done piecemeal :+1:

5 Likes

SNT is already exist :slight_smile: Status project. Imho SAFE is a cool ticker for the new network

4 Likes

For fork sake

We need to acknowledge what exists.
Unicode confusables are formally defined and resolved with https://www.unicode.org/Public/security/8.0.0/confusables.txt

I wonder necessarily, NRS registration should follow the Unicode confusables table regardless and force many to one instance. … and then, everything else is app level aesthetic option for future.

Having just suggested “should” then …

Is an assumption - so approach with caution…

This is fair for urls

and the newline character too.

However, beyond those I would hesitate - even with whitespace! All characters in Unicode exist for a reason?.. so, U+205F [Medium Mathematical Space] might matter to some just for the asthetic?

Some questions were:

  • how are people entering these variations?

does not matter.

  • how do we display these variations to people?

as the owner of the url prefers… as the user of the url prefers (follows from suggestion below; and nicely in keeping with empowering users)… and/or a default, if there’s not an option to lookup that preference.

So, I wonder perhaps

safe:/fr-ca/ᑶeᑶᑶers
safe:/en/peppers

or similar could suggested at app level, for which context is preferred - or better perhaps some block level unique reference Unicode - Unicode Character Table
… and safe:// perhaps the usual unicode default… available as an option for all.

  • what are the risks and benefits of allowing confusables / homoglyphs / symbols / emoji etc?

So long as each string resolves to one instance at registration via NRS, there is no confusion - it’s just difference of opinion on how that string is presented.

:thinking: … undecided about capital letters… always preferred lowercase myself but old unSafe internet has a legacy of confusion on that dependant on the host registrar… there is confusion in English even for I(i) and l(L).

Yeah we need an organic alternative. SAFE is available as a ticker? SN is another viable suggestion as well. People revert to SNT but Like you said it’s status network so I twinge a little bit every time I see that disseminated.

Probably best not to announce the ticker at all until formalised! :sweat_smile:

12 Likes