NRS Brainstorming Megathread

UTC is the standard zone used in the hardware clock of every motherboard. It may or not be set correctly by the vault operator, but it is a common standard that exists.

It will also need to resolve a specific version of a document won’t it? Specific version and timestamp are synonymous/analogous. One is more amenable to a software dev, the other more easy to use/understand by mortal humans.

1 Like

To what tolerance do you consider proper time verification and reliability? +/- 1 year? or 1 day? or 1 hour? or 1 second? or 1 millisecond? or 1 nanosecond? 1 femto second?

Everything has a tolerance. You can either reject data because it doesn’t meet some a priori tolerance requirement, or collect the data and determine the tolerance or accuracy a posteriori.

The latter provides a view of what the close group thought the year and day was near version publication.

1 Like

Time belongs to the application layer. Not the core system. Even the current internet has time at the application layer

And there are very good reasons protocols do not use time (as in clock)

We’re not talking about communication protocol, we’re talking about metadata supplementation. Application based and you break pweb consistency. It becomes a hodgepodge.

Did I say communication protocol. I said protocols, and has a broader meaning than packets being sent over wires.

In the broader sense of protocol, it is still an application layer issue. Time is unreliable and recognised as such. It is a human construct and should be kept as such.

If you want to know if one version is newer or older than another file’s version then you need precise accuracy. Is +/- 5 minutes close enough? That could make checking versions by time (which came first or after) can make it a dog’s breakfast if referencing another site’s JS libraries where the timestamps were out by minutes or hours with the versions referencing them. This will cause real issues, so why try and use it? This is why protocols do not use time (as in clock) because separate systems (or sections in Safe) will reliably have differing times and by varying amounts.

Just use version that the web page wants and you have no need for timestamps.

And it still has Nothing to do with NRS

You want timestamps if you want to roll back to a certain point in time… and you can only do so to the accuracy that is possible - without time there is no accuracy, with time there is something much better.

That there is rough consensus on what time it is on planet Earth, I wonder is not very contentious. Yes, time is a human construct but the consensus on what time is generally agreed can be noted, with all the caveats about what that suggests adopted as with any other quality.

You’re contesting as if timestamps would replace version… but the suggestion is what would be a compound of version+timestamp. So, that is entirely manageable; if there is ever a conflict, then it’s a curiousity.

Files already use creation; modified; and access dates… and those are rough and useful and don’t break code… immutable files would naturally only carry the creation date.

2 Likes

I am contesting that timestamps should be part of the Safe protocols (data or comms)

Have it at the application layer. The browser works at the application layer.

Keep timestamps at the application layer and implement this

That is at the directory layer. And time is still user supplied or server supplied at the application layer

The application storing the web page or data file can supply an optional date/time

Sure but it’s almost semantics.
… the consensus is endorsed by Elders who can be trusted. Not an onerous task…

It’s not the hard drive that does the time stamps on files… it’s the OS.
This is OS level not application level. The Elders are motherboard.

NO its who is doing it. Safe or the application

It’s a matter of consensus, if the Safe Network is significantly rooted in the idea of perpetual or permanence that can be rolled back, then the OS that is Elders perhaps can support that… and only by communicating the confidence in the time… not the actual of timestamping… we want Elders not burdened.

Hey folks. Distributed timekeeping is a difficult problem, but we are not the first to encounter it.

I’ve already pointed out opentimestamps.org, which is a federated timestamping service for the bitcoin network, made by Peter Todd, a bitcoin core developer. He states:

Every Bitcoin block header has a field in it called nTime. For a Bitcoin block to be accepted by the network the Bitcoin protocol requires that field to be set to approximately the time the block was created. Exactly how accurate that field must be for a block to be valid is a complex topic, but for our purposes it’s fair to say it’ll very likely be accurate to within two or three hours - even if a sizable minority of Bitcoin miners are trying to create invalid timestamps - and almost certainly within a day.

Thus we can say Bitcoin is a notary, and we can use Bitcoin block headers as time attestations: proof that a notary that we trust attested to the fact that some data existed at some point in time. That means that if Bitcoin is working correctly these 80 bytes - the block header for block 358,391 - existed on May 28th, 2015:

So, for a first iteration, it could be as simple as consulting the bitcoin network for the latest timestamp in the bitcoin blockchain when publishing page/file data. This would provide a fairly low time resolution, but it would be near impossible for an attacker to inject arbitrarily wrong times, eg yesterday or tomorrow.

Later we could figure out our own timekeeping protocol. But as I understand it, that is not a priority initially.

I understand we may not want any external dependencies. Still, I am throwing it out there to demonstrate that:

  1. Cryptocurrencies have already dealt with this problem to some extent, and likely other projects as well. IPFS? We could learn from those efforts.
  2. We could possibly leverage an external network for timestamping documents only until such time as we develop something native to the Safe Network.
3 Likes

I think you missed the point. It’s not either/or. Obviously the version is the deciding factor. All the timestamp does is to provide additional info that is more natural/intuitive to inquisitive humans when they wonder when the version was published.

This is not what we’re talking about.

Certainly we do not want to root any dependencies but if there is consensus on the time… or just accept what is the difference as +/- error and also note that… then the group can suggest the time as a feature… use at risk. Simpler and faster than looking to an external service. The time sources for OS in an extreme could be compromised for a time and we would not want time to matter that much that it ever affects the way the network operates.

Still, if you want to rollback then time enhances version.

Well said. Enhances, not supplants. The publisher client can provide a UTC timestamp on publication, but this is the least trusted. Once you have 8 timestamps from the close group then you get a more reliable consensus. All we care is that by storing a set of readings it is more reliable than a single timestamp from the publisher alone. Other clients reading the timestamps at a later date can decide how they want to determine reliability of the timestamp group to assign an average or majority rule based standard for publication date/time. There is your app layer. All the core needs to do is collect the raw metadata and associate it with a published version.

PS. Considering the distributed consensus operations going on during the process of Putting data to the network, we can’t expect a single timestamp anyway. It’s more like a time window starting with the client and ending with the last elder or close group member to provide a UTC estimate. Knowing the start of the publication time window to at least the year and day, even if hour or minute are inconclusive, is far better than knowing nothing at all.

If all vaults can be queried for UTC time, and have to sign that, then it can be asserted that the section has an agreed clock that ticks away by an hour (or whatever) and can be queried and verified by any client.

If a vault gets out of step by more than a certain threshold, this can be a signal for unreliability and punishment.

So I think each section could maintain a reasonably accurate clock, at the cost of adding to the load on vaults and Elders, and it becomes a question of whether or not this is a useful thing to implement and if it can be used to support timestamps in the way we’re thinking above.

Maybe with CRDTs it would be possible to lump most of the work in the clients, but I’m not clever enough to think of how that could work.

3 Likes

I was thinking something similar but this may be too harsh… we don’t want to punish good operators just because of a weak CMOS battery.

Edit: If a timestamp provided by a vault is consistently off by some ridiculous margin (many hours, days, weeks, years) compared to the rest of the group it probably should lose age and be punished since it is increasing standard deviation and uncertainty in the publication date by a ridiculous amount according to the close group.

I like “enhance not supplant” point of view. Just collect and store the vault date/time estimates when versions are published and let the Browser or App make sense of them by whatever self consistent statistic the client wants at any later time.

Edit2: Also seems like collecting these timestamps from the section or elders is trivial for CRDT causal order. The actual time stamp of a vault would not be used by the CRDT consensus process, it would only be collected during the process and then stored as metadata.

1 Like

If the goal is to be able to align versions in a rough timeline only, maybe the size of the Safe Network itself could be used instead of actual clock/calendar time, assuming there is a way to keep this metric readily available to the entire network. In other words, if the size of the network at the time of the version upload is n then the version number is n (or a derivative of n). The feasibility of this approach assumes the network will continue to grow which should be a fairly safe assumption and that a running approximation of the size is readily available at all times. The application itself could then convert n to an approximate time by referring to historical records of the growth of the network. There could be some overlap, of course, of version numbers if a page or document was updated rapidly (every few seconds) but even if your search for a previous version that existed on a certain date returned multiple copies, that might still be acceptable for the task at hand.

Reminds of halflife but we don’t need anything complex for simple timestamping. Still, metrics like growth will be interesting for other reasons… and offtopic perhaps lend to economics.

We do not want to root any dependencies on external services… and OS time keeping is that, with many updating against internet ntp services… if there was a glitch in time (measure not real), then network should be unaffected. If one vault is in xorspace at a real distance and some odd lag in the internet or in the time perception there, then that should just be noted as a curiousity not a measure of the quality of the vault.

That said, in law they use the idea of compound … so, if many oddities are arising from a vault, the perhaps the sum of those is a question mark??

There already is a section timestamp similar to a lamport clock.

The Section Key is held in a linked list. Where every key is signed by the previous key.

Data on the network is signed by the key as valid network data.

The section key changes with every Elder change (Elder is 7 Oldest Adults).

So we can say this data was signed 3 keys after this other data.

How much resolution is that though? Well it’s what we can manage right now :slight_smile:

Thoughts follow

More seriously time is weird and does not work very well IMO. If you look at tree rings then there is 1 per year, but some are tight and some have large gaps between then because every year is different. We can extrapolate then every minute/second etc. is different.

On the smallest artificial resolution 2 docs are published, one claims to be first, we cannot tell. However if they are in one of our crdt containers like lseq then we may possibly tell which was first and make that provably so, but how much first by? Do we really care? should we?

So entropy is the universal clock, each tick has more complexity, no two ticks are the same, the ticks are infinitesimally small and cannot be measured.

Anyway I pose these as thought experiments, not statements of fact and leave you with the notion we can provide a “timestamp” based on entropy of one source, Elder groups. We can possibly go further, but not guaranteed and the bigger question is should we?

8 Likes

Potentially there is error beyond just the difference within section elders’ perception of the time, which doesn’t help resolve which of two files was the first. So, two sections might have a slight difference of opinion, aside from the complexities of network lag to query any of that.

Still, it’s unclear if we need to prove exactly… a bit of fuzzy is always nice to tease the ocd carriers :slight_smile:

One route out, is to state an error +/- on each timestamp, which introduces an uncertainty that reflects what is fairly known. So, if the error on both timestamps tempts that one file could have been created before another OR the inverse, then uncertainty can be acknowledged by those making use of it. Otherwise, it’ll be clear which file was created first because the error was not great enough to make that difference. The trivial problem with this is that error is another point of data… but perhaps can still be a single data point if combined into a string of numbers [time+error] is not too much.

1 Like