RFC for timestamping in SAFE

[digipl]

New RFC about timestaping in the SAFE Network

Thierry61/rfcs_dates_in_SD/blob/7cd96dba914f9f1b5bbe7f658bb45b5dd80f8c41/proposed/0000-structured-data-timestamping.md

[anon81773980]

Awesome! That’s gonna help so much. I was actually about to make a thread about time stamping, I been thinking about it in the last few days.

[Anders]

Impressive. That’s a really powerful feature. Can it be used for replacing blockchain functionality?

[polpolrene]

Great work! This is how we could compete with blockchains.

[fergish]

Very, very cool. Look forward to this getting sorted and included.

[polpolrene]

Thinking about it, would it cost a lot of extra overhead data to timestamp all PUT’s?

[smacz]

This does not rely on any other input other than the device’s own system clock? (The embedded one that runs on the CMOS battery)

Also, there is no verification of this time?

[polpolrene]

You verify with 32 nodes.

I personally checked at work (12 stations: servers and desktops) and at home (9 devices: desktops, laptops, tablets, smartphones) and they are all in the range of actual time ± one minute, which is more than enough.

So when there’s consensus that the date is October 17th and the time is 21:24 that would be enough to sign a Chunk/File or whatever.

[janitor]

Maybe there’s an RFC… oh wait.

This is possible because most devices are already approximatively synchronized in the range of a few minutes and timestamping of documents doesn’t need to be precise.

Concretely, to handle the date variability among the nodes, the client defines a date range in the SD. During the validation consensus a manager has only to check that its own date is inside this range.

I don’t know how the manager is selected (can it be selected if it has wrong time and who decides). And what happens if wrong system time is set after it’s been selected.

[smacz]

@janitor

During the validation consensus a manager has only to check that its own date is inside this range.

That’s what I was looking for, thank you.

[tfa]

Following my analysis in this post:

Any time consensus within SAFEnet?

The problem is that maintaining a global time across the network adds another complex layer in the core. A much simpler solution which would need a minimal modification of the core is to add the following fields in StructuredData: min_date: i64 (default value 0) max_date: i64 (default value std::i64::MAX) The i64 type represents the sec part of a struct Timespec generated from an UTC time (a number of seconds since the beginning of the epoch). Each nodes in the data manager group handling a …

I implemented this functionality in routing crate and sent a PR because it was so simple. But then David indicated that a RFC was needed, this is the reason why things are in the wrong order (I did the PR before the RFC).

Let us hope that MaidSafe team doesn’t find anything blocking in this proposal.

@janitor

I don’t know how the manager is selected

The managers are the standard ones, either:

• The group managing the client (around client name in XOR space)
• The group managing the SD (around SD name in XOR space)

(or both)

I searched for the call sites of functions checking the signatures of a SD and didn’t find them. The function checking the date must probably called at the same places (that I didn’t find).

[tfa]

@polpolrene

Thinking about it, would it cost a lot of extra overhead data to timestamp all PUT’s?

The cost is 16 bytes in a SD. Immutable data is not concerned.

@polpolrene

So when there’s consensus that the date is October 17th and the time is 21:24 that would be enough to sign a Chunk/File or whatever.

Suppose that actual UTC time is 2015/10/17 21:24 and it is also the client time. If the client needs to timestamp a SD with a 10 mn precision then it will specify an interval from 2015/10/17 21:19 to 2015/10/17 21:29. The PUT will succeed only if at least 28/32 of nodes in the managing group have a time inside this range.

Analyzing the devices at my disposal this precision seems easily reachable. But we don’t really need such a precision. A one hour precision would be enough and is even more reachable. For this precision he would specify an interval from 2015/10/17 20:54 to 2015/10/17 21:54

[Anders]

@tfa

A one hour precision would be enough and is even more reachable.

A 10 minute precision would be possible too I imagine. The system clocks in the clients probably almost always have a correct UTC time within some minute I would guess.

[tfa]

@Anders

A 10 minute precision would be possible too

I think so and this was my first example.

@Anders

The system clocks in the clients probably almost always have a correct UTC time within some minute I would guess

It is in the interest of the client to precisely synchronize its system clock.

[Anders]

@tfa

It is in the interest of the client to precisely synchronize its system clock.

And today’s operating systems probably have automatic clock synchronizations. Even smartphones. Linux may require a manual configuration like (unless it’s a default setting): UbuntuTime - Community Help Wiki But Linux users are usually more tech savvy than Windows and OS X users.

[davep]

Nicely done. I like the idea of natural network convergence on more accurate times.

I would suggest one change to the with_dates constructor though. If everyone is supposed to subtract a little for min_date and add a little for max_date, just accept a “current_time” and “range” (or “acceptable_variance” or whatever…) value and do the math for them. Then it is obvious what the use is, everyone should just get it right without thinking about it, and getting it wrong would almost have to be intentional.

[neo]

@polpolrene

Thinking about it, would it cost a lot of extra overhead data to timestamp all PUT’s?

I read it as only timestamping when requested.

If its for every SD, then maybe the RFC could be changed to only specific SD types. That way it would reduce overheads. No need to have it for every SD types. Or just when requested.

Remember also that SD data costs the same as one chunk (that is 10x the PUT cost) so there is room for account for the additional overheads.

[tfa]

@davep

I would suggest one change to the with_dates constructor though. If everyone is supposed to subtract a little for min_date and add a little for max_date, just accept a “current_time” and “range” (or “acceptable_variance” or whatever…) value and do the math for them.

This would restrict possible usage of asymmetric dates:

• People who only need a proof that a document was stored before a certain date can set the default value for min_date (0).
• One could take into account the estimated time for the request to reach the managers. For example someone targeting a 10 s precision, could try the following interval: [current time - 4 s … current time + 6 s]

The math is better done in the client and it’s only a matter of HMI. A simple one could display a few radio buttons:

• no timestamping
• 1 hour precision
• 10 mn precision
• 1 hour precision (max date only)
• 10 mn precision (max date only)

@neo

If its for every SD, then maybe the RFC could be changed to only specific SD types. That way it would reduce overheads. No need to have it for every SD types. Or just when requested.

This would suppress the usage of the type_tag for timestamped data, and:

• We would like to be able to timestamp a standard directory.
• Most use cases for timestamping need their own type_tag.

The cost is minimal in terms of:

• Data size: 16 bytes over 100KB (that’s 0.016%)
• CPU: a call of a function doing a simple if test (this function can be optimized by not reading the UTC time when min_date/max_date fields have their default values).

[polpolrene]

@tfa

Analyzing the devices at my disposal this precision seems easily reachable. But we don’t really need such a precision. A one hour precision would be enough and is even more reachable. For this precision he would specify an interval from 2015/10/17 20:54 to 2015/10/17 21:54

For things like a forum etc. you might even want to have a 1 min. precision. Don’t know if that could be done. But when Safecoin transactions take place under a second, time verification should be possible in a short range as well I think.

[davep]

@tfa

This would restrict possible usage of asymmetric dates:

People who only need a proof that a document was stored before a
certain date can set the default value for min_date (0).

One could
take into account the estimated time for the request to reach the
managers.

OK, I get this as an engineer, but what’s the practical use of selecting “no min date” vs “min date”? If the concern is that there may be enough variance that the timestamp is rejected, don’t you have that same concern on the max_date side as well?

If “one” needs to take into account request travel time, don’t we all?