Block Chain's Time Stamp is Nonsequential?!

[crabel]

I am developing metrics and macro analytics for bitcoin. As a result I need to access the information contained in the blockchain. This resulted in some interesting findings. I use Abe and checked one point on blockchain explorer.

1. The block chain time stamp is non sequential.  I did not think this was possible. I found this when trying to compute the network hashrate using the (t_stamp_i - t_stamp_(i-1))/600*difficulty*4295032833 to find the network hash rate. When I plotted this I had negative numbers, which if the timestamp monotonically increases is not possible.

As an example, blocks 139793 (time stamp 1312599459) and 139792 (time stamp 1312599808) are not monotonically increasing.

How does one determine the appropriate time stamp? And then, estimate the hash rate for each block?

2. I tried to find an explicit function for finding the difficulty (needed in finding the hash rate) Here I used a ratio of the difference in timestamps for blocks 0 to 2015, 2016 to 4031, etc to the 1209600 seconds that it "should" take to find the 2016 blocks. This gives a multiplier for each time step. I then multiplied every previous multiplier to obtain the difficulty for any particular block.  The result is close to the difficulty reported everywhere (1,180,941,214.99 from Abe for current) ( 1,233,468,670.05 by my method). Intuitively the method I used makes sense (I couldn't find a description online that did not involve hexadecimal (sorry I am a base 10 kinda cat) or written in C (I don't program outside of Matlab). Basically every method I found was unintelligible.  My method relies upon accurate timestamps and any accumulated small error will give an erroneous number, and goes back to my first problem.

Any help/explanation here will be appreciated.

[Cryddit]

Yeah, it's a bit of a mess.

Bitcoin protocol has allowed people whose clocks disagree by up to two hours to participate in the network.  I suppose the idea may have been something to do with clocks disagreeing until people reset them at daylight saving time transitions.

And some folks have tried to take advantage of that one way and another -- I think that at one time it may have been advantageous for miners to have their clocks set differently from the rest of the network or something a bit crazy like that.

So, yes, you're going to get a few later blocks that are timestamped earlier than the blocks they come after.  It didn't really happen that way of course; it just looks like it because some of the people had their clocks set wrong.

Personally I think that was a mistake.  If someone can't be bothered to keep her computer's clock set right, then kick her off the network. 

[deepceleron]

The block timestamp is only part of the Bitcoin protocol because an average of it is used for adjusting the difficulty. Nothing about the timestamp affects the selection or propagation of blocks, except for the two hour rule that other miners use.

[michagogo]

A block is considered valid if the timestamp is not more than two hours in the future, and the timestamp is not before the median timestamp of the past 11 blocks.

[DannyHamilton]

Personally I think that was a mistake.  If someone can't be bothered to keep her computer's clock set right, then kick her off the network.  

And in a decentralized system with no central authority to provide a time server that everyone must synchronize with, who gets to decide which timestamp is "right"?

The blockchain provides sequencing in the absence of a reliable source of synchronized time.  Transactions in more recently added blocks "always" occur after transactions in less recently added blocks, regardless of what "time" they happened.

[crabel]

Thank you all for that information. There are ways of resolving the timestamp in a decentralized manner. when conducting the comparison of the size of the proof of work, each submitted proof of work will have an associated timestamp. Take the average and use that as the timestamp for the longest.  There are also methods of synchronizing the network clock, using the clocktime of participants.

From an analytics standpoint accurate time measurements are very important.

To the second part:

Where can I find the equations that are used to calculate difficulty (base 10 for dummies please) and network hash rate for each block?

Thanks again

[deepceleron]

Thank you all for that information. There are ways of resolving the timestamp in a decentralized manner. when conducting the comparison of the size of the proof of work, each submitted proof of work will have an associated timestamp. Take the average and use that as the timestamp for the longest.  There are also methods of synchronizing the network clock, using the clocktime of participants.

From an analytics standpoint accurate time measurements are very important.

To the second part:

Where can I find the equations that are used to calculate difficulty (base 10 for dummies please) and network hash rate for each block?

Thanks again

The actual equations that calculate the next difficulty are at line 1000 or so of https://github.com/bitcoin/bitcoin/blob/master/src/main.cpp
The difficulty is recalculated every 2016 blocks based on the average time of the last evaluation period, to maintain the rate of six blocks per hour when network hashing rate changes.

The equations you would use to calculate difficulty depend on what your question is.

https://en.bitcoin.it/wiki/Difficulty

Difficulty * 4295032833.0 / 600 = estimated hash rate

Difficulty = 600 / 4295032833.0 * estimated hash rate (hash/s)

Difficulty = 139.6962545641144970431923866272 * estimated hash rate (Ghash/s)

[michagogo]

Thank you all for that information. There are ways of resolving the timestamp in a decentralized manner. when conducting the comparison of the size of the proof of work, each submitted proof of work will have an associated timestamp. Take the average and use that as the timestamp for the longest.  There are also methods of synchronizing the network clock, using the clocktime of participants.

I'm not sure what you mean by "each submitted proof of work" and "when conducting the comparison of the size of the proof of work". As far as the Bitcoin network is concerned, there is no proof of work other than each block. In addition, the timestamp is part of the block header, meaning that it's part of what gets hashed in the PoW process. There are other ways of associating blocks with times, just like there are ways to associate transactions with times, but the block timestamp cannot be relied upon to be completely accurate.

[gmaxwell]

Thank you all for that information. There are ways of resolving the timestamp in a decentralized manner.

"Ways"
...

Miners express their own local view of the time in the timestamps, subject to some constraints of plausibility that don't endanger the decentralized nature of the system. You can freely apply whatever postprocessing you like to these values— or, better, find parties who have already recorded times for them.

It does not do what you want it to do, it does what the system needs it to do.

And some folks have tried to take advantage of that one way and another -- I think that at one time it may have been advantageous for miners to have their clocks set differently from the rest of the network or something a bit crazy like that.

Would you please reduce your level of nonsense emission?

There hasn't ever been such a situation in Bitcoin. Though I believe some altcoins may have self destructed based on trying to "fix" timestamps as you've outlined.

In any case, if you did want to try to produce a precise consensus "real time", though Bitcoin doesn't— and avoids a lot of unnecessary complexity on account of it— you'd still need a mechanism for participants to express their true opinion about the time, and that metric would not be monotonic.

[Cryddit]

And some folks have tried to take advantage of that one way and another -- I think that at one time it may have been advantageous for miners to have their clocks set differently from the rest of the network or something a bit crazy like that.

There hasn't ever been such a situation in Bitcoin. Though I believe some altcoins may have self destructed based on trying to "fix" timestamps as you've outlined.

Glad to hear it.  :-)  As I said, I wasn't sure.  Sorry to be passing on rumors.

Also, it's good to know that I need to pay close attention when I'm doing things that depend closely on time in some altcoin code I'm working on.  I'm considering it completely reasonable that two computers whose clocks disagree by more than about ten seconds should not be able to peer with one another.  And if you do get a 'cannot peer due to time difference' error that should trigger a query to public nntp servers to check the offset from system time. 

[DannyHamilton]

trigger a query to public nntp servers to check the offset from system time. 

Providing a source of "centralization".

If any nntp servers differ, they can split and isolate sections of the network.

Seems like a bad idea.

[Cryddit]

The whole purpose of public NNTP servers is to NOT disagree about the time.  And because there are thousands of them, I rather disagree about them providing a source of centralization.

In the same sense that the purpose of the Bitcoin network is to secure transactions. 

[DannyHamilton]

The whole purpose of public NNTP servers is to NOT disagree about the time.  And because there are thousands of them, I rather disagree about them providing a source of centralization.

In the same sense that the purpose of the Bitcoin network is to secure transactions.

You are assuming that every NNTP server can be trusted.

The whole purpose of bitcoin is to create a way to create a system where you don't have to trust anyone else to engage in a transaction.

[scintill]

Thank you all for that information. There are ways of resolving the timestamp in a decentralized manner.

"Ways"
...

Miners express their own local view of the time in the timestamps, subject to some constraints of plausibility that don't endanger the decentralized nature of the system.

I got a chuckle out of this too (can't be too haughty because I'm no expert myself.)  I'm not sure exactly what crabel was saying here*, but a peer-to-peer proof-of-work hash chain that incorporates timestamps that are supposed to be within some bounds to be part of the winning chain, IS A "way of resolving the timestamp in a decentralized manner."

If you think the clock is wrong, or not monotonic enough, take it up with the entire bitcoin network.  They effectively approve each timestamp as it comes in, by starting another block on top.  Sorry, but I don't think they care much about your analytics.  I started this thought as a joke, but seriously, if you think it's a big deal and think everyone will agree, the window could be made stricter.  But I am pretty sure keeping an accurate clock is pretty much the last thing anybody is going to risk orphaning their mined coins for, or having their transactions reversed or delayed over.  The bitcoin network does not have accurate timestamps as a design goal.

*It sounded like he may have been describing a way he normalizes the timestamp for his purpose, but I'm not sure.