Bitcoin Forum

I have been spending some time trying to understand how much of what we see on a Day by Day / Week by Week basis is variance and how much of it is real hash rate additions or subtractions from the network?

First a word of warning... I am no statistician all of my observations are just that with no mathematical foundation to support them. A lot of the thinking and head scratching has been preparation for Phil's guess the next Difficulty increase contest, so thanks to Phil for running the Contest. Anyway on with the show.

There are plenty of Graphs on the net but having your own helps because you can play with the data & the way it is presented. So here is my graph, very much a work in progress as at each iteration you have some ideas, some of which work and some do not. It probably raises more questions than it answers, but it has helped me get to grips with what is happening on a short term basis.

http://www.slotforum.com/forums/uploads/1453650842/gallery_2150_2322_65864.jpg

So to explain it covers the current 14 day difficulty period  with the Block number across the top, Days across the bottom.

The Blue graph is a Block by Block cumulative hash rate change for the 14 day period read from the scale on the Right, so at the end of the period it will be showing the % increase. Period is not over yet so do not pay too much account to the projection....
The Purple graph is the same except that each Day it is reset showing just the Days % increase and is read from the left hand scale. First few points again very wild until the average builds.
The vertical Yellow lines are the Day markers & the Triangles mark the Days % increase.  

I have tried but chosen to not use moving averages as often the data dropping off the back has more effect than the new more interesting data on the front. So here are some of my observations.

There appear to be 3 phases. Phase 1 Days 1-5 Hash rate drops and then recovers back to where you would expect around zero.
Phase 2 Days 6-9 Hash rate is stable varying but close to zero.
Phase 3 Days 10-14 Hash rate Increases

Finally to kick things off a few thoughts / possibilities for each of the phases.

Phase 1

So the first question is there really a drop off or is this just network variance? My feeling is that is too large to be entirely variance, some of it will be as we have very few data points, resulting in some wild points initially as particularly shown in the first partial Day.

So the theory I am going to suggest for phase 1 is that if you are managing a large mine and you have some major work to do, removing old miners, making significant infrastructure changes that involve down time etc. Then given the choice between doing these tasks at the end of a Difficulty period or the beginning, you would make the most of the lower difficulty and delay the work until the new period.

Phase 2

A period of apparent stability, small variance to the hash rate which I think could be explained as being normal network variance.

Phase 3

On Day 10 Hash is added to the network, yes of course there is variance in this but looking at the daily figure it indicates that as much as 20% has been added to the network hash rate which then of course is there each day until the end of the period.

Finally a thought that to some degree contradicts the above, and may be blindingly obvious to some but was not to me....

If the network was otherwise stable and Hash was added at the beginning of a period, say 12%. Then that would result in a 12% increase to the measured Hash rate for that period and when the Difficulty change was made we would be back at zero.

However, and here is where it contradicts phase 1. If the 12% was added half way through the 14 Day period then it would only result on a 6% increase in difficulty.
Finally and here's the key point. You would be carrying into the next difficulty period an uncorrected for 6% to kick off the next period?

And on that though I welcome any thoughts you have?


Rich

I think in Phase 1 the drop was caused because the price crashed due to those comments by the Dev. With the diff up and lower price, some S3 maybe where turned off.

Yes that is a good point and I am sure it will have had an impact and helped the dip. However..... I am preparing the data, quite a lot of work, for the previous 14 day period in the same format and it bears some striking similarities to the graph above.

Rich

Here is the same graph for the previous 14 day period (Dec31 - Jan12) As you can see there are a lot of similarities.

http://www.slotforum.com/forums/uploads/1453650842/gallery_2150_2322_28192.jpg

The same 3 phases, however on entering phase 2 we go straight into additional hash / positive territory of around +4% and then further hash added in phase 3 taking us through to an exit at just over +9%

Other than the similarity, no further thoughts at this end for the moment as I have only just got all the data in, however thought I would put it up for others to see.

I have started on the Data for the previous 14 days (Dec19 - Dec31), this is the period with the Day that we say +42% and 205 Blocks. From what I have done so far the overall graph is going to look quite different from the two above...

BTW I accept that there may be nothing in this exercise, and I suspect it's not possible to predict the future, however it may facilitate a better understanding of the past?


Rich

Interesting topic!

Would it be possible to compare this data to different pools hashrate?
aka can you see which pool are mainly affected by drop in hashrate after difficulty increase? or is it evenly distributed?

Yes good idea, it's on my list of things to try, especially around the times that there is significant change. The Spreadsheet I have put together with the Block Timings does also include who found the Block so should be possible at some point.

Rich

Few remarks:

1. How the variance works in reality can be observed at antpool stats. With their 200+ PH/s in the pool their results have significant statistical relevance. For example their daily luck chart here https://www.antpool.com/poolStats.htm shows that daily results oscilate between 80% and 120% almost every 5th day. These 20% oscillations are purely variance based and can be interpolated to whole network and can easy answer those hundreds petahashes jumps in global daily hashrate calculations.

2. Theoretical background to variance is nicely described here, but is not that easy to read:
http://organofcorti.blogspot.sk/2015/07/faq-bitcoin-mining-and-luck-statistic.html

Hi thanks for the comments and links, will take a look later. I have this feeling that more of what we see is just variance based than people give it credit for. I am just blundering around here a bit, but have never let lack of knowledge or experience get in way of trying things and gaining some first hand experience.

Rich