Price drives difficulty

[Fcx35x10]

now that the rewards halved. waiting for the price hike. any speculations on the chain of effects?

[silverfuture]

Difficulty may not affect price but supply does. Block reward halving reduces supply. Demand is another story.

[crazyates]

now that the rewards halved. waiting for the price hike. any speculations on the chain of effects?

Don't expect it to hike to $20 overnight.

[organofcorti]

Here we go my friend (example chart of the data, which is "difficulty and hashrate by day"):

......


^{donations: 1MoLECHau3nb9zV78uKraNdcZsJ2REwxuL}

Sweet! I'll give it a test run when I get home, see how close my interpolated values were, and redo some network hashrate predictions.

Thanks for sharing your data.

Nice, glad you're happy! Looking forward to your updates.

I gave it a test run last night, after converting it to weekly averages. It looks good, and I didn't see any obvious difference between it and my data. However when I checked the ccf, acf and pacf again, I found a weak autocorrelation at lag 11. The pacf spike was also there in my dataset but it wasn't significant. In your dataset it became significant, and when adding a lag11 term for the network hashrate to the lag1 linear model, I got a decrease in the 95% CI for %error of about 2%.

I had also changed the way I bound weekly average network hashrate to weekly median price to something that was more accurate, so this may have had an effect on the linear model's accuracy too.

I can't be certain, but I think the model based on your data is likely more accurate as a predictor since you based it on actual rounds per day rather than (as I did) an interpolation between groups of 144 blocks.

I also completed forecast models for up to four weeks. I'll post the results tonight if I get some time. The forecast models have surprisingly good accuracy if there are no significant changes to conditions, less so when there are large changes in hashrate or price.

[molecular]

I gave it a test run last night, after converting it to weekly averages. It looks good, and I didn't see any obvious difference between it and my data. However when I checked the ccf, acf and pacf again, I found a weak autocorrelation at lag 11. The pacf spike was also there in my dataset but it wasn't significant. In your dataset it became significant, and when adding a lag11 term for the network hashrate to the lag1 linear model, I got a decrease in the 95% CI for %error of about 2%.

I had also changed the way I bound weekly average network hashrate to weekly median price to something that was more accurate, so this may have had an effect on the linear model's accuracy too.

I can't be certain, but I think the model based on your data is likely more accurate as a predictor since you based it on actual rounds per day rather than (as I did) an interpolation between groups of 144 blocks.

I also completed forecast models for up to four weeks. I'll post the results tonight if I get some time. The forecast models have surprisingly good accuracy if there are no significant changes to conditions, less so when there are large changes in hashrate or price.

If I understand correctly the data seems good and the higher resolution (or linear time) even has a slightly positive effect on your model?

I'm playing around myself, still stuck at the early stages, though. What correlation coefficient do you use? Pearsons Correlation?

[organofcorti]

If I understand correctly the data seems good and the higher resolution (or linear time) even has a slightly positive effect on your model?

I was having interpolating from sets of 144 blocks to weekly data. It's good enough, but your dataset is easily converted to weekly averages, and yes, the models seem more accurate when I use it. there's a (weak) autocorrelation for network hashrate at lag11 which was not significant for my dataset, but does seem to add accuracy to the models. I've writen a post using your dataset it here, and the post on the forum about it is here.

I'm playing around myself, still stuck at the early stages, though. What correlation coefficient do you use? Pearsons Correlation?

For the network Difficulty / price correlation, yes. However for the network hashrate / price correlation (which involves lagged variables) I used the autocorrelation function (acf), partial autocorrelation function (pacf) and the cross correlation function (ccf). You can learn more about them here.

Thanks for your help. I'm sorry the donation was paltry, but I did put your address at the end of the "Long range forecasts" blog post at organofcorti.blogspot.com, so perhaps some kind soul will add to that.

[molecular]

If I understand correctly the data seems good and the higher resolution (or linear time) even has a slightly positive effect on your model?

I was having interpolating from sets of 144 blocks to weekly data. It's good enough, but your dataset is easily converted to weekly averages, and yes, the models seem more accurate when I use it. there's a (weak) autocorrelation for network hashrate at lag11 which was not significant for my dataset, but does seem to add accuracy to the models. I've writen a post using your dataset it here, and the post on the forum about it is here.

I'm playing around myself, still stuck at the early stages, though. What correlation coefficient do you use? Pearsons Correlation?

For the network Difficulty / price correlation, yes. However for the network hashrate / price correlation (which involves lagged variables) I used the autocorrelation function (acf), partial autocorrelation function (pacf) and the cross correlation function (ccf). You can learn more about them here.

Thanks for your help. I'm sorry the donation was paltry, but I did put your address at the end of the "Long range forecasts" blog post at organofcorti.blogspot.com, so perhaps some kind soul will add to that.

Just read through your blog post. Thanks a lot for your donation, for mentioning me in the post and for the pointer about the correlation functions .

[organofcorti]

"Price" does indeed "drive difficulty"
and an analysis of correlation coefficients tells us that the "price" is the log of the volume weighted average USDBTC price from two difficulty periods previously, and difficulty is actually the log of D.

http://m.xkcd.com/552/

You didn't read that blog post or any of the following posts, did you? Please read the following quote from the subsequent blog post:

from http://organofcorti.blogspot.com/2012/11/102-forecasting-network-hashrate.html :

0. Introduction
In the last post I showed in a simple way that changes in the MTGOX US$-BTC exchange rate correlated well with later changes in the mining Difficulty. Since there have been some misunderstandings on the subject I want to explain the correlation in a bit more detail before starting on forecasting the network hashrate.

Clearly, the exchange rate is not influencing mining difficulty directly, but through an intermediate process:

US$-BTC  ->  cost per hash and earnings per hash  ->  network hashrate  ->  mining Difficulty

Of course, this assumes that there are no possible changes to the cost per hash or earnings per hash, or the network hashrate other than the MTGOX US$-BTC exchange rate. Some of the factors could be:

Technology changes that make the cost per hash lower or the earnings per hash higher, or both -  for example the change from CPU mining to GPU mining, to a lesser extent FPGAs, and to a much greater extent, the upcoming ASIC mining devices. This would likely increase the network hashrate without a corresponding increase in the MTGOX US$-BTC exchange rate.
An influx of new miners due to publicity. If new bitcoin enthusiasts start to mine rather than buy, the network hashrate will increase without a corresponding increase in the MTGOX US$-BTC exchange rate.
Non-profit driven miners.  A given miner that mines at difficulties beyond which a return can be made could be mining irrationally - either due to the assumption that the local currency - BTC exchange rate will rise significantly, not wanting to turn off expensive equipment that has yet to pay for itself, or perhaps a less irrational reason such as wanting to contribute to network security.
Changes to local electricity costs - price per kWh is unlikely to decrease, which means cost per hash is likely to increase, probably in the near term for many miners. This would, if a large enough increase, prevent the network hashrate from expanding beyond some maximum beyond which it would only be profitable for those not paying for electricity to mine,  without a corresponding decrease in the MTGOX US$-BTC exchange rate.

The first has already occurred, but does not seem to have had an obvious effect on network hashrate that could not be predicted by the exchange rate. This may be because the cost of purchasing a GPU was sufficiently prohibitive that an increase in  MTGOX US$-BTC was required to make it possible. ASICs however are much cheaper to purchase (in terms of $/hash) and much cheaper to run (hash/joule) and should cause a significant discontinuity in the correlation.

The second possibility has certainly occurred at times. This however does not seem to have caused an increase in difficulty that could not be predicted by the exchange rate.

I am uncertain as to the likelihood of the third possibility having occurred. It would be a good explanation for the distinct lack of the mining Difficulty's response to the MTGOX US$-BTC crash after June 2011. If correct, mining difficulty should reduce much more slowly than MTGOX US$-BTC.

The last is yet to occur, but will at some point in the not too distant future. After this point, the only increases in network hashrate not due to publicity or non-profit driven miners will be either incremental as technology gradually improves efficiency and cost, or the exchange rate.

My original intention for this post was to find out if any of the first three were had affected mining difficulty in any significant way, or at least find specific points at which there was a significant departure from the MTGOX US$-BTC correlation and determine a possible reason.
........

[organofcorti]

You didn't read that blog post or any of the following posts, did you?

Nope.

Please read the following quote from the subsequent blog post:

No, because I just wanted to post a funny comic.

It would be nice if you posted your of topic comments in the off-topic thread. I can help you if you're not sure how to start a topic there

[organofcorti]

Sorry, I'll just remove them then.

Now you make me feel like a censor, which was not my aim. Plus I do love XKCD and I liked the cartoon to which you linked.

But your link did imply I was somehow wrong - I don't mind that at all since I am often wrong, but an off-hand link implying I meant something I didn't was just confusing.

[organofcorti]

I'm now including Difficulty estimates up to 3 retargets in advance at the "Long range forecast" thread:
https://bitcointalk.org/index.php?topic=125171.msg1441129#msg1441129

And of course on my blog:
http://organofcorti.blogspot.com/2013/01/weekly-network-forecast-7th-january-2013.html

Here's a taste:

Enjoy!