USD/BTC Decoupling from forum activity and search trends

[ElectricMucus]

So does that mean the current price increase is of a deflationary nature?

[1714143170]

1714143170

[1714143170]

1714143170

[bb113]

How about treating the search hits as a first derivative of the price?  And maybe superimpose that over the inflation.

Sounds interesting. Can you give an example of how to do this?

Sorry - I have no idea.  I never did this stuff.  It comes from the intuition of a model where new people buy bitcoins and keep them - driving the price up (the assumption that the price would go up in proportion of how much bitcoins are bought is not very sound though), and the opposite force of the supply of newly mined coins.

sounds like a solid theory to me.

By the way 123, how did you get this data?
Might be interesting to do pearson correlation as well: http://www.alcula.com/calculators/statistics/correlation-coefficient/

I've been learning how to harvest data into R. Its just a couple scripts of downloading data and Regex. I'd post it all but its really ugly right now. Here is the forum harvester if you wanna check it out. It should just run and give you a table if you install the packages at the top.

Code:

#install.packages("RCurl")
#install.packages("XML")

require(RCurl)
require(XML)


forum2=NULL
for(y in 2009:2013){
for (m in 1:12){

start<-paste(month.name[m],y)
if(m==1){
end<-paste(month.name[12],(y-1))
}else{
end<-paste(month.name[m-1],y)
}

date<-as.numeric(paste(y,m,sep=""))

forum<-getURL(paste("https://bitcointalk.org/index.php?action=stats;expand=",
date,"#",date,sep=""),
ssl.verifypeer = FALSE, useragent = "R"
)
forum<-readLines(tc <- textConnection(forum)); close(tc)
pagetree <- htmlTreeParse(forum, error=function(...){}, useInternalNodes = TRUE)
 
x <- xpathSApply(pagetree, "//*/table", xmlValue)  
x <- unlist(strsplit(x, "\n"))
x <- gsub("\t","",x)
x <- sub("^[[:space:]]*(.*?)[[:space:]]*$", "\\1", x, perl=TRUE)
x <- x[!(x %in% c("", "|"))]

if(any(x==start) & any(x==end)){
temp<-as.data.frame(rbind(x[which(x==start)[2]:(which(x==end)[2]-1)]))

out=NULL
for(j in 2:(length(temp)/6)){
n<-seq(1,length(temp),by=6)[j]
temp2<-as.matrix(temp[n:(n+5)])
out<-rbind(out,temp2)
}

}else{
out<-matrix(rep(NA,6),nrow=1,ncol=6)
}

forum2<-rbind(forum2,out)

}
}
forum2<-forum2[!is.na(forum2[,1]),]
colnames(forum2)<-c("Date","New Topics","New Posts","New Members","Most Online","Page Views")

forum<-as.data.frame(forum2)
for(i in 2:6){
forum[,i]<-as.numeric(as.character(forum[,i]))
}

head(forum)

I'm not sure about using pearson's R. Residuals are all over the place. I'm trying to figure out how people usually analyze these things.

[phelix]

[...]

A fixed amount of interest generates a larger price increase because new coins are flowing at a lower rate now?

[...]

this. 25 coins instead of 50.

but it seems like something changed on the forum, too:



note how the posts/member figure drops mid October. when was the watchlist feature added?

[Spaceman_Spiff]

So does that mean the current price increase is of a deflationary nature?

What do you mean?

I've been learning how to harvest data into R. Its just a couple scripts of downloading data and Regex. I'd post it all but its really ugly right now. Here is the forum harvester if you wanna check it out. It should just run and give you a table if you install the packages at the top.

Ah k, I am not yet familiar with R (only heard about it), but thanks for making the info available.

I'm not sure about using pearson's R. Residuals are all over the place. I'm trying to figure out how people usually analyze these things.

My understanding of Pearsons r is very basic to be honest.  The closer to 1, the stronger the correlation, that's about the extent of my knowledge :-) .  What else were you thinking of?  Other normalisation or something like that? 
It would of course be more interesting to have a number that showed the "predictive power of future price" somehow. Not sure if this exists.

By the way, in my edited previous post I suggested using an integral, but given that you are working with discrete data, that would just be the sum (of the wikipedia pageviews etc.).

[thezerg]

Maybe it means you no longer have to be an enthusiast to use bitcoins.  Probably a combination of that and bigger investors entering the picture.

[da2ce7]

anodically, I believe the post quality has never been higher on bitcoin talk than now.

[notme]

anodically, I believe the post quality has never been higher on bitcoin talk than now.

As long as we aren't going by spelling

anecdotally- Based on casual observations or indications rather than rigorous or scientific analysis

[ElectricMucus]

So does that mean the current price increase is of a deflationary nature?

What do you mean?

That the prices of BTC increase more because people do not spend their coins rather than more people using coins.

[organofcorti]

Have you tried to assess auto- and cross correlations between the variables? You can do that using the R functions acf() and ccf ( in library(stats) ). You might find there's a lagged correlation there, or perhaps the independent variables are not so independent, or something else that's more complex than a simple linear relationship.

Lots of good info on ARIMA models here:

https://onlinecourses.science.psu.edu/stat510/

I used this method to model the network hashrate and difficulty. More info here:

http://organofcorti.blogspot.com/2012/11/102-forecasting-network-hashrate.html

but it's a bit light on detail.

[bb113]

Have you tried to assess auto- and cross correlations between the variables? You can do that using the R functions acf() and ccf ( in library(stats) ). You might find there's a lagged correlation there, or perhaps the independent variables are not so independent, or something else that's more complex than a simple linear relationship.

Lots of good info on ARIMA models here:

https://onlinecourses.science.psu.edu/stat510/

I used this method to model the network hashrate and difficulty. More info here:

http://organofcorti.blogspot.com/2012/11/102-forecasting-network-hashrate.html

but it's a bit light on info.

I tried using auto.arima (package forecast):



The results don't seem awfully informative. Maybe I didn't use it correctly. I'll check out your link.

[organofcorti]

Rather than using ARIMA models to forecast, try using auto and cross correlative methods to determine if there are lagged relationships between any of the time series variables. then you can use lm() to create a linear model (an example is in the first link somewhere)

[bb113]

Rather than using ARIMA models to forecast, try using auto and cross correlative methods to determine if there are lagged relationships between any of the time series variables. then you can use lm() to create a linear model (an example is in the first link somewhere)

I see what you mean with this. Have you ever used breakpoints? I was thinking rather than model price as a function of the search indicators, it would be better to instead model the probability of breakouts in either direction. Then each spline (or whatever it would be called) between the breakpoints would be an independent sample.

A library of these could be built up over time and searched for multidimensional clusters of slope, preceding and contemporaneous search indicators, variance, autocorrelation, volume, etc. It may then be possible to match one of these clusters to the current trend since the last breakpoint and thus allow improved forecasting. I'm not really familiar with this field so maybe there is a name for this already.



Red points are breakpoints, the heatmap is in the style of https://ferroh.com/charts except its trades per day (mt gox) rather than volume.

[organofcorti]

I see what you mean with this. Have you ever used breakpoints? I was thinking rather than model price as a function of the search indicators, it would be better to instead model the probability of breakouts in either direction. Then each spline (or whatever it would be called) between the breakpoints would be an independent sample.

I've never used breakpoints, but it's an interesting thought - breaking the sample into subsamples based on the breakout points. I don't know enough about it though.

I wonder how trades per day correlates with the price after the breakout? I also wonder if trades per day is also another actor on the network hashrate, modifying the effect of price. Interesting questions there.

[bb113]

[...]

A fixed amount of interest generates a larger price increase because new coins are flowing at a lower rate now?

[...]

this. 25 coins instead of 50.

but it seems like something changed on the forum, too:



note how the posts/member figure drops mid October. when was the watchlist feature added?

.

How was the filtering done?

Also you should edit your post or do this in the future so the chart fits on normal monitors:

Code:

[img width=900]http://blockchained.com/stuff/bitcointalk_stats.png[/img]

[dree12]

Also you should edit your post or do this in the future so the chart fits on normal monitors:

Code:

[img width=900]http://blockchained.com/stuff/bitcointalk_stats.png[/img]

This is a very stupid way of doing this. Rather than restricting the size on the image, get theymos to do it on the stylesheet.

Code:

IMG {
  max-width: 100%;
}

Then it will fit on all monitors and also won't be artificially shrunk on widescreen monitors.

[Raize]

I guess Bitcoin must not be the future of money then.  What are we all doing here?

Best post in thread, IMHO.

The fact is, if Bitcoin was adopted outside of this forum (which a lot of these indicators seem to be based on), it would still mean an increase in price. As another person here says:

Larger capital holders getting in (fewer searches per buck)?

It could be Bitcoin has attracted the attention of people that don't *want* to post.

[organofcorti]

I'm not sure about using pearson's R. Residuals are all over the place. I'm trying to figure out how people usually analyze these things.

I missed this before -sorry. Try looking for log(price) ~ hits or diff(log(price)) ~ diff(hits) relationships. Also the non-price variables are probably not independent from each other, so it might be asking a lot to get a simple correlation there.

If you don't mind using p values then you could try some linear models and associate the lowest p-vals with the most useful variables.

[notme]

Also you should edit your post or do this in the future so the chart fits on normal monitors:

Code:

[img width=900]http://blockchained.com/stuff/bitcointalk_stats.png[/img]

This is a very stupid way of doing this. Rather than restricting the size on the image, get theymos to do it on the stylesheet.

Code:

IMG {
  max-width: 100%;
}

Then it will fit on all monitors and also won't be artificially shrunk on widescreen monitors.

That would be ideal, but until the forum gods can oblige us with better stylesheets, please do the "very stupid way of doing this".

[phelix]

[...]

How was the filtering done?

simple iir. filter constant 512. data points every 17-18mins

Also you should edit your post or do this in the future so the chart fits on normal monitors:

Code:

[img width=900]http://blockchained.com/stuff/bitcointalk_stats.png[/img]

hmm thanks for the hint.

[solex]

bitcoinbitcoin113

Impressive analysis. Yet these metrics appear to show recent exponential growth which may be a good proxy for interest in / take-up of BTC...

http://blockchain.info/charts/my-wallet-n-users
http://blockchain.info/charts/my-wallet-transaction-volume
http://blockchain.info/charts/my-wallet-n-tx