My side project: a chart that tracks Bitcoin Twitter Sentiment and Tweet Volume

[bitcointweets]

Hi guys,

I'm a student at the University of British Columbia studying Computer Engineering.
I'm interested in using alt data to predict price movements for Bitcoin.

Unfortunately, most of these services are behind a paywall, so I made a free version myself.

The tweet data is pulled from the Official Twitter API. I chart tweet frequency (volume of tweets about Bitcoin per minute) and Tweet sentiment as well.
You can see the Live chart here: Bitcoin Tweets Sentiment Analysis

I also made a Bitcoin News Sentiment and volume chart.
This one pulls data directly from Google's RSS news feed every 10 minutes to track the release of major events that could affect the price.

[LeGaulois]

I have been trying to find out about the AFINN sentiment analysis model, if I understand it correctly it is a list of words containing each its score.
https://github.com/fnielsen/afinn/tree/master/afinn/data

It makes me think about Microsoft Flow (edit: the name isn't Flow but I'm unable to remember it). Does this tool use the number of tweets or more about the feelings expressed?
Or truly a mix of both?

The first question I asked myself concerns all the Twitter bots that auto-post a tweet every XX minutes (using IFTTT or Zapier for example).
The charts could therefore be easily manipulated, by creating hundreds of accounts, collecting subscribers, and then automating the accounts.
It is very easy to create a hundred accounts per day.

I believe 1000 accounts could be enough to manipulate the charts. Currently, there are 15.000 tweets/hours, an account can post every 10 minutes (to avoid the Twitter algorithm) 1.000 accounts x 6 tweets/hours= 6.000 tweets. It's about 40% of the 15.000

How the tool filters this?

[bitcointweets]

I have been trying to find out about the AFINN sentiment analysis model, if I understand it correctly it is a list of words containing each its score.
https://github.com/fnielsen/afinn/tree/master/afinn/data

It makes me think about Microsoft Flow (edit: the name isn't Flow but I'm unable to remember it). Does this tool use the number of tweets or more about the feelings expressed?
Or truly a mix of both?

The first question I asked myself concerns all the Twitter bots that auto-post a tweet every XX minutes (using IFTTT or Zapier for example).
The charts could therefore be easily manipulated, by creating hundreds of accounts, collecting subscribers, and then automating the accounts.
It is very easy to create a hundred accounts per day.

I believe 1000 accounts could be enough to manipulate the charts. Currently, there are 15.000 tweets/hours, an account can post every 10 minutes (to avoid the Twitter algorithm) 1.000 accounts x 6 tweets/hours= 6.000 tweets. It's about 40% of the 15.000

How the tool filters this?

Good Points. You clearly had an in-depth look at the tool.

Yes, that is the correct understanding of AFINN.

The tool does take into account volume of tweets along with sentiment plus a few other factors (Twitter account age and activity is used to filter out fresh accounts).
Also, Twitter is very good at detecting spam. How do I know? Because I actually tried to create a bot that would tweet out current sentiment and the API was very restrictive on how often it would let me post.
There are no explicit rate-limits to posting statuses on twitter, but the algorithm doesn't want you to automate tweets that are too similar to your recent statuses etc.

[LeGaulois]

Pretty smart

I see you used the API from developer.twitter.com that's why you found it very limited. If you go through a third-party application it becomes much less restrictive.

For example, you could write 100 tweets, spin the text 10 times to have 1000 tweets, upload all of them in a WordPress database that will auto-post on a regular basis, you will see much less restrictions.

And yeah, Twitter improved a lot to detect the spams over the years. About the rate-limit...



But as you surely know, it's a case by case depending on the account used

[tranthidung]

Thank you for creating that website with visual real-time updated charts.

Some contributions:

• So far, there are 2 options for time frames: 4h and 24h at Bitcoin News Sentiment Analysis. If you can, I'd like to see the chart for one to 3 months.
• Reasons
  
  • Bitcoin is volatile in beginning, middle or ending of months. A wider time frame can help to see such changes and correlations with twitter sentiment/ volume
• The visual charts
  
  • The animation (in lines) on charts are annoying, in my opinion. You should change it to static.
  • It distract viewers' eyes and challenging to focus on different lines and figure out what is implied behind them (correlations, ie.)
• Data table: how to dowload it (with wider period)?

[bitcointweets]

Thank you for creating that website with visual real-time updated charts.

Some contributions:

• So far, there are 2 options for time frames: 4h and 24h at Bitcoin News Sentiment Analysis. If you can, I'd like to see the chart for one to 3 months.
• Reasons
  
  • Bitcoin is volatile in beginning, middle or ending of months. A wider time frame can help to see such changes and correlations with twitter sentiment/ volume
• The visual charts
  
  • The animation (in lines) on charts are annoying, in my opinion. You should change it to static.
  • It distract viewers' eyes and challenging to focus on different lines and figure out what is implied behind them (correlations, ie.)
• Data table: how to dowload it (with wider period)?

Yes, absolutely. These are my thoughts as well, after monitoring the 24hr chart for a few months. I'm currently developing a monthly and yearly chart. It'll be out very soon.
Interesting point about volatility depending on time of month, I did not know about this.

Yes, that is a very common critique, people do hate the moving lines. I'll make them static.
The reason I animated them was because of a UI book I read but everyone hates it.

I can expose an API that you can use to pull the data as JSON, that should be very simple as I'm already set up to do that.

[bitcointweets]

Pretty smart

I see you used the API from developer.twitter.com that's why you found it very limited. If you go through a third-party application it becomes much less restrictive.

For example, you could write 100 tweets, spin the text 10 times to have 1000 tweets, upload all of them in a WordPress database that will auto-post on a regular basis, you will see much less restrictions.

And yeah, Twitter improved a lot to detect the spams over the years. About the rate-limit...

https://i.imgur.com/JgUZWQh.png

But as you surely know, it's a case by case depending on the account used

Yes, I am currently looking at TWINT etc to get around rate limits.
Right now I'm actually using AWS for the backend. I love the scalability of using microservices for everything. The computing, databases, notification system, REST api, all handled through AWS.

[tranthidung]

I'm currently developing a monthly and yearly chart. It'll be out very soon.
Interesting point about volatility depending on time of month, I did not know about this.

That thread is not exactly answer this question, but somewhat and you can handle it by your side I think. You also can visually detect it with Heikin Ashi candle on trading view, 1D chart.

• Bitcoin price history -- most profitable and bloodiest months, days

Yes, that is a very common critique, people do hate the moving lines. I'll make them static.

It is your good choice and will be a good upgrade for your site.

I can expose an API that you can use to pull the data as JSON, that should be very simple as I'm already set up to do that.

I am not a coder so I can not handle with JSON. It will be good if you dump your data in csv format that is one of standard for machine-readable data.
Another note is Twitter is full of farm accounts so do you exclude them from your analysis. Farm accounts can bias your result a lot.

• Twitter Audit score
• I guess you know it, and if you can include this criteria for your analysis (with or without Audit score -- then with same categories based on Audit scores) it can be great
• I meant, effects from farm accounts and from famous accounts are different
• I don't believe in such tool (it is biased-tool as same as Google Trend) to predict bitcoin price but as you planned to do it, it is my suggestion