Machine Learning (Keras LSTM and K-Clustering) to predict Bitcon Price

[Vlad.Sas]

I'm trying for some time now to develop a neural network that can predict bitcoin prices. Bitcoin is a great asset for such an experiment because it has fewer fundamentals than the other stocks or derivatives and it relies more on technical analysis than other assets. Nevertheless, it is impacted from time to time by news and other events in the crypto-world.

I am working with TF + Keras and LSTM (Long-Short Term Memory) cells to try and predict future prices.

If I let the network guess freely, without adjusting it to each OHLC Tick, the predictions look like this:




If i adjust it at every point in the dataset, it is needless to say that the graph of predictions is way more precise:


There are few steps I would like to take to improve this

• Factor in Technical indicator (BB, Ichimoku Cloud, SAR, etc...
• Place a factor for fundamental analysis in the dataset (for example, a score for the positivity of the news at that specific tick (0=bad news for btc, 10=great news)
• Collect data from the order books of the largest exchanged and factor in the order book states at every tick
• Unsupervised learning and K-Clustering to find co-relations between bitcon and other foreign currency (for example, it might be that there's an inverse relationship between bitcoin and the Yuan (generally when Yuan drops, Bitcoin rises). K-Clustering can find such relashionships
• Merging all of the above into a trading bot that aims for around 30%-40% return a year

Does anyone pursue such a goal? If yes, would you like to share the accumulated knowledge/ideas?
This is a serious project and currently, only me and a two of my partners are involved in it, but we could sure use some more man-power. I'd also like to know if someone would like to fund such a venture and get involved.

Cheers to you all!

[1713459612]

1713459612

[hugeblack]

I remember that one of the members develop such a neural network using a train of images that were taken periodically and at a high rate to track price patterns ---> https://bitcointalk.org/index.php?topic=5213806.0 Check it here ---> https://vafed.it/signals.php.

The only thing I disagree with is that following historical patterns without taking other factors, news, hacking, more altcoins, and others will lead to wrong measurements.

You added a good amendment by including the news as an influencer, but I think this is an area for research as it has to learn and then predict how and the effect of such news on prices ---> if it is correct then see its effect on the price.

[bitmover]

I think this ai can predict price in common situations... but life isn't common all the time.
Would it predict corona virus? Oil crisis?

Market cannot be predicted,  because life can't.

[Vlad.Sas]

I think this ai can predict price in common situations... but life isn't common all the time.
Would it predict corona virus? Oil crisis?

Market cannot be predicted,  because life can't.

It could if you include fundamental analysis, but you cannot predict it exactly, ofc.

[bitmover]

Would it predict corona virus? Oil crisis?

It could if you include fundamental analysis, but you cannot predict it exactly, ofc.

Nobody predicted corona virus. Things like that are humanly impossible to predict.
This is not related to fundamental analysis of bitcoin, but real world issues that affect everything.

[Vlad.Sas]

Would it predict corona virus? Oil crisis?

It could if you include fundamental analysis, but you cannot predict it exactly, ofc.

Nobody predicted corona virus. Things like that are humanly impossible to predict.
This is not related to fundamental analysis of bitcoin, but real world issues that affect everything.

That's not quite right. Markets reacted in march to covid. For example i managed to short few airlines before the crash. Markets don't react instantly

[davis196]

Would it predict corona virus? Oil crisis?

It could if you include fundamental analysis, but you cannot predict it exactly, ofc.

Nobody predicted corona virus. Things like that are humanly impossible to predict.
This is not related to fundamental analysis of bitcoin, but real world issues that affect everything.

You can't predict when a virus will mutate and spread,but you can predict how it will spread.
I think that with machine learning,you can predict the Bitcoin price,but only in the short term,because short term price fluctuations are partially based on past price performance.By "past",I mean in the last few hours.
The Coronavirus pandemic surprised everyone,but there was a possibility that such a new virus will sooner or later infect a large part of the human population.You can't predict the exact event,but you can somehow measure the possibility/probability of that event happening.Viruses and bacteria are constantly mutating and evolving,we can't stop them from going that.

[Vlad.Sas]

Would it predict corona virus? Oil crisis?

It could if you include fundamental analysis, but you cannot predict it exactly, ofc.

Nobody predicted corona virus. Things like that are humanly impossible to predict.
This is not related to fundamental analysis of bitcoin, but real world issues that affect everything.

You can't predict when a virus will mutate and spread,but you can predict how it will spread.
I think that with machine learning,you can predict the Bitcoin price,but only in the short term,because short term price fluctuations are partially based on past price performance.By "past",I mean in the last few hours.
The Coronavirus pandemic surprised everyone,but there was a possibility that such a new virus will sooner or later infect a large part of the human population.You can't predict the exact event,but you can somehow measure the possibility/probability of that event happening.Viruses and bacteria are constantly mutating and evolving,we can't stop them from going that.

I completely agree with the above. Attempts in this project are to predict the following 3-10 hours.

[bitmover]

You can't predict when a virus will mutate and spread,but you can predict how it will spread.

Ok, so you are proposing to protect yourself for the next pandemia, which will happen like every 400 years or so.
Or the next big earthquake which may affect california in the next months or maybe 200 years?

Or should this AI prepare for the situation while Trump  get Alzheimer and made some shit?

What I am saying is that the world cannot be predicted. And the world affects bitcoin price a lot (por positive and negative).

Maybe a world war will be good for btc. Or it can be the end. Who knows?

As you said, next 3-10 hours is ok to try to predict. But you can't do much about longer periods. This is not something I invented, this is an old discussion in traditional financial markets.

Those random events of high impact are called Black Swan events, you can google about it later. This term was created by Nassim taleb, who actually loves bitcoin.

[hatshepsut93]

Those random events of high impact are called Black Swan events, you can google about it later. This term was created by Nassim taleb, who actually loves bitcoin.

Nassim Talem himself said that coronavirus was by no means a black swan event, and that he and others have predicted it long before it happened. And it wasn't even a hard prediction to make, China is responsible for multiple outbreaks already, and it will happen again, unless they seriously stop this wildlife meat trade, which they probably won't, as they have already reopened their wet markets.

[bitmover]

Nassim Talem himself said that coronavirus was by no means a black swan event, and that he and others have predicted it long before it happened. And it wasn't even a hard prediction to make, China is responsible for multiple outbreaks already, and it will happen again, unless they seriously stop this wildlife meat trade, which they probably won't, as they have already reopened their wet markets.

Thanks for sharing this article. I didn't know about it..
Anyway  even if corona is or not a black swan, it could not be predicated by any trading algorithm.  Real world events affect bitcoin price, this is why all ta fail in mid term imo.

[hatshepsut93]

Thanks for sharing this article. I didn't know about it..
Anyway  even if corona is or not a black swan, it could not be predicated by any trading algorithm.  Real world events affect bitcoin price, this is why all ta fail in mid term imo.

It's actually very interesting, because the article tells how a long time ago Nassim Taleb developed this trading strategy which was losing money while the market was calm, but it greatly profited from a big and sudden movement, the black swans. So, even if a neural network trained on Bitcoin's price can't predict a global pandemic, maybe it can be trained to see patterns of sudden big price changes, and act similarly to the described Nassim Taleb's strategy.

[Chlotide]

The idea sounds awesome and wish you all the best!

Thing is this project would have a very hard start. At first yes, you need to calculate all sorts of variables, take into account news and asses if they may be true or fake, use 2 quantum computers to grind the numbers (exaggerating to make a point) but would really work for 1 user. And would be a hell of a project if it achieved 75% accuracy or somewhere in that ballpark.

But once the project would reach a critical number of users the "Bitcoin Wall Observer AI" would just turn into a HYIPE IMO.
If the AI says BTC will increase in price the users will start buying and price would actually increase. If the AI says BTC will go down 20% users will sell and it will decrease in price (how close to the speculated 20% is unknown but users will still think it works). This is why I would say it could be ground breaking if it's used by a handful of users and could just as well be done in notepad(not even ++) at a certain point.

Wish you all the best and I'll be throwing a look at future development info

P.S thanks for the article @hatshepsut93 , very interesting read