Machine Learning (Keras LSTM and K-Clustering) to predict Bitcon Price

[Vlad.Sas]

I'm trying for some time now to develop a neural network that can predict bitcoin prices. Bitcoin is a great asset for such an experiment because it has fewer fundamentals than the other stocks or derivatives and it relies more on technical analysis than other assets. Nevertheless, it is impacted from time to time by news and other events in the crypto-world.

I am working with TF + Keras and LSTM (Long-Short Term Memory) cells to try and predict future prices.

If I let the network guess freely, without adjusting it to each OHLC Tick, the predictions look like this:




If i adjust it at every point in the dataset, it is needless to say that the graph of predictions is way more precise:


There are few steps I would like to take to improve this

• Factor in Technical indicator (BB, Ichimoku Cloud, SAR, etc...
• Place a factor for fundamental analysis in the dataset (for example, a score for the positivity of the news at that specific tick (0=bad news for btc, 10=great news)
• Collect data from the order books of the largest exchanged and factor in the order book states at every tick
• Unsupervised learning and K-Clustering to find co-relations between bitcon and other foreign currency (for example, it might be that there's an inverse relationship between bitcoin and the Yuan (generally when Yuan drops, Bitcoin rises). K-Clustering can find such relashionships
• Merging all of the above into a trading bot that aims for around 30%-40% return a year

Does anyone pursue such a goal? If yes, would you like to share the accumulated knowledge/ideas?
This is a serious project and currently, only me and a two of my partners are involved in it, but we could sure use some more man-power. I'd also like to know if someone would like to fund such a venture and get involved.

Cheers to you all!

[1714734944]

1714734944

[1714734944]

1714734944

[1714734944]

1714734944

[jackg]

I mean it's an interesting idea, I'm not sure how much its being/been done and how accurate it has been, I'm assuming you've just started it now and are piling past info into the neural network (as an average of several markets)?

[Vlad.Sas]

I mean it's an interesting idea, I'm not sure how much its being/been done and how accurate it has been, I'm assuming you've just started it now and are piling past info into the neural network (as an average of several markets)?

Your assumptions are quite accurate. I'm 1 month into this project and still in the research phase. I'm reading all papers that I can find to see how others applied ML to price predictions, what worked, what didn't, what can be improved in their approach, etc... The neural network in this post is just an intro phase really. A much more serious period of research and data gathering will follow. After the plan is put together, I'll attempt to have a nice prototype and then, of course, try to get funding for the project.

[coiningz]

I guess, if there were the real neutral network that can predict the price there already will be ~100% accurate trading bots but we dont have them

[Vlad.Sas]

I guess, if there were the real neutral network that can predict the price there already will be ~100% accurate trading bots but we dont have them

That's not possible, but it's possible to have a bot that gives an educated guess.
The aim of this project isn't to make a 100% accurate bot, but to increase the accuracy of the prediction as much as possible

[hatshepsut93]

I guess, if there were the real neutral network that can predict the price there already will be ~100% accurate trading bots but we dont have them

Nothing can be 100% accurate, but you don't need to, as long as your accuracy is above 50% you will be making profit. And there already been some projects here that advertised trading bots based on machine learning, I think I saw some threads on the Project Development board, perhaps OP will be interested to look at them.

[Vlad.Sas]

I guess, if there were the real neutral network that can predict the price there already will be ~100% accurate trading bots but we dont have them

Nothing can be 100% accurate, but you don't need to, as long as your accuracy is above 50% you will be making profit. And there already been some projects here that advertised trading bots based on machine learning, I think I saw some threads on the Project Development board, perhaps OP will be interested to look at them.

Will do! Thx.

[Vlad.Sas]

I guess, if there were the real neutral network that can predict the price there already will be ~100% accurate trading bots but we dont have them

Nothing can be 100% accurate, but you don't need to, as long as your accuracy is above 50% you will be making profit. And there already been some projects here that advertised trading bots based on machine learning, I think I saw some threads on the Project Development board, perhaps OP will be interested to look at them.

50%+ accuracy might not do it. You can have a model with 30% accuracy, but which doesn't miss the big spikes and drops in the price or a model with 70% accuracy that misses the spikes and drops. In this case, you might find that the 30% accuracy model is more profitable than the other one.

[hatshepsut93]

50%+ accuracy might not do it. You can have a model with 30% accuracy, but which doesn't miss the big spikes and drops in the price or a model with 70% accuracy that misses the spikes and drops. In this case, you might find that the 30% accuracy model is more profitable than the other one.

Yes, 50%+ accuracy assumes that a bot trades with a constant amount every time, if you have some complex strategy, then your performance can only be measured by the profit that it makes.

I'm really curious if such algorithm is able to predict huge movements, or if it's better for day trading with high frequency. And if it can do both, then which is better - waiting for a big shot, or steadily making profit every day?

[franziskakuhn]

You have a good observation in the bitcoin price chart on the market,
I think it would be great if you place your thread here https://bitcointalk.org/index.php?board=8.0

[buwaytress]

I used to in 2017/18 follow quite a few of these neural learning projects. At least two quite closely and even interacting with their founders, who at the very least were experts in their fields (Academics with very useful experience too) -- but once 2017/18 gave up the gains, the began failing just like most other crypto hedge funds.

And last year eToro did something with a purely "based on Twitter sentiment" thing and they've been posting heavy losses since too.

How far back has your data gone? I don't see any timeframe there. I would start also just charting from now and use alternative scenarios. One a bear outlook (looking to sell) and one a bull outlook (looking to hold). I know this is a predictor but outlook does affect prediction.

[Vlad.Sas]

I used to in 2017/18 follow quite a few of these neural learning projects. At least two quite closely and even interacting with their founders, who at the very least were experts in their fields (Academics with very useful experience too) -- but once 2017/18 gave up the gains, the began failing just like most other crypto hedge funds.

And last year eToro did something with a purely "based on Twitter sentiment" thing and they've been posting heavy losses since too.

How far back has your data gone? I don't see any timeframe there. I would start also just charting from now and use alternative scenarios. One a bear outlook (looking to sell) and one a bull outlook (looking to hold). I know this is a predictor but outlook does affect prediction.

I didn't plot the dates. I've trained the AI on hourly OHLCV since 2017. Currently, the structure of the model looks like this:


(first branch: open prices, second branch: high/low, third: volume)

It doesn't yet take into account any external factors other than the OHLCV. The impact of twitter sentiment and other fundamentals is yet to be studied, but thanks for the insights, I'd appreciate if you share more about your studies .

Cheers!

[hugeblack]

I didn't plot the dates. I've trained the AI on hourly OHLCV since 2017. Currently, the structure of the model looks like this:

OHLCV may give you patterns of prices to a certain extent or when there are no external factors (real influences.)
If you manually compare and delete these external factors, how accurate will the results be? I think you will get excellent accuracy as the price tends to repeat after the historical resistances when we ignore many external influences.
You can also track some of the effects and try to give them price adjustments, for example, hacking platforms and others.

Can you add more details to the charts?

[Sanugarid]

I guess, if there were the real neutral network that can predict the price there already will be ~100% accurate trading bots but we dont have them

Nothing can be 100% accurate, but you don't need to, as long as your accuracy is above 50% you will be making profit.

I think we should not be talking about accuracy here since we are dealing with decentralized market that is very uncertain for the price. And I don't think a 50% accuracy , or even 70% can really make you gain a profit.

And there already been some projects here that advertised trading bots based on machine learning, I think I saw some threads on the Project Development board, perhaps OP will be interested to look at them.

yes, we already have these applications, bot integrated system that analyzes old data, from past to present to compare the movements but none of these makes me that interested to look after it. The fact the the whole scheme is decentralized is enough for me to know that every thing is under an unclear path, you have to shoot in the dark if you gain then you're lucky, not otherwise.