GregH37
Legendary
 Offline
Activity: 1302
Merit: 1037
|
 |
March 30, 2019, 07:55:27 AM |
|
That's a lot of data that can feed AI bot. You can test your AI bot on those data. 1 clever algorithm and you have AI bot that has multiple times better performance than traditional bots and is "learning" and improving with every new trade that came to database.
This is not data than average trader will look in and perform his trades better. Its huge database for smart programists and their algorithm
Keep pushing it and probably those who are interested will have a rethink of it and see the need for it. Is there no way you can provide a free test for this that any trader can try and see if it will work or not, because it is only your work that can speak for you because getting people to commit fund now will be quite hard as everyone is virtually a suspect now, kudos to those idiots called ICO scammers that is now making everyone to be scared and also see a good project as a bad one. Anyway, I wish you luck but that $1500 is not feasible, you can make the first test very cheap, like $200, from the test, many of them will come to testify, then you can increase the value for your work. |
|
|
|
|
glaz330 (OP)
Newbie
Offline
Activity: 21
Merit: 0
|
|
March 31, 2019, 04:44:47 PM
Last edit: March 31, 2019, 05:27:11 PM by glaz330 |
|
OP, honestly if anyone were interested initially and then seeing that you're just a Newbie would make them lose interest in your project. Who even pays for a bot services that high in price. BTW, there's nothing special about what you intend to supply as it can be found online already.
How exactly did you conclude that I am a Newbie? (on the forum yes) It can be really funny how people can just judge the expertise of a forum user by their forum tag, I do that a lot too and until one day, it dawn on me that I was already an expert in trading before getting to know about bitcointalk and opening an account with on it. So for the fact that you are a newbie here doesn’t mean you are not an expert in your field but the service you intend rendering is quite on a high side for now and if you can proof to us that it will help the market out of the situation it is now, maybe I can try it out too and probably broadcast it to more crypto users too who might find it useful. Thank you for your constructive criticism.... I am testing the backend right now at it is working as expected. I am capturing data as we speak. My database is already one week old with all BTC pairs. I 'll provide a link, when I finish the validation and write a README with the specs of the database. Plus, I'll keep it free on github probably, for the time being. |
|
|
|
|
glaz330 (OP)
Newbie
Offline
Activity: 21
Merit: 0
|
|
March 31, 2019, 04:55:14 PM |
|
That's a lot of data that can feed AI bot. You can test your AI bot on those data. 1 clever algorithm and you have AI bot that has multiple times better performance than traditional bots and is "learning" and improving with every new trade that came to database.
This is not data than average trader will look in and perform his trades better. Its huge database for smart programists and their algorithm
Keep pushing it and probably those who are interested will have a rethink of it and see the need for it. Is there no way you can provide a free test for this that any trader can try and see if it will work or not, because it is only your work that can speak for you because getting people to commit fund now will be quite hard as everyone is virtually a suspect now, kudos to those idiots called ICO scammers that is now making everyone to be scared and also see a good project as a bad one. Anyway, I wish you luck but that $1500 is not feasible, you can make the first test very cheap, like $200, from the test, many of them will come to testify, then you can increase the value for your work. Thank you. Please see my answer above. |
|
|
|
|
|
Indamuck
|
|
March 31, 2019, 05:04:00 PM |
|
Data has immense value. This is why all these free platforms exist, they want to collect every bit of information they can about you.
As people have said before you need a lot of trading data if you are going to make a trading bot that has any chance of turning consistent profit.
|
|
|
|
|
glaz330 (OP)
Newbie
Offline
Activity: 21
Merit: 0
|
|
March 31, 2019, 05:16:36 PM |
|
First 50 entries starting at 2019-03-30 00:00:00 UTC to get a glimpse id |event_type|event_time |symbol |aggregate_trade_id|price |quantity|first_trade_id|last_trade_id|trade_time |market_maker|ignore|
---|----------|-------------|---------|------------------|----------|--------|--------------|-------------|-------------|------------|------|
1|aggTrade |1553904000025|NULSBTC | 4213613| 0.000187| 108| 4834249| 4834249|1553904000023|false |true |
2|aggTrade |1553904000059|CNDBTC | 3980877|0.00000407| 9766| 5473717| 5473717|1553904000058|false |true |
3|aggTrade |1553904000059|CNDBTC | 3980878|0.00000408| 10000| 5473718| 5473718|1553904000058|false |true |
4|aggTrade |1553904000059|CNDBTC | 3980879|0.00000409| 18663| 5473719| 5473721|1553904000058|false |true |
5|aggTrade |1553904000062|ARNBTC | 10846535|0.00011232| 19| 12083863| 12083863|1553904000058|false |true |
6|aggTrade |1553904000062|ARNBTC | 10846536|0.00011233| 744| 12083864| 12083864|1553904000058|false |true |
7|aggTrade |1553904000062|ARNBTC | 10846537|0.00011248| 100| 12083865| 12083865|1553904000058|false |true |
8|aggTrade |1553904000062|ARNBTC | 10846538| 0.0001126| 1039| 12083866| 12083867|1553904000058|false |true |
9|aggTrade |1553904000073|ADABTC | 19667521|0.00001733| 231| 25492811| 25492811|1553904000070|false |true |
10|aggTrade |1553904000109|ADABTC | 19667522|0.00001733| 84| 25492812| 25492812|1553904000106|false |true |
11|aggTrade |1553904000131|BCHSVBTC | 5616643| 0.015673| 0| 6848467| 6848467|1553904000129|true |true |
12|aggTrade |1553904000138|BNBBTC | 35103603| 0.0040185| 10| 40594951| 40594951|1553904000136|false |true |
13|aggTrade |1553904000138|BNBBTC | 35103604| 0.0040198| 27| 40594952| 40594952|1553904000136|false |true |
14|aggTrade |1553904000144|ETHBTC | 104442122| 0.034891| 0| 114690882| 114690882|1553904000142|true |true |
15|aggTrade |1553904000153|ARNBTC | 10846539| 0.0001126| 22| 12083868| 12083868|1553904000152|false |true |
16|aggTrade |1553904000192|NULSBTC | 4213614| 0.000187| 391| 4834250| 4834250|1553904000191|false |true |
17|aggTrade |1553904000248|ADABTC | 19667523|0.00001733| 5164| 25492813| 25492813|1553904000246|false |true |
18|aggTrade |1553904000251|EVXBTC | 3361090|0.00028006| 8| 4288698| 4288699|1553904000250|true |true |
19|aggTrade |1553904000251|EVXBTC | 3361091|0.00028001| 3| 4288700| 4288700|1553904000250|true |true |
20|aggTrade |1553904000278|BNBBTC | 35103605| 0.00402| 23| 40594953| 40594953|1553904000275|true |true |
21|aggTrade |1553904000278|BNBBTC | 35103606| 0.0040185| 0| 40594954| 40594954|1553904000275|true |true |
22|aggTrade |1553904000304|PPTBTC | 3740263| 0.0003651| 40| 4315780| 4315780|1553904000303|false |true |
23|aggTrade |1553904000343|NULSBTC | 4213615| 0.000187| 352| 4834251| 4834251|1553904000341|false |true |
24|aggTrade |1553904000351|ETHBTC | 104442123| 0.034896| 3| 114690883| 114690883|1553904000349|false |true |
25|aggTrade |1553904000352|ETHBTC | 104442124| 0.034898| 3| 114690884| 114690884|1553904000352|false |true |
26|aggTrade |1553904000367|BATBTC | 7191680| 0.0000693| 1375| 8602061| 8602061|1553904000365|false |true |
27|aggTrade |1553904000369|ETHBTC | 104442125| 0.034898| 1| 114690885| 114690885|1553904000367|false |true |
28|aggTrade |1553904000372|ETCBTC | 11879170| 0.001187| 51| 14399174| 14399177|1553904000369|true |true |
29|aggTrade |1553904000443|ADABTC | 19667524|0.00001733| 8440| 25492814| 25492814|1553904000440|false |true |
30|aggTrade |1553904000477|NULSBTC | 4213616| 0.000187| 2631| 4834252| 4834252|1553904000476|false |true |
31|aggTrade |1553904000496|AMBBTC | 3279987|0.00001682| 400| 3942009| 3942009|1553904000493|true |true |
32|aggTrade |1553904000496|AMBBTC | 3279988| 0.0000168| 11234| 3942010| 3942012|1553904000493|true |true |
33|aggTrade |1553904000515|EVXBTC | 3361092|0.00028007| 107| 4288701| 4288701|1553904000514|false |true |
34|aggTrade |1553904000521|NULSBTC | 4213617| 0.000187| 248| 4834253| 4834253|1553904000520|false |true |
35|aggTrade |1553904000521|ADABTC | 19667525|0.00001733| 7197| 25492815| 25492815|1553904000520|false |true |
36|aggTrade |1553904000530|ADABTC | 19667526|0.00001733| 53442| 25492816| 25492818|1553904000527|false |true |
37|aggTrade |1553904000533|LTCBTC | 20357376| 0.014972| 0| 22637179| 22637179|1553904000532|false |true |
38|aggTrade |1553904000545|ADABTC | 19667527|0.00001733| 5146| 25492819| 25492819|1553904000543|false |true |
39|aggTrade |1553904000586|EVXBTC | 3361093|0.00028007| 85| 4288702| 4288702|1553904000585|false |true |
40|aggTrade |1553904000657|DOCKBTC | 1600866|0.00000329| 1367| 2530555| 2530556|1553904000655|true |true |
41|aggTrade |1553904000665|ADABTC | 19667528|0.00001733| 1809| 25492820| 25492820|1553904000663|false |true |
42|aggTrade |1553904000721|NULSBTC | 4213618| 0.000187| 443| 4834254| 4834254|1553904000721|false |true |
43|aggTrade |1553904000743|EVXBTC | 3361094|0.00028007| 84| 4288703| 4288703|1553904000741|false |true |
44|aggTrade |1553904000826|NULSBTC | 4213619| 0.000187| 211| 4834255| 4834255|1553904000825|false |true |
45|aggTrade |1553904000850|ADABTC | 19667529|0.00001732| 115| 25492821| 25492821|1553904000847|true |true |
46|aggTrade |1553904000858|FUELBTC | 2706758|0.00000339| 2950| 4152898| 4152898|1553904000856|false |true |
47|aggTrade |1553904000946|NULSBTC | 4213620| 0.000187| 144| 4834256| 4834256|1553904000943|false |true |
48|aggTrade |1553904001006|OAXBTC | 2748544|0.00006215| 359| 3605942| 3605942|1553904001004|false |true |
49|aggTrade |1553904001019|BNBBTC | 35103607| 0.0040184| 3| 40594955| 40594955|1553904001016|true |true |
50|aggTrade |1553904001060|NULSBTC | 4213621| 0.000187| 93| 4834257| 4834257|1553904001058|false |true |
|
|
|
|
|
|
Theb
|
|
March 31, 2019, 05:27:02 PM |
|
So the price range for Binance's historical data range from 350$ to 1500$ which I don't think normal traders would be interested to buy since most of the data can already be provided by other free services online. The only thing I can think off that will buy this data are either their competitors or some companies in the financial industry trying to create data solutions for others. So if you are thinking that you can piece out their data and sell it cheaper it may work but you will only have a limited market for it as I don't think people will be comfortable on shelling out hundreds of dollars for a data they don't even know how to take advantage of.
|
|
|
|
glaz330 (OP)
Newbie
Offline
Activity: 21
Merit: 0
|
|
March 31, 2019, 05:43:07 PM |
|
So the price range for Binance's historical data range from 350$ to 1500$ which I don't think normal traders would be interested to buy since most of the data can already be provided by other free services online. The only thing I can think off that will buy this data are either their competitors or some companies in the financial industry trying to create data solutions for others. So if you are thinking that you can piece out their data and sell it cheaper it may work but you will only have a limited market for it as I don't think people will be comfortable on shelling out hundreds of dollars for a data they don't even know how to take advantage of.
Could you please provide a link with free raw historical Binance data, like the one showing in my post above? Thank you. |
|
|
|
|
|
hyunee
|
|
March 31, 2019, 05:44:50 PM |
|
Thank you for your answer. I am talking for tick by tick raw data - all trades. Can you find this kind of info for free? Ty in advance! I'm really sorry but I don't know what you're talking about. Can you explain it in details why do we need tick raw data? |
|
|
|
|
Sujoris
Member
Offline
Activity: 80
Merit: 46
|
|
March 31, 2019, 06:56:27 PM |
|
Thank you for your answer. I am talking for tick by tick raw data - all trades. Can you find this kind of info for free? Ty in advance! Get an API access and pull all the data to your own server. So yes, it is available for free. You only need to be smart & code-literate enough to access that data. |
|
|
|
|
glaz330 (OP)
Newbie
Offline
Activity: 21
Merit: 0
|
|
March 31, 2019, 07:37:11 PM |
|
Thank you for your answer. I am talking for tick by tick raw data - all trades. Can you find this kind of info for free? Ty in advance! Get an API access and pull all the data to your own server. So yes, it is available for free. You only need to be smart & code-literate enough to access that data. You are making this look like the simplest thing on the planet. In my research I couldn't find some smart & code-literate enough to access that data AND give it for free. |
|
|
|
|
glaz330 (OP)
Newbie
Offline
Activity: 21
Merit: 0
|
|
April 01, 2019, 07:01:44 AM |
|
Thank you for your answer. I am talking for tick by tick raw data - all trades. Can you find this kind of info for free? Ty in advance! I'm really sorry but I don't know what you're talking about. Can you explain it in details why do we need tick raw data? your strategy + train an AI bot + various analyses of the data |
|
|
|
|
glaz330 (OP)
Newbie
Offline
Activity: 21
Merit: 0
|
|
April 01, 2019, 02:56:07 PM |
|
|
|
|
|
|
glaz330 (OP)
Newbie
Offline
Activity: 21
Merit: 0
|
|
April 02, 2019, 12:20:45 PM |
|
OP, honestly if anyone were interested initially and then seeing that you're just a Newbie would make them lose interest in your project. Who even pays for a bot services that high in price. BTW, there's nothing special about what you intend to supply as it can be found online already.
How exactly did you conclude that I am a Newbie? (on the forum yes) It can be really funny how people can just judge the expertise of a forum user by their forum tag, I do that a lot too and until one day, it dawn on me that I was already an expert in trading before getting to know about bitcointalk and opening an account with on it. So for the fact that you are a newbie here doesn’t mean you are not an expert in your field but the service you intend rendering is quite on a high side for now and if you can proof to us that it will help the market out of the situation it is now, maybe I can try it out too and probably broadcast it to more crypto users too who might find it useful. I have updated the post with a link... I 'll really appreciate your feedback. Thank you |
|
|
|
|
glaz330 (OP)
Newbie
Offline
Activity: 21
Merit: 0
|
|
July 31, 2019, 06:59:46 AM |
|
New dates added.
|
|
|
|
|
Frank37
Sr. Member
Offline
Activity: 770
Merit: 250
CryptoTalk.Org - Get Paid for every Post!
|
|
August 07, 2019, 06:21:48 PM
Last edit: August 07, 2019, 06:41:32 PM by Frank37 |
|
Unfortunately, many users do not understand what you are trying to do here. Even experienced BTCT users overlook the fact that you want to collect tick by tick raw data. All here mentioned free services and the binance API can deliver this: https://api.binance.com/api/v1/klines?symbol=BTCUSDT&interval=1mBut the smallest time interval is 1 min. It is not trivial to do this task tick by tick, because you have to read out the trade list cyclically this way: https://api.binance.com/api/v1/trades?symbol=BTCUSDT&limit=50and merge it at the right place together again. Since sometimes 10 or 20 trades per second are executed, this is not easy, because your script have to act fast. But if your script requests data to fast you can quickly get a HTTP 429 return code (breaking a request rate limit). And this request is only for one pair - BTCUSDT. All BTC-pairs create much more traffic and timing problems. So it would need several data collection clients which are working together. Keep up your work. |
|
|
|
silviumatei
Newbie
Offline
Activity: 1
Merit: 0
|
|
September 04, 2019, 06:17:09 AM |
|
This is incomplete in my opinion and might be confusing for AI bots without full order book all the time. Big (fake or real) walls in order book very often push price in various directions that won't be visible for AI bot with only trade history.
Fair enough.... So do you think, that raw data in sync with an order book of say 100 positions, in both directions, is a good sample? Ty in advance Hi, how r u doing? Very interesting project, particularly for reinforcement learning. Did you see this by any chance? Looks pretty much like what you are describing: https://tardis.dev/ |
|
|
|
|
K_I_C_K
Newbie
Offline
Activity: 1
Merit: 0
|
|
December 02, 2019, 05:41:42 PM |
|
I've looked for historical trade data as well .. and I can't believe the price that is being asked for this data. I designed a python program to grab all the trade data from the api and place it in a database server .. the only problem is that the api only allows you to grab a maximum of 1000 trades per request ... so in order to grab a years worth of data, it's going to take about 10-20 days. If any one knows of a better solution, could you let me know ? and if any one would like the python code to grab the data, let me know and i'll pass it on.
|
|
|
|
|
tardis_thad
Newbie
Offline
Activity: 4
Merit: 0
|
|
December 29, 2019, 01:32:16 PM |
|
Can't speak for other solutions but in case of https://tardis.dev price is not only for trade data, but also for tick by tick order book data which given it's vast size (in case of Binance only it's ~150GB compressed data per month) you may find price not that unbelievable (add VM + storage costs and it quickly adds up). |
|
|
|
|
|
