bomberb17 (OP)
|
|
September 07, 2018, 02:46:10 AM |
|
Hello I am looking for historic data for mining pools to perform a study. e.g. for Antpool, how many blocks it mined in January 2018, and for each block what was the Bitcoin network hashrate and what was the pool's hashrate. I would also like to export them in readable format e.g. JSON. I only found this in btc.com https://btc.com/stats/pool/AntPoolwhich the data is not exportable (you have to click the button on the bottom a million times) Blockchain.info only has history for 4 days.. Any suggestions? Thanks! (P.S. To moderators: Please do not delete my post again for no reason. If I'm posting on the wrong section please let me know which forum section would be applicable and move it there.)
|
|
|
|
|
|
|
|
Remember that Bitcoin is still beta software. Don't put all of your money into BTC!
|
|
|
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
|
|
|
|
|
pooya87
Legendary
Offline
Activity: 3444
Merit: 10546
|
|
September 07, 2018, 04:18:46 AM |
|
I substitute pool_name with a pool but it doesn't work..
you should replace $pool_name with the name: eg: https://blockchain.info/blocks/AntPool?format=jsoni believe it gives you the past 24 hours blocks found by that pool. in case you needed more you either have to seek other APIs to see whether they can help or just take the first transaction of each block (the coinbase tx) and decode their OP_Return output if they have any to see which pool that belongs to (similar to what explorers do) for example block #540291 has this: 6a 24aa21a9eda1df4536a0b991727e4caf58f1aae977786a13708f015a589b8e0d02d67e9306
which contains the string "Mined by AntPool60" in it.
|
. .BLACKJACK ♠ FUN. | | | ███▄██████ ██████████████▀ ████████████ █████████████████ ████████████████▄▄ ░█████████████▀░▀▀ ██████████████████ ░██████████████ █████████████████▄ ░██████████████▀ ████████████ ███████████████░██ ██████████ | | CRYPTO CASINO & SPORTS BETTING | | │ | | │ | ▄▄███████▄▄ ▄███████████████▄ ███████████████████ █████████████████████ ███████████████████████ █████████████████████████ █████████████████████████ █████████████████████████ ███████████████████████ █████████████████████ ███████████████████ ▀███████████████▀ ███████████████████ | | .
|
|
|
|
|
|
|
pooya87
Legendary
Offline
Activity: 3444
Merit: 10546
|
|
September 07, 2018, 05:10:15 AM |
|
However if I set a limit higher than 1000 then it only returns 10 blocks...
limit is the number of results (blocks mined) that the call returns. if you check out the documentation at the bottom of my previous post you can see that it only accepts a number between 1 and 1000 so if you give it something outside of that range it defaults to 10 hence the 10 returned blocks. if you need more you need to go to next page using "next" like this: https://api.smartbit.com.au/v1/blockchain/pool/AntPool?limit=1000&sort=height&next=1https://api.smartbit.com.au/v1/blockchain/pool/AntPool?limit=1000&sort=height&next=2...
|
. .BLACKJACK ♠ FUN. | | | ███▄██████ ██████████████▀ ████████████ █████████████████ ████████████████▄▄ ░█████████████▀░▀▀ ██████████████████ ░██████████████ █████████████████▄ ░██████████████▀ ████████████ ███████████████░██ ██████████ | | CRYPTO CASINO & SPORTS BETTING | | │ | | │ | ▄▄███████▄▄ ▄███████████████▄ ███████████████████ █████████████████████ ███████████████████████ █████████████████████████ █████████████████████████ █████████████████████████ ███████████████████████ █████████████████████ ███████████████████ ▀███████████████▀ ███████████████████ | | .
|
|
|
|
bomberb17 (OP)
|
|
September 07, 2018, 05:24:45 AM |
|
|
|
|
|
nc50lc
Legendary
Offline
Activity: 2408
Merit: 5588
Self-proclaimed Genius
|
|
September 07, 2018, 05:49:18 AM |
|
You can try https://bitcoinstrings.com/, but it's totally converted to text and the blocks' data aren't displayed. OP_return data were translated too, just ignore them or highlight all the " Mined by" lines then search from other source about which blocks are included in that particular blkXXXXX.dat number. Sort it out, and you might find what you need. However, the site's updates were not that frequent ( about daily to four times a week)
|
. .HUGE. | | | | | | █▀▀▀▀ █ █ █ █ █ █ █ █ █ █ █ █▄▄▄▄ | ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ . CASINO & SPORTSBOOK ▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄ | ▀▀▀▀█ █ █ █ █ █ █ █ █ █ █ █ ▄▄▄▄█ | | |
|
|
|
pooya87
Legendary
Offline
Activity: 3444
Merit: 10546
|
|
September 07, 2018, 06:00:17 AM |
|
sorry, that was my bad. i always make the same mistake with pages in API calls. basically you make the first call: https://api.smartbit.com.au/v1/blockchain/pool/AntPool?limit=20&sort=height (used 20 so it opens up fast) then you look for this key in the JSON: [pool][block_paging][next_link] which contains the link to next page: https://api.smartbit.com.au/v1/blockchain/pool/AntPool?next=NTQwMjA2&limit=20in case you are wondering what the heck "NTQwMjA2" is, it is base64 encoding of the last block height in previous page. in this case the block at index 19 of the array result of the first call which is "540206"
|
. .BLACKJACK ♠ FUN. | | | ███▄██████ ██████████████▀ ████████████ █████████████████ ████████████████▄▄ ░█████████████▀░▀▀ ██████████████████ ░██████████████ █████████████████▄ ░██████████████▀ ████████████ ███████████████░██ ██████████ | | CRYPTO CASINO & SPORTS BETTING | | │ | | │ | ▄▄███████▄▄ ▄███████████████▄ ███████████████████ █████████████████████ ███████████████████████ █████████████████████████ █████████████████████████ █████████████████████████ ███████████████████████ █████████████████████ ███████████████████ ▀███████████████▀ ███████████████████ | | .
|
|
|
|
bomberb17 (OP)
|
|
September 07, 2018, 06:09:18 AM |
|
You can try https://bitcoinstrings.com/, but it's totally converted to text and the blocks' data aren't displayed. OP_return data were translated too, just ignore them or highlight all the " Mined by" lines then search from other source about which blocks are included in that particular blkXXXXX.dat number. Sort it out, and you might find what you need. However, the site's updates were not that frequent ( about daily to four times a week) Thanks, however I would need to devote time to convert text to usable data... Great thanks! Looks like I will have to find a way to "glue" all these JSONs into a big one..
|
|
|
|
ABCbits
Legendary
Offline
Activity: 2870
Merit: 7464
Crypto Swap Exchange
|
|
September 07, 2018, 01:04:30 PM Last edit: September 07, 2018, 01:21:41 PM by ETFbitcoin |
|
This might be crazy idea, but you could try using Bitcoin API JSON-RPC and your own script to get coinbase text which pool usually use to mark it's mined block. getblockhash, getblock true and getrawtransaction {true/false} with script should able to do the trick. You can try https://bitcoinstrings.com/, but it's totally converted to text and the blocks' data aren't displayed. OP_return data were translated too, just ignore them or highlight all the " Mined by" lines then search from other source about which blocks are included in that particular blkXXXXX.dat number. Sort it out, and you might find what you need. However, the site's updates were not that frequent ( about daily to four times a week) I was going to mention that website, but there's no info of the block height and it's useless for OP unless he know about how Bitcoin Core manages it's blocks/leveldb and have full copy of blockchain/run full nodes.
|
|
|
|
spirali
Member
Offline
Activity: 65
Merit: 61
|
|
September 09, 2018, 10:38:15 AM |
|
Great thanks! Looks like I will have to find a way to "glue" all these JSONs into a big one..
"jq" ( https://stedolan.github.io/jq/) is a very good tool to achieve that (and any JSON processing in general). Learning the syntax is a bit cumbersome at the beginning but it's definitely worth it because it's very powerful.
|
1 BTC = 1 BTC
|
|
|
bomberb17 (OP)
|
|
September 10, 2018, 03:43:43 PM |
|
I eventually used the JSON data from api.smartbit.com.au, saved me a lot of time of putting the data together myself.
Thank you all for your inputs!
|
|
|
|
|