Bitcoin Forum

Bitcoin => Bitcoin Technical Support => Topic started by: georgerick80 on April 11, 2021, 08:40:11 PM



Title: Open source address index & query software
Post by: georgerick80 on April 11, 2021, 08:40:11 PM
I want to be able to run address queries (balance, tx count, etc.) similar to the API provided by Block Explorer, blockchain.info, Blockcypher, etc. For performance reasons I want to run my own node & server and not depend on someone else's API (most/all of which cost money for making a large number of requests, or rate limit their free requests).

The Bitcoin Core RPC / REST API does not support general address queries, only queries against addresses in your own wallet and then some general blockchain status info on top of that.

Blockcypher Explorer publishes open sources for the web front end, but apparently not for the back end server that does the actual address indexing & search: https://github.com/blockcypher/explorer

I found "iquidus explorer" but this comment says it is too slow to practically use with Bitcoin:
https://github.com/iquidus/explorer/issues/452

I found several other projects that work with Ethereum, Cardano, and other coins but nothing for Bitcoin. Any others you know of that work with Bitcoin?

Thanks.


Title: Re: Open source address index & query software
Post by: PrimeNumber7 on April 12, 2021, 12:01:13 AM
You will want to put every transaction into a database, and run a database query anytime you want to lookup information about an address. This is in essence what any block explorer will do whenever you lookup an address or transaction.


Title: Re: Open source address index & query software
Post by: ABCbits on April 12, 2021, 11:07:23 AM
There are many open source Bitcoin block explorer out there, there are few that you might haven't tried such as
1. https://github.com/mempool/mempool (https://github.com/mempool/mempool). This one can run Raspberry Pi, so i'd recommend this.
2. https://github.com/Blockstream/esplora (https://github.com/Blockstream/esplora)

Both of them have some dependency though (Bitcoin Core with txindex enabled and Electrum server).


Title: Re: Open source address index & query software
Post by: LoyceV on April 12, 2021, 11:15:36 AM
If it doesn't need to be real-time (but updated once a day), you could use this instead of running a full node:
Related topics
Bitcoin block data available in CSV format (https://bitcointalk.org/index.php?topic=5246271.0)
List of all Bitcoin addresses with a balance (https://bitcointalk.org/index.php?topic=5254914.0)
List of all Bitcoin addresses ever used (https://bitcointalk.org/index.php?topic=5265993.0)
[~500 GB] Bitcoin block data: inputs, outputs and transactions (https://bitcointalk.org/index.php?topic=5307550.0)

If you create your own API, will you make it accessible to the public?


Title: Re: Open source address index & query software
Post by: dkbit98 on April 12, 2021, 11:18:18 AM
I found several other projects that work with Ethereum, Cardano, and other coins but nothing for Bitcoin. Any others you know of that work with Bitcoin?

I am not sure how they compare with iquidus explorer but there are several open source explorers for Bitcoin an you can run them on your own and see if you find them useful for address index and query:

Mempool is use mostly for checking current state of Bitcoin mempool and fees but you can use it as explorer:
https://github.com/mempool/mempool

Yogh.io you can run on top of your full Bitcoin node as explorer:
https://github.com/yogh-io/bitcoin-transaction-explorer

BTC RPC Explorer is simple self-hosted explorer you can run on your own full node:
https://github.com/janoside/btc-rpc-explorer





Title: Re: Open source address index & query software
Post by: NotATether on April 13, 2021, 07:15:03 PM
If you create your own API, will you make it accessible to the public?

What would be the point of doing that though, since OP would most likely have to rate-limit requests, which he wants to avoid?

BTC RPC Explorer is simple self-hosted explorer you can run on your own full node:
https://github.com/janoside/btc-rpc-explorer

This one looks the most similar to Blockchain.info, you just need a non-pruned node to run it because some information cannot be grabbed if running with a pruned node. Same thing happens if txindex is off.


Title: Re: Open source address index & query software
Post by: georgerick80 on April 19, 2021, 07:15:07 PM
Thanks all, I decided to give mempool a try and it seems to be meeting my needs so far.