Bitcoin Forum
April 27, 2024, 07:07:00 PM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1] 2 »  All
  Print  
Author Topic: [Question] Fast way to check bulk address bitcoin balance  (Read 341 times)
DanielHoffman (OP)
Newbie
*
Offline Offline

Activity: 26
Merit: 0


View Profile
November 14, 2023, 04:25:58 AM
 #1

Hi i want to check a bulk addresses balance, actually i use request.get in python way with blockstream but it's too slow (around 100 addresses in 46 seconds)

Really appreciate.
1714244820
Hero Member
*
Offline Offline

Posts: 1714244820

View Profile Personal Message (Offline)

Ignore
1714244820
Reply with quote  #2

1714244820
Report to moderator
"There should not be any signed int. If you've found a signed int somewhere, please tell me (within the next 25 years please) and I'll change it to unsigned int." -- Satoshi
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1714244820
Hero Member
*
Offline Offline

Posts: 1714244820

View Profile Personal Message (Offline)

Ignore
1714244820
Reply with quote  #2

1714244820
Report to moderator
1714244820
Hero Member
*
Offline Offline

Posts: 1714244820

View Profile Personal Message (Offline)

Ignore
1714244820
Reply with quote  #2

1714244820
Report to moderator
1714244820
Hero Member
*
Offline Offline

Posts: 1714244820

View Profile Personal Message (Offline)

Ignore
1714244820
Reply with quote  #2

1714244820
Report to moderator
Yamane_Keto
Sr. Member
****
Offline Offline

Activity: 462
Merit: 477



View Profile WWW
November 14, 2023, 04:50:04 AM
Merited by vapourminer (2), pooya87 (2)
 #2

you can try  https://blockchair.com/API for a limit of 1,000 calls per day Addresses per request (25k) Requests per minute ( 30 MIn) and If it is not enough, you can subscribe to their paid services according to the prices https://blockchair.com/api/plans

if you care about Requests per minute only try blockonomics.co as you can get 100 address every 2 min.
If you tell me specifically what you are looking for, I may give you better options, but https://blockchair.com provides excellent service and diverse options.

.BEST.CHANGE..███████████████
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
███████████████
..BUY/ SELL CRYPTO..
seek3r
Legendary
*
Offline Offline

Activity: 1260
Merit: 1954



View Profile
November 14, 2023, 07:58:41 AM
Merited by LoyceV (6), El duderino_ (5), tyz (3), bitmover (3), pooya87 (2), ABCbits (2), Halab (2), bullrun2024bro (2), vapourminer (1), DdmrDdmr (1)
 #3

Using requests.get is indeed very slow.

To check the balance of Bitcoin addresses in bulk, you would want an API that supports batch requests.
This would speed up the whole process since you can query multiple addresses within a single API call. The Blockchain.com API supports batch requests for address balances.

Code:
import requests

def fetch_balances(addresses):
    address_list = '|'.join(addresses)

    api_endpoint = f'https://blockchain.info/balance?active={address_list}'

    response = requests.get(api_endpoint)
    data = response.json()

    for address, info in data.items():
        balance = info['final_balance']
        print(f"Address: {address}, Balance: {balance} ")

addresses = ['address1', 'address2', 'address3', ...]

fetch_balances(addresses)

A simple python script that would help you to fetch the balances of multiple addresses. The output would be in satoshis due to API of Blockchain.com

.
.HUGE.
▄██████████▄▄
▄█████████████████▄
▄█████████████████████▄
▄███████████████████████▄
▄█████████████████████████▄
███████▌██▌▐██▐██▐████▄███
████▐██▐████▌██▌██▌██▌██
█████▀███▀███▀▐██▐██▐█████

▀█████████████████████████▀

▀███████████████████████▀

▀█████████████████████▀

▀█████████████████▀

▀██████████▀▀
█▀▀▀▀











█▄▄▄▄
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
.
CASINSPORTSBOOK
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▀▀▀▀█











▄▄▄▄█
albert0bsd
Hero Member
*****
Offline Offline

Activity: 849
Merit: 660



View Profile WWW
November 14, 2023, 01:28:30 PM
Merited by vapourminer (2), pooya87 (2), NotATether (2), nc50lc (1)
 #4

Hi i want to check a bulk addresses balance, actually i use request.get in python way with blockstream but it's too slow (around 100 addresses in 46 seconds)

Why people always want to do this? It is always slow.

The correct way to do this faster is download an updated list of address with balance List of all Bitcoin addresses with a balance

Then load that file into a database/bloom filter or some other custom method.

Check your list against that method.

WIth this way you can be able to check million of address in less than a second

bitmover
Legendary
*
Offline Offline

Activity: 2282
Merit: 5901


bitcoindata.science


View Profile WWW
November 14, 2023, 06:06:07 PM
 #5

I think the problem with those free api is that you might even get blocked doing 100 requests in less than a minute.

I made this javascript tool which allows you to do search for address balances, but it can't handle 100 addresses  (I even used a try/catch method with a seocnd api when you get to api request limits)

Take a look

https://bitcoindata.science/bitcoin-balance-check


Hi i want to check a bulk addresses balance, actually i use request.get in python way with blockstream but it's too slow (around 100 addresses in 46 seconds)

Why people always want to do this? It is always slow.

The correct way to do this faster is download an updated list of address with balance List of all Bitcoin addresses with a balance

Then load that file into a database/bloom filter or some other custom method.

Check your list against that method.

WIth this way you can be able to check million of address in less than a second

This is not faster than 46 seconds (as you have to download a huge file)... and the data might not be updated.

.
.BLACKJACK ♠ FUN.
█████████
██████████████
████████████
█████████████████
████████████████▄▄
░█████████████▀░▀▀
██████████████████
░██████████████
████████████████
░██████████████
████████████
███████████████░██
██████████
CRYPTO CASINO &
SPORTS BETTING
▄▄███████▄▄
▄███████████████▄
███████████████████
█████████████████████
███████████████████████
█████████████████████████
█████████████████████████
█████████████████████████
███████████████████████
█████████████████████
███████████████████
▀███████████████▀
█████████
.
albert0bsd
Hero Member
*****
Offline Offline

Activity: 849
Merit: 660



View Profile WWW
November 14, 2023, 06:29:01 PM
 #6

This is not faster than 46 seconds (as you have to download a huge file)... and the data might not be updated.

Well that is true the data may bot be updated, but it is a good point to start no? If someone do this he can keep updated their database with other sources scripts and this may avoid all  external web APIs limits.

About the 46 Seconds... it is not bvious?? The time that i mention is after download and process the data.

bitmover
Legendary
*
Offline Offline

Activity: 2282
Merit: 5901


bitcoindata.science


View Profile WWW
November 14, 2023, 07:32:17 PM
 #7

This is not faster than 46 seconds (as you have to download a huge file)... and the data might not be updated.

Well that is true the data may bot be updated, but it is a good point to start no? If someone do this he can keep updated their database with other sources scripts and this may avoid all  external web APIs limits.

About the 46 Seconds... it is not bvious?? The time that i mention is after download and process the data.

It is amazing to use this file, especially for privacy purposes. As he is looking all addresses,  nobody knows which one is his.

But data probably don't get updates in every block (like a block explorer). And looks like op is looking for a performance boost

.
.BLACKJACK ♠ FUN.
█████████
██████████████
████████████
█████████████████
████████████████▄▄
░█████████████▀░▀▀
██████████████████
░██████████████
████████████████
░██████████████
████████████
███████████████░██
██████████
CRYPTO CASINO &
SPORTS BETTING
▄▄███████▄▄
▄███████████████▄
███████████████████
█████████████████████
███████████████████████
█████████████████████████
█████████████████████████
█████████████████████████
███████████████████████
█████████████████████
███████████████████
▀███████████████▀
█████████
.
CryptoDeivid
Jr. Member
*
Offline Offline

Activity: 40
Merit: 24


View Profile
November 15, 2023, 12:50:11 AM
 #8

This is not faster than 46 seconds (as you have to download a huge file)... and the data might not be updated.

Well that is true the data may bot be updated, but it is a good point to start no? If someone do this he can keep updated their database with other sources scripts and this may avoid all  external web APIs limits.

About the 46 Seconds... it is not bvious?? The time that i mention is after download and process the data.

This time may vary depending on the specific data being used and the processing involved, I thing, but 46 seconds is a relatively short amount of time.
yhiaali3
Legendary
*
Offline Offline

Activity: 1680
Merit: 1849


#SWGT CERTIK Audited


View Profile WWW
November 15, 2023, 07:08:33 AM
 #9

There are many sites that provide this service for free or paid. You can try this free site:

https://cointool.app/batchCheckBalance/btc

It gives good and fast results, and the other good thing about this site is that it gives aggregate address balance results for coins other than Bitcoin on many networks such as:
BTC, ETH, Arbitrum, BSC, Optimism, TRX, HECO, Solana, Polygon,.......etc

albert0bsd
Hero Member
*****
Offline Offline

Activity: 849
Merit: 660



View Profile WWW
November 15, 2023, 01:53:57 PM
 #10

And looks like op is looking for a performance boost

In that case a custom host like mempool.space can be installed to handle all the block chain data already parsed and processed in this way you can change your API end point to this localhost server without any restriction..

This time may vary depending on the specific data being used and the processing involved, I thing, but 46 seconds is a relatively short amount of time.

Once that the data is downloaded an proccessed the waiting time will be just small.

Yamane_Keto
Sr. Member
****
Offline Offline

Activity: 462
Merit: 477



View Profile WWW
November 15, 2023, 02:08:14 PM
Merited by vapourminer (2)
 #11


Code:
import requests

def fetch_balances(addresses):
    address_list = '|'.join(addresses)

    api_endpoint = f'https://blockchain.info/balance?active={address_list}'


A simple python script that would help you to fetch the balances of multiple addresses. The output would be in satoshis due to API of Blockchain.com
The blockchain.info API has a long history of sudden shutdowns, in which you will not receive a notification or alert, and the support team responds slowly and may continue for several days without knowing the cause of the problem.
blockchair.com/API is better.


The correct way to do this faster is download an updated list of address with balance List of all Bitcoin addresses with a balance

It seems that addresses.loyce.club depends on the data that comes from blockchair_bitcoin_addresses_and_balance_DATE.tsv.gz. Therefore, even if you download this file and try to manage the database and then create the API, you are accessing the indirect blockchair/API, which I think would be slower, if If you have the resources, buy a paid service.

.BEST.CHANGE..███████████████
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
██
███████████████
..BUY/ SELL CRYPTO..
seek3r
Legendary
*
Offline Offline

Activity: 1260
Merit: 1954



View Profile
November 15, 2023, 02:36:17 PM
 #12

A simple python script that would help you to fetch the balances of multiple addresses. The output would be in satoshis due to API of Blockchain.com
The blockchain.info API has a long history of sudden shutdowns, in which you will not receive a notification or alert, and the support team responds slowly and may continue for several days without knowing the cause of the problem.
blockchair.com/API is better.
But the OP asked for a solution with batch requests and afaik blockchair's API does not support this in a single API call.  Wink
Blockchairs API typically works with individual queries for each address or transaction. This means you would need to send a separate request for each address you want to check, which can be less efficient than a batch request, especially for a large number of addresses.

An alternative to blockchain.com could be the Blockcypher API, but I have no experience with it.

.
.HUGE.
▄██████████▄▄
▄█████████████████▄
▄█████████████████████▄
▄███████████████████████▄
▄█████████████████████████▄
███████▌██▌▐██▐██▐████▄███
████▐██▐████▌██▌██▌██▌██
█████▀███▀███▀▐██▐██▐█████

▀█████████████████████████▀

▀███████████████████████▀

▀█████████████████████▀

▀█████████████████▀

▀██████████▀▀
█▀▀▀▀











█▄▄▄▄
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
.
CASINSPORTSBOOK
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▀▀▀▀█











▄▄▄▄█
DanielHoffman (OP)
Newbie
*
Offline Offline

Activity: 26
Merit: 0


View Profile
November 15, 2023, 07:36:57 PM
 #13

Using requests.get is indeed very slow.

To check the balance of Bitcoin addresses in bulk, you would want an API that supports batch requests.
This would speed up the whole process since you can query multiple addresses within a single API call. The Blockchain.com API supports batch requests for address balances.

Code:
import requests

def fetch_balances(addresses):
    address_list = '|'.join(addresses)

    api_endpoint = f'https://blockchain.info/balance?active={address_list}'

    response = requests.get(api_endpoint)
    data = response.json()

    for address, info in data.items():
        balance = info['final_balance']
        print(f"Address: {address}, Balance: {balance} ")

addresses = ['address1', 'address2', 'address3', ...]

fetch_balances(addresses)

A simple python script that would help you to fetch the balances of multiple addresses. The output would be in satoshis due to API of Blockchain.com

Thanks but already do check balance in same 50 seconds  Embarrassed, honestly i want to do this very fast but it's okay
albert0bsd
Hero Member
*****
Offline Offline

Activity: 849
Merit: 660



View Profile WWW
November 15, 2023, 11:57:37 PM
 #14

Thanks but already do check balance in same 50 seconds  Embarrassed, honestly i want to do this very fast but it's okay

You are very lazy, you omit a lot of good answers.

btc_enigma
Hero Member
*****
Offline Offline

Activity: 688
Merit: 565


View Profile
November 16, 2023, 06:22:18 AM
 #15

Feel free to see the following github repo that uses several blockexplorer APIs to check address balances

https://github.com/xylevy/BTCSteroids


PS: Blockonomics API actually allows to lookup multiple addresses in one call and should be quite useful for this task

apogio
Sr. Member
****
Offline Offline

Activity: 420
Merit: 948



View Profile WWW
November 16, 2023, 09:43:00 AM
Last edit: November 16, 2023, 12:55:23 PM by apogio
 #16

Another solution to the problem:

Look into LoyceV's datasets: https://bitcointalk.org/index.php?topic=5254914.0

1. Download the daily data from there (it includes every address that has a balance).
2. Import them in a DB
3. Set your code to ask the DB if it includes the address you want. This will be much faster.

Optional tip: If you import batch addresses from the DB into a hashmap then you can check if it contains the address ULTRA FAST.



EDIT: It has already been mentioned by Yamane_Keto & albert0bsd above.

albert0bsd
Hero Member
*****
Offline Offline

Activity: 849
Merit: 660



View Profile WWW
November 16, 2023, 12:45:27 PM
 #17

Another solution to the problem:
...

Do you read all the posts before reply?

...
The correct way to do this faster is download an updated list of address with balance List of all Bitcoin addresses with a balance
...

OP is a lazy newbie, he want the code already done, just to run in one-click, he is not developer, maybe he doesn't know what a hashmap or a bloom filter is... if you see he only reply to the person who actually offer some code snippet

apogio
Sr. Member
****
Offline Offline

Activity: 420
Merit: 948



View Profile WWW
November 16, 2023, 12:59:42 PM
 #18

Do you read all the posts before reply?

I edited my post above and explained you have already mentioned it. I didn't carefully read your post, sorry.

OP is a lazy newbie, he want the code already done, just to run in one-click, he is not developer, maybe he doesn't know what a hashmap or a bloom filter is... if you see he only reply to the person who actually offer some code snippet

Ok, but OP can also import the file from LoyceV's website to an excel or something else and do the process manually. Excel would crash, now that I think about it.

In fact, OP wants something that can't be done with just one click.

LoyceV
Legendary
*
Offline Offline

Activity: 3290
Merit: 16557


Thick-Skinned Gang Leader and Golden Feather 2021


View Profile WWW
November 16, 2023, 01:50:10 PM
 #19

The correct way to do this faster is download an updated list of address with balance List of all Bitcoin addresses with a balance

Then load that file into a database/bloom filter or some other custom method.
This is what I would use to quickly check millions of addresses:
How to use
The most likely use is to check a long list of Bitcoin addresses for a remaining balance.
On Linux, use this to find matching addresses (after extrating the compressed .gz file of course):
Code:
comm -12 Bitcoin_addresses_LATEST.txt <(cat mylist.txt | sort | uniq)
  • Bitcoin_addresses_LATEST.txt: the extracted latest version downloaded from addresses.loyce.club.
  • mylist.txt: your own list of addresses, one address per line.
This takes only seconds to check millions of addresses.

This is not faster than 46 seconds (as you have to download a huge file)... and the data might not be updated.
Three things:
1. The server can spit out that file within 46 seconds Tongue
2. OP mentioned "bulk", and I don't consider his 100 address test to be bulk.
3. The data is updated daily. I assume OP is trying to brute-force Bitcoin addresses, in which case that doesn't matter.

Even if it takes a few minutes to download, after that you can check billions of addresses per hour.



@OP: what are you trying to accomplish?

█▀▀▀











█▄▄▄
▀▀▀▀▀▀▀▀▀▀▀
e
▄▄▄▄▄▄▄▄▄▄▄
█████████████
████████████▄███
██▐███████▄█████▀
█████████▄████▀
███▐████▄███▀
████▐██████▀
█████▀█████
███████████▄
████████████▄
██▄█████▀█████▄
▄█████████▀█████▀
███████████▀██▀
████▀█████████
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
c.h.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▀▀▀█











▄▄▄█
▄██████▄▄▄
█████████████▄▄
███████████████
███████████████
███████████████
███████████████
███░░█████████
███▌▐█████████
█████████████
███████████▀
██████████▀
████████▀
▀██▀▀
apogio
Sr. Member
****
Offline Offline

Activity: 420
Merit: 948



View Profile WWW
November 16, 2023, 01:53:01 PM
 #20


3. The data is updated daily. I assume OP is trying to brute-force Bitcoin addresses, in which case that doesn't matter.


Are there still people trying to brute-force keys? I am amazed...

Pages: [1] 2 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!