Best way or API to fetch UTXOS

[mahurovihamilo]

Hi All

Does anyone know the best way to fetch UTXOs from specific addresses? via API calls or any other way?

Thank you !

[nc50lc]

Does anyone know the best way to fetch UTXOs from specific addresses? via API calls or any other way?

Have you tried any of the famous blockexplorers' APIs?

For example, Blockstream's public API:

Code:

blockstream.info/api/address/1BitcoinEaterAddressDontSendf59kuE/txs/chain

Returns with the latest 25 transaction of the specified address.

To continue to the next set of transactions, add the last txid from the result above. (check if it's the transaction's txid, not its input)
e.g.:

Code:

https://blockstream.info/api/address/1BitcoinEaterAddressDontSendf59kuE/txs/chain/fab012bec43a909d787e95a1afcbced1a66fb1a6a62d0d22cad9578c560627cc

[RickDeckard]

In addition to nc50lc reply, you can also try using the API provided by mempool.space[1] to access that information:

Code:

curl -sSL "https://mempool.space/api/address/1KFHE7w8BhaENAswwryaoccDb6qcT6DbYY/utxo"

Returns:

Code:

[
  {
    txid: "12f96289f8f9cd51ccfe390879a46d7eeb0435d9e0af9297776e6bdf249414ff",
    vout: 0,
    status: {
      confirmed: true,
      block_height: 698642,
      block_hash: "00000000000000000007839f42e0e86fd53c797b64b7135fcad385158c9cafb8",
      block_time: 1630561459
    },
    value: 644951084
  },
  ...
]

mempool.space doesn't publish what are their API limits so once you start getting HTTP 429 error you will know that you've reached them. If you intend to make several calls perhaps it would be wise to balance them between some blockexplorers' APIs to avoid being blocked/rejected by their APIs.
[1]https://mempool.space/docs/api/rest#get-address-utxo

[mahurovihamilo]

Thank you for your replies gentlemen. That is very helpful.

I had tried some other API frameworks before, like:

wget -qO- https://api.blockcypher.com/v1/btc/main/addrs/{line.rstrip()[:]} > index.data'

But this is getting me a lot of transaction information but not specifically the UTXOs so I was having to parse through a lot of needlessly useless data.

Thank you again.

[mahurovihamilo]

Soooo if you don't mind I have a further question about formatting:

I did: wget -qO- https://mempool.space/api/address/1Lpu1LJAhYTWLveYW5mwi9T2kF3NKWQcJP/utxo  > data.json

using the mempool API, however the file is not quite formatted as usual json files are  and that makes parsing a lot harder with bash and python. However if from that page I manually click on raw date and then click on pretty print and manually save,  the saved file (through graphical interface) is actually formatted as usual json files.

I tried adding &type=json at the end of the wget command, but that messes up and the end point is not found.

So my question is, do you know in mempool how to insure the downloaded UTXO file (via wget) is an actual formatted json?

This is necessary to parse for batches of UTXO calls from script...

Thank you !!

[mahurovihamilo]

In addition to nc50lc reply, you can also try using the API provided by mempool.space[1] to access that information:

Code:

curl -sSL "https://mempool.space/api/address/1KFHE7w8BhaENAswwryaoccDb6qcT6DbYY/utxo"

Returns:

Code:

[
  {
    txid: "12f96289f8f9cd51ccfe390879a46d7eeb0435d9e0af9297776e6bdf249414ff",
    vout: 0,
    status: {
      confirmed: true,
      block_height: 698642,
      block_hash: "00000000000000000007839f42e0e86fd53c797b64b7135fcad385158c9cafb8",
      block_time: 1630561459
    },
    value: 644951084
  },
  ...
]

mempool.space doesn't publish what are their API limits so once you start getting HTTP 429 error you will know that you've reached them. If you intend to make several calls perhaps it would be wise to balance them between some blockexplorers' APIs to avoid being blocked/rejected by their APIs.
[1]https://mempool.space/docs/api/rest#get-address-utxo

I can't get an output formatted like that. When I do the call: curl -sSL "https://mempool.space/api/address/1KFHE7w8BhaENAswwryaoccDb6qcT6DbYY/utxo" > data.json

The ooutput comes formated like this, which is a pain to parse:

[{"txid":"33717a68856ecf397455b12b2a21321bf64eb6b7062939b46f630365c79d5604","vout":22,"status":{"confirmed":true,"block_height":838458,"block_hash":"0000000000000000000269654a31dbdfad3f0b7589433b1977a511e5641897c6","block_time":1712672676},"value":8021},{"txid":"5f3dafcdd142358b332e0939d37174b76472735df8df29325901d2c7d18a152c","vout":476,"status":{"confirmed":true,"block_height":834708,"block_hash":"00000000000000000000292964b4b00d1d414583bd18d1ebea7a8041d4409c6d","block_time":1710447063},"value":546},{"txid":"5557fbd7f4504d3b1886c4540db80fe5e56e326bbf419bcdfeecddf232bd31d1","vout":87,"status":{"confirmed":true,"block_height":838624,"block_hash":"000000000000000000002ffad6c02ed22351bd4df615b0fad3e1fbbb4bdc24dc","block_time":1712764942},"value":6040},{"txid":"988774c19446c5d2b174588f9839b43e86334b59aa5b0a8f05255bd29be53807","vout":2,"status":{"confirmed":true,"block_height":842495,"block_hash":"00000000000000000001a6c7110b453f369db51cbcb2d108f6909de4460fa6c8","block_time":1715122220},"value":546},{"txid":"e9abd6cf3b26236ef8b7f819a8fbd4a001379a74f86a1fff43b000382858abdb","vout":670,"status":{"confirmed":true,"block_height":840255,"block_hash":"000000000000000000020cf2611ecdda237904fcf1aa2edb532e1571b80614a9","block_time":1713735799},"value":888},{"txid":"b694b0fe40acde2855a1097350c68ce2c8015c364d8945a357b903ae9bbad4ec","vout":6,"status":{"confirmed":true,"block_height":841064,"block_hash":"0000000000000000000017f1db3dd49d05d4eea6761daaa25ad345a9985bcb18","block_time":1714201794},"value":15120},{"txid":"9cbf451cb05ac019c0cac3d7eb0d175c99ebd587fc4fe3e6b0662f724adf92be","vout":454,"status":{"confirmed":true,"block_height":840885,"block_hash":"000000000000000000023f33c91331450913710361a6f34426ebbe853efc18ee","block_time":1714085239},"value":546},{"txid":"9fd0b6e5c96f0537f0a61a4e1c32167c91511cd1ccfc34a92801842701c4df95","vout":0,"status":{"confirmed":true,"block_height":841305,"block_hash":"0000000000000000000077f28728313f4085275acb1745a438b71f197fee2a00","block_time":1714353023},"value":4287},{"txid":"0c2d34ad6c724b6d2554cd577e0db66d2ed4786edeab9d2246b5698fe842e215","vout":565,"status":{"confirmed":true,"block_height":840253,"block_hash":"0000000000000000000069440489b457550d1df7da1ec2ec52475d6353245948","block_time":1713735717},"value":888},{"txid":"9587acf268975b518733bfa432a1a2153a06e3de39ed118005d6b645120aaff7","vout":1,"status":{"confirmed":true,"block_height":841305,"block_hash":"0000000000000000000077f28728313f4085275acb1745a438b71f197fee2a00","block_time":1714353023},"value":52500},{"txid":"826f8191cc6ece6a2f895fbed111d80022a2efd4ee5482a25ae8398b4687bfff","vout":59,"status":{"confirmed":true,"block_height":841064,"block_hash":"0000000000000000000017f1db3dd49d05d4eea6761daaa25ad345a9985bcb18","block_time":1714201794},"value":19216},{"txid":"8a006d51d3173a36d257b357f55bc5155331e0c5f21cab754fd73957a250fc34","vout":378,"status":{"confirmed":true,"block_height":840657,"block_hash":"00000000000000000002d4a9bbd7a2437fe32df0d442148635dfae522514b5af","block_time":1713962541},"value":546},{"txid":"d0eb2d477784a892f012943dab7f11dc47fcf1d0fd236ac483237e3048e87ff2","vout":490,"status":{"confirmed":true,"block_height":836194,"block_hash":"00000000000000000000361121795cda72a2494ef23648f19685977610c1d55a","block_time":1711337405},"value":546}]

In priciple should be the same but my SEd and Python scripts that work on regular json, dont work on this output. I started customising Sed commands to reformat but it takes forever.

How do you get it to download in that json format?

Thanks !

[RickDeckard]

I can't get an output formatted like that. When I do the call: curl -sSL "https://mempool.space/api/address/1KFHE7w8BhaENAswwryaoccDb6qcT6DbYY/utxo" > data.json

What if you try the following code:

Code:

curl -sSL "https://mempool.space/api/address/1KFHE7w8BhaENAswwryaoccDb6qcT6DbYY/utxo" | jq '.' > data.json

All I did was adding | jq '.' to your initial input. jq is a JSON processing tool[1] - and since API usually return JSON data when called - it should get the job done by formatting the data before saving it to your named file (in this case data.json).
[1]https://www.digitalocean.com/community/tutorials/how-to-transform-json-data-with-jq

[mahurovihamilo]

 Thank you @RickDeckard

That is perfect! it does download in the right format. Thanks.

[mahurovihamilo]

Good day gentlemen...

I did the mempool call, but I got a message back on my shell telling the the request for UTXOs for that address  exceeded  limit of outputs > 500

curl -sSL "https://mempool.space/api/address/XXXXXXXXXXXXXXXXXXX/utxo"  > index.data

The mempool documentation is a little scarce in detail. Do you know of a flag I can add to the CURL call to limit it to 500 or less? I cant really afford a payed plan, not at those prices....

I tried: curl -sSL "https://mempool.space/api/address/XXXXXXXXXXXXXXXXXXX/utxo&limit=450 , as this works somewhere else, but not here.....


Thank you!

[RickDeckard]

I think you have two options that you can try:

• Run your local instance of mempool.space (by using Docker[1] for example) + Local Node. I would highly recommend doing this considering the amount of freedom and privacy that you'll get;
• mempool.space doesn't seem to have a capability to filter UTXO's. You either look for another blockexplorer API or perhaps you can try to add more arguments to jq. Considering the documentation[2] you could try filtering the results? For example, if you use [0:15] as an additional argument, it supposedly only prints the first 15 UTXO's. It should be something like this:

Code:

curl -sSL "https://mempool.space/api/address/AddressGoesHere/utxo" | jq '.[0:15]' > data.json

[1]https://github.com/mempool/mempool/tree/master/docker
[2]https://jqlang.github.io/jq/manual/

[mahurovihamilo]

Thank you @RickDeckard

I am going to explore running my own instance of mempool! Cheers!

[mahurovihamilo]

Hello RickDeckard,

So I went ahead and installed Docker and mempool, is up and running with a few glitches, but for now my question is, for this to work, do I also need to be running a full BTC node as well?

Thank you.

[RickDeckard]

I advise you to follow a guide[1] that covers the full installation of mempool.space correctly. As far as  know you need both a Bitcoin Core + Electrum Server in order to fully benefit from mempool.space capabilities.
[1]https://www.youtube.com/watch?v=I2SzBqcsXaE

[mahurovihamilo]

Ok, thank you!