Calculate how many BTC an address own from RPC calls

[TheWolf666]

I am trying to figure out how https://www.blockchain.com/ calculates the source and destinations of a transaction with the goal to know how much BTC every address owns.
I have no problem to get the VOUT

getblockhash thehash
then for each nTx I get the txhash
getrawtransaction txhash
which give me the vin and the value of all the transactions.

The problem is that I do not have the information as from which address the transactions are coming from.
I know the value and the destination, but not the source.

In consequence, I can only do the sum of all BTC that were sent to that address, but I cannot subtract the spent BTC.

However in https://www.blockchain.com/ they seems to have the source and destination, but for the source, I ignore how they are doing it from the txhash.

Anyone could suggest somewhere to look to solve my problem?

[odolvlobo]

If you want to know the amount of btc currently at an address, you would sum all the UTXOs associated with that address (using "scantxoutset"?).

[NotATether]

getblockhash thehash
then for each nTx I get the txhash
getrawtransaction txhash
which give me the vin and the value of all the transactions.

getrawtransaction will only find mempool transactions, and only if you pass -txindex=1, so your setup already can't handle confirmed transactions.  But it does support fetching them from a particular block if you know it's block hash, which you have just got from getblockhash (alternatively you can retrieve them from Blockchair).

getrawtransaction txhash true blockhash

Will also give you the vouts, and their values under the vout array --> but also, more importantly, their addresses in the vout array --> scriptPubKey --> addresses. With this method the address contained in vin is not directly available but you can retrieve it from the vouts of the connected transaction listed in vin --> txid.

The second argument is verbose=true it tells bitcoin core to return the result as JSON instead of a string. It even gives you the transaction hex in case you want to run decoderawtransaction on it.

Note that all this only works for transactions contained in the block identified by the block that you passed.

[TheWolf666]

Thanks for these answers, it helps to give me new idea to dig!

 

[odolvlobo]

Do you have example of command that works? I read the description of scantxoutset, but there's no example and i'm confused about 2nd parameter.

Nope. I have as much information as you. I read the description and it looks like the right command to use, but I have never used it.

[HCP]

Do you have example of command that works? I read the description of scantxoutset, but there's no example and i'm confused about 2nd parameter.

Which bit is confusing you?

Code:

Scans the unspent transaction output set for entries that match certain output descriptors.
Examples of output descriptors are:
    addr(<address>)                      Outputs whose scriptPubKey corresponds to the specified address (does not include P2PK)
    raw(<hex script>)                    Outputs whose scriptPubKey equals the specified hex scripts
    combo(<pubkey>)                      P2PK, P2PKH, P2WPKH, and P2SH-P2WPKH outputs for the given pubkey
    pkh(<pubkey>)                        P2PKH outputs for the given pubkey
    sh(multi(<n>,<pubkey>,<pubkey>,...)) P2SH-multisig outputs for the given threshold and pubkeys

In the above, <pubkey> either refers to a fixed public key in hexadecimal notation, or to an xpub/xprv optionally followed by one
or more path elements separated by "/", and optionally ending in "/*" (unhardened), or "/*'" or "/*h" (hardened) to specify all
unhardened or hardened child keys.
In the latter case, a range needs to be specified by below if different from 1000.
For more information on output descriptors, see the documentation in the doc/descriptors.md file.

More reading on descriptors as indicated in the descriptors.md file: https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md


The simplest is addr():

Code:

scantxoutset "start" "[\"addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)\"]"

Code:

{
  "success": true,
  "txouts": 70264820,
  "height": 668945,
  "bestblock": "00000000000000000008fa308c94179d3b6c4d4501cde114d14b33593bbeae59",
  "unspents": [
    {
      "txid": "f9a7240ad23bc84044ab83b593e11dd6686a2cfbc13731944259a6ee0f10295f",
      "vout": 1,
      "scriptPubKey": "76a9147ddb236e7877d5040e2a59e4be544c65934e573a88ac",
      "desc": "addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)#nsvz5zvh",
      "amount": 2.15076162,
      "height": 668945
    },
    {
      "txid": "4aa8d886f4a799b08b0063562f76dfd1bbe200ca3dc2bf39935f341fa9e44ffa",
      "vout": 1,
      "scriptPubKey": "76a9147ddb236e7877d5040e2a59e4be544c65934e573a88ac",
      "desc": "addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)#nsvz5zvh",
      "amount": 1.14902796,
      "height": 668944
    }
  ],
  "total_amount": 3.29978958
}

[TheWolf666]

Do you have example of command that works? I read the description of scantxoutset, but there's no example and i'm confused about 2nd parameter.

Which bit is confusing you?

Code:

Scans the unspent transaction output set for entries that match certain output descriptors.
Examples of output descriptors are:
    addr(<address>)                      Outputs whose scriptPubKey corresponds to the specified address (does not include P2PK)
    raw(<hex script>)                    Outputs whose scriptPubKey equals the specified hex scripts
    combo(<pubkey>)                      P2PK, P2PKH, P2WPKH, and P2SH-P2WPKH outputs for the given pubkey
    pkh(<pubkey>)                        P2PKH outputs for the given pubkey
    sh(multi(<n>,<pubkey>,<pubkey>,...)) P2SH-multisig outputs for the given threshold and pubkeys

In the above, <pubkey> either refers to a fixed public key in hexadecimal notation, or to an xpub/xprv optionally followed by one
or more path elements separated by "/", and optionally ending in "/*" (unhardened), or "/*'" or "/*h" (hardened) to specify all
unhardened or hardened child keys.
In the latter case, a range needs to be specified by below if different from 1000.
For more information on output descriptors, see the documentation in the doc/descriptors.md file.

More reading on descriptors as indicated in the descriptors.md file: https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md


The simplest is addr():

Code:

scantxoutset "start" "[\"addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)\"]"

Code:

{
  "success": true,
  "txouts": 70264820,
  "height": 668945,
  "bestblock": "00000000000000000008fa308c94179d3b6c4d4501cde114d14b33593bbeae59",
  "unspents": [
    {
      "txid": "f9a7240ad23bc84044ab83b593e11dd6686a2cfbc13731944259a6ee0f10295f",
      "vout": 1,
      "scriptPubKey": "76a9147ddb236e7877d5040e2a59e4be544c65934e573a88ac",
      "desc": "addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)#nsvz5zvh",
      "amount": 2.15076162,
      "height": 668945
    },
    {
      "txid": "4aa8d886f4a799b08b0063562f76dfd1bbe200ca3dc2bf39935f341fa9e44ffa",
      "vout": 1,
      "scriptPubKey": "76a9147ddb236e7877d5040e2a59e4be544c65934e573a88ac",
      "desc": "addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)#nsvz5zvh",
      "amount": 1.14902796,
      "height": 668944
    }
  ],
  "total_amount": 3.29978958
}

This does not answer to the question: "how to know how much an address own" or at least "how to know the source of a transaction".
It only gives you the sum of all the BTC spent by an address (in the case of the 'addr' command)
Ideally, it would be great to start from an address like 1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2 and then have a list of RPC calls that will get how much this address owns. If the wallet can do it, it should be also possible to do it from RPC.


Thanks for pointing that command that is not very well known though.

[NotATether]

If the wallet can do it, it should be also possible to do it from RPC.

Wallet software have it easy because they have already scanned the blockchain for all transactions related to a group of addresses (or have just queried that information from an SPV who already did that heavy work), and they can look for transactions from the block verification code itself instead of having to make RPC calls, since that would introduce more latency in the block verification step.

[odolvlobo]

The simplest is addr():

Code:

scantxoutset "start" "[\"addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)\"]"

Code:

{
  ...,
  "unspents": [
    {
      "txid": "f9a7240ad23bc84044ab83b593e11dd6686a2cfbc13731944259a6ee0f10295f",
      "vout": 1,
      "scriptPubKey": "76a9147ddb236e7877d5040e2a59e4be544c65934e573a88ac",
      "desc": "addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)#nsvz5zvh",
      "amount": 2.15076162,
      "height": 668945
    },
    {
      "txid": "4aa8d886f4a799b08b0063562f76dfd1bbe200ca3dc2bf39935f341fa9e44ffa",
      "vout": 1,
      "scriptPubKey": "76a9147ddb236e7877d5040e2a59e4be544c65934e573a88ac",
      "desc": "addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)#nsvz5zvh",
      "amount": 1.14902796,
      "height": 668944
    }
  ],
  "total_amount": 3.29978958
}

This does not answer to the question: "how to know how much an address own" or at least "how to know the source of a transaction".
It only gives you the sum of all the BTC spent by an address (in the case of the 'addr' command)

It looks to me like it gives you the amount of unspent. Isn't that what you want?

[TheWolf666]

The simplest is addr():

Code:

scantxoutset "start" "[\"addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)\"]"

Code:

{
  ...,
  "unspents": [
    {
      "txid": "f9a7240ad23bc84044ab83b593e11dd6686a2cfbc13731944259a6ee0f10295f",
      "vout": 1,
      "scriptPubKey": "76a9147ddb236e7877d5040e2a59e4be544c65934e573a88ac",
      "desc": "addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)#nsvz5zvh",
      "amount": 2.15076162,
      "height": 668945
    },
    {
      "txid": "4aa8d886f4a799b08b0063562f76dfd1bbe200ca3dc2bf39935f341fa9e44ffa",
      "vout": 1,
      "scriptPubKey": "76a9147ddb236e7877d5040e2a59e4be544c65934e573a88ac",
      "desc": "addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)#nsvz5zvh",
      "amount": 1.14902796,
      "height": 668944
    }
  ],
  "total_amount": 3.29978958
}

This does not answer to the question: "how to know how much an address own" or at least "how to know the source of a transaction".
It only gives you the sum of all the BTC spent by an address (in the case of the 'addr' command)

It looks to me like it gives you the amount of unspent. Isn't that what you want?

If I do the math on a real wallet, the total_amount is the sum of all VOUT which is not what is left unspent in the wallet, but what was spent (out) of the wallet.
I am working on a stacking system where I want to check how much an address is holding in order to reward a %APR for the BTC holders.
What I need yes is the unspent, but this command is not doing this, it lists all the transaction OUT using this address, but not sum of the transactions IN

I guess that with using getrawtransaction and save all the VOUT and the addresses, that would be possible to sum up all the BTC received from the specific address and then substract the total_amount to these numbers to get what is left holded by the address.
I will make some tests and post the result here.

[TheWolf666]

Ok I get it!

Let's do it with pseudo code.

Step 1 saving all the VOUT received by addresses in a database.

Code:

foreach tx {
  //save in a SQL database every VOUT and and its addresses
  $raw = getrawtransaction($txid)
  save (addresses, value)
}

Step 2 scan all addresses previously saved and do a scantxout

Code:

// use the addresses from the SQL database 
foreach address {
    $sum= sum(value)
    $scan=scantxoutset "start" "[\"addr(address)\"]"
    $owned = $sum-$scan("total_amount")
    save(address, $sum)
    save(address, $owned)
}

This should do the job. Now in the SQL database you can query an address, and get the money received, spent and owned.

[TheWolf666]

Note listunspent "address" will list all the unspent UTXO from the wallet. Can be entered into the console. Sum of them will give the total of the unspent UTXO available.

[HCP]

This does not answer to the question: "how to know how much an address own" or at least "how to know the source of a transaction".
It only gives you the sum of all the BTC spent by an address (in the case of the 'addr' command)

I think you're a little confused as to what scantxoutset is showing. It is scanning the UTXO set... which is the set of all unspent transaction outputs. It returns the UTXOs currently controlled by the descriptors that you provide... in other words "how much the address own".

My example address (as of 2021-02-04 23:20 UTC):




Is that not exactly what you're after?

If I do the math on a real wallet, the total_amount is the sum of all VOUT which is not what is left unspent in the wallet, but what was spent (out) of the wallet.

No... those "vout"'s are the vout's from the txid's that created that particular UTXO... which sent it to that address   (They're not vout's spending from the address)

Note listunspent "address" will list all the unspent UTXO from the wallet. Can be entered into the console. Sum of them will give the total of the unspent UTXO available.

Note that listunspent only works on addresses that are "in" your wallet.dat... it will not work for "external" (aka "random") addresses.

[TheWolf666]

I have a small problem but very boring.
I cannot find a way to code the scantxoutset command in a json valid format.

While the format

Code:

scantxoutset "start" "[\"addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)\"]" 

is working in the console, encoding it like this

Code:

{"method":"scantxoutset","params":["\"start\" [\"addr(1F1tAaz5x1HUXrCNLbtMDqcw6o5GNn4xqX)\"]"]}

is a correct json format, but it is returning Invalid Method error.

Anyone see what is wrong?

[NotATether]

I have a small problem but very boring.
I cannot find a way to code the scantxoutset command in a json valid format.

While the format

Code:

scantxoutset "start" "[\"addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)\"]" 

is working in the console, encoding it like this

Code:

{"method":"scantxoutset","params":["\"start\" [\"addr(1F1tAaz5x1HUXrCNLbtMDqcw6o5GNn4xqX)\"]"]}

is a correct json format, but it is returning Invalid Method error.

Anyone see what is wrong?

Core is interpreting other parts of the RPC method as the name.

It looks like you are using CURL to access the API, or a C wrapper that uses curl. Did you make sure to backslash all special characters and wrap everything, including the curly braces, between single quotes, or double quotes if using C?

[HCP]

What OS are you running this on? And what language are you using for making the call?

It's likely, as NotATether has suggested, an "escaping" issue... I've had so many issues with Bitcoin Core and getting the JSON formatting "just so"

[TheWolf666]

I use PHP this way

Code:

        $method='"start";
        $params= "[\"addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)\"]";

        // request is converted in JSON
        $request = json_encode(array(
            'method' => $method,
            'params' => $params,
        ));

[HCP]

Have you tried it without the \ escape characters within the PHP?

I suspect that they are not required... the \" stuff seems to be a "console" requirement... I would expect that the json_encode() function would take care of the escaping/formatting required, so maybe try something like:

Code:

    $method= "start";
    $params= "addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2";

    // request is converted in JSON
    $request = json_encode(array(
        'method' => $method,
        'params' => $params,
    ));

Failing that... debug and just dump out the value of $request before executing it... you'll likely see some weird escaping happening would be my guess... like, it'll be escaping the \'s or something stupid like that.

[TheWolf666]

Have you tried it without the \ escape characters within the PHP?

I suspect that they are not required... the \" stuff seems to be a "console" requirement... I would expect that the json_encode() function would take care of the escaping/formatting required, so maybe try something like:

Code:

    $method= "start";
    $params= "addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2";

    // request is converted in JSON
    $request = json_encode(array(
        'method' => $method,
        'params' => $params,
    ));

Failing that... debug and just dump out the value of $request before executing it... you'll likely see some weird escaping happening would be my guess... like, it'll be escaping the \'s or something stupid like that.

Not working, the resulting JSON is like this:

Code:

{"method":"scantxoutset","params":["\"start\" \"addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2\""]}

This drives me crazy, there should be JSON examples in the Bitcoin documentation.

[HCP]

oops... I missed a ) in my code...    

Code:

   $method= "start";
    $params= "addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)";

    // request is converted in JSON
    $request = json_encode(array(
        'method' => $method,
        'params' => $params,
    ));

Although, with that debug output, you might need to use:

Code:

$params = "[addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)]";

As I'm fairly sure it needs to be an array and requires the [ ]'s

[TheWolf666]

It seems that there is a problem specific to windows.

For example

This is working on Unix

Code:

./bitcoin-cli scantxoutset start  '["addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)"]'

But this is not on windows

Code:

bitcoin-cli.exe scantxoutset start  '["addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)"]'

error: Error parsing JSON:'[addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)]'

Escaping like this bitcoin-cli.exe scantxoutset start  '[\"addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)\"]'
does not change anything, it fails.

Any clue?

[HCP]

It'll be the way that the Windows "shell" is interpreting the characters... are you using cmd.exe/"command prompt" or powershell?

The format for using scantxoutset on the Windows command prompt seems to be:

Code:

bitcoin-cli.exe scantxoutset start [\"addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)\"]

[TheWolf666]

It'll be the way that the Windows "shell" is interpreting the characters... are you using cmd.exe/"command prompt" or powershell?

The format for using scantxoutset on the Windows command prompt seems to be:

Code:

bitcoin-cli.exe scantxoutset start [\"addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)\"]

Actually you are right, it is working, I am using cmd because I call it from PHP and that's what it does.,
Here is the solution:

From the command prompt, no " or ' are accepted.

Code:

bitcoin-cli.exe scantxoutset start [\"addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)\"]

From PHP retval is an array with the response and $r is the return code true or false

Code:

$address="1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2";
$s='bitcoin-cli.exe scantxoutset start [\"addr('.$address.')\"] ';
exec($s,$retval,$r);

This solution is working but it sucks because it is really slow compared to RPC.
I could not get a positive result with any JSON formatted submission to RPC.  

[NotATether]

It seems that there is a problem specific to windows.

...

But this is not on windows

Code:

bitcoin-cli.exe scantxoutset start  '["addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)"]'

error: Error parsing JSON:'[addr(1CUTyyxgbKvtCdoYmceQJCZLXCde5akiX2)]'

In Windows' command prompt, the left and right parentheses are considered special characters and have to be escaped with a ^ character, not a backslash. They are not special in Linux which is why it works there. So try escaping those in your RPC call and try it again.

The Windows command-line interpreter uses a caret character (^) to escape reserved characters that have special meanings (in particular: &, |, (, ), <, >, ^).[4] The DOS command-line interpreter, though it has similar syntax, does not support this.