How to efficiently extract values of vin in blockchain transactions

[PVminer]

I'm trying to analyze fees in the blockchain and I wrote a script in Python using bitcoinrpc library and getrawtransaction RPC call. However, getrawtransaction does not print vin values, only corresponding txids so I had to read all those transactions and extract the values from vout of the parent transactions. The script works but is very slow, each block takes a few minutes to process.

Is there a faster way to do it? I want a local solution (I can patch Bitcoin Core if necessary) but downloading e.g. from blockchain.info will not work because they have limits on the number of queries.

[1715443648]

1715443648

[1715443648]

1715443648

[DannyHamilton]

Is there a faster way to do it? I want a local solution (I can patch Bitcoin Core if necessary) but downloading e.g. from blockchain.info will not work because they have limits on the number of queries.

You could write a program to maintain your own indexed UTXO list, and then just grab the values from your list.

[PVminer]

You could write a program to maintain your own indexed UTXO list, and then just grab the values from your list.

The problem is that the input transaction does not have to be unspent and the list of all outputs is going to be a large database. And initializing it via RPC will be slow.

[DannyHamilton]

the list of all outputs is going to be a large database.

And this is why it is so slow for you to look it up right now.  The blockchain is effectively a database of all outputs.  The value of the input is NOT stored in the transaction.  It is ONLY stored in the output that is being spent, so you have to go find it (either in the blockchain or your own database).

[byteball]

Have a look at https://github.com/metrodexcoin/litecoin/blob/getblocksummary/src/rpc/misc.cpp#L651

getblocksummary RPC method.

The code that might help you to calculate each tx fee is commented out, in lines 691-700.

The reason I commented it out (after writing it myself) was that I only needed the fee for entire block,
which I can find in the coinbase transaction. Line 706 has the calculation.

In your case, you would need a similar method that returns fee for every transaction in the block.
Having -txindex is vital for this (see line 660), as transactions with all outputs already spent are not linked from UTXO
and therefore not easily findable, unless you know their block hash or height.

I can do this for a small donation or bounty, when I have free time, if C++ is alien to you