Read and convert UTXO to normal view!

[Tigr1987]

Good day! Faced one of the problems... After I completely unloaded all the data of their UTXO and led them to this view:
         {key--value}
{0e006f62667573636174655f6b6579--0868e8ea3a22510927}
{42--41cf808cc8db2348caffb27a963e7ffaa905feb62431222768e8ea3a22510927}{430000030d4e89146c95367f242332ecc7059d29fda2bff18e0aa6e8e9a4d1c33d00--aa3e8a2a23017c2ab0278593e4539f0aed9e0b177c61d2fbfc}{43000003502797e99d1cc6f9497253b2290514fdf647e12487c20c4b20e3833e0300--d16a9c98ce2609ecaa7085c38c878b7e4846242e883e5aa5a2bd6a}{4300000662484b459022b01156d71a584d67654e859c9da2566e155a887a46c9e101--fe7c8a9dedd30e27b1bf28694e715703eb5edff435e3e9075e90676e}{43000006b4e26afc5d904f239930611606a97e730727b40d1d82d4f3f1438cf2a101--a86cccbacf080984e7ddbbb7c6197537e6d0fadc5b1ebfac707561}{43000006eea2b069136698e059f0b1c95730814e68244c5d76da69de3e9e5b8f8100--dd3892202203a46392869022566d7fed1f3a4f05e190a94d04}{4300000706dd42cd45a2b9855215ed482eea768d1f5845a19365072d1539e4f5d200--d37eda3c22c307fad6c24889683b8deb8df6ed9d17e12623b2}{430000076acf9ecf1ab3911b5adc3b30cf0705fc8f947117b0569538d2fb690cea01--d370cab4ba6209203b0d19ef3e3b36ac884cd041de9d868445ed7c}
..................

What method can lead them to a normal form?
             Public key -- balance?
72e0d74f683d5ca6a5ae59b2db39be2dda20bddd4272895d4ee39f323cf9c113 -- 1.111111 BTC

where can I read about the conversion algorithm to this type? I ask for your help in this matter!

[HeRetiK]

I think a bit more context is needed here to make helping easier. What tool / API did you use to get this dataset?

[Tigr1987]

I think a bit more context is needed here to make helping easier. What tool / API did you use to get this dataset?

I wrote a script for node js that pulls the entire base of the key -- value pair and saves everything to a file

Code:

var level = require('level')
const fs = require("fs");
var i=0;
var db = level('chainstate' ,{keyEncoding:'hex',valueEncoding:'hex'})
var s='';


function hextoString(str){
const buf = new Buffer(str, 'hex');
return buf.toString('utf-8');
}

db.createReadStream({ keys: true, values: true })
  .on('data', function (data) {
	
s='{'+data.key+'--'+data.value+'}'
console.log(s);	

fs.appendFileSync("base.db", s);
i=i+1;
	console.log(i);
	
    
 })

console.log('**************************************');

[HeRetiK]

I think a bit more context is needed here to make helping easier. What tool / API did you use to get this dataset?

I wrote a script for node js that pulls the entire base of the key -- value pair and saves everything to a file

What are you trying to do? Unless you are after a very specific use case I'm fairly certain there are easier ways to achieve your goal rather than trying to parse Bitcoin's chainstate LevelDB.

[Tigr1987]

I think a bit more context is needed here to make helping easier. What tool / API did you use to get this dataset?

I wrote a script for node js that pulls the entire base of the key -- value pair and saves everything to a file

What are you trying to do? Unless you are after a very specific use case I'm fairly certain there are easier ways to achieve your goal rather than trying to parse Bitcoin's chainstate LevelDB.

how does it transform into this kind of thing?:
Public key -- balance?
72e0d74f683d5ca6a5ae59b2db39be2dda20bddd4272895d4ee39f323cf9c113 -- 1.111111 BTC

of course, if there is another option to unload chainstate as a public key balance.... well not bad, but only part of the writing of the script direct to core without any third party apps!

[pooya87]

this sounds a lot like indexing that the block explorers do. so you may want to check out the source code of some of the open source block explorers to see how they are doing that like insight https://github.com/bitpay/insight maybe this can clarify your question so others can help you more.

and you keep giving this example:

Public key -- balance?
72e0d74f683d5ca6a5ae59b2db39be2dda20bddd4272895d4ee39f323cf9c113 -- 1.111111 BTC

but 72e... is not a public key! you seem to be doing something wrong so far to this point in your code too.

[Tigr1987]

and you keep giving this example:

Public key -- balance?
72e0d74f683d5ca6a5ae59b2db39be2dda20bddd4272895d4ee39f323cf9c113 -- 1.111111 BTC

but 72e... is not a public key! you seem to be doing something wrong so far to this point in your code too.

this public key was generated by me from the lantern as an example ... which is a hash from private.... from which you can get the address of the purse after dancing with a tambourine in the form of ripemd 160 and recode in base58
But the crux of the issue is: by what algorithm we can get these data .... the ones we pulled from UTXO?

the topic which was discussed here before ... as part of the decoding of UTXO data is obtained today is not relevant... either change the encoding algorithm or am I missing something!

https://bitcointalk.org/index.php?topic=647198.0;all

Code:

/** pruned version of CTransaction: only retains metadata and unspent transaction outputs
 *
 * Serialized format:
 * - VARINT(nVersion)
 * - VARINT(nCode)
 * - unspentness bitvector, for vout[2] and further; least significant byte first
 * - the non-spent CTxOuts (via CTxOutCompressor)
 * - VARINT(nHeight)
 *
 * The nCode value consists of:
 * - bit 1: IsCoinBase()
 * - bit 2: vout[0] is not spent
 * - bit 4: vout[1] is not spent
 * - The higher bits encode N, the number of non-zero bytes in the following bitvector.
 *   - In case both bit 2 and bit 4 are unset, they encode N-1, as there must be at
 *     least one non-spent output).
 *
 * Example: 0104835800816115944e077fe7c803cfa57f29b36bf87c1d358bb85e
 *          <><><--------------------------------------------><---->
 *          |  \                  |                             /
 *    version   code             vout[1]                  height
 *
 *    - version = 1
 *    - code = 4 (vout[1] is not spent, and 0 non-zero bytes of bitvector follow)
 *    - unspentness bitvector: as 0 non-zero bytes follow, it has length 0
 *    - vout[1]: 835800816115944e077fe7c803cfa57f29b36bf87c1d35
 *               * 8358: compact amount representation for 60000000000 (600 BTC)
 *               * 00: special txout type pay-to-pubkey-hash
 *               * 816115944e077fe7c803cfa57f29b36bf87c1d35: address uint160
 *    - height = 203998
 *
 *
 * Example: 0109044086ef97d5790061b01caab50f1b8e9c50a5057eb43c2d9563a4eebbd123008c988f1a4a4de2161e0f50aac7f17e7f9555caa486af3b
 *          <><><--><--------------------------------------------------><----------------------------------------------><---->
 *         /  \   \                     |                                                           |                     /
 *  version  code  unspentness       vout[4]                                                     vout[16]           height
 *
 *  - version = 1
 *  - code = 9 (coinbase, neither vout[0] or vout[1] are unspent,
 *                2 (1, +1 because both bit 2 and bit 4 are unset) non-zero bitvector bytes follow)
 *  - unspentness bitvector: bits 2 (0x04) and 14 (0x4000) are set, so vout[2+2] and vout[14+2] are unspent
 *  - vout[4]: 86ef97d5790061b01caab50f1b8e9c50a5057eb43c2d9563a4ee
 *             * 86ef97d579: compact amount representation for 234925952 (2.35 BTC)
 *             * 00: special txout type pay-to-pubkey-hash
 *             * 61b01caab50f1b8e9c50a5057eb43c2d9563a4ee: address uint160
 *  - vout[16]: bbd123008c988f1a4a4de2161e0f50aac7f17e7f9555caa4
 *              * bbd123: compact amount representation for 110397 (0.001 BTC)
 *              * 00: special txout type pay-to-pubkey-hash
 *              * 8c988f1a4a4de2161e0f50aac7f17e7f9555caa4: address uint160
 *  - height = 120891
 */

[pooya87]

this public key was generated by me from the lantern as an example ... which is a hash from private.... from which you can get the address of the purse after dancing with a tambourine in the form of ripemd 160 and recode in base58
But the crux of the issue is: by what algorithm we can get these data .... the ones we pulled from UTXO?

you don't hash a private key to get the public key! you use the elliptic curve's generator point and multiply it by the private key value. and you don't "pull it from UTXO"! the only data that the transactions contain are the signature (ie. R and S value) and the public key and the scriptpubkey which can be used to obtain the address from.

so by having a raw transaction (i am assuming you are aware that UTXO is unspent transaction output) you can get the following:
1. sender's public key which you will have no way of turning into an address unless you also check the scriptpub of the transaction that it is spending by looking at the output's txid field and getting the raw transaction from your blockchain.
2. receiving address by simply taking the scriptpub of this transaction that you are parsing and converting it to its appropriate address.

the topic which was discussed here before ... as part of the decoding of UTXO data is obtained today is not relevant... either change the encoding algorithm or am I missing something!

so you are trying to decode the UTXO database that bitcoin core stores? in which case you have to check the latest version of bitcoin core to see how it is de/serializing them

[Tigr1987]

you don't hash a private key to get the public key! you use the elliptic curve's generator point and multiply it by the private key value. and you don't "pull it from UTXO"! the only data that the transactions contain are the signature (ie. R and S value) and the public key and the scriptpubkey which can be used to obtain the address from.

128b87b572c379e92d8dd9f2f89118ec0bd8fe7ad6214312486d572d8718f8c7 -Private Key
a7b9b9a51a294e09d06e5d4d91b720a3a74be2935c1b570f565daed1a8a8d184 - Public Key
19C1yVYeWBouc4ZWc1zm3xaUzNBNqWyB3G - adress
https://bitnovosti.com/wp-content/uploads/2014/01/but1.png

[pooya87]

128b87b572c379e92d8dd9f2f89118ec0bd8fe7ad6214312486d572d8718f8c7 -Private Key
a7b9b9a51a294e09d06e5d4d91b720a3a74be2935c1b570f565daed1a8a8d184 - Public Key
19C1yVYeWBouc4ZWc1zm3xaUzNBNqWyB3G - adress

first of all, a public key (according to the agreed upon standard) must start with 0x04 if it is uncompressed, and 0x02 or 0x03 if it is compressed. yours doesn't start with either.
additionally that is not the public key of the private key you posted. the correct public key is:
04C7F4A880E496EE7335424E9613E7B29D37CEF9151DE1751D44E9159C699E07A20038BF72B3FF4 FC35F1A6FF2E43F56DC4D5743BE3AFF67F1A7C5F6A9F227B47C
or the compressed form:
02C7F4A880E496EE7335424E9613E7B29D37CEF9151DE1751D44E9159C699E07A2