Verifying OP_CHECKSIG in a non-segwit block

[vimrull]

I am trying to validate a transaction from block 170. There are two outputs 10+40=50 bitcoins total. The txo is coming from block 9.

Now I have loaded both of the blocks from hex data downloaded from blockchain.info site:

https://blockchain.info/rawblock/000000008d9dc510f23c2657fc4f67bea30078cc05a90eb89e84cc475c080805?format=hex
https://blockchain.info/rawblock/00000000d1145790a8694403d4063f323d499e655c83426834d4ce2f8dd4a2ee?format=hex

I am able to load it using some code I wrote and validate most of the parameters of the blocks. But when I try to validate the ECDSA signature using openssl functions using this code: https://github.com/vimrull/vimrull/blob/b2dcb47ec109c021e67d9088c1e7eca69335f521/tests/crypto/test_open_ssl.cpp#L5  I am unable to validate it. To validate I load the block and separate the transaction and double hash it. Then get the public key and the signature from the block and call openssl validate method. Here is my test case: https://github.com/vimrull/vimrull/blob/b2dcb47ec109c021e67d9088c1e7eca69335f521/tests/vm/testscript.cpp#L35

When it did not work I took the raw transaction from https://en.bitcoin.it/wiki/OP_CHECKSIG and use the public key and signature from the block and it validates successfully.
 
Transaction that can be validated (TXV):

Code:

0100000001c997a5e56e104102fa209c6a852dd90660a20b2d9c352423edce25857fcd37040000000043410411db9
3e1dcdb8a016b49840f8c53bc1eb68a382e97b1482ecad7b148a6909a5cb2e0eaddfb84ccf9744464f82e160bfa9b8b
64f9d4c03f999b8643f656b412a3acffffffff0200ca9a3b00000000434104ae1a62fe09c5f51b13905f07f06b99a2f7159
b2225f374cd378d71302fa28414e7aab37397f554a7df5f142c21c1b7303b8a0626f1baded5c72a704f7e6cd84cac002
86bee0000000043410411db93e1dcdb8a016b49840f8c53bc1eb68a382e97b1482ecad7b148a6909a5cb2e0eaddfb
84ccf9744464f82e160bfa9b8b64f9d4c03f999b8643f656b412a3ac0000000001000000

Double sha256 of the above data is 7a05c6145f10101e9d6325494245adf1297d80f8f38d4d576d57cdba220bcb19 which I cannot find in the blockchain.info site. But this hash and public key and signature combination is valid according to openssl.

Here is the transaction from the actual block (TXB):

Code:

0100000001c997a5e56e104102fa209c6a852dd90660a20b2d9c352423edce25857fcd3704000000004847304402204
e45e16932b8af514961a1d3a1a25fdf3f4f7732e9d624c6c61548ab5fb8cd410220181522ec8eca07de4860a4acdd1290
9d831cc56cbbac4622082221a8768d1d0901ffffffff0200ca9a3b00000000434104ae1a62fe09c5f51b13905f07f06b99a2
f7159b2225f374cd378d71302fa28414e7aab37397f554a7df5f142c21c1b7303b8a0626f1baded5c72a704f7e6cd84cac
00286bee0000000043410411db93e1dcdb8a016b49840f8c53bc1eb68a382e97b1482ecad7b148a6909a5cb2e0eaddf
b84ccf9744464f82e160bfa9b8b64f9d4c03f999b8643f656b412a3ac00000000

The reverse of the double hash of the transaction is f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16 which can be found on blockchain.info site but if I take signature and public key found on block 170 it can not be validated.

I am guessing the serialization of the transaction during validation may be a little different than what is in raw transaction (or blockchain.info has wrong block data - which is unlikely). In my code I serialize the transaction that matches the binary format of the block. But the transaction raw value that works can not be found in actual block.

So, I am wondering how do I get/generate the data TXV when I have the data TXB in the block.

Explaining this seems to be hard- but the test case may make it a bit clear. At the HEAD of the repository the test case is passing because I have just hardcoded the raw tx in the code.

I appreciate if you take time to point any issue you see with my test case approach. As a newbie I am probably missing something very obvious.

And please excuse my code. I am trying to learn modern C++ as well while trying this project.

[pooya87]

I am guessing the serialization of the transaction during validation may be a little different than what is in raw transaction (or blockchain.info has wrong block data - which is unlikely).

Serialization is the same, but some parts of the transaction must be modified before being hashed. Your first raw bytes in the code tag with the hash = 0x7a05c6... looks correct, you replace the signature script of the transaction with the pubkey script of the output that is being spent (set the rest of the inputs signature script to empty, here there is only one) add the SigHashType at the end as 4 bytes and then hash it.

Read this for details: https://bitcoin.stackexchange.com/questions/32628/redeeming-a-raw-transaction-step-by-step-example-required

In my code I serialize the transaction that matches the binary format of the block. But the transaction raw value that works can not be found in actual block.

Only the signed transaction is going to be included in the block not the modification or the hash of it.

The reverse of the double hash of the transaction is f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16 which can be found on blockchain.info site but if I take signature and public key found on block 170 it can not be validated.

That is the transaction ID and it is only used to reference transactions not to be used for signature verification.

[vimrull]

Serialization is the same, but some parts of the transaction must be modified before being hashed. ... you replace the signature script of the transaction with the pubkey script of the output that is being spent

Perfect - thanks for the pointer. Basically just after the previous transaction hash and index, goes the pubkey script from old blocks transaction:

Code:

01000000 # version
01       # one input
c997a5e56e104102fa209c6a852dd90660a20b2d9c352423edce25857fcd3704  # tx hash from block 9
00000000 # input index 0
43       # script length
# 0x41 bytes of data = public key + OP_CHECKSIG
410411db93e1dcdb8a016b49840f8c53bc1eb68a382e97b1482ecad7b148a6909
a5cb2e0eaddfb84ccf9744464f82e160bfa9b8b64f9d4c03f999b8643f656b412a3ac
ffffffff
02       # we have 2 outputs
00ca9a3b00000000 # 10 coins
43               # length of script
# first outout sending 10 coints to a new address
4104ae1a62fe09c5f51b13905f07f06b99a2f7159b2225f374cd378d71302fa28414e7
aab37397f554a7df5f142c21c1b7303b8a0626f1baded5c72a704f7e6cd84cac
00286bee00000000 # 40 coins
43               # length of script
# second output - sending 40 coins to address of coin origin
410411db93e1dcdb8a016b49840f8c53bc1eb68a382e97b1482ecad7b148a6909a5cb2e
0eaddfb84ccf9744464f82e160bfa9b8b64f9d4c03f999b8643f656b412a3ac
00000000 # lock time
01000000 # hash type

This makes sense - before the transaction signature is generated we do not have it and putting the signature inside the transaction that is being signed and generating hash creates an impossible chicken egg scenario.

This seems like solves the validation problem of most non-segwit signature - with slightly altered version to use bitcoin address instead of public  key in later versions of the payment script. I'll have to understand multi-sig and segwits signature validation. Good thing is as a side effect of this exercise, I now know how to generate a raw transaction.

[If you see this, you are not pooya87 and have spendable merit, please send some for the previous reply - helped me a lot and I do not have sMerit to signal that to the system]

[pooya87]

This seems like solves the validation problem of most non-segwit signature - with slightly altered version to use bitcoin address instead of public  key in later versions of the payment script.

Keep in mind that things are a bit more complicated than that when serializing a transaction for signing. The SE link and the explanation above works for standard outputs such as P2PK, P2PKH and for SigHash_All type, however if the SigHash type is different or if there is OP_CodeSeparator inside the output scripts being spent, things will be very different.
Basically:
- If the spending script contains the signature (eg. signature inside the pubkey script of the output being spent, or signature in redeem script of a P2SH) it is removed but only if it was being pushed correctly to the stack.
- The actual script used during serialization starts from the last executed OP_CodeSeparator so if there were other OP codes before it they will be ignored.
- SigHash types change what inputs and outputs we sign.
-- ALL signs everything and is most common
-- Single signs 1 output
-- None signs no outputs
-- AnyoneCanPay signs 1 input
-- There are other various details about how they modify the tranasction

Check out FindAndDelete and CTransactionSignatureSerializer in interpreter.cpp for details.

[vimrull]

Keep in mind that things are a bit more complicated than that when serializing a transaction for signing...

Ha ha - absolutely. It was beginners high moment when the signature validation worked for the first time . It has been long time (after school) I am doing a project like this.

Thanks for the sighash type and code_separator comment. I'll delay the transaction generation part until I am able to do the validation of most common types of signature and the script interpreter engine. But, when I am back with tx generation, I'll be able to refer back to that.

So, with this the test case is now complete with only two raw blocks and without any hard coded values (except for sanity checks): https://github.com/vimrull/vimrull/blob/b2dcb47ec109c021e67d9088c1e7eca69335f521/tests/vm/testscript.cpp#L35