Using pybitcointools to validate tx_valid.json test vectors

[HeadsOrTails]

I'm using this fork of the pybitcointools library and am trying to use the Bitcoincore JSON test vectors for unittesting.

Can someone advise:

1. How ECDSA is used to verify transactions..?
2. Ideally, how pybitcointools can be used do #1

From tx_valid.json, the first test vector is:

Code:

[
["The following are deserialized transactions which are valid."],
["They are in the form"],
["[[[prevout hash, prevout index, prevout scriptPubKey], [input 2], ...],"],
["serializedTransaction, enforceP2SH]"],
["Objects that are only a single string (like this one) are ignored"],

["The following is 23b397edccd3740a74adb603c9756370fafcde9bcc4483eb271ecad09a94dd63"],
["It is of particular interest because it contains an invalidly-encoded signature which OpenSSL accepts"],
["See http://r6.ca/blog/20111119T211504Z.html"],
["It is also the first OP_CHECKMULTISIG transaction in standard form"],
[[["60a20bd93aa49ab4b28d514ec10b06e1829ce6818ec06cd3aabd013ebcdc4bb1", 0, "1 0x41 0x04cc71eb30d653c0c3163990c47b976f3fb3f37cccdcbedb169a1dfef58bbfbfaff7d8a473e7e2e6d317b87bafe8bde97e3cf8f065dec022b51d11fcdd0d348ac4 0x41 0x0461cbdcc5409fb4b4d42b51d33381354d80e550078cb532a34bfa2fcfdeb7d76519aecc62770f5b0e4ef8551946d8a540911abe3e7854a26f39f58b25c15342af 2 OP_CHECKMULTISIG"]],
"0100000001b14bdcbc3e01bdaad36cc08e81e69c82e1060bc14e518db2b49aa43ad90ba26000000000490047304402203f16c6f40162ab686621ef3000b04e75418a0c0cb2d8aebeac894ae360ac1e780220ddc15ecdfc3507ac48e1681a33eb60996631bf6bf5bc0a0682c4db743ce7ca2b01ffffffff0140420f00000000001976a914660d4ef3a743e3e696ad990364e555c271ad504b88ac00000000", true]

Now, we have the following:

Code:

# inputs
ins = ["%s:%d" % ("60a20bd93aa49ab4b28d514ec10b06e1829ce6818ec06cd3aabd013ebcdc4bb1", 0)]	# ins = ["prevout hash:prevout index"]

# scriptPubKey of inputs
scriptPubKey = '514104cc71eb30d653c0c3163990c47b976f3fb3f37cccdcbedb169a1dfef58bbfbfaff7d8a473e7e2e6d317b87bafe8bde97e3cf8f065dec022b51d11fcdd0d348ac4410461cbdcc5409fb4b4d42b51d33381354d80e550078cb532a34bfa2fcfdeb7d76519aecc62770f5b0e4ef8551946d8a540911abe3e7854a26f39f58b25c15342af52ae'

outs = [{'value': 1000000, 'script': '76a914660d4ef3a743e3e696ad990364e555c271ad504b88ac'}] 
raw = mktx(ins, outs)      
# "0100000001b14bdcbc3e01bdaad36cc08e81e69c82e1060bc14e518db2b49aa43ad90ba2600000000000ffffffff0140420f00000000001976a914660d4ef3a743e3e696ad990364e555c271ad504b88ac00000000"

# from test_vector ()or fetching the Tx: 23b397edccd3740a74adb603c9756370fafcde9bcc4483eb271ecad09a94dd63:0)
signed = "0100000001b14bdcbc3e01bdaad36cc08e81e69c82e1060bc14e518db2b49aa43ad90ba26000000000490047304402203f16c6f40162ab686621ef3000b04e75418a0c0cb2d8aebeac894ae360ac1e780220ddc15ecdfc3507ac48e1681a33eb60996631bf6bf5bc0a0682c4db743ce7ca2b01ffffffff0140420f00000000001976a914660d4ef3a743e3e696ad990364e555c271ad504b88ac00000000"

der_sig = "0048304402203f16c6f40162ab686621ef3000b04e75418a0c0cb2d8aebeac894ae360ac1e780220ddc15ecdfc3507ac48e1681a33eb60996631bf6bf5bc0a0682c4db743ce7ca2bab01"    # NOTE: this signature is not canonical because both the r and s values have the highest bit set (ie ideally r and s should be padded with nullbytes)

So given all the information from the test vectors, how does one verify the signature against the raw transaction?

(NB. I'm doing this as a means to learn the way Bitcoin signs/verifies transactions. So far I've had no issue with these unittests)

[fbueller]

It looks like the verify_tx_input function is the smallest atomic function in the library that can validate signatures.

`verify_tx_input      : (tx, i, script, sig, pub) -> True/False`


Since you're in the process of learning:

Each input in a transaction has a different signature to the other. Have a look at https://en.bitcoin.it/wiki/OP_CHECKSIG to see in detail how a transaction is altered to capture all the necessary data for this SIGHASH type. The final step is to sha256d() this modified version, and return this as the hash for ECDSA.

(The SIGHASH flags all have different meanings, but generally speaking, they just mean different data is committed to in the hash. The consequences are beyond the scope of this, but if you read the article, it's the best place to start).

So each transaction input has a signature valid for this [txid, vout, scriptPubKey] combination. Because input signatures are unique to the input, it seems pybitcointools only includes a verify_tx_input function. The likely reason for this is, as you can see, this functions argument signature has 5 arguments.. That's a lot of data just for each input, so a verify_tx() function would depend on a lot of data, which mightn't be available if you're not a full node.

The `txi`, `i`, `script` fields are properties of the transaction you are spending (script == prevOut's scriptPubKey), and are used to produce the hash for ECDSA. Everyone with the blockchain and is doing full validation would now check the signature. The public key is known because it was pushed to the stack either in the prev out scriptPubKey (pay-to-pubkey) or the scriptSig (pay-to-pubkey-hash). And the signature will also be known.

So this is sufficient information for any node on the network to validate a transaction. When you sign a transaction, you need to know the same information, but need to modify the script afterwards, and if it's fully signed, you'll broadcast it.  

Note; in bitcoin core, the script language is the means of pushing signatures/pubkeys/opcodes to the stack. After the sig/pubkey is pushed at some point, OP_CHECKSIG is called

On to your code section.

Der_sig appears not to be just the sig - It's the full scriptSig. See the null byte at the start.
The public key, can be 1-of-2, because the scriptPubKey you put up there is a multisig output script. So both public keys will be checked by bitcoin core.

Code:

txid = "60a20bd93aa49ab4b28d514ec10b06e1829ce6818ec06cd3aabd013ebcdc4bb1"
vout = 0
scriptPubKey = '514104cc71eb30d653c0c3163990c47b976f3fb3f37cccdcbedb169a1dfef58bbfbfaff7d8a473e7e2e6d317b87bafe8bde97e3cf8f065dec022b51d11fcdd0d348ac4410461cbdcc5409fb4b4d42b51d33381354d80e550078cb532a34bfa2fcfdeb7d76519aecc62770f5b0e4ef8551946d8a540911abe3e7854a26f39f58b25c15342af52ae'
outs = [{'value': 1000000, 'script': '76a914660d4ef3a743e3e696ad990364e555c271ad504b88ac'}] 
...

# This is actually a der sig plus a byte at the end indicating the SIGHASH type. This must be included. 
der_sig = "304402203f16c6f40162ab686621ef3000b04e75418a0c0cb2d8aebeac894ae360ac1e780220ddc15ecdfc3507ac48e1681a33eb60996631bf6bf5bc0a0682c4db743ce7ca2bab01" 

# I just picked a random one from the scriptPubKey, this mightnt return TRUE. 
pubkey = "04cc71eb30d653c0c3163990c47b976f3fb3f37cccdcbedb169a1dfef58bbfbfaff7d8a473e7e2e6d317b87bafe8bde97e3cf8f065dec022b51d11fcdd0d348ac4"

# Verify the input
verify_tx_input(txid, vout, scriptPubKey, der_sig, pubkey)

See how I magically defined der_sig and pubkey as constants? Well, there are different output script types, and each is satisfied differently. (in this case, a sig from 1 of 2 keys will do). Depending on the script type, the public keys can be related in different ways. Getting into this is beyond the scope, but studying the OP_CHECKSIG wiki page will show you how it works for pay-to-pubkey-hash.

The multisig-in-the-output-script type is quite similar to this, though calls OP_CHECKMULTISIG. This is mainly just a loop over OP_CHECKSIG.

The next thing to look at this the scripting engine, which ties all of this together. https://en.bitcoin.it/wiki/Script -  at the bottom you'll find step-by-step analysis of how scripts are parsed, ultimately to lead to a CHECKSIG call.

Also - here's the bitcoin core code for OP_CHECKSIG: https://github.com/bitcoin/bitcoin/blob/master/src/script/interpreter.cpp#L827-L860

And the actual function TransactionSignatureChecker::CheckSig: https://github.com/bitcoin/bitcoin/blob/master/src/script/interpreter.cpp#L1108-L1127

[HeadsOrTails]

Thanks so much for the reply!

A quick query: how would I redeem this Tx I created? (If possible)

I'm trying to push the 03... public key to redeem (the IF clause)