Script unit tests

[ThePiachu]

I currently started implementing Scripts in my Bitcoin client code. As I can't find too much data on interesting unit tests for many different Script opcodes, I decided to put this topic up for anyone interesting in collaborating on this.

Some simple tests I've created so far:

Code:

0x01, 0x00, 0x01, 0x01, OP_MAX, OP_1
01000101A451
0101

OP_0, OP_1, OP_2, OP_3, OP_4, OP_5, OP_6, OP_7, OP_8, OP_9, OP_10, OP_11, OP_12, OP_13, OP_14, OP_15, OP_16, OP_DEPTH
005152535455565758595A5B5C5D5E5F6074
0102030405060708090A0B0C0D0E0F1011

OP_NOP, OP_NOP1, OP_NOP2, OP_NOP3, OP_NOP4, OP_NOP5, OP_NOP6, OP_NOP7, OP_NOP8, OP_NOP9, OP_NOP10, OP_DEPTH
61B0B1B2B3B4B5B6B7B8B974
00

OP_1, OP_DUP, OP_1, OP_2, OP_NIP, OP_1, OP_2, OP_OVER, OP_1, OP_2, OP_3, OP_ROT, OP_1, OP_2, OP_SWAP, OP_1, OP_2, OP_TUCK, OP_1, OP_2, OP_2DROP, OP_1, OP_DROP, OP_1, OP_2, OP_2DUP, OP_1, OP_2, OP_3, OP_3DUP, OP_1, OP_2, OP_3, OP_4, OP_2OVER, OP_1, OP_2, OP_3, OP_4, OP_5, OP_6, OP_2ROT, OP_1, OP_2, OP_3, OP_4, OP_2SWAP
51765152775152785152537B51527C51527D51526D517551526E5152536F5152535470515253545556715152535472
01010201020102030102010201020102010201020301020301020304010203040506010203040102

format:

Readable opcodes by their name
Hash representation of the whole input
Hash representation of the output, with all the items remaining on the stack being concatenated

Anyone interesting in creating and testing more unit tests together?

[1714019969]

1714019969

[1714019969]

1714019969

[etotheipi]

A couple months ago, I extracted all the non standard scripts in the testnet blockchain.  There's some pretty interesting ones:

https://gist.github.com/1329788

However, there's no guarantee that each TxOut is spendable, so, only the ones that have corresponding TxIn scripts spending the TxOuts should be used.  Luckily, if it's a non-std TxOut script, there's usually a non-std TxIn script to spend it, so they should both be in that list.  Since TxHashes are listed you can do a search for matching scripts...

Of those, I extracted three for my unit tests (in Armory):

https://gist.github.com/1421550

That is the complete debugging state at every step of a 2-of-2 and 2-of-3 (with addr-verify but no OP_CHECKMULTISIG) and another 2-of-3 that does addr-verify and uses OP_CHECKMULTISIG.  Not only do they make good unit tests, but the debugging states are great for seeing how these scripts actually work. 

Keep in mind, those multi-sig txs are not the "standard" multi-sig types that will be in the upcoming client releases, but they will stress your script processor, and they are correct since they were accepted by the Satoshi client on the Testnet.

-Eto

[ThePiachu]

Hmm, are those dumps from the multi-sig complete, or abbreviated? Since you seem to be calling checksig on "30460221  047cf315", which would make one of them a public key. Aren't those supposed to be longer?

[etotheipi]

Hmm, are those dumps from the multi-sig complete, or abbreviated? Since you seem to be calling checksig on "30460221  047cf315", which would make one of them a public key. Aren't those supposed to be longer?

Sorry, they are abbreviated to make it readable, and they are unique so there is no ambiguity.  The debugging states were only intended to be examined by eye, to make sure that the script processor was doing the right thing.  If you want to run those tests yourself, you can grab the TxHash from the top of the debugging output, and search for it in the full list of Testnet scripts. 

In each of the testnet scripts, I included the raw script (in hex) and the full raw tx (in hex).  From that, you should have just about everything you could possibly need to run those scripts through your scripting engine.

[Gavin Andresen]

Anyone interesting in creating and testing more unit tests together?

Good Idea.  How are you creating the files?

What language are you writing in?  Anybody want to volunteer to write a Satoshi-bitcoin unit tester in C++ that parses ThePiachu's format? Or do you already have code that runs them? Committing a gazillion Script unit test cases to src/test/scriptTests/ sounds like a very good idea to me.

[ThePiachu]

For now, I'm not creating any files, just writing the tests by hand in my source file. Probably could write them down in a txt file and upload them somewhere (like either a directory in Bitcoin github, or some new repository).

I'm writing my code in Go and I guess at some point I might create some website that can run the Bitcoin Script from a text input. For now though I need to finish writing the code for a few last OPs.

Should we also write unit tests for operations that will generate errors?

[ThePiachu]

Created a repository for various Bitcoin Unit Tests and uploaded my initial tests:
https://github.com/ThePiachu/Bitcoin-Unit-Tests

[ThePiachu]

Uploaded a couple different tests, and created a new topic for general Unit Tests:
https://bitcointalk.org/index.php?topic=56323.0

[etotheipi]

Piachu,

I just pulled out the three transactions I referenced in my first post, and put them into a form that you should be able to copy them right into your tests.  Since all of them contain signature-checks, you will need the full serialized transaction -- which I have included.  If you don't want the scripts separately, you can just concatenate the scripts:  <TxIn | TxOut>


2-of-2 Transaction without OP_CHECKMULTISIG

Code:

Testnet Tx Hash:  1c9608650a912be7fa88eecec664e6fbfa4b676708697fa99c28b3370005f32d

PrevTxOutScript:  526b006b7dac7ca914fc1243972b59c1726735d3c5cca40e415039dce9879a6c936b7dac7ca914375dd72e03e7b5dbb49f7e843b7bef4a2cc2ce9e879a6c936b6c6ca2
CurrTxInScript:   483045022017462c29efc9158cf26f2070d444bb2b087b8a0e6287a9274fa36fad30c46485022100c6d4cc6cd504f768389637df71c1ccd452e0691348d0f418130c31da8cc2a6e8014104e83c1d4079a1b36417f0544063eadbc44833a992b9667ab29b4ff252d8287687bad7581581ae385854d4e5f1fcedce7de12b1aec1cb004cabb2ec1f3de9b2e60493046022100fdc7beb27de0c3a53fbf96df7ccf9518c5fe7873eeed413ce17e4c0e8bf9c06e022100cc15103b3c2e1f49d066897fe681a12e397e87ed7ee39f1c8c4a5fef30f4c2c60141047cf315904fcc2e3e2465153d39019e0d66a8aaec1cec1178feb10d46537427239fd64b81e41651e89b89fefe6a23561d25dddc835395dd3542f83b32a1906aeb
CurrRawTx:        01000000011c9608650a912be7fa88eecec664e6fbfa4b676708697fa99c28b3370005f32d01000000fd1701483045022017462c29efc9158cf26f2070d444bb2b087b8a0e6287a9274fa36fad30c46485022100c6d4cc6cd504f768389637df71c1ccd452e0691348d0f418130c31da8cc2a6e8014104e83c1d4079a1b36417f0544063eadbc44833a992b9667ab29b4ff252d8287687bad7581581ae385854d4e5f1fcedce7de12b1aec1cb004cabb2ec1f3de9b2e60493046022100fdc7beb27de0c3a53fbf96df7ccf9518c5fe7873eeed413ce17e4c0e8bf9c06e022100cc15103b3c2e1f49d066897fe681a12e397e87ed7ee39f1c8c4a5fef30f4c2c60141047cf315904fcc2e3e2465153d39019e0d66a8aaec1cec1178feb10d46537427239fd64b81e41651e89b89fefe6a23561d25dddc835395dd3542f83b32a1906aebffffffff01c0d8a700000000001976a914fc1243972b59c1726735d3c5cca40e415039dce988ac00000000


Pretty TxIn Script:
   [PUSHDATA -- 72 BYTES:]
   3045022017462c29efc9158cf26f2070d444bb2b087b8a0e6287a9274fa36fad30c46485022100c6d4cc6cd504f768389637df71c1ccd452e0691348d0f418130c31da8cc2a6e801
   [PUSHDATA -- 65 BYTES:]
   04e83c1d4079a1b36417f0544063eadbc44833a992b9667ab29b4ff252d8287687bad7581581ae385854d4e5f1fcedce7de12b1aec1cb004cabb2ec1f3de9b2e60
   [PUSHDATA -- 73 BYTES:]
   3046022100fdc7beb27de0c3a53fbf96df7ccf9518c5fe7873eeed413ce17e4c0e8bf9c06e022100cc15103b3c2e1f49d066897fe681a12e397e87ed7ee39f1c8c4a5fef30f4c2c601
   [PUSHDATA -- 65 BYTES:]
   047cf315904fcc2e3e2465153d39019e0d66a8aaec1cec1178feb10d46537427239fd64b81e41651e89b89fefe6a23561d25dddc835395dd3542f83b32a1906aeb

Pretty TxOut Script:
   2
   OP_TOALTSTACK
   OP_0
   OP_TOALTSTACK
   OP_TUCK
   OP_CHECKSIG
   OP_SWAP
   OP_HASH160
   [PUSHDATA -- 20 BYTES:]
   fc1243972b59c1726735d3c5cca40e415039dce9
   OP_EQUAL
   OP_BOOLAND
   OP_FROMALTSTACK
   OP_ADD
   OP_TOALTSTACK
   OP_TUCK
   OP_CHECKSIG
   OP_SWAP
   OP_HASH160
   [PUSHDATA -- 20 BYTES:]
   375dd72e03e7b5dbb49f7e843b7bef4a2cc2ce9e
   OP_EQUAL
   OP_BOOLAND
   OP_FROMALTSTACK
   OP_ADD
   OP_TOALTSTACK
   OP_FROMALTSTACK
   OP_FROMALTSTACK
   OP_GREATERTHANOREQUAL

2-of-3 Transaction without OP_CHECKMULTISIG

Code:

Testnet Tx Hash:  2f654d4d1d7246d1a824c5b6c5177c0b5a1983864579aabb88cabd5d05e032e2

PrevTxOutScript:   526b006b7dac7ca9143cd1def404e12a85ead2b4d3f5f9f817fb0d46ef879a6c936b7dac7ca9146a4e7d5f798e90e84db9244d4805459f87275943879a6c936b7dac7ca914486efdd300987a054510b4ce1148d4ad290d911e879a6c936b6c6ca2
CurrTxInScript:    4730440220151ad44e7f78f9e0c4a3f2135c19ca3de8dbbb7c58893db096c0c5f1573d5dec02200724a78c3fa5f153103cb46816df46eb6cfac3718038607ddec344310066161e01410459fd82189b81772258a3fc723fdda900eb8193057d4a573ee5ad39e26b58b5c12c4a51b0edd01769f96ed1998221daf0df89634a7137a8fa312d5ccc95ed8925483045022100ca34834ece5925cff6c3d63e2bda6b0ce0685b18f481c32e70de9a971e85f12f0220572d0b5de0cf7b8d4e28f4914a955e301faaaa42f05feaa1cc63b45f938d75d9014104ce6242d72ee67e867e6f8ec434b95fcb1889c5b485ec3414df407e11194a7ce012eda021b68f1dd124598a9b677d6e7d7c95b1b7347f5c5a08efa628ef0204e1483045022074e01e8225e8c4f9d0b3f86908d42a61e611f406e13817d16240f94f52f49359022100f4c768dd89c6435afd3834ae2c882465ade92d7e1cc5c2c2c3d8d25c41b3ea61014104ce66c9f5068b715b62cc1622572cd98a08812d8ca01563045263c3e7af6b997e603e8e62041c4eb82dfd386a3412c34c334c34eb3c76fb0e37483fc72323f807
CurrRawTx:         01000000012f654d4d1d7246d1a824c5b6c5177c0b5a1983864579aabb88cabd5d05e032e201000000fda0014730440220151ad44e7f78f9e0c4a3f2135c19ca3de8dbbb7c58893db096c0c5f1573d5dec02200724a78c3fa5f153103cb46816df46eb6cfac3718038607ddec344310066161e01410459fd82189b81772258a3fc723fdda900eb8193057d4a573ee5ad39e26b58b5c12c4a51b0edd01769f96ed1998221daf0df89634a7137a8fa312d5ccc95ed8925483045022100ca34834ece5925cff6c3d63e2bda6b0ce0685b18f481c32e70de9a971e85f12f0220572d0b5de0cf7b8d4e28f4914a955e301faaaa42f05feaa1cc63b45f938d75d9014104ce6242d72ee67e867e6f8ec434b95fcb1889c5b485ec3414df407e11194a7ce012eda021b68f1dd124598a9b677d6e7d7c95b1b7347f5c5a08efa628ef0204e1483045022074e01e8225e8c4f9d0b3f86908d42a61e611f406e13817d16240f94f52f49359022100f4c768dd89c6435afd3834ae2c882465ade92d7e1cc5c2c2c3d8d25c41b3ea61014104ce66c9f5068b715b62cc1622572cd98a08812d8ca01563045263c3e7af6b997e603e8e62041c4eb82dfd386a3412c34c334c34eb3c76fb0e37483fc72323f807ffffffff01b0ad5407000000001976a9146a4e7d5f798e90e84db9244d4805459f8727594388ac00000000


Pretty TxIn Script: 
   [PUSHDATA -- 71 BYTES:]
   30440220151ad44e7f78f9e0c4a3f2135c19ca3de8dbbb7c58893db096c0c5f1573d5dec02200724a78c3fa5f153103cb46816df46eb6cfac3718038607ddec344310066161e01
   [PUSHDATA -- 65 BYTES:]
   0459fd82189b81772258a3fc723fdda900eb8193057d4a573ee5ad39e26b58b5c12c4a51b0edd01769f96ed1998221daf0df89634a7137a8fa312d5ccc95ed8925
   [PUSHDATA -- 72 BYTES:]
   3045022100ca34834ece5925cff6c3d63e2bda6b0ce0685b18f481c32e70de9a971e85f12f0220572d0b5de0cf7b8d4e28f4914a955e301faaaa42f05feaa1cc63b45f938d75d901
   [PUSHDATA -- 65 BYTES:]
   04ce6242d72ee67e867e6f8ec434b95fcb1889c5b485ec3414df407e11194a7ce012eda021b68f1dd124598a9b677d6e7d7c95b1b7347f5c5a08efa628ef0204e1
   [PUSHDATA -- 72 BYTES:]
   3045022074e01e8225e8c4f9d0b3f86908d42a61e611f406e13817d16240f94f52f49359022100f4c768dd89c6435afd3834ae2c882465ade92d7e1cc5c2c2c3d8d25c41b3ea6101
   [PUSHDATA -- 65 BYTES:]
   04ce66c9f5068b715b62cc1622572cd98a08812d8ca01563045263c3e7af6b997e603e8e62041c4eb82dfd386a3412c34c334c34eb3c76fb0e37483fc72323f807

Pretty TxOutScript:
   2
   OP_TOALTSTACK
   OP_0
   OP_TOALTSTACK
   OP_TUCK
   OP_CHECKSIG
   OP_SWAP
   OP_HASH160
   [PUSHDATA -- 20 BYTES:]
   3cd1def404e12a85ead2b4d3f5f9f817fb0d46ef
   OP_EQUAL
   OP_BOOLAND
   OP_FROMALTSTACK
   OP_ADD
   OP_TOALTSTACK
   OP_TUCK
   OP_CHECKSIG
   OP_SWAP
   OP_HASH160
   [PUSHDATA -- 20 BYTES:]
   6a4e7d5f798e90e84db9244d4805459f87275943
   OP_EQUAL
   OP_BOOLAND
   OP_FROMALTSTACK
   OP_ADD
   OP_TOALTSTACK
   OP_TUCK
   OP_CHECKSIG
   OP_SWAP
   OP_HASH160
   [PUSHDATA -- 20 BYTES:]
   486efdd300987a054510b4ce1148d4ad290d911e
   OP_EQUAL
   OP_BOOLAND
   OP_FROMALTSTACK
   OP_ADD
   OP_TOALTSTACK
   OP_FROMALTSTACK
   OP_FROMALTSTACK
   OP_GREATERTHANOREQUAL

2-of-3 Transaction with OP_CHECKMULTISIG

Code:

Testnet Tx Hash:      bb664ff716b9dfc831bcc666c1767f362ad467fcfbaf4961de92e45547daab87

PrevTxOutScript :     537a7652a269537a829178a91480677c5392220db736455533477d0bc2fba65502879b69537a829178a91402d7aa2e76d9066fb2b3c41ff8839a5c81bdca19879b69537a829178a91410039ce4fdb5d4ee56148fe3935b9bfbbe4ecc89879b6953ae
CurrTxInScript:       00493046022100d73f633f114e0e0b324d87d38d34f22966a03b072803afa99c9408201f6d6dc6022100900e85be52ad2278d24e7edbb7269367f5f2d6f1bd338d017ca460008776614401473044022071fef8ac0aa6318817dbd242bf51fb5b75be312aa31ecb44a0afe7b49fcf840302204c223179a383bb6fcb80312ac66e473345065f7d9136f9662d867acf96c12a42015241048c006ff0d2cfde86455086af5a25b88c2b81858aab67f6a3132c885a2cb9ec38e700576fd46c7d72d7d22555eee3a14e2876c643cd70b1b0a77fbf46e62331ac4104b68ef7d8f24d45e1771101e269c0aacf8d3ed7ebe12b65521712bba768ef53e1e84fff3afbee360acea0d1f461c013557f71d426ac17a293c5eebf06e468253e00
CurrRawTx:            0100000001bb664ff716b9dfc831bcc666c1767f362ad467fcfbaf4961de92e45547daab8701000000fd190100493046022100d73f633f114e0e0b324d87d38d34f22966a03b072803afa99c9408201f6d6dc6022100900e85be52ad2278d24e7edbb7269367f5f2d6f1bd338d017ca460008776614401473044022071fef8ac0aa6318817dbd242bf51fb5b75be312aa31ecb44a0afe7b49fcf840302204c223179a383bb6fcb80312ac66e473345065f7d9136f9662d867acf96c12a42015241048c006ff0d2cfde86455086af5a25b88c2b81858aab67f6a3132c885a2cb9ec38e700576fd46c7d72d7d22555eee3a14e2876c643cd70b1b0a77fbf46e62331ac4104b68ef7d8f24d45e1771101e269c0aacf8d3ed7ebe12b65521712bba768ef53e1e84fff3afbee360acea0d1f461c013557f71d426ac17a293c5eebf06e468253e00ffffffff0280969800000000001976a9140817482d2e97e4be877efe59f4bae108564549f188ac7015a7000000000062537a7652a269537a829178a91480677c5392220db736455533477d0bc2fba65502879b69537a829178a91402d7aa2e76d9066fb2b3c41ff8839a5c81bdca19879b69537a829178a91410039ce4fdb5d4ee56148fe3935b9bfbbe4ecc89879b6953ae00000000
Pretty TxIn Script: 
   OP_0
   [PUSHDATA -- 73 BYTES:]
   3046022100d73f633f114e0e0b324d87d38d34f22966a03b072803afa99c9408201f6d6dc6022100900e85be52ad2278d24e7edbb7269367f5f2d6f1bd338d017ca460008776614401
   [PUSHDATA -- 71 BYTES:]
   3044022071fef8ac0aa6318817dbd242bf51fb5b75be312aa31ecb44a0afe7b49fcf840302204c223179a383bb6fcb80312ac66e473345065f7d9136f9662d867acf96c12a4201
   2
   [PUSHDATA -- 65 BYTES:]
   048c006ff0d2cfde86455086af5a25b88c2b81858aab67f6a3132c885a2cb9ec38e700576fd46c7d72d7d22555eee3a14e2876c643cd70b1b0a77fbf46e62331ac
   [PUSHDATA -- 65 BYTES:]
   04b68ef7d8f24d45e1771101e269c0aacf8d3ed7ebe12b65521712bba768ef53e1e84fff3afbee360acea0d1f461c013557f71d426ac17a293c5eebf06e468253e
   OP_0
Pretty TxOut Script: 
   3
   OP_ROLL
   OP_DUP
   2
   OP_GREATERTHANOREQUAL
   OP_VERIFY
   3
   OP_ROLL
   OP_SIZE
   OP_NOT
   OP_OVER
   OP_HASH160
   [PUSHDATA -- 20 BYTES:]
   80677c5392220db736455533477d0bc2fba65502
   OP_EQUAL
   OP_BOOLOR
   OP_VERIFY
   3
   OP_ROLL
   OP_SIZE
   OP_NOT
   OP_OVER
   OP_HASH160
   [PUSHDATA -- 20 BYTES:]
   02d7aa2e76d9066fb2b3c41ff8839a5c81bdca19
   OP_EQUAL
   OP_BOOLOR
   OP_VERIFY
   3
   OP_ROLL
   OP_SIZE
   OP_NOT
   OP_OVER
   OP_HASH160
   [PUSHDATA -- 20 BYTES:]
   10039ce4fdb5d4ee56148fe3935b9bfbbe4ecc89
   OP_EQUAL
   OP_BOOLOR
   OP_VERIFY
   3
   OP_CHECKMULTISIG

Note that all of these scripts are non-standard.  They are actually designed to allow for 20-byte public keys hashes to be used as input instead of 65-byte public keys.  As such, they are very complicated because the script has to verify that the public keys match 2 of the hashes in the TxOut script.

Also, be careful with you Hash160 calls.  As I learned from testing my own scripting engine, OP_0 is interpretted as []  (empty string), which means it can be hashed and has zero length.  Those are both important for passing those script tests.