cbitcoin - Bitcoin implementation in C. Currently in development.

[Gavin Andresen]

I'm not familiar enough with Satoshi client internals to easily understand what script_tests is doing.  Could you confirm my understanding of the format of the two JSON files?

The format is a list of pairs, where the first item is the scriptSig required to spend and the second is the scriptPubKey. The third item is ignored (useful for comments).

script_valid.json contains only valid scriptSig/scriptPubKey pairs.
script_invalid.json contains only invalid scriptSig/scriptPubKey pairs.

valid/invalid are defined by the rules of transaction validation, and the unit test actually constructs transactions and runs the verification routine on them to make sure that they succeed or fail, as expected.

[1714812251]

1714812251

[1714812251]

1714812251

[1714812251]

1714812251

[realnowhereman]

Thanks.

I note that the difficult ones, CHECKSIG, etc, aren't included.  I guess that's because there is no transaction for the script to operate on?

Is there a plan to include those difficult ones in the unit tests later?  Or will they be part of the transaction unit tests?

I ask only so I know whether to consider creating a fake transaction for the test scripts to run inside?

[MatthewLM]

The transaction tests are here: https://github.com/bitcoin/bitcoin/blob/master/src/test/transaction_tests.cpp

Hopefully I'll get around to starting transaction validation next week, though I'll be busy over the next few days.

If anyone would like to contribute, improving the test code would certainly be helpful.

[MatthewLM]

I just had a brain wave. There is no need to do copying of data for serialisation and de-serialisation of bitcoin messages like bitcoinj does. I can store the serialised byte data and just access that through pointers. Should be more efficient.

It's problematic trying to use a Java library as a basis for a C library. :-)

I need to store integers separately in the systems endianness.

[MatthewLM]

Now I've implemented transaction serialisation and deserialisation. Here is the test code for that: https://github.com/MatthewLM/cbitcoin/blob/master/cbitcoin/test/testCBTransaction.c

Next I'll move onto transaction signing and validation. After that I can work on the blocks, block chain, block validation and then the networking.

Once again, if anyone wishes to aid in anyway the best way is to scrutinise the code for bugs and problems and improve the tests.

[grondilu]

(registering to this thread)

[MatthewLM]

I'm assuming with SIGHASH_SINGLE the output scripts are blank but the VarInt is still there, so each output up to before the input index will have 9 bytes, 8 for the value and 1 for the VarInt. Can anyone verify that? Or is it just the value with 8 bytes? Trying to track it down in the C++ client was taking too long.

Also how are output value signs represented as serialised data? Similar to the script, except as little endian? Twos compliment? The serialisation code in the C++ client is just too over my head for me to understand.

Edit: Apparently the C++ client is platform dependent on the x64 architecture meaning the signed integers should be twos compliment and little endian.

[grondilu]

I'm assuming with SIGHASH_SINGLE the output scripts are blank but the VarInt is still there, so each output up to before the input index will have 9 bytes, 8 for the value and 1 for the VarInt. Can anyone verify that? Or is it just the value with 8 bytes? Trying to track it down in the C++ client was taking too long.

Also how are output value signs represented as serialised data? Similar to the script, except as little endian? Twos compliment? The serialisation code in the C++ client is just too over my head for me to understand.

I'm not sure I understand exactly your problem, but maybe it could help if I tell you that I had lots of difficulty trying to understand the bitcoin serialisation protocol, until I read Gavin's bitcointools python code.  It is much easier to understand than C++.  It helped me a lot for my perl library.

[MatthewLM]

Unfortunately from what I see it doesn't help with my SIGHASH issue. I made a question here: http://bitcoin.stackexchange.com/questions/3890/for-sighash-single-do-the-outputs-other-than-at-the-input-index-have-8-bytes-or

[fellowtraveler]

If I had to write something in straight-C, I would use the D language.

[MatthewLM]

Except D isn't C. 

OpenSSL is terribly undocumented. I'm currently looking at checking transaction ECDSA signatures using:

bool (*ecdsaVerify)(u_int8_t *,u_int8_t *,const u_int8_t *); /**< Function for verifying ECDSA signatures for a 70 byte signature against a 32 byte hash using a 66 byte ECDSA public key. */

The first parameter is the 70 byte signature, the second is the 32 byte SHA256 digest and the last is the public key data. For testing I need to implement this function. I'm using OpenSSL for the tests but it doesn't document converting public key bytes to an EC_KEY object. Not good at all! But I looked at the bitcoin source code and I saw o2i_ECPublicKey so I'll use that...

bool ecdsaVerify(u_int8_t * signature,u_int8_t * hash,const u_int8_t * pubKey){
   EC_KEY * key = EC_KEY_new_by_curve_name(NID_secp256k1);
   o2i_ECPublicKey(&key, &pubKey, 66);
   ECDSA_verify(0, hash, 32, signature, 70, key);
}

Hopefully this is the right way. Am I right thinking that the signatures are 70 bytes and the public keys are 66 bytes?

[Gavin Andresen]

Hopefully this is the right way. Am I right thinking that the signatures are 70 bytes and the public keys are 66 bytes?

Signatures are BER-encoded data structures, and can be an arbitrary number of bytes (if they're DER-encoded, which is the strict subset of BER encoding, then they're 70-something bytes).

Public keys are either 33 or 65 bytes (not counting the "push the next N bytes onto the stack" CSCript opcode).

I've got to say you make me nervous; you seem to be following a "make it work for a couple of test cases then move on" style of development, which is a bad way to create a secure, robust codebase.

PS: I sympathize with you RE: OpenSSL's lack of documentation....

[kokjo]

If I had to write something in straight-C, I would use the D language.

if i had to use X, i would use something different and unrelated Y.

WTF? the reason for using C is that its simple, small and portable. which i suspect that D is not.

[MatthewLM]

@Gavin Andresen

Thanks for the information. Don't get worried about the tests. I'm fully aware they are incomplete. The tests can always be fully implemented later on. I welcome anyone that is interested to contribute to the testing code.

So I'm following "make it work for a couple of test cases then move on and come back later to fully implement the tests".

I'll also need to come back later and complete the documentation. I'm documenting to some degree as I go along but there are parts that I'll be better able to document later.

[MatthewLM]

New problem: http://stackoverflow.com/questions/10906524/openssl-i2o-ecpublickey-not-providing-any-bytes

Any ideas?

[fellowtraveler]

If I had to write something in straight-C, I would use the D language.

if i had to use X, i would use something different and unrelated Y.

WTF? the reason for using C is that its simple, small and portable. which i suspect that D is not.

Anyone who writes C programs would do well to research the D language.

Ditto for C++, but for entirely different reasons.

D is the perfect replacement for both (each for different reasons.)

I have programmed C and C++ for over 20 years, everything from controller cards to Windows, to Linux, to Mac, to Android, and FreeBSD. (Do your own research, just giving my opinion. Take it for whatever it's worth.)

-FT

[kokjo]

If I had to write something in straight-C, I would use the D language.

if i had to use X, i would use something different and unrelated Y.

WTF? the reason for using C is that its simple, small and portable. which i suspect that D is not.

Anyone who writes C programs would do well to research the D language.

Ditto for C++, but for entirely different reasons.

D is the perfect replacement for both (each for different reasons.)

I have programmed C and C++ for over 20 years, everything from controller cards to Windows, to Linux, to Mac, to Android, and FreeBSD. (Do your own research, just giving my opinion. Take it for whatever it's worth.)

-FT

Language fanboy detected.

hey, if i had to write something in C, i would use python.

[fellowtraveler]

Language fanboy detected.

hey, if i had to write something in C, i would use python.

Fair enough. However, Python was not specifically written as a replacement for C and C++.

D *was* specifically written as a replacement for C and C++.

Therefore, I think you are comparing apples and oranges.

For example, D, like C, would be an appropriate tool for writing a system-level driver.  Python, on the other hand, most likely would NOT be an appropriate tool for such a task.

(Apples and oranges...)

[MatthewLM]

Does anyone know how to install OpenSSL 1.0.1c on OSX? After changing to the unarchived directory I ran:

Code:

./config
make
make test
sudo make install

But I still have the old shipped version.

I had a feeling updating might fix my i2o_ECPublicKey problem.

[MatthewLM]

The OpenSSL problems are solved. I've worked on a script compiler (text to byte code) and this is the syntax:

Code:

<digit> = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9" | "a" | "b" | "c" | "d" | "e" | "f" | "A" | "B" | "C" | "D" | "E" | "F"
 <hex_digits> ::= <digit> | <digit> <hex_digits>
 <hex> ::= "0x" <hex_digits>
 <operation_name> = "0" | "FALSE" | "1NEGATE" | "RESERVED" | "1" | "TRUE" | "2" | "3" | "4" |  "5" | "6" | "7" | "8" | "9" | "10" | "11" | "12" | "13" | "14" | "15" | "16" | "NOP" | "VER" | "IF" | "NOTIF" | "VERIF" | "VERNOTIF" | "ELSE" | "ENDIF" | "VERIFY" | "RETURN" | "TOALTSTACK" | "FROMALTSTACK" | "2DROP" | "2DUP" | "3DUP" | "2OVER" | "2ROT" | "2SWAP" | "IFDUP" | "DEPTH" | "DROP" | "DUP" | "NIP" | "OVER" | "PICK" | "ROLL" | "ROT" | "SWAP" | "TUCK" | "CAT" | "SUBSTR" | "LEFT" | "RIGHT = 0x81" | "SIZE" | "INVERT" | "AND" | "OR" | "XOR" | "EQUAL" | "EQUALVERIFY" | "RESERVED1" | "RESERVED2" | "1ADD" | "1SUB" | "2MUL" | "2DIV" | "NEGATE" | "ABS" | "NOT" | "0NOTEQUAL" | "ADD" | "SUB" | "MUL" | "DIV" | "MOD" | "LSHIFT " | "RSHIFT" | "BOOLAND" | "BOOLOR" | "NUMEQUAL" | "NUMEQUALVERIFY" | "NUMNOTEQUAL" | "LESSTHAN" | "GREATERTHAN" | "LESSTHANOREQUAL" | "GREATERTHANOREQUAL" | "MIN" | "MAX" | "WITHIN" | "RIPEMD160" | "SHA1" | "SHA256" | "HASH160" | "HASH256" | "CODESEPARATOR" | "CHECKSIG" | "CHECKSIGVERIFY" | "CHECKMULTISIG" | "CHECKMULTISIGVERIFY" | "NOP1" | "NOP2" | "NOP3" | "NOP4" | "NOP5" | "NOP6" | "NOP7" | "NOP8" | "NOP9" | "NOP10"
 <operation> ::= "OP_" <operation_name>
 <lines> ::= "\n" | "\n" <lines>
 <seperator> ::= <lines> | " "
 <operation_or_hex> ::= <operation> | <hex>
 <seperated_operation_or_hex> ::= <operation_or_hex> <seperator>
 <script_body> ::= <seperated_operation_or_hex> | <seperated_operation_or_hex> <script_body> | ""
 <lines_or_none> ::= <lines> | ""
 <operation_or_hex_or_none> ::= <operation_or_hex> | ""
 <script> ::= <lines_or_none> <script_body> <operation_or_hex_or_none> 

What do people think? The compiler automatically converts hex into push operations.