Technical details on claim script/signatures - Help Please

[Realpra]

I have poured over the available information I could find such as:

https://en.bitcoin.it/wiki/File:Bitcoin_OpCheckSig_InDetail.png
https://en.bitcoin.it/wiki/Protocol_specification#tx
https://en.bitcoin.it/wiki/Protocol_specification#Variable_length_integer
https://en.bitcoin.it/wiki/Protocol_specification#Signatures
https://en.bitcoin.it/wiki/Transactions#Pay-to-PubkeyHash

Aswell as mathematical descriptions of how ESDSA works:
http://kakaroto.homelinux.net/2012/01/how-the-ecdsa-algorithm-works/

However I have some questions I hope someone knows a resource for or the answer to:
1. In the linked picture what is "script part 4"?
2. Is it correct that script part 1 is R and S concatenated and compressed with DER encoding?
3. Is script part 2 the pub key?
4. If yes, is the  pubkey also DER encoded?
5. Where is the implied script part 3?
6. Can OP_CODESEPARATOR be left out entirely without invalidating the script?
7. Basically what is the byte-wise format of the claim script, this is my own educated guess:
START: DER(RS){approx. 70 bytes} VARINT_PUBKEY_LENGTH pubkeybytes{32 bytes, x coordinate of Qa} :END

(The out script would not be repeated when claiming, but is seen here:
OP_DUP OP_HASH160 VARINT_PUSH_BYTES PUSHEDBYTES{hash160 of Qa/address} OP_EQUALVERIFY OP_CHECKSIG )


A good exact and in-depth explanation of DER encoding would also be nice.

[mriou]

I hear you, it takes some effort to get op_checksig right. Here are some answers:

> 1. In the linked picture what is "script part 4"?

Whatever comes after OP_CHECKSIG in the script. Nothing says OP_CHECKSIG has to be the last operation of the script.

> 2. Is it correct that script part 1 is R and S concatenated and compressed with DER encoding?

No, in that picture, script part 1 and 2 are previous instructions in the script. Nothing says the OP_CHECKSIG sequence of instructions has to come first in the script, any arbitrary processing supporting by the scripting language can happen before.

> 3. Is script part 2 the pub key?

No, per step 1, the pubkey and signature are on the stack. What you see in step 2 is the script. Running the script affects the stack but they're still distinct structures.

> 4. If yes, is the  pubkey also DER encoded?

There are more than one possible encoding. Read this too:

https://en.bitcoin.it/wiki/Technical_background_of_Bitcoin_addresses

> 5. Where is the implied script part 3?

It's the actual OP_CHECKSIG standard sequence, from OP_DUP to OP_CHECKSIG itself.

> 6. Can OP_CODESEPARATOR be left out entirely without invalidating the script?

Yes, it's optional. The separator is only needed for some scripts that may have more than one OP_CHECKSIG operation in them for example.

> 7. Basically what is the byte-wise format of the claim script, this is my own educated guess:

It's a little more complicated. I'd recommend looking at the source code of bitcoind and other implementations out there, it helps.

[etotheipi]

You're getting confused by the generality of that diagram.  It is showing how to do it for an ultra-complex script, but the vast majority will be much simpler, and no OP_CODESEPARATORs. 


For every standard script, and probably most of the useful scripts, the output of Step4 is just the script being spent.  Whatever is the raw script of the TxOut being spent by this TxIn, is what's going to be inserted into that TxIn in Step8. 

In fact, there was some calls to remove OP_CODESEPARATOR, and it seems there's not really any reason to use it that can't be done without it.

Therefore, just assume that Step8 is always just copying the raw script being spent, into the TxIn before signing.

[Realpra]

Thank you to all of you for helping me out and special thanks to Etotheipi for making some of the documentation in the first place.
I found that the BouncyCastle library should be able to do DER encoding.

I'm still not entirely sure how the claim script is structured and encoded though.