creating a raw unsigned tx, Updated with an example

[Coding Enthusiast]

Update 2016-11-29:
Since this example helped me I decided to add it here for anyone else who may be interested in the future.
Scroll Down to new questions
following this: http://bitcoin.stackexchange.com/a/32695
TL;DR: what is "76 a9 14" and "88 ac" which is being added to the hash160 and why is it different when using other wallets to make the raw tx.

i have gone as far as creating a raw unsigned tx myself:

Code:

01000000
01
902dbe4ae9a20b85f53cb6813396e6aae272ee4e3b699e8c2498d2ce8f5833ee
00000000
19
76a914  fdf8bdeae5fe5fb35328dc04a95b17b8ed70bed1  88ac
ffffffff
01
c027090000000000
19
76a91437ba8314fcd8bbbf49ed9a1d6d27f9797e7d60f188ac
00000000

i also get the same thing with https://coinb.in/#newTransaction

but with Electrum (since that is the only wallet i have) i get this:

Code:

01000000
01
902dbe4ae9a20b85f53cb6813396e6aae272ee4e3b699e8c2498d2ce8f5833ee
00000000
19
01ff16fd00  fdf8bdeae5fe5fb35328dc04a95b17b8ed70bed1
ffffffff
01
c027090000000000
19
76a91437ba8314fcd8bbbf49ed9a1d6d27f9797e7d60f188ac
00000000

so as you can see first method is using 76a914 {hash160} 88ac format
whole Electrum is using 01ff16fd00 {hash160} format

- so why is it different? and how can both of them be valid?

[1713473723]

1713473723

[achow101]

The 76a914...88ac is the standard form for a pay-to-pubkey-hash output script. When you sign a transaction, the output script goes into the scriptsig, hence that format there. You will notice it down below by the outputs.

As for what Electrum is doing, I'm not sure, It doesn't seem like it should work, my best guess is that it's something internal to help it figure things out. I suggest taking a look through the source code (https://github.com/spesmilo/electrum) to see what is happening.

[Coding Enthusiast]

I simply used Electrum to double check and see if the TX that i generate myself is right, since i don't have any other wallet.

I am trying to learn the code and next step is Bitcoin the hard way + developers guide for me to learn more about signing that raw tx.

[DannyHamilton]

Look closer, in all three cases (Electrum, coinb.in, and your raw UNSIGNED transaction) the scriptPubKey is exactly the same.

What you are describing in your post is not the scriptPubKey, it's the scriptSig.  Since your transaction is UNSIGNED, it doesn't yet have a scriptSig.  Therefore you, coinb.in, and Electrum appear to have put some placeholder garbage in there.  Eventually, when the transaction is signed, this placeholder data will be replaced with the actual scriptSig.

[Coding Enthusiast]

OK, i got some more time to work on this.

Look closer, in all three cases (Electrum, coinb.in, and your raw UNSIGNED transaction) the scriptPubKey is exactly the same.

What you are describing in your post is not the scriptPubKey, it's the scriptSig.  Since your transaction is UNSIGNED, it doesn't yet have a scriptSig.  Therefore you, coinb.in, and Electrum appear to have put some placeholder garbage in there.  Eventually, when the transaction is signed, this placeholder data will be replaced with the actual scriptSig.

i understand it now but i am still confused about how can this have meaning and still be different:
yeah i understood that it is a temporary placeholder to be filled with scriptSig later.
but what confuses me is how can it be garbage when it has a meaning and has to be parsed when passed into the code to validate and then sign.
reading the Script page:

Code:

OP_DUP OP_HASH160 <don't know what 0x14 is, couldn't find> ... OP_EQUALVERIFY OP_CHECKSIG

2 more questions:

• it is said that the In-counter and Out-counter are "Variable length integers" can anyone give me an example of an In-counter with "3", "5" and "9" byte so i know how it is going to look like. also is This how you get the size then value of the VarInt
• are all the numbers (amount, lengths, lock time,...) Little-endian?

[DannyHamilton]

i am still confused about how can this have meaning and still be different:
yeah i understood that it is a temporary placeholder to be filled with scriptSig later.
but what confuses me is how can it be garbage when it has a meaning and has to be parsed when passed into the code to validate and then sign.

A transaction signature is computed over the entire transaction.  However, since the scriptSig is part of the transaction, and the scriptSig is in the process of being computed, it is impossible to have the scriptSig properly populated with the signature (which you haven't computed yet) while you are computing the signature.  Therefore, as part of the process of signing the transaction, the scriptSig values are all set to 0. When later validating a signature, the scriptSig values must again be set to 0 during the validation.  This way a consistent result can be expected.

It is rather useless to populate the scriptSig with large placeholder values, and I don't know why Electrum or coinb.in would be doing that. When I use Bitcoin Core to generate the unsigned raw transaction, the entire scriptSig placeholder value is just 00.

Code:

createrawtransaction [{\"txid\":\"ee33588fced298248c9e693b4eee72e2aae6963381b63cf5850ba2e94abe2d90\",\"vout\":0}] {\"165fbdntWGih7i9mfs9v5ZHgGyNxhHS4Wb\":0.00600000}

0100000001902dbe4ae9a20b85f53cb6813396e6aae272ee4e3b699e8c2498d2ce8f5833ee0000000000ffffffff01c0270900000000001976a91437ba8314fcd8bbbf49ed9a1d6d27f9797e7d60f188ac00000000

reading the Script page:

Code:

OP_DUP OP_HASH160 <don't know what 0x14 is, couldn't find> ... OP_EQUALVERIFY OP_CHECKSIG

Any OP code value from 0x01 through 0x4b is just the number of bytes to push onto the stack.  So a code of 0x14 indicates that the next 0x14 bytes are to be pushed onto the stack.  In the scriptPubKey, those 20 bytes are the RIPEMD160 hash value.

it is said that the In-counter and Out-counter are "Variable length integers" can anyone give me an example of an In-counter with "3", "5" and "9" byte so i know how it is going to look like.

EDIT: It appears I didn't understand this next part as well as I thought I did.  See comments from achow101 below

You'll never see an in-counter or out-counter larger than 3 bytes. That would require the transaction to have more than 2097151 inputs or outputs.

Here is the 2 byte bigendian hexadecimal representation of the decimal value 258 in a varInt:

Code:

- REDACTED -

0x8102

Here is the 3 byte bigendian hexadecimal representation of the decimal value 65535 in a varInt:

Code:

- REDACTED -

0x82FE7F

Here's the 4 byte bigendian hexadecimal representation of the decimal value 4294967296 in a varInt:

Code:

- REDACTED -

0x8EFEFEFF00

also is This how you get the size then value of the VarInt[/li][/list]

No.

You get the value like this:
https://github.com/bitcoin/bitcoin/blob/master/src/serialize.h#L361

You get the size like this:
https://github.com/bitcoin/bitcoin/blob/master/src/serialize.h#L331

• are all the numbers (amount, lengths, lock time,...) Little-endian?

I don't think VarInt values are ever Little-endian, but I'm not 100% certain.  Everything else seems to be.

[achow101]

A transaction signature is computed over the entire transaction.  However, since the scriptSig is part of the transaction, and the scriptSig is in the process of being computed, it is impossible to have the scriptSig properly populated with the signature (which you haven't computed yet) while you are computing the signature.  Therefore, as part of the process of signing the transaction, the scriptSig values are all set to 0. When later validating a signature, the scriptSig values must again be set to 0 during the validation.  This way a consistent result can be expected.

It is rather useless to populate the scriptSig with large placeholder values, and I don't know why Electrum or coinb.in would be doing that. When I use Bitcoin Core to generate the unsigned raw transaction, the entire scriptSig placeholder value is just 00.

When signing happens, the scriptsig is populated with the previous scriptpubkey. It appears that Electrum and coinb.in are doing that so that when signing happens, they know what to put in the scriptsig without having to find it in the blockchain.

I don't think VarInt values are ever Little-endian, but I'm not 100% certain.  Everything else seems to be.

Varint is described at https://bitcoin.org/en/developer-reference#compactsize-unsigned-integers

[DannyHamilton]

- snip -
It is rather useless to populate the scriptSig with large placeholder values, and I don't know why Electrum or coinb.in would be doing that. - snip -

- snip -
It appears that Electrum and coinb.in are doing that so that when signing happens, they know what to put in the scriptsig without having to find it in the blockchain.

Ok, I guess that makes sense.

When calculating the scriptSig, you'd need to know which private key to sign with. This can be determined by looking in the blockchain at the output that is being spent, but if you store this information in the scriptSig space of the unsigned transaction, then you can save yourself the time of needing to go back and look it up again.

I don't think VarInt values are ever Little-endian, but I'm not 100% certain.  Everything else seems to be.

Varint is described at https://bitcoin.org/en/developer-reference#compactsize-unsigned-integers

Wow.  Here I thought I already understood the digital representation of a transaction pretty well, and yet for the past 4.5 years I've completely misunderstood how the variable-length integers are represented.  I guess I've just never had to deal with (or look closely at) any variable-length integers with a value larger than 252, so I never noticed that I was mistaken.

Thanks.

So, if compactSize unsigned integers are used in the transaction for the quantity of inputs and outputs, then where does Bitcoin use VarInt, and/or why is it in serialize.h?

[achow101]

So, if compactSize unsigned integers are used in the transaction for the quantity of inputs and outputs, then where does Bitcoin use VarInt, and/or why is it in serialize.h?

Compactsize unsigned integers are typically referred to as varint.

But in Bitcoin Core, the VarInt is used for something else internally. It has to do with the CVarint class which is used for some internal representation, IIRC it has something to do with on disk serialization e.g. blk*.dat files.

[Coding Enthusiast]

thanks a lot for the help

- snip -
It is rather useless to populate the scriptSig with large placeholder values, and I don't know why Electrum or coinb.in would be doing that. - snip -

- snip -
It appears that Electrum and coinb.in are doing that so that when signing happens, they know what to put in the scriptsig without having to find it in the blockchain.

Ok, I guess that makes sense.

When calculating the scriptSig, you'd need to know which private key to sign with. This can be determined by looking in the blockchain at the output that is being spent, but if you store this information in the scriptSig space of the unsigned transaction, then you can save yourself the time of needing to go back and look it up again.

i made a little code to serialize the raw txs to understand it better especially this demo on wiki and the rawtx you made with core and now i see exactly what you mean. they are filled with zero.
so i suppose this is why you spoon-feed "scriptPubKey":hex bitcoin core offline when you want to signrawtransaction

♯♯Varint is described at https://bitcoin.org/en/developer-reference#compactsize-unsigned-integers

i was looking for the example 515=0xfd0302 which was not available on wiki for some reason!
thanks, i guess i have to start reading both from now on

i have spend a couple of days on understanding only a raw "unsigned" transaction and make a little code for it and i have not yet started on the signing part and that is the scariest, not because of losing money (i am just testing things here) but because it is so complicated and also since it is cryptographic, it is very unforgiving!
but it is at least good kind of scary+enjoyable

[DannyHamilton]

- snip -
i have not yet started on the signing part and that is the scariest, not because of losing money
- snip -

If you are working on testnet, then you can try out your transactions without worrying about losing money.

[Coding Enthusiast]

If you are working on testnet, then you can try out your transactions without worrying about losing money.

ok thanks for the suggestion. i made myself an address and claimed faucet, ♯readyToMoveForward

[pebwindkraft]

>> i have spend a couple of days on understanding only a raw "unsigned" transaction
>> and make a little code for it and i have not yet started on the signing part and that
>> is the scariest, not because of losing money (i am just testing things here) but
>> because it is so complicated and also since it is cryptographic, it is very unforgiving!
>> but it is at least good kind of scary+enjoyable

no need to fear 
Draw a picture, and then go through the details, what is required. A single input signing process is not that difficult. You create your unsigned raw trx, and leave the PKScript (76A914 + previous trx script + 88AC) in its position. You sign the whole structure. And then you replace this PKScript with the generated signature, which is then just a string concatenation. Pay attention to the length fields, they almost always change...

Where it get's a bit more advanced is with two or more inputs, cause all signatures have to be done specifically, see here:
http://bitcoin.stackexchange.com/questions/41209/how-to-sign-a-transaction-with-multiple-inputs

good luck!
(btw: which coding language are you using?)

[Coding Enthusiast]

♯
http://bitcoin.stackexchange.com/questions/41209/how-to-sign-a-transaction-with-multiple-inputs
good luck!
(btw: which coding language are you using?)

hey thanks for the link, i just have to get some free time to start on the signing part. may do it this weekend.

and i use C♯ (i am a newbie though)

[Coding Enthusiast]

I am updating this with an example transaction which contains those CompactSize Unsigned Integers that we were talking about for someone who may be interested in a real tx example.

https://blockchain.info/tx/ef492f5dd516f51061a77bc148b8925e74557b1f31a42ca51098f26b82c23fb0?format=hex
This tx that I have found has 290 inputs (tx_in count) which is shows with 0xfd2201