How to sign raw transaction with openssl?

[kzv]

Hello

I know that transactions with non standard script is can not be signed with bitcoin-core. So I am looking for the way to sign transaction with other methods that not required any programming. For example openssl.

Look at this standard, but not signed transaction
02000000018d5379c5460adf4a83e75b8c14c9832268aa4539b3ce3f102dd59f4fc154b0ea00000 00021029501ef2a97b855ab35898a6f5901c6538bab08d8244cd88b213e777095a17c6ffeffffff 01105522000000000017a914e4b3d47cc59f2c895a4c09107c98f8acc476c9688700000000

when I signed it with

Code:

signrawtransactionwithwallet 02000000018d5379c5460adf4a83e75b8c14c9832268aa4539b3ce3f102dd59f4fc154b0ea0000000021029501ef2a97b855ab35898a6f5901c6538bab08d8244cd88b213e777095a17c6ffeffffff01105522000000000017a914e4b3d47cc59f2c895a4c09107c98f8acc476c9688700000000 

I got this output

020000000001018d5379c5460adf4a83e75b8c14c9832268aa4539b3ce3f102dd59f4fc154b0ea0 0000000171600142fcfe0ee792965d852f392c5fe53e2be71274ca8feffffff0110552200000000 0017a914e4b3d47cc59f2c895a4c09107c98f8acc476c968870247304402206a59b9212efdacc2e6367c4a45b4e26b3a73a64d564c983cb82edcd2ae545647022 04a5ec3e3e729009cfc855b0c23bf2468e423344993847a37394d3810a1c7f1bb0121029501ef2a 97b855ab35898a6f5901c6538bab08d8244cd88b213e777095a17c6f00000000

So I guess that this is a signature part

0247304402206a59b9212efdacc2e6367c4a45b4e26b3a73a64d564c983cb82edcd2ae54564702204a5 ec3e3e729009cfc855b0c23bf2468e423344993847a37394d3810a1c7f1bb0121029501ef2a97b855ab35898a6f5901c6538bab08d8244cd88b213e777095a17c6f

Where this is my public key
029501ef2a97b855ab35898a6f5901c6538bab08d8244cd88b213e777095a17c6f

And this is my signature
304402206a59b9212efdacc2e6367c4a45b4e26b3a73a64d564c983cb82edcd2ae54564702204a5 ec3e3e729009cfc855b0c23bf2468e423344993847a37394d3810a1c7f1bb01


I do not known: what is 02
And got main question: can I obtain this part with openssl or other well-known utilities?

[khaled0111]

020000000001018d5379c5460adf4a83e75b8c14c9832268aa4539b3ce3f102dd59f4fc154b0ea0 0000000171600142fcfe0ee792965d852f392c5fe53e2be71274ca8feffffff0110552200000000 0017a914e4b3d47cc59f2c895a4c09107c98f8acc476c968870247304402206a59b9212efdacc2e6367c4a45b4e26b3a73a64d564c983cb82edcd2ae545647022 04a5ec3e3e729009cfc855b0c23bf2468e423344993847a37394d3810a1c7f1bb0121029501ef2a 97b855ab35898a6f5901c6538bab08d8244cd88b213e777095a17c6f00000000

I am not 100% certain but I think what is in bold is the witness data and 02 is the witness count.

02 <- witness count 47 <-witness length 71 bytes  304402206a59b9212efdacc2e6367c4a45b4e26b3a73a64d564c983cb82edcd2ae54564702204a5 ec3e3e729009cfc855b0c23bf2468e423344993847a37394d3810a1c7f1bb01 <- witness data 21 <- witness length 33 bytes 029501ef2a97b855ab35898a6f5901c6538bab08d8244cd88b213e777095a17c <- witness data

[BitMaxz]

Honestly, I don't know how to read the hash but since you are looking for ways to sign raw transaction I tried to research in Google and found this post here and according to the post you need to convert the private key into pem.key to be able to sign the raw transaction through openSSL.

Just added this below to make sure you won't miss the link included from the link above.

Check this it is connected to the link above Verifying a bitcoin trx on the Unix cmd line with OpenSSL

You can see some command there that you can use for signing raw transactions.

About the 02 mean I think it is transaction output count according to this "Transaction Data Decode Raw Format"

[pooya87]

So I am looking for the way to sign transaction with other methods that not required any programming. For example openssl.

it actually requires a lot of programming since you have to first create the message digest for hashing on your own using 2x SHA256 hash(es)* and pass that to OpenSSL for signing and then you have to account for a couple of consensus related rules such as having low s, strict DER encoding,... and a bunch more which OpenSSL doesn't care about. then you also have to insert this inside your transaction's appropriate field with the correct encoding, length indicators,...

* for legacy transactions it will be one doubleSha256 hash of a special serialized tx. but for SegWit transactions it will be multiple doubleSha256 hash of different serialization of different parts explained in BIP143 then a final hash of the concatenated results.

So I guess that this is a signature part

you are correct.
since your transaction is a P2SH-P2WPKH after you sign it, you have to first put a special script inside the script sig (00142fcfe0ee792965d852f392c5fe53e2be71274ca8) then you have to place the signature + pubkey inside the witness part of the transaction.
inside witness, you insert "items" so instead of reporting the length of the script like you do in a scriptSig you instead report the number of items inside of the witness. since you have 2 items (signature and the public key) you start by 0x02 followed by signature followed by public key each having their length encoded using Compact Int (aka varint).

[kzv]

Thank you for explain. This is really too hard without programming ((
I have oppened the issue with feature request https://github.com/bitcoin/bitcoin/issues/17400

[retweeting]

And got main question: can I obtain this part with openssl or other well-known utilities?

You might find this thread to be relevant to your query - https://bitcointalk.org/index.php?topic=5103419.0

[pooya87]

Thank you for explain. This is really too hard without programming ((
I have oppened the issue with feature request https://github.com/bitcoin/bitcoin/issues/17400

since a couple of days has passed and there has been no answer to this, here is my 2 cents.
i don't think what you want here is even possible. and the problem is not with "not knowing where to place signature" it is about the step before that, meaning not knowing how to sign.
when you give core (or anything else) your transaction with scripts to sign it looks at those scripts and then decides how to sign it. so it has to have those script types defined first which is why it can not sign anything non-standard.
for example for "OP_DROP OP_CHECKSIGVERIFY" it has to first have a pre-defined code to know it has to ignore the "drop" and sign using the key, then add the arbitrary data to be dropped. the code can't guess it on its own.

[kzv]

since a couple of days has passed and there has been no answer to this, here is my 2 cents.
i don't think what you want here is even possible. and the problem is not with "not knowing where to place signature" it is about the step before that, meaning not knowing how to sign.
when you give core (or anything else) your transaction with scripts to sign it looks at those scripts and then decides how to sign it. so it has to have those script types defined first which is why it can not sign anything non-standard.
for example for "OP_DROP OP_CHECKSIGVERIFY" it has to first have a pre-defined code to know it has to ignore the "drop" and sign using the key, then add the arbitrary data to be dropped. the code can't guess it on its own.

I think that signing process is very standard:
1. For all inputs
2. Read scriptPubKey
3. Hash scriptPubKey
4. Sign hashed result

I don't see any problem with if next step will be
5. Save signature in array

This should be work for any scriptPubKey. The user will decide for himself what to do with the array from step 5.

[pooya87]

~

I think that signing process is very standard:
1. For all inputs
2. Read scriptPubKey
3. Hash scriptPubKey
4. Sign hashed result

I don't see any problem with if next step will be
5. Save signature in array

This should be work for any scriptPubKey. The user will decide for himself what to do with the array from step 5.

it is not that simple. you are missing a lot of branches after step 2 and your step 3 is wrong, you don't hash the script, you hash  the "modified" transaction. look at https://en.bitcoin.it/wiki/OP_CHECKSIG and https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki for all the details but the summary of it is something like this:

1. for "each" input
2. read related outpoint's scriptpub
2.1. figure out the type of it
2.2. prepare the message digest or digests based on the type
3. hash it/them and get a final digest
4. sign it.

the steps 2.1 and 2.2 can be entirely different depending on what step 2. returned. and it has to be a pre-defined type. for example if it returned P2PKH type then it empties all the other txin scriptsigs and replaces the corresponding input's script with that P2PKH script and then signs the whole modified transaction.
if it was P2WPKH then the entire hashing mechanism changes. you have to concatenate different parts together hash them then concatenate the hashes together and hash them again (BIP143).

if it were non-standard or not pre-defined, like say your scriptpub was this:

Code:

OP_HASH160 <4266fc6f2c2861d7fe229b279a79803afca7ba34> OP_EQUAL

and redeem script was:

Code:

OP_2DUP OP_EQUAL OP_NOT OP_VERIFY OP_SHA1 OP_SWAP OP_SHA1 OP_EQUAL

there is no way any code could predict how to produce a valid signature for this. you HAVE TO do it manually since it is "non-standard". by the way there is no signing or hashing (the transaction) involved in this script link

[kzv]

1. for "each" input
2. read related outpoint's scriptpub
2.1. figure out the type of it
2.2. prepare the message digest or digests based on the type
3. hash it/them and get a final digest
4. sign it.

the steps 2.1 and 2.2 can be entirely different depending on what step 2. returned. and it has to be a pre-defined type. for example if it returned P2PKH type then it empties all the other txin scriptsigs and replaces the corresponding input's script with that P2PKH script and then signs the whole modified transaction.
if it was P2WPKH then the entire hashing mechanism changes. you have to concatenate different parts together hash them then concatenate the hashes together and hash them again (BIP143).

if it were non-standard or not pre-defined...
there is no way any code could predict how to produce a valid signature for this.

I mean the case when the step 2 is returned one of the predefined standard scriptpub. I need to sign nonstandard redeem script only.
For example when I have an P2SH transaction hash (from my grandfather as an inheritance)  and I known that redeem script should by like this

Code:

<signature><data> OP_DROP <public key> OP_CHECKSIG

So I can create a raw transaction with this redeem script manually and only one thing that I can not to do - I can not to sign RAW transaction because redeem script is nonstandard. Of course, if I am a programmer, I just can write a program. But I think that this can be done without programming with openssl for example as a hard way. Or this feature can be placed in one of the next version of bitcoin core API

BTW may be I do not undestand the signing process and API "signrawtransactionwithkey" can sign my unstandard redeem script. Can you explain me?