Sign Transaction Bitcoin with Openssl

[barno]

I created Legacy address in testnet enviroment.

Code:

    $ openssl ecparam -genkey -name secp256k1 -rand /dev/urandom -out chiave_priv.pem
    -----BEGIN EC PARAMETERS-----
    BgUrgQQACg==
    -----END EC PARAMETERS-----
    -----BEGIN EC PRIVATE KEY-----
    MHQCAQEEIIec9IrjQkpFhCAqMMNHQuU/6X7+DDH7p/TPr2on9j+XoAcGBSuBBAAK
    oUQDQgAEJNcNWmojf5IU23BoNL6mG1VLcV08UAqXp/YrE6AbMkaVj05xJJCPg2FN
    wu/j5c/5zp6GoBla+XuYagwHl0VwAQ==
    -----END EC PRIVATE KEY-----

    $ openssl ec -in chiave_priv.pem -outform DER|tail -c +8|head -c 32 |xxd -p -c 32 > btc_priv.key
    879cf48ae3424a4584202a30c34742e53fe97efe0c31fba7f4cfaf6a27f63f97

It's my private key WIF

Code:

    cS8KGbfbAxW6DLWvDQAfR4ZPXC2yJVXa7Qj4gYrFZNNGcWtbyPJ6

it is my compressed public key

Code:

    0324d70d5a6a237f9214db706834bea61b554b715d3c500a97a7f62b13a01b3246

it's my transaction data that I want to sign with my private key

Code:

    0200000001bc010c04e1c44a991760fa1cb7af1076bf755077d1cba60d3d4ca3f0de670d210000000000ffffffff01c0aff629010000001976a914f6b61a355d427c892e51bdb261d4f56c4e93b16f88ac00000000

I know that I have to do double sha256

Code:

    $ printf 0200000001bc010c04e1c44a991760fa1cb7af1076bf755077d1cba60d3d4ca3f0de670d210000000000ffffffff01c0aff629010000001976a914f6b61a355d427c892e51bdb261d4f56c4e93b16f88ac00000000 | xxd -r -p | sha256sum -b | xxd -r -p | sha256sum -b > a.txt

Now Sign it (maybe here the error?)

Code:

    $ openssl dgst -sha256 -hex -sign chiave_priv.pem a.txt  
    EC-SHA256(a.txt)= 30440220113f4da0dab33c850b13c3f8e8239acdb9d474808d9c65e91a9304c4573ac3fc0220514df1aaf115535de6e3a7da19011b416b84bd2ec8737958d2d97c6eed6985f7

Create ScriptSig

 - 6A ScriptSig length => 212 char hex
 - 47 Signature Length+SIGHHASH => 142 char hex
 - < signature > 01
 - 01 => SIGHASH_ALL
 - 21 Public key length => 66 char hex
 - < public key >

Below you can find the whole transaction data

Code:

0200000001bc010c04e1c44a991760fa1cb7af1076bf755077d1cba60d3d4ca3f0de670d21000000006A4730440220113f4da0dab33c850b13c3f8e8239acdb9d474808d9c65e91a9304c4573ac3fc0220514df1aaf115535de6e3a7da19011b416b84bd2ec8737958d2d97c6eed6985f701210324d70d5a6a237f9214db706834bea61b554b715d3c500a97a7f62b13a01b3246ffffffff01c0aff629010000001976a914f6b61a355d427c892e51bdb261d4f56c4e93b16f88ac00000000

When I try to sentransaction, I get a error

Code:

    $ bitcoin-cli sendrawtransaction 0200000001bc010c04e1c44a991760fa1cb7af1076bf755077d1cba60d3d4ca3f0de670d21000000006A4730440220113f4da0dab33c850b13c3f8e8239acdb9d474808d9c65e91a9304c4573ac3fc0220514df1aaf115535de6e3a7da19011b416b84bd2ec8737958d2d97c6eed6985f701210324d70d5a6a237f9214db706834bea61b554b715d3c500a97a7f62b13a01b3246ffffffff01c0aff629010000001976a914f6b61a355d427c892e51bdb261d4f56c4e93b16f88ac00000000
        error code: -26
        error message:
        mandatory-script-verify-flag-failed (Signature must be zero for failed CHECK(MULTI)SIG operation) (code 16)

Decoderawtransaction:

Code:

bitcoin-cli decoderawtransaction 0200000001bc010c04e1c44a991760fa1cb7af1076bf755077d1cba60d3d4ca3f0de670d21000000006A4730440220113f4da0dab33c850b13c3f8e8239acdb9d474808d9c65e91a9304c4573ac3fc0220514df1aaf115535de6e3a7da19011b416b84bd2ec8737958d2d97c6eed6985f701210324d70d5a6a237f9214db706834bea61b554b715d3c500a97a7f62b13a01b3246ffffffff01c0aff629010000001976a914f6b61a355d427c892e51bdb261d4f56c4e93b16f88ac00000000
    {
      "txid": "3f360942dca166a75397cbc3f481771bea8f1c2b0c79313b034fb067882b3b36",
      "hash": "3f360942dca166a75397cbc3f481771bea8f1c2b0c79313b034fb067882b3b36",
      "version": 2,
      "size": 191,
      "vsize": 191,
      "weight": 764,
      "locktime": 0,
      "vin": [
        {
          "txid": "210d67def0a34c3d0da6cbd1775075bf7610afb71cfa6017994ac4e1040c01bc",
          "vout": 0,
          "scriptSig": {
            "asm": "30440220113f4da0dab33c850b13c3f8e8239acdb9d474808d9c65e91a9304c4573ac3fc0220514df1aaf115535de6e3a7da19011b416b84bd2ec8737958d2d97c6eed6985f7[ALL] 0324d70d5a6a237f9214db706834bea61b554b715d3c500a97a7f62b13a01b3246",
            "hex": "4730440220113f4da0dab33c850b13c3f8e8239acdb9d474808d9c65e91a9304c4573ac3fc0220514df1aaf115535de6e3a7da19011b416b84bd2ec8737958d2d97c6eed6985f701210324d70d5a6a237f9214db706834bea61b554b715d3c500a97a7f62b13a01b3246"
          },
          "sequence": 4294967295
        }
      ],
      "vout": [
        {
          "value": 49.99000000,
          "n": 0,
          "scriptPubKey": {
            "asm": "OP_DUP OP_HASH160 f6b61a355d427c892e51bdb261d4f56c4e93b16f OP_EQUALVERIFY OP_CHECKSIG",
            "hex": "76a914f6b61a355d427c892e51bdb261d4f56c4e93b16f88ac",
            "reqSigs": 1,
            "type": "pubkeyhash",
            "addresses": [
              "n41SfwRvQD7tXJsbZsEcDWKFQ7DLYi3P7i"
            ]
          }
        }
      ]
    }

[1714011027]

1714011027

[1714011027]

1714011027

[1714011027]

1714011027

[1714011027]

1714011027

[BrewMaster]

I know that I have to do double sha256

Code:

    $ printf 0200000001bc010c04e1c44a991760fa1cb7af1076bf755077d1cba60d3d4ca3f0de670d210000000000ffffffff01c0aff629010000001976a914f6b61a355d427c892e51bdb261d4f56c4e93b16f88ac00000000 | xxd -r -p | sha256sum -b | xxd -r -p | sha256sum -b > a.txt

this is your mistake.
in bitcoin you don't just hash the raw transaction itself, you have to first modify it then hash the modified result.
this hex is missing 2 things:
1. the scriptsig is empty which should have been replaced by the scriptpub of the input you are spending
2. the sighashtype at the end which is a 4 byte integer in little endian order.
you can find more information on bitcoin wiki's checksig or on SE:
https://bitcoin.stackexchange.com/questions/32628/redeeming-a-raw-transaction-step-by-step-example-required

these steps are more complicated for different script types though.

[barno]

Thanks for reply.

I followed the steps but I have some issues.

Paste the new parts here, public keys and private keys are the same.
My enviroment is regtest.

this is my UTXO

Code:

  {
   {
    "txid": "3bd88800252c4d049ced8813b1d6fff76b99d84e56289dfd60eca8898814a8e2",
    "vout": 0,
    "address": "mzjCn3ojXBGWcRjWYfJdNdrM9yr5UbpXgq",
    "label": "",
    "scriptPubKey": "76a914d2bb7890f3f6356d89673367b44e9a7d0265009188ac",
    "amount": 49.99900000,
    "confirmations": 6,
    "spendable": true,
    "solvable": true,
    "desc": "pkh([d2bb7890]03d6e5ff918b8388dc49cdfc115ddc1ea9fd4d884dd8b87ef6602a6bd07b5c1748)#67xu85z0",
    "safe": true
  },

there are my parameters for my transaction:

Code:

TXID=3bd88800252c4d049ced8813b1d6fff76b99d84e56289dfd60eca8898814a8e2
VOUT=0
AMOUNT=49.998
ADDR_MITT=msPjuNgbmbRSkNGJPvquKJRRmrbzS96s62

Now create my transaction data

Code:

$ bitcoin-cli createrawtransaction '[{"txid":"'$TXID'","vout":'$VOUT'}]' '[{"'$ADDR_MITT'":'$AMOUNT'}]'
0200000001e2a8148889a8ec60fd9d28564ed8996bf7ffd6b11388ed9c044d2c250088d83b0000000000ffffffff01c0e4022a010000001976a914824441111b374bec1952a5b3fa9dd4e3ed679b3888ac00000000

Now I put the scriptPubKey and SigHash.

Code:

0200000001e2a8148889a8ec60fd9d28564ed8996bf7ffd6b11388ed9c044d2c250088d83b000000001976a914d2bb7890f3f6356d89673367b44e9a7d0265009188acffffffff01c0e4022a010000001976a914824441111b374bec1952a5b3fa9dd4e3ed679b3888ac0000000001000000

(I tried even with ScriptPub of previous transaction. The UTXO of 3bd88800252c4d049ced8813b1d6fff76b99d84e56289dfd60eca8898814a8e2.)

Double SHA256

Code:

printf 0200000001e2a8148889a8ec60fd9d28564ed8996bf7ffd6b11388ed9c044d2c250088d83b000000001976a914d2bb7890f3f6356d89673367b44e9a7d0265009188acffffffff01c0e4022a010000001976a914824441111b374bec1952a5b3fa9dd4e3ed679b3888ac0000000001000000 | xxd -r -p | sha256sum -b | xxd -r -p | sha256sum -b > a.txt

Code:

result:
f1f941f66a7bd080f14506ff91a05bd3d525dae236a4d86bac5dfa6e3bce4563

Signature

Code:

openssl dgst -sha256 -hex -sign chiave_priv_3.pem a.txt
EC-SHA256(a.txt)= 30450220120a27e6e275545d4b1c79d4fad40a5d6dc9d44f512a00d5b2b71b68e5e4d376022100bf6284c39500565c089efec95db1d82869882a08b30513a3de0287ff223b834f

add 01 at the end.

Code:

30450220120a27e6e275545d4b1c79d4fad40a5d6dc9d44f512a00d5b2b71b68e5e4d376022100bf6284c39500565c089efec95db1d82869882a08b30513a3de0287ff223b834f01

48 is the signature Length.

Public key not compressed is

Code:

04d6e5ff918b8388dc49cdfc115ddc1ea9fd4d884dd8b87ef6602a6bd07b5c1748321ac6d22737633f56538a28158d2558ca09407d9f6cc221c54b873e4cb9b999

41 is the public key length.

Signature+public key is 278 char => 8B

The whole transaction is

Code:

0200000001e2a8148889a8ec60fd9d28564ed8996bf7ffd6b11388ed9c044d2c250088d83b000000008B4830450220120a27e6e275545d4b1c79d4fad40a5d6dc9d44f512a00d5b2b71b68e5e4d376022100bf6284c39500565c089efec95db1d82869882a08b30513a3de0287ff223b834f014104d6e5ff918b8388dc49cdfc115ddc1ea9fd4d884dd8b87ef6602a6bd07b5c1748321ac6d22737633f56538a28158d2558ca09407d9f6cc221c54b873e4cb9b999ffffffff01c0e4022a010000001976a914824441111b374bec1952a5b3fa9dd4e3ed679b3888ac00000000

Code:

$ bitcoin-cli sendrawtransaction 0200000001e2a8148889a8ec60fd9d28564ed8996bf7ffd6b11388ed9c044d2c250088d83b000000008B4830450220120a27e6e275545d4b1c79d4fad40a5d6dc9d44f512a00d5b2b71b68e5e4d376022100bf6284c39500565c089efec95db1d82869882a08b30513a3de0287ff223b834f014104d6e5ff918b8388dc49cdfc115ddc1ea9fd4d884dd8b87ef6602a6bd07b5c1748321ac6d22737633f56538a28158d2558ca09407d9f6cc221c54b873e4cb9b999ffffffff01c0e4022a010000001976a914824441111b374bec1952a5b3fa9dd4e3ed679b3888ac00000000
error code: -26
error message:
mandatory-script-verify-flag-failed (Script failed an OP_EQUALVERIFY operation) (code 16)

Code:

$ bitcoin-cli decoderawtransaction 0200000001e2a8148889a8ec60fd9d28564ed8996bf7ffd6b11388ed9c044d2c250088d83b000000008B4830450220120a27e6e275545d4b1c79d4fad40a5d6dc9d44f512a00d5b2b71b68e5e4d376022100bf6284c39500565c089efec95db1d82869882a08b30513a3de0287ff223b834f014104d6e5ff918b8388dc49cdfc115ddc1ea9fd4d884dd8b87ef6602a6bd07b5c1748321ac6d22737633f56538a28158d2558ca09407d9f6cc221c54b873e4cb9b999ffffffff01c0e4022a010000001976a914824441111b374bec1952a5b3fa9dd4e3ed679b3888ac00000000
{
  "txid": "0101c9603426d1476d813617bc9af7c83eb62d1efec973c096ba55ad74c3501b",
  "hash": "0101c9603426d1476d813617bc9af7c83eb62d1efec973c096ba55ad74c3501b",
  "version": 2,
  "size": 224,
  "vsize": 224,
  "weight": 896,
  "locktime": 0,
  "vin": [
    {
      "txid": "3bd88800252c4d049ced8813b1d6fff76b99d84e56289dfd60eca8898814a8e2",
      "vout": 0,
      "scriptSig": {
        "asm": "30450220120a27e6e275545d4b1c79d4fad40a5d6dc9d44f512a00d5b2b71b68e5e4d376022100bf6284c39500565c089efec95db1d82869882a08b30513a3de0287ff223b834f[ALL] 04d6e5ff918b8388dc49cdfc115ddc1ea9fd4d884dd8b87ef6602a6bd07b5c1748321ac6d22737633f56538a28158d2558ca09407d9f6cc221c54b873e4cb9b999",
        "hex": "4830450220120a27e6e275545d4b1c79d4fad40a5d6dc9d44f512a00d5b2b71b68e5e4d376022100bf6284c39500565c089efec95db1d82869882a08b30513a3de0287ff223b834f014104d6e5ff918b8388dc49cdfc115ddc1ea9fd4d884dd8b87ef6602a6bd07b5c1748321ac6d22737633f56538a28158d2558ca09407d9f6cc221c54b873e4cb9b999"
      },
      "sequence": 4294967295
    }
  ],
  "vout": [
    {
      "value": 49.99800000,
      "n": 0,
      "scriptPubKey": {
        "asm": "OP_DUP OP_HASH160 824441111b374bec1952a5b3fa9dd4e3ed679b38 OP_EQUALVERIFY OP_CHECKSIG",
        "hex": "76a914824441111b374bec1952a5b3fa9dd4e3ed679b3888ac",
        "reqSigs": 1,
        "type": "pubkeyhash",
        "addresses": [
          "msPjuNgbmbRSkNGJPvquKJRRmrbzS96s62"
        ]
      }
    }
  ]
}

[BrewMaster]

Code:

mandatory-script-verify-flag-failed (Script failed an OP_EQUALVERIFY operation) (code 16)

you should pay attention to the error messages you see. they usually tell you what was wrong.
in this case the problem is the public key you included in the scriptsig.

the script verification is like this:
- push 1 item (sig)
- push 1 item (pubkey)
- duplicate top stack item
- pop and hash top stack item with RIPEMD160 of SHA256 and push the result
- pop top 2 stak items and see if they are equal

this last step is where it stops because the hashes are not equal.
with a closer look you are using the "uncompressed" public key (it starts with 4 and is 65 bytes) which is wrong. you should replace it with compressed key which is 03d6e5ff918b8388dc49cdfc115ddc1ea9fd4d884dd8b87ef6602a6bd07b5c1748
then the evaluation will pass.

there is also another point to keep in mind which OpenSSL is not doing which is you MUST have a low S value otherwise the signature verification will fail.
to do that parse the signature, if (S > N/2) then S = N - S

[barno]

Thanks again.
I created a script to replicate my goal.
I change public key uncompressed with compressed.

Sometimes I get that error:
mandatory-script-verify-flag-failed (Non-canonical signature: S value is unnecessarily high) (code 16)
It's the issue that you explain in the last post.

and sometimes I get that error:
mandatory-script-verify-flag-failed (Signature must be zero for failed CHECK(MULTI)SIG operation) (code 16)

[BrewMaster]

i made a small mistake above, your S should not be bigger than N/2 not N. so the check is this: if (S > N/2) then S = N - S
i don't see any other issue with your transaction though.

[barno]

Thanks for your reply, I saw that links about that issue "mandatory-script-verify-flag-failed (Non-canonical signature: S value is unnecessarily high) (code 16)".

https://github.com/bitcoin-core/secp256k1/blob/544435fc90a5672d862e2a51f44c10251893b97d/src/ecdsa_impl.h#L310-L315

Do you know If I can check S before signature or If I can do that function 

Code:

if (S > N/2) then S = N - S

in bash without use external library?

thanks

[BrewMaster]

in bash without use external library?

i doubt that it is even possible because these values (S and N) are a lot bigger than the normal integer sizes that any basic language supports which is 64 bit while those are 256 bit so you must need an external library that provides that functionality for you.

[barno]

Ok I'm back!

The problem is

Code:

openssl dgst -sha256 -hex -sign chiave_priv_3.pem a.txt

In that way I do another SHA256! I did SHA256 3 times!  

To resolve this issue, I can do something like

Code:

$ openssl pkeyutl -inkey chiave_priv_3.pem -sign -in a.txt -pkeyopt digest:sha256 | xxd -p -c 256

or I can do single SHA256 and apply another SHA256 with openssl

Code:

$ printf 0200000001e2a8148889a8ec60fd9d28564ed8996bf7ffd6b11388ed9c044d2c250088d83b000000001976a914d2bb7890f3f6356d89673367b44e9a7d0265009188acffffffff01c0e4022a010000001976a914824441111b374bec1952a5b3fa9dd4e3ed679b3888ac0000000001000000 | xxd -r -p | sha256sum -b | xxd -r -p > a.txt
$ openssl dgst -sha256 -hex -sign chiave_priv_3.pem a.txt

I prefer the first solution!

About "mandatory-script-verify-flag-failed (Non-canonical signature: S value is unnecessarily high) (code 16)" it's more complicated than that.
I converted the S (DER signature) to base10. (it's another signature, not the same of thread, sorry but I have my notes)
For example:

Code:

$ s=`echo "ibase=16; $(printf 00f00e64e164ce4fee984165ba8205a8544ece37458006687cdaa53d4e6e1859bc  | tr '[:lower:]' '[:upper:]')" | bc |  tr -d '\n' | tr -d '\' | awk '{print $1}'`
$ echo $s
108580515770129610852831425129233053758690240817412348750872366071983533218236

Then convert N to base 10, and get N/2

Code:

$ N=`echo "ibase=16;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141" | bc  |  tr -d '\n' | tr -d '\' | awk '{print $1}'`
$ echo $N
108580515770129610852831425129233053758690240817412348750872366071983533218236

$ N2=`echo "$N/2" | bc |  tr -d '\n' | tr -d '\' | awk '{print $1}'` 
$ echo $N2
57896044618658097711785492504343953926418782139537452191302581570759080747168

You can find very cool stuffs if u search 108580515770129610852831425129233053758690240817412348750872366071983533218236 or 57896044618658097711785492504343953926418782139537452191302581570759080747168 in google

Now I Check if s is greater than N/2, if it is I need to subtract it. (N-S)

Code:

$ s=`echo "$N - $s" | bc |  tr -d '\n' | tr -d '\' | awk '{print $1}'`  
$ echo $s
7211573467186584570739559879454854094147323461662555631732797069534628276101

Convert the result to base16

Code:

$ s=`echo "obase=16;$s" | bc`
$ echo $s
E2412F237BCDCA1AD1AD7DA1075D8C0AD258A07066D695F99DEA0AAEC7034A4

Sometimes you can get odd bytes, in that case I have 63 hex.

Code:

$ printf FF19B1E9B31B01167BE9A457DFA57AA6BE0A5A12F4237BEE52D213E621DE785 | wc -c
  63

It's very similar when you get seed phrase, if you don't have a block of 11 bits, you need to add some "padding", then I add 0 at the beginning.

Code:

s=0FF19B1E9B31B01167BE9A457DFA57AA6BE0A5A12F4237BEE52D213E621DE785

Now I can make a "new" DER signature, replace the old s with the new one, calculate the length of it and the length of signature!
And it works!
I hope to help someone!
Thanks to Andrew Chow and BrewMaster for your time guys
(English is not my mother tongue; please excuse any errors on my part)