Need help creating a raw transaction for testnet in Java

[Sir_Krovlood]

Hello!

Me and my friend have recently started learning the basics of bitcoin transactions following various tutorials such as these: https://medium.com/deqode/bitcoin-transactions-made-easy-c3aa838545d4, https://www.javatips.net/api/bitseal-master/src/org/bitseal/crypt/SigProcessor.java

We were looking at BitcoinJ library, but found its code to be cumbersome and unnecessarily confusing, thus deciding to make our own code utilising simpler libraries.

So far we've managed to generate a raw hexcode of a transaction which is able to be decoded by the api, but unable to be pushed to the blockchain:
0100000001f0d7ada8c89b1700b8a63024ba3a7da3e591fe48098cc566575ef19ea691eddb01000 0008b4830450221008c85c45905cb0d222dd2b802209b1af39ea59aec08cd04899a837222639e12 95022030d7e67a596a643904f552389814a6ca47cee4ad4c186dce6b945ca3f8b450a60141045b0 b7d4cd97db3f2bbb79460523dc5e8200fd0b3a614803f08fe87abbcec240aac43fa4aaf5c210a78 0f4e3eb6237379595ae92ad2ad7324901c7bbb8ef33136ffffffff0150c30000000000001976a91 4344a0f48ca150ec2b903817660b9b68b13a6702688ac00000000

Here is the data of transaction we want to make

Our private key (WIF) is 92hTi6bHXN6twTModrsFjVnJHE1x6izMKiYi957d7WhPLghaenj
Our public key is 045b0b7d4cd97db3f2bbb79460523dc5e8200fd0b3a614803f08fe87abbcec240aac43fa4aaf5c2 10a780f4e3eb6237379595ae92ad2ad7324901c7bbb8ef33136
Our public address is mwCBgxjH8EsGj2KQ5HNRAZMozhe3x8kwBU

Previous transaction is dbed91a69ef15e5766c58c0948fe91e5a37d3aba2430a6b800179bc8a8add7f0
Transaction sum is 0.0005

Destination address is mkHS9ne12qx9pS9VojpwU5xtRd4T7X7ZUt

We suspect that the problem arises during digital singing process. We use  java.security and bouncycastle "SHA256WithECDDSA" for signing.

Any help would be appreciated and if necessary we can provide our code.
Thank you in advance!

[pooya87]

your signature is wrong and it is possibly because you signed the wrong thing. it is not possible to know what the mistake was without seeing the code or the bytes you hashed (before hashing). so you have to do it yourself. here is what you should have hashed for signing:

Code:

0100000001f0d7ada8c89b1700b8a63024ba3a7da3e591fe48098cc566575ef19ea691eddb010000001976a914abf54a72e0420000a594c71b210459718a7f062088acffffffff0150c30000000000001976a914344a0f48ca150ec2b903817660b9b68b13a6702688ac0000000001000000

notice that it is the same transaction you posted but with 2 main changes:
1. the signature script is replaced with the pubkey script of the transaction you are spending (notice the correct script length 0x19)
2. the sighash is added at the end as a 4 byte integer.

hash that with SHA256 twice and the result should be this:

Code:

51939b6328d8a041def47b795a709dd871223493191fafe68b86f9278e8a9359

place some breaks in your code and get the intermediate values and check with these to see where you went wrong.

[Sir_Krovlood]

your signature is wrong and it is possibly because you signed the wrong thing. it is not possible to know what the mistake was without seeing the code or the bytes you hashed (before hashing). so you have to do it yourself. here is what you should have hashed for signing:

Code:

0100000001f0d7ada8c89b1700b8a63024ba3a7da3e591fe48098cc566575ef19ea691eddb010000001976a914abf54a72e0420000a594c71b210459718a7f062088acffffffff0150c30000000000001976a914344a0f48ca150ec2b903817660b9b68b13a6702688ac0000000001000000

notice that it is the same transaction you posted but with 2 main changes:
1. the signature script is replaced with the pubkey script of the transaction you are spending (notice the correct script length 0x19)
2. the sighash is added at the end as a 4 byte integer.

hash that with SHA256 twice and the result should be this:

Code:

51939b6328d8a041def47b795a709dd871223493191fafe68b86f9278e8a9359

place some breaks in your code and get the intermediate values and check with these to see where you went wrong.

Thank you for your reply!
We checked the message to be signed and the result of double SHA256 and they are all correct.
It appears that we cannot find the proper signing algorithm for Signature.getInstance() method, which we use for message signing.
We tried SHA256withECDSA, SHA256withECDDSA (with only a single prior hashing), NONEwithECDSA, ECDSA but nothing helped so far

We also checked the transaction you made https://api.blockcypher.com/v1/btc/test3/txs/4cb390243cb1bd8ccfc00a96b66ce117a21e01d12187ee6159a026d54d0951ac?limit=50&includeHex=true
We only fail at digital signature.

Could you tell us what you are using for message signing?

[pooya87]

SHA256WithECDDSA is performing another SHA256 on whatever you give it. so when you give it

Code:

51939b6328d8a041def47b795a709dd871223493191fafe68b86f9278e8a9359

it hashes that and uses this:

Code:

e8ec17b45db131f1ad4cf1dac1c2b4e3bdcea7b5f5217a9d6e50dfe83a234007

instead (which is SHA256 of 5193....). and your generated signature (30450221008c85...) is valid with the above hash (e8ec...) but the hash is not what should have been signed so it is technically an invalid transaction signature.

you could use a trick and pass a single SHA256 to that method and let it perform the second one*!
use this:

Code:

87204d576b96159f5e277ae5b6a1c21872d1749e28e6ad50d216c2aca33e56d7

i think you should get the valid signature. and since that method is deterministic you should get this signature:

Code:

3045022100b380b383a1ca2e93e5acbd57033bfabdae32bbb7b1001ad04d6e05bc0f70828b0220086558b527827b547953f5102a66ccae2210c315c3f4f9a21f08774df540cf9d01

* that is only if you are trying to learn how things work. otherwise it is best if you use a bitcoin library instead of a general purpose cryptography library to perform these tasks. for example signature's S value has been altered here to be lower than n/2 which is something that other libraries won't do.

[Sir_Krovlood]

i think you should get the valid signature. and since that method is deterministic you should get this signature:

Code:

3045022100b380b383a1ca2e93e5acbd57033bfabdae32bbb7b1001ad04d6e05bc0f70828b0220086558b527827b547953f5102a66ccae2210c315c3f4f9a21f08774df540cf9d01

* that is only if you are trying to learn how things work. otherwise it is best if you use a bitcoin library instead of a general purpose cryptography library to perform these tasks. for example signature's S value has been altered here to be lower than n/2 which is something that other libraries won't do.

Could you explain this part about "n/2 length"?

We used SHA256withECDDSA on message 87204d576b96159f5e277ae5b6a1c21872d1749e28e6ad50d216c2aca33e56d7 and got result

Code:

3046022100b380b383a1ca2e93e5acbd57033bfabdae32bbb7b1001ad04d6e05bc0f70828b022100f79aa74ad87d84ab86ac0aefd5993350989e19d0eb53a699a0c9e73edaf571a4

There might be problem with creating ECPrivateKey object. We create it like this

Code:

            BigInteger ourPrivateKeyDValue = decodePrivKeyDvalueFromWIF(ourPriK);

            ECNamedCurveParameterSpec params = ECNamedCurveTable.getParameterSpec("secp256k1");
            ECCurve curve = params.getCurve();

            KeyFactory fact = KeyFactory.getInstance("ECDSA", bouncyCastleProvider);

            java.security.spec.EllipticCurve ellipticCurve = EC5Util.convertCurve(curve, params.getSeed());
            java.security.spec.ECParameterSpec params2 = EC5Util.convertSpec(ellipticCurve, params);
            java.security.spec.ECPrivateKeySpec keySpec = new java.security.spec.ECPrivateKeySpec(ourPrivateKeyDValue, params2);

            ECPrivateKey ourDecoPriK = (ECPrivateKey) fact.generatePrivate(keySpec);

            String pubK = bytesToHex1((params.getG().multiply(ourPrivateKeyDValue)).getEncoded());
            System.out.println("Public Key \t" + pubK);

            Signature sig = Signature.getInstance("SHA256withECDDSA", bouncyCastleProvider);
            System.out.println(sig.getProvider());

            sig.initSign(ourDecoPriK, new SecureRandom());
            sig.update(messageToBeSigned);
            byte [] signature = sig.sign();

Since we get the correct Public key, we suppose that decodePrivKeyDvalueFromWIF(ourPriK) gives us the correct result.

[pooya87]

it is not length it is value (as 256-bit integers) and N is the curve's order.
in elliptic curves cryptography signature (r,s) and (r,-s) are the same and are both valid.
in bitcoin, to protect against malleability there is a consensus rule that every signature must have an s value that is smaller than N/2 if it wasn't then the reported signature must be (r,-s).
we always report numbers in positive and in modular arithmetic -s ≡ N -s (mod N)

the signature you posted is:
30-46
  02-21-00b380b383a1ca2e93e5acbd57033bfabdae32bbb7b1001ad04d6e05bc0f70828b
  02-21-00f79aa74ad87d84ab86ac0aefd5993350989e19d0eb53a699a0c9e73edaf571a4
and as you can see your "s" is bigger than N/2 so it has to be reduced. if you compute N-s (s being the second integer) you will get the same result as i posted above.

Code:

115792089237316195423570985008687907852837564279074904382605163141518161494337 - 111994522657517726718599769603728457093585839206032994055308250032212630925732 = 3797566579798468704971215404959450759251725073041910327296913109305530568605
in hex 0x086558b527827b547953f5102a66ccae2210c315c3f4f9a21f08774df540cf9d

[Sir_Krovlood]

Big thanks to you, friend!
We updated our code for shifting negative values to positive ones and now everything is working and we managed to push another transaction.

Stay safe with all this pandemia running!