[BOUNTY: 2.0 BTC] [CLAIMED] Message Signing in Armory

[etotheipi]

Claimed by jackjack!

Original message:


I was going to do this myself, but it should be fairly straightforward to implement and I have some other priorities right now.  But there has been demand for it, so I'll let someone else take a shot it for me.  The string manipulations should be trivial in python.

Goals:

• (1) Create a new message signing algorithm that I can easily wrap with a nice GUI
• (2) It should have three modes:
•  
  • (a) Bare signatures, compatible with signatures from Bitcoin-Qt/bitcoind
  • (b) RFC2440-compatible, ASCII-armored signature blocks (Base64), with the CRC-24
  • (c) RFC2440-compatible, ASCII-armored signature blocks (Clearsign)
• (3) It would be neat if the clear-signed messages were somehow compatible with bare signatures: i.e. you could copy the text into Bitcoin-Qt and signature into Bitcoin-Qt and it would verify properly.  But I highly doubt that's possible... maybe for simple messages
• (4) Unit-tests that demonstrate dash-escaping, newline behavior, and unicode support
• (5) Be willing to release the copyright of your solution to public domain

The reason I think (3) is not possible is that

Sections 6 and 7 of RFC2440 indicate changing newlines (\n) to Windows newlines (\r\n).  And dash-escaping the clear-signed text, but using the original text for signing.  I haven't looked too deeply into it, but what I've seen is that it won't be too complicated, just requires some attention to detail.  Such as getting CRC-24 coded correctly (also looks simple).  

For Base64 encoding: go figure, python has it built-in:  "import base64;  base64.b64encode(msg), and base64.b64decode(sigBlock)"

This doesn't need to be integrated into the GUI at all.  Just have it operate on raw strings, and I will integrate it into the GUI.  The GUI would probably have radio buttons that say "Version 0 (Bare Signature/Bitcoin-Qt)", "Version 1 (Base64)", "Version 1 (Clearsign)", and the appropriate signature would be produced for the text that is in the text box (which may include newlines, dashes, and unicode).

Anyone up for the task?  [/list][/list]

[etotheipi]

Raised the bounty to 2 BTC!

[jackjack]

I'm interested. In how much time this needs to be done?

[etotheipi]

I'm interested. In how much time this needs to be done?

I'd like to have it done in a couple weeks.  But if there are snags and it can't be done before then, I guess there's not much I can do, eh?  But if you look at the RFC, it doesn't look like it will be terribly difficult.  Especially in python.

[jackjack]

Great, I thought your deadline would have been shorter. I'm ok to look at it if you're ok.
I just want to be sure I understand: you want to sign a message with a private key? I thought Armory already signed messages

[etotheipi]

Great, I thought your deadline would have been shorter. I'm ok to look at it if you're ok.
I just want to be sure I understand: you want to sign a message with a private key? I thought Armory already signed messages

Armory's signmessage is not compatible with Bitcoin-Qt.  Their signatures cannot be cross-verified (not even the same format).  I didn't want to follow Bitcoin-Qt because I hate bare signatures.  So I proposed I would become compatible if they started to support "proper" signatures:  which is the original, properly encoded text, along with the signature.  I didn't even get any commitment from them on this but someone suggested that RFC2440 was the way to go, and if I lead, I might get them to follow.  Especially if it builds off an existing standard.

"Version 0" will be compatible with the existing Bitcoin-Qt, so that bare-signatures will cross-verify.  I know some people still want the backwards compatibility. 

"Version 1" implements the target behavior:  copy this base64 block of text into and it pops up a windows saying "The following message has a verified signature from Address X:  <message>".  This is how message signing should be.  Not multiple copy&paste ops which fail if you miss a whitespace character.

[jackjack]

OK, I'm up for that

[slothbag]

Can I add to the bounty.. I need a small standalone C/C++ program written to do the same message signing/verification. I dont need the version 1 proposal, just version 0 which is compatible with bitcoin-qt.

You can copy the code from bitcoin-qt or use something like libccoin, i'm not fussed.  Just as long as its small and fast.

I can add another 1 btc for it

[etotheipi]

Can I add to the bounty.. I need a small standalone C/C++ program written to do the same message signing/verification. I dont need the version 1 proposal, just version 0 which is compatible with bitcoin-qt.

You can copy the code from bitcoin-qt or use something like libccoin, i'm not fussed.  Just as long as its small and fast.

I can add another 1 btc for it

Sure.  I assume jackjack will be making some python blackboxes, one for each version.  I saw your other thread about this, and it does sound like a good match.  You can easily invoke the python directly.  Alternatively, I have all the ECDSA operations wrapped up nicely in the C++ code which is wrapped up in the python.  You could actually access the C++ code directly in a main.cpp file, but you do need a little bit extra to get it working.  Just a thought...

[slothbag]

The 1 btc additional bounty has been claimed by scintill.  He has provided a patch to the official bitcoin repo that includes a separate verifysig program.

Cheers

[scintill]

The 1 btc additional bounty has been claimed by scintill.  He has provided a patch to the official bitcoin repo that includes a separate verifysig program.

I've rebuilt it outside of the bitcoin repo (copying over stuff it depends on) and put it on github: https://github.com/scintill/bitcoin-signature-tools .  It's a C++ command-line verifier for the bitcoin-qt bare signatures.

It's easier to build than what I gave slothbag originally and won't break if they shuffle files around in the bitcoin repo.  I've just noticed that I somehow missed the fact that slothbag asked for signing, too, and this is only verifying right now.  He paid the bounty so I guess he's happy with it , but since those were the original terms he can still require me to add signing for free if he'd like.  Others can request it too, but I don't have time to make it a priority unless there's a bounty.

[etotheipi]

Thanks scintill,

That's a nice compact implementation.  I like it.  I was just looking through the code...

@jackjack

I forgot that Bitcoin-Qt uses key recovery for the signatures.  Meaning, that you only need to provide the signature, and the public key can only be one of four different values.  Thus, signature and a couple extra bits.  This is something I don't have implemented in any of libraries yet, and it doesn't look trivial.   But the code is all there in the C++ from scintill, and I am considering putting it into my own "EncryptionUtils.h/.cpp".

I assume you're not going into the C++ code in Armory...?  Just making standalone python files?  That will work for now, though I can see that having the C++ key recovery would be useful.

If you want to take a shot at implementing it in EncryptionUtils.h/.cpp, I will throw in another 0.5 BTC (on success, of course!).  I already have EC math operations available in the library.  I'm pretty sure you can do most of it with what's already there.  Though, you might have to add a sqrt-function or something (I cheated with UncompressPoint, by just letting Crypto++ do the uncompression for me, so  I didn't actually implement it).

[techwtf]

Here is a working bitcoin-qt compatible signer & checker in python, with ECDSA code embedded, it even supports compressed pubkeys.
https://gist.github.com/anonymous/da91bd30fa068ae4a3bb

if I'm qualified to get a proportion of the bounty: 1HFhtvzNEiy9eooLSJRMyZ1XB2FXgYGvCH. I'm still happy even not anyway.

the sign example is taken from brainwallet.org.

[techwtf]

Here is a working bitcoin-qt compatible signer & checker in python, with ECDSA code embedded, it even supports compressed pubkeys.
https://gist.github.com/anonymous/da91bd30fa068ae4a3bb

if I'm qualified to get a proportion of the bounty: 1HFhtvzNEiy9eooLSJRMyZ1XB2FXgYGvCH. I'm still happy even not anyway.

the sign example is taken from brainwallet.org.

missing code:

Code:

import struct
def encode32(i):
  return struct.pack('<I', i)

def encode64(i):
  return struct.pack('<q', i)

[jackjack]

Thanks scintill,

That's a nice compact implementation.  I like it.  I was just looking through the code...

@jackjack

I forgot that Bitcoin-Qt uses key recovery for the signatures.  Meaning, that you only need to provide the signature, and the public key can only be one of four different values.  Thus, signature and a couple extra bits.  This is something I don't have implemented in any of libraries yet, and it doesn't look trivial.   But the code is all there in the C++ from scintill, and I am considering putting it into my own "EncryptionUtils.h/.cpp".

I assume you're not going into the C++ code in Armory...?  Just making standalone python files?  That will work for now, though I can see that having the C++ key recovery would be useful.

If you want to take a shot at implementing it in EncryptionUtils.h/.cpp, I will throw in another 0.5 BTC (on success, of course!).  I already have EC math operations available in the library.  I'm pretty sure you can do most of it with what's already there.  Though, you might have to add a sqrt-function or something (I cheated with UncompressPoint, by just letting Crypto++ do the uncompression for me, so  I didn't actually implement it).

I do have the key recovery in my python code
Techwtf's implementation too

[etotheipi]

I do have the key recovery in my python code
Techwtf's implementation too

Okay, great. 

I mentioned expanding the Armory C++ utilities because it doesn't use OpenSSL.  It uses Crypto++.  I definitely want that code upgraded, and it might just be one night's worth of work to match the EC-math operations I already have, with the key-recovery algorithm.  Maybe a tad bit of Crypto++ doc searches looking for the needed operations.  That's worth 0.5 BTC to me.

For now, if the key recovery only works in python, that's fine.  It will fit into my interface.  I just wanted the key recovery as part of the C++ operations for other reasons.

[jackjack]

Oh ok I misunderstood, I thought you wanted that for Python. I might look at that once the current problem is solved.

That problem is this one: I thought I was making mistakes in my implementation but looking at some codes I realized that there is currently not a single implementation of OpenPGP that accepts secp256k1.
I tried to add it in gnupg-ecc, but no luck yet...

So... I have done what you asked for (message signing with secp256k1 keys that follows RFC2440) but nobody can read those signatures yet.
This leads to two questions

• 1. Is it enough to claim (a part of) the bounty? I'm not being greedy, it's just that I have other projects and I see that continuing this one (ie implementing OpenPGP verifying) will require some additional hours of work. In all cases, I'll finish and implement that verifying but the ETA's would be different.
  
  
• 2. Say that I'm going to implement that now (bounty or not, it's more about the 'global' 'future' of secp256k1 signing with RFC2440). There's still a problem:
  
  • GPG needs to have the public key stored in your keyring to verify. AND that keyring doesn't accept secp256k1 keys.
  • GPG doesn't accept to verify signatures if the public key and the signature are in the same block. (Not a word about such behavior in RFCs though.)
  So, well... What should I do?
  
  • Create two blocks (key export + signature) ? (OK with GPG behavior BUT (1) two blocks to process: importing, then verifying, and (2) GPG doesn't support secp256k1 key import yet)
  • Stick to RFCs, break compatibility with GPG and create big signatures that contain the pubkey? (simpler to verify: only one block BUT will never work with GPG)
  
  

Anyway, I'll publish my signing code soon. This week hopefully. With both possibilities for problem 2.


PS: All this post is about Version 1 / OpenPGP signatures, not Version 0, which is done

[etotheipi]

Oh ok I misunderstood, I thought you wanted that for Python. I might look at that once the current problem is solved.

That problem is this one: I thought I was making mistakes in my implementation but looking at some codes I realized that there is currently not a single implementation of OpenPGP that accepts secp256k1.
I tried to add it in gnupg-ecc, but no luck yet...

So... I have done what you asked for (message signing with secp256k1 keys that follows RFC2440) but nobody can read those signatures yet.
This leads to two questions

• 1. Is it enough to claim (a part of) the bounty? I'm not being greedy, it's just that I have other projects and I see that continuing this one (ie implementing OpenPGP verifying) will require some additional hours of work. In all cases, I'll finish and implement that verifying but the ETA's would be different.
  
  
• 2. Say that I'm going to implement that now (bounty or not, it's more about the 'global' 'future' of secp256k1 signing with RFC2440). There's still a problem:
  
  • GPG needs to have the public key stored in your keyring to verify. AND that keyring doesn't accept secp256k1 keys.
  • GPG doesn't accept to verify signatures if the public key and the signature are in the same block. (Not a word about such behavior in RFCs though.)
  So, well... What should I do?
  
  • Create two blocks (key export + signature) ? (OK with GPG behavior BUT (1) two blocks to process: importing, then verifying, and (2) GPG doesn't support secp256k1 key import yet)
  • Stick to RFCs, break compatibility with GPG and create big signatures that contain the pubkey? (simpler to verify: only one block BUT will never work with GPG)
  
  

Anyway, I'll publish my signing code soon. This week hopefully. With both possibilities for problem 2.


PS: All this post is about Version 1 / OpenPGP signatures, not Version 0, which is done

Crap.  I hadn't considered the fact that the ECDSA keys could be used in conjunction with OpenPGP.  I actually just wanted the message formatting/armoring, which could then be signed with a Bitcoin private key exactly as Bitcoin-Qt does it now.  But I never wanted interoperability with OpenGPG.  On the upside, you may have already finished this, actually.

Right now, Bitcoin-Qt takes in a message and a private key, and spits out a bare signature.  Verison 0 matches that.   Right?  I assume you tested that it's interoperable with Bitcoin-Qt.

Version 1 would instead spit out an ASCII-armored block that follows RFC 2440 formatting, and only the part specified by RFC 2440 would be signed, but it would be signed "the Bitcoin way."  Which is to HMAC it with "Bitcoin Signed Message:" and sign the result just like you would sign a Bitcoin transaction.  But the signed text and signature would be included together in a single block, using dash-escaped formatting.  Even if there's a slight variation between making a Bitcoin signature fit/make sense where an OpenGPG signature would go, that's fine.  The goal was just to have the message encoding and data-to-be-signed serialized the same way.  Please don't spend any more time digging into OpenGPG. 

Can you post a few examples of what you have?  Specifically, what the input and output looks like the way you have implemented Version 1 (for both Base64 and clearsign)? 

[jackjack]

Ok then!
I think/hope that will anyway be useful in the future

For now I have that:
v0

Code:

{
 'message': 'jjj',
 'signature': 'G52Qg26N+v9BE7WlaG+MQUYF+35Or0UF6cWUp9bRVM46LFT8AP+spHCVLds9gpCh+IGcKLOCLYdLWKgFqvP82PY=', // =base64(signature)
 'address': '1BCwRkTsYzK5aNK4sdF7Bpti3PhrkPtLc4'
}

v1 Clear sig

Code:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

ldld
-----BEGIN PGP SIGNATURE-----
iHkEABMIACEFAlGOIpYCGwMGCwkIBwMCBhUIAgkKCwMWAgECHgECF4AACgkQQAqjLBlarMRT3wEA1IED    (OpenPGP packets)
GsDYPz4OTY0bn35RTz6RkyCh59i47DBnjih4S5IA/R3qcXG/V9Mx8uHzjaU1uM6CrS1II1aij+JqU6vR
2Gtp
=D1/a
-----END PGP SIGNATURE-----

v1 Base64

Code:

-----BEGIN SIGNATURE-----
kA0DAAgTU4HmRG72SLABywpiAFGOJKpsZGxkiHkEABMIACEFAlGOJKoCGwMGCwkIBwMCBhUIAgkKCwMW   (OpenPGP packets)
AgECHgECF4AACgkQU4HmRG72SLBCWAD8Db4nEv/poywtioVXy3nRCIwVkVJc8kULlRVpEeW4Os8BAJ2B
8qnmdGqEmyYbl3ZDV+Osp7440Cdl8WgSv3EHvAMf
=5wDl
-----END SIGNATURE-----

So instead of my clear sig, you'd want that, right?

Code:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

ldld
-----BEGIN PGP SIGNATURE-----
G52Qg26N+v9BE7WlaG+MQUYF+35Or0UF6cWUp9bRVM46LFT8AP+spHCVLds9gpCh+IGcKLOCLYdLWKgFqvP82PY= (same data than v0: 1+64 bytes, may be different if contains \n)
=crc24
-----END PGP SIGNATURE-----

But what about the base64 one? Just this?

Code:

-----BEGIN PGP SIGNATURE-----
G52Qg26N+v9BE7WlaG+MQUYF+35Or0UF6cWUp9bRVM46LFT8AP+spHCVLds9gpCh+IGcKLOCLYdLWKgFqvP82PY= (same data than v0: 1+64 bytes)
=crc24
-----END PGP SIGNATURE-----

[etotheipi]

This is looking good!   

A couple questions/comments.

--GPG keeps the armored blocks to 64-characters wide, not 80.
--Does this do proper dash-escaping? 
--Handles newlines properly?
--Signs the non-dash-escaped, windows-styled-newline string, or something like that (I don't remember the details, that was your job )
--Make it say "-----BEGIN BITCOIN SIGNED MESSAGE-----" and "-----BEGIN BITCOIN SIGNATURE-----"
--Is the v1-Base64 you identified correctly?  It says "BEGIN SIGNATURE", but it should probably be "BEGINE BITCOIN MESSAGE", I assume the text and signature are both bundled in there.

And these signatures use the key recovery?  I assume that's what the 1+64 bytes comment is. 

I really did want python signing.  But I was saying I would pay an extra 0.5 BTC to additionally implement key recovery in cppForSwig/EncryptionUtils.cpp using Crypto++, but it's not strictly necessary or part of the original bounty.