New Attack Vector

[kjj]

Well, good luck with that, I guess.

so you are okay with that the developers is destroying bitcoin by trying to make it better?

Destroying?  Nonsense.

Leave your hyperbole at home and go read up on why non-canonical encodings are bad, and why the developers decided to mark them as non-standard.  (That the satoshi client refuses to emit anything it considers to be nonstandard should be obvious.)

[TierNolan]

go make a standard and stick to it.

Oh, the irony .

You basically want openSSL to be the standard.  That makes it harder for clients to be sure they are compatible.  

With the "beauty" checks, a script parser can be sure that it covers every valid signature.

[piotr_n]

Fuck you liar! sipa added the check only 11 months ago.

Don't talk to me like this - I only tried to help you, but I can swear as well.
11 months ago is long enough to cause a hard fork, if they revert the change now.

You should have mined your ugly tx before this change - that would surely prevent it

[desired_username]

BITCOIN IS SHIT, it does not accept signatures from the most well know implementation of crypto algorithms: openssl.

Code:

ThreadRPCServer method=sendrawtransaction
ERROR: Non-canonical signature: wrong length marker
ERROR: CScriptCheck() : f57a2c4d3b8f9653eaee0d5611fcf7c918bcc8903894e148c5b56486fb3f8eaa VerifySignature failed
ERROR: CTxMemPool::accept() : ConnectInputs failed f57a2c4d3b8f9653eaee0d5611fcf7c918bcc8903894e148c5b56486fb3f8eaa

why the fuck is this stuff implemented the way it is?

bitcoin and especially the satoshi client is a stinking pile faulty patches on other patches of bad and stupidly written code!

All main developers is bad at coding, and should feel bad about it.

/rant over

Is there a kokjo-coin available?

[Mike Hearn]

kokjo, what was the OpenSSL command/code you used to generate that signature?

There are reasons canonical checks were adopted (it will fix a variety of attacks and sharp edges in the protocol). However if your OpenSSL is generating non canonical sigs, we'd very much like to find out why.

[kokjo]

Fuck you liar! sipa added the check only 11 months ago.

Don't talk to me like this - I only tried to help you, but I can swear as well.
11 months ago is long enough to cause a hard fork, if they revert the change now.

no its not. there already exist transactions that violates the beauty checks, and other clients accept these transactions too.

[kokjo]

Is there a kokjo-coin available?

No. But it will certainly come if the devs continue to behave like tards

[piotr_n]

Fuck you liar! sipa added the check only 11 months ago.

Don't talk to me like this - I only tried to help you, but I can swear as well.
11 months ago is long enough to cause a hard fork, if they revert the change now.

no its not. there already exist transactions that violates the beauty checks, and other clients accept these transactions too.

no. such transactions don't exist. not in the blockchain.
it doesn't matter what other clients accept - it only matters what miners accept, and 99% of them are already at satoshi 0.8.x

[Pieter Wuille]

fix bitcoin, go make a standard and stick to it.

No offence, but all this work is being done to actually make it comply to the standard.

The problem was that even though OpenSSL always created transactions that were compliant, it accepts signatures in non-standard encodings. This is bad, as every alternate implementation is required to implement the rules exactly as OpenSSL is. We've known this issue for a while, brought it up several times on the development mailing list and on this forum, investigated who was creating such transactions and gotten them to upgrade, until the frequency of such transactions was low enough. Then finally we added a rule to make bitcoind not relay or mine such transaction - without making them invalid even.

In this case, if the signature created made by bitcoind is non-standard, then we have a serious bug in the signature creation code. AFAIK, no bitcoin(d/qt) ever has ever created signatures that are currently considered non-standard however. If you're creating them yourself, it's possible to create valid transactions that are being rejected now. I'm sure you prefer your bitcoin node to reject the transaction immediately over only have the network reject it with you wondering why it doesn't confirm.

There is no risk for a hard fork at all right now, by the way. This is just local client policy about which transactions are relayed.

[gmaxwell]

As this thread politely explains, OpenSSL violates the ASN.1/DER standard and will accept encoding which are invalid in a multitude of ways.  Because of this any implementation of the Bitcoin protocol— no matter what library it used— have to be bug for bug compatible with OpenSSL.  Worse, the behavior creates malleability of signatures which create the attack describe in this thread and which actually turns out to be even somewhat worse than described here.

Malleability  was being actively employed on the network in order to double spend and dos attack.  To close the attacks a strict, canonical, valid encoding must be enforced. As a bonus, if the enforcement is eventually made a network rule alternative implementations will not have to emulate OpenSSL's signature handling bugs.

It's interesting to hear that it's _creating_ these signatures: IIRC, we did not need to change Bitcoin in any way to avoid creating signatures with non-canonical encoding, OpenSSL's weird behavior is on the decoding side. It would be useful to know how you're (ab)using OpenSSL to accomplish this and/or what version of it you're using. It would be concerning, indeed, if there were some way to get the reference software to produce such a thing— but it's certainly not done so on normal systems in the past.

In any case, this was tested for months, deployed in the codebase as non-enforced for months prior to activation, announced widely. Bitcoin will not remain vulnerable to attack just so you can be lazy and produce sloppy inefficiently encoded signatures. If you are unwilling to take the effort to understand this and get it right it seems unlikely that you'll get the rest of the protocol right. Cryptographic distributed consensus systems may not be rocket surgery, but perhaps they're a little too complicated for some people— perhaps the kind of people whos first recourse is to start slinging insults at people for correcting a security vulnerability— to safely implement.

Piotr_n, this is not yet a network rule. Presumably someday it will be. Today this signature correctness is not enforced against blocks, just against free transactions. This means that miners can still perform the related malleability attacks but other users cannot, and it still means that alternative implementations must remain bug for bug compatible with OpenSSL's buggy validation. This is also why it's not a forking-anything to change.

[piotr_n]

There is no risk for a hard fork at all right now, by the way. This is just local client policy about which transactions are relayed.

Are you sure?
From what I see, the only place where IsCanonicalSignature is called from is EvalScript - the same function the blockchain parser uses.

Code:

bool fSuccess = (!fStrictEncodings || (IsCanonicalSignature(vchSig) && IsCanonicalPubKey(vchPubKey)));
if (fSuccess)
    fSuccess = CheckSig(vchSig, vchPubKey, scriptCode, txTo, nIn, nHashType, flags);

[kokjo]

kokjo, what was the OpenSSL command/code you used to generate that signature?

There are reasons canonical checks were adopted (it will fix a variety of attacks and sharp edges in the protocol). However if your OpenSSL is generating non canonical sigs, we'd very much like to find out why.

Code:

ssl = ctypes.cdll.LoadLibrary(ctypes.util.find_library('ssl'))
NID_secp256k1 = 714 # from openssl/obj_mac.h

def check_result (val, func, args):
    if val == 0:
        raise ValueError
    else:
        return ctypes.c_void_p (val)

ssl.EC_KEY_new_by_curve_name.restype = ctypes.c_void_p
ssl.EC_KEY_new_by_curve_name.errcheck = check_result

...

    def __init__(self):
        self.k = ssl.EC_KEY_new_by_curve_name(NID_secp256k1)

...

    def sign(self, hash):
        sig_size = ssl.ECDSA_size(self.k)
        mb_sig = ctypes.create_string_buffer(sig_size)
        sig_size0 = ctypes.POINTER(ctypes.c_int)()
        assert 1 == ssl.ECDSA_sign(0, hash, len(hash), mb_sig, ctypes.byref(sig_size0), self.k)
        return mb_sig.raw

[piotr_n]

There is no risk for a hard fork at all right now, by the way. This is just local client policy about which transactions are relayed.

Are you sure?
From what I see, the only place where IsCanonicalSignature is called from is EvalScript - the same function the blockchain parser uses.

Code:

bool fSuccess = (!fStrictEncodings || (IsCanonicalSignature(vchSig) && IsCanonicalPubKey(vchPubKey)));
if (fSuccess)
    fSuccess = CheckSig(vchSig, vchPubKey, scriptCode, txTo, nIn, nHashType, flags);

Oh, I get it - its the fStrictEncodings.
Thanks for explaining

Yeah, it would not have caused a hard fork - sorry for messing up.
I once analyzed the <hashtype> byte problem (I mentioned before) and then realized that fixing it would require a hard fork. And this mislead me.

[gmaxwell]

Oh, I get it - its the fStrictEncodings.
Thanks for explaining

Yup. The chain is stuffed full of old violations too. You can rig it to always be strict and reindex the chain and watch it reject it if you're bored.

[Pieter Wuille]

@kokjo: mind sharing the code how to embed the OpenSSL-created signature into a sigScript? I expect the problem is caused by how you wrap/pad it, rather than with the signature itself.

[piotr_n]

but the bottom line is: your transaction might get mined after all.
send it to luke-jr - he seems to be running the most liberal mining node out there.
or mine it yourself, if the copper feeding electricity to your house is thick enough

as long as the blockchain protocol allows it, I don't mind any software to reject relaying any transaction, by its own subjective rules.

[jackjack]

Why should Bitcoin protocol accept all openssl formats?

What we need is a protocol, not mad people whining in a forum. That sure won't make them change.

[kokjo]

@kokjo: mind sharing the code how to embed the OpenSSL-created signature into a sigScript? I expect the problem is caused by how you wrap/pad it, rather than with the signature itself.

Code:

sc = [script.OP_PUSH(sig+"\x01"), script.OP_PUSH(key.publickey)]
tx.inputs[0].script = script.encode_script(sc)

as said before its the statoshi clients problem, not me generating bad(wrongly padded) signatures. i already have tracked down the problem in the satoshi client.

but the bottom line is: your transaction might get mined after all.
send it to luke-jr - he seems to be running the most liberal mining node out there. or mine it yourself

wasn't that the guy that put prayers in the blockchain?

I think i have trolled him a bit too much to ask him to accept my transaction with out appropriate fees, and the satoshi client will not accept the transaction unless its over 11 months old.

[gmaxwell]

What we need is a protocol, not mad people whining in a forum. That sure won't make them change.

There is a formal specification for the signature encoding being used. That didn't stop OpenSSL from accepting weirdly encoded signatures. In most kinds of applications it doesn't matter— in the context of a distributed consensus algorithm being overly permissive in what you accept can be a devistaing global consensus failure producing vulnerability.  Practically every blockchain behavior which you'd normally specify has this kind of tight bounding. It's a fairly unusual challenge— the coupling in Bitcoin is much tighter than other protocols, even network routing protocols.

[piotr_n]

wasn't that the guy that put prayers in the blockchain?

I think i have trolled him a bit too much to ask him to accept my transaction with out appropriate fees, and the satoshi client will not accept the transaction unless its over 11 months old.

No - then don't ask him. Just find the IP of his node and send it there...