BIP 322 draft design and improvement thread

[achow101]

Did you even read the BIP or are you just spouting nonsense based on the one sentence summary?

It is just true for Bitcoin Core, how is it supposed to be implemented by third party wallets and services, or maybe it is never going to happen, right?

Conveniently, the BIP addresses this. Wallets without script interpreters can use template matching and verify the common, standards scripts. If it finds a script that it is unable to verify, then it should return a result of "inconclusive". A template matching verifier can look at the address it is given and understand what data to expect.

So third party wallets and services can implement the common single key, multisig, lightning htlc, etc. scripts and probably get 99% of the signed messages. It's not that hard to figure out.

Tying a message to a txo as metadata, is the only relevant application of message signing with bitcoin keys.

Where is this being claimed? I don't see that in the BIP text anywhere. And just so you know, if you heard this from luke-jr, not every opinion that luke-jr has is shared by other developers (or anyone else).

Which is absolutely wrong:
First and foremost, this so-called 'tying metadata'(!) is not that important, if it is relevant at all, in the context of 'signing'. The common application is, and has always been, proving authenticity, data integrity and non-repudiability of a message, using bitcoin keys with a possible reference to a txo in the blockchain, if the owner finds sych a reference useful. i.e., the main application is using bitcoin as a public key infrastructure. In the concrete world we are living in, (not the abstract world created by the above claim) signed messages are not supposed to be 'metadata' of a txn, signing is not binding anything to any other thing. It is just about authenticity and other characteristics we mentioned above.

And how does BIP 322 fail to do this? You do know that you can sign arbitrary messages, right? You do know that tying to specific UTXOs is an optional feature of BIP 322, not a requirement? Frankly, it sounds like you did not read the BIP at all and are getting your entire understanding of it based on what other people are saying about it, or perhaps you read it and just failed to understand it.

it is not a use-case, just an imaginary scenario, projected from the solution domain to the problem domain, in an upside-down world. Interestingly, even this "use-case" is not fulfilled by the design:
I provide your application with a utxo reference and a proof bound to some message as being a "sign-by script" BIP322, convincing it, your app. that I'm able to unlock the said utxo, right?
Now what? Your application decides to give me a credit or something, either with respect to or ifnoring the content of my message? Who am I? The one who provided the proof? How this scheme is supposed to resist a simple man-in-the middle attack? What happened to authenticity? What about non-repudiation?

Who said anything about ignoring your message? How does Bitcoin script not have authenticity or non-repudiation?

[aliashraf]

Conveniently, the BIP addresses this. Wallets without script interpreters can use template matching and verify the common, standards scripts.

I purposefully avoided it to stay on the main line which was criticizing the concept, and not the BIP.
As you've brought it up: BIP322 supports both worlds at the same time!
How is it even possible? Is the BIP trying to invent another third universe where paradoxes are considered normal and tolerable?

I got this standard script behind a txo which somehow I'm able to unlock, and I'm trying to tie a message to it,  the problem is I got two different interested parties, one uses Bitcoin Core while the other party has another wallet.
What do I have to do when trying to convince them?
More critically important: When the said parties finish verifying (probably different versions of) my message, the party using the interpreter is convinced about a message being supplied by "somebody" who is able to unluck the txo, but what about the other party? Should they ignore the fact that their wallet already knows the public key(s) that actually was used?

See? Two different messages, two different signing processes, two different verification algorithms, and finally two, radically different application domains. What kind of BIP is it? A derailed BIP!

If it finds a script that it is unable to verify, then it should return a result of "inconclusive". A template matching verifier can look at the address it is given and understand what data to expect.

So third party wallets and services can implement the common single key, multisig, lightning htlc, etc. scripts and probably get 99% of the signed messages. It's not that hard to figure out.

Yes, it's easy when you see it, but you should look, just look. Essentially, the BIP is trying to get implemented by any means without insisting on any principle or philosophy, I suppose I could forge an arbitrary third world and successfully convince Karl Johan to support it as well!

Tying a message to a txo as metadata, is the only relevant application of message signing with bitcoin keys.

Where is this being claimed? I don't see that in the BIP text anywhere. And just so you know, if you heard this from luke-jr, not every opinion that luke-jr has is shared by other developers (or anyone else).

It's Greg Maxwell, not Luke, I like Luke, even when he goes too far. As for Greg? No ways! I mean, I'm a fan, not comfortable with the polemic but a fan, still, I don't know why I frequently find myself on the opposite side of where Greg stands, what's wrong with him?  

You do know that tying to specific UTXOs is an optional feature of BIP 322, not a requirement?

No, it is not optional, the other way is supposed to be optional, you just admitted that Bitcoin Core reference implementation is focused on "tying" stuff. I've already refuted this mix-and-support design policy.

Who said anything about ignoring your message? How does Bitcoin script not have authenticity or non-repudiation?

Bitcoin script, has it through employing ES technologies like ECDSA and Schnorr, but script signing? No! It doesn't. It just implies that the script interpreter has passed testing some witness data for unlocking a txo. It is silent about the identity of the party who prepared this data, hence there is no established mechanism for authentication and repudiation unless you/your app would examine the witness data beyond what the interpreter does, e.g., by checking whther the message is signed and if so, is the signature identical with the one used for unlocking? See? It's so ugly!

[NotATether]

Guys can we stop with these arguments that aren't accomplishing anything? The point of this thread is to discuss how to solve the edge cases of BIP322 to bring it to final status:

Do not derail this thread with off-topic arguments. Proposals made here will be relayed to the mailing list. I am not self-moderating this thread, but if this thread gets derailed, I will lock it, and open a new, self-moderaded thread.

I dropped the old BIP because luke-jr told me it's better to help finalize BIP322, and I want to make change happen [as opposed to whining about it]. That's the end of the story. This thread is not for rambling about why my BIP was (or shouldn't have been) rejected, open a new thread if you want to discuss that.
Message signing with transactions looks ugly but is the only way to demonstrate proof-of-spendability-from-UTXOs. Let's leave it at that.

Anyway, what I did come here to post was that there is apparently a bunch of TODOs for addressing interoperability with various BIPs and proposals that are likewise in draft or maybe not submitted yet - one of them is interoperability with Silent Payments so this is what I have suggested on the Github issue for it (see the second post of this thread):

Here's a solution at least for the Silent Transactions. Since they can't decide which address formats they will use for it (there is a lot of to-and-fro between taproot and segwit), and all such address formats can be represented in the Simple format anyway, just write in  this BIP that Silent Transactions can use the Simple format to sign their messages, and the scriptPubKey/scriptSig/Witness will all be the same.

[aliashraf]

@NotATether,
It is called "BIP322 discussion thread", isn't it?

Change its title to "I want to join the club by any means", or lock it, whatever, just don't get mad and angry because it doesn't make sense to stop discussing in a discussion thread. 

[NotATether]

I have addressed some more TODOs in a mailing list post, which I will quote verbatim:

Although there is a Github issue/PR at https://github.com/bitcoin/bips/pull/1347 for addressing all the TODO items of BIP322, I decided to throw it in the mailing list again to see if anyone else has suggestions for dealing with them.

So in an older copy of the draft at https://github.com/bitcoin/bips/blob/b6b0126e2d04793ba52a40f05d24538fa3f2c9ad/bip-0322.mediawiki , I found the some TODO items, and I will copy-paste the ones in the Specification section (for full proofs) here:

> TODO: How does this interact with as-of-yet-unspecified "Silent Transactions"?
> TODO: Some invalid opcode to allow only in various proof types?
> TODO: A way for the initial signer to delegate to another scriptPubKey; needed for better privacy and CoinJoin/Lightning compatibility

So to start with, I believe it will be very helpful to limit what opcodes scriptPubKeys to be elligible to sign from them. The specification already does so to a point, but in order for these to be recognizable, it's my opinion that oneof the NOPs should be placed at the beginning of the script to activate proof parsing mode.

Of course, an opcode is not necessary at all, if the program is able to infer from context where the proof is coming from. After all, since they cannot be broadcasted, they can't be mined in blocks, so will never be encountered in a full node's usual verifier. I'm not sure what is to be gained from adding an opcode - the only source for real transactions is from P2P-obtained blocks, so when a human inputs a signature to be verified, it can check that a real transaction is not being inserted by looking for the invalid input.

For Silent Transactions, I have already given my suggestion in the PR, that some subsection can be made saying that it can operate with them by using its scriptPubKey (and other stuff that may be necessary - I am not excatly sure what goes inside the Witness stack of message_signature).

In the case of the last TODO, related to delegation to another scriptPubKey, Iam not quite sure at the moment what to do about it - perhaps you guys can place a MAST (two Merkle branches, to be specific) - the first branch has the original signer's scriptPubKey, the second branch contains the delegated signer's scriptPubKey.

At this point, the core of the BIP is already done, but such cases need to be addressed as "how can I sign a message from the Lightning Network" and "how can I allow another person to sign a message on my behalf, without giving him/her access to my private keys?"

Side note: I don't think most people here even know what a Merkle branch is, the form that is already used in Taproot, so if I prepared an explanation about this data structure, would it help you guys understand what is being proposed here?

Side note 2: I'm starting to suspect that the witness stack which contains the solution to a "challenge" in the second transaction which the spec calls 'to_sign' just has an ecdsa signature and public key (from bottom-to-top order) in the case of P2[W]{SH|PKH} - I'm not really sure what is pushed on a Taproot address, so I may have to make a guide for that too.

[silent transactions as referred to in the mailing list and Github is the same thing as Silent Payments thread started by @witcher_sense].

[garlonicon]

> I suppose in the case of legacy P2PKH signing, a hypothetical OP_CHECKDATASIG can take <signature> <pubkeyhash> off the stack and perform an ECDSA public key recovery

You can always perform key recovery for legacy ECDSA: "<signature> OP_SWAP OP_CHECKSIG" is always spendable, for any valid <r,s> DER-encoded pair. Here, if "<signature> <messageHash> <pubkey> OP_CHECKDATASIG" works for signature verification BIP, then you only need to provide "<signature>" as an input, the rest could be placed in some output, and your program could easily compute that. Or you can use "OP_DUP OP_HASH160 <address> OP_EQUALVERIFY <messageHash> OP_SWAP OP_CHECKDATASIG" as your output script, then the signer has to provide a valid signature, and a valid public key explicitly, but if you only have some signature, then you can always recover that key by yourself.

Because if OP_CHECKDATASIG is not handled, then still, signing by scripts can handle that, but the Script would be unnecessarily complex. And because there is no need to introduce OP_CHECKDATASIG on-chain, it can be safely done inside this BIP.

> I'm not sure how an OP_RESERVED in an unexcuted OP_IF is going to help implement an ECDSA pubkey recovery + DUP/HASH160/EQUALVERIFY hybrid instruction.

Because if you want to support any script, and you want to add a new opcode, you have to make it somehow compatible with existing schemes. You have to assign some byte for OP_CHECKDATASIG or for any new opcodes you want to add. And you don't want to assign just the next free opcode, because you will get OP_CHECKSIGADD or other opcodes from TapScript, so your script decoder will no longer handle two script versions at once. Unless you want to make it also compatible with altcoins like BCH, but I guess it is not needed.

No new opcodes are needed. Just use "OP_DUP OP_HASH160 <address> OP_EQUALVERIFY OP_SWAP OP_SHA256 <messageDoubleHash> OP_EQUALVERIFY OP_CHECKSIG", and then check two things:
1) is "<signature> <messageSingleHash>" valid for this transaction, according to BIP-322
2) is "<signature> <messageSingleHash>" valid for a given "Bitcoin Message"
Then, you can hash "Bitcoin Message" and check the second case in your software, and also consider such signature valid. By making the above script "standard", you can expand verification, just in the same way as P2SH was introduced.

[NotATether]

No new opcodes are needed. Just use "OP_DUP OP_HASH160 <address> OP_EQUALVERIFY OP_SWAP OP_SHA256 <messageDoubleHash> OP_EQUALVERIFY OP_CHECKSIG", and then check two things:
1) is "<signature> <messageSingleHash>" valid for this transaction, according to BIP-322
2) is "<signature> <messageSingleHash>" valid for a given "Bitcoin Message"
Then, you can hash "Bitcoin Message" and check the second case in your software, and also consider such signature valid. By making the above script "standard", you can expand verification, just in the same way as P2SH was introduced.

I understood the whole script up to OP_CHECKSIG.

From my understanding, OP_CHECKSIG will compute the hash of all the inputs, outputs and script combined together.

For simple and full proof, we can simply compute such signature by hand and place it on the stack for the purposes of equality comparison.

But in the case of legacy proofs, which are supposed to be backward-compatible with the ECDSA (r,s) signatures, it would result in a different signature from when computing r,s from the private key.

This raises an interesting question: P2PKH message signing as it is now won't be removed out of backward compatibility concern, so does the legacy proof simply imply doing the standard ECDSA signing while skipping the entire script interpreter and tx format?

[Pieter Wuille]

The Bitcoin script system *is* a (programmable) digital signature scheme on its own, which achow101 is referring to as "script signatures". This digital signature scheme is distinct from the ECDSA/Schnorr signature algorithms available in Bitcoin script (but builds upon it).

• The "public keys" of this digital signature scheme are the scriptPubKeys in transaction outputs.
• The "messages" of this scheme are the spending transactions, excluding witness data.
• The "signatures" of this scheme are the scriptSigs and witnesses in transaction inputs.

It's a programmable signature scheme in that it supports more complex assertions than "a single party with key X agrees"; e.g. it can express agreement of multiple parties (using e.g. a P2SH scriptPubKey with a redeemscript that requires signatures with multiple keys). The actual script semantics are more-or-less irrelevant for this. It just suffices to express the kinds of assertions we care about.

BIP322 is taking this script signature system, and transposing it to a different context: messages that aren't transactions. Everything else remains the same: the scriptPubKeys remain the "public keys" verified against (=addresses), the "signatures" remain scriptSigs/witnesses (but now embedded in a BIP322 signature encoding, rather than being placed in a transaction). However, the "messages" are replaced with a virtual transaction derived from the message being signed, rather than any real transaction. This permits reusing all the script logic a signer and/or verifier may have (however complex or simplistic that may be) for transactions to be immediately applied to messages.

No new opcodes are needed or involved - the only thing that changes is that in BIP322 context, OP_CHECKSIG (and friends) don't compute a sighash from a spending transaction, but from a virtual transaction that is derived from the message being signed. If you want to BIP322-sign for "I have the ability to spend funds sent to address X" with message M, all you need to do is demonstrate your ability to "spend" funds sent to X, using a virtual transaction that commits to M. If you have the capacity (in terms of access to private keys, and in terms of having signing algorithms for it) to do that for the script corresponding to X (whether that's a simple single-key constructor, or something far more complicated) for real transactions, you also have the ability to do that for this BIP322 virtual transaction.

I don't understand any of the concerns brought up here. AFAICT, the only thing necessary to move BIP322 forward is finalizing the last details of the specification, and implementing it (which may mean bring the Bitcoin Core implementation up-to-date, if BIP changes are made, and/or implementations for other software, which may be template based).

[aliashraf]

The Bitcoin script system *is* a (programmable) digital signature

No!
It is just an invention, a bad one: Programmable digital signature. There was no such thing in the world before you guys made it up, right? No problem, as super devs you are entitled to do inventions, but as an observer I have all the rights to review and to resist.

The "messages" of this scheme are the spending transactions, excluding witness data.[
{-snip-}
BIP322 is taking this script signature system, and transposing it to a different context: messages that aren't transactions.

What happened now? Bitcoin script is a digital signature with transactions as their messages, then it evolved to a general pupose one for arbitrary messages? So, there are two consecutive inventions : first we invent programmable DS, and immediately we go all-in to make it even more general purpose? Fantastic but absurd, isn't it?
Or maybe BIP322 is just hijacking (taking) your invention, landing it where it was not supposed to land (through a transposition).  

I understand how it feels, getting high by doing abstractions and generalizations, but it needs to pass some review and criticisn. I'm afraid the vague and loose terminology is a bad sign.

Bitcoin scripting system is not a digital signature scheme, it may use this technology (we have signature free scripts, don't we?). Using a technology doesn't make a system identical with that technology.

However, why should anybody do it? Ruining an established terminology.
What's the catch? Proof-of -ownership or something? Why should it be considered an act of signing? Why don't we simply open a BIP for this one with an appropriate terminology? It is not "message signing" per se, it is "virtual transacting", whatever.

I don't understand any of the concerns brought up here. AFAICT, the only thing necessary to move BIP322 forward is finalizing the last details of the specification, and implementing it (which may mean bring the Bitcoin Core implementation up-to-date, if BIP changes are made, and/or implementations for other software, which may be template based).

Then how is it possible to have both, template based and interpreter based implementations in one world?
Above thread, I've already discussed it, for now let's check just one issue (there are more, believe me):

For a p2sh where the script is well-formed and can be supported by both, either the message produced by interpreter friendly style is not compatible with template based verification and fails, or it is compatible.
For compatibility, you should
1- Prepare and sign a message template based.
2- Put the signed message in the virtual txn.
3- Supply witness data, using your signature again!
4- Every time check the verifier and supply properly either the virtual txn or the message.

The last step is necessary unless you IMPOSE dependencies on template based wallets.

Ugly & Disgusting.

My Suggestions: (I want to be constructive)

1- First and foremost, stop scrambling the terminology, I know, we all know, Greg is genius, you are genius, ..., no doubts, but we are all humans, sometimes we go too far.

2- Ripping this sign-by-script concept off from BIP322, let it focus on true signing with support for references to standard txns (with well-formed scripts).

3- Make another BIP Just for proof-by-interpretation thing. Find a proper label.

Do not mix signing-with-ref-to-txo and proof-by-interpretation/whatever.
[/list]

[Pieter Wuille]

It is just an invention, a bad one: Programmable digital signature. There was no such thing in the world before you guys made it up, right? No problem, as super devs you are entitled to do inventions, but as an observer I have all rights to review and to resist.

It's okay if you don't feel this terminology is warranted. Do you disagree it is useful abstraction?

Then how is it possible to have both, template based and interpreter based implementations in one world?

A template-matching based verifier will only be compatible with a subset of potential BIP322 signatures, reporting "inconclusive" for others. That's the price for not having a full script interpreter.

But even if *all* implementations use just template-matching based verification, this approach still has the advantage of defining a single format that is compatible with *all* potential future extensions that correspond to script features. Because addresses are encodings of scripts, and what we're signing for is the ability to spend outputs sent to a certain address, using script for the message signing too is just the obvious match in my view.

2- Ripping this sign-by-script concept off from BIP322, let it focus on true signing with support for references to standard txns (with well-formed scripts).

I believe it is entirely uninteresting to work on any kind of message signing system that is restricted to a subset of what script can do. That is postponing another inevitable future problem again, when that subset no longer suffices.

[garlonicon]

There was no such thing in the world before you guys made it up, right?

Wrong. You can always send your coins to "<messageHash> OP_DROP <pubkey> OP_CHECKSIG" and spend them, then it will be quite similar to OP_CHECKDATASIG, the difference will be just in the format of the message: in "Bitcoin Message" you have this famous prefix, here you would have transaction data as your prefix. So, the different message will be signed, but the whole functionality will be the same. Or you could have "<pubkey> OP_CHECKSIG" and use different sighashes that would take your message from the stack instead. There are many options.

No problem, as super devs you are entitled to do inventions, but as an observer I have all the rights to review and to resist.

Good luck banning "OP_SHA256 <messageDoubleHash> OP_EQUALVERIFY <pubkey> OP_CHECKSIG" from the script, and all other variants of that.

Why should anybody do it?

Because you cannot stop people from implementing it on their own. Having it standardized would mean that their signatures will at least be compatible with some BIP. Now we have a situation, where different wallets implement their own ways of handling signatures, so there is a choice between getting it standardized or not. You cannot stop it, because it is not something that has to go through some kind of soft-fork, it is purely informational no-fork, trying to resist is like trying to stop HD wallets.

Also note that we have almost-BIP-322-compatible network called signet, and note that it can be hijacked to serve the same purpose: the recipient could set some signet challenge, and wait for a valid signed block. The only catch in hijacking the signet is that mining is needed to make a valid signature. But one simple change, and you can get fully working script verifier, if you set the difficulty to the same value as in regtest.

Then how is it possible to have both, template based and interpreter based implementations in one world?

Guess what, P2SH is template-based and interpreter-based: you have some template, some fixed opcodes here and there, and it works. It is fully compatible with other nodes, because just more data are pushed into the stack, and old nodes simply check if the hash of the script is correct, but upgraded nodes actually execute the script. And here it could be similar: some nodes could know how to handle transactions, and other nodes could simply require some magic bytes in some format, and then perform signature checking on some hashed data, and report everything non-standard as "not implemented".

First and foremost, stop scrambling the terminology, I know, we all know, Greg is genius, you are genius, ..., no doubts, but we are all humans, sometimes we go too far.

So, go on, and propose some BIP to get rid of the entire Script, and go back to P2PK-only-world. Or make a sidechain for that, and put your coins there. Because as long as we have Script, and as long as it gets more and more complex, there will be use cases for signing different outputs. Unless you want to tell someone that "You have your coins on P2SH/P2WSH/P2TR? Too bad, please move to P2PKH, and sign it, then I will verify it".

[aliashraf]

Do you disagree it is useful abstraction?

For sure, if it holds any water, it would be poisonous, imo.

A template-matching based verifier will only be compatible with a subset of potential BIP322 signatures, reporting "inconclusive" for others.

NO! It will be incompatible with ALL BIP322 generated signatures and vice versa, They belong to two different worlds and only with lots of dependencies and ugly stuff can agree upon a message being signed: you need to generate and sign a template friendly message then put this message in BIP322 virtual txn then again sign it!! lots of other ugly details here.

The question is: why should they communicate ever? One being a conventional message signing algorithm, the other being a special proof-of-authority "thing".

But even if *all* implementations use just template-matching based verification, this approach still has the advantage of defining a single format that is compatible with *all* potential future extensions that correspond to script features. Because addresses are encodings of scripts, and what we're signing for is the ability to spend outputs sent to a certain address, using script for the message signing too is just the obvious match in my view.

In my view, it is a too long shot which ends in a total miss-fire. There is no reasonable way for circumventing the versioning requirements when you add features, trust me, it will never happen, there is no general solution for all the problems. Soft-forks do not happen in bitcoin on a daily basis, it is not a "project", like what Etherem is, so, it'd be easy to add new templates ever few years or so.

I believe it is entirely uninteresting to work on any kind of message signing system that is restricted to a subset of what script can do. That is postponing another inevitable future problem again, when that subset no longer suffices.

With all due respect, Peter, you need to re-think this approach. You are free to implement your script version and promote it as whatever you wish.
I'm serious, it is useful, but as far as it is not introduced as a signing mechanism, hence "postponing" the development and implementation of a standardized template based (universal) signing system.

The interpreter based system you suggest is good and useful, but in its own domain and universe, for its own (future?) applications.

[aliashraf]

There was no such thing in the world before you guys made it up, right?

Wrong. You can always send your coins to "<messageHash> OP_DROP <pubkey> OP_CHECKSIG" and spend them.

Really?  
Just because we have scripts that USE Digital Signature schemes, then we have a Programmable Digital Signature system? Are you kidding or what?
Before rushing to the keyboard, always do your homework: Google it! Programmable Digital Signature System.

Done? As I said, and you quoted, there is no such thing in the world, Right?

As for your show-off with what you've learned about the way interpreter is implemented in bitcoin client:

Please note: We are comparing message signing using templates with this job done using the interpreter, the latter, according to BIP32 is done by putting the message in a virtual/invalid transaction and signing this transaction, these two are incompatible, so, no, they don't reach to an agreement. The only way for this to happen is the ugly dependent way, I briefly described.

[achow101]

NO! It will be incompatible with ALL BIP322 generated signatures and vice versa, They belong to two different worlds and only with lots of dependencies and ugly stuff can agree upon a message being signed: you need to generate and sign a template friendly message then put this message in BIP322 virtual txn then again sign it!! lots of other ugly details here.

NO! I think you misunderstand what "template matching" means here. This isn't jamming some other signature format into a BIP 322 signature. There's no "template friendly message" to create, and no signing again.

In this context, template matching occurs against the output script for which the BIP 322 signature signs for. This output script is produced by the address provided, as addresses encode output scripts. A template matching verifier would simply look at the address, see if that address type is one that it understands. Because standard scripts have fixed witness structures, the verifier can look at the scriptSig and witness to see if those match the template, then extract the components that it expects, and verifies that those are correct. This also generalizes to things that don't have addresses, such as Lightning HTLCs, as the script has a fixed structure and so the witnesses will also have a fixed structure.

For example, a template verifier for P2PKH would be given a P2PKH address. It understands P2PKH addresses and knows that it should expect a scriptSig that begins with a byte between 0x07 and 0x49, followed by however many bytes that number was, followed by a byte that is either 0x21 or 0x41, followed by however many bytes that number is. It knows that the first of those is an ECDSA signature, and the second is a public key. It knows that it should check that the public key's hash160 should match the one encoded in the provided address. It knows that the signature, pubkey, and the sighash of the current transaction, should validate using ECDSA. It can do all of this, because the programmer hard coded these things in. If anything doesn't match, it just immediately says the signature is invalid. If it gets an address it doesn't know, it tells the user that verification is inconclusive because it doesn't know that address type.

Template matching is literally just the inverse of signing. It's just that instead of putting things into a scriptSig or witness, it's pulling data out. This technique works for any set of scripts where the opcodes remain in (relatively) fixed positions, the only things that differentiate them are preimages and pubkeys.

[aliashraf]

NO! I think you misunderstand what "template matching" means here. This isn't jamming some other signature format into a BIP 322 signature. There's no "template friendly message" to create, and no signing again.

Reading through your post, it looks that there is no template matching, it is verification without interpreter. The fake BIP322 transation is to be verified! A client without the interpreter! People have hell of a job to support it, don't they? Very few are invited to this party, I suppose.

It is not the way you design software, seriously. It is just contingent dependency, we need the message to be signed explicitly, otherwise we are dependent on this verifier.  Now, check a real template based solution and compare the elegance and beauty:
We have three parties: the signer, the verifier and the consumer.

The signer:
 already knows which address should be used, it signs a message with keys for the address, then it attaches the address the script behind it (if any).

The verifier:
First and foremost, checks whether signature is correct per se. If passed, the message is authenticated, but it continues checking if the signer has included supplementary, bitcoin blockchain related info, if so, it compares the address received directly or extracted from the supplied script (template based), comparing it with the keys used, if passed, the address reference is attached to the message.

The consumer module:
Has a signed message with public keys disclosed and a reference to a bitcoin address for profiling purposes and as a metadata.

Such a signed message is independent of bitcoin-like clients/modules, standing on its foot, it is self-contained.

Note that we have a total different scenario here with what Greg Maxwell has proposed:
Here the signed message is the core data, relation with bitcoin is metadata. Greg has put it upside-down because in the real world, we focus on the message. If this up-side down world has an excuse to exist, who am I to oppose, what I'm insisting on is to do it separately and in its own application domain (being a very narrow domain, I suppose).

P.S.
Note how downward compatible and inclusive is the scheme I suggested, with just standard Digital signature libraries, very simple applications can communicate (just ignoring the bitcoin address related meta data) with each other as well as with much more sophisticated applications.

Also note that there is no need to reject the upside-down world, it can exist and do whatever it is  supposed to do.

[NotATether]

At any rate, it means that besides TODOs, all we have to change in the BIP is specify that "The behavior of OP_CHECKSIG and related opcodes is changed when verifying proofs to hash `to_spend` instead of a real transaction" or words to that effect (apologies if I missed something in this correction).

Note that we have a total different scenario here with what Greg Maxwell has proposed:
~

As far as I know, gmaxwell did not help with the BIP322 design, so why do you keep bringing him up here?

[garlonicon]

No new opcodes are needed.

Even better: we have OP_PUBKEY and OP_PUBKEYHASH. They make scripts automatically invalid when occured, so they behave as OP_RESERVED, but they are just named. So, "OP_PUBKEY OP_CHECKSIG" and "OP_DUP OP_HASH160 OP_PUBKEYHASH OP_EQUALVERIFY OP_CHECKSIG" could be used to mark transaction as invalid, and turn on backward-compatible verification at the same time.

At any rate, it means that besides TODOs, all we have to change in the BIP is specify that "The behavior of OP_CHECKSIG and related opcodes is changed when verifying proofs to hash `to_spend` instead of a real transaction" or words to that effect (apologies if I missed something in this correction).

Exactly. And if we want to make it somehow compatible with "Bitcoin Message", then OP_PUBKEYHASH could represent P2PKH addresses, and OP_PUBKEY could be used to check P2PKs. Then, it can be safely combined with other opcodes, because it would behave exactly as OP_RESERVED, and will mark transaction as invalid when used on-chain, but will be accepted when skipped.

[NotATether]

OK, I just figured out how to solve the delegation problem.

In BIP322, there is a [planned] provision for some person to delegate signing to another person. That means the second person can do all the signing stuff that the first person could do, and the signature is as if the first person signed it.

What this could be useful for?

- L2/Lightning Network, a channel is just 2-of-2 multisig, so a prospective channel co-creator "delegate" signing to the channel itself, by signing a UTXO inside the multisig - signing along with the other party, creating a dummy output (see below), which can be signed to prove liquidity on behalf of the channel, while keeping the channel itself anonymous.
- CoinJoin, To prove that some CoinJoin coordinator is liquid without violating its anonymity by revealing public keys, the person managing the CoinJoins delegates signing from all UTXOs to be used in the CJ, at once, delegating signing to another dummy output.
- By the same token, Mixers can prove their liquidity without revealing their UTXO set.
- Silent Payments, where the public key is not even known in the first place, the address of a silent payment can delegate signing to another dummy output which only the sender and receiver know about.

So how does this delegation work? It's very simple:

1. All UTXOs that want to delegate signing to a different party must sign a preliminary transaction of this format:
- All input/output amounts are zero.
- input 1 is an invalid input of the kind in BIP322
- the rest of the inputs are the UTXOs wanting to delegate signing to a different party, with valid signatures/witness stacks.
- there is only one output, and it is a P2WSH output with the following script:
OP_PUSH <hash-of-address-hash> OP_SWAP OP_HASH160 OP_EQUALVERIFY
- And the witness stack that will "spend" the transaction in the "to_spend" tx is simply:
<address-hash>
- Likewise, the "to_spend" tx has only one input, refering to the txid of the delegating transaction with output point 0 i.e. the UTXO <delegation-txid>:0. Outputs of "to_spend" remain the same.
-- Contrary to the use of Hash160, we are NOT hashing a public key or script. We are hashing an address hash, implying that we are using addresses.

Do you know why I said "delegating to a different party"? Because it could be a functionally different entity, just like how CEO is diffferent from LLC company even if it has only 1 employee. The "address" here represents a kind of company <but is not a smart contract> - it can represent a channel, it can represent a coinjoin, it can represent a silent payment. The channel/CJ/etc. only has to hash the decoded RIPEMD160 of an address, with another SHA256-RIPEMD160, to make an "address" that can be used to sign messages from.

This "address" aka. LLC company can even be encoded with Bech32 to make it look like a real address - obviously don't send any funds directly to that address because they will be lost - and in fact, it *should* be Bech32-encoded when transmitting the Signed Message.

A signed message has these three parts:

Message
Address
Signature

BIP322 specifies the signature is just the raw transaction format of "to_sign". Normally, the address would simply be the address you are signing from, but in the case of delegation, it is desireable for the original addresses to remain anonymous. So since an address must be filled in somewhere, the Bech32 "hash-of-address-hash" created above can be used as a P2WSH address.

Advantages of this scheme:

- The real addresses/UTXOs signing the transaction are only known to each other and whoever is managing the "to_delegate" transaction.
- Only the real signers and the person who is in charge of the P2WSH output can sign the "to_delegate" output and thus sign a message from it (note that they could be the same person).
- There can be an arbitrary number of delegations before the transaction is actually signed (the new person who is in charge of signing, i.e. has the P2WSH output of the "to_delegate" transaction can simply generate another address hash, and delegate to that "address" in another transaction, giving some other person that "address" if they want to)
- Delegated signatures can wrap Full and Full with UTXOs signing formats, so Light clients do not have to directly support those two formats, either for complexity reasons, or because they have no UTXO set.
- And crucially: **There is no on-chain transaction, so the delegation is private and cannot be traced back by the public**.

And there are virtually no disadvantages to this.

I should emphasize that you don't delegate signing to another person, you delegate signing to another party that may just be comprised by one person. I say this because the delegation does not make any new on-chain UTXOs that someone could posess, but it simply creates a hash160 of some address hash that was generated by the delegators, and the hash-of-address-hash does not necessarily have to represent a person, it can also represent a service.

FAQ:

Q: Does this utilize the Full format?
A: Yes.

Q: How to represent the delegation in a signed transaction?
A: Just encode the hash-of-address-hash in Bech32 version 0 and put it in the Address field.

Q: If the delegation is private, then how can the address-hash be known and the transaction signed?
A: The UTXO signers take a random address associated with them <could be one of their own address, could be the address of a multisig that is being used elsewhere etc.> and then make the hash160 of that address. Whoever they give this to, can sign a BIP322 transaction.

Q: How can the public verify a delegated BIP322 transaction if the address-hash is private?
A: The hash-of-address-hash is revealed in the Address, not in the Signature. BIP322 states that the signature only contains the "to_sign" transaction, which does not contain the witness stack of "to_spend" that has the hash-of-address-hash (because THAT was already spent in "to_spend") therefore the address hash is Bech32 (version 0) encoded and can be decoded to re-construct "to_spend" transaction and from there "to_sign".

Q: How to differentiate between non-delegated and delegated signatures?
A: You can't. But then again, BIP322 doesn't differentiate between "message signatures from an address" and a signature from a set of UTXOs so it wouldn't be able to identify a delegated transaction anyway.
Rather, a full-blown verification software should present a list-box or a set of radio buttons, that toggles between "Legacy", "Simple", "Full", "Full with UTXOs" and "Full with Delegation" - Each of these controls the content in the Address field - this would already be required to support validating Legacy signatures anyway, which are otherwise incompatible with the transaction-based signing (and this is the workaround BIP322 specifies to support that).

Q: What if the verifier does not have a UTXO set (light clients)?
A: Then present three toggles or radio buttons: "Legacy", "Simple", and "Delegated" - each of these options only require a single encoded address to be specified in the field, and Full, and Full With UTXOs signatures can be wrapped with a delegation to support a single address. Consequentially, these control how the signed transaction is [re]constructed for sign/verify.

[garlonicon]

OP_PUSH <hash-of-address-hash> OP_SWAP OP_HASH160 OP_EQUALVERIFY

Just use "OP_HASH160 OP_PUSH <hash-of-address-hash> OP_EQUAL". This OP_VERIFY is needed only if this part should be concatenated with something else as "<condition> OP_VERIFY <condition2>". Also note that OP_HASH160 is in practice "OP_SHA256 OP_RIPEMD160", so you could use OP_RIPEMD160 instead of OP_HASH160, and then just push 256-bit hash, reached as SHA-256("something").

[NotATether]

OP_PUSH <hash-of-address-hash> OP_SWAP OP_HASH160 OP_EQUALVERIFY

Just use "OP_HASH160 OP_PUSH <hash-of-address-hash> OP_EQUAL". This OP_VERIFY is needed only if this part should be concatenated with something else as "<condition> OP_VERIFY <condition2>".

Nice catch. I should probably update the corresponding mailing list entry.

Also note that OP_HASH160 is in practice "OP_SHA256 OP_RIPEMD160", so you could use OP_RIPEMD160 instead of OP_HASH160, and then just push 256-bit hash, reached as SHA-256("something").

I could've just used OP_RIPEMD160, but I followed the example of P2WPKH-P2SH where the address hash is hashed with SHA256 and RIPEMD160 to create a new, nested address hash, so I prefer to leave it like that.


There is actually one flaw I found in this scheme - it's that you can't verify that the message is indeed signed with the UTXOs, because they are not even known.

This stems from a tradeoff where you can't have both the privacy of delegation, and verifiability of the delegation, at the same time.

I'm not sure what to do about this.

[I should probably collect all this info on my website instead of dumping it all on the mailing list]