Old P2SH debate thread

[Luke-Jr]

Not sure if you've all been following the latest push-to-script-hash (p2sh aka OP_EVAL) discussions. OP_EVAL was Rejected because it isn't staticly analyzable (this is a useless property for Bitcoin scripts, but a nice-to-have attribute for idealistic reasons). Now Gavin has proposed a new BIP 16 "/P2SH/" to replace it. However, this replacement is far worse: it basically replaces the scripting system Bitcoin is built on with a single special-cased template. While this might be fine if BIP 16 actually proposed abolishing the current scripting system (while using the templated design to retain backward compatibility), it does not explicitly say so, and Gavin refuses to make this implied decision part of the specification/BIP. In effect, what BIP 16 proposes is the same thing as if a computer were to look for a specific program and stop you from using it unless your computer had a magic USB key for it (that is, it's not the program checking for the USB key, which would be the sane solution, but instead the computer that would be running it).

Since this is a protocol change, Gavin cannot, however, go ahead with it entirely on his own. He needs at the very least a majority of miners to go along with it, and he's mentioned 70% as the cutoff point.

Here's the catch: Gavin is forcing everyone using the latest Bitcoin code to vote for BIP 16. If you want to oppose this insane protocol change, you will need to modify your bitcoind source code or you will be voting IN FAVOUR OF IT by default (as of git master commit 922e8e2). I have written code that allows you the freedom to vote for (-p2sh) or against (-nop2sh):
    https://github.com/bitcoin/bitcoin/pull/755
You can apply it to your bitcoind code like so:
    curl https://github.com/bitcoin/bitcoin/pull/755.diff | patch -p1

Tycho (DeepBit) has expressed dislike of /P2SH/, but has not yet committed to opposing it either. To stop this from going through, the easiest way to achieve the needed 30% is with the support of BTC Guild, BTCMine, EclipseMC, and Eligius combined. However, every little bit helps!

Also, there ARE other solutions to the P2SH need/problem. A compromise between OP_EVAL and BIP 16 would be workable, and there is at least one eval-less proposal that is even easier to statically analyze without breaking the scripting system (CODEHASHCHECK aka CHC). Gavin has thus far ignored this because he just wants to rush into a protocol upgrade to get two-factor addresses out there.

For reference:

• OP_EVAL (BIP 12) is at https://en.bitcoin.it/wiki/BIP_0012
• /P2SH/ (BIP 16) is at https://en.bitcoin.it/wiki/BIP_0016
• CODEHASHCHECK is summarized at https://bitcointalk.org/index.php?topic=56969.msg687795#msg687795

Please let me know if your pool will support or oppose BIP 16.

Luke
Eligius

P.S. Why is static analysis useless for Bitcoin? Because you need to evaluate all scripts to validate them anyway. Sure, you could reject invalid ones with too many signature checks early, but there are equal and better ways to attempt a DoS without these (for example, including exactly the maximum signature checks, then failing for some entirely unrelated reason).

[Gavin Andresen]

Luke, you try my patience.

I'm going to step away from the code for a few days to calm down before I do something stupid because my patience is wearing thin.

[kjj]

However, this replacement is far worse: it basically replaces the scripting system Bitcoin is built on with a single special-cased template.

I've been trying to figure out your opposition to P2SH for a while now.  Your premise seems to be wrong, as far as I can tell, so you are coming to an incorrect conclusion.

Your premise, clearly stated here, is that Gavin is trying to replace the current script system.  That does not appear to be the case at all.  P2SH uses the exact same script system that we've been using all along, the only difference is when the script appears.

In the current system, the script is presented to the network when the transaction is created.  In P2SH, the script is presented when the transaction is redeemed.

There is no other change that I can see.  No one is suggesting that we abandon the current scripting system.  P2SH actually uses the current system to expand itself.  It also enables addresses that correspond to arbitrary protection schemes, rather than needing a new type of special address for every type of recipient wallet.

Also, can you try to explain OP_CODEHASHCHECK better?  I really can't see what problem it is attempting to solve.

[Nachtwind]

Luke.. from a miners position i got to tell you: Just stop it. You make yourself as ridicolous as RealSolid is for a long while and for the same reasons: in the last few days you act like an arrogant self-righteous asshole. I know you will end up telling us about freedom of open source and democtratic elections and stuff, but tell you what - i dont give a shit about that. I like the way bitcoin was handled till now and if Gavin decides one way i trsut HIM more than you to make the right choice since lately you have shown a grade of mental instability that is only comparable to RealSolid.

So, just stop your erratic behaviour or fuck off. Bitcoin can survive better without you at the moment than with you with your current attitude.

Sorry to all for the harsh tone, but i tried to keep out of trouble a while now but the day came i finally had to respond...

[Luke-Jr]

Luke.. from a miners position i got to tell you: Just stop it. You make yourself as ridicolous as RealSolid is for a long while and for the same reasons: in the last few days you act like an arrogant self-righteous asshole. I know you will end up telling us about freedom of open source and democtratic elections and stuff, but tell you what - i dont give a shit about that. I like the way bitcoin was handled till now and if Gavin decides one way i trsut HIM more than you to make the right choice since lately you have shown a grade of mental instability that is only comparable to RealSolid.

If you want a monarchial currency, why not just use the Fed's USD? Gavin isn't perfect, and this is one example. Sorry you consider making people aware of a problem to be "mental instability".

[rjk]

If you want a monarchial currency, why not just use the Fed's USD? Gavin isn't perfect, and this is one example. Sorry you consider making people aware of a problem to be "mental instability".

I would prefer to hear your response to kjj, who seems to be making salient points.

[gmaxwell]

Luke, shame on you for just dropping this message and calling for a forum vote without giving people who disagree with you the fair time to write an opposing position.  (Edit: Specifically I'm complaining about the forum vote aspect of this, Luke has been pretty upfront with his complaints at least since he started raising them)

As with a lot of things, the subtle details here are technical can be tricky to understand.  It's hard to find an expression of the situation to help people reason about the consequences without having to learn everything that the developers know.

Here is my take on the situation:   We have a couple of related needs:  We need to have receiver controlled disposition of funds (e.g. you should be able to elect to have a multi-party secured wallet _personally_ without burdening people who send to you),  we need this to have a compact representation both as an address (600 byte address have severe usability problems) and in the chain (so the sender doesn't have to pay larger fees just because you have a complicated escrow for your account),  and ideally we'd like to move more of the blockchain storage from outputs to inputs because inputs are always prunable so this will help tame chain bloat in the future.

After some discussion it was proposed that we add a new opcode, OP_EVAL which would allow a transaction validated by a script provided in the transaction itself.  With a solution in hand, no more alternatives were considered and implementation was performed.

Things were looking pretty good for about a month, but then Roconnor went to implement OP_EVAL in his own implementation of the bitcoin blockchain validation code and he immediately sounded alarms:  He found some serious bugs in the OP_EVAL implementation (which were quickly fixed), but he also had a more fundamental complaint:  OP_EVAL makes script turing complete, and breaks the ability to statically analyze scripts.

At least the former part wasn't unknown to the other developers— but I don't think we'd thought through the consequences completely because adding turing completeness wasn't at all one of the goals (see the list above), it was a side effect. A side effect we thought was tamed by recursion limits (which weren't actually working), but the recursion limits aren't enough to recover the analyizability of script that is lost.

Analyiziability is important because its what allows you to make definite, accurate, statements about what a script will and won't do without actually executing it.  It also makes it possible to write implementations of bitcoin with stronger concrete and auditable statements about their security.   It's the property that lets you uphold the security principle of "validate input; then act on it".   Because the bitcoin system's security comes _PURELY_ from software— it doesn't matter how great Gavin is, he can't reverse your transactions if they go wrong due to software bugs—  its very important for the continued adoption of bitcoin that we be able to back it up with the most secure software possible, and that we be able to prove that security to skeptics.

Roconnor's concerns came very late in the process, but they deserved a lot of weight: As one of few people who have implemented the whole system from scratch (not even something Gavin can say of himself) he has a important and almost-unique perspective,  he can also rightfully be considered an expert in the field of formal software validation, and his concerns came packaged with real security vulnerabilities that everyone else had missed— proving his deep understanding of the code.   TD, author of the BitcoinJ implementation, had already expressed serious reservations about the whole thing.

Perhaps most significantly for me is this simple fact:   Script was very clearly designed by Satoshi to NOT be turing complete, and considerable effort was made in this design to achieve this property. The second sentence of the description of script on our wiki says "It is purposefully not Turing-complete, with no loops.".  It seems foolish to me to abandon a core design principle of script, one which greatly helps in reasoning about its security, without a darn good reason.  And we don't have a reason at all— our purpose had nothing to do with turing-completeness, that was just a side effect.  (the size limits of script and limited IO make it pretty hard to come up with anything where the turing completeness is actually useful)

After further discussion and iteration over roughly a half dozen other proposals it was realized that it was possible to accomplish _exactly_ the original goals, in a much simpler manner, without introducing Turing completeness.  This is what P2SH does.   I personally think that if we'd come up with the idea of P2SH first, we would have stopped there and never considered OP_EVAL.

Unfortunately,  Luke has taken this position that I consider weird:  That P2SH is bad because it's a special case instead of being a regular opcode that executes arbitrary code, and this special-caseness makes it inelegant.  My own position is that none of this is natural law: every behavior is a special case and happens only because the software says it so— some parts are more regular than others but what we should care about is implementation complexity and P2SH's complexity is very low.

What perplexes me more is that Luke stated he would withhold his complains if the 'other developers' announced that "Bitcoin 2.0" (whatever that would be) would only use P2SH style transactions and that non-P2SH would be deprecated.  I think a lot of people who have considered this consider it a likelihood— the potential pruning savings of P2SH style transactions are tremendous and could significantly reduce spam risks,  but no one can make promises about a future system which doesn't exist and may not involve the current developers or may never exist.

[It was also unfortunate that Luke had other commitments at the weekly development meeting where ~everyone else decided that P2SH was an acceptable compromise,  so he sort of appeared two days later complaining about it after it was perceived by many other people as already settled. As a result not a lot of care has been given to his complaints, even though he has been pretty persistent about them]


I fail to see a lot of genuineness in complaints that take the form of "do it completely, or not at all, a little isn't acceptable" unless there is a good reason that the moderate path is a bad one, and I don't see any reason why the moderate path is a bad one here.

P2SH is a good technical solution which addresses our needs and which has decreased complexity and risk compared to OP_EVAL.   Should there ever be a need for the turing completeness that OP_EVAL offers enough to justify the costs/risks then it could be deployed, P2SH does not preclude the possibility of OP_EVAL in the future.

[jimrandomh]

This would've been a lot better received if it was brought up while the technical discussions were still ongoing. I originally proposed something like OP_EVAL in https://bitcointalk.org/index.php?topic=46429, and I think P2SH achieves everything I wanted out of that. CODEHASHCHECK is arguably better, but only slightly - certainly not by enough to justify delaying deployment. Getting some sort of multisig transactions working is rather urgent, and P2SH is the implementation we have, and it carefully avoids opening any cans of worms.

What this really seems to be about is politics - LukeJr is uncomfortable with Gavin having the power to alter the Bitcoin protocol. I think that this power should be phased out eventually--but not yet. We'll mark that transition with a "1.0" version number. We still need protocol improvements. Gavin's shown no indication that he'd ever abuse his position, and he's the only one who's actually putting in the work required. And this is a really bad time to start a power struggle, because LukeJr isn't really going against Gavin - he's going against the output of a community discussion which he didn't take part in.

(As for the technical side - the issue seems to be that P2SH introduces a new interaction between the concept of "standard transactions" and the scripting system. Before, a transaction would be accepted if it was standard and the script returned success; now there's an additional requirement, which is that one of the standard transaction types now calls for you to rerun the script in a different way. This means that the "standard transactions" concept is now actually part of the scripting system, rather than a secondary sanity check, so it can't be fully dropped in the future. However, the special case is on the sender-script side, and I don't think allowing nonstandard scripts there was ever plausibly a good idea. I do still want to see the standard-transaction types broadened to include fancy tricks with time locking, but that's considerably less urgent.)

[Luke-Jr]

Luke, shame on you for just dropping this message and calling for a forum vote without giving people who disagree with you the fair time to write an opposing position.

I've raised the issue with Gavin more than once. He seems to have been going out of his way to ignore it and force his decision on everyone. I can't just give him all the time in the world because he has set a deadline only a month away; so miner awareness needs to begin now.

Analyiziability is important because its what allows you to make definite, accurate, statements about what a script will and won't do without actually executing it.  It also makes it possible to write implementations of bitcoin with stronger concrete and auditable statements about their security.   It's the property that lets you uphold the security principle of "validate input; then act on it".   Because the bitcoin system's security comes _PURELY_ from software— it doesn't matter how great Gavin is, he can't reverse your transactions if they go wrong due to software bugs—  its very important for the continued adoption of bitcoin that we be able to back it up with the most secure software possible.

Please don't imply this is security-related, or that static analysis provides any security advantage. While it does optimize rule checking in cases that will never matter, it doesn't provide any practical benefit for security.

Perhaps most significantly for me is this simple fact:   Script was very clearly designed by Satoshi to NOT be turing complete, and considerable effort was made in this design to achieve this property. The second sentence of the description of script on our wiki says "It is purposefully not Turing-complete, with no loops.".  It seems foolish to me to abandon a core design principle of script, one which greatly helps in reasoning about its security, without a darn good reason.  And we don't have a reason at all— our purpose had nothing to do with turing-completeness, that was just a side effect.  (the size limits of script and limited IO make it pretty hard to come up with anything where the turing completeness is actually useful)

More foolish to abandon the concept of scriptPubKey being a script entirely (as BIP 16 does), yet still support it being used as a script. OP_EVAL wasn't perfect, but at least the script remained a script.

What perplexes me more is that Luke stated he would withhold his complains if the 'other developers' announced that "Bitcoin 2.0" (whatever that would be) would only use P2SH style transactions and that non-P2SH would be deprecated.  I think a lot of people who have considered this consider it a likelihood— the potential pruning savings of P2SH style transactions are tremendous and could significantly reduce spam risks,  but no one can make promises about a future system which doesn't exist and may not involve the current developers or may never exist.

I wasn't asking for any promises about the future. I was simply saying "if we're going to replace the script with a single hash, let's actually do that"; that is, the BIP should explicitly say "scriptPubKey is considered deprecated and is replaced with hashRedeemScript. For the sake of backward compatibility, hashRedeemScript must be formatted <like so>, and if it is found not to be formatted like this, it is to be interpreted as a legacy scriptPubKey for compatibility only."

[piuk]

Your premise, clearly stated here, is that Gavin is trying to replace the current script system.  That does not appear to be the case at all.  P2SH uses the exact same script system that we've been using all along, the only difference is when the script appears.

In the current script system every operation has a purpose and scripts executes in a methodical order. When P2SH is introduced scripts can no longer be executed simply by interpreting the instructions. P2SH undermines the scripting language because it requires that if the interpreter spots a specific template do something else rather than execute the script.

Take the a standard P2SH template:

scriptSig: [signature] [serializedScript]
scriptPubKey: OP_HASH160 [serializedScriptHash] OP_EQUAL

If it were to follow the rules of the scripting system exactly as it is now it would always return true.

[signature] [serializedScript] OP_HASH160 [serializedScriptHash] OP_EQUAL
[signature] [serializedScript] [serializedScriptHashA] [serializedScriptHash] OP_EQUAL

There is no operation to stay that  [serializedScript] should be deserialized and run. At least OP_EVAL & OP_CODEHASHCHECK follow the rules.

Btw I am not opposed to this for any political reasons, I think Gavin does an absolutely superb job as project leader, it just seems like a short sighted change to me.

[Luke-Jr]

(As for the technical side - the issue seems to be that P2SH introduces a new interaction between the concept of "standard transactions" and the scripting system. Before, a transaction would be accepted if it was standard and the script returned success; now there's an additional requirement, which is that one of the standard transaction types now calls for you to rerun the script in a different way. This means that the "standard transactions" concept is now actually part of the scripting system, rather than a secondary sanity check, so it can't be fully dropped in the future. However, the special case is on the sender-script side, and I don't think allowing nonstandard scripts there was ever plausibly a good idea. I do still want to see the standard-transaction types broadened to include fancy tricks with time locking, but that's considerably less urgent.)

That's a good point, but it's a negative point. "Standard" scripts is merely miner bias. It's a flaw that the software developer is allowed to force his biases on miners, as it is right now, and making "standard" transactions a protocol thing would make the problem even worse. Miners are free to choose what transactions they accept on an individual basis, and work should be done to make that easier, not harder. That is, "standard" transactions should be undefined entirely. Eligius already will accept any transaction, "standard" or not, without bias.

While I disagree strongly with the monarchial powers being given to Gavin, that is not the basis for my objection to BIP 16 specifically.

[gmaxwell]

"Standard" scripts is merely miner bias. It's a flaw that the software developer is allowed to force his biases on miners,

The vendor of the software you're running (the bitcoin development team overall) doesn't believe that the risks (of software bugs, DOS, and bloat attacks) is well enough understood in the software they're releasing to leave that functionality fully open.  Especially because use of the corner cases in the chain may make it difficult to fix them.   It's a conservative position that some people don't agree with— but the people who don't agree aren't stepping up to provide the QA evidence to convince other people of their confidence.  ::shrugs::

I'm happy the reference software takes a conservative position, though I also hope this position is a short term one that will go away with more experience and testing.

Eligius already will accept any transaction, "standard" or not, without bias.

Indeed, and Eligius' willingness to do so indirectly cost MTGOX many thousands of bitcoins when a software bug caused them to send some weird transactions that Eligius mined— though this is clearly not Eligius' fault as MTGOX compounded their own error by then explicitly directing eligius to mine them.

Can you point to a positive use of Eligius' mining of non-standard transaction yet?  When bluematt and I tried using a non-standard transaction for a puzzle a few months back we couldn't manage to get Eligius to mine it.  So I have no examples to suggest.

I ... don't mean to argue that you're doing a bad thing there, but I think the case that what you're doing is an important improvement is fairly weak.   With only one moderate size pool (and perhaps some tiny solo miners) accepting non-standard txn at least the DOS/bloat exposure is limited by the small number of blocks mined with those rules.

[goblin]

This would've been a lot better received if it was brought up while the technical discussions were still ongoing. I originally proposed something like OP_EVAL in https://bitcointalk.org/index.php?topic=46429, and I think P2SH achieves everything I wanted out of that. CODEHASHCHECK is arguably better, but only slightly - certainly not by enough to justify delaying deployment. Getting some sort of multisig transactions working is rather urgent, and P2SH is the implementation we have, and it carefully avoids opening any cans of worms.

Who says it's urgent? Is someone paying money for developing it fast?

The proposed change is quite major as it changes the logic of accepting blocks in the blockchain and it'll require all miners to upgrade. I was under the impression that such changes (i.e. increasing the block size limit) would be implemented and discussed years in advance of going live, and here we have a proposal that's only a month away!

OP_EVAL is one example where too quick a change could've been catastrophic and no-one had really thought of it until Roconnor came along.

Therefore my vote is a NO - this change is way too quick. Rushing it faster by flagging it as urgent is in my opinion dangerous.

EDIT - added:

DISCLAIMER: I don't fully understand the change, so may be wrong there.

[theymos]

In the current script system every operation has a purpose and scripts executes in a methodical order. When P2SH is introduced scripts can no longer be executed simply by interpreting the instructions. P2SH undermines the scripting language because it requires that if the interpreter spots a specific template do something else rather than execute the script.

I agree. P2SH would probably work OK, but it's not consistent with the rest of Script, so I would prefer that it not be used.

(I voted for both P2SH and CODEHASHCHECK in the poll, since I think they're both acceptable solutions.)

[theymos]

Exactly what protocol problem is this trying to solve? Is something broken that needs fixing? Or are we talking about 'fixing' something that isn't broken?

Did you read gmaxwell's long post above?

[rjk]

Exactly what protocol problem is this trying to solve? Is something broken that needs fixing? Or are we talking about 'fixing' something that isn't broken?

Did you read gmaxwell's long post above?

I'm guessing he's looking for a TL;DR explanation
That text wall (I did read it) covered an entire page on my 2048x1154 monitor at full screen

[DeathAndTaxes]

This is kind of where I stand as well. Why is this so important? If I understand correctly, it's to provide more security against stolen wallets? I'm going by this comment:

But the use case I REALLY care about is the secure, multiple-signatures-required-to-spend wallet.

It isn't just stolen wallets.  It allows a range of potential applications where it takes 2 entities to release funds.

Bitcoin needs mutli-sig.  I don't know enough to weigh in on the merits and dangers of the 3 proposed solutions but mult-sig is essential for Bitcoin to grow to the next phase.

[jimrandomh]

Epoch: The problem this is solving is M-of-N multi-key addresses. It lets you set up your wallet so that there are multiple keys, stored in different places - for example, one on your computer, one on your smartphone, and one in a safe - and require that several of them be used together in order to spend coins. That way, if your computer is compromised, the attacker can't steal your coins without also compromising one of the other keys. This feature is extremely important; it would have prevented the allinvain attack, it allows corporations to hold coins without having to completely trust any one employee, and it makes escrow work better. Implementation has to be done in two stages - first adding support for it in the protocol, then second adding support for it in the user interface. We're currently on the first part, and talking about the relative merits of different ways of doing it. All of the proposals currently on the table are protocol changes that enable M-of-N multi-key addresses, and pretty much everyone agrees that at least one of the proposals must be adopted.

[kjj]

Exactly what protocol problem is this trying to solve? Is something broken that needs fixing? Or are we talking about 'fixing' something that isn't broken?

Did you read gmaxwell's long post above?

I'm guessing he's looking for a TL;DR explanation
That text wall (I did read it) covered an entire page on my 2048x1154 monitor at full screen

The short version is that right now the sender has to provide everything up front.  Say I want to set up my wallet to require "(A and B) or C" keys.  To create a transaction that can only be redeemed in that way, I need to provide three public key hashes, or about 480 bits, which makes for long addresses.  It gets worse if I'm a large corporation that wants to require 7 of 13 keys, for example, since then the address is 13*160 bits long.

P2SH lets me create my own script, then give only a single 160 bit hash (of my secret script) to someone that wants to pay me.

I'm sorta coming around to Luke's side, at least a little.  Piuk's reply to me makes it much more clear.  I still haven't figured out how OP_CODEHASHCHECK is intended to work, so I wouldn't say that I'm a convert exactly.

The problem is that there doesn't seem to be any good way to do this.  Anything we do will break old clients, potentially in dangerous ways.  Dangerous for the people using them, of course, not the network as a whole (except that one could argue that harming some users harms us all).

[DeathAndTaxes]

Exactly what protocol problem is this trying to solve? Is something broken that needs fixing? Or are we talking about 'fixing' something that isn't broken?

Did you read gmaxwell's long post above?

Certainly, twice. gmaxwell's post was well put together; I was hoping to get another perspective on it. He mentions 2 main needs: The 1st (needing receiver controlled disposition of funds) is something I don't the understand the purpose/intent of. Perhaps someone could volunteer to briefly explain the key points of why this is needed.

The super simple version on "why":
Currently Bitcoin only supports a single signature for transactions.  That leads to a whole host of potential issues, risks, and need for implicit trust (something Bitcoin was designed to reduce/eliminate).  EVENTUALLY Bitcoin needs some form of multi-sig.  Maybe not today but it does need it.  Without multi-sig too many transactions are "trust me it is safe".  That is fine for an academic experiment but for Bitcoin to grow to the next level multi-sig is essential.  Without some change mult-sig is not implementable in any useful fashion.

The proposal and debate is on HOW to allow effective implementation of a NEW capability (multi-sig).

The super simple version of "how":
We need multi-sig (see how why above)
The initial plan for multi-sig was OP_EVAL.  Simple version: it doesn't work.

Gavin's proposed solution was P2SH.  It was discussed and work is going forward.  Luke missed some meetings so his concerns are coming lake.  Luke believes it is "flawed" because it breaks existing method of handling scripts which is seems a "hack around" and "clunky".  (I think Reread his post for more detailed explanation).

There is an alternative CODEHASHCHECK but that would mean going back to the drawing board and could have problems of it's own.  This has been in the works for some time so I can see Gavin's desire to get it finished.

Luke claim is at this point Gavin is unwilling to back down from P2SH and it is a mistake.  Gavin counter is that P2SH is viable if not perfect.  Some believe Luke fears are excessive or overblown (read gmaxwell post for a rebutal).  I don't think it can be simplified more than that.  This is low level protocol nuts and bolts decisions.