Bitcoin Forum

I haven't seen discussion of BIP 17 (https://en.bitcoin.it/wiki/BIP_0017) anywhere besides IRC, so I thought I'd start one.

I'll start by saying that I'm trying hard to put aside my biases and dispassionately evaluating the proposal on its merits (I'll just say that I'm not happy with the way BIP 17 came to be, but it is what it is).

Quick executive summary of BIP 17:

A new opcode is proposed, OP_CODEHASHVERIFY, that replaces OP_NOP2.

It is used in a new "standard" scriptPubKey that looks like:


... which is redeemed using a scriptSig like (for example, a 2-of-2 CHECKMULTISIG):



OP_CODEHASHVERIFY is defined to take the hash of everything in the scriptSig from the last OP_CODESEPARATOR and compare it to the top item on the stack. If the hashes match, then it is a no-op, otherwise script validation fails. (see the spec for all the details for what happens if there is no CODESEPARATOR or a CODEHASHVERIFY is put in the scriptSig)


BIP 17 is an alternative to BIP 16, which has a scriptPubKey:


... which is redeemed with:



I see the appeal of BIP 17 -- the redeeming opcodes aren't "hidden" as serialized bytes, they're right there in the scriptSig. That feels less like a hack.

However, there are a couple of practical reasons I like BIP 16 better:

• Old clients and miners count each OP_CHECKMULTISIG in a scriptSig or scriptPubKey as 20 "signature operations (sigops)."  And there is a maximum of 20,000 sigops per block.  That means a maximum of 1,000 BIP-17-style multisig inputs per block.  BIP 16 "hides" the CHECKMULTISIGs from old clients, and (for example) counts a 2-of-2 CHECKMULTISIG as 2 sigops instead of 20. Increasing the MAX_SIGOPS limit would require a 'hard' blockchain split; BIP 16 gives 5-10 times more room for transaction growth than BIP 17 before bumping into block limits.
• With BIP 17, both transaction outputs and inputs fail the old IsStandard() check, so old clients and miners will refuse to relay or mine both transactions that send coins into a multisignature transaction and transactions that spend multisignature transactions.  BIP 16 scriptSigs look like standard scriptSigs to old clients and miners. The practical effect is as long as less than 100% of the network is upgraded it will take longer for BIP 17 transactions to get confirmed compared to BIP 16 transactions.
• Old clients and miners will immediately accept ANY scriptSig for BIP 17 transactions as valid. That makes me nervous; if anybody messes up and sends coins into a BIP 17 transaction before 50% of hashing power supports it anybody can claim that output. An advantage of BIP 16 is the "half-validation" of transactions; old clients and miners will check the hash in the scriptPubKey.

I also have some theoretical, "just makes me feel uncomfortable" reasons for disliking BIP 17:

• OP_CHECKSIG (https://en.bitcoin.it/wiki/OP_CHECKSIG) feels like it was originally designed to be in the scriptPubKey-- "scriptSig is for signatures." Although I can't see any way to exploit an OP_CHECKSIG that appears in the scriptSig instead of the scriptPubKey, I'm much less confident that I might have missed something.  I'm much more confident that BIP 16 will do exactly what I think it will (because it is much more constrained, and executes the CHECKSIG exactly as if it appeared directly in the scriptPubKey).
• Changing from the scriptSig being just "push data onto the stack" to "do the bulk of verification" also makes me nervous, especially since nodes that relay transactions can add whatever they like to the beginning of the scriptSig before relaying the transaction. Again, I can't think of any way of leveraging that into an exploit, but the added complexity of code in the scriptSig and requiring OP_CODESEPARATORs in the right place makes me nervous.
• I've never liked OP_CODESEPARATOR-- it is not like the other opcodes, the way it isn't affected at all by OP_IF and the way it 'steps out' and causes the raw bytes of the transaction to be hashed.  Nobody has been able to figure out how to use it, and the best guess is it is like your appendix:  maybe useful in the past, but not useful now.  Safer to get rid of it entirely, in my opinion.

Gavin's signature:

Help me out:
Ask your favorite miner or mining pool operator to support P2SH for a more secure Bitcoin

I was waiting until I finished a reference implementation (http://luke.dashjr.org/programs/bitcoin/w/bitcoind/luke-jr.git/shortlog/refs/heads/checkhashverify_backport) to post about it, but thanks for the early review. :)

Thanks, and sorry for not being more tactful with introducing it.

This is indeed a benefit to the evaluation-based solutions (both OP_EVAL and BIP16), but I personally feel it's not needed nearly as urgently (how many blocks have over 1000 txns of any kind so far?) and there are many other such hidden-from-old-versions improvements that could be thrown in. BIP 17 does not preclude adding such a solution (ideally without the crucial flaws in BIP 16) in the future should it become necessary, after people have had more time to figure out just what should be added and implement them (for example, I think everyone agrees it would be nice to provide pubkey extraction from signatures).

Since scriptSigs must always follow scriptPubKey, does this really make a big difference? ie, if people can't send them, they can't receive them anyway.

Old clients don't know how to receive any P2SH transactions right now, so they won't display them in any circumstance.. Old miners won't accept them because they fail IsStandard. There is a slight risk of producing a block stealing BIP 17 funds and resending them immediately, but the same attack vector also affects the other solutions and has no effect on people who follow the best practices of waiting for 6 confirmations.

It's evaluated the exact same way in all 3 scripts, and already accepted in scriptPubKey. If there is an attack vector here (which seems very unlikely), it is there both with or without BIP 17.

Isn't it?

This is only true when BIP 17 is in use. Why is it a problem to hash the script executed, as opposed to execute the script hashed?

I support BIP 17 or a proposal like it. I think that a standard method should be used. Worrying about old clients is not a valid concern, IMO. Bitcoin is still evolving. If every change had to worry about old clients, then in 10 years the bitcoin protocol is going to be a mess. Creating hackish instead of elegant solutions will only make the bitcoin protocol a convoluted mess several years down the road.

On release, a big warning can be placed on it to not use the feature until > 50% of clients support. Easy as that. Bitcoin has survived years without these features. It will survive a couple more months for a solid implementation to be created / adopted.

I'm glad to see discussion about these proposals.  It's always nice to have working code now, but considering we are building the currency of the future, lets take time and make sure we do it right.

I'm still not sure which proposal I like the best.

Is either proposal easier for a client to manage?  It looks like both require the client to keep track of the script since it is not in the blockchain.  Is this correct?  Is there a potential to "lose" the script, or can it be recovered in some way?

That's the same for all 3 BIPs; the client already needs to manage a private key. The scripting sugar is easy to reproduce if you have that.

What if we implement both approaches on two different networks: BIP16 on Bitcoin and BIP17 on Litecoin.
I consider Litecoin a healthy alternative to Bitcoin, it might be lacking some marketing as of now, but that might change in the future.
This way we will have a backup solution for Bitcoin in case the approach taken was a mistake.

How about we leave scams out of this? Off topic.

By the way... if there is no fully-functional reference implementation yet, you really shouldn't be putting "CHV" in your coinbases yet. The string in the coinbase really aught to mean "this code is all ready to support this feature,"  because full support from a majority of hashing power is what we want to measure.


Imagine you're an early adopter.  You ask people to send you money into your spiffy new ultra-secure wallet.

With BIP 16, transactions TO you will take longer to get into a block because not everybody is supporting the new feature.

But transactions FROM you will look like regular transactions, so the people you are paying won't have to wait.

That is not a big difference, but it is an advantage of the BIP 16 approach.


No, they are not evaluated in the same way.  The bit of code in bitcoin transaction validation that makes me nervous is:
... in SignatureHash(), which is called from the CHECKSIG opcodes.  scriptCode is the scriptPubKey from the previous (funding) transaction, txTmp is the transaction being funded.

This is the "Copy the scriptPubKey into the scriptSig before computing the hash that is signed" part of what OP_CHECKSIG does.

I like BIP 16 better than OP_EVAL/BIP 17 because BIP 16 does two complete validations, once with the scriptPubKey set to HASH160 <hash> OP_EQUAL and then once again with the scriptPubKey set to (for example) <pubkey> OP_CHECKSIG.

BIP 16 essentially says "If we see a P2SH transaction, validate it, then treat it is a normal, standard transaction and validate it again."

BIP 17 will run OP_CHECKSIGs when it is executing the scriptSig part of the transaction, which is a completely different context from where they are executed for the standard transactions we have now.

Again, I can't see a way to exploit that but it makes me very nervous.

I sat down and studied these proposals this morning.  However these proposals came about should be set aside in favor of doing what's best for bitcoin.  I have no idea what transpired, so I feel my judgement isn't clouded in that respect (could be in other respects, but not that one).

Disclaimers: I've not considered any backward compatibility issues which might trump everything I have to say.  I've also not considered any implementation concerns (complexity of the code, etc).  That might also trump what I have to say.  And finally, while I think I have a good grasp of bitcoin scripting, I am by no means an expert.

I think sending coins to a script hash is good thing and perhaps the way it should have always been done, even for simple transactions.  The BIPs say that P2SH makes it possible for the recipient to define the script required to spend coins.  But here's an observation:  It has always been the case that the recipient defines the script required to spend coins.  It's just that there has, to this point, been a universal assumption that when the recipient sends you an address, that the recipient desires that a specific kind of script (a simple single signature).  It's also not true that with P2SH that the recipient doesn't need to tell the sender the form of the script that is desired.  The recipient is telling the sender the form of the script.  It's just that since the script is only executed when spending coins, the only thing necessary to communicate to the sender is a hash of the script, not the entire script.  All transactions should work this way (the sender will never have to ask whether this is a simple, single signature transaction, or a P2SH transaction…they will all be a P2SH).

I think BIP17 is a superior proposal to BIP16 and BIP12.  BIP12 (OP_EVAL) simply adds complexity to the method of executing scripts.  Validation of scripts now needs to look into the content of data push opcodes.  I can't see that it introduces any particular security holes (i.e. AFAIK there's no way to put something onto the stack that didn't originate inside the script itself …akin to a SQL injection attack).  However, I also can't see that it adds value.  BIP16 feels like it's caught somewhere between BIP12 and BIP17.  It's neither a general purpose OP_EVAL proposal and yet it still requires special handling and execution of code pushed onto the stack for no other reason than to compute its hash.  This just feels a bit hacky to me, especially in light of OP_CODESEPARATOR, which seems to be designed for this sort of thing.

I'm not sure why the use of OP_CODESEPARATOR is a concern.  It seems to me that the way BIP17 is using it is exactly what it was intended for.  It's purpose as far as I can tell is to provide a means of isolating the portions of a script that should be subject to a hash/signature from those that shouldn't (i.e. you obviously wouldn't want a signature to actually be included as part of the hash used for signing purposes).  I assume that there is an implicit OP_CODESEPARATOR that sits between the ScriptSig and ScriptPubKey (the terminology here should probably be updated to better reflect what these two things really are…but I'm not sure how I would describe them).  I supposed if I had it to do over, I would have created an opcode to push a segment of script onto the stack…it would push just the portion of code since the last OP_CODESEPARATOR.  Then you could have something like

   scriptSig: [signature] OP_CODESEPARATOR [pubkey] OP_CHECKSIG OP_PUSHCODE
   scriptPubKey: OP_HASH160 [20-byte-hash of {[pubkey] OP_CHECKSIG} ] OP_EQUAL

Now that I type this, I realize this would only require one new opcode, OP_PUSHCODE instead of OP_CHECKHASHVERIFY…maybe BIP18?  Like BIP17 (unlike BIP12 and BIP16), there are no special execution semantics, and OP_PUSHCODE seems a little more generally useful than OP_CHECKHASHVERIFY (which is generally desirable for things that are going to consume an opcode).

I don't see any issue with using OP_CHECKSIG in the ScriptSig.  And, in fact, it's in the scriptSig in both BIP16 and BIP17 (the only difference is a meaningless difference the mechanics of how it gets executed as far as I can see).  But if someone can come up with a plausible reason to be concerned, they should certainly voice it.  I don't however think it's good to make a decision based on a vague feelings (for or against).

There used to be, but before the protocol was made immutable it was changed so the scripts are instead executed in sequence with only the main stack inherited between them.

I agree this would be better, but I'm pretty sure it's impossible to make it backward compatible.

On further thought, I would revise this:

   scriptSig: [signature] OP_CODESEPARATOR [pubkey] OP_CHECKSIG OP_PUSHCODE
   scriptPubKey: OP_HASH160 [20-byte-hash of {[pubkey] OP_CHECKSIG} ] OP_EQUAL

To be:

   scriptSig: [signature] OP_CODESEPARATOR [pubkey] OP_CHECKSIG
   scriptPubKey: OP_PUSHCODE OP_HASH160 [20-byte-hash of {[pubkey] OP_CHECKSIG} ] OP_EQUAL

The reason is that in the previous form if you wanted to attack it, it's only necessary to reverse the hash160 function with whatever the scriptSig leaves on the stack.  In the second form, not only would you need to reverse the hash160 function, but it would also have to be a valid script sequence (a bit more secure).

Regarding backward compatibility, I think Gavin makes good points about BIP-17 in the OP (especially the point about spend transactions automatically being considered valid no matter what by old nodes).

The more I study this, the more I'm starting to feel that allowing just a specific set of standard transaction types is an unnecessary constraint on the system.  Is it an irrational fear?  Why not allow all valid scripts?  What is the risk?  It seems like it's going to be a real problem going forward if every time someone comes up with some new type of transaction they want to craft that they need to get the entire network to upgrade in order to do it (especially with P2SH).  This kind of coordinated upgrade should only be necessary if you're adding or changing the behavior of the opcodes.  Soon after the P2SH transition happens, you're then going to need to convince miners to start accepting various multi-sig transaction types.  Each time you do this, you have to be very careful or risk serious chain split.  It seems to me that the need for more interesting transaction types, and the risk of chain split when adopting them, warrants serious study of lifting the "standard transactions" restrictions.

From what I understand, part of the risk is that scripts could consume unpredictably large amounts of resources (time, memory, etc.), so we try to constrain the system to put upper bounds on what is possible.  The different proposals are seeking to find the best way to add as many new features as possible (including the ones we haven't dreamed up yet), without making the scripts too "powerful."

I suppose the others can explain more, or correct me if I'm wrong. ;)

That's exactly what this is: "adding or changing the behavior of the opcodes". A non-P2SH multisig could be released today and individual miners could start accepting it immediately. The only downside is that the newly-whitelisted transactions wouldn't be considered standard yet by most clients, so it won't be as easily relayed to the miners that would accept the transaction. Also, the transactions wouldn't be mined as quickly.

I'm aware of that.  But right after P2SH is supported, you then have to whitelist new transactions to get multi-sig.  This whitelisting process requires coordination.  I just wonder whether most (if not all) valid scripts should be allowed along with the p2sh change (subject to limitations on the number and complexity of operations).  Since the script language is not turing complete, it is possible to estimate the cost of executing a script and impose such limitations.  Otherwise, every time a new type of transaction becomes desired, you have to go through this process of coordinating the upgrade of clients to support it.

I'm wondering, which of the two approaches is easier to deprecate?
Or is it cut in stone after it is deployed?
Imagine somebody comes up with a new way of doing multisig in a year and everybody agrees that it's the right way.

Also what approach makes it easier to add new features on top of?
Think of the list of features for the next couple of years and see if the discussed multisig proposals fit nicely or not.

It's easy to deprecate (= stop using) just about anything. However, dropping support for it is not quite as simple - in any case, we'd need everyone to spend transactions still using it.

BIP 12 is the most flexible. BIP 16 gets a one-shot at changing any aspect of the scripting system (and it's being wasted on merely recounting sigops...). BIP 17 cannot change any scripting fundamentals, but can be used for one additional check at any part of a scriptPubKey. If BIP 17 is deployed, something like a combination of BIP 12 and 16 could be still easily used in the future as an upgrade.

Can someone remind me why BIP-12 has fallen out of favor?  OP_EVAL might add some amount of flexibility (regarding where a script is defined vs when it is actually executed), but none of these proposals seems radically different form one another.

BIP 12 cannot be statically analyzed. AFAIK no practical need for static analysis has surfaced yet, but so long as things are being rushed, it's not safe to say they won't in the future either...

Ah, right.  Why do you say there's no practical need?  One practical need is to determine a priori whether a script is too computationally expensive to be allowed.  With OP_EVAL, I could push some code on the stack and evaluate many times over…it's possible you could trivially mitigate that problem by limiting the number of allowed OP_EVALs, but it does make it difficult (if not impossible) to determining up front, in all cases, the cost of running a given script.  You could push code onto the stack that pushes more code onto the stack and executes another OP_EVAL (creating an infinite recursion that may be difficult to detect).  For reasoning similar to the omission of loops and jumps, I would be hesitant to have an OP_EVAL.  I think the code separator & checkhashverify (or ideally pushcode) is the cleaner approach.  The whole objective here is that you want a mechanism to hash the script required to spend a transaction.  OP_EVAL goes way beyond that.  And even BIP16, which also evaluates code you push on the stack, seems wrong to me (and would make the implementation more complex and static analysis more difficult).

With a general OP_CODESEPARATOR OP_PUSHCODE, you would have the flexibility of pushing any sequence of code onto the stack for the purposes of later computing a hash value, but you would never be executing code that was pushed onto the stack.  One improvement upon that would be to somehow ensure that only code which will actually execute would be hashed (to make it impossible to just push a random value onto the stack and then use its hash).  OP_CODEHASHVERIFY has that advantage, but requires that the hashed code immediately precede the operation.  It's possible I'm being overly paranoid though.

As for backward compatibility & chain forks, I think I would prefer a clean solution rather than one that is compromised for the sake of backward compatibility.  Then I would lobby to get people to upgrade to clients that accept/propagate the new transactions and perhaps create patches for some of the more popular old versions designed just to allow and propagate these new types of transactions.  Then when it's clear that the vast majority of nodes support the new transactions, declare it safe to start using them.  Any stragglers that haven't updated might find themselves off on a dying fork of the block chain…which will be a great motivator for them to upgrade.  ;)

…and then stop evaluating when you hit the limit. Even with a static-analysis limit in place, I could waste just as much of your computation time by putting just under the limit, then failing with OP_0 or such. Knowing the cost beforehand doesn't stop any known attacks.

Here is a BIP 17 transaction on testnet (http://blockexplorer.com/testnet/tx/6ac20a9a6e53c53f13b4b3e2c17b259faab4b788dff089ca2b98ef14535e66d0) created with my new checkhashverify branch (https://github.com/luke-jr/bitcoin/tree/checkhashverify). Please help test/review!

BIP 16 explicitly states:
"Validation fails if there are any operations other than "push data" operations in the scriptSig."

Let me try again for why I think it is a bad idea to put anything besides "push data" in the scriptSig:

Bitcoin version 0.1 evaluated transactions by doing this:


That turned out to be a bad idea, because one person controls what is in the scriptPubKey and another the scriptSig.

Part of the fix was to change evaluation to:


That gives a potential attacker much less ability to leverage some bug or flaw in the scripting system.

Little known fact of bitcoin as it exists right now: you can insert extra "push data" opcodes at the beginning of the scriptsigs of transactions that don't belong to you, relay them, and the modified transaction (with a different transaction id!) may be mined.


Are you volunteering to make that happen? After working really hard for over four months now to get a backwards-compatible change done I'm not about to suggest an "entire network must upgrade" change...

And BIP 16 makes this true again: the receiver now controls (to a degree) both scriptSig and "scriptPubKey". BIP 17 retains the current rules.

You can insert extra non-PUSH opcodes too, and mine them yourself... Basically, we already can put non-PUSH stuff in scriptSig, so if there is a vulnerability here, it's already in effect.

I think just about every developer agrees with Gavin that this is not worth a blockchain fork...

So, instead of a fork, you try and create a hacky solution that old clients won't completely reject, but at the same time won't actually fully verify?  It doesn't seem so clear cut that this is preferable to a fork.  It seems like a bigger risk that you have clients passing along transactions that they aren't really validating.  And, I'm not sure a block chain fork is the end of the world.  Consider that when a fork occurs (the first block with a newer transaction type not accepted by the old miners and clients), most transactions will still be completely valid in both forks.  The exceptions are coin base transactions, p2sh transactions and any down stream transactions from those.  If you've convinced the majority of miners to upgrade before the split occurs (and miners take steps to avoid creating a fork block until some date after it's been confirmed that most miners support it), then miners that have chosen not to upgrade will quickly realize that they risk their block rewards being un-marketable coins.  So, I'm pretty sure they'll quickly update after that point.  People running non mining clients will also quickly follow when they realize mining activity on their fork is quickly dying off (but the vast majority of the transactions that appear valid to them will also be just as valid in the newer fork).  The big risk for people that haven't upgraded is that someone double spends by sending them a plain old transaction after they've already spent those coins via a new style transaction that the old clients don't accept.  But even then it might be difficult for such a transaction to propagate if the vast majority of people have upgraded.

The only practical difference between these is that by restarting evaluation you ensure that all execution context other than stack is cleared.  I think you could have made OP_CODESEPARATOR ensure that everything other than the stack is wiped and have achieved essentially the same objective.  If you have good tests for every opcode that ensure it behaves correctly and leaves all execution context in the correct state, you could breath a little easier about such possible exploits.

Both BIP-16 and BIP-17 have an OP_CHECKSIG in the scriptSig.  It seems you're concerned about whether it's executing in the same context as the rest of the scriptSig or it's run in some other context (i.e. scriptPubKey).  It's seems the concern is the same concern that originally motivated you to split the execution of scriptSig and scriptPubKey.  It feels like an irrational fear, but maybe the implementation of the opcodes has been historically buggy and the fear is warranted.

Well that seems bad (though I can't imagine how it could be actually exploited…other than creating confusion about transaction IDs).  Is there a problem with the scope of what is getting signed?

Unless I'm entirely mistaken there was a rather nasty vulnerability in OP_EVAL caused by this added bit of complexity, one that BIP 16 would've inherited if you hadn't spotted and fixed it. While technically it was only a denial of service vulnerability that prevented nodes that supported it from mining any blocks, a denial of service vulnerability of this kind is enough to create transactions spending other people's bitcoins from their P2SH addresses and get non-upgraded nodes to accept them even after the switch-on date, which is kind of a big deal.

Why the hard dates?
You both are struggling and rushing because the dates you set keep coming again and again, before miners notice and upgrade. Here's an alternate idea:

• Have P2SH* implemented, and announce P2SH support in the coinbase, but it's disabled until a certain condition is met.
• The condition is to have 55% or more blocks in any 2016-block span announcing support for P2SH.
• When that condition is met, the remaining 45% hashing will have two weeks to update.
• Remove the P2SH announcement and just reject blocks with invalid P2SH transactions. Also the changes in the block limits will be made effective here.
• A future version of the software will remove the automatic switch logic.

* With P2SH I'm referring to both BIP 16 and 17. I have no preference for any of them if a soft schedule is chosen.


BIP 16 was made specifically to address this. All concerns expressed in this thread are solved IMHO with a soft schedule with an automatic 55% switchover, as long as clients honors the 6-confirmation convention.

I agree (though I might suggest 64 or 70%…and maybe a month instead of 2 weeks for the activation).  It seems to me that people are compromising design out of an irrational fear of a chain fork.  If you get the overwhelming majority of people to add p2sh support (without actually activating it yet), you've then built a consensus that people want it.  If you then set a date for activation, you give everyone else that hasn't yet upgraded a chance to do so…and they will do it because the consequence of not doing so is that they end up with some unspendable coins in their wallet (it's also worth noting that the majority of coins in old wallets would actually still be usable on both chain forks long after the activation).  After activation, you can then begin the work needed to add support for these transaction in the user interface.

This is an excellent idea. It would give everyone in the network network a longer time to upgrade. Also voting process should be standardised so you include a BIP number e.g. "/BIP_0016/" rather than "/P2SH/" or "CHC" etc.

Yes, my biggest problem with either BIP is the timeframe as well.  I've disabled all of my mining daemons from broadcasting support for either BIP at the moment for my pool.  I would like to see a consensus reached and a sane timeframe proposed before re-enabling support.

Two, three, four weeks is not a sane timeframe for the types of changes proposed to be tested, vetted and deployed.

I set the vote/deadline for BIP 17 soon because I figured Gavin wouldn't accept any delays. If Gavin is willing to tolerate a later schedule for BIP 17, I can update it.

That's non-trivial to implement; it seems to me that a conscious decision by the miners/pools to support or not support is less work and safer.

Luke proposed something similar earlier, though; I'm surprised his patches don't implement it.

I like whoever proposed that the string in the coinbase refer to the BIP, in the future that's the way it should be done.


RE: schedules:

Deadlines, as we've just seen, have a way of focusing attention.  OP_EVAL got, essentially, zero review/testing (aside from my own) until a month before the deadline.

It seems to me one-to-two months is about the right amount of time to get thorough review and testing of this type of backwards-compatible change. Longer deadlines just mean people get busy working on other things and ignore the issue.

So far, I favor BIP 16 because of this point alone.  Eyeballing the client version distributions (http://bitcoinstatus.rowit.co.uk/) suggests that roughly 70% of users are running clients older than 0.5.  We probably can't expect that 70% to upgrade anytime soon.

For Bitcoin businesses, slow propagation is a customer support headache.  In my experience, customers send their payment and expect prompt acknowledgement that it was received even if they still have to wait for confirmations.  The longer the propagation delay, the more customer support emails I get.  I'm guessing propagation delay will slow adoption of BIP 17 transactions among those needing them most: businesses with large balances.

Incidentally, I agree with sentiments expressed elsewhere on this thread that IsStandard() should be replaced with actual resource consumption metrics in the scripting evaluation engine.  It seems fairly straightforward to assign a cost to each opcode and fail any script that hits the resource limits.  That seems both more flexible and more durable than IsStandard().

BIP 16 has the some problem as BIP 17 when sending to them.

IsStandard() is a permanent part of the protocol with BIP 16.

It would be pretty easy to fix this. All you need to do is to modify the networking code (as well as the dnsseeds) to make sure there is connectivity between eveyrone running the new version without needing to pass through old versions in the process. Could someone more experienced at p2p networks than me propose an algorithm?

No, it really isn't.

Here's a possible future implementation of IsStandard():


I like the idea of a future IsStandard() that allows more transaction types, but only if they're under some (sane) resource limits.

Can you elaborate why that's the case?  If true, I think it's very bad.  IsStandard() needs to be lifted at some point (probably when there is a suite of tests around each and every opcode that verify that it does the specified thing and leaves all execution context in a valid state).

Edit: by "lifted" I mean to not restrict scripts to a small set of well known ones…but limits on resource utilization by a script is definitely needed.

Gavin is correct that the actual function named IsStandard() could be removed or replaced. However, with BIP 16, all implementations are required to check for the specific BIP 16 standard transaction and treat it differently.

+1
I'm still with Luke on this one.
While Gavin admits that he would love to make IsStandard more generic (moving from pre-defined scripts to generic resource-constrained ones)
he still leaves handling of multisig as an eternal "special case" for some reason.

On the other hand this "special case" might just be a first step into another way of doing things, and if there are other "special cases" in the future it wouldn't look that odd. What frightens me is that there is no way back if we make this step and it turns out to be a wrong direction.
With Luke's solution we are not making any steps into unknown, we just implement the feature within the current framework.
This will allow us to have multisig as soon as we want it, while giving us more time to think what kind of steps and in what direction we want to make.

Please explain to me how ANY of the proposals (the original OP_EVAL, BIP 16, and BIP 17) are any different in the "what if we change our minds and want to remove support" case?

Removing support for BIP 17 would be harder than removing support for BIP 16, because if the network doesn't support it all BIP 17 transactions can be stolen as soon as they're broadcast.

Imagine there are a bunch of un-redeemed BIP 17 transactions in the block chain and support for BIP 17 goes away.  Every single one of them could be immediately redeemed by anybody.

The situation is better with BIP 16, because of the "half validation" done by old nodes.  Admittedly not a lot better, but it is the "belt and suspenders" nature of BIP 16 that makes me prefer it.

Isn't it still possible for anyone to spend out of a BIP-16 transaction if someone has ever revealed the public key (so, if you ever spend from a BIP16 transaction, you would certainly want to spend out of every transaction funding that address to avoid a chance that someone will spend those other transactions)?  

So, let me see if I can sum up the choices when changing the protocol (regardless of which proposal is adopted):

1) Make things backward compatible in order to avoid a chain split - this creates a risk that coins can be stolen because old nodes/miners aren't performing full validation on the new transactions…to avoid this risk you do what?  wait until 100% of all mining capacity has upgraded before using the new transaction types?

2) Ensure that no nodes (old/new, miner or non miner) allow any transaction that doesn't pass all validation and accept that a chain split could (will) happen - this creates a risk that people who don't upgrade can end up with un-spendable coins in their wallets (a double spend could be executed by creating a transaction accepted in the new chain, but not in the old, then spend those coins again on the old chain).

I'm not sure which is the better approach…just trying to make sense of it.  However, it feels like it's better to ensure coins can't be accidentally spent and that full validation is always performed.  If the super majority of the network is switching, then people will upgrade or patch…that's just part of the deal with bitcoin IMO.  I also question the wisdom of OP_NOP bytecodes.  Seems like it would be smarter to make them cause a script to immediately fail (if that were the case, we wouldn't be having this discussion, we would be talking about a clean implementation and a proper transition of the network).

Also, this whole debate makes me wonder whether it wouldn't be worthwhile factoring out the script validation/execution engine to allow that bit to be independently upgraded from the rest of the bitcoin client.  It might also help with unit testing and verifying a critical bit of code to keep it separated from everything else.

... and with every such proposal, lack of network support means the transactions can all be stolen as soon as someone else tries to redeem them. Not really much better.

Yes, since any of the selected proposals will be cut in stone (in order for multisig transactions of this type to remain spendable) we will have to stick to it indefinitely. What I meant is that by making this brave step into unknown with BIP16 we might be violating some other properties of original Satoshi script which we haven't even thought of. Staying within current framework with BIP17 it is safer to assume that those properties will remain intact.

And as Luke pointed out it is still possible to make this transition later and support second type of multisig transactions or maybe new scripting system all together.

Personally, I'd love to see Haskell code being used for that. It _is_ a rather big dependency though. Haskell code is, to the best of my knowledge, completely sandboxed if you disallow usage of IO Monad. Haskell is a strictly typed pure functional language which gives it some very nice properties, especially when writing code that needs to work just right and have no surprises.

Maybe the future BIP 12/16 replacement can evaluate Haskell? ;)

That's a bad idea for precisely the same reasons as OP_EVAL is a bad idea :P

Actually, it is somewhat better. Even if BIP 16 is one day removed (which would be stupid, so that's incredibly unlikely), the "half validation" provides the same protection as address transactions do if ECC is one day completely compromised. The built-in protection means that the owner of the transaction can still safely spend their funds if:
1) They wait until BIP 16 support is re-enabled.
2) They never broadcast the redeeming transaction themselves until it is in a block (either that they mined themselves, or they have someone trusted mine).

If support for BIP 16 is ever dropped, you'd be correct. However, just like normal addresses, each transaction should use a unique P2SH address. When I realized this, it blew my mind. The more things change, the more they stay the same...

Nope. We just need 50%. Any block containing an invalid P2SH transaction will be seen as invalid to the part of the network that supports P2SH. If that part of the network is over 50% of the hash power, it will eventually overtake the bad chain, keeping old clients connected to the correct chain.

Again, the only way a permanent chain split could happen is if over 50% of the hash power doesn't fully validate P2SH transactions.

A full protocol update would NOT be "clean". Old clients would permanently be stuck on their own network.

There is a problem with this approach.  Due to FUD spread by some people (well, mostly one), many see this as a kind of "vote" unrelated to the actual implementation of BIP 16.  Just put /P2SH/ in the coinbase to "vote".  If miners "vote" without actually validating the pay-to-script-hash transactions, there will be a miscount and chain splits may occur.  This is even more dangerous to BIP 17.  If I understand this correctly, until more than 50% of the hashing power validate BIP 17 transactions correctly, anyone can steal a BIP 17 transaction before it enters a block.  If a large pool "votes" BIP 17 in the coinbase without actually supporting the specification, this may well happen.

While somewhat incidental, I'd like to note that this seems to me to be a very strong, pragmatic argument in favor of BIP 16.

From a theoretical perspective I also feel that BIP 16 is better. If the goal is to store code differently, it is best to handle this before execution by a preprocessor and not via a special opcode that changes execution flow in non-trivial ways. I have a very easy time implementing and feeling comfortable with BIP 16 as it only affects the way scripts are stored/loaded and not how they are executed. There are no new opcodes which can be used in unexpected places or unexpected combinations and there is no change to EvalScript, which is an argument that anyone who has actually tried to implement this function securely has to give a lot of weight to.

I quoted your post Stefan in the new thread under general Bitcoin discussion here:
https://bitcointalk.org/index.php?topic=61125.msg714569#msg714569
It seams more heated and there are more participants now.

Thanks, much appreciated. :)

I think it comes down to this:
Do people want to introduce something new to the bitcoin scripting system or use a hack and keep the old one.
Gavin's main concerns seem to be keeping old clients functional.
My preference is to introduce a new opcode. But i do not feel that bip17 is mature enough since it introduces new security risks.
So i think something like bip17 should be implemented, but in a way that cannot be exploited/misenterpreted by older clients even if BIPXX has less than 50% support.

edit... oops, posted in the wrong thread

BIP 17 does not introduce any new security risks that aren't also risks with BIP 16.

it does. if you send a new transaction before 50% of the hash power is updated it will be redeemable by anyone
with bip16, you will at least need to know the script - its not much, but its not nothing either.

I think the whole "how to do updates" issue should be discussed on general terms, and not just in the context of these BIPs. Its not the last major change you will do to bitcoin.
Currently there are n't even any update notifications - so people have to manually check the website/forum every month or so...

And to redeem it, you have to make the script known even before your redemption gets confirmed. So any rogue miner/relayer can easily swipe it the moment you try to spend.

still better than nothing

Right, so it's clear that with either BIP-16 or BIP-17, it's not a prudent thing to use the new transactions until we're well beyond 50% support.  But you have to admit that with BIP-17, it's a hole you could drive a truck through while with BIP-16 it's more like threading a needle (you'd have to get really lucky that you saw a transaction well before the rest of the network saw it or you'd need control over a substantial amount of hashing power (yet another argument against large pools).

My bigger concern is Gavin's concern about the state of the script implementation code (how well it's understood, how well it's tested).  I think everyone is largely in agreement about how transactions should work in ideal circumstances (at least, I pretty much agree with Gavin's other post about it).  But it's prudent to consider the risk of the unknowns in code that people don't seem to fully understand or trust to be free of potentially devastating bugs.

you dont need computing power, you just send a competing transaction, and wichever gets first into a block will be accepted

BTW, why is the address of BIP17 longer than BIP16?

BIP17 and BIP16 both use BIP13 addresses...

is there already an implementation of both BIPs?

BIP 17 could gain the same robustness by the following (in python-like pseudocode):

Where MetaEvaluate is a function that:

I'm not going to write pseudocode for code_split but the first argument is the code to be split and the second is the instruction by which to separate it. I'm not sure if it makes sense to be able to split at any instruction but OP_CODESEPARATOR though. code_split does need to understand not to split in the middle of data though, so pure split won't work.

Yes, BIP 16 is in mainline git master, and BIP 17 has implementations for 0.3.19 onward (https://en.bitcoin.it/wiki/BIP_0017#Reference_Implementation).

Interesting, but it seems more likely to break something subtle to me, at least in regard to backward compatibility. But perhaps this option should be explored more if we have the time.

On the contrary, you don't have to make the script known publicly to redeem it. You can mine it yourself, or get someone that you trust completely to mine it for you. Once it's in a block, the only danger is that the block could be orphaned, and since the script is now publicly released, other people could try redeeming it.

However, that is significantly better than BIP 17 where anyone can try redeeming it the second the BIP 17 transaction appears on the network.

Not considering both BIPs require a supermajority hashpower of support to become active. ;)

At the moment I slightly prefer BIP 16 over 17. BIP 16 seems a little more security-conservative because the scripting system isn't being modified and non-P2SH scripts will not be touched at all. There's no chance that anyone will actually want to use "OP_HASH160 [20-byte-hash-value] OP_EQUAL" as their entire scriptPubKey unless they're using P2SH, so I don't believe that BIP 16 undermines Script.

I find P2SH's use of special-case-scripts to be inelegant, though Script is already inconsistent and inelegant, so I don't care that much. Hopefully when the max block size is increased, Script will be totally revamped to fix all of the imperfections.

I'm amazed at how much argument there is over this tiny issue. BIP 16 and 17 have few real differences (and these differences are very technical), but it seems like everyone on this forum has some strongly-held opinion about it.

The differences seem small and I'm confident that technical issues are not what is causing the controversy.
Of course, I would have liked OP_EVAL to succeed as I thought it led Bitcoin in a direction I thought would be healthy in the long run.

The current arguments bode ill for getting a vague consensus for more profound changes probably required in future. Imagine Satoshi trying to deploy IsStandard() if he had to deal with the current situation.

Part of the problem is that the pool operators are vital to the success of upgrades and they only really care about mining and the exchange rate!

ByteCoin

http://blockchain.info/p2sh

I'm disapointed.

Me too. Too many votes for BIP16

I don't think it bodes ill at all.  I think, in the future, if something like this is handled properly as a roll out, things will go very well.  This entire thing was botched from the beginning as far as the rollout goes and then quickly devolved into this mess. 

I also highly disagree that the pool operators only care about mining and the exchange rate.  Speaking personally, I care about what's best for the miners and the stability of the pool.  For my part, my objection over this is the fact that the timeframe was so short to make changes/deployment, after evaluating what, exactly, is being proposed.

Oh, slush just started to support P2SH, it'll be more.

But surely P2SH will be delayed, again, because of you. Nice Work, Luke.

We all learn from our mistakes.

Would you be willing to write an "informational BIP" describing how to do a rollout right in the future?

I'm terrible at that kind of "first we'll need to get buy-in from groups X, Y, and Z by doing A, B, and C, then each of those groups will elect representatives who will have three days to agree to a schedule, which will be announced blah blah blah blah"

I'll have to review the BIP format and requirements, but I can probably put something together.  It's based on the Python development RFC format or something, yeah?

Yes, BIPs are based on PIPs and BEPs (for Python and BitTorrent).  See BIP 0001 for the process.

i think it would had a better success as BRFC (Bitcoin Request For Comments)

The only unnecessary delay at this point is from people trying to push BIP 16 through when we have a sane alternative. If everyone was focussed on BIP 17, we'd be there in no time. ;)

My assessment is that the current state of disagreement is a rather freakish conspiracy of circumstances and would have been impossible to predict and/or avoid. Please outline how it was botched. Here's a little history as I see it:

In the beginning there was casascius' proposal (https://bitcointalk.org/index.php?topic=45211.msg538756#msg538756) which virtually nobody bothered to understand. This rapidly transformed into a proposal to "reinterpret" OP_CHECKSIG with some code to recognize legacy cases to ensure back compatibility. This was rejected because the special case interpretation was seen as a hack and destroyed the "purity" of the scripting language - rather ironic in light of the current hacky solutions. Gavin then gathered fairly broad support around my OP_EVAL proposal and everything was going smoothly for a few months until roconnor finds some bugs and some low-to-zero-practical-impact disadvantages. Gavin lobotomizes the scheme in an ugly fashion to allay these concerns, probably assuming that the adoption of the more cautious scheme will be uncontroversial. With reasonable justification but with little regard for propriety or the long-term consequences, Luke-Jr objects to Gavin's scheme's violation of the script and uses his considerable technical and social capital to engineer the scheme's failure. This brings us up-to-date.

To apportion the blame fairly, it should be mentioned that roconnor was threatening to spread FUD about the "dangers" of OP_EVAL if his demands were not met, regardless of the incidental damage to Bitcoin.

ByteCoin

So which BIP are you voting for?  ;)

You forgot to mention "engineering its failure" includes "engineering a better, sane proposal"... BIP 17

I left out the outline you already gave, as I have no objective assessment as to it's accuracy (or inaccuracy) - the part I would add, that I would consider the important botched part:

BIP 16 proposed on 3 Jan 12, deadline for changes to bitcoind for miners (realistically, the big pools) to be 1 Feb 12.  Giving less than 4 weeks to implement changes, test, deploy and mine.  No direct contact of the major players making up 55%+ of the mined blocks, with the possible exception of Slush(?) and/or Tycho (?) in "secret." 

The time frame is too short, and the lack of communication as to the ramifications/reasons to the people who mine the majority of the blocks is the problem.  If either of those two situations had been handled, the other one probably would have sorted itself out, but since neither issues was handled properly, the entire operation was botched.

Just FYI: I contacted the top ten mining pools (as listed in the stickied threads in the Mining Pools subforum) directly via email way back in October, copied the email to them in the Mining Pools forum, and kept them 'in the loop' on all of this.

I probably should have posted all of my followup emails to them in the Mining Pools forum, also; as I said, I'm terrible at that kind of "keep track of who you've told what and make sure you've got 'buy-in' from X Y and Z" thing.

I think my only really major blunder was being too stubborn about inserting some wording into BIP 16 that Luke wanted inserted; I let my emotions cloud my judgement.

This was about OP_EVAL (BIP 12). All communications about BIP 16, if any, were kept secret until I brought it into public light... I don't recall getting any myself, despite running one of the top 10 pools.

gavin, do you have any commercial interest in implementing P2SH?

somebody suggested that he had contacted a commercial company for making HW keys for bitcoin wallets - similar to what mtgox has.
to use such a device with bitcoin P2SH might be very helpful - 1 key on the PC , 1 key on the device

Even if Gavin does have a monetary incentive to implement P2SH, that doesn't mean he will only support BIP 16. It just means that he will want the code written quickly.  If he wants a company behind it, that is just more incentive to make sure that P2SH works properly.

The WTF image was posted because your question has little relevance, not because of some misunderstanding.

one of the reasons bip17 is not well accepted is because it will take more time to implement and test - delaying the introduction of P2SH.
I just want to make sure the reasons to opposing BIP17 are not commercial (aka a set release date for a commercial product).

Any bip16/17 thread gets off topic quite fast :\
if anyone wanted to know the details - they are posted in the wiki
I cant see any chance of an objective and/or productive discussion of this topic on the forums...

BIP 17 is already implemented, and easily tested. The delays at this point are because people won't let BIP 16 go.

It does seem like giving in to a guy with such one-sided interpretations of events is like giving into a terrorist. Since he is quite willing to approach controversies from an extremist and catastrophic point of view, there is no way to establish bargaining power with him unless you hold your ground.

quid pro quo won't work

I don't have an opinion this, but I have some related questions.

1) Where is MAX_SIGOPS validated? In isStandard(), right?

I think it should be there. I'm ok with having that function for sanity, but I think the door should be open to stop using it in the future, to leave it out of the protocol (although maybe each implementation runs its own version of the function, depending on its performance variables). Not sure how each proposal affects that part I just read something about it somewhere in these discussions.

2) Probably there's a good reason but...Why a scripting language is needed in the first place?
Why don't just a set of validations that must be all true, operators with parameters and references to other elements in the script?

Standard Transaction to Bitcoin address


Multi-signature


another valid scriptSig

Wouldn't something like this be more secure and achieve the same things?
Of course, this would not be feasible for compatibility issues, I just want to know the reasons behind the stack language.

EDIT
Sorry, I just found this thread:
https://bitcointalk.org/index.php?topic=61248.0

why not just add another field into a transaction at the output and implement the "ideal" 3 parts solution gavin sugested?
Is there a way to make old clients ignore an unknown field?
 

We could add a hashScriptPubKey field and let scriptPubKey be presented at redemption. My question remains.
Why do we need a stack based language?

3) Also, the scriptPubKey it's going to end up in the chain anyway. Why the moment matters so much?


As Gavin explains, it's not possible for backwards compatibility.
I'm not making a proposal, just trying to understand bitcoin better.
I'm not directly involved in the bitcoin development and I don't think my voice counts much in this purely technical controversy.
I mean, if both proposals end up being equally secure, for the end user, both do the same thing.

If you are asking about the public key, then usually it appears in the blockchain only when it doesn't matters anymore (for one-time addresses).
So if someone can "break" the signature using only pubkey, he won't be dangerous because if pubkey is used for redeeming then that address is empty already.

I found better answers for the third question:



The other questions are still open.

As someone who only understands the Bitcoin algorithm on a high level, I want to get a better understanding of what multi-sig is for.

My impression is that the feature involves the creation of an address with two private keys, each generated by a party who doesn't have to know the other one. In other words, an address where both key-generators would have to be compromised for the coins at that address to be compromised.

My impression was also that ECDSA already has that feature, based on some kind of magic where you multiply Party 1's private key and Party 2's public key (and vice versa). I remember Mike Caldwell (the Casascius guy) talking about this method with whoever runs Printbills as a possible means of adding confidence in the security of the Bitcoins backing a piece of physical currency.

Was one of these impressions incorrect? Does that magic of multiplying ECDSA keys not work really, or is there something else that these BIPs do that justifies a change in the protocol?

As opposed to what?  A register based language?  Note that what bitcoin implements is just a bytecode interpreter, not a language.  You could chose to implement a more user friendly language that compiles to bitcoin bytecode if you chose to, but the benefits would be marginal at best given that the scripts are very small.  It's not like you have to wade through pages and pages of the stuff when talking about your typical script.  People have already invented a defacto language when talking about bitcoin scripts (using "OP_CHECKSIG" instead of its bytecode…or {} to denote stuff pushed on the stack).

That bytecode is also a formal language. I'm not talking about performance improvements, so it doesn't matter much if the interpreter interprets bytecodes or strings containing operators names such as "OP_CHECKSIG". Both would be equivalent in what they can do (but the second notably worse for storage).
My point is, why have such a potent language?
Why not just a list of bytecodes that express conditions to be validated?
Separating from the beginning the operator scripts from the data?
You have a list of data and a stack of operators that can refer to that data as variables or use constants for the parameters of the operators.
You wouldn't have flow control or stack words, for example.
I guess the reason is to allow use cases that aren't thought of yet, but new words can be added later, as is the case for p2sh.
Well, kind of answered myself now: a more potent language needs to be extended less times and extending the language is really an issue with a blockchain to agree.

Right…and with p2sh, the impetus is really on the receiver of coins to ensure whatever script they come up with is free from any defects that might allow unauthorized people to spend their coins.  Which is how it should be.  Someone could create a script that allowed anyone to spend the coins, but that's an issue for the script author, not bitcoin itself.  If anything, scripts should be allowed to access even more data related to a transaction or the block chain.  For instance, your script might check whether the input transaction has at least N confirmations.  Or it might want to check that a certain percentage of the input amount goes to certain output addresses (you can imagine this might be useful for a payment processor like bit-pay that doesn't want to ever have control over the full amount of a transaction, but instead just redirect a percentage of the value of the transaction).

I have a small question.
If two private keys always need to be kept separate,
how one would generate a single address which would combine both of them without having both of them on a single device at one point?

for the address you only need the public keys and script - you can store those on the PC

for the redeeming transaction you need the signature, public key and script
you need the private key to make a signature
so:
PC creates <sig1> using key1
phone creates <sig2> using key2
phone sends <sig2> (not the key) to PC
the sig is valid only for this transaction, and it must belong to the public key in the address, this will end up being in the transaction anyway, so there are no "secrets" transmitted over the internet
PC puts everything together and creates a redeeming transaction

Ok that explains it, thanks!