BIP 16 analysis from a miner's point of view

[paraipan]

Hey guys just found something interesting on BIP 16, i opened another thread no to get off-topic this one https://bitcointalk.org/index.php?topic=61125
I should state that i'm not biased in any way pro/against this BIP the only thing is the way it was introduced to the community opened my appetite for researching it a bit more.

Going backwards on the updates (commits) of the bitcoin source code i found this reference

https://github.com/bitcoin/bitcoin/commit/922e8e2929a2e78270868385aa46f96002fbcff3

so thinking a bit, Gavin stated in some threads that he's been pushing this change for a few months now, but that could only be true with the scraped OP_EVAL because p2sh change is authored January 05, 2012. So when did bip 16 appeared first time ?
The BIP states 03-01-2012, https://en.bitcoin.it/wiki/BIP_0016, with a very small deadline to "vote" for it, Feb 1, 2012, and a two week time to implement, without thoroughly testing it, on Feb 15. That deadline was familiar... yes, you guessed it, it was the OP_EVAL's

https://en.bitcoin.it/wiki/BIP_0012
https://github.com/bitcoin/bitcoin/commit/a0871afb2b1d6d358c833fd08bca2f13c840fd4d

Meh, this doesn't sound quite right in a project like this. So i digg'ed more. Looking at BIP 16 page history on the wiki i saw that the original proposal was done by user "2112" on Dec 18, 2011 and scraped by Genjix then put in it's actual form by Gavin on Jan 4, 2012.

https://en.bitcoin.it/w/index.php?title=BIP_0016&action=history

Couldn't help myself to look at the original proposal and find a reason why it was changed in such a manner

https://en.bitcoin.it/w/index.php?title=BIP_0016&oldid=21016

Some citing:

The proposed changes are far-reaching and as such are not suitable for immediate implementation. They are so extensive that it is certain that a complete reimplementation will be required

Self explanatory

The centerpiece of this proposal is the idea of “digital prospectus”: a program whose main functionality is to do perform a verification of the submitted blocks and transactions. This program will be cryptographically hashed and will become a “root prospectus hash”

At first i didn't get it but then it hit me, remembered what some guys said about bypassing the present scripting system by putting only the hash of the script and doing the actual checking in memory (stack) by executing the script code there.

The implementation will provide a means of recording the “digital prospectus amendments” which in effect will patch the original prospectus.

So someone would be able to do "amendments" on those scripts running on bitcoin clients.

In other words it would change the Bitcoin government from the democracy to the republic. The last but not least change allowed by the existence of the “digital prospectus” will be the change in scripting engine.

Man, this doesn't sound right...

If the prospectus writer decides to allow general scripting with looping she can include in the prospectus a relatively simple theorem

Now i felt shivers on my spine

Another benefit of using LISP (or any similar language) for scripting lies in its transformability. There exist a body of research of ultra-reliable computing that used “SIMD-like” and/or “Hamming distance 3 or higher” coding for error detection and correction.

Meh, so the script could be checking up it's code too

tl;dr

In summary this proposal encompasses three main changes: (1) explicit cryptographically signed and software-executable contract included in the root block, (2) cooperative DHT-based networking protocol that does away with IRC, dedicated ports and 4-byte identifiers, (3) general prefix script notation backed by strong syntax and semantic checkers. Because of this proposal is very far-reaching I suggest that it will be immediately placed in the dormant state. Initially we can work on clarifying its wording, but the full implementation will require a lot of discussion and research

(1) Arbitrary code that uses as support p2sh transactions
(2) Network communication between those small scripts
(3) Self aware of it's code

woot, dunno what to say

edit: cut off the part about skynet

[theymos]

That stuff is totally unrelated to BIP 16. It was removed because 2112 was using that BIP number without having it assigned to him.

[Costia]

That stuff is totally unrelated to BIP 16. It was removed because 2112 was using that BIP number without having it assigned to him.

its a consipracy! where is my tin foil hat 

[paraipan]

That stuff is totally unrelated to BIP 16. It was removed because 2112 was using that BIP number without having it assigned to him.

dunno, so all my post is mot ? the 2112's paper resembles actual bip 16 implementation, a script hash that always verifies, being backward compatible and the new clients actually running the scripts.

[paraipan]

That stuff is totally unrelated to BIP 16. It was removed because 2112 was using that BIP number without having it assigned to him.

its a consipracy! where is my tin foil hat 

i should put mine right away 

[genjix]

The reason I closed it down was because Gavin works for the CIA, and I wanted to shut down all the dissenters. The BIP process is so that we can scheme together, make it BIP law then force everyone to comply.

Occam's razor.

[paraipan]

The reason I closed it down was because Gavin works for the CIA, and I wanted to shut down all the dissenters. The BIP process is so that we can scheme together, make it BIP law then force everyone to comply.

Occam's razor.

great, finally someone to tell the truth

[owowo]

The reason I closed it down was because Gavin works for the CIA, and I wanted to shut down all the dissenters. The BIP process is so that we can scheme together, make it BIP law then force everyone to comply.

Occam's razor.

great, finally someone to tell the truth

Yeah, what's the CIA's agenda with Bitcoin? Let me guess: "no need to know".

I really don't like the discussion about some "Commitees" or letting some small group of ppl decide about this over here: https://bitcointalk.org/index.php?topic=61922.0;all.

I think major changes should be tested thoroughly and not rushed.
 

[paraipan]

The reason I closed it down was because Gavin works for the CIA, and I wanted to shut down all the dissenters. The BIP process is so that we can scheme together, make it BIP law then force everyone to comply.

Occam's razor.

great, finally someone to tell the truth

Yeah, what's the CIA's agenda with Bitcoin? Let me guess: "no need to know".

I really don't like the discussion about some "Commitees" or letting some small group of ppl decide about this over here: https://bitcointalk.org/index.php?topic=61922.0;all.

I think major changes should be tested thoroughly and not rushed.
 

+1    

meh this will clear out in the end and every one will know better it's place, developers write programs, miners make them work, so users can use them without hassle

[2112]

That stuff is totally unrelated to BIP 16. It was removed because 2112 was using that BIP number without having it assigned to him.

Actually genjix himself assigned 0016 for that purpose.

My PM to genjix:

BIP registration
« Sent to: genjix on: December 16, 2011, 12:39:31 AM »
Quote  Reply  Delete 
Hi there!

Would you please allocate an official BIP number for my proposal that I tentatively called BIP 2112?

https://bitcointalk.org/index.php?topic=54382.0

This is very forward-looking proposal, not something for an immediate implementation. I think it is somewhat important to publicize it early from the intellectual property ownership point of view. With your help I'm hoping it will stay in the public domain.

Also may I suggest that you create some discontinuity in the proposal numbers? If not the discontinuity in numbers then at least a separate table on the wiki page for the far-out proposals that will require significant re-implementation effort. I think it would be helpful to somehow distinguish between short-term and long-term proposals. They will have very different readership with disparate demands for writing style and documentation.

Thanks.

genjix's reply:

genjix
Hero Member

Posts: 1455

Re: BIP registration
« Sent to: 2112 on: December 16, 2011, 10:27:36 AM »
Quote  Reply  Delete 
Hey,

Not checking here often. My email is genjix@riseup.net

Are you on IRC? I'm genjix on Freenode

I can give you BIP 0016, otherwise why do you want discontinuity in the numbers? Python only did it for where they have moved from Python 2 -> 3

http://www.python.org/dev/peps/

Because it was a huge breaking change and a fork in development (to avoid confusion on their part).

OK, so AFAIK your proposal is for a standard for a program that validates blocks. This program is how the standard validation is accepted or rejected.

For BIPs you can explicitly state:

== Copyright ==

This document has been placed in the public domain.

Otherwise it is implicitly licensed under the openpub license. However BIPs need an Author (so people can contact you wish new suggestions far in the future, or if they're working on a new standard).

Just give this a quick skim, https://en.bitcoin.it/wiki/BIP_0001

Add your proposal to this page https://en.bitcoin.it/wiki/BIP_0016

Keeping the same format as the other BIPs. Then let me know and we'll go through it. I can suggest ways to reword things, or if things are difficult to understand.

Then you can email the mailing list with the proposal. During the discussion, people will suggest things and you should make changes and clarify things. At the end a consensus may begin to emerge about whether to accept or reject the proposal.

Just to settle the score.

[gmaxwell]

That stuff is totally unrelated to BIP 16. It was removed because 2112 was using that BIP number without having it assigned to him.

dunno, so all my post is mot ? the 2112's paper resembles actual bip 16 implementation, a script hash that always verifies, being backward compatible and the new clients actually running the scripts.

Since no one has given you a serious reply—  No, this is completely unrelated.

At the level of similarity you're perceiving there the already existing bitcoin system also fits.   BIP16/BIP17 don't change the behavior of the scripting system except in changing _when_ the script requirements for the disposition of a transaction are provided. With P2SH you commit to your requirements when you pay (by giving a hash of them) but they're provided by the spending transaction.

It doesn't change the scripting language, make it more powerful or turing complete.  BIP12 did increase the expressive power some by adding limited recursion, but no one wants to go that way now.

[paraipan]

That stuff is totally unrelated to BIP 16. It was removed because 2112 was using that BIP number without having it assigned to him.

dunno, so all my post is mot ? the 2112's paper resembles actual bip 16 implementation, a script hash that always verifies, being backward compatible and the new clients actually running the scripts.

Since no one has given you a serious reply—  No, this is completely unrelated.

At the level of similarity you're perceiving there the already existing bitcoin system also fits.   BIP16/BIP17 don't change the behavior of the scripting system except in changing _when_ the script requirements for the disposition of a transaction are provided. With P2SH you commit to your requirements when you pay (by giving a hash of them) but they're provided by the spending transaction.

It doesn't change the scripting language, make it more powerful or turing complete.  BIP12 did increase the expressive power some by adding limited recursion, but no one wants to go that way now.

Thanks for being you the one to try give a "serious reply".

The way i understand it BIP16 changes completely the scripting system for the p2sh transactions. How ? bypassing it. Seen lot of code examples already. BIP17 uses an existent placeholder OP code to fit it's needs and another one that seems to have been put there by Satoshi himself.

[Gavin Andresen]

The way i understand it BIP16 changes completely the scripting system for the p2sh transactions. How ? bypassing it. Seen lot of code examples already. BIP17 uses an existent placeholder OP code to fit it's needs and another one that seems to have been put there by Satoshi himself.

You understand wrong.

BIP 16 recognizes a certain pattern of bytes, and says "If you see those bytes, then the script will be provided by the person who has the coins instead of the sender (the sender just provides a secure hash of that script)."

The script that is provided is then used EXACTLY as if it was a "normal" sender-provides-the-script transaction.

BIP16 is the most conservative possible solution to the problem we want to solve.

[gmaxwell]

Thanks for being you the one to try give a "serious reply".

The way i understand it BIP16 changes completely the scripting system for the p2sh transactions. How ? bypassing it. Seen lot of code examples already. BIP17 uses an existent placeholder OP code to fit it's needs and another one that seems to have been put there by Satoshi himself.

It's not possible to bypass the scripting system while remaining compatible with bitcoin.

BIP16 uses the script "OP_HASH160 [20-byte-hash-value] OP_EQUAL"

BIP17 uses the script " [20-byte-hash-value] OP_CHECKHASHVERIFY OP_DROP"  but OP_CHECKHASHVERIFY isn't something that existed already, it's something that Luke made up for the purpose of BIP17.  In order to make it compatible it's overlaid on top of OP_NOP2.

From the perspective of prior software nodes BIP16 transactions say "Check that the data the redeemer is providing hashes to this value from the address, if so then allow redemption" and BIP17 transactions say "allow redemption" (no condition).

From the perspective of new nodes both say "Check that the provided bitcoin script hashes to the value from the address, then execute it".   The execution is implicit in both cases, the BIP16 style uses an existing way of saying hash some data and check it (which is why old nodes are able to do that much validation) but gives it special meaning for this case, the BIP17 style introduces a completely novel way of checking hashes which is ignored by old nodes in order to make the implied execution special to just the new opcode it introduces.

[paraipan]

Interesting. I don't understand completely but i'm making efforts

So the most secure and conservative way to implement the feature would be BIP16, a little more error prone too, given the amount of coding work that was needed, but nothing that can't be managed.

My understanding was that OP_NOP codes were left there exactly for these purposes that's why i thought you would be using them.

[Steve]

The thing that bothered me most about BIP16 is that the execution of the script in scriptSig is triggered by recognition of a special pattern of opcodes in the scirptPubKey.  BIP17 seemed cleaner in that it adds a new opcode that essentially says "compute the hash of the code following the most recent OP_CODESEPARATOR and verify it."  Execution of this code happens because it's simply in the normal flow of opcodes.  From the perspective of what these BIPs enable, both of them are workable.  But Gavin's concern should give us all concern.  He's earned enough credibility that you shouldn't really disagree with his position unless you take the time to really study the current script engine as well as the history of how it evolved to where it is today.  And my superficial understanding is that BIP16 is less of a departure from the historic conventions in terms of how scripts get executed and what types of opcodes have been traditionally used in the scriptSig. 

BIP16 might be overly conservative, but at the same time its hard to argue against being overly conservative when the objections to BIP16 are largely just a matter of aesthetics.

[gmaxwell]

Interesting. I don't understand completely but i'm making efforts

So the most secure and conservative way to implement the feature would be BIP16, a little more error prone too, given the amount of coding work that was needed, but nothing that can't be managed.

My understanding was that OP_NOP codes were left there exactly for these purposes that's why i thought you would be using them.

BIP17 advocates (mostly Luke) argue that BIP17 is more conservative, because the change in behavior is conceptually more clean and less "special".  BIP16 advocates counter that the BIP16 change is more focused and narrow (And point out that old nodes still do at least superficial validation of BIP16, and other minor technical details) and that this makes it more conservative. BIP17 points out that its patches are somewhat smaller, BIP16 could counter that thats because BIP17 leaves some minor technical shortcomings open.  I don't think there is an objective way to settle that, they're all earnest perspectives.

During the discussion that was to become BIP16 I suggested that we satisify Luke by simply sticking a completely pointless OP_NOP at the end.. an OP_MAKEITSO, if you will:

09:35 < gmaxwell> sipa: okay, so lets end this argument and add a freeking op_code for this for luke.
[...]
09:36 < gavinandresen> I don't like adding an extra byte to make luke-jr feel better.

But doing this would perpetually bloat the blockchain a bit just so that Luke (and now a few others) would feel better about the cleanliness of the solution.  It's a real cost that will cost bitcoin users real money now and in the future and will contribute to the loss of decentralization as bitcoin grows.

I think Gavin is also rightly concerned about placating Luke for the sake of placating Luke. Luke's argument style is a bit like terrorism: He's a hard working contributor, and sometimes easy to compromise with, but sometimes when things don't go his way he just keeps turning up the rhetoric until everyone else burns out or folds.

Ignoring the bloat issue, it's good to avoid burning up NOPs:  We only have 10 of them, 9 really because the first one has been randomly used in toy transactions, when people really weren't thinking about what the NOPs were actually there for. Because they are our only method for adding explicit behavior in a backwards compatible way we must steward them carefully.

Luke's BIP17 avoids creating that bloat by making his new OPcode also do what two of our existing opcodes (HASH160, OP_EQUAL) do in addition to the MAKEITSO part.  The primary cost of this is that his transactions look like pay to anyone, rather than pay to hash-secret to old nodes.

In discussion Luke also indicated that he'd find BIP16 more acceptable if it was written so that BIP16 style transaction were expressed in the documentation just as a series of special bytes, not as a script and if old style transactions were deprecated (at least for new usage). His opinion there is that doing this would just make BIP16 the new way of doing things, not a special case (but rather the old outputs would be the special case).  I think Gavin made a political error by not just going along with that.   (Luke would point out that he's glad Gavin didn't because it caused him to propose BIP17 which he believes to be better than BIP16 with the "right" description).

[kjj]

During the discussion that was to become BIP16 I suggested that we satisify Luke by simply sticking a completely pointless OP_NOP at the end.. an OP_MAKEITSO, if you will:

09:35 < gmaxwell> sipa: okay, so lets end this argument and add a freeking op_code for this for luke.
[...]
09:36 < gavinandresen> I don't like adding an extra byte to make luke-jr feel better.

But doing this would perpetually bloat the blockchain a bit just so that Luke (and now a few others) would feel better about the cleanliness of the solution.  It's a real cost that will cost bitcoin users real money now and in the future and will contribute to the loss of decentralization as bitcoin grows.

Making an OP_P2SH really seems like the best option.  The objection that BIP16 creates a special case isn't confined to just Luke; plenty of other people also see that as a valid concern.  One name that stands out in my memory because I specifically asked him about it is Tycho.

Of course, Gavin also has objections to OP_P2SH, and while I don't think those objections are fatal (see my reply), I'm not the one doing the work so my opinion doesn't mean shit.

In particular, I think that we could define OP_P2SH in a way that has very strict limits today, but the ability to be used in a more flexible way tomorrow if we determine that it is safe to do so.

For example, we could require that it appear no more than once, and if present it must be the first (or last) opcode in a script and reject any transaction that includes it in any other position, or even reject any transaction where the script doesn't exactly match the existing BIP16 template.  This would be pretty simple to add to the existing BIP16 code.

But then down the road, we could gradually relax things as needed.

Say you want to make a transaction that can be redeemed by either of two different P2SH payloads.  You rewrite the first stage script to include two hashes and verify that the script matches at least one of them.  OP_P2SH still must be first, but now there are two possible templates.

Down the road, someone wants to write a conditional script that can be redeemed either with a key or a P2SH payload.  You allow OP_P2SH to be found elsewhere in the script, maybe even inside of a OP_IF block.

What I'm trying to get at is that OP_P2SH could be added today, which would ease some valid concerns, and for relatively low cost, and in a way that allows us room to change in the future.

And for what it's worth, I'm a strong supporter of P2SH, even without OP_P2SH.  My miners are currently set to only use pools that either support or intend to support P2SH very soon and my experimental p2pool deployment tool defaults to supporting it.  I trust Gavin's judgment on this, my only real disagreement is over how to sell it to the rest of the community.