Bitcoin Forum

Bitcoin => Bitcoin Discussion => Topic started by: genjix on January 29, 2012, 03:54:08 AM



Title: The Truth behind BIP 16 and 17 (important read)
Post by: genjix on January 29, 2012, 03:54:08 AM
http://bitcoinmedia.com/the-truth-behind-bip-16-and-17/

Now I am not usually posting (spamming) Bitcoin Media stories on the forum, but I feel this is an important read. All the posts I've seen thus far are by partisan supporters of either scheme with no objectivity. They've also been highly technical in their language whereas here I've put mucho effort to make it readable and understandable for non-bitcoin developers.

Other developers disagree with giving this information away and feel like you as users should trust their judgement. I strongly disagree. I'd rather people have a say in such fundamental matters such as this, even if it makes the developer's lives harder because they have to explain their decisions thoroughly.

My worry is bitcoin someday becomes corrupted. Developers: see this extra scrutiny as an opportunity to build a culture of openness. It is not at all bad.

EDIT:

http://bitcoinmedia.com/cathartic-progress/

Michael Marquardt (theymos) suggests compiling a list of everyone intimate with the bitcoin protocol to invite to a two-week email discussion. After those two-weeks a vote is taken. It will be the job of the champions of each idea (BIP 16, BIP 17 and no change) to win over the committee into supporting them. If an idea has necessary support, bitcoin clients will be programmed to apply the new rules for 3 months in the future.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Phinnaeus Gage on January 29, 2012, 04:49:26 AM
http://bitcoinmedia.com/the-truth-behind-bip-16-and-17/

Now I am not usually posting (spamming) Bitcoin Media stories on the forum, but I feel this is an important read. All the posts I've seen thus far are by partisan supporters of either scheme with no objectivity. They've also been highly technical in their language whereas here I've put mucho effort to make it readable and understandable for non-bitcoin developers.

Other developers disagree with giving this information away and feel like you as users should trust their judgement. I strongly disagree. I'd rather people have a say in such fundamental matters such as this, even if it makes the developer's lives harder because they have to explain their decisions thoroughly.

My worry is bitcoin someday becomes corrupted. Developers: see this extra scrutiny as an opportunity to build a culture of openness. It is not at all bad.


WoW! Good read, genjix, albeit I didn't understand half of it. But that's not important. What's important is that you've taken the time to present this in a different light. One which may open eyes as to what's at stack. I wish I was able to supply input into this important issue, but I can't. Not because of any side I like better than another, but because of my ignorance on this topic. Again, thank you, genjix, for taking the incentive in re-presenting this.

~Bruno~


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: stcupp on January 29, 2012, 05:08:06 AM
+1


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: JusticeForYou on January 29, 2012, 05:16:37 AM
+1


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: cbeast on January 29, 2012, 05:23:40 AM
I propose a voluntary shutdown of exchanges during the implementation of such a major patch until stabiity is fully tested. A bank holiday, so to speak.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Ranvier on January 29, 2012, 05:36:45 AM
+1 @Phinnaeus


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Maged on January 29, 2012, 05:52:04 AM
Quote
A minor issue is that old clients will not propagate CHV payments. The old clients will accept blocks with CHV payments, but they will not pass them onto other nodes in the network. This is not a problem as transaction propagation speed is usually far quicker than confirmation speed (especially with 6 confirms or more).
Huh? Because hash power percentage is significantly different from the percentage of Bitcoin network nodes, this is a huge point. Until the relay nodes upgrade, it can take a well-connected node up to 24 hours to get a transaction to a miner that would accept it. At this point, BIP 16 has a huge advantage: although it will take that long to get the transaction to a BIP 16-based address, once it's there, it can be spent just as quickly as a normal address-based transaction. BIP-17 transactions have to wait on both ends. Hope you aren't buying anything with Bit-Pay... (they only wait 15 minutes for the transaction)


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: paraipan on January 29, 2012, 06:44:50 AM
+1 thanks for explaining this dude, now i feel i can make a more educated voting  :)


My view:

                         BIP        | 16 | 17 |
20-byte short address        |  x  |  x |
unexpected behavior         |  x  |  -  |
harder to manage             |  x  |  -  |
max. number of SIGOPS    |  x  |  -  |
uses an existing operation  |  -  |  x  |
(to define a subscript)
uses a new operation        |  -  |  x  |
(script’s actions)
need more than 55% sup.  |  x  |  x  |
backward compatible        |  x  |  x  |


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: chsados on January 29, 2012, 09:06:15 AM
somebody needs to explain this like im 5....

do normal non miners make an difference?


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: [Tycho] on January 29, 2012, 01:41:05 PM
http://bitcoinmedia.com/the-truth-behind-bip-16-and-17/
"There are four functional fundamental implementations of the bitcoin protocol" - looks like you forgot about the Ufasoft client.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: [Tycho] on January 29, 2012, 01:46:54 PM
I wonder why rush to deploy such extremally dangereous changes
without months of torturing testing on the testnet ??

I mean BOTH proposals (16 and 17).
Gavin created a bot that makes BIP17 testing impossible on the testnet.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: joulesbeef on January 29, 2012, 01:55:37 PM
I'm concerned about lukejr continuing to work on bitcoin after he proved his is an untrustworthy individual. He abused the people using his pool. And they want to continue to let him change the bitcoin code? I tell you what as a business owner, I would say fuck that shit. ANd really I am going to make sure as many business owners as possible who are considering bitcoin know who helps program it. And let them decide with full information that they have to trust something produced by a person who proved that they could not be trusted.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: vuce on January 29, 2012, 02:05:09 PM
I wonder why rush to deploy such extremally dangereous changes
without months of torturing testing on the testnet ??

I mean BOTH proposals (16 and 17).
Gavin created a bot that makes BIP17 testing impossible on the testnet.
Is this true?


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: genjix on January 29, 2012, 02:41:23 PM
I'm concerned about lukejr continuing to work on bitcoin after he proved his is an untrustworthy individual. He abused the people using his pool. And they want to continue to let him change the bitcoin code? I tell you what as a business owner, I would say fuck that shit. ANd really I am going to make sure as many business owners as possible who are considering bitcoin know who helps program it. And let them decide with full information that they have to trust something produced by a person who proved that they could not be trusted.

This is the wrong attitude that I am campaigning against. Ideas should be selected based on technical merit, not the people behind them. And Luke's idea is technically sound.

Bitcoin is not a business either, it is a system and a community. Having authorities start banning individuals is fascist.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: wachtwoord on January 29, 2012, 02:48:06 PM
Thanks a lot for that post. I have a background in computer science (MSc title and work experience) and have had no time as of yet to form a good opinion. If the facts in that article are correct I am in favor of NEITHER.

Both (especially BIP16, sorry Gavin) sound like complete hacks. Furthermore I don't see a lot of value in the advantage of this, reducing the address lengths of complex operations. I suggest we wait until the 20k limit becomes an issue and solve it by forking the block chain.

This was a fairly high level article though and a lot of people have much more in depth knowledge of the system and the benefits so I could very well be wrong here. Either way I thought I'd voice my opinion (and mention on what information it is based).


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Steve on January 29, 2012, 03:07:13 PM
Both (especially BIP16, sorry Gavin) sound like complete hacks.
They are both hacks to a degree, everyone knows that (there are many other warts in the scripting system too btw).  The questions/concerns are of a more pedestrian nature.  How to make the transition with the least amount of disruption.

I've come to the conclusion that there's very little risk in the transactions themselves.  The risk is just about bugs that might be introduced as a result of the code changes being made in the scripting system.  

A different way to arrive at a consensus is for Gavin and Luke-Jr to put their forks of the bitcoin script engine out there and start lobbying for miner support now (assuming they've already done a prudent amount of testing with tesnet).  For miners, there's little risk in validating these new style transactions if they do it right.  They won't be creating a fork if they have these new transactions in their blocks.  All they're basically doing it some additional validation that might reject some transactions that other miners would include.  If they validate BIP16/17 without rejecting blocks that others mine that don't satisfy the full validation of BIP16/17, then they won't go off on their own fork.  You simply need to caution people against actually creating these new transactions until the majority of the network is fully validating them (i.e. don't provide any UI that uses them until some time after a super majority of miners are validating them).

The first BIP to achieve a majority of support will then quickly achieve super majority support and that BIP can be declared the winner.  After that point, miners can simply never include transactions of the other kind in their blocks (so they don't need to carry forward the validation code of the losing BIP).


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: [Tycho] on January 29, 2012, 03:32:20 PM
I wonder why rush to deploy such extremally dangereous changes
without months of torturing testing on the testnet ??

I mean BOTH proposals (16 and 17).
Gavin created a bot that makes BIP17 testing impossible on the testnet.
You are kidding, right ?
No, I'm not:

Luke has had to test BIP 17 on the main network instead of testnet because I wrote a BIP-17-stealing robot and ran it on testnet

* Disclaimer: I don't think that BIP17 is better than BIP16. Both are ugly hacks. I will support one only if most other miners will.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: hazek on January 29, 2012, 03:39:12 PM
Quote
The risk is loosing the entire bitcoin market. The return is a better blockchain.

Just based on this I'll tell you right now, that if the bitcoin systems's integrity is compromised even for a second, I'm out and will never be back. It's whole value to me depends on it and I know I'm in the majority who think like this.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: wachtwoord on January 29, 2012, 03:48:37 PM
 How to make the transition with the least amount of disruption.

Imo that would be doing neither.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: hazek on January 29, 2012, 03:54:44 PM
 How to make the transition with the least amount of disruption.

Imo that would be doing neither.

Exactly. There's no urgency, so I really don't understand why the urgency to ram through a proposal with any risk attached with the excuse that nothing will get done otherwise? Gavin?


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Steve on January 29, 2012, 03:57:33 PM
 How to make the transition with the least amount of disruption.

Imo that would be doing neither.

Exactly. There's no urgency, so I really don't understand why the urgency to ram through a proposal with any risk attached with the excuse that nothing will get done otherwise? Gavin?
I think this is wrong.  There is urgency.  It's clear the p2sh makes for a better bitcoin in many respects.  If bitcoin cannot make the transition, then another coin most certainly will (maybe litecoin, or something else).  That coin will be superior to bitcoin and will start to attract investment.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: wachtwoord on January 29, 2012, 04:04:51 PM
What is the added value beyond shorter addresses for complex transactions? Because I don't see that as very important.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: fornit on January 29, 2012, 04:09:24 PM
bip 17 sounds more elegant, straightforward and less complex. i think it deserves some time for proper testing.

that being said i am not sure if i can fully assess the downsides of implementing neither bip. longer addresses, bigger blockchain (is there an estimation how much bigger?). any security problems?

edit: btw: its stated as an advantage of bip 16 that you can put more multisig transactions in a block. doesnt that make miners favor multisig transactions? (higher fees and you can still include many in a block)
if so, wouldnt the fees of normal transactions converge to the fees of multisig transactions once the blocks get fuller and there is actual competetion about which transactions go into the first possible block?


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Gavin Andresen on January 29, 2012, 04:41:53 PM
Exactly. There's no urgency, so I really don't understand why the urgency to ram through a proposal with any risk attached with the excuse that nothing will get done otherwise? Gavin?

It is completely artificial urgency, so a decision is reached and implemented in a reasonable amount of time.

When I set the deadline, I had no idea Luke would:
  a) Decide that a bugfix 0.5.2 release was a good idea. I thought the minor bugfixes weren't worth taking the time to make a release but arguing with Luke is like arguing with a brick wall, so I went "meh, whatever."
  b) Propose BIP 17 and go on a war against BIP 16.

Both of those distracted from implementation and testing of BIP 16.

But what's past is past, the question is what do do from here; I'm planning on starting that discussion in the Dev&Tech forum tomorrow.

PS: RE: risk:  the risks of either proposal to the overall health of the bitcoin network are small.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Steve on January 29, 2012, 04:53:17 PM
What is the added value beyond shorter addresses for complex transactions? Because I don't see that as very important.
I think it's important.  An address doesn't have to be a hash of anything…an address could be the entire script that you want someone to send coins to.  However, that would be rather unwieldy and there has been considerable investment in the notion of a bitcoin address being a relatively short thing.  Both investment in terms of software and investment in terms of people learning and understanding bitcoin.  To start passed around very long addresses would be rather confusing I think.  It also has the drawback of revealing more about the destination script than is necessary.

P2SH is the way that bitcoin should have been designed from the beginning IMO.  Outputs of transactions (scriptPubKey) should have always been just a hash and the validation always been that a script hashes to that value and executes successfully.  Using long addresses would be moving in the wrong direction in my opinion.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: oOoOo on January 29, 2012, 08:05:10 PM
Quote
P2SH provides a way to hide these SIGOPs from Satoshi’s rule to allow a newer more precise method for counting them to be developed.

(...)

P2SH does not solve this issue, but it buys more time.

... and ...

Quote
Rather than introducing a new paradigm of executing a piece of data on the stack upon recognising a particular script schema, CHV (BIP 0017) uses an existing operation that seemed to be included for this purpose by Satoshi: CODESEPARATOR.

CHV’s disadvantage over P2SH is nodes will accept any transactions spending an output as valid. To old clients it looks like a transaction that is spendable by anybody while P2SH requires the hashed data.

Judging on your explanation it appears as if BIP 17 is the superior approach. BIP 16 appears to be sacrificing elegance and consistency just for the sake of backwards compatibility ("older clients"). I don't see any reason to do this, an economically critical software such as bitcoin should be maintained and updated regularly by all users just like you would update any anti-virus software.

Quote
Summarising:

  • P2SH introduces a complexity-inducing solution to a tough problem.
  • CHV introduces a new operation code. Unexpected combinations of opcodes have been a problem in the past.
  • Remain where we are with all of bitcoin’s imperfections and problems. Use 70+ character addresses for multiple person transactions instead of 20 character ones.

I don't believe the last point to be a big issue at all. People will rarely write down an address character-by-character. Compared to weird/untested hacks and unnecessary complexity, a 70 symbol address seems to be the smalles evil.
.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: gmaxwell on January 29, 2012, 08:06:50 PM
Gavin created a bot that makes BIP17 testing impossible on the testnet.

This isn't true.  Yes, Gavin created a bot that exploits a differential weakness of BIP17 vs BIP16. This was pretty helpful as luke was insisting there was no difference, and this let us feel out how the difference actually mattered.

This doesn't make testing impossible, and there are in fact test transaction there. It actually aided testing and helped find some misbehavior in the software Luke was running with BIP17.

I have mined testnet for Luke several times to aid in his testing (and, in fact, did a costly 40 blockish reorg for him).

I think it's unfortunate that both Tycho and Genjix are both spreading overt misinformation here in order to create controversy.  (Tycho making insane claims that testing BIP17 is impossible, Genjix with falsely describing this as a "vote", claiming that people don't want you to know about this open and widely discussed matter, the over the top subject line)

The kind of hysteria being promoted here is a very big disincentive to contributing technically to bitcoin. I encourage everyone to be patient and thoughtful and recognize that bitcoin can not survive if people with powerful media presences are able to disrupt development and exhaust the developers at will.  Discussion is great, but consider whos interest panic is well aligned with: Certainly not the interest of the Bitcoin using community. Don't let your emotions be manipulated.



Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: paraipan on January 29, 2012, 08:20:14 PM
@gmaxwell the pressure was created by Gavin and Luke, in the first place, with not so accurate statements, personal attacks and a close deadline ahead. Now you only see the waves coming back.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Luke-Jr on January 29, 2012, 08:49:50 PM
Gavin created a bot that makes BIP17 testing impossible on the testnet.
Not quite. You can still test by using -p2shtime=0

I'm concerned about lukejr continuing to work on bitcoin after he proved his is an untrustworthy individual. He abused the people using his pool. And they want to continue to let him change the bitcoin code? I tell you what as a business owner, I would say fuck that shit. ANd really I am going to make sure as many business owners as possible who are considering bitcoin know who helps program it. And let them decide with full information that they have to trust something produced by a person who proved that they could not be trusted.
As a business owner, you should know better than to buy into these libelous lies and use such crude language.

When I set the deadline, I had no idea Luke would:
  a) Decide that a bugfix 0.5.2 release was a good idea. I thought the minor bugfixes weren't worth taking the time to make a release but arguing with Luke is like arguing with a brick wall, so I went "meh, whatever."
I don't see how 0.5.2 is at all related to this. I'm pretty sure there was a general consensus that the mlock fix (which drastically improved startup time and initial blockchain download) warranted it.
 b) Propose BIP 17 and go on a war against BIP 16.
Rather, I was "on a war against" BIP 16 from the day it was proposed. You chose to basically ignore my objections, and nobody else seemed to be coming up with an alternative to BIP 16 (since BIP 12 had been killed off), so I spent the time to create BIP 17 as a solution to provide P2SH without the problems posed by BIP 16.

Yes, Gavin created a bot that exploits a differential weakness of BIP17 vs BIP16. This was pretty helpful as luke was insisting there was no difference, and this let us feel out how the difference actually mattered.
The same bot could be created to steal BIP16 transactions instead.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: gmaxwell on January 29, 2012, 09:03:32 PM
@gmaxwell the pressure was created by Gavin and Luke, in the first place, with not so accurate statements, personal attacks and a close deadline ahead. Now you only see the waves coming back.

Please show me a statement made by Gavin which is inaccurate or a personal attack.

The whole characterization of Luke vs Gavin is nonsense to begin with. BIP16 arose out of combining a half dozen proposals, it was a _consensus_ proposal. At the regular development meeting that produced it no one was objecting to it at the end (partially no one because Luke had to leave early, before he left he indicated that he would only support it if some language about deprecating non-BIP16 transactions was added to the BIP).

Yes, Gavin created a bot that exploits a differential weakness of BIP17 vs BIP16. This was pretty helpful as luke was insisting there was no difference, and this let us feel out how the difference actually mattered.
The same bot could be created to steal BIP16 transactions instead.

Come on, you know it's not the same. A single BIP17 stealer hidden on tor with only one connection to the network will be 100% effective (in an network without P2SH validation, of course). A BIP16 stealer is only 100% effective if run by a large miner, otherwise it is only X% effective in cases where the stealer is on the shortest path between the spender and X% of the mining hash power.   I agree that it's a corner case, but there is a difference there.



Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: paraipan on January 29, 2012, 09:10:43 PM
@gmaxwell the pressure was created by Gavin and Luke, in the first place, with not so accurate statements, personal attacks and a close deadline ahead. Now you only see the waves coming back.

Please show me a statement made by Gavin which is inaccurate or a personal attack.

The whole characterization of Luke vs Gavin is nonsense to begin with. BIP16 arose out of combining a half dozen proposals, it was a _consensus_ proposal. At the regular development meeting that produced it no one was objecting to it at the end (partially no one because Luke had to leave early, before he left he indicated that he would only support it if some language about deprecating non-BIP16 transactions was added to the BIP).



just look around you will find some of Gavin's threads. I'm into in the loop with you guys, dev team, so my opinions are based on what was made public on the forum lately.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: gmaxwell on January 29, 2012, 09:13:46 PM
just look around you will find some of Gavin's threads. I'm into in the loop with you guys, dev team, so my opinions are based on what was made public on the forum lately.

If you're going to slander like that you at least owe the participants here a link to a specific message/a quote.

I think what you're saying is untrue and unfounded but if you go around and repeat it as a fact people who, unlike me, haven't read almost all the discussion are going going to believe it.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Luke-Jr on January 29, 2012, 09:19:02 PM
When I set the deadline, I had no idea Luke would:
  a) Decide that a bugfix 0.5.2 release was a good idea. I thought the minor bugfixes weren't worth taking the time to make a release but arguing with Luke is like arguing with a brick wall, so I went "meh, whatever."
  b) Propose BIP 17 and go on a war against BIP 16.

Here's the catch: Gavin is forcing everyone using the latest Bitcoin code to vote for BIP 16.

I don't see how 0.5.2 is at all related to this. I'm pretty sure there was a general consensus that the mlock fix (which drastically improved startup time and initial blockchain download) warranted it.

What is this?
Latest code = from git, to-be-0.6.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: paraipan on January 29, 2012, 09:23:15 PM
just look around you will find some of Gavin's threads. I'm into in the loop with you guys, dev team, so my opinions are based on what was made public on the forum lately.

If you're going to slander like that you at least owe the participants here a link to a specific message/a quote.

I think what you're saying is untrue and unfounded but if you go around and repeat it as a fact people who, unlike me, haven't read almost all the discussion are going going to believe it.

let's not elevate on this issue more than is necessary, and you can think whatever you want, at least we have that freedom. I'm not going to link/repeat what has been discussed because the forum does an awesome job keeping a record of it.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: genjix on January 29, 2012, 09:42:18 PM
I think it's unfortunate that both Tycho and Genjix are both spreading overt misinformation here in order to create controversy.  (Tycho making insane claims that testing BIP17 is impossible, Genjix with falsely describing this as a "vote", claiming that people don't want you to know about this open and widely discussed matter, the over the top subject line)

The kind of hysteria being promoted here is a very big disincentive to contributing technically to bitcoin. I encourage everyone to be patient and thoughtful and recognize that bitcoin can not survive if people with powerful media presences are able to disrupt development and exhaust the developers at will.  Discussion is great, but consider whos interest panic is well aligned with: Certainly not the interest of the Bitcoin using community. Don't let your emotions be manipulated.

Here is the list of developers i contacted for feedback before publishing:
- justmoon
- jgarzik
- gavin
- roconnor
- gmaxwell
- sipa
- wladimir
- Mike Hearn
- luke-jr

I got help with copy-editing and feedback from the following people:
- luke-jr
- justmoon
- tcatm

Quote
21:31 < genjix> gmaxwell, roconnor: i'm putting this out tomorrow if you want to give some thoughts on this: http://privatepaste.com/c8b40edb00
21:31 < occulta> also that wiki is very old, relating to client 0.3 *
21:31 < BlueMatt> "minimum TX fee for new transactions reduced to 0.0005 BTC."
21:31 < BlueMatt> its still true
21:31 < genjix> whether it captures the entirety of the EVAL, P2SH, CHV discussion
21:32 < gmaxwell> genjix: ugh. that makes me feel sick. Representing it as a vote is simply misleading.
21:32 < gmaxwell> It's not that kind of 'vote'.
21:32 < genjix> what would you call it then?
21:32 < gmaxwell> I give up.
21:32 < genjix> it basically is, and this is informing the voters to ensure they make a better decision

21:33 < gmaxwell> This process is all broken.
21:33 < gmaxwell> No, it's pissing all over the walls.
21:34 < gmaxwell> genjix: The reason for the coinbase tags is _NOT_ to conduct a vote (if it were, I suppose the software would also tally the result) but simply because there needs to be a hash power measurement because the new rules are only safe if the majority of all future hashpower enforces them.
21:34 < gmaxwell> genjix: there is also no way this is going active on Feb 15th now. So the representation of that is creating false urgency. Though I suppose its up to gavin to announce moving that back.
21:35 < gmaxwell> genjix: you're also characterizing this as gavin vs luke, which is a complete load of rubbish.
21:35 < Joric> are there any pure-python parsers for berkeley db? my google skills are failing me
21:35 < Joric> or at least format documentation
21:36 < BlueMatt> bdb parsers are impossibly hard to find
21:36 < genjix> i asked for feedback to write a better article and you're simply attacking me
21:36 < Joric> yeah )
21:36 < BlueMatt> there may be a bdb wrapper
21:36 < genjix> Joric: pybsbdb
21:36 < Joric> want to get rid of bsddb dependency, gae doesn't have it
21:36 < genjix> it is a good bdb wrapper
21:36 < BlueMatt> genjix: his feedback is that there should be no article
21:36 < BlueMatt> (and I agree)
21:37 < genjix> yes let the mere users fester in ignorance
21:37 < gmaxwell> genjix: sorry. This whole "dispute" thing has basically pushed my interest in contributing to bitcoin technically negative.

21:37 < Joric> etotheipi_, do you need pure python bdb parser aswell?
21:37 -!- graingert [~graingert@unaffiliated/graingert] has joined #bitcoin-dev
21:37 < genjix> i'm only trying to inform people how bitcoin works (if you look at my past articles)
21:38 < gmaxwell> And I'm irate because I feel like I've invested time in something that now has net negative return (it stresses me out). I shouldn't be taking that out on you.

21:38 < Diablo-D3> [04:35:13] <gmaxwell> genjix: you're also characterizing this as gavin vs luke, which is a complete load of rubbish.
21:38 < Diablo-D3> yes really
21:38 < Diablo-D3> because if there was some sort of cage match between the two
21:38 < Diablo-D3> gavin would be going in dry.
21:38  * BlueMatt suggests you leave authors out of the article
21:38 < gmaxwell> genjix: explaining what the P2SH stuff does is fantastic. People seemed to like it when I explained it in #bitcoin-mining the other day.
21:38 < BlueMatt> (as its irrelevant)
21:39 -!- Ahimoth_ [~Ahimoth@75.80.19.176] has joined #bitcoin-dev
21:39 -!- Ahimoth [~Ahimoth@75.80.19.176] has quit [Disconnected by services]
21:39 -!- Ahimoth_ is now known as Ahimoth
21:39 < gmaxwell> genjix: inviting people into taking positions over technical minutia which they won't be qualified to really have an opinion on without a lot more understanding than you can put into the article... meh.
21:40 < BlueMatt> esp when their opinion wont have any effect on the outcome aside from making the pissing match bigger
21:40 < gmaxwell> BlueMatt: exactly.

21:40 < BlueMatt> (kinda doubt any will switch pools)
21:40 -!- wirehead [~a@154.5.144.145] has quit [Ping timeout: 260 seconds]
21:41 < gmaxwell> The solution to this needs to be consensus of the interested and compentent. Not a bigger dispute decided by whomever can convince more people to come to their side.
21:41 < genjix> i'd rather people have a say in the matter even if it makes life tougher for developers to explain their decisions.

21:41 -!- erle- [~m@g225119098.adsl.alicedsl.de] has joined #bitcoin-dev
21:41 < gmaxwell> The latter case has no winners.
21:41 < genjix> these kinds of decisions should always be deliberately difficult and hard
21:41 < BlueMatt> what?
21:41 < BlueMatt> we should make all of our decisions harder?
21:42 < genjix> no big decisions to the protocol or system
21:42 < genjix> implementation decisions - fine.

21:42 < BlueMatt> but we should make the big decisions harder on ourselves?
21:42 < gmaxwell> genjix: So what, some statemen with a prominant website gets people all upset over their half understandings of some technical details and they go against the decisions of the people who are actually spending time to work on the software? You know what the outcome of that is? The people working on it _leave_, and the only people left to make additions are the people who are either too clueless or lazy to contribute now, and luke.
21:43 < BlueMatt> by making the decisions into huge pissing matches where politics takes on more importance than technical arguments?
21:43 < genjix> umm hello?
21:43 < genjix> it's already that way
21:43 < gmaxwell> genjix: when we had the discussion about what would become BIP16 the discussion ended without anyone objecting to that decision. (except luke, who'd left early)
21:44 < genjix> sure. but i feel a bit apprehensive about telling our users this is how it will be, you have no say and then giving them the finger
21:44 < BlueMatt> if they feel like really getting involved, great
21:44 < BlueMatt> they can come in here and chat, and post on the forum
21:45 < BlueMatt> s/forum/mailing list/
21:45 < gmaxwell> genjix: your article is also full of factual errors. For example, the maximum recusion depth was always part of OP_EVAL. roconnor's important contribution was realizing that the implementation was buggy and the limit didn't work.
21:45 < BlueMatt> (freudian slip)
21:45 < gmaxwell> genjix: I don't think any user would be opposed to P2SH as it is— when I explained it in bitcoin-mining the other day people were very excited about it.
21:46 < gmaxwell> genjix: the reason to oppose it is just the risk of unknown bugs— a very important concern, but not one that causual users are qualified to reason about, unfortunately.
21:47 -!- graingert [~graingert@unaffiliated/graingert] has quit [Remote host closed the connection]
21:47 < gmaxwell> genjix: if you think this needs to be delayed in order to address that, then great— but where is the plan that converts extra time into extra software quality?
21:47 < genjix> i dont have a viewpoint on this.
21:47 < genjix> if p2sh, chv or none comes along then i'll implement them.
21:47 < BlueMatt> so why are you encouraging others to make one?

21:48 -!- b4epoche_ [~textual@dssl.mne.psu.edu] has joined #bitcoin-dev
21:48 < genjix> my experience is in software architecture not protocols, so i'm not commenting
21:48 -!- b4epoche [~textual@dssl.mne.psu.edu] has quit [Read error: Operation timed out]
21:48 -!- b4epoche_ is now known as b4epoche
21:48 < genjix> BlueMatt: because i like people to have choice and freedom
21:48 < genjix> it is not harmful to give people choice or information

21:49 < BlueMatt> they do, if they actually want to come and reason about the issues, there is always someone here to discuss with
21:49 < BlueMatt> but they dont have choice
21:49 < BlueMatt> and unless they all switch to p2pool overnight, they wont get one
21:50 -!- shazooun [~shazooun@70-90-104-13-ma-ne.hfc.comcastbusiness.net] has quit [Ping timeout: 245 seconds]
21:50 < gmaxwell> genjix: there can't be choice and freedom without understanding. People don't bother even switching pools when their pools cost them money.
21:51 -!- shazooun [~shazooun@70-90-104-13-ma-ne.hfc.comcastbusiness.net] has joined #bitcoin-dev
21:51 < genjix> my worry is someday bitcoin becomes corrupted. see this extra scrutiny as an opportunity to build a culture of openness
21:51 < genjix> it is not at all bad.
21:51 < tcatm> it's not so much about having a choice but discovering "the best" way to implement more complex transactions types...
21:51 < gmaxwell> genjix: if you're concerned about the ecosystem why aren't you out there figuring out why people are paying 110%-115% PPS for secret mining projects? and telling the people who are contributing who don't currently know where their hash power is going.
21:52 < genjix> gmaxwell: i am writing an article on that
21:52 < gmaxwell> Oh. :)
21:52 < roconnor> genjix: s/Both BIP 0017 and BIP 0018 are/Both BIP 0016 and BIP 0017 are/
21:52 < genjix> but i have never mined a block so a lot of this is news to me.
21:52 < roconnor> PPS?
21:52 < genjix> like proportional mining being a scam and understanding the various statistical measures.
21:53 < gmaxwell> roconnor: pay per share.
21:53 < genjix> pay per share
21:53 < genjix> thanks roconnor
21:53 < gmaxwell> roconnor: people are being given a signficant premium on mining above the expected rewards, with ~zero payout variance.
21:54 < roconnor> gmaxwell: paid it bitcoin?
21:54 < roconnor> *in
21:54 < gmaxwell> roconnor: yes. If it were paid in USD it would be completely sensible.
21:54 < gmaxwell> roconnor: paid in bitcoin daily too.
21:55 < roconnor> that doesn't sound sustainable
21:55 < gmaxwell> roconnor: if it were being used to promote a new mining pool, for example, it would also be sensible... but this is for private projects with no published hash rates, so it doesn't have promotional value.
21:56 < BlueMatt> gmaxwell: link?
21:56 < gmaxwell> My best theory is that they're doing something useful with merged mining, next best is that they've got some idiot money laundering scheme (e.g. give miners dirty silkroad coins), worst outcome is that it's for some idiot attack. :(
21:57 < gmaxwell> BlueMatt: https://bitcointalk.org/index.php?topic=54467.0
21:57 < gmaxwell> https://bitcointalk.org/index.php?topic=61117.0
21:57 < gmaxwell> https://bitcointalk.org/index.php?topic=55819.0 (A meta service to aggregate these offers into a bidding market)
21:57 < gmaxwell> https://bitcointalk.org/index.php?topic=61570.0
21:58 < BlueMatt> now thats just weird
21:58 < gmaxwell> There are more... there are at least a half dozen people doing this all of a sudden.
21:58 -!- wirehead [~a@154.5.144.145] has joined #bitcoin-dev
21:58 < roconnor> gmaxwell: I don't really see how it could be used to launder money
21:59 < gmaxwell> roconnor: I said idiot for a reason! :)
21:59 < roconnor> ah
21:59 < BTC_Bear> gmaxwell: quick question as to this:
21:59 < BTC_Bear> the checkpoints are there to keep from overtaking the blockchain, or at least it is a side benefit. gmaxwell would know more than I. But I believe, I am correct.
22:00 < roconnor> BTC_Bear: it is there to stop DOS attacks by sending you long but low work chains.
22:00 < gmaxwell> BTC_Bear: nah, not really— the checkpoints do that as a side effect but they're so far back that you couldn't realistically overtake even with a good multiple of the hash power for a short window.
22:00 < CIA-97> bitcoin: jedi95 * rec8af03cfdf7 Phoenix-Miner/minerutil/RPCProtocol.py: Fixed expire= for X-Roll-Ntime http://tinyurl.com/7xwchtb
22:00 < BTC_Bear> thanx
22:00 < gmaxwell> what roconnor said, they avoid some stupid DOS attacks.
22:00 < JFK911> where can I get 115%
22:01 < gmaxwell> I guess the risk of a new checkpoint being set does discourage someone from working in secret on a very long overtaking fork, but thats also some speculative side benefit.
22:01 < BlueMatt> as a sidenote, we need a new checkpoint for 0.6
22:02 < BlueMatt> s/for/before/
22:04 -!- dr_win [~dr_win@147.32.31.193] has joined #bitcoin-dev
22:06 < roconnor> genjix: gavin knew about the looping behaviour in OP_EVAL well before december.
22:06 < roconnor> genjix: the maximum iteration code was there from the beginning
22:06 < genjix> roconnor: i've corrected that

Notes:
- I was directly involved with the development of BIP 0016
- I wrote an article trying to present the facts. it was exhausting and took a lot of time so there may be some slip ups, errors or bad phrasing, but i want to inform the users and put out truth. I don't appreciate general hand waving trashing the article, i do appreciate constructive criticism about how to reword paragraphs or change sections to be more accurate.
- Went to great effort to make that article factually accurate, neutral, fair and balanced.
- Have no preference for BIP 16, BIP 17 or none. I will implement whatever comes along. Others have been thinking this over far longer than me.
- I asked the developers before asking them to write an article. Nobody seemed interested so I took on the task.
- Will definitely publish an article challenging mine from another developer.

I am not sure what has changed since this time and I am not sure why you did not bring to my attention what you feel is 'overt misinformation' in the article before hand. If there is something worth editing I will certainly consider it even now. I know you have helped a lot and I really have appreciated your efforts and the leaking of those documents before. I know you're a good guy, but lets talk through this more amicably without the attacks.

I run the BIP standardisation process and it is my responsibility to ensure that a BIP gets adequate discussion and approval before becoming a standard.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: hazek on January 29, 2012, 09:45:20 PM
I think you did a great service to our community with your effort that you put into your article, so thank you very much!


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: paraipan on January 29, 2012, 09:57:26 PM
I think you did a great service to our community with your effort that you put into your article, so thank you very much!

+1


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Costia on January 29, 2012, 10:04:33 PM
Latest code = from git, to-be-0.6.

Normal people don't read "from git" as "latest code" even if it is technically true. Just sayin'.
note the "code" in latest code - not "release"
but yeah if you dont mess with CVS yourself , you probably won't notice :)


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: westkybitcoins on January 29, 2012, 10:06:53 PM
Thanks for the article, genjix. Definitely cleared some things up for me.

My conclusion (for what little it's worth:)

BIP 17 seems to clearly be a superior method coding-wise. BIP 16 seems like overkill.

But my vote, in part made by continuing to run a pre-Qt client, is for neither.

Ultimately we seem to be fracturing the community (and potentially the blockchain!) just for the sake of convenience. If complex transactions require 70-character addresses, then so be it. To risk permanent harm to the system to avoid them just seems short-sighted.

I also don't buy the whole concern about who pays transactions costs... considering the recipient can charge whatever he wants for whatever he's selling, that point is more academic than anything. And the blockchain bloat? That almost seems like a distraction. Complex transactions are going to take up extra space in the blockchain; arguing over where and when the bloat occurs seems pretty academic too.

I'm thinking Tycho made the right call here.

Note: Again, I'm just stating my perspective on all this; no insult to anyone intended.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: gmaxwell on January 29, 2012, 10:16:16 PM
Quote
I am not sure what has changed since this time and

Nothing changed. Thats the point. You didn't correct the highly exaggerated an "political" representation which I  (and _many_ other people) pointed out was confusing and misleading.

You've continued with your misrepresentation here with your selective bolding in your quotation.  For example, I'd spent some time making what I thought was a pretty clear and non "political" explanation of the subject in #bitcoin-mining, which I recommended you borrow from.  But you've bolded my commentary in a way which presents a false description of my position.

Please completely unbold my comments or completely bold them. I consider it unacceptable that you skipped over things like "< gmaxwell> genjix: explaining what the P2SH stuff does is fantastic. People seemed to like it when I explained it in #bitcoin-mining the other day."  but then bold the "but" part of the statement that I said next.

You make it sound like you're somehow more neutral than anyone else. This is nonsense, you might be less informed than other people— but that doesn't make you more neutral.  I gain _nothing_ from any of the particular proposals, nor did I write any of their code.  (I've done more testing work on BIP17 (the proposal that I don't prefer) than BIP16, simply because Luke needed help with testing and asked for it).  But somehow you paint me as a partisan? thats weird to me.





Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Atheros on January 29, 2012, 10:16:27 PM
To risk permanent harm to the system...

There is no risk of permanent harm.

And the blockchain bloat? That almost seems like a distraction. Complex transactions are going to take up extra space in the blockchain; arguing over where and when the bloat occurs seems pretty academic too.

You forget that completed transactions can be pruned. They can and will be deleted by many miners after the bitcoins are spent further. Unspent bitcoins can not. They are kept by all miners forever. These BIPs would reduce the size of unspent transactions, no matter how complex, to the lowest safe size. It is not merely academic. It is significant.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: oOoOo on January 29, 2012, 10:18:53 PM
Ultimately we seem to be fracturing the community (and potentially the blockchain!) just for the sake of convenience. If complex transactions require 70-character addresses, then so be it. To risk permanent harm to the system to avoid them just seems short-sighted.

Agreed!


There is no risk of permanent harm.


Besides the split of the chain...


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Costia on January 29, 2012, 10:23:01 PM
Quote
These BIPs would reduce the size of unspent transactions, no matter how complex, to the lowest safe size.
never understood why that would happen, the bips just move the large script from the output to the input
unspent blocks have both
inputs of unspent transacions can be pruned? (afaik pruning is actually not supported by the client)
I thought the merkle tree contained hashes of transaction and not hashes of separate inputs/outputs...


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: gmaxwell on January 29, 2012, 10:24:55 PM
Ultimately we seem to be fracturing the community (and potentially the blockchain!) just for the sake of convenience. If complex transactions require 70-character addresses, then so be it. To risk permanent harm to the system to avoid them just seems short-sighted.
Agreed!
There is no risk of permanent harm.
Besides the split of the chain...

Any split would resolve, assuming a simple majority of the future hash power was enforcing the new rules. It's not an unresolvable split risk.

70 characters is more like the minimum address length for a non P2SH secure wallet address.  An actual complex transaction might be more like 1000-2000 characters.

I gave genjix a pretty nice list of reasons why P2SH is important other than the long addresses. Unfortunately, he decided to only quote from people saying negative things about P2SH in his post.  Here is what I wrote to him:

Quote

...  No it's not a mistake.  P2SH _prevents_ needing long addresses.

Lets unpack the acronym "pay to script _hash_".  Hashes only need to
be 128-256 bits in size or so to have acceptable security, so you
don't need something longer than that for paying to a hash.

Note that gavin is saying 70 characters, not bytes.

Without some form of P2SH then only way for you to make a personal
choice of asking people to pay to a two-factor protected account or
two a multiparty trust that manages the finances of an organization
is using some form of "P2S", pay-to-script.

In other words, you'd have to have an address that encodes a full
script specification for the sender to pay to,  instead of just
encoding its hash.  As a result these addresses would be much longer
(and potentially very long).

The minimum size of a two address involving encoded script would be on
that order, but they get bigger quite quickly if you add more options
to the script (actually 70 sounds quite small, it should be more like
100 for a minimum two pubkey script).

In addition to the unworkability of very long addresses as described
by gavin (amusingly I am unable to copy and paste the quoted example
in one go) a P2S solution has several problems which you might
consider more or less important:


(1) They are highly vulnerable to invisible substitution.  E.g. I can
trivially take a P2S address, change one or two characters and get a
script which is redeemable by anyone.  With P2SH you have to do
computation which is exponential in the number of unchanged digits to
get a look alike address.

(2) The sender is fully responsible for fees related to the enlarged
transactions. Even if _you're_ willing to take the txn-processing time
and fee burden of a 30 person joint trust address,  random e-commerce
sites will not be and will randomly reject your addresses.

(3) They create another input vector for non-trivial data which must
be inspected and validated, potentially presenting an attack surface.

(4) They leave the complicated (long) release rules in the transaction
outputs.  When a transaction is mined we can't be sure if it will ever
be redeemed. The outputs are unprunable.   In a future world where
many nodes prune output space is far more important than input space
and it would make sense to require more fees for it because we're
never sure how long it would need to be stored (making it an
attractive target for someone who wants to make Bitcoin unusable by
spamming it with worthless data).  P2SH reduces output sizes to the
absolute minimum without inflating the total data size.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: gmaxwell on January 29, 2012, 10:34:10 PM
Quote
These BIPs would reduce the size of unspent transactions, no matter how complex, to the lowest safe size.
never understood why that would happen, the bips just move the large script from the output to the input
unspent blocks have both
inputs of unspent transacions can be pruned? (afaik pruning is actually not supported by the client)
I thought the merkle tree contained hashes of transaction and not hashes of separate inputs/outputs...

Pruning in this case doesn't actually have too much to do with the merkle tree here.  Once a transaction has been spent, and the spend is burred in your chain, you'll never need any of it's inputs (or the outputs they spend) again and so you can save the storage.   (the connecting tree isn't relevant, because you've already validated the transactions for yourself, you don't need to continually prove to yourself that you weren't lying to yourself)

When you get a transaction you don't know when it will be spent— or _if_ it ever will be— maybe you'll need to store its outputs for 6 months or 60 years.   But you do know that you'll be able to forget about its inputs as soon as they're burred deep enough in the chain.   So given equal total sizes we should strongly prefer to put more of the bytes in the script signature (the input side).

The current reference software doesn't take advantage of this... but the chain is only a gigabyte now.  The storage requirements from these decisions have an impact for basically forever.



Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Costia on January 29, 2012, 10:44:54 PM
Quote
(the connecting tree isn't relevant, because you've already validated the transactions for yourself, you don't need to continually prove to yourself that you weren't lying to yourself)
it becomes relevant if you add it to the default client.
then everyone will remove the inputs, and new clients downloading the blockchain will have trouble finding a record they can validate. (and if a new client cant validate, maybe an attacker will be able to generate a longer chain by manipulating some of the hashes)
the way i see it, if you decided to keep the blockchain - you should never prune so it cant be validated. if you don't want to keep it, you can delete all of the blocks right after validation and just keep a signed (by yourself) copy of unspent transactions.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Atheros on January 29, 2012, 10:49:26 PM
it becomes relevant if you add it to the default client.
then everyone will remove the inputs, and new clients downloading the blockchain will have trouble finding a record they can validate.

Pruning will not be so naïvely implemented.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: gmaxwell on January 29, 2012, 10:56:47 PM
then everyone will remove the inputs, and new clients downloading the blockchain will have trouble finding a record they can validate.

This is always the case for pruning, or at least pruning naively implemented. The reference client will support both modes of operation some day. I have a half dozen copies of the blockchain at home, thats silly and non-scalable. :)

The alternative to pruning isn't not pruning— FWIW— if the reference client doesn't gain the ability to prune more and more people will move to nodes like multibit which don't validate _at all_, and bitcoin will lose its decentralization.  Bootstrapping is important, but its irrelevant if you can't afford the runtime costs.

Quote
the way i see it, if you decided to keep the blockchain - you should never prune so it cant be validated. if you don't want to keep it, you can delete all of the blocks right after validation and just keep a signed (by yourself) copy of unspent transactions.

You need to keep enough to allow you to reorganize, but yes. P2SH minimizes the size of that set of that unspent data (by a factor of 2-4 or so, depending on what kind of assumptions you make on the frequency of complicated transactions). It does this without any particular 'costs', except the cost of deploying something new now. It's just one of the several advantages of using P2SH style transactions.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Costia on January 29, 2012, 11:02:22 PM
you can prune in a way that you can still validate. you will only nee to keep 1 transaction per block and the merkle tree hashes.
I guess it will only matter for light clients then.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: westkybitcoins on January 30, 2012, 02:04:12 AM
To risk permanent harm to the system...

There is no risk of permanent harm.

Hmm. I'm going by what I've read on the other thread. I seem to recall one of the big points of contention was that, essentially, "we can't screw this up, because if we do, we won't ever be able to fix it."

One example:

And, at the same time, there's nothing stopping anyone from implementing both.  It's conceivable someone could create a client that did the validation for both BIP-16 and BIP-17.  Miners could also validate both style transactions.  Since they are both designed to be backward compatible, they will all be recognized as legitimate transactions by the whole network if they get into a block (even if not all clients relay them).  The downside of course is that if you create either a BIP-16 or BIP-17 style transaction and the majority of mining power isn't doing the full validation, you run the risk of losing your coins to theft.

Perhaps its more accurate to say implementing either risks causing serious problems.



Quote
And the blockchain bloat? That almost seems like a distraction. Complex transactions are going to take up extra space in the blockchain; arguing over where and when the bloat occurs seems pretty academic too.

You forget that completed transactions can be pruned. They can and will be deleted by many miners after the bitcoins are spent further. Unspent bitcoins can not. They are kept by all miners forever. These BIPs would reduce the size of unspent transactions, no matter how complex, to the lowest safe size. It is not merely academic. It is significant.

Unspent bitcoins are only kept by miners until they are spent.

Sure, there will be some "lost" coins... coins sent using complex transactions where the recipient loses the private key, dies before spending the coins, etc. But I think that's a negligible situation that shouldn't dictate the protocol.

I think the real question is, how long on average will coins sent via complex transactions remain unspent? If the intention is for most transactions in the future to become complex (which I'm seeing hints of, and if so, I'd love to be pointed to where this is actually discussed openly,) then I see your point, but I don't see that that as a given.

Even if the average time delay of spending coins from complex transactions is months (which sounds highly unlikely,) it still just delays the pruning. A "rolling amount" of temporary bloat from some unspent complex-tx coins doesn't seem like it would be enough to worry about.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: gmaxwell on January 30, 2012, 02:13:45 AM
I think the real question is, how long on average will coins sent via complex transactions remain unspent? If the intention is for most transactions in the future to become complex (which I'm seeing hints of, and if so, I'd love to be pointed to where this is actually discussed openly,) then I see your point, but I don't see that that as a given.

The expectation that many in the future would be 'complex' because all two-factor wallets transactions would be complex.  I'd certainly encourage typical users to go this route.

But it's also not the entirely right question to ask here, because the chain doesn't just carry earnest transactions.  If someone wants to DOS attack bitcoin to make it unusable or ruin its decentralization, they'll use the cheapest method available to add the most immortal data to the blockchain. By reducing the amount of potentially immortal data needed in normal financial transactions we reduce that attack, because it is either easily distinguishable from normal transactions (thus subject to higher fees), or isn't as much data.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Luke-Jr on January 30, 2012, 02:19:26 AM
FYI, earlier this morning Gavin solved the 1000-multisig-input limitation on BIP 17. I'm in the process of drafting a new BIP to cover these scripts (this isn't directly P2SH related) now.

P.S. I don't claim that Gavin is supportive of BIP 17 in light of this solution. I don't know if he is or not.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: N12 on January 30, 2012, 02:26:17 AM
I’m with gmaxwell and BlueMatt. Why turn this technical stuff into a FUD political power struggle debate? It’s not like everyone had an opinion on how encryption should have been done, too. People don’t have to have an opinion on this, it’s up to the developers to achieve consensus.

I’m also with Gavin and Steve. Two-factor Authentication is extremely important for safely using Bitcoin without having the hassle of paper wallets and extra offline computers! Noone except us enthusiasts would go for such lengths, and that could be a reason we are stagnating in terms of users and acceptance.

I don’t understand why this debate exists.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: westkybitcoins on January 30, 2012, 02:39:36 AM
I think the real question is, how long on average will coins sent via complex transactions remain unspent? If the intention is for most transactions in the future to become complex (which I'm seeing hints of, and if so, I'd love to be pointed to where this is actually discussed openly,) then I see your point, but I don't see that that as a given.

The expectation that many in the future would be 'complex' because all two-factor wallets transactions would be complex.  I'd certainly encourage typical users to go this route.

I do see the utility of this, but I find myself wondering if this will really be how the masses will handle most of their transactions; having to always use two devices for every spend can get inconvenient, and people like convenience. Still, yes, were this to become standard (which I personally would dislike) I could see a lot more unspent complex-tx outputs sitting around.


Quote
But it's also not the entirely right question to ask here, because the chain doesn't just carry earnest transactions.  If someone wants to DOS attack bitcoin to make it unusable or ruin its decentralization, they'll use the cheapest method available to add the most immortal data to the blockchain. By reducing the amount of potentially immortal data needed in normal financial transactions we reduce that attack, because it is either easily distinguishable from normal transactions (thus subject to higher fees), or isn't as much data.

Hmm. I see your point.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: makomk on January 30, 2012, 10:21:38 PM
So apparently, despite the fact that the miners are currently getting to make the decision and are the ones most directly affected by the bugs resulting from the code being deployed in the way it has been, letting them decide is an attack on Bitcoin:

Quote
23:32 < gmaxwell> luke-jr: perhaps because you're in charge of 400GH of other people's hash power you're happy to say that a coinbase tally is the right way to determine support, but I don't think thats good for bitcoin.
23:34 < gmaxwell> If miners were to force this by simple majority of hashing power, but the community opposed it — we'd call that a 50% attack.

However, trying to give users enough information to make a decision is also apparently unnecessary and everyone should just trust the developers to make the right decision on their behalf. Sigh. Wonder if people have underestimated just how fundamentally Bitcoin relies on its developers remaining honest.

This isn't true.  Yes, Gavin created a bot that exploits a differential weakness of BIP17 vs BIP16. This was pretty helpful as luke was insisting there was no difference, and this let us feel out how the difference actually mattered.
On the other hand Gavin didn't even try and create a bot to exploit the weakness that exists in BIP16. It also took him quite a while to admit that he'd done it leaving everyone puzzled as to what was going on and who was doing it. (Either would have been quite easy to exploit on testnet and a bit harder to exploit elsewhere from what I recall; testnet isn't an entirely realistic place to test this kind of thing.)


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Gavin Andresen on January 30, 2012, 10:42:08 PM
For context: makomk is the creator of CoiledCoin, a bitcoin alternative:
  https://bitcointalk.org/index.php?topic=56675

And RE: creating bots:  I created a BIP-17-stealing bot because it was really easy (took about 10 minutes of hacking).  A BIP-16-stealing bot would be a lot harder (because it would have to 'lie in wait' until the sender was redeeming the coins, and then race to relay/mine a 'stealing' version of the transaction before the rest of the network mined the original).


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: makomk on January 30, 2012, 11:06:54 PM
For context: makomk is the creator of CoiledCoin, a bitcoin alternative:
  https://bitcointalk.org/index.php?topic=56675
For additional context, as a result of developing it I found and reported a couple of fairly nasty security vulnerabilities in the OP_EVAL implementation that BIP 16 could've inherited - though I believe Gavin may have already found and fixed the second one and just not notified the pools about it. Also, CoiledCoin has been dead for a long time now courtesy of luke-jr having more than 51% of the hash power and using it to block transactions. It's complicated and mostly irrelevant which is why I didn't mention CoiledCoin in the first place.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: markm on January 31, 2012, 12:33:24 AM
It seems to me that one of the arguments for these new transactions has been that two devices (device "Alice" and device "Bob", presumably in crypto parlance) cannot work together to make a secret that neither of them alone possess?

But I thought that in fact crypto has already solved the problem of how "Alice" and "Bob" can work together to create keys that enigher of them actually has solo ability to use?

Is that incorrect?

If not, then surely the standard way for Alices and Bobs to do such things can work on the device side without the chain having to know anything about it?

-MarkM-


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: genjix on January 31, 2012, 12:52:11 AM
My stance:

http://bitcoinmedia.com/cathartic-progress/

Michael Marquardt (theymos) suggests compiling a list of everyone intimate with the bitcoin protocol to invite to a two-week email discussion. After those two-weeks a vote is taken. It will be the job of the champions of each idea (BIP 16, BIP 17 and no change) to win over the committee into supporting them. If an idea has necessary support, bitcoin clients will be programmed to apply the new rules for 3 months in the future.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: [Tycho] on January 31, 2012, 01:02:25 AM
Michael Marquardt (theymos) suggests compiling a list of everyone intimate with the bitcoin protocol to invite to a two-week email discussion. After those two-weeks a vote is taken. It will be the job of the champions of each idea (BIP 16, BIP 17 and no change) to win over the committee into supporting them. If an idea has necessary support, bitcoin clients will be programmed to apply the new rules for 3 months in the future.
Good idea, but it solves a problem that never existed (chosing between BIP16 and BIP17).


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Technomage on January 31, 2012, 01:20:53 AM
My stance:

http://bitcoinmedia.com/cathartic-progress/

Michael Marquardt (theymos) suggests compiling a list of everyone intimate with the bitcoin protocol to invite to a two-week email discussion. After those two-weeks a vote is taken. It will be the job of the champions of each idea (BIP 16, BIP 17 and no change) to win over the committee into supporting them. If an idea has necessary support, bitcoin clients will be programmed to apply the new rules for 3 months in the future.
I really hope this is the next step.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Steve on January 31, 2012, 01:26:11 AM
My stance:

http://bitcoinmedia.com/cathartic-progress/

Michael Marquardt (theymos) suggests compiling a list of everyone intimate with the bitcoin protocol to invite to a two-week email discussion. After those two-weeks a vote is taken. It will be the job of the champions of each idea (BIP 16, BIP 17 and no change) to win over the committee into supporting them. If an idea has necessary support, bitcoin clients will be programmed to apply the new rules for 3 months in the future.
And the nice thing about this is that if the committee comes to a decision the miners disagree with, the miners aren't bound to actually follow the decision of the committee.  :)

I posted some stuff about using the Condorcet method in replies to the article.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Andrew Vorobyov on January 31, 2012, 08:26:08 AM
I just want to underline that if miners will not agree with it they will be responsible for future of Bitcoin.

Please, make sure that in the long run we will find solution... but it can cost us chain forks (blood and revolutions)...

I hope we can stay away from it and keep unity... because otherwise things can go pretty nasty... I really would think twice before submitting myself to be responsible for BTC's future..

BTW, I like Tyco's approach... if he had taken any side it will be irresponsible for him...



Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: EhVedadoOAnonimato on January 31, 2012, 10:32:02 AM
Thanks gengix for the article.

I agree with Tycho and Mike Hearn. There is no urgency in this. The risks are high when compared to the small feature improvement.

Huge addresses are not a major problem. If people can get along with huge URLs, there's no reason to believe they would be lost with large addresses. Even if an address is thousands of chars long, shortening services could come along and provide a short URL that you can put into those QR codes again.
And transaction fees are and will remain very cheap to the end user for several years yet. People actually argue they will remain low forever, and that the main financing to mining will come from other sources. Even if that's not the case, there's plenty of time to do this calmly.

Why not just let every potentially chain forkable change to be made when the block size limit is increased/removed, since that will have to happen one day anyway?


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: cbeast on January 31, 2012, 11:45:39 AM
Huge addresses can make difficult to read QR codes. They become harder to scan with a camera phone and even print clearly. Just an added observation.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: EhVedadoOAnonimato on January 31, 2012, 12:41:36 PM
Huge addresses can make difficult to read QR codes. They become harder to scan with a camera phone and even print clearly.

As do huge URLs. People would just use a shortener, as I suggested on my post above yours.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: da2ce7 on January 31, 2012, 12:48:28 PM
As do huge URLs. People would just use a shortener, as I suggested on my post above yours.

That is the problem... it takes up extra space in the chain... where the sender pays the fees; not the spender.

This is bad; as say mtgox would only except 'plain' bitcoin addresses.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: EhVedadoOAnonimato on January 31, 2012, 02:25:10 PM
As do huge URLs. People would just use a shortener, as I suggested on my post above yours.

That is the problem... it takes up extra space in the chain... where the sender pays the fees; not the spender.

And transaction fees are and will remain very cheap to the end user for several years yet. People actually argue they will remain low forever, and that the main financing to mining will come from other sources. Even if that's not the case, there's plenty of time to do this calmly.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Luke-Jr on January 31, 2012, 03:25:21 PM
Huge addresses are not a major problem. If people can get along with huge URLs, there's no reason to believe they would be lost with large addresses. Even if an address is thousands of chars long, shortening services could come along and provide a short URL that you can put into those QR codes again.
I don't think it's a good idea to encourage people to trust URI shortening services with their money.



Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: EhVedadoOAnonimato on January 31, 2012, 03:33:15 PM
If they are already trusting a service similar to an eWallet to hold half the keys.... but yeah, you have a point, shortening services wouldn't work in most cases. But still, big addresses and transaction fees are not that big of a problem to provoke all this rush and heated discussions.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Atheros on February 02, 2012, 10:05:27 PM
But still, big addresses and transaction fees are not that big of a problem to provoke all this rush and heated discussions.

If we don't fix it soon, it will never get fixed. And it needs to get fixed because it will be an enormous problem several years from now. Proper scrutiny and testing is certainly necessary- but these changes are urgent.

Protocol changes are early or never. And never is very very bad.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Luke-Jr on February 02, 2012, 10:32:11 PM
But still, big addresses and transaction fees are not that big of a problem to provoke all this rush and heated discussions.

If we don't fix it soon, it will never get fixed. And it needs to get fixed because it will be an enormous problem several years from now. Proper scrutiny and testing is certainly necessary- but these changes are urgent.

Protocol changes are early or never. And never is very very bad.
There are much more important fixes that need to be made for Bitcoin to scale, and they are hard-forking.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Atheros on February 02, 2012, 10:38:02 PM
Like the maximum block size (1,000,000 bytes) and maximum number of signature operations per block (20,000)?

Maybe this is naïve, but I'm personally fine with that being taken care of a little later- the very fact that it *must* be done means that it actually will get done. And hopefully by that point we can take care of a few other things on the hardfork wishlist (https://en.bitcoin.it/wiki/Hardfork_Wishlist).


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: 2_Thumbs_Up on February 02, 2012, 11:07:14 PM
Huge addresses can make difficult to read QR codes. They become harder to scan with a camera phone and even print clearly. Just an added observation.
But RFID tags will most likely be a good competitor to QR codes as NFC becomes standard in smart phones, no?

What is the added value beyond shorter addresses for complex transactions? Because I don't see that as very important.
I think it's important.  An address doesn't have to be a hash of anything…an address could be the entire script that you want someone to send coins to.  However, that would be rather unwieldy and there has been considerable investment in the notion of a bitcoin address being a relatively short thing.  Both investment in terms of software and investment in terms of people learning and understanding bitcoin.  To start passed around very long addresses would be rather confusing I think.  It also has the drawback of revealing more about the destination script than is necessary.

P2SH is the way that bitcoin should have been designed from the beginning IMO.  Outputs of transactions (scriptPubKey) should have always been just a hash and the validation always been that a script hashes to that value and executes successfully.  Using long addresses would be moving in the wrong direction in my opinion.
Am I right in my understanding of the issue if I think that the problem arises from trying to implement P2SH and keeping backwards compatibility? If P2SH had been the way bitcoin works from the beginning then it would be fine, but since we now have an "old" block chain to think about the solutions become hackish?


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Atheros on February 02, 2012, 11:26:10 PM
But RFID tags will most likely be a good competitor to QR codes as NFC becomes standard in smart phones, no?

This would still be the data one would need to copy and paste. They would still be called "Bitcoin addresses" by everyone except perhaps the developers themselves. They should be as short as possible. Also, all of that data is necessarily going to be stored in the blockchain- much of it forever and ever.

Am I right in my understanding of the issue if I think that the problem arises from trying to implement P2SH and keeping backwards compatibility? If P2SH had been the way bitcoin works from the beginning then it would be fine, but since we now have an "old" block chain to think about the solutions become hackish?

Yes, this is my impression. The fact that we have an old blockchain isn't the problem, it is the fact that we would have old clients. They would need to upgrade.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: 2_Thumbs_Up on February 02, 2012, 11:44:25 PM
But RFID tags will most likely be a good competitor to QR codes as NFC becomes standard in smart phones, no?

This would still be the data one would need to copy and paste. They would still be called "Bitcoin addresses" by everyone except perhaps the developers themselves. They should be as short as possible. Also, all of that data is necessarily going to be stored in the blockchain- much of it forever and ever.
I agree with those points, I just don't think that QR codes will stay around forever so that part shouldn't really be a factor in determing the future of bitcoin as far as I'm concerned.

Am I right in my understanding of the issue if I think that the problem arises from trying to implement P2SH and keeping backwards compatibility? If P2SH had been the way bitcoin works from the beginning then it would be fine, but since we now have an "old" block chain to think about the solutions become hackish?

Yes, this is my impression. The fact that we have an old blockchain isn't the problem- it is guaranteeing that enough people upgrade.
Ok. I'm not a dev so I'm very cautios in suggesting anything. This is more out of curiosity and for understanding. But would it be technically possible to start over from scratch with a new block chain and altered protocol that allows P2SH in a clean manner and put all the current owners of bitcoins in the genesis block?

I'm just a user of bitcoin so I know my voice doesn't have the most weight. But I do see the value in the features that P2SH provides but hackish solutions still make me very wary, and I'm probably not the only one. So if it's at all possible to implement this in a clean manner that would put a lot of my worries to rest, even if it would be more cumbersome to transition.

And I don't really want to go into the dangers of such a transition with the merkle tree and whatever. I'm just wondering if it's technically possible, or if a clean implementation of P2SH is incompatible with the current adress syntax even with a new block chain.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: [Tycho] on February 02, 2012, 11:44:44 PM
Also I would like to say that at least partially all this mess is caused by the opcodes that Satoshi disabled more than a year ago.

I think that multisigs with short addresses can't be implemented without P2SH because OP_CAT op is disabled. May be.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Atheros on February 03, 2012, 12:15:46 AM
RE: 2_Thumbs_Up

That new 'genesis block' would be just as big as the current pruned blockchain, so there is no point in doing it that way. Luckily there is no need: clean and simple P2SH would be easy to do with the current blockchain and transactions. The only challenge would be, apparently, the network-wide upgrade process.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: westkybitcoins on February 03, 2012, 06:52:13 PM
At the risk of rehashing a settled issue, I have a stupid question:

It is already cryptographically possible to have two or more devices each have access to a portion of a private key, and be able to combine these portions to spend funds in such a way that no device gains access to any more of the private key than it already had, correct?

If this is the case, I'm wondering why it's considered mission-critical to implement scripting for multisig and for extra security measures in general by altering the protocol, rather than letting the owners of the private keys take responsibility for securing them and preventing unauthorized spending.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Luke-Jr on February 03, 2012, 07:02:33 PM
It is already cryptographically possible to have two or more devices each have access to a portion of a private key, and be able to combine these portions to spend funds in such a way that no device gains access to any more of the private key than it already had, correct?
AIUI, no.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: fivemileshigh on February 03, 2012, 07:55:04 PM
* Disclaimer: I don't think that BIP17 is better than BIP16. Both are ugly hacks. I will support one only if most other miners will.

Tycho, I mean this with all the kindness and respect I can possibly muster:

If you think both are ugly, make your case and stand by it. "Just doing what everybody else is doing" is groupthink. Don't be a sheep. Nothing good or worth having lies that way.




Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: ByteCoin on February 04, 2012, 12:45:20 AM
It is already cryptographically possible to have two or more devices each have access to a portion of a private key, and be able to combine these portions to spend funds in such a way that no device gains access to any more of the private key than it already had, correct?
AIUI, no.
Yes it's possible. As far as I can recall, you need to use the additive homomorphic property of the Pallier scheme. It has been discussed on the forum before but I can't find the reference.

ByteCoin


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: casascius on February 04, 2012, 12:57:03 AM
RE: 2_Thumbs_Up

That new 'genesis block' would be just as big as the current pruned blockchain, so there is no point in doing it that way. Luckily there is no need: clean and simple P2SH would be easy to do with the current blockchain and transactions. The only challenge would be, apparently, the network-wide upgrade process.

This may belong in another thread, but I have heard this point made, and I am not sure I agree with it, just doing a mental estimation in my head.

A new genesis block would be the same size as the current pruned blockchain, minus all of the following VERY BIG wastes of space:
  • Transaction inputs (these are HUGE: 130+ bytes each, compared to outputs around 30 bytes each) - and are worthless information except for their verification value... when multisig arrives, typical inputs will be double and triple this per transaction.
  • Stubs left behind when you prune merkle trees (since you need the full hash of the branches you pruned to be able to verify the hash of what's left of tree, which in many cases these hashes are a decent fraction in size of the transactions they replace)
  • Spent outputs of multi-output transactions (which can't be pruned if ANY unspent outputs remain - especially bloatful when you consider, for example, that pools like P2Pool generate huge transactions with numerous penny-sized outputs to pay off miners, of which most but not all get spent... which directly means that most of the space they take up is a waste).

Remove all of this fluff from a pruned block chain, make a "regenesis" block consisting of nothing but unspent outputs, and I bet it's less than a third the size of even a pruned block chain, and less than a tenth or twentieth the size of our current chain (a disparity that increases toward infinity as bitcoin is used - by 2013, this disparity might quadruple).


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: finway on February 04, 2012, 03:47:13 PM
Gonna read this long post someday.


Title: Re: The Truth behind BIP 16 and 17 (important read)
Post by: Atheros on February 05, 2012, 06:10:17 PM
Remove all of this fluff from a pruned block chain, make a "regenesis" block consisting of nothing but unspent outputs, and I bet it's less than a third the size of even a pruned block chain, and less than a tenth or twentieth the size of our current chain (a disparity that increases toward infinity as bitcoin is used - by 2013, this disparity might quadruple).

Ok, I definitely concede that it would be smaller. All of those things individual miners can do on their own. The chain wouldn't be verifiable anymore but we basically threw that condition out of the window when we made a re-genesis block. I'm sure miners will be doing this before long as long as the full block chain is still available in a distributed manner.