Request to remove getmemorypool before it gets beyond being a release candidate

[kano]

I seems that an issue posted on the bitcoin git is not the correct way to do this.

I'll try the suggested method then

Looking for others to actually post their opinions on this of course.

I'll firstly look at it from a technical point of view:
The getmemorypool command is an interface to allow anyone to create any type of block they like that bitcoin will validate with very generic rules.
This allows someone to bypass the limited set of approved transactions that are within bitcoin and create non-standard and possibly non-functional transactions and put them permanently in the block-chain.

Is this where bitcoin is now heading?
No desire to attempt to enforce the types of transaction and only add new types with general consensus?

Of course others will bypass the defined transaction set with their own code, but I see "getmemorypool" as saying quite clearly - "Everyone do as you please to the block-chain."

However, for the actual reason why it has been created:
It was created to allow merged mining with namecoin, so the namecoin devs did not have to modify the bitcoin code each release.
It is also an issue in my opinion that the reason was not part of the pull request and clearly avoided.

The side effect namecoin merged mining has for the block-chain is to add around 46 extra bytes to each bitcoin block created with merged mining.

The namecoin web site also hides this fact by denying that any extra data is put into the bitcoin block-chain on their main page about merged mining http://dot-bit.org/Merged_Mining
I have requested around 2 weeks ago for that error to be corrected and no one has done so.

Not only does this sanction putting non-bitcoin, namecoin data in the bitcoin block chain, but it effectively says that any other alternate chain can do the same thing also.

The size of the bitcoin block-chain has already been discussed as being an issue, so why open the flood gates and sanction that anyone can add whatever they like to it?

I'd say that disallowing this commit would put out the word that bitcoin does not want any extra non-bitcoin data thrown in the bitcoin block-chain and that clearly seems to be an issue of note to some around.

The problem with going for a middle ground is that what amount of extra non-bitcoin data is considered OK?
If 46 bytes is OK, then why not 56 bytes or 106 bytes or ...
Clearly a line needs to be drawn, but I can't see any line making a statement other than to set the boundary at zero.

The normal generation block coinbase is 8 or 9 bytes - keep it at that.
Even putting a short pool name in there can be done within 8 or 9 bytes.

[Pieter Wuille]

The size of the coinbase is limited to at most 100 bytes, and at least 2 bytes. This means that by using the 98 free bytes for merged mining or whatever purpose, the effect on the block chain is limited to 5 megabyte per year.

[shads]

This is not required for the current merged mining implementation.  In fact it doesn't allow you to tell bitcoind what to put into the coinbase.  It's a read only operation.  I see where you're coming from but blocking this feature is not to going to stop people generating blocks with their own data.  There's nothing to stop someone collecting transactions by listening to the p2p protocol and generating their blocks that way.  This gives the opportunity to leverage code in bitcoind i.e. you can assume what's contained in the response of getmemorypool has passed bitcoind's validations but there's nothing stopping you from implementing this yourself or even accepting transactions with different criteria.

At the end of the day your concerns can't be addressed by blocking features in one client.  They need to be addressed at the protocol level.  If the protocol and therefore the majority of block generating clients define > n bytes in coinbase to be invalid then that's the way it is.  Simply preventing one client from generating blocks like that doesn't stop anyone else from doing it.

If you wanted to block merged mining for example all you'd have to do is create a patch to reject blocks with more than 39 bytes of arbitrary data in the coinbase.  Then get enough pools to represent 51% to run the patch.  It's not part of the official client but it becomes the defacto protocol by virtue of adoption.

[kano]

This is not required for the current merged mining implementation.  In fact it doesn't allow you to tell bitcoind what to put into the coinbase.  It's a read only operation.
...

Sorry, it is a bi-directional interface (vaguely similar to getwork)
However, the data sent back is the complete hex code of a block so it does enable anything to be put in there at all.
The block is given a very basic verification then accepted.

[Luke-Jr]

I seems that an issue posted on the bitcoin git is not the correct way to do this.

Seriously, quit trolling. It doesn't matter where.

I'll firstly look at it from a technical point of view:
The getmemorypool command is an interface to allow anyone to create any type of block they like that bitcoin will validate with very generic rules.

It's a mechanism to export part of bitcoind's functionality as an API so that external software can use bitcoind's implementation of transaction acceptance rules.

This allows someone to bypass the limited set of approved transactions that are within bitcoin and create non-standard and possibly non-functional transactions and put them permanently in the block-chain.

This mindset is extremely dictatorish. There is no organization/person to "approve" transactions allowed. The current status quo of the bitcoind developers controlling things in practice is a bad thing which needs to be solved by more implementations and miners/users stepping up to customize their own rules.

Is this where bitcoin is now heading?
No desire to attempt to enforce the types of transaction and only add new types with general consensus?

It better be. Replacing the Fed with Gavin isn't exactly a good thing (no offense, Gavin).

However, for the actual reason why it has been created:
It was created to allow merged mining with namecoin, so the namecoin devs did not have to modify the bitcoin code each release.
It is also an issue in my opinion that the reason was not part of the pull request and clearly avoided.

No, it was created for p2pool, which had previously been making blocks without any transactions at all, thus harming the Bitcoin network. getmemorypool gives p2pool access to transactions it knows are safe to put in blocks.

The side effect namecoin merged mining has for the block-chain is to add around 46 extra bytes to each bitcoin block created with merged mining.

29 actually.

The namecoin web site also hides this fact by denying that any extra data is put into the bitcoin block-chain on their main page about merged mining http://dot-bit.org/Merged_Mining
I have requested around 2 weeks ago for that error to be corrected and no one has done so.

It's a wiki: correct it yourself. The point (I presume) is that it isn't extra data. It's just part of the standard Bitcoin protocol being used. On the other hand, there is quite a bit of actually extra data required in the Namecoin blocks.

Not only does this sanction putting non-bitcoin, namecoin data in the bitcoin block chain, but it effectively says that any other alternate chain can do the same thing also.

Actually, the data put in the coinbase isn't even namecoin-specific. That same 29 bytes is designed to support up to 32 different aux chains, and could reasonably be extended to more in the future.

I'd say that disallowing this commit would put out the word that bitcoin does not want any extra non-bitcoin data thrown in the bitcoin block-chain and that clearly seems to be an issue of note to some around.

"Bitcoin" is not an entity, and Satoshi himself was the original innovator behind (and explicitly endorsed) merged mining.

The problem with going for a middle ground is that what amount of extra non-bitcoin data is considered OK?
If 46 bytes is OK, then why not 56 bytes or 106 bytes or ...
Clearly a line needs to be drawn, but I can't see any line making a statement other than to set the boundary at zero.

The line has been drawn in the Bitcoin protocol since the first block: there are 100 bytes for miners to put whatever arbitrary data they want into blocks. The original uses were (block 0) political propaganda and (bitcoind code) extranonce, but it was intentionally designed to be arbitrary for future extensions.

[Gavin Andresen]

Replacing the Fed with Gavin isn't exactly a good thing (no offense, Gavin).

None taken.  I've been trying pretty hard lately NOT to be The Central Authority, I don't want the job and I think the whole idea is contrary to the Zen of Bitcoin.

[kano]

The side effect namecoin merged mining has for the block-chain is to add around 46 extra bytes to each bitcoin block created with merged mining.

29 actually.

Interesting that someone has implemented this in Eligius' pool and you don't know the correct number.

It is 46 extra bytes except in the case of Eligius where it is a few more.

The coin base is usually 04 followed by a 4 byte number, then either 02 followed by a 2 byte or 03 followed by a 3 byte number.
Thus the basic coinbase is 8 or 9 bytes
e.g. block 148904 coinbase (8) : 045dee091a02a406 (.]......)

A merged bitcoin block 148895: (would have been 8 as you can see from the start of it)
coinbase (54) : 044ffd934e02590b522cfabe6d6dc6ce9152ca514033e47d58b38e7e9d89138c1def5fc5aad7d4b f69897dc151ff0100000000000000 (.O..N.Y.R,..mm...R.Q@3.}X..~......_.....i.}.Q.........)

A merged bitcoin block 148892: (would have been 9)
coinbase (55) : 045dee091a03ee9e00522cfabe6d6d3cabe85a239caf9fd955a96cb854c130827fc4a821b58366b 853bbbe023de58c0100000000000000 (.].......R,..mm<..Z#....U.l.T.0....!..f.S...=..........)

A merged Eligius bitcoin block 148882
coinbase (57) : 07456c69676975730308f8002c4d4d3d3daf19f6508599c1061e7969aa623f0437f4286b219d809 1bc49132be0951549b00100000000000000 (.Eligius....,MM==...P.....yi.b?.7.(k!....I.+...I.........)

[DeathAndTaxes]

The side effect namecoin merged mining has for the block-chain is to add around 46 extra bytes to each bitcoin block created with merged mining.

29 actually.

Interesting that someone has implemented this in Eligius' pool and you don't know the correct number.

It is 46 extra bytes except in the case of Eligius where it is a few more.

The coin base is usually 04 followed by a 4 byte number, then either 02 followed by a 2 byte or 03 followed by a 3 byte number.
Thus the basic coinbase is 8 or 9 bytes
e.g. block 148904 coinbase (8) : 045dee091a02a406 (.]......)

A merged bitcoin block 148895: (would have been 8 as you can see from the start of it)
coinbase (54) : 044ffd934e02590b522cfabe6d6dc6ce9152ca514033e47d58b38e7e9d89138c1def5fc5aad7d4b f69897dc151ff0100000000000000 (.O..N.Y.R,..mm...R.Q@3.}X..~......_.....i.}.Q.........)

A merged bitcoin block 148892: (would have been 9)
coinbase (55) : 045dee091a03ee9e00522cfabe6d6d3cabe85a239caf9fd955a96cb854c130827fc4a821b58366b 853bbbe023de58c0100000000000000 (.].......R,..mm<..Z#....U.l.T.0....!..f.S...=..........)

A merged Eligius bitcoin block 148882
coinbase (57) : 07456c69676975730308f8002c4d4d3d3daf19f6508599c1061e7969aa623f0437f4286b219d809 1bc49132be0951549b00100000000000000 (.Eligius....,MM==...P.....yi.b?.7.(k!....I.+...I.........)

Ok you win it is 46 to 49 bytes on average and at most 92 bytes.  It is a non-issue, at least for me.  

The blockchain is big and dealing with that will require innovation  to allow embedded and mobile devices to use bitcoin but 100 bytes here or there isn't going to do anything.  The average transaction is 300 bytes.  The average block is 50K and growing.  Hopefully someday we will have 20x, 50x, 100x as many transactions in a block.  Block size may reach 1MB.  A few bytes is trivial.  100 bytes less won't make the block chain small enough for mobile devices but a restriction here and restriction there MAY stifle innovation. 

There should be no central authority.  Miners can choose to exclude transactions, they are the gatekeepers.  Form a mining build called the purists and reject any "non-standard" transaction.  The way to solve this is through community outreach.  Convincing mining pools to exclude "bloated" transactions, or convincing people who bloat the transactions to pay for that cost.   Nobody should be deciding what bitcoin allows.  The rules were left intentionally open ended.  What I might consider worthless bloat today may be the killer app of tomorrow.

[Serith]

I'd say that disallowing this commit would put out the word that bitcoin does not want any extra non-bitcoin data thrown in the bitcoin block-chain and that clearly seems to be an issue of note to some around.

"Bitcoin" is not an entity, and Satoshi himself was the original innovator behind (and explicitly endorsed) merged mining.

I would like to know what exactly Satoshi said about merged mining, could you link the source?

[Luke-Jr]

https://bitcointalk.org/index.php?topic=1790.msg28696#msg28696
https://bitcointalk.org/index.php?topic=1790.msg28715#msg28715

[kano]

https://bitcointalk.org/index.php?topic=1790.msg28696#msg28696
https://bitcointalk.org/index.php?topic=1790.msg28715#msg28715
https://bitcointalk.org/index.php?topic=1790.msg28696#msg28696

Yeah that's 2 (same link in the 1st and 3rd)

Again, so that means what I said above:

Not only does this sanction putting non-bitcoin, namecoin data in the bitcoin block chain, but it effectively says that any other alternate chain can do the same thing also.

I don't care who Satoshi is, that is a bad idea.

(Satoshi left here 4 days after those comments and has never returned ...)

[Mike Hearn]

Kano, let me see if I can explain better.

In theory merged mining is supposed to add 33 bytes to the coinbase. I'm not sure why Vince chose to add some marker bytes, like I said on another thread, it's not necessary.

However, 33 bytes per block is not an issue. Satoshi did indeed endorse this as the right way to solve the problem. I suggest you read this page:

  https://en.bitcoin.it/wiki/Alternative_Chains

Let me summarize the arguments as to why merged mining is OK. The system was created to allow alternative systems based on the block chain algorithm to benefit from Bitcoins miners, without needing to put their transaction data into the block chain. The reason we don't want random-alt-chain transaction data in the Bitcoin blocks:

• The Bitcoin tx format is very specific to currency and is also very inflexible. It can be abused as a container of random data but it wasn't actually designed for that, and trying to use it as such leads to very obscure and poorly designed protocols.
• Bitcoin blocks and transactions are size-limited. Space used for DNS or whatever is space not used for financial transactions. That's a problem because ....
• Everyone who runs Bitcoin today has opted in to a specific set of features, and decided they're willing to donate their resources to it. If one alternative use case grows to a significant fraction of total resource consumption that implicit social contract is violated.

Merged mining solves all those problems. The amount of overhead is, and always will be, limited to ~33 bytes. It doesn't matter how much data exists in your alt chains, or how many alt chains there are, the impact to Bitcoin users is the same small number of bytes. This overhead is both trivial and - more importantly - entirely constant, compared to the cost of taking part in the Bitcoin protocol, and as such can be discarded.

Merged mining is the solution to people putting unrelated data in the chain, not the cause of it.

[finway]

Merged mining 。。。brilliant idea

[kano]

Kano, let me see if I can explain better.

In theory merged mining is supposed to add 33 bytes to the coinbase. I'm not sure why Vince chose to add some marker bytes, like I said on another thread, it's not necessary.

However, 33 bytes per block is not an issue. Satoshi did indeed endorse this as the right way to solve the problem. I suggest you read this page:

  https://en.bitcoin.it/wiki/Alternative_Chains

Let me summarize the arguments as to why merged mining is OK. The system was created to allow alternative systems based on the block chain algorithm to benefit from Bitcoins miners, without needing to put their transaction data into the block chain. The reason we don't want random-alt-chain transaction data in the Bitcoin blocks:

• The Bitcoin tx format is very specific to currency and is also very inflexible. It can be abused as a container of random data but it wasn't actually designed for that, and trying to use it as such leads to very obscure and poorly designed protocols.
• Bitcoin blocks and transactions are size-limited. Space used for DNS or whatever is space not used for financial transactions. That's a problem because ....
• Everyone who runs Bitcoin today has opted in to a specific set of features, and decided they're willing to donate their resources to it. If one alternative use case grows to a significant fraction of total resource consumption that implicit social contract is violated.

Merged mining solves all those problems. The amount of overhead is, and always will be, limited to ~33 bytes. It doesn't matter how much data exists in your alt chains, or how many alt chains there are, the impact to Bitcoin users is the same small number of bytes. This overhead is both trivial and - more importantly - entirely constant, compared to the cost of taking part in the Bitcoin protocol, and as such can be discarded.

Merged mining is the solution to people putting unrelated data in the chain, not the cause of it.

No, let me explain better.

You are saying you need to be able to put data in the coinbase.

Fine - so why have an interface that does WAY MORE than that?
This interface allows for doing anything you like in a block.
That is NOT what you have said.

Rewrite the interface to limit it to ONLY the coinbase and then it matches what you said (except again, it's 46 not 33 - there is no point arguing that coz 46 is correct - even if it isn't what you wanted - go look at the blockchain)

...
However, 33 bytes per block is not an issue. Satoshi did indeed endorse this as the right way to solve the problem. I suggest you read this page:

  https://en.bitcoin.it/wiki/Alternative_Chains

That was written 6 months after Satoshi left ... so using his name in reference to it is misleading ...

[Mike Hearn]

You're still confused and mixing up two unrelated things.

getmemorypool is not to do with merged mining. It's for the p2pool system. It does not let you do anything you can't already do if you're skilled enough to be a pool operator.

Merged mining would use the setauxwork RPC that is discussed elsewhere. It does indeed only set the coinbase.

You're worrying about a non issue.

[kokjo]

kano: fork the client, remove stuff you don't like, problem solved

[gmaxwell]

kano: fork the client, remove stuff you don't like, problem solved

I don't agree with his concerns here— merged mining is, in fact, the _cure_ to the problem he's worried about.  But your response is unreasonable and unfair.  Forking the client doesn't solve the problems with _other_ people throwing garbage in the blockchain which you are forced to store forever in order to participate in bitcoin.

[Luke-Jr]

kano: fork the client, remove stuff you don't like, problem solved

I don't agree with his concerns here— merged mining is, in fact, the _cure_ to the problem he's worried about.  But your response is unreasonable and unfair.  Forking the client doesn't solve the problems with _other_ people throwing garbage in the blockchain which you are forced to store forever in order to participate in bitcoin.

Neither does preventing mainline from doing things already allowed.

[gmaxwell]

kano: fork the client, remove stuff you don't like, problem solved

I don't agree with his concerns here— merged mining is, in fact, the _cure_ to the problem he's worried about.  But your response is unreasonable and unfair.  Forking the client doesn't solve the problems with _other_ people throwing garbage in the blockchain which you are forced to store forever in order to participate in bitcoin.

Neither does preventing mainline from doing things already allowed.

Enh, I dunno. Having to maintain a separately compiled fork is a hurdle.

Look at things like the deterministic wallet generator here that uses some crack ass and probably insecure novel cryptographic crap— most bitcoin users are insulated from insecure key generators, for the moment, by the fact that they can't import the keys that crap generates.    Not the most solid protection— but don't underestimate the power of the default.

But it's moot because the complain here is misguided...