Miners that refuse to include transactions are becoming a problem

[kano]

Meanwhile ... some pools send an LP with no txn to speed up the fact that they need to send an LP to every miner in the pool as quickly as possible when a block is found.

e.g. it's not hard to find a quick DeepBit block with no txns.

So yeah your not gonna get too far with trying to enforce rules like this silly one.

[Haplo]

The only purpose of miners in BTC is to include and record TX.

1: Any miner not including tx for technical reasons is a communist parasite collecting a welfare check and contributing nothing. These people should be excluded 100%. One day there will be 1000tx+ per block, and miners trying to take shortcuts to just win their block subsidy will fuck the network if they dump 1000tx per block on everyone else. This is especially true once people are actually paying a fee to get their tx into a block faster, since they will still end up paying a fee but will get nothing in return.

2: The blockchain is a contentious shared resource, the only purpose of which is to record and verify tx. Any miner who excludes all tx for fee reasons should be considered a monopolist and be excluded.

3: Because miners are a communist entitlement-sucking union, they should be forced to meet a quota in order to regulate how much control they have over prices. If miners are forced to include 10% of tx, then getting into that 10% spot means bidding on the price of those slots. This doesn't stop miners from choosing the 10% they decide to include, or from excluding everyone who doesn't pay their price assuming enough people are willing to do so.

Case in point: D&T ran out as fast as possible and commissioned fee software, calculating how much he has to charge people in order to regain 50BTC per block after the subsidy is cut down, as though miners "deserve" to be paid 50BTC per block in fees, regardless of the market value of 1BTC, regardless of the value of the service they provide, and regardless of what users are willing to pay (100% exclusion for those paying less than his fee). Because the blockchain is a time-limited, shared resource, if all miners adopt similar policies, either because of what they think they're entitled to or because they want to save bandwidth so they can find a block a few seconds before the next pool, then the whole blockchain might soon be empty of tx.

The same type of problem occurred in the whaling industry, and because individual whalers had no personal incentive to limit their hunting, collectively whalers nearly hunted all species to extinction. By the same token, a subsidy with no regulation is likely to lead to all miners destroying the currency, because it is in their stupid, short term self-interest to cut corners and extort high fees regardless of whether or not BTC collapses as a currency.

[DeathAndTaxes]

Haplo I demand you back up your LIES with a cite.

Please provide a specific cite where I stated:
a) fees should always remain the same regardless of the market value of BTC
b) miners "deserve" 50 BTC per block in fees
c) how much fees are necessary to regain 50 BTC per block.

You will learn reputation is earned here and with only 49 posts you are starting out as a blatant liar.  I expect that when you can't find evidence to back up your lies you will correct your blatantly false statements about me.  We can disagree on the facts but I won't stand by when some fraking noob start lying to my face.    Regardless of your actions the quote will remain so others can see how little your word is worth.


Quoted for prosperity how little value Haplo's word has:

Case in point: D&T ran out as fast as possible and commissioned fee software, calculating how much he has to charge people in order to regain 50BTC per block after the subsidy is cut down, as though miners "deserve" to be paid 50BTC per block in fees, regardless of the market value of 1BTC, regardless of the value of the service they provide, and regardless of what users are willing to pay (100% exclusion for those paying less than his fee).

[MoonShadow]

The only purpose of miners in BTC is to include and record TX.

This is wrong from the start.

[MoonShadow]

This says all that I need to know about your worldview.  The free market isn't theoretical at all, it's the default emergent order in the absence of coercion.  And capitalism isn't remotely the same thing.

The "absence of coercion", see that is what I really doubt can or will be possible anytime soon or maybe anytime at all, which makes its true implementation a theoretical exercise for me. You don't really know much about my worldview, but you are right, I don't buy into that "absence of coercion" scenario. That is an ideal.

Okay, you are entitled to your own opinion, even though reality doesn't care what my or your opinion may be.

[kano]

There is at least one more obvious and just as important purpose to mining ... securing the block-chain by generating new blocks.
You need both to happen since mining blocks with a million txn's will not help anything if you are not actually generating required difficulty blocks.

OK I guess I need to explain my previous post?

When a block is generated, every standard pool on the internet needs to send a message to every pool miner to start work on the new block and include the details of that work
This needs to get to the miners as quickly as possible to stop them wasting their precious mining resources on work that doesn't secure the block-chain, but rather switch to work that does help secure the block-chain.
Thus if the pool simply creates the smallest possible block and thus can process it as quickly as possible so it can send out these change work details as quickly as possible, then it makes sense to do that (this processing needs to be done repeatedly, for each miner)
As soon as that work is processed and if it is found to not provide a block, each miner will next work on a block header for the same block but that contains the transactions that the pool decided need to be processed.

So there is even a rather simple technical reason for there being blocks that only contain the coinbase txn and no other txn.

So, yeah, being god and saying everyone must do this coz you think it is right, doesn't really work that well in the real world where "science" decides

[JoelKatz]

Thus if the pool simply creates the smallest possible block and thus can process it as quickly as possible so it can send out these change work details as quickly as possible, then it makes sense to do that (this processing needs to be done repeatedly, for each miner)

What processing do you think the pool has to do per-miner that has anything to do with the size of the block?

[kano]

Thus if the pool simply creates the smallest possible block and thus can process it as quickly as possible so it can send out these change work details as quickly as possible, then it makes sense to do that (this processing needs to be done repeatedly, for each miner)

What processing do you think the pool has to do per-miner that has anything to do with the size of the block?

It must process the full block to produce the 80 byte header and the Midstate for each miner.
This processing is directly related to the number of txn's in the block - i.e. generation of the merkle root for the blocks merkle tree.

The tree root (part of the 80 bytes) is simply the transaction number of the coinbase for a block with no extra txns - thus requires no merkle tree processing

If you don't understand the merkle tree processing read here for starters:
https://en.bitcoin.it/wiki/Dump_format

[JoelKatz]

Thus if the pool simply creates the smallest possible block and thus can process it as quickly as possible so it can send out these change work details as quickly as possible, then it makes sense to do that (this processing needs to be done repeatedly, for each miner)

What processing do you think the pool has to do per-miner that has anything to do with the size of the block?

It must process the full block to produce the 80 byte header and the Midstate for each miner.

No, it doesn't.

This processing is directly related to the number of txn's in the block - i.e. generation of the merkle root for the blocks merkle tree.

Generating the merkle tress, less the coinbase transaction, need only be done once per block. Adding the coinbase transaction at the tip has constant expense regardless of what else is in the tree.

The tree root (part of the 80 bytes) is simply the transaction number of the coinbase for a block with no extra txns - thus requires no merkle tree processing

The processing would be done only once anyway, unless you wanted to include newer transactions.

If you don't understand the merkle tree processing read here for starters:
https://en.bitcoin.it/wiki/Dump_format

Once your merkle tree of transactions is build, adding a coinbase to it has constant cost regardless of the size of the tree.

I think the real reason miners are building blocks with no transactions is because they don't want to have to validate transactions and a block with an invalid transaction would be rejected. I don't think it has anything to do with generating work units.

[MoonShadow]

Thus if the pool simply creates the smallest possible block and thus can process it as quickly as possible so it can send out these change work details as quickly as possible, then it makes sense to do that (this processing needs to be done repeatedly, for each miner)

What processing do you think the pool has to do per-miner that has anything to do with the size of the block?

It must process the full block to produce the 80 byte header and the Midstate for each miner.

Doesn't have to.  Usually, the pool master does the merkle tree work and just passes the merkle root & the rest of the header to be hashed to the miners, so the miners don't see more with or without transactions.

[JoelKatz]

Thus if the pool simply creates the smallest possible block and thus can process it as quickly as possible so it can send out these change work details as quickly as possible, then it makes sense to do that (this processing needs to be done repeatedly, for each miner)

What processing do you think the pool has to do per-miner that has anything to do with the size of the block?

It must process the full block to produce the 80 byte header and the Midstate for each miner.

Doesn't have to.  Usually, the pool master does the merkle tree work and just passes the merkle root & the rest of the header to be hashed to the miners, so the miners don't see more with or without transactions.

We're talking about the work the pool master has to do per miner. My point is that it only has to build the merkle tree once. It can then inject a different coinbase transaction in without having to recompute the rest of the tree. (The coinbase is at the tip of the tree.)

[Haplo]

Haplo I demand you back up your LIES with a cite.

Please provide a specific cite where I stated:
a) fees should always remain the same regardless of the market value of BTC
b) miners "deserve" 50 BTC per block in fees
c) how much fees are necessary to regain 50 BTC per block.

You will learn reputation is earned here and with only 49 posts you are starting out as a blatant liar.  I expect that when you can't find evidence to back up your lies you will correct your blatantly false statements about me.  We can disagree on the facts but I won't stand by when some fraking noob start lying to my face.    Regardless of your actions the quote will remain so others can see how little your word is worth.


Quoted for prosperity how little value Haplo's word has:

Case in point: D&T ran out as fast as possible and commissioned fee software, calculating how much he has to charge people in order to regain 50BTC per block after the subsidy is cut down, as though miners "deserve" to be paid 50BTC per block in fees, regardless of the market value of 1BTC, regardless of the value of the service they provide, and regardless of what users are willing to pay (100% exclusion for those paying less than his fee).

Since all users are free to edit and/or delete their own posts, there is no way for me to prove absolutely what you said or did not say. That said, I can no longer find the post I was referring to, but that could be simply because I haven't looked far enough. Either way I can't find it on search and I don't feel like looking any further just to prove that you're a communist. Also, I could give a shit less what my post count is or what other people think.

However, there is something I'm curious about.

One related question.  Is it possible to have bitcoind create a tx but not broadcast it?  If so is it then part of the memorypool?  If not could bitcoind be modified to make it part of the memorypool?  Essentially I want the ability to include a tx in a block that hasn't been broadcasted (coin melting).

Is there a purpose for doing that besides intentionally destroying coins, or perhaps making a finney attack?

There is at least one more obvious and just as important purpose to mining ... securing the block-chain by generating new blocks.
You need both to happen since mining blocks with a million txn's will not help anything if you are not actually generating required difficulty blocks...

So, yeah, being god and saying everyone must do this coz you think it is right, doesn't really work that well in the real world where "science" decides

Securing tx and including tx are inseparable. Security with no tx and tx with no security are equally useless, since tx are ultimately what is being secured. We would never consider paying miners just to sit around spamming an empty blockchain, since that by itself serves no purpose.

That said, I can see that there may be other technical problems with the minimum inclusion proposal, but since everyone seems to have a different idea of how tx inclusion and mining pools work (which may be true, since there are different software versions), I can't even intelligibly decide whether I'm full of shit or not, however..

(1) If pool owners are purposefully excluding tx because it is inefficient for them in some way, that is entirely their fault, or possibly a shortcoming in the BTC protocol itself; but if that is the case, then shouldn't P2mining be impractical? If distributed mining isn't practical (especially now when there are only ~80tx per block), then how practical is BTC going to be when there are 1000tx+ per block? Otherwise what do we need major pools and pool operators for to begin with?

(2) If a pool owner is excluding tx because they can't keep the blockchain on their machines, we should force blockchain validation a la gmaxwell, assuming that his proposal (or something similar) would in fact solve the problem. I have my doubts.

(3) The monopolist/commons problem may or may not become relevant. A minimum inclusion requirement wouldn't necessarily solve that problem, either. The main problem is that, unlike a free market, only one "seller" gets to sell his product every ~10 minutes or so, no matter what the demand is, so there is an inherent limitation in price matching between buyers and sellers, which biases towards unilateral miner monopoly a bit more than in a "normal" market. If history serves as any reasonable guide, the next step is that monopolists arise (probably the owner of DeepBit, for that matter), then the next thing you know they own the currency and you've got full-on fascism all over again. An external fascist nation with enough resources (ie the USA) could easily produce the same result if they wanted to purposefully destroy the currency.

Some form of proof of stake might mitigate that possibility, but I don't know of any practical way to make that work. BTC combines both technological concerns along with economic concerns, which makes it far more complicated than any other currency, period. Considering it's less than 4 years old, it's not even clear that it will actually work at all at this point, but the sheer number of ways in which it could be attacked from within or without doesn't look very promising. Without tangible benefits over fiat or gold, there's no reason to use it at all. Anyone using BTC right now is really speculating on an experiment, which is fine for personal use, but unacceptable from a business standpoint.

Doesn't have to.  Usually, the pool master does the merkle tree work and just passes the merkle root & the rest of the header to be hashed to the miners, so the miners don't see more with or without transactions.

Makes sense. Then the question is "how consistently could a tx list be kept across different nodes on the network?"

D&T seems to think this would make tx inclusion less efficient (which it might depending on the mechanics of the inclusion quota) however I don't see how that could be the general case. As long as miners are using similar inclusion criteria, their lists should be very similar overall, so it should be possible to reach a reliable agreement over the network.

The way I see it, if a miner is happily accepting charity from their 25-50BTC block reward, paid by all BTC stakeholders and potential stakeholders, then they should happily accept a minimum quota of charity returned to those stakeholders to ensure the reliability and longevity of the network. It doesn't even require that they necessarily accept free tx, just as long as they include enough tx to prove that they're contributing something. The only outlier case is where the next block is found before any new tx are broadcast, which is possible, but there are ways of allowing such blocks to be accepted anyway.

[kano]

Thus if the pool simply creates the smallest possible block and thus can process it as quickly as possible so it can send out these change work details as quickly as possible, then it makes sense to do that (this processing needs to be done repeatedly, for each miner)

What processing do you think the pool has to do per-miner that has anything to do with the size of the block?

It must process the full block to produce the 80 byte header and the Midstate for each miner.

Doesn't have to.  Usually, the pool master does the merkle tree work and just passes the merkle root & the rest of the header to be hashed to the miners, so the miners don't see more with or without transactions.

We're talking about the work the pool master has to do per miner. My point is that it only has to build the merkle tree once. It can then inject a different coinbase transaction in without having to recompute the rest of the tree. (The coinbase is at the tip of the tree.)

OK I'm not correct, but neither are you

It has to process the complete left side of the tree and the depth of the tree is based on the number of Txn's being included.
It is not a constant.

Edit: also before the tree it has to generate a coinbase txn for each miner and after the tree the midstate for each miner

[DeathAndTaxes]

Since all users are free to edit and/or delete their own posts, there is no way for me to prove absolutely what you said or did not say. That said, I can no longer find the post I was referring to, but that could be simply because I haven't looked far enough. Either way I can't find it on search and I don't feel like looking any further just to prove that you're a communist. Also, I could give a shit less what my post count is or what other people think.

So you lied.  Good enough.

However, there is something I'm curious about.

One related question.  Is it possible to have bitcoind create a tx but not broadcast it?  If so is it then part of the memorypool?  If not could bitcoind be modified to make it part of the memorypool?  Essentially I want the ability to include a tx in a block that hasn't been broadcasted (coin melting).

Is there a purpose for doing that besides intentionally destroying coins, or perhaps making a finney attack?

It is right there in the quote.  The concept is called "coin melting".  It would render asinine ideas like "tainted coin lists" and "Bitcoin Police" beyond infeasible.  Currency must be fungible.  Having blacklists reduces the fungibility of currency and I see it as a threat to Bitcoin.

[MoonShadow]

Thus if the pool simply creates the smallest possible block and thus can process it as quickly as possible so it can send out these change work details as quickly as possible, then it makes sense to do that (this processing needs to be done repeatedly, for each miner)

What processing do you think the pool has to do per-miner that has anything to do with the size of the block?

It must process the full block to produce the 80 byte header and the Midstate for each miner.

Doesn't have to.  Usually, the pool master does the merkle tree work and just passes the merkle root & the rest of the header to be hashed to the miners, so the miners don't see more with or without transactions.

We're talking about the work the pool master has to do per miner. My point is that it only has to build the merkle tree once. It can then inject a different coinbase transaction in without having to recompute the rest of the tree. (The coinbase is at the tip of the tree.)

That's mostly true, but so what?  I don't understand the point you are trying to make?  The coinbase transaction is the one that all miners must make, and correctly, for their block to be valid at all, whether that is done by a pool master, a lone miner or a botnet.

[Haplo]

It is right there in the quote.  The concept is called "coin melting".  It would render asinine ideas like "tainted coin lists" and "Bitcoin Police" beyond infeasible.  Currency must be fungible.  Having blacklists reduces the fungibility of currency and I see it as a threat to Bitcoin.

Nope, still not following. I'm afraid I don't know what you mean by "memory pool". What does it mean for a tx to be in the memory pool? What does it mean for a coin to be included in the memory pool without being broadcast?

AFAIK a tx not broadcast will not be included in a block, and therefore is effectively not a tx. It sounds to me like you want miners to be able to include arbitrary tx which are neither received nor coinbase into their blocks, the only purpose of which that I'm aware of would be for executing a finney attack with the cooperation of a mining pool.

[MoonShadow]

It is right there in the quote.  The concept is called "coin melting".  It would render asinine ideas like "tainted coin lists" and "Bitcoin Police" beyond infeasible.  Currency must be fungible.  Having blacklists reduces the fungibility of currency and I see it as a threat to Bitcoin.

Nope, still not following. I'm afraid I don't know what you mean by "memory pool". What does it mean for a tx to be in the memory pool? What does it mean for a coin to be included in the memory pool without being broadcast?

AFAIK a tx not broadcast will not be included in a block, and therefore is effectively not a tx. It sounds to me like you want miners to be able to include arbitrary tx which are neither received nor coinbase into their blocks, the only purpose of which that I'm aware of would be for executing a finney attack with the cooperation of a mining pool.

This is correct.  If you create a transaction but don't broadcast it, it doesn't exist.  The same is true for a block.

[dooglus]

This is correct.  If you create a transaction but don't broadcast it, it doesn't exist.  The same is true for a block.

He wants to mine it himself, not broadcast it to other miners.

I'm guessing it's a transaction with tainted coins as input, and much less (or zero, if that's allowed) output value.  The difference between input value and output value goes to whoever mines the transaction, which D&T wants to be himself.  That way the tainted coins lose their taint (unless the taint calculation takes fees into account, of course).

[Polvos]

He wants to mine it himself, not broadcast it to other miners.

I'm guessing it's a transaction with tainted coins as input, and much less (or zero, if that's allowed) output value.  The difference between input value and output value goes to whoever mines the transaction, which D&T wants to be himself.  That way the tainted coins lose their taint (unless the taint calculation takes fees into account, of course).

Exact!

You can put all the tainted coins as fee or only a percentage in a transaction to another address you own.

This way you melt the tainted coins with the real transactions fee mined in the block and the block finding reward.

[MoonShadow]

This is correct.  If you create a transaction but don't broadcast it, it doesn't exist.  The same is true for a block.

He wants to mine it himself, not broadcast it to other miners.

I'm guessing it's a transaction with tainted coins as input, and much less (or zero, if that's allowed) output value.  The difference between input value and output value goes to whoever mines the transaction, which D&T wants to be himself.  That way the tainted coins lose their taint (unless the taint calculation takes fees into account, of course).

I can see how that might work to undermine blocklists, but selling the private keys works just as well.  Anyway, he might have to wait a long time to mine his one transaction.