I know this has been brought up before, but confirmation times are getting weird

[IsaacGoldbourne]

I've not noticed it, I've just noticed that confirmations have been more bunched (like 3 in 3 minutes and then 17 minute wait). I'm guessing this is because of the huge differences in hashpower between pools.

[1714899064]

1714899064

[1714899064]

1714899064

[DeathAndTaxes]

Or not.  The backlog was prior to Eligus making the change and Eligus is only ~15% of the network while confirmation times have risen 80%.   Tx volume is higher, blocks aren't, tx have to wait longer.

[MoonShadow]

Anybody know what's going on?

I think I might...

http://arxiv.org/pdf/1311.0243v2.pdf

http://threatpost.com/researchers-debate-value-of-new-bitcoin-attack

Looks like a mining pool has attempted this 'selfish mining' theory, to gain a profit advantage.

[DeathAndTaxes]

Selfish mining wouldn't change the number of tx confirmed.
One could engage in selfish mining actually make larger blocks and thus have less of a backlog.

The vast majority of significantly delayed tx are free.  The default block rules cap free tx at 27KB or ~60 transactions per block.  That is a mere 0.1 tps.  If people are creating more than 1 free tx every 10 seconds unless a miner decides to miner MORE than the default free tx which increases their orphan rate for exactly 0 BTC more revenue then there is going to be a backlog.  If the rate of free transaction creation rises then the backlog will get longer and longer and longer.

[Mondy]

This is actually quite surprising! Could there be a flaw?

[DeathAndTaxes]

This is actually quite surprising! Could there be a flaw?

https://bitcointalk.org/index.php?action=profile;u=151190;sa=showPosts
What is with the dozens and dozens of one line posts in the past two days?  
Trying to boost up your activity score to appear older and pull off a scam?

[MoonShadow]

Selfish mining wouldn't change the number of tx confirmed.

I agree, but I think that it could still push up the average time to confirm.  Transactions that were not yet available when the selfish pool found a block solution cannot be added to that block, but would likely be found in the public block.  Then if the selfish pool releases their hidden block, any transactions that made it into the public block but not the selfish block would have to re-enter the transaction queue.  But then there would be a delay, as most miners that accepted the public block would have already removed those transactions from their queues, and can't replace them until after that particular mining node had already discarded the public block as an orphan. 

What we are seeing here could be a side effect of selfish mining put into practice.

[DeathAndTaxes]

Selfish mining wouldn't change the number of tx confirmed.

I agree, but I think that it could still push up the average time to confirm.  Transactions that were not yet available when the selfish pool found a block solution cannot be added to that block, but would likely be found in the public block.  Then if the selfish pool releases their hidden block, any transactions that made it into the public block but not the selfish block would have to re-enter the transaction queue.  But then there would be a delay, as most miners that accepted the public block would have already removed those transactions from their queues, and can't replace them until after that particular mining node had already discarded the public block as an orphan.  

What we are seeing here could be a side effect of selfish mining put into practice.

Gotcha.  That is an interesting angle.  I don't think that is the case but it might be a good way to design a "selfish miner" warning system.


The average doesn't tell the whole story but a distribution would.
If the move from ~10 minutes to ~18 minutes is due to selfish miner what we should see is a reduction in the 0-10 minute times and an increase in the longer but relatively short times say 11-30 minutes.

My guess (and just a guess at this point) is that isn't what the distribution would look like.  What I think is we have a minority of very very old tx which bring up the average so it would look something like 90% of tx have a confirmation time in the 0-30 minute range and then you have and increased amount of tx in the long tail with confirmation times ranging from 31 to 1000 minutes.

[chriswilmer]

Selfish mining wouldn't change the number of tx confirmed.

I agree, but I think that it could still push up the average time to confirm.  Transactions that were not yet available when the selfish pool found a block solution cannot be added to that block, but would likely be found in the public block.  Then if the selfish pool releases their hidden block, any transactions that made it into the public block but not the selfish block would have to re-enter the transaction queue.  But then there would be a delay, as most miners that accepted the public block would have already removed those transactions from their queues, and can't replace them until after that particular mining node had already discarded the public block as an orphan. 

What we are seeing here could be a side effect of selfish mining put into practice.

This was one of my original thoughts when starting this thread, especially given that it started shortly after the selfish mining authors seemed keen on proving that they were right.

[niniyo]

Pools have an incentive to keep their blocks small so that they propagate fast, reducing their chance of being orphaned.  There is certainly no incentive to include free transactions, and low-fee transaction are probably also a negative prospect.  eg. adding an extra 500KB to a block for 0.1BTC might not be worth it : it adds 0.4% more returns to the 25BTC base reward, but if that means 3 seconds extra propogation time then it adds a 0.5% extra chance of being orphaned, so it would not be worth doing.

I've just made these numbers up based on my own assumptions of transaction size and propogation times, but hopefully you see the point.  Maybe it's the case that some pools have recently done the math and tweaked their thresholds to optimise their returns.

[DeathAndTaxes]

Pools have an incentive to keep their blocks small so that they propagate fast, reducing their chance of being orphaned.  There is certainly no incentive to include free transactions, and low-fee transaction are probably also a negative prospect.  eg. adding an extra 500KB to a block for 0.1BTC might not be worth it : it adds 0.4% more returns to the 25BTC base reward, but if that means 3 seconds extra propagation time then it adds a 0.5% extra chance of being orphaned, so it would not be worth doing.

I've just made these numbers up based on my own assumptions of transaction size and propagation times, but hopefully you see the point.  Maybe it's the case that some pools have recently done the math and tweaked their thresholds to optimise their returns.

Well if a pool was looking to maximize the short term revenue, block with the highest possible net revenue is one with no transactions (other than the coinbase).  A miner would simply mine only empty blocks and ignore all transactions even paying ones.  The best estimate (with current protocol) is that the orphan cost is about 3.3 mBTC per kB.  Paying tx are ~0.1 mBTC so the inclusion of any tx is a net loss of revenue.  That being said hopefully miners and pool ops realize that having a lot of coins doesn't do you much if you cripple the growth of Bitcoin.  2% less coins which are each worth 10x as much because you helped grow the adoption of Bitcoins is a pretty good deal.

One thing Gavin pointed out, is that if all miners have higher orphan rates they don't lose any revenue.  What matters is RELATIVE orphan rates.  If you have an orphan rate of 2% and the network on average has an orphan rate of 1% then you are losing 1% revenue.   However if you have an orphan rate of 5% and the network on average also has an orphan rate of 5% you aren't losing anything. 5% of your blocks are orphaned but so are everyone elses and as a result the difficulty is 5% lower.  The big miners should sit down and discuss mutually raising block sizes.   Even if you can't get everyone onboard the orphan costs can be reduced if miners agree to all raise block sizes.   Getting 50%, 60%, 70% of hashpower in agreement should seem possible as honestly that is just what getting the top 6 or 7 pools and major solo miners to find common ground?

Longer term a more efficient method of block propagation is possible which should reduce that orphan cost.  Today when a block is broadcast it all the tx inside the block are also broadcast as part of the block message.  Most nodes already have these txs so it is just wasted bandwidth.  One could instead include just the tx hash in the block message and that would cut the size of a block message by up to 80%.  For larger savings a reduced length hash could be used instead.  Collisions here are not a security risk and would still be incredibly rare and that would allow reducing the block message size (and thus propagation delay) by 95% or more. 

Still don't want to be all doom & gloom, even if nothing is done over time Moore's law will mean higher bandwidth at lower costs which will reduce the propagation delay.  Also the block subsidy being cut in half again in ~3 years will reduce the "distortion" that the large subsidy has no fee pricing.

[Technomage]

I'm with DeathAndTaxes on this issue. There is a REAL problem here. The problem is NOT only 0-fee tx that are being delayed by days in some cases, that is actually okay - free transactions have been charity which is finally over.

The problem is that I've been seeing transactions with the current standard fee of 0.0001 per kb delayed for multiple blocks, all the time now. That is the real issue. To really get priority now, you have to pay something like 0.001 per kb. That seems quite silly to me when pools are deliberately keeping the blocks small.

I don't know the correct solution to this issue, but it would be in the best interest of Bitcoin for large pool operators to come together and simply decide they will all start producing larger blocks in a more relaxed way. They don't need to start adding 0-fee tx but maybe decide that they will add standard fee tx all the way to 1MB? How is that for a quick solution?

If enough of the large pools agree to do that, the relative orphan rate will be similar across the board. Bitcoin wins, everyone wins.

[Minor Miner]

Perhaps a bunch of solo miners also have enacted parameters because they were sick of certain "large users of the blockchain" not paying any fare at all even though they were making fantastic profits.   Free Riding is great until the guy that pays for the gas gets tired of it.....

[DannyHamilton]

I'm with DeathAndTaxes on this issue. There is a REAL problem here. The problem is NOT only 0-fee tx that are being delayed by days in some cases, that is actually okay - free transactions have been charity which is finally over.

The problem is that I've been seeing transactions with the current standard fee of 0.0001 per kb delayed for multiple blocks, all the time now. That is the real issue. To really get priority now, you have to pay something like 0.001 per kb. That seems quite silly to me when pools are deliberately keeping the blocks small.

I don't know the correct solution to this issue, but it would be in the best interest of Bitcoin for large pool operators to come together and simply decide they will all start producing larger blocks in a more relaxed way. They don't need to start adding 0-fee tx but maybe decide that they will add standard fee tx all the way to 1MB? How is that for a quick solution?

If enough of the large pools agree to do that, the relative orphan rate will be similar across the board. Bitcoin wins, everyone wins.

There is already a financial incentive to do so.  Any pool that chooses not to include transactions that have paid fees while there is still room in the block is intentionally choosing not to get paid the maximum amount that they could.  If even 1 pool starts accepting these smaller fee transactions, they will be able to reward their members with higher payments than the other pools.  This will attract additional miners which will lead to even higher profits.

[DeathAndTaxes]

Well not exactly Danny.  The larger the block the larger the chance of orphans.  So as a hypthetical if you collect 2% more gross revenue but increase your chance of orphans 3% you actually (after accounting for losse due to orphan blocks) make less revenue.   The large block subsidy kinda distorts the economics of mining.

[niothor]

So after a few days where I thought times are getting back to normal it seems they are going back up.
For now there are only a few minutes , but we're talking about 50k transactions/day if those are the cause. Make them 10x and somebody will have to make those block changes.


Also , funny:
Height   Age   Transactions   Total Sent   Relayed By   Size (kB)
270900   2 minutes   382   5,327.14 BTC   GHash.IO   151.13
270900   2 minutes   369   5,037.41 BTC   BTC Guild   145.65
270899   5 minutes   313   7,642.39 BTC   Deepbit   138.48

[MoonShadow]

   The large block subsidy kinda distorts the economics of mining.

In which case, it's a self-correcting issue.  So long as that's what's going on.

[DannyHamilton]

Well not exactly Danny.  The larger the block the larger the chance of orphans.  So as a hypthetical if you collect 2% more gross revenue but increase your chance of orphans 3% you actually (after accounting for losse due to orphan blocks) make less revenue.   The large block subsidy kinda distorts the economics of mining.

I suppose, but seriously if miners want to squeeze every last drop of blood from the profitability stone, they should seriously look into implementing "child pays for parent".  I've had multiple transactions I've received where I would have been happy to pay a large enough fee on a child transaction to make sure that both my transaction and the parent transaction get included in the next block.

If they can't include a 1 kb transaction with a 0.0001 because of the cost of orphan risk, then they should absolutely be willing to include in their next block both the 1 kB transaction with zero fees that I received as well as the 1 kB child transaction with 0.002 BTC fees that I sent to increase their profits.

[chriswilmer]

Well not exactly Danny.  The larger the block the larger the chance of orphans.  So as a hypthetical if you collect 2% more gross revenue but increase your chance of orphans 3% you actually (after accounting for losse due to orphan blocks) make less revenue.   The large block subsidy kinda distorts the economics of mining.

I suppose, but seriously if miners want to squeeze every last drop of blood from the profitability stone, they should seriously look into implementing "child pays for parent".  I've had multiple transactions I've received where I would have been happy to pay a large enough fee on a child transaction to make sure that both my transaction and the parent transaction get included in the next block.

If they can't include a 1 kb transaction with a 0.0001 because of the cost of orphan risk, then they should absolutely be willing to include in their next block both the 1 kB transaction with zero fees that I received as well as the 1 kB child transaction with 0.002 BTC fees that I sent to increase their profits.

Posts on the Bitcoin Foundation forum by Gavin, Mike Hearn and Luke seem to imply that mining pool operators are just not paying much attention to this one way or another. They are using some default settings for accepting transactions and just focusing on making sure they find blocks... I have no idea how true this is.

[MikeyVeez]

Well not exactly Danny.  The larger the block the larger the chance of orphans.  So as a hypthetical if you collect 2% more gross revenue but increase your chance of orphans 3% you actually (after accounting for losse due to orphan blocks) make less revenue.   The large block subsidy kinda distorts the economics of mining.

What's with your bend over fest with Bitcoin? Nobody gives a fuck that 90% empty blocks are still being made every 10min. What people care about is that transactions are much slower than usual,  why must you take so much bitcoin penis?  Bitcoin is getting messed up by higher activity which it cant cope with.