I know this has been brought up before, but confirmation times are getting weird

[MikeyVeez]

Well not exactly Danny.  The larger the block the larger the chance of orphans.  So as a hypthetical if you collect 2% more gross revenue but increase your chance of orphans 3% you actually (after accounting for losse due to orphan blocks) make less revenue.   The large block subsidy kinda distorts the economics of mining.

I suppose, but seriously if miners want to squeeze every last drop of blood from the profitability stone, they should seriously look into implementing "child pays for parent".  I've had multiple transactions I've received where I would have been happy to pay a large enough fee on a child transaction to make sure that both my transaction and the parent transaction get included in the next block.

If they can't include a 1 kb transaction with a 0.0001 because of the cost of orphan risk, then they should absolutely be willing to include in their next block both the 1 kB transaction with zero fees that I received as well as the 1 kB child transaction with 0.002 BTC fees that I sent to increase their profits.

Posts on the Bitcoin Foundation forum by Gavin, Mike Hearn and Luke seem to imply that mining pool operators are just not paying much attention to this one way or another. They are using some default settings for accepting transactions and just focusing on making sure they find blocks... I have no idea how true this is.

What we can gather from this is Bitcoin sucks, and there will never be mass adaption because it's broken. Gavin and the foundation suck as they have known about this for years and never bothered to fix or address it. Bitcoin its on its final bubble stage before it spazes and implodes upon itsself.

The final stage can of course take it to thousands of dollars and still last a few years, but we are now in its final leg instead of the beginnings.

[phillipsjk]

I thought of a way to reduce orphans: more full nodes on the network. Unfortunately, non-mining full nodes are not rewarded directly by the protocol. Also, most "miners" are not full nodes: they simply rely on pools.

More nodes reduce orphans by speeding up block propagation:

• By having more aggregate CPU power verifying transactions. Slow CPUs probably don't help too much for preventing orphans.
• By having more aggregate bandwidth for relaying transactions and blocks.
• More nodes make any successful DDOS more expensive to run.

My personal Bitcoin node has been in the planning stages for months. When up, it will have an average bandwidth of ~463kbps (burstable to 5Mbps). Calculation: (300GB/month(cap))/(30days/month)/(24hours/day)/(3600seconds/hour)*(8bits/byte)/2(symmetric bandwidth usage)

If I get a second (or better paying) job, I will consider renting a dedicated server with ~7.7Mbps (burstable to 100Mbps) (5TB cap)

Edit2: with 5Mbps burstable bandwidth, I may want to limit the block-size to 256kB so it has a chance of being transmitted in less than 1 second. With 8 connections, it would take 3.3 seconds to broadcast to all 8.
Edit: read page 4: sending only block headers sounds like it just might work (assuming the node has seen all the transactions)

[Trongersoll]

I thought of a way to reduce orphans: more full nodes on the network. Unfortunately, non-mining full nodes are not rewarded directly by the protocol. Also, most "miners" are not full nodes: they simply rely on pools.

More nodes reduce orphans by speeding up block propagation:

• By having more aggregate CPU power verifying transactions. Slow CPUs probably don't help too much for preventing orphans.
• By having more aggregate bandwidth for relaying transactions and blocks.
• More nodes make any successful DDOS more expensive to run.

My personal Bitcoin node has been in the planning stages for months. When up, it will have an average bandwidth of ~463kbps (burstable to 5Mbps). Calculation: (300GB/month(cap))/(30days/month)/(24hours/day)/(3600seconds/hour)*(8bits/byte)/2(symmetric bandwidth usage)

If I get a second (or better paying) job, I will consider renting a dedicated server with ~7.7Mbps (burstable to 100Mbps) (5TB cap)

Edit: read page 4: sending only block headers sounds like it just might work (assuming the node has seen all the transactions)

Isn't every Wallet a node, providing that they are using bitcoin-QT?

[MoonShadow]

I thought of a way to reduce orphans: more full nodes on the network. Unfortunately, non-mining full nodes are not rewarded directly by the protocol. Also, most "miners" are not full nodes: they simply rely on pools.

More nodes reduce orphans by speeding up block propagation:

• By having more aggregate CPU power verifying transactions. Slow CPUs probably don't help too much for preventing orphans.
• By having more aggregate bandwidth for relaying transactions and blocks.
• More nodes make any successful DDOS more expensive to run.

My personal Bitcoin node has been in the planning stages for months. When up, it will have an average bandwidth of ~463kbps (burstable to 5Mbps). Calculation: (300GB/month(cap))/(30days/month)/(24hours/day)/(3600seconds/hour)*(8bits/byte)/2(symmetric bandwidth usage)

If I get a second (or better paying) job, I will consider renting a dedicated server with ~7.7Mbps (burstable to 100Mbps) (5TB cap)

Edit: read page 4: sending only block headers sounds like it just might work (assuming the node has seen all the transactions)

Isn't every Wallet a node, providing that they are using bitcoin-QT?

Yes, Bitcoin-QT is a full node.

[phelix]

Well if a pool was looking to maximize the short term revenue, block with the highest possible net revenue is one with no transactions (other than the coinbase).  A miner would simply mine only empty blocks and ignore all transactions even paying ones.  The best estimate (with current protocol) is that the orphan cost is about 3.3 mBTC per kB.  Paying tx are ~0.1 mBTC so the inclusion of any tx is a net loss of revenue.

Are your numbers from Gavin's calculation? https://gist.github.com/gavinandresen/5044482  Back-of-the-envelope calculations for marginal cost of transactions  (almost the same result, 3.2mBTC/kB)

I guess we will see much higher TX fees then...

[MoonShadow]

Well if a pool was looking to maximize the short term revenue, block with the highest possible net revenue is one with no transactions (other than the coinbase).  A miner would simply mine only empty blocks and ignore all transactions even paying ones.  The best estimate (with current protocol) is that the orphan cost is about 3.3 mBTC per kB.  Paying tx are ~0.1 mBTC so the inclusion of any tx is a net loss of revenue.

Are your numbers from Gavin's calculation? https://gist.github.com/gavinandresen/5044482  Back-of-the-envelope calculations for marginal cost of transactions  (almost the same result, 3.2mBTC/kB)

I guess we will see much higher TX fees then...

If that is what it takes to get miners to pay attention to transactions, that's what it takes.

[zebedee]

Confirmation time =/= block time.

Average time between blocks is <10 minutes.  I pointed this out in the prior thread but it will likely get ingored again.  Any "answer" which involves miners solving less blocks would mean the time between blocks would be greater than 10 minutes and that is not the case.

Miners are on average producing a block size of ~160KB.
http://blockchain.info/charts/avg-block-size

To clear the backlog and reduce the average wait time to ~1 block would require blocks to be roughly 50% larger.  Miners are chosing not to do that.  The average block has ~300 tx vs the ~2,500 tx limit imposed by the 1MB or ~600 tx it would take to clea the backlog

Simple version: blocks are 90% empty, tx wait longer to be included in a block

A free market is emerging in transaction fees!  Imagine that.  Clearly miners have, to some extent collectively, decided to delay including "marginal" zero-fee txns in blocks.  I think by 24-48 hrs in some cases.  They still get in eventually.

This is good and healthy.  All the whiners are just upset because the freebie has been reduced; whining is to be expected.

What I'm wondering is where is MisterBigg?  He was the dude saying if we removed the blocksize limit then miners would be stuffing everything into one block, even things paying 1 satoshi.  I think that idea has been thoroughly debunked, particularly recently - his school of thought cannot explain the current phenomenon, and we're nowhere near the 1MB limit.  More evidence we should just get rid of it.

[Technomage]

What I'm wondering is where is MisterBigg?  He was the dude saying if we removed the blocksize limit then miners would be stuffing everything into one block, even things paying 1 satoshi.  I think that idea has been thoroughly debunked, particularly recently - his school of thought cannot explain the current phenomenon, and we're nowhere near the 1MB limit.  More evidence we should just get rid of it.

Indeed it seems that the issue of orphan blocks alone will stop miners from creating blocks that are too large in size. The fee market will be more based on that than the enforced limit.

[DeathAndTaxes]

What I'm wondering is where is MisterBigg?  He was the dude saying if we removed the blocksize limit then miners would be stuffing everything into one block, even things paying 1 satoshi.  I think that idea has been thoroughly debunked, particularly recently - his school of thought cannot explain the current phenomenon, and we're nowhere near the 1MB limit.  More evidence we should just get rid of it.

Indeed it seems that the issue of orphan blocks alone will stop miners from creating blocks that are too large in size. The fee market will be more based on that than the enforced limit.

Agreed.  It is something I hadn't considered but the "orphan cost" does constrain block size.  I do think retaining a hard cap for the interim future (possibly raised to 5 or 10 MB) is a good thing.  A malicious actor (and thus not interested in orphan cost) could do some damage to adoption by bloating the blockchain with very large blocks.

[tvbcof]

Just as a point of reference:

I broke open one of my savings wallets last night for the first time in a year it being a monster hassle to retrieve them.

I'm working on a source built bitcoind from around April 2013 codebase, and my client is only caught up to block 25k-ish.  I'm on a satellite connection with limited monthly quota, so I only run the client when I need to.

I transferred all the coins out of the wallet (100BTC) directly to my Coinbase account and two blockchain.info accounts.  I have bitcoind set to pay .001 transaction fee.

 - The first transaction (to Coinbase) was visible within a minute on blockchain.info.

 - The second transaction (to blockchain.info) took many minutes to show up.  It seemed to show right after a block was mined.

 - The third transaction (also to blockchain.info) happened fairly quickly as I recall.  A few minutes perhaps.

---

I've never believed that Bitcoin is a appropriate tool for dealing with small purchases for coffee or whatever.  I want something reliable and independent (from corporate and government interference) for large value transactions.  .001 BTC on a 50BTC transaction is a price I'm perfectly happy to pay for good service.  It is

 - vastly less than I pay for international wires,
 - and is much faster,
 - more hassle free,
 - more private,
 - and more reliable (e.g., Mt. Gox cannot get my $5k international wire to me after more than a quarter.)

So far Bitcoin is has served me well for my needs here.

[tvbcof]

Isn't every Wallet a node, providing that they are using bitcoin-QT?

Yes, Bitcoin-QT is a full node.

It isn't mentioned much, but my understanding is that it's a fairly useless full node if one isn't running UPnP or has set up a port forward.  Just sucks out without giving back.  For better or worse, many people probably don't even know what this means, or even that they are doing UPnP since lots of routers have it on as a default setting.  I don't trust UPnP since I didn't compile it (or more accurately, I didn't compile the OS on my phone, printer, etc), and I turn it off.

[BurtW]

I was under the impression that UPnP was just one of several discovery methods in the code.  With your UPnP turned off is your full node client still able to discover and connect to other nodes?  If so then it is not necessary for node discovery.

[tvbcof]

I was under the impression that UPnP was just one of several discovery methods in the code.  With your UPnP turned off is your full node client still able to discover and connect to other nodes?  If so then it is not necessary for node discovery.

One's own process (bitcoin[-qt|d]) discovering and connecting to services outside one's internal network is not a problem.  Firewalls typically allow this without restriction.  The problem is if someone discovers your, can they connect to you?  The only thing they will know is the IP or your router (port 8333 can be inferred.)  When they hit that, the router need to forward the message to the appropriate device within one's internal network.

Discovery and connection are two different problems.  Bitcoin used to use IRC for discovery back when I started, but I think that is fully deprecated.  The current bitcoin.it wiki page on running Bitcoin doesn't say much about UPnP except that there is a flag.

There are some conversation about this here and there, but not as many as I would have expected.  I kind of feel that perhaps a development decision was made that enough routers have UPnP on be default that many people will be running the code in 'good citizen' mode whether they know it or not, and talking about it will mostly just scare people.

[MoonShadow]

No, no.  Runing a full node that is 'quiet' is not "taking without giving back".  Dark nodes are a certainty that not all nodes are discoverable.  This is a feature, not a bug.  There are attack vectors, against the p2p network model itself, that are undermined by the very concept that dark nodes can (and do) exist.  They become a repository of the true blockchain, should (as an example) a viral vector is discovered within the network itself, permitting a malicious player to exploit all visable nodes at network speeds.  Dark nodes (probably) promise some degree of insulation from such an exploit, as well as the certainty that the attacker cannot ever be certain of success of such an attempt, since it's not verifiable.  The fact that some nodes are discoverable and others are not isn't a problem.

[Peter Todd]

What I'm wondering is where is MisterBigg?  He was the dude saying if we removed the blocksize limit then miners would be stuffing everything into one block, even things paying 1 satoshi.  I think that idea has been thoroughly debunked, particularly recently - his school of thought cannot explain the current phenomenon, and we're nowhere near the 1MB limit.  More evidence we should just get rid of it.

AFAIK I'm the one who pointed out that incentive first; MisterBigg either independently realized the same thing or was repeating me.

Either way, I did my analysis more carefully here and found that the incentive to stuff blocks full of garbage was even stronger than I originally thought. However that incentive only is true under specific circumstances that are not present in the current Bitcoin system, but will be in the future. Specifically doing that decreases the number of blocks you find per unit hashing power, but it decreases that number even more for your competitors - right now blocks are themselves worth a lot of money so there's no reason to use that strategy, but in the future when mining income is mainly from transaction fees the strategy appears to make sense.

tl;dr: the idea is anything but debunked, it's just not yet relevant.

[phelix]

What I'm wondering is where is MisterBigg?  He was the dude saying if we removed the blocksize limit then miners would be stuffing everything into one block, even things paying 1 satoshi.  I think that idea has been thoroughly debunked, particularly recently - his school of thought cannot explain the current phenomenon, and we're nowhere near the 1MB limit.  More evidence we should just get rid of it.

AFAIK I'm the one who pointed out that incentive first; MisterBigg either independently realized the same thing or was repeating me.

IIRC he gave completely different reasons that were proven wrong by Gavin's paper showing that including TXs is quite expensive due to orphan costs.

The orphan cost of including a TX for a standard 500byte TX is now a whopping $1,30

[asdf]

Is anything being done about this?

Since transaction fees are proportional to the BTC price, this amounts to a cap on bitcoin adoption; less people will use bitcoin as the tx fee becomes uncompetitive. We are already PAST this point, imo. I feel that there is some urgency to fix this problem now.

The obvious solution is to reduce the orphan cost. To do this solved-block propogation time needs to be lowered. I don't really understand it, but could CoinWitness be the key solution? https://bitcointalk.org/index.php?topic=277389.0;all

I don't know of anything in the development pipeline that addresses this issue.

[MoonShadow]

Is anything being done about this?

Yes.

The obvious solution is to reduce the orphan cost. To do this solved-block propogation time needs to be lowered.

This is being worked on by migrating from a complete block to a skeleton block, with only the headers and merkle tree included. This alone would reduce the orphan risk cost by about a factor of 10.

I don't really understand it, but could CoinWitness be the key solution? https://bitcointalk.org/index.php?topic=277389.0;all

Not for Bitcoin.

[asdf]

Is anything being done about this?

Yes.

The obvious solution is to reduce the orphan cost. To do this solved-block propogation time needs to be lowered.

This is being worked on by migrating from a complete block to a skeleton block, with only the headers and merkle tree included. This alone would reduce the orphan risk cost by about a factor of 10.

Thanks for the reply.

According to the above analysis, the fees are already about 1/30th of what they should be for miners to break even. The skeleton block solution gets us to 1/3. That's just at the current price.

Don't you think this is a big problem? I assume that, at some point, economic reality will kick in and miners will start to demand much higher fees. Suddenly bitcoin isn't so great compared to alternatives.

[DeathAndTaxes]

Is anything being done about this?

Yes.

The obvious solution is to reduce the orphan cost. To do this solved-block propogation time needs to be lowered.

This is being worked on by migrating from a complete block to a skeleton block, with only the headers and merkle tree included. This alone would reduce the orphan risk cost by about a factor of 10.

Thanks for the reply.

According to the above analysis, the fees are already about 1/30th of what they should be for miners to break even. The skeleton block solution gets us to 1/3. That's just at the current price.

Don't you think this is a big problem? I assume that, at some point, economic reality will kick in and miners will start to demand much higher fees. Suddenly bitcoin isn't so great compared to alternatives.

Not really, I recommend reading the entire thread.  Switching to hashes only is more like a 20x reduction in size.   Also the fees being 1/30th of "orphan cost" is just one estimate.  Other estimates put it closer to 1/10th.  Miners also receive a very large subsidy and they have an indirect incentive to keep the network going (a few less Bitcoins which are worth something is more than a few more worthless Bitcoins).  Also Moore's law is alive and well and it applies to bandwidth as well.

Still even if fees rose 300% (which I strongly doubt) that would make the cost of a Bitcoin tx something on the order of $0.20 which compares very well to other payment methods.   What I think is more likely is miners dropping or reducing the number of free transactions they support and maybe the min fee doesn't go down.   That combined with the subsidy, improvement to block propogation, and the power of Moore's law will likely keep tx fee costs below $0.10 or so.