Bitcoin Forum

There seems to be a substantial backlog (6000) of Bitcoin transactions building up thanks to the increasing number of users taking to the technology (https://bitcointalk.org/index.php?topic=338999.0) and numerous questions in the [Technical Support] board (https://bitcointalk.org/index.php?board=4.0) over why transactions are seemingly now stuck in Limbo even when they include a fee.

Suggestions have been mooted such as "include a higher fee" and "be more generous", but how does this work in the context of most users? The average person will do what their wallet tells them. They'll have no idea what constitutes a suitable fee for inclusion in a block and when they realize that even if they include the required 0.0001 BTC minimum fee, this puts them in the same position and requires they still wait 12-48 hours for their transaction to confirm - it's going to raise serious questions over Bitcoin's scalability...

In addition, according to CoinMill - the cost of 1 BTC is currently 641.84 USD and a transaction fee of 0.0001 BTC is 0.06 USD (6 cents). 0.0005 (the old fee) is 30 cents. As Bitcoin appreciates (if it does), the cost of transactions relative to the value of a Bitcoin grows substantially.

This makes it pretty uneconomical for any merchant to engage in using Bitcoin as a currency as A) they'll be waiting ages for their transaction to confirm if the user incorrectly attaches an 'unacceptable' fee and B) the cost of moving BTC is likely to rapidly meet or exceed that of preexisting payment methods such as ACH, Debit and eventually Credit cards... and in the case of the latter two it doesn't generally take 24-48 hours to even acknowledge your transaction exists.

Are there any solutions to this conundrum?

This is way Bitcoin was always meant to work: more fees = faster confirmations; less fees = slower confirmations. At first, when transaction volume was lower, it didn't work that way; most or all transactions were confirmed quickly regardless of fees, and apparently some people were fooled into thinking that was normal, and now that the fee system is working properly, people are complaining that it's broken.

This can easily be fixed by adding an "Urgency" option to the Send Coins tab, allowing users to easily select between "Urgent" and "Not Urgent" transactions, and hopefully showing the expected fee and confirmation time for each option. Average users will easily accept that they can pay more fees to get faster transactions (they already do for things like postage), we just need to make that option more accessible.

The fees can be lowered if the price rises too high. They have already been lowered several times for exactly this reason, and there's no reason to think it won't happen again.

EDIT: Typo

Current problem here is that Bitcoin price moves signifcantly more rapidly than fee amendments. In the last 72 hours, the price of one Bitcoin on BTC China peaked at circa 1,100 USD. Moving one BTC with the minimum fee of 0.0001 BTC would cost you 11 cents. Amazon pay less than that to process a credit card transaction. 

Transaction fees are likely to go up (in dollar terms) for a while, until some engineering work is done to reduce the "orphan cost" for miners to include more transactions in their blocks OR mining pools / miners collectively agree to include more transactions for the good of the whole system.

In the very short term, you can ask mining pool operators to create larger blocks. If they refuse, then switch your miners to a pool that does.

If they all create larger blocks, then we get more transactions and more orphan blocks, but the cost of those extra orphan blocks is spread across everybody mining, so everybody gets just as many bitcoins (on average) as they would with smaller blocks.

What's the problem? That the value of your holding increases thousands of dollars while you have to pay an additional fee of only several cents per transaction? The Amazon example is red-herring, the cost has already been included in the price of the goods, either willingly or unwillingly, and dollar never fluctuates as much.

Sometimes I feel that it's not only the Wall street bankers who are snobbish and stingy.

You are taking things out of context.

Please show me a single user that
a) is using standard transactions
b) paid the MINIMUM fee
c) didn't create a tx w/ unconfirmed inputs (which have to be confirmed before the tx can confirm)

and has a tx stuck for hours.

The fee is currently 0.1 mBTC (~$0.05).  While this is not free it is pretty much cheaper than any other payment system.
Credit Cards: $0.30 + 2%
PayPal: $0.30 + 3%
ACH: $0.25 to $0.50
Bank Wire:  $10 to $25
International Bank Wire: $25 to $40
Check processing (business): $0.50 ea
etc


Most users with tx "stuck" issues involve either
a) a free as in no fee at all
b) giant bloated dust spam tx that miners are reluctant to include in blocks
c) tx which have no fee issues like using unconfirmed outputs as inputs.

When the price stabilizes the min fee can be reduced.  It doesn't have to be exactly fixed in fiat terms.  Throughout the history of Bitcoin despite the exchange rate rising 60,000,000% the min fee has been ~0.5 cents to 10 cents per KB. 

The last min fee reduction happened when Bitcoin sustained an exchange rate >$100 and after the reduction the fee was ~1 US cent. 

I can't show this, but that isn't the issue. Since you *can* spend unconfirmed outputs - and this was done with the stock Bitcoin daemon - , it doesn't matter if your transaction includes a fee if one of the ancestors did not. Every downstream transaction is delayed as a consequence. This actually happened to this transaction: https://blockchain.info/tx/35c4e8c86075cf3f5335e029abd2d981af011c142b56e04c0a6d8b0d588d32ae created by the stock Bitcoin Daemon with the 'Sendtoaddress' call. This ancestor transaction https://blockchain.info/tx/99e6c22980571d1733986d950f564854393777ba0905d08aae36d7f4f64e4a3a included no-fee and as a consequence because it was picked as an input the downstream transaction was delayed. Both were confirmed +300 minutes after being broadcast... 5 hours. The downstream transaction included a fee.


Only... there is no minimum fee for some transactions - 0.0 is perfectly acceptable. My understanding of Transaction Fees is that they were probably conceived as a antidote to potential abuses of the transaction mechanism (spamming - you're less likely to attack something if it costs you money to do so). As for financial remuneration, the block reward was designed to suffice for miners until such time as it became inconsequential. Transaction fees were a bonus, but not the driver. Cherry picking is clearly happening and it's causing problems (Example: https://bitcointalk.org/index.php?topic=339871.0)


Again, your citing some arcane knowledge of the inner workings of Bitcoin to make this happen. The stock Bitcoin client happily uses unconfirmed outputs to generate new transactions. These transactions *will not confirm* until the ancestors have.

Consider also the scenario where a business receives and sends Bitcoin funds (same way as PayPal merchants do). If the business's wallet software decides to use an unconfirmed output as input from a client who paid using a 0 fee transaction as part of a downstream transaction or chain of downstream transactions and that client subsequently double spent the original transaction with a fee - all the downstream transactions would be invalidated if the subsequent fee including transaction were prioritized. This could mean significant financial losses for the business not to mention the nightmare of tracing and unwinding of the now invalid transactions.


Do you really believe Amazon, Wallmart, Best Buy etc - some of the worlds largest retailers pay 0.30 USD per credit card transaction? The fact is in the last 72 hours Bitcoin on at least one exchange (BTC China) priced 1 Bitcoin at 1,100 USD. At this point, users on that exchange moving their Bitcoins and paying the minimum transaction fee were paying 0.11 USD per transfer. That is not cheap.


Which are perfectly permissible. The sudden 'requirement' for a fee even though it is not mandated by the protocol just to get even remotely reasonable confirmation time is ridiculous.


Yes. 0.0001 BTC (currently) per 1KB.


Which the stock Bitcoin Daemon happily permits and you are none the wiser until your transaction suddenly doesn't go through.

Wouldn't a simple solution be to disallow unconfirmed inputs to be used for downstream output ? It's like your bank, sometimes your available balance and ledger balance are different.

The fees going up rapidly are a symptom of the increasing value of bitcoin. That will be adjusted when the value is more stable. From a long term perspective this shouldn't be a problem if bitcoin value doesn't jump orders of magnitude ($100 -> $1000) in a few days.

I've moved bitcoins with almost instant confirmation while paying a $0.05 transaction fee. I just choose the fee manually on my client though..

The biggest problem is not the fee (people are used to much, much higher fees), but that many new users are unaware of it. They only find out about it after searching for answers why their transaction does not get confirmations.

The wallets need to make it easier to set the fee. Some wallets won't even show it because it was left out for simplicity/space.

It should be in everyone's interest to support mining. The exchange rate can double within a day. Increasing the hashing power, and with it the security of the network, takes time.

I agree. Blockchain.info's simple "Send Money" does not include any settings for a fee. The MUCH scarier sounding "Advanced Send" (which does not actually look scary at all) does include a setting for miner's fees.

I think the notion that YOU can choose to pay or not pay a fee or even choose how much to pay is a libertarian way of setting things up, something that works well with the notion of what bitcoin is supposed to be. However, this isn't necessarily the best option from a usability perspective - people don't want to be bothered with thoughts of transaction fees. I've had a few orders getting stuck on my bitcoin business for 24+ hours because users don't know they should include a small fee for faster confirmation.

Maybe clients setting a default fee that ensures transactions will go through quickly is a good idea ?

Basically make the fees opt-out and keep it reasonably small (under 5 cents of a dollar, this is hard with the fluctuating exchange rate of course..). This will also give huge incentive for miners to continue operating the network because after all, this would not be possible without all those nodes.

But isn't the increased value of Bitcoin with respect to FIAT reward enough? @ 1000 USD / BTC - Mining pools are receiving 25,000 USD worth of value for every block successfully mined. If there are 144 blocks / day and a mining pool gets 8 of those blocks (~5.5%) - that's over 200,000 USD in revenue to share per day. Surely, a more reasonable approach is to discount fees and work on increasing the value of Bitcoin through adoption as a global payment mechanism. You don't do this by trying skim extra gravy off the top and penalizing those who don't pay.

Certainly right now clients should probably all be at least including the minimal fee at the moment (I changed my bitcoin-qt's setting to 0.0005 after having several very slow and later two stuck txs).

The other problem here is simply that currently (as in the current version of) Bitcoin does not scale - so until this is better resolved then there is simply nothing else that can be done (although at least if miners included closer to the 1MB of txs per block we'd have far less backlog).

The way mining and difficulty works, the infrastructure is directly proportional to the block rewards (base reward + fees).  More fees and higher exchange rates will always directly improve the hashing power of the network. No such thing as enough if you are interested in improving the hash rate.

What I still want to know is how much of a potential risk this scenario is:

"Consider also the scenario where a business receives and sends Bitcoin funds (same way as PayPal merchants do). If the business's wallet software decides to use an unconfirmed output as input from a client who paid using a 0 fee transaction as part of a downstream transaction or chain of downstream transactions and that client subsequently double spent the original transaction with a fee - all the downstream transactions would be invalidated if the subsequent fee including transaction were prioritized. This could mean significant financial losses for the business not to mention the nightmare of tracing and unwinding of the now invalid transactions."

As I mentioned above, the stock Bitcoin daemon (as shipped in Bitcoin-QT) appears happy to chain transactions with unconfirmed outputs.

Adding transactions to a block takes time, in which the miner/pool is idle. The transaction fee has to make up for that idle time. It doesn't need to be much, but certainly more than zero. I am sure the blocks will be filled if there are TX with fees available.



That's why it is good practice to wait for 7 confirmations before accepting a balance.

I've sent a few transactions recently with a 0.0005 BTC fee, the waiting time is normal, so I guess it's the minimum fee to have no issues. If you want to save on fees, then I guess exchange your BTC to LTC for micro-payments. BTC is getting into heavy-weight category, can't be bothered with micro-payments (as measured in USD).

But this doesn't make sense. The whole way Bitcoin is architected means that whether you've got one person mining on a single PC, or an entire continent on high power ASICS, the base reward is fixed with respect to the number of Bitcoins mined thus far (100->50->25->[...]->0 etc). Since we won't reach a mining reward that is near zero before 2040? The performance of the network and the faith people have in it is being significantly hampered for the sake of fractions of a Bitcoin.

Take block #270632. This includes a fairly typical 469 transactions, and gets 25 BTC reward (13949.34 USD according to CoinMill). They only took 0.14727399 BTC (82.17 USD) in fees. Basically, they received an extra 0.58% in reward fees. Bitcoin price has dropped by 1.07% (src: Bitstamp - Open 536.01, Close - 530.25) today alone. Does 0.58% extra really justify killing the performance and usability of Bitcoin?

How? The stock Bitcoin Daemon certainly doesn't discriminate between confirmed and unconfirmed. It just seems to pick appropriately valued outputs to use as inputs - regardless if they are confirmed or not. Literally; "bitcoind sendtoaddress {ADDRESS} {VALUE}". There is no choice.

I doesn't kill anything. It just takes a little longer. Paying a few USD cent gets rid of the wait time. It's everyone's choice what to do. Personally, I don't think twice about those few cents. And on top of that, I know that those few cents even go towards improving the network (in a tiny way).



If you are the receiver of a transaction, you only accept it as valid if it has 7 transactions. Mind, this is only for high value transactions. It is unlikely that someone tries to double spend small amounts, so merchants often accept transactions that were just broadcast to the network and have not even been included in a block.

Or are you asking how to see how many confirmations a transactions has? Your client should show you. You could also use https://blockchain.info/address/<your address here>.

I think you are misunderstanding my question. How, exactly, do you do this? If I run a bitcoin Daemon with 5 Bitcoins in and you send me 1 Bitcoin with no fee and it's stuck unconfirmed, I then fire off a "sendtoaddress" command from my wallet to send 3 Bitcoin to another person and the bitcoind chooses to use your unconfirmed input, my new transaction is now dependent on your preceding transaction being confirmed. There is no "sendfromconfirmedinputsonly" command - it's an algorithmic lottery which inputs the daemon will use to satisfy the outbound transaction. If your transaction is used, it doesn't matter if I include a 1 BTC fee ... I'm still stuck waiting for your transaction to confirm before my new one will.

My bad. I am not an expert, but maybe this will help you:

https://en.bitcoin.it/wiki/Raw_Transactions


Also sendfrom allows you to specify [minconf=1]. See:

https://en.bitcoin.it/wiki/Original_Bitcoin_client/API_calls_list

To be fair, as much as I do love Bitcoin and what it brings to world transfers.

Debit Card:  Usually $0 for a lot of major websites.
Bank Transfer (UK - UK bank account)  $0

Bitcoins primary strength is still making small to very large fast transfers across great distances around the world with tiny fees.

The stock bitcoind will NOT send until inputs have 1 CONFIRM when using sendtoaddress
The stock bitcoind will NOT increase the balance as seen with getbalance until you have 1 CONFIRM.
This is done INTENTIONALLY to avoid these types of situations.   The 0 confirm input could potentially NEVER confirm (as in a double spend) and thus the tx created with 0 confirm inputs will be forever invalid.

The daemon is also not the "normal clueless user tool".  The stock QT CLIENT will NEVER allow you to create a tx with unconfirmed inputs.  As far as the client is concerned you simply don't have the funds available for spending until they have 6 confirmations.

If other clients/wallets/exchages are allowing users to spend unconfirmed inputs that is a problem with those clients and should be fixed by the authors.

But this simply does not seem to be the case. See:

https://blockchain.info/tx/35c4e8c86075cf3f5335e029abd2d981af011c142b56e04c0a6d8b0d588d32ae

and

https://blockchain.info/tx/99e6c22980571d1733986d950f564854393777ba0905d08aae36d7f4f64e4a3a

Indeed, you commented on a similar situation here: https://bitcointalk.org/index.php?topic=339792.msg3645862#msg3645862

So, which is it?

How are you determining that those tx were created using the stock bitcoin daemon using sendtoaddress by looking at the tx id?

I told the user to stop sending unconfirmed inputs SPECIFICALLY because that is NON STANDARD BEHAVIOR and one that is likely going to cause more problems than it solves.

The fact that a tx exists doesn't mean it was created using the QT client or sendtoaddress command.  I can show you a tx which 2600 BTC was sent to nowhere.  It certainly wasn't done using the QT client.  You do know there are highly advanced calls like createrawtx which will allow the user to make all kinds of tx, even impossible ones which will result in permanent loss of coins.  


Bitcoind is an advanced tool.  Your grandmother is never going to use it.  The QT CLIENT DOES NOT ALLOW SPENDING UNCONFIRMED INPUTS FOR THIS EXACT REASON (AND OTHER GOOD REASONS AS WELL).

Because I created them, and I created them with the stock Bitcoin Daemon with the stock BitcoinD command "sendtoaddress". What's more, I had to use the blockchain.info/pushtx functionality to make the downstream transactions show up because it seems transactions with unconfirmed inputs are just ignored by blockchain.info. It was only when I did this that I saw the fact it was spending unconfirmed inputs.

Then report it as a bug.  I often get annoyed that sendtoaddress does not allow sending unconfirmed inputs.

On edit:
Is 19VuPF5XHfp435YBMG3f1F3oXZ9BvS2eTe also an address in the same wallet?  i.e. did you send your balance from one address you own to another one you owned?  If so IIRC bitcoind allows spending that unconfirmed input because it knows that it will be eventually confirmed.  Maybe that behavior should be changed?  Still I would ask WTF did you do that if it is the case?

@Arksun Those fees are usually charged to the MERCHANT. This is why you don't see a charge. They are already priced into whatever you are paying for.

No. 19VuPF5XHfp435YBMG3f1F3oXZ9BvS2eTe is not in the wallet. You seem to be under the erroneous impression that some sort of black-art has been attempted with Bitcoin daemon that's gone awry. I'm telling you now that the only command that has been fired at BitcoinD to make it spend the Bitcoins is "sendtoaddress".

In this case I would be reporting a bug (those higher level RPC commands should not be like the raw tx ones).

Like I said file a bug report.  Not sure how or why bitcoind allow spending an unconfirmed output.  I use it for my automated backend and I have to drop to a lower level command to force a tx when I have funds but they are unconfirmed and I need it to go out immediately. I actually wish sendtoaddress had a [minconf=1] option with a default of 1 that would allow finer control but it doesn't.

A couple of times I have resorted to doing a "getrawtransaction <id>" then pasted the raw tx into https://blockchain.info/pushtx (and that has actually worked).

I have done exactly the same, but that's just because people I send BTC to get nervous when the txID I give them doesn't appear on the network (well, blockchain.info). That's what led to this ... discovery.

As an aside, I've raised the issue here: https://github.com/bitcoin/bitcoin/issues/3288

Look at this transaction:

http://blockchain.info/de/tx/5038108b9f275e8adbd86bf89df695e59b7f03ca72144ca51752904dc989c4e6 (http://blockchain.info/de/tx/5038108b9f275e8adbd86bf89df695e59b7f03ca72144ca51752904dc989c4e6)

Without any fees, it needed 24 hours to get confirmed!  :o

A symptom of the untempered greed of mining pools unfortunately. Everyone wants more magical internet money.

I took a quick (re)look at the code and sendtoaddress is pretty simple.  It does some parsing and error checking and then calls.  SendMoneyToDestination()

https://github.com/bitcoin/bitcoin/blob/d980f9b7d687a1e60eecf3691b592d9806a30f4a/src/wallet.cpp#L1467


Note the balance check calling GetBalance()

https://github.com/bitcoin/bitcoin/blob/d980f9b7d687a1e60eecf3691b592d9806a30f4a/src/wallet.cpp#L961

GetBalance gets the sum of confirmed unspent outputs.

The only thing I can think of is at the time you created the second tx the unspent output used WAS CONFIRMED.  Possibly there was a short re-org in the blockchain AFTER the second tx was broadcast which made the first tx unconfirmed.  Then because of high number of free tx and miners including a limited number you ended up with the first tx not confirming for a while and that led to the second tx not confirming for a while.


IMHO the default behavior for a lot of things probably need to change.  I doubt the core devs will see it as a high priority but I think the entire system would be simpler and less prone to unexpected (from user standpoint) behavior if we saw the following.

1) Drop min fee to 0.02 mBTC.  The last time this was done the exchange rate was ~$100 USD.  The drop would put min fees back to ~$0.01 per KB.
2) Change default behavior of client to include a fee on ALL tx (not just low priority ones) and make the default fee the min fee.  Users can override this but they should be warned "including no fee may result in substantial delays before confirmation".
3) Implement "parent pays child".
4) Implement "tx replacement".

Use blogs and other public forums to advocate that authors of other wallets/eWallets/exchanges/etc do the same.

That sounds plausible, but how often would you expect such a reorganization to occur? I have just dug through the logs and found another example:

This transaction https://blockchain.info/tx/2b7bec664c3552c605dd687404e1c7088501e88369ceabf4da04c2bd15dc2ed4 was broadcast at 02:28 GMT on the 19-11-2013. It did not confirm for 670 minutes and the transaction had to be manually pushed through blockchain.info/pushtx to appear on the blockchain.info database. The reason it didn't appear is again, because of an unconfirmed input being used: https://blockchain.info/tx/3386d4afcd5506a7bd77637b63f0ed061b878fb1557278bc3717048ca10d4c31 - This ancestor transaction was broadcast at 01:42 GMT on the 19-11-2013 and included no fee. It took 716 minutes to confirm.

You can see the overlap in confirmation times means https://blockchain.info/tx/3386d4afcd5506a7bd77637b63f0ed061b878fb1557278bc3717048ca10d4c31 could not have possibly been confirmed when it was used as an input for https://blockchain.info/tx/2b7bec664c3552c605dd687404e1c7088501e88369ceabf4da04c2bd15dc2ed4

Here's a live list of Mt. Gox Bitcoin transactions stuck waiting for confirmation for more than 2 hours:
http://skanner.net/MtGox/mtgox_tx.php (http://skanner.net/MtGox/mtgox_tx.php)

There's a huge backlog of "no fee" transactions.   It looks like 0.001 BTC isn't enough to consistently get a transaction through, but 0.002 BTC is. So it now costs about $1 to do a Bitcoin transaction.

Bitcoin has now been priced out of the micropayments business. It's too expensive for, say, an iTunes track or a video rental.

Would you mind linking a couple of those with 2h wait time and proper fee?

This seems to be inconsistent behavior though. I've paid a 0.0001BTC fee (roughly 6 cents) and had transactions go through in less than 10 minutes.

min fee for low priority tx is 0.1 mBTC per kB not 1 mBTC.  So your estimate is off by a factor of at least 10x.

I'm not sure if these complaints were recent.  For me, I bought something on Monday,my first purchase with Bitcoins, and to my astonishment it was quick in terms of confirmation of bitcoin transaction.  I created another account and transferring a small amount from my main account to the new account, confirmation was pretty much instantaneous.  Once I purchased my items with the small funded account, that transaction of bitcoins was very quick.  I received confirmation on my phone literally 1 sec later.  The one thing that made me concerned was that retail's website took a while to send me any confirmation email letting me know the purchase was complete.  As for transaction fee, it was a small one .0005 BTC I believe, or whatever the decimal place is suppose to be but it was not very high at all.  That's my recent experience.

If you include a fee and especially if you include a generous fee which 0.0005 BTC is, you shouldn't encounter any problems with rapid confirmations. You're basically being fast-tracked to the front of the queue.

Your correct the inclusion of tx in a block is pretty trivial.  Sure it is a non zero time but a CPU can validate about 15,000 tps.  If the UXTO is kept in memory (UXTO = unspent portion of blockchain and much smaller) there is no reason that including even thosands of tx in a block should take more than a fraction of a second.  Pools also use various tricks to speed up work generation when a block change occurs (send work to faster workers first, build a 0 tx block to get all miners working rapidly and then go back and build the larger block).  So for all intents and purposes we can consider this to be zero cost.

The prior poster misstates the "cost".  The real cost that miners (pools) are worried about is increased orphan losses.  For a given pool if they do something different which increases their orphan rate by 1% they lose ~0.25 BTC, if that changes produces less than 0.25 BTC in additional revenue they are net-net losing revenue.  The larger the block is, the longer it takes to propagate through the network for a given amount of bandwidth.  Those peers then need to validate it, and relay it to their peers, who validate it and relay it to their peers.  Eventually all nodes (and more importantly miners) have the new block and work to build off it.

The time between when a miners finds a block and all miners are building off that block can be considered the "critical window".  If another miners finds a competing block (same height) in that critical window one of those blocks will be orphaned.   The larger the block the longer the propagation delay, and the greater the losses due to orphans.  You can consider the chance of an orphan relative to the value of the block the "cost" (in lost revenue) for a miner adding another tx to the block.   Smaller block = less gross revenue but less chance of orphaned block, larger block = higher gross revenue (except for free tx which is why I believe they will die off) but higher chance of losing the whole block.

Gavin corrected me (in another thread) that the best estimate for the "orphan cost" is ~3.3 mBTC per kB, which is pretty huge.  This is why miners are reluctant to build larger blocks.  If you look at it from game theory perspective, they get 25 BTC for "free" (baseline orphan risk), The average paying tx is ~0.1 to 0.3 mBTC per kB but it costs them about 3.3 mBTC in orphan losses to include that tx.   There is "good news" though.  The current method of relaying blocks is relatively inefficient.  It likely with some small soft (no hard fork necessary) changes reduce the orphan cost by 80% (a more controversial change could cut that to 99% or more).  Also as bandwidth becomes cheaper and more available the orphan cost also falls, so Moore's law is on ourside.  Miners can also reduce the orphan cost by making sure other miners are direct peers.  The faster other miners learn of "your block" the faster the critical window closes.  It doesn't really matter how long it takes the whole network to know about a new block just how long it take for all (well most) miners to know about a new block.  Non-mining nodes learning about a block a few seconds later is fine for their needs.

Remember Bitcoin is an experiment, it is in beta.  For those who want a perfectly polished protocol with no rough edges check back in a decade. :)

I think we should raise awarness and pettition the larger pool operators to include the max block size of I believe 1Mb. Can we as a community organize this together?

Hell at this point I would settle for just having it out in the open. 

What ARE the parameters you use for a block?
How many free txs?
How many paid txs?
What is the threshold for paying vs fee? (by default it is 0.1 mBTC).

If you can start a thread and get 10 pool admins to provide that kind of insight the community can start making better decisions.  The nice thing is the blockchain keeps them honest.  If pool xyz claims they allocate 27KB per block for free tx and there is a backlog and they produce multiple blocks with <27KB we can prove they are not being honest.  Likewise if pool abc claims they will include up to 500KB of paying txs with theshold of 0.1 mBTC per KB and they produce multiple blocks smaller than 500 KB while paying tx are waiting it can be proven they also are being dishonest.

If we KNOW what all the major pools (and major solo miners with more than say 5% of global hashrate) have as their transactions selection parameters we an put that together and start to get a better view of how long various tx are going to take.  If you know combined all the pools on average only devote an average of 27KB (the default) of space for free transactions that is ~60 free transactions per block.  If there are 2000 waiting it is going to take 50 blocks (assumming no new tx) to confirm all of them.

True but selfish mining actually means a net reduction in current income.  The attack is that an attacker would lose money today to generate a larger than fair share of revenue in the future.  I don't find it a particularly viable attack method as if you want to expend massive amounts of cost today to hurt the Bitcoin network you could just do a traditional 51% attack.

Simple version if a pool has 20% of the hashing power and due to longer propagation another miner wins the race to all other miners then the pool has a 20% chance of winning both blocks but an 80% chance of losing the already solved block.   In reality that type of simplified race outcome is very unlikely it is more likely that a pool broadcast a block and it reaches some % of miners first (including itself) and a competing block reaches some % of miners first and for the next block the network is split.  How split really depends on how close the race was and how different in size the blocks are.

I don't get it. I checked this a few times. I tranfered 0.0001 + 0.0001 fee, just to check. My tx was always in the next block. I always sent this btc from my mobile app.

Which tx with fees have this problems?

Example: This was a couple of minutes ago and the it went into the following block:
https://blockchain.info/tx/8e78d8c99d8c3cd62102f2b78befe3d171e15f496c725a09aadd3fe4e8fbb765

I haven't seen many problems with "normal" paying tx however:
a) Someone user have reported issues with paying txs and the actual issue is the tx uses as an input the output of a prior tx which hasn't confirmed. Fee or not a tx isn't going to confirm until the inputs confirm.  If the input has no fee it may take a long time.  The issue isn't the paying tx it is the unpaid one that it relies on.  The QT client prevents spending unconfirmed outputs for this reason. If other clients allow this it should probably be disabled by default.  A protocol change to improve this scenario is called "child pays parent".

b) Most clients by default most clients include no fee for high priority txs. That may have been optimal at one time however today that can mean waiting a long time.  I recommend setting the default fee to 0.1 mBTC for ALL transactions if you want timely inclusion in a block.  The days of pay a fee for low priority but get into a block quickly for free if tx is high priority are coming to a close.  Free tx should be considered charity and the time to confirmation may be hours, days, or even weeks.   Clients should probably warn users to expect potentially extended confirmation times if they try.

c) Some users mistakenly think that a very low fee (say 1 satoshi) helps but by default miners consider fees below 0.1mBTC (10,000 sat) to be free so really there is no point to pay a fee below 0.1 mBTC you are just wasting money the tx isn't going to confirm any faster than a no fee one will.  Clients should probably warn users about this behavior if user tries to pay a fee below the default threshold.

In my experience baring those scenarios most paying tx due confirm very quick, at most they may be delayed a block or two.  So it isn't an issue for all tx all the time.

And there is another issue - in case your tx stuck, in most (in all?) clients it is not possible to increase fee. It can be done by sending same tx with different fee, but why there is no such option?

The problem is that you cannot send the same tx whilst the old one is known to the network (any attempt to do so will simply be ignored as a "double spend" attempt) so it is not something that can very easily be handled by clients (basically you have to wait until the old tx has been "forgotten" before you could send a "replacement" tx).

@DeathAndTaxes

Yeah, I read about this guy who used a fee but the output he wanted to use was not confirmed (no fee). Then it makes sense that it take a long time and is just a mistake of the user. So when i scroll down I see that many tx have less than 0.1mBTC per kb or an unconfirmed output.

But what I saw now: At the moment there are waiting 2000 tx and the total fee to collect is 50btc (now it dropped mysteriosly to 25btc). This means the average fee is 0.025btc = $12,5 ????
-> I think this is because many tx with maaany outputs are stuck. So they pay a large amount of tx but it is still less than 0.1mBTC per kb.
https://blockchain.info/de/unconfirmed-transactions

For reference, in Canada:
Debit card: fees vary from $0 (rare) to $0.15 generally
Bank Transfer: $20

The democratic solution would be that if you use bitcoins then you should also be mining bitcoins as the system gradually moves from being rewards based to transaction fee based with the increasing difficulty of bitcoins. That way electricity costs ect would be hugely spread out among users and average trasaction costs would stay low. To get a hosted wallet u should be required to devote a small part of ur computers time to mining IMO.