Can you prove this?

Ahh, so that explains why I haven't been paid my mining profits yet today.  The mining site shows that the payment went out but it's not registered in my wallet yet.  Is this sort of thing normal?  I've seen delays a few times but this seems bigger than usual.

And isn't the price of btc up today too?  I'd think this is the sort of issue that would drive the price down more.

On June 22nd, we initiated a relatively limited stress test on the Bitcoin blockchain to determine whether or not we alone could have a large impact on the ecosystem as a whole. While our initial tests merely created full blocks for a multiple hour period, transaction confirmation times remained within 6 blocks for most transactions. This test was not as successful as planned.

See here why the first stress test was not as successful as planned: CLICK

The result was roughly 12 hours of full blocks combined with increased confirmation times for many Bitcoin transactions. By selecting random transactions that were not initiated by our team, we were able to determine that many standard fee transactions were taking ~80 blocks before receiving a single confirmation.

Bitcoin is at a breaking point, yet the core developers are too wound up in petty arguments to create the required modifications for long term sustainability. If nothing is done, Bitcoin will never be anything more than a costly science project. By stress testing the system, we hope to make a clear case for the increased block size by demonstrating the simplicity of a large scale spam attack on the network.

The plan - We setup 32 Bitcoin servers that will send approximately 1 transactions per second each (up from 10 servers sending 2 transactions per second each) - Each of these transactions will be approximately 3kb in size and will each spend to 10-20 addresses - The outputs will then be combined to create large 15-30kb transactions automatically pointing back to the original Bitcoin servers.
Example: https://blockchain.info/tx/888c5ccbe3261dac4ac0ba5a64747777871b7b983e2ca1dd17e9fc8afb962519

•    Certain servers will be configured to include marginally larger than standard fees, thus guaranteeing delays from standard SPV wallets.

The target will be to generate 1mb worth of transaction data every 5 minutes. At a cost of 0.0001 per kb (as per standard fees) this stress test will cost approximately 0.1 BTC every five minutes. Another way to look at the cost is 0.1 BTC per full block that we generate. We have allocated 20 BTC for this test, and therefore will be able to single handedly fill 100 blocks, or 32 hours worth of blocks. However, we will stop pushing transactions after 24 hours at 13:00 GMT Tuesday June 30.

Predictions

•    Our above estimates are based on 1 block being mined every 10 minutes. This is standard, however deviations are likely to create temporary blips in our testing, in particular when there is an hour gap between confirmations.
•    The above calculations are based on each block being exactly 1mb and no other transactions appearing in the blocks. This is obviously unrealistic due to the fact that the average block size without our intervention is approximately 600kb. Furthermore, at least 30% of miners continue to cap blocks at 731kb. Others cap at 976kb.

Conclusions

For the sake of avoiding un-necessary calculations, lets assume that each block is 926kb in size, the average normal Bitcoin transaction volume is 600kb per block, and CoinWallet.eu will be pushing 2mb of transaction data into the network every 10 minutes. Under these conditions, the amount of transaction data being pushed to the network every 10 minutes (or every average block) will be ~2600kb. This will result in a 1674 kb backlog every 10 minutes.

By 13:00 GMT Monday June 29, the mempool of standard fee transactions will be 10mb By 24:00 GMT Monday June 29nd, the mempool of standard fee transactions will be 130mb By 13:00 GMT Tuesday June 30, the mempool of standard fee transactions will be 241mb

At this point the backlog of transactions will be approximately 241 blocks, or 1.67 days. When the average new transactions are factored into the equation, the backlog could drag on for 2-3 days. At this point, questions are raised such as whether or not this will cause a "crash landing." It is impossible to know with certainty, however we are anxiously looking forward to Monday.

On monday, CoinWallet.eu conducted a stress test of the Bitcoin blockchain. Not only was our plan to see the outcome, but also to see how easy it would be for a malicious entity or government to create havoc for the Bitcoin community. As you will see from the analysis below, delayed transactions are not the only issue that Bitcoin users experienced.

Surprisingly, executing tens of thousands of transactions that correctly propagate to the network simultaneously is not as easy as we had expected. One of the methods that was used to increase the kb size of our transactions was to send transactions consisting of numerous small outputs (usually 0.0001) to make a single transaction of 0.01. A simple transaction is usually 225-500 bytes, while many of our transactions were 18 kb (A number which limits the blockchain to 5 transactions per minute). In our preliminary testing this was effective, however in practice it caused our servers to crash. Throughout the day and evening, our strategy and methodology changed multiple times.

Initially the plan was to spend 20 BTC on transaction fees to flood the network with as many transactions as possible. Due to technical complications the test was concluded early, with less than 2 BTC spent on fees. The events of yesterday were accomplished with less than €500.

Timeline

    11:57 GMT - Transaction servers initiated. Thousands of 700 kb transactions completed within the first 20 minutes. Transactions were used to break coins into small 0.0001 outputs.
    12:30 GMT - Servers begin sending larger 18kb transactions.
    14:10 GMT - Mempool size increases dramatically. Blockchain.info breaks.
    14:20 GMT - Our servers begin to crash. It becomes apparent that BitcoinD is not well suited to crafting transactions of this size.
    14:30 GMT - Our test transactions are halted while alternate solutions are created. The mempool is at 12 mb.
    17:00 GMT - Alternate transaction sending methods are started. Servers are rebooted. Mempool has fallen to 4mb.
    21:00 GMT - The stress test is stronger than ever. Mempool reaches 15 mb and more than 14000 transactions are backlogged. The situation is made worse by F2Pool selfishly mining two 0kb blocks in a row.
    23:59 GMT - 12 hours after starting, the test is concluded. Less than 2 BTC (€434) is spent on the test in total.

The following graph depicts the entire test from start to finish: https://anduck.net/bitcoin/mempool.png

Observations

Delayed confirmation times and large mempool buildups were not the only observation that came from our testing. Many more services were impacted than we had initially envisioned.

Blockchain.info

Over the past few months, Blockchain.info has become increasingly unreliable, however we are confident that yesterday's stress test had an impact on their website being offline or broken for 1/3 of the day. During periods where we sent excessive transactions, Blockchain.info consistently froze. It appeared as though their nodes were overwhelmed and simply crashed. Each time this occurred, the site would re-emerge 10-30 minutes later only to fail again shortly thereafter. Users of the blockchain wallet were unable to send transactions, login or even view balances during the downtimes. In response to our heavy Bitcoin usage, blockchain.info began to exclude certain transactions from their block explorer. This issue is explored further by the creators of Multibit, who can confirm that some transactions sent from their software were ignored by Blockchain, but were picked up by Blockr.

Bitcoin ATMs

Many ATMs operate as full nodes, however some ATMs rely on third party wallet services to send and receive transactions. The most prominent Bitcoin ATM of this type is Lamassu, which uses the blockchain.info API to push outgoing transactions from a blockchain.info wallet. Due to the blockchain.info issues, all Lamassu ATMs that use blockchain.info's wallet service were unavailable for the day.

MultiBit

Both versions of MultiBit suffered delayed transactions due to the test. Gary and Jim from MultiBit have created a full analysis from Multibit's perspective which can be read at https://multibit.org/blog/2015/06/23/bitcoin-network-stress-test.html

The outcome was that transactions with the standard fee in Multibit HD took as many as 80 blocks to confirm (Approximately 13 hours). Standard 10000 satoshi fee transactions took an average of 9 blocks to get confirmed. Multibit has stated that they will be making modifications to the software to better cope with this type of event in the future.

Tradeblock

With Blockchain.info broken, we frequently referred to Tradeblock to track the backlog. Unfortunately Tradeblock was less than perfectly reliable and often failed to update when a new block had been mined. Regardless, at one point 15,000 unconfirmed transactions were outstanding.

Bitpay

Users reported issues with Bitpay not recognizing transactions during the test.

Price

Increase of $2. Contrary to some predictions, we did not short Bitcoin.

Green Address

While this app was not hindered directly by our test, we did send a series of 0.001 payments to a green address wallet. When attempting to craft a transaction from the wallet, an error occurs stating that it is too large. It appears that the coins that were sent to this wallet may be lost.

Conclusions

From a technical perspective, the test was not a success. Our goal of creating a 200mb backlog was not achieved due to miscalculations and challenges with pushing the number of transactions that we had desired. We believe that future tests may easily achieve this goal by learning from our mistakes. There are also numerous vulnerable services that could be exploited as part of a test, including Bitcoin casinos, on-chain gambling websites, wallets (Coinbase specifically pointed out that a malicious user could take advantage of their hosted wallet to contribute to the flooding), exchanges, and many others. Users could also contribute by sending small amounts to common brain wallets.

We also learned that the situation could have been made worse by sending transactions with larger fees. We sent all transactions with the standard fee of 10000 satoshis per kb. If we had sent with 20000 satoshis per kb, normal transactions would have experienced larger delays. In our future stress tests, these lessons will be used to maximize the impact.

But shapshift.io charges quite a hefty fee, no?  At least that's what I heard.  I've used bitmixer.io multiple times without a problem, simply for testing purposes.