On monday, CoinWallet.eu conducted a stress test of the Bitcoin blockchain. Not only was our plan to see the outcome, but also to see how easy it would be for a malicious entity or government to create havoc for the Bitcoin community. As you will see from the analysis below, delayed transactions are not the only issue that Bitcoin users experienced.
Surprisingly, executing tens of thousands of transactions that correctly propagate to the network simultaneously is not as easy as we had expected. One of the methods that was used to increase the kb size of our transactions was to send transactions consisting of numerous small outputs (usually 0.0001) to make a single transaction of 0.01. A simple transaction is usually 225-500 bytes, while many of our transactions were 18 kb (A number which limits the blockchain to 5 transactions per minute). In our preliminary testing this was effective, however in practice it caused our servers to crash. Throughout the day and evening, our strategy and methodology changed multiple times.
Initially the plan was to spend 20 BTC on transaction fees to flood the network with as many transactions as possible. Due to technical complications the test was concluded early, with less than 2 BTC spent on fees. The events of yesterday were accomplished with less than €500.Timeline 11:57 GMT - Transaction servers initiated. Thousands of 700 kb transactions completed within the first 20 minutes. Transactions were used to break coins into small 0.0001 outputs.
12:30 GMT - Servers begin sending larger 18kb transactions.
14:10 GMT - Mempool size increases dramatically. Blockchain.info breaks.
14:20 GMT - Our servers begin to crash. It becomes apparent that BitcoinD is not well suited to crafting transactions of this size.
14:30 GMT - Our test transactions are halted while alternate solutions are created. The mempool is at 12 mb.
17:00 GMT - Alternate transaction sending methods are started. Servers are rebooted. Mempool has fallen to 4mb.
21:00 GMT - The stress test is stronger than ever. Mempool reaches 15 mb and more than 14000 transactions are backlogged. The situation is made worse by F2Pool selfishly mining two 0kb blocks in a row.
23:59 GMT - 12 hours after starting, the test is concluded.
Less than 2 BTC (€434) is spent on the test in total.The following graph depicts the entire test from start to finish:
https://anduck.net/bitcoin/mempool.pngObservationsDelayed confirmation times and large mempool buildups were not the only observation that came from our testing. Many more services were impacted than we had initially envisioned.
Blockchain.infoOver the past few months, Blockchain.info has become increasingly unreliable, however we are confident that yesterday's stress test had an impact on their website being offline or broken for 1/3 of the day. During periods where we sent excessive transactions, Blockchain.info consistently froze. It appeared as though their nodes were overwhelmed and simply crashed. Each time this occurred, the site would re-emerge 10-30 minutes later only to fail again shortly thereafter. Users of the blockchain wallet were unable to send transactions, login or even view balances during the downtimes. In response to our heavy Bitcoin usage, blockchain.info began to exclude certain transactions from their block explorer. This issue is explored further by the creators of Multibit, who can confirm that some transactions sent from their software were ignored by Blockchain, but were picked up by Blockr.
Bitcoin ATMsMany ATMs operate as full nodes, however some ATMs rely on third party wallet services to send and receive transactions. The most prominent Bitcoin ATM of this type is Lamassu, which uses the blockchain.info API to push outgoing transactions from a blockchain.info wallet. Due to the blockchain.info issues, all Lamassu ATMs that use blockchain.info's wallet service were unavailable for the day.
MultiBitBoth versions of MultiBit suffered delayed transactions due to the test. Gary and Jim from MultiBit have created a full analysis from Multibit's perspective which can be read at
https://multibit.org/blog/2015/06/23/bitcoin-network-stress-test.htmlThe outcome was that transactions with the standard fee in Multibit HD took as many as 80 blocks to confirm (Approximately 13 hours). Standard 10000 satoshi fee transactions took an average of 9 blocks to get confirmed. Multibit has stated that they will be making modifications to the software to better cope with this type of event in the future.
TradeblockWith Blockchain.info broken, we frequently referred to Tradeblock to track the backlog. Unfortunately Tradeblock was less than perfectly reliable and often failed to update when a new block had been mined. Regardless, at one point 15,000 unconfirmed transactions were outstanding.
BitpayUsers reported issues with Bitpay not recognizing transactions during the test.
PriceIncrease of $2. Contrary to some predictions, we did not short Bitcoin.
Green AddressWhile this app was not hindered directly by our test, we did send a series of 0.001 payments to a green address wallet. When attempting to craft a transaction from the wallet, an error occurs stating that it is too large. It appears that the coins that were sent to this wallet may be lost.
ConclusionsFrom a technical perspective, the test was not a success. Our goal of creating a 200mb backlog was not achieved due to miscalculations and challenges with pushing the number of transactions that we had desired. We believe that future tests may easily achieve this goal by learning from our mistakes. There are also numerous vulnerable services that could be exploited as part of a test, including Bitcoin casinos, on-chain gambling websites, wallets (Coinbase specifically pointed out that a malicious user could take advantage of their hosted wallet to contribute to the flooding), exchanges, and many others. Users could also contribute by sending small amounts to common brain wallets.
We also learned that the situation could have been made worse by sending transactions with larger fees. We sent all transactions with the standard fee of 10000 satoshis per kb. If we had sent with 20000 satoshis per kb, normal transactions would have experienced larger delays.
In our future stress tests, these lessons will be used to maximize the impact.