28 000 unconfirmed TXs

[Hyena]

I noticed my CPU was at 100%. I started to investigate and verified that my block chain monitor process was running at 100%, consuming 100 MiB of RAM and totally frozen. After restarting I was amazed to witness such logs:

Code:

Wed Jul  8 01:40:13 2015 :: Checked 20 of 28281 unconfirmed TXs in 2.00 seconds.
Wed Jul  8 01:40:15 2015 :: Checked 46 of 28261 unconfirmed TXs in 2.00 seconds.
Wed Jul  8 01:40:17 2015 :: Checked 46 of 28215 unconfirmed TXs in 2.00 seconds.
Wed Jul  8 01:40:19 2015 :: Checked 48 of 28169 unconfirmed TXs in 2.00 seconds.
Wed Jul  8 01:40:21 2015 :: Checked 60 of 28121 unconfirmed TXs in 2.00 seconds.
Wed Jul  8 01:40:23 2015 :: Checked 46 of 28061 unconfirmed TXs in 2.00 seconds.
Wed Jul  8 01:40:25 2015 :: Checked 34 of 28015 unconfirmed TXs in 2.00 seconds.
Wed Jul  8 01:40:27 2015 :: Checked 24 of 27981 unconfirmed TXs in 2.00 seconds.
Wed Jul  8 01:40:38 2015 :: Checked 11 of 27957 unconfirmed TXs in 11.00 seconds.
Wed Jul  8 01:40:41 2015 :: Checked 30 of 27946 unconfirmed TXs in 2.00 seconds.

How on Earth can there be 28 000 unconfirmed TXs. That's probably one of the reasons my process froze. Is the bitcoin network under attack or being stress tested by someone? What's happening?

edit:
for the record, I upgraded my bitcoin-core to version v0.10.2 (64-bit) yesterday.

edit2:
if the raw mempool is getting SOOO freaking huge then we need API commands to get data from the raw mempool in pages rather than as a one giant json object.

[1714135714]

1714135714

[1714135714]

1714135714

[RussianRaibow]

How on Earth can there be 28 000 unconfirmed TXs. That's probably one of the reasons my process froze. Is the bitcoin network under attack or being stress tested by someone? What's happening?

Bitcoin was under attack. Now, it is recovering. Bitcoin is resilient.

[Hyena]

How on Earth can there be 28 000 unconfirmed TXs. That's probably one of the reasons my process froze. Is the bitcoin network under attack or being stress tested by someone? What's happening?

Bitcoin was under attack. Now, it is recovering. Bitcoin is resilient.

I think bitcoin-core should start discarding unconfirmed TXs if they start to consume too much memory. TXs with the lowest priority should be discarded first. Also, individual TXs should contain a proof of work in addition to the fee. The proof of work should be optional and should increase the priority of the TX so that it would not be discarded so easily.

For example, let's say the network is under attack. The nodes would not be able to distinguish between malicious TXs and honest TXs. However, a honest node would attach a proof of work to the TX so that honest TXs would be favoured by the network since their proof of work has more zeroes in front of the hash than the malicious TXs. When the attacker chooses to attach their own PoW the priority of the malicious TXs rises naturally to the point that it could push out the honest TXs. However, because the client can freely choose the difficulty of the PoW, it will notice that their TX is not accepted by other nodes so the client increases the difficulty, calculates a new PoW and resends the honest TX. That way, the bitcoin network would become super resilient to the given attacks.

[domob]

How on Earth can there be 28 000 unconfirmed TXs. That's probably one of the reasons my process froze. Is the bitcoin network under attack or being stress tested by someone? What's happening?

Bitcoin was under attack. Now, it is recovering. Bitcoin is resilient.

I think bitcoin-core should start discarding unconfirmed TXs if they start to consume too much memory. TXs with the lowest priority should be discarded first. Also, individual TXs should contain a proof of work in addition to the fee. The proof of work should be optional and should increase the priority of the TX so that it would not be discarded so easily.

For example, let's say the network is under attack. The nodes would not be able to distinguish between malicious TXs and honest TXs. However, a honest node would attach a proof of work to the TX so that honest TXs would be favoured by the network since their proof of work has more zeroes in front of the hash than the malicious TXs. When the attacker chooses to attach their own PoW the priority of the malicious TXs rises naturally to the point that it could push out the honest TXs. However, because the client can freely choose the difficulty of the PoW, it will notice that their TX is not accepted by other nodes so the client increases the difficulty, calculates a new PoW and resends the honest TX. That way, the bitcoin network would become super resilient to the given attacks.

I think that we need some kind of control on the size of the mempool eventually if it grows too big.  I'm not sure that PoW is the way to go, though - I imagine that the outcome would be that spamming is still trivial with, say, a Blockerupter, while ordinary users need to compute very long to get a suitable PoW.

[btc_enigma]

my bitcoind crashed yesterday .. memory and cpu usage is high

Someone suggested I increase txrelayfees , what fees should I set so that my service doesn't miss out non spamming transactions while blocking spam tx

[ChetnotAtkins]

There are currently 60000 (!) unconfirmed transactions. Mempool at 50 MB

https://tradeblock.com/blockchain/

[NoBit]

How long will this "attack" last? Does it have anything to do with the recent LTC-pump?

[Vezalke]

Hey, thanks for link. I'm waiting for my transaction for 20 hours now and the fees was included. 

[el kaka22]

I see that blockchain.info have some bug on showing unconfirmed txs, they only show 17k txs for now (but it is still a large number).

There are currently 60000 (!) unconfirmed transactions. Mempool at 50 MB

https://tradeblock.com/blockchain/

And at this page it is showing 66k transactions! And there are some f... miners who still mining empty blocks (at block 364389 and the latest 364399 at the time or writing). I wonder when my tx with 2000 satoshi fee will be confirmed

[Hyena]

I think that we need some kind of control on the size of the mempool eventually if it grows too big.  I'm not sure that PoW is the way to go, though - I imagine that the outcome would be that spamming is still trivial with, say, a Blockerupter, while ordinary users need to compute very long to get a suitable PoW.

We could start identifying TXs by the nodes where the TX initiated. That way, a honest node would only have to calculate the PoW once during its existence. The calculation should be difficult. When other nodes detect TX flood from a single node they would revoke the PoW and not relay those TXs any more until a new PoW is attached.

It is easy to revoke a PoW from a malicious node and it is very difficult to calculate a new PoW. The malicious user would have to constantly calculate new PoWs while the honest nodes with their existing PoW can keep sending TXs normally. At times of high number of unconfirmed TXs the TX PoW difficulty should be risen in correlation to the number of those TXs.

When a honest node receives a TX they would check the internal database for a PoW that was somehow attached to the TX. If such a PoW exists and is not flooding the network, the node relays the TX. If the PoW exists but is flooding the network with spam TXs, the PoW will be revoked locally and the TXs with that PoW are no longer relayed. If the PoW does not exist locally the node checks the current number of unconfirmed TXs, calculates the difficulty and checks whether the PoW meets the criteria. If the criteria is met, the PoW is added to the local database. Otherwise, TXs with that PoW are not relayed.

[Hyena]

How are developers responding to this severe limitation of Bitcoin's usage. There are currently 72000 (!) unconfirmed transactions but it seems they don't really want to acknowledge it.

Perhaps set a limit of tx/s to discourage spamming the mempool and block malicious nodes.

I start to suspect that the devs have become corrupt. They are all part from their own precious altcoin scams and seeing bitcoin crash and burn will make them only happy. They deliberately do nothing.

[cesmak]

How are developers responding to this severe limitation of Bitcoin's usage. There are currently 72000 (!) unconfirmed transactions but it seems they don't really want to acknowledge it.

Perhaps set a limit of tx/s to discourage spamming the mempool and block malicious nodes.

This is not right at now there are "only" near 18.000 unconfirmed transactions....

https://blockchain.info/it/unconfirmed-transactions?offset=0

[achow101]

I think bitcoin-core should start discarding unconfirmed TXs if they start to consume too much memory. TXs with the lowest priority should be discarded first. Also, individual TXs should contain a proof of work in addition to the fee. The proof of work should be optional and should increase the priority of the TX so that it would not be discarded so easily.

Most of the dust transactions that are spamming the network are being dropped by most node's mempools. It is primarily these dust transactions that cause the mepool to grow so large.

For example, let's say the network is under attack. The nodes would not be able to distinguish between malicious TXs and honest TXs. However, a honest node would attach a proof of work to the TX so that honest TXs would be favoured by the network since their proof of work has more zeroes in front of the hash than the malicious TXs. When the attacker chooses to attach their own PoW the priority of the malicious TXs rises naturally to the point that it could push out the honest TXs. However, because the client can freely choose the difficulty of the PoW, it will notice that their TX is not accepted by other nodes so the client increases the difficulty, calculates a new PoW and resends the honest TX. That way, the bitcoin network would become super resilient to the given attacks.

So you are suggesting that we must mine txs in order to have it be accepted and confirmed? What if the malicious spammer has an ASIC that can calculate these PoWs faster than everyone else and makes them more difficult? That means that he just raised the difficulty of his transactions, thereby increasing his priority. This causes other people to have a lower difficulty than the majority and thus their priority is lower. So what happens to those people that don't have sufficient computing power to calculate a PoW more difficult than the spammers? Does this make their transaction have a lower priority and they are essentially locked out of making transactions?

We could start identifying TXs by the nodes where the TX initiated. That way, a honest node would only have to calculate the PoW once during its existence. The calculation should be difficult. When other nodes detect TX flood from a single node they would revoke the PoW and not relay those TXs any more until a new PoW is attached.

If a PoW calculation is difficult, what happens if my computer does not have the computing power to calculate a sufficiently difficult PoW? Is it rejected and blocked from the network? Does that mean I will have to wait even longer for Bitcoin Core to start the first time so that it can generate a PoW that identifies the node as me?

How are developers responding to this severe limitation of Bitcoin's usage. There are currently 72000 (!) unconfirmed transactions but it seems they don't really want to acknowledge it.

Perhaps set a limit of tx/s to discourage spamming the mempool and block malicious nodes.

While there may be 72000 unconfirmed transactions, the standard mempool and standard Bitcoin Core software drops certain transactions. The site, tradeblock, where you are getting this number from, records the transactions differently. They record all transactions broadcasted, which will include all of the dust and spam transactions that other nodes and miners will drop from their mempools. The 72000 number is not an accurate representation of what the actual number of transactions in the mempool is.

Edit: Take 2, because apparently the first one was wrong.

[Hyena]

knightdk, you simply did not understand what I was suggesting about TX PoW. All your replies were ill formed and overly pessimistic without understanding my idea. If you are a troll, then I would like to have you vaporized.

TX "mining" can be done without impact on devices that have less computing power.

Distinguishing between nodes can be done without linking them to the IP. For example, you can do it based on the signing public key.

That makes your whole long reply invalid.

[achow101]

knightdk, you simply did not understand what I was suggesting about TX PoW. All your replies were ill formed and overly pessimistic without understanding my idea. If you are a troll, then I would like to have you vaporized.

Ok. Perhaps I did not understand. Perhaps you can inform me. I will re-read your posts and see if my understanding changes.

TX "mining" can be done without impact on devices that have less computing power.

How so?

Distinguishing between nodes can be done without linking them to the IP. For example, you can do it based on the signing public key.

Even when done with public keys, if the spammer has setup a ton of nodes and each one has the exact same wallet but different signing keys and ip addresses, each node could send a couple of transactions per second yet when all of the nodes are combined, it is still spamming the network. This solution still wouldn't work. Nodes could filter based on addresses, but even that could still be circumvented.

[Hyena]

TX "mining" can be done without impact on devices that have less computing power.

How so?

2 options. Either the difficulty should be pre set to a reasonable level similarly to how a maximum block size is a constant.
Or, TX PoW difficulty should be dynamic so that at times of attack the difficulty rises temporarily. The latter is much better alternative to what we currently have because then the mempool won't get flooded but instead TXs would get longer to compile.

Distinguishing between nodes can be done without linking them to the IP. For example, you can do it based on the signing public key.

Even when done with public keys, if the spammer has setup a ton of nodes and each one has the exact same wallet but different signing keys and ip addresses, each node could send a couple of transactions per second yet when all of the nodes are combined, it is still spamming the network. This solution still wouldn't work. Nodes could filter based on addresses, but even that could still be circumvented.

So let the spammer setup a ton of nodes. That will be a limited resource which will eventually get depleted. Also, if you read my answer above you will understand that as the attack is carried out the TX PoW difficulty goes dynamically up rendering the attacker's precalculated PoWs useless.

That combined with a block size limit of 8 MiB and minimum TX fees would be enough to stop such attacks. In addition, the protocol can be adjusted to respond to a sudden increase in the number of unconfirmed TXs so that minimum TXs fees would be risen programmatically.

[achow101]

TX "mining" can be done without impact on devices that have less computing power.

How so?

2 options. Either the difficulty should be pre set to a reasonable level similarly to how a maximum block size is a constant.
Or, TX PoW difficulty should be dynamic so that at times of attack the difficulty rises temporarily. The latter is much better alternative to what we currently have because then the mempool won't get flooded but instead TXs would get longer to compile.

consider my edited response quoted below

So you are suggesting that we must mine txs in order to have it be accepted and confirmed? What if the malicious spammer has an ASIC that can calculate these PoWs faster than everyone else and makes them more difficult? That means that he just raised the difficulty of his transactions, thereby increasing his priority. This causes other people to have a lower difficulty than the majority and thus their priority is lower. So what happens to those people that don't have sufficient computing power to calculate a PoW more difficult than the spammers? Does this make their transaction have a lower priority and they are essentially locked out of making transactions?

Distinguishing between nodes can be done without linking them to the IP. For example, you can do it based on the signing public key.

Even when done with public keys, if the spammer has setup a ton of nodes and each one has the exact same wallet but different signing keys and ip addresses, each node could send a couple of transactions per second yet when all of the nodes are combined, it is still spamming the network. This solution still wouldn't work. Nodes could filter based on addresses, but even that could still be circumvented.

So let the spammer setup a ton of nodes. That will be a limited resource which will eventually get depleted. Also, if you read my answer above you will understand that as the attack is carried out the TX PoW difficulty goes dynamically up rendering the attacker's precalculated PoWs useless.

That combined with a block size limit of 8 MiB and minimum TX fees would be enough to stop such attacks. In addition, the protocol can be adjusted to respond to a sudden increase in the number of unconfirmed TXs so that minimum TXs fees would be risen programmatically.

The attacker can do what I said above, use ASICs to continuously drive up the tx difficulty.

[BombaUcigasa]

I noticed my CPU was at 100%.

CPU at 9%: http://188.166.81.128/localdomain/localhost.localdomain/cpu-day.png

43000 transactions: http://188.166.81.128/localdomain/localhost.localdomain/bitcoind_mempoolcount-day.png

78Mb mempool: http://188.166.81.128/localdomain/localhost.localdomain/bitcoind_mempoolbytes-day.png

[Taras]

We've recently crossed over the 80,000 transaction threshold

How are developers responding to this severe limitation of Bitcoin's usage. There are currently 72000 (!) unconfirmed transactions but it seems they don't really want to acknowledge it.

Perhaps set a limit of tx/s to discourage spamming the mempool and block malicious nodes.

This is not right at now there are "only" near 18.000 unconfirmed transactions....

https://blockchain.info/it/unconfirmed-transactions?offset=0

The blockchain.info transaction feed has been proven to be untrustworthy last time we were flooded with transactions, but TradeBlock and other newer sites keep a more accurate number

edit: TradeBlock just died, I give up

[Hyena]

The attacker can do what I said above, use ASICs to continuously drive up the tx difficulty.

The attacker could also have a quantum computer or majority of the mining power or whatever fictional contra argument you can fantasize about.

Since TX PoW is not used for mining and is solely used to reduce spam, the algorithm can be changed with super consensus. Only spammers would be against such protocol changes. Their ASICs would become worthless. You start to seem more and more like a bitcoin traitor, unable and unwilling to come up with solutions and only trying to bash ideas for improvements with rather lame counter arguments.