Users of testnet3 don't have full nodes, running 24/7

[stwenhao]

Just some observation, which can be interesting for some testnet3 users: there are many transactions, displayed on https://mempool.space/testnet which stay unconfirmed for days, even though some blocks, mined by some miners, are far from being full, and these transactions could easily be included by them (because they already include transactions with lower fee rates).

So, why a lot of blocks are empty? Because a lot of users just send their transactions once, and close their clients.

And, what happens, when their nodes are offline? Well, these transactions are never re-broadcasted by anyone. So, when new miners come in, then they don't receive those transactions. They don't include them in their block templates, and they are waiting longer and longer, until some old miner include it in some block.

Some example: https://mempool.space/testnet/tx/882cf91eb089fd83ae26f6f557dc0d5ff4a670b07b405282200c5fd963267329#vin=0

This transaction was unconfirmed for 4 days. And it would probably stay unconfirmed for much longer than that. However:

Code:

$ ./bitcoin-cli -testnet sendrawtransaction 020000000001016e8cdd2c2715e1c9453929489182ab9af4cd8d74241518155a000faa29e96d920200000000fdffffff0242d813000000000022512043d25bc290142d1031a0b22367991bfe1a9f1334c2b232e230377bcf11cdcec642d8130000000000225120d7e7bc85cc1012351c33fc2211ce46a8edba5aa204d75334643a1210f65020da0140853bf59989a9e3a21ec7af6f558626287f14c1e7e48bd590bc22310f09811758044191b7f23c300e8d98cd23fe2f07941e6e49766261ddf762cf4ee3c565c1e700000000
882cf91eb089fd83ae26f6f557dc0d5ff4a670b07b405282200c5fd963267329

And, just one minute later, it is confirmed! (the same was true for more than this one transaction, so I am sure, that it is not just some coincidence; everything I rebroadcasted, was confirmed in minutes, even if it was first seen days ago)

Conclusions: it is a good idea to run a full node (even in pruning mode), if you want to get your transactions confirmed sooner than later. You can have "fee rate" of 856 sat/vB, and "effective fee rate" of 3,048 sat/vB, and wait days for a single confirmation. However, if you just run a full node, and it will re-broadcast your transactions periodically, then not only you can get it confirmed sooner, but also you can make it cheaper, when you hit a miner, who just joined, and don't have fully filled mempool yet (not to mention about full-RBF potential, where you can replace more expensive transaction with some cheaper version, if some miner does not have the properly synchronized mempool state).

Also, it is an interesting observation, when you notice, that people are shouting about Ordinals as "not a spam", but they don't even care about re-broadcasting their own transactions, and they end their journey, when their transactions are displayed in block explorers.

[BlackHatCoiner]

You'll be surprised by how many users of mainnet run their own node.  

So, why a lot of blocks are empty? Because a lot of users just send their transactions once, and close their clients.

You could blame the miners as well. If your only source of (testnet) income is through transaction fees, then you should increase your storage limit and allow more transactions in your mempool, and for longer time.

BTW, why does it have an average block time of 43 seconds? What's the name of the attack it currently experiences?

[stwenhao]

You'll be surprised by how many users of mainnet run their own node.

The same issues can happen in each chain, which copy-pasted Bitcoin Core source code. Which is why even if you are just a regular user, you are incentivized to run some kind of node, to re-broadcast your own transactions.

BTW, why does it have an average block time of 43 seconds?

For the same reasons we have testnet4: every 2016 blocks, the difficulty is adjusted. If you mine a block with CPU difficulty in exactly the right time, then, the code assumes, that it was the network difficulty for the last two weeks. And then, it is resetted. And then, after each re-adjustment, ASICs bring back the difficulty from 1 to the real one.

What's the name of the attack it currently experiences?

Just a regular blockstorm, which is why we have 17 halvings and 38146 satoshis as the base reward in testnet3, even though the chain started in 2011.

You could blame the miners as well.

Of course. I still don't understand, why some users run their ASICs, and include a single coinbase transaction, getting 38146 satoshis, instead of getting 7 tBTC for the same work, and including transactions properly.

In general, all of that is just a remainder, that full nodes are important, and what can happen, when there are not enough of them, running 24/7.

Also, it is yet another remainder, that by default, transaction propagation is quite weak, and there are some tricks, which you can use for a while, as long as some miners won't fill their mempools properly (for example by replacing 100 sat/vB transaction with just 1 sat/vB transaction through full-RBF, if you send it to the right node).

[BlackHatCoiner]

Just a regular blockstorm, which is why we have 17 halvings and 38146 satoshis as the base reward in testnet3, even though the chain started in 2011.

So, in testnet3, when no block is found for 20 minutes, you can quickly mine a long chain of difficulty=1 blocks, and immediately broadcast them, whereas in testnet4, this chain of blocks cannot go more than 2 hours into the future and each block has to have a time distance of 20 minutes + 1 second. Is that right?

[stwenhao]

you can quickly mine a long chain of difficulty=1 blocks, and immediately broadcast them

Exactly. It is called time-warp attack, it is why BIP-94 was introduced, and it is also described here: https://blog.lopp.net/griefing-bitcoin-testnet/

whereas in testnet4, this chain of blocks cannot go more than 2 hours into the future

Two hours rule is present in all networks. But: if you don't have restrictions from BIP-94, then you can pretend, that something took two weeks, even if it is not the case. I think Jameson Lopp described it quite well, how you can fake timestamps.

and each block has to have a time distance of 20 minutes + 1 second

No. If you reset the difficulty, during difficulty adjustment, then you can produce six blocks per second, and after 2016 blocks (mined in 5 minutes and 36 seconds), the difficulty will be changed from 1 to 4. Then, after the next 2016 blocks, it will change from 4 to 16, and so on, until you reach the real difficulty.

But: if you apply Lopp's attack, then you can for example switch the difficulty in different ways, for example: 1 -> 4 -> 1 -> 4 -> 1 ...

The difference between testnet4 and testnet3, when it comes to difficulty adjustments, is that after 2016 blocks, testnet3 difficulty can go for example from 123456789 into 1. But in testnet4, ASICs have to mine at least a single block per two weeks, with the real difficulty. And then, on top of that, you have timewarp rules from BIP-94, which are restricting timestamps further (and are still discussed by developers, because it is still not fully clear, which rules will be deployed into mainnet).

[pooya87]

So, why a lot of blocks are empty? Because a lot of users just send their transactions once, and close their clients.

And, what happens, when their nodes are offline? Well, these transactions are never re-broadcasted by anyone. So, when new miners come in, then they don't receive those transactions. They don't include them in their block templates, and they are waiting longer and longer, until some old miner include it in some block.

There is no correlation between running a node + staying online and having your transaction mined. When a transaction is pushed to the network (whether through your own full node or a SPV client or even an online tool to push it) it becomes part of the transactions in other full nodes' (aka peers') mempool on the network.

A new miner connects to a node or a pool that was already online. If a new node comes online they will request mempool from other peers and they will receive the transactions in their mempool which will include the tx pushed by someone who went offline.

And, just one minute later, it is confirmed! (the same was true for more than this one transaction, so I am sure, that it is not just some coincidence; everything I rebroadcasted, was confirmed in minutes, even if it was first seen days ago)

It could be a bug with mempool.space website and node they run in the backend! My guess is that it is a lack of relay problem. For example if you check some of the transactions found on https://mempool.space/testnet/mempool-block/0 in other block explorers you can see that they cannot be found elsewhere.
Example:
https://mempool.space/testnet/tx/9249f922fbf243ccd1107f9a25db1ed1372dc876b585bdd8f98aab8ad5eea0e5
Says it was first seen 3 days ago but it cannot be found on other explorers
https://live.blockcypher.com/btc-testnet/tx/9249f922fbf243ccd1107f9a25db1ed1372dc876b585bdd8f98aab8ad5eea0e5
https://blockstream.info/testnet/search?q=9249f922fbf243ccd1107f9a25db1ed1372dc876b585bdd8f98aab8ad5eea0e5

Maybe these transactions were pushed directly through their service here https://mempool.space/testnet/tx/push but their node doesn't relay them to the rest of the network. Therefore those txs stay only in their node and show up only in their platform. Meaning no miner sees them to mine them.
This explains why they were mined when you manually download the raw tx from their website and push it using your own node.

Edit:
Here is an experiment to show you don't need a full node and the problem is with mempool.space.
Take any tx from their UI like https://mempool.space/testnet/tx/0ad727680449c102b36baacc1f33cff09da69c42655710af25588632291cf0c6#vout=0
and download the raw hex. Then go to any explorer like https://live.blockcypher.com/btc/pushtx/ or https://blockstream.info/testnet/tx/push to push the tx. It gets confirmed right away like the one I experimented with and was confirmed already after 2 months.

[stwenhao]

It could be a bug with mempool.space website and node they run in the backend!

I guess you are right. I saw transaction counter increased from 100k into 400k, but my node received none of that.

Which means, that someone is just pushing transactions to their block explorer, without sending any of that to the rest of the network.

[BlackHatCoiner]

Mempool.space probably has more bugs. In the testnet4 explorer, this address[1] is shown as having minus 20 thousand something bitcoin. It's probably due to a miscalculation that occurs in their backend when the chain is reorged. (It happens when this miner reorgs the chain, while there is a transaction that spends from his address reorged.)

[1] https://mempool.space/testnet4/address/tb1qwq8yt3w6k2yrjr2nwq2a59wny3f3ns2xx3fgg9