Future of testnet3 network.

[ABCbits]

I didn't see anyone discuss about it in this forum, but last week Jameson Lopp decide to attack testnet3 network to prove a vulnerability[1]. If you either attempt to use testnet3 recently or follow the news, you probably know it's not easy to obtain testnet3 coins these days. On top of that, recent Runes hype makes some people decide to add arbitrary data to testnet3 blockchain[3]. With some developer discussing new testnet network (not Signet)[4], i wonder how long will testnet3 remain usable and supported.

[1] https://beincrypto.com/jameson-lopp-bitcoin-testnet-griefing-attack/
[2] https://blockchair.com/bitcoin/testnet/
[3] https://protos.com/halving-excitement-pumps-worthless-bitcoin-testnet/
[4] https://groups.google.com/g/bitcoindev/c/9bL00vRj7OU

[bitmover]

Sharing his tweets about the attack

https://twitter.com/lopp/status/1781276372729524640
https://twitter.com/lopp/status/1780548492219318402

I discovered this website where this guy is claiming that users get rewarded with airdrops when using testnet, other sell testnet coins in swap services... so some users/dev are unsatisfied and are trying to get it down or change it.

https://coinfaucet.eu/en/btc-testnet/
All of this arose because the testnet is used to test applications (which, by the way, is the essence of the Bitcoin testnet) and people are rewarded for it in the form of AirDrops. Another problem they see is that the testnet started trading publicly about a year ago on altquick.com (the closed market has been around for a long time...) which is completely logical given its similarity to BTC (limited supply, mining...) Yes, I have no problem with that. The market also has positives, for example, people who need a quantity they cannot obtain from available sources will not misuse services such as faucets. People will not HODL testnet, it will circulate again, and people who buy testnet, will sell it again after use...

[Timelord2067]

There are still faucets dispensing testnet coins - I donated a small portion of my stockpile to one of the newer ones that was launched here a year or two ago.

I note also that they are being traded - that was only a matter of time.  Now, it looks as though people want to block those trades by resetting the testnet itself (which could cause consensus issues down the track)...

[pooya87]

it's not easy to obtain testnet3 coins these days.

There has always been faucets giving coins and the only complains I've seen so far have been from people who want large amounts and the worst part is that I have not seen any reasonable arguments for needing such amounts as opposed to using something like regtest.
So I wouldn't say "it is not easy to obtain testnet coins".

What the future holds?
This attack is not going to change anything, after all this is not the first time testnet is under a miners' attack.
But if the trades continue and testnet coins start gaining a value and scams form around it, we can see testnet3 having the same fate as its predecessors and we may enter testnet4 era.
Although if we do, I think we should come up with better mining rules that are not complicated like signet and not as vulnerable like testnet3.

[NotATether]

What the future holds?
This attack is not going to change anything, after all this is not the first time testnet is under a miners' attack.
But if the trades continue and testnet coins start gaining a value and scams form around it, we can see testnet3 having the same fate as its predecessors and we may enter testnet4 era.
Although if we do, I think we should come up with better mining rules that are not complicated like signet and not as vulnerable like testnet3.

It is absolutely impossible to test anything on testnet as long as the coins have a value. Testnet coins are supposed to be worthless in order to deter people from swiping them from automated testing processed. Or in the event that they do get swiped, it's supposed to be easy to replenish them with more coins obtained from somewhere else. The fact that testnet coins are so scarce means that you can't even rely on that anymore, and if you want to test stuff then you need to simulate your own blockchain with a few (~3-5) blocks.

You don't need a lot of testnet coins to test stuff but you do need enough to cover the fees for long enough that the tests can run for several thousand iterations. So we are looking at around 0.1 to 1 tBTC for each project.

[vjudeu]

I think we should come up with better mining rules that are not complicated like signet and not as vulnerable like testnet3.

True. The big question is: what should be the new consensus rules of testnet4?

Because the story is that developers already tried to make a better version of the testnet, and they came up with signet. Which means, that in 2020, nobody had any good idea, how to make a good test network, without creating a permissioned network, where only creators could mine new blocks.

new testnet network (not Signet)

I know it is not about signet, but about testnet4, but I guess those two networks will be very similar. In general, if nobody has any good idea, then the status quo is what happens. Which means, that if nobody will invent anything better, then testnet4 will be the same as testnet3, but with a different Genesis Block, and without blockstorms. So, basically, it will be just like signet, but with signet challenge set to OP_TRUE.

it's not easy to obtain testnet3 coins these days

This is not a bug, it is a feature. In case of the network reset, we may lose one important property: having a chain with a lot of halvings. And this is a problem, because the amount of test coins is what directly represents your ability of publicly sharing your tests. Which means, that if you have 50 tBTC, then you can push 5 GB of test cases. So, the amount of test coins is not something, that should correspond to the real values, used in mainnet. It is mainly about anti-spam features, and preventing people from flooding the network with their data.

On top of that, recent Runes hype makes some people decide to add arbitrary data to testnet3 blockchain

This is the reason, why I agree with Garlo Nicon, that the new testnet should preserve existing halvings. So, even if all existing coins will be burned, then the block reward should not start from 50 tBTC, but rather from 0.01220703 tBTC, which we have today. Because the amount of coins you have, is what decides about your ability to push data. Which means, that if testnet4 will start from 50 tBTC, then each and every miner will get an ability to push 5 GB of data, and the new network will be flooded with data pushes from the day one, instead of being used to test payments.

[NotATether]

it's not easy to obtain testnet3 coins these days

This is not a bug, it is a feature. In case of the network reset, we may lose one important property: having a chain with a lot of halvings. And this is a problem, because the amount of test coins is what directly represents your ability of publicly sharing your tests. Which means, that if you have 50 tBTC, then you can push 5 GB of test cases. So, the amount of test coins is not something, that should correspond to the real values, used in mainnet. It is mainly about anti-spam features, and preventing people from flooding the network with their data.

This can be rectified by creating a new halfchain which is like testnet but the halving block lengths are significantly lower.

Maybe even signet can be adapted for this so that a new chain doesn't have to be created, but I don't know if that is possible or if it would require a new chain entirely. That being said, the Bitcoin Core team is not too thrilled about the prospect of modifying the current chains.

[vjudeu]

Maybe even signet can be adapted for this so that a new chain doesn't have to be created, but I don't know if that is possible or if it would require a new chain entirely.

You can always claim less coins in the coinbase transaction, than you are allowed to. It is a perfect no-fork, not even a soft-fork. If you can claim 0.01220703 tBTC today, then it is the upper amount. You can as well always claim zero coins in the coinbase transaction, and your blocks will be perfectly valid in all networks: mainnet, signet, regtest, and testnet3.

That being said, the Bitcoin Core team is not too thrilled about the prospect of modifying the current chains.

Yes, but the situation is not so simple. If developers could simply pull off the plug, and destroy testnet3, by detaching it from the current client, then they would probably do so. But instead, they ask, if there should be any "grace period" or something, because they know, that people can simply stay with the old software. And in that case, testnet3 may become just another altcoin, probably much closer to the mainnet than many other altcoins.

And also, there is some risk of replaying coinbase transactions between networks. Which means, that if you have some block 1,234,567 in the current testnet3, then you can migrate exactly the same coinbase transaction into testnet4. And similar things can happen between signet and testnet4, or even mainnet and testnet4. One of the things, preventing that from happening, is the fact that testnet3 has much more blocks than other networks. Which means, that blocks from testnet3 can be copy-pasted into other networks, but not the other way around, because it would require a huge chain reorganization in testnet3.

[ABCbits]

I don't know whether any of you run testnet full node, but i just found out my testnet full node has total size 71GB.

Code:

$ du -d 2 -h
594M    ./indexes/blockfilter
361M    ./indexes/coinstats
6.0G    ./indexes/txindex
6.9G    ./indexes
450M    ./blocks/index
56G     ./blocks
7.8G    ./chainstate
71G     .

I'm also surprised chainstate folder has size 7.8GB. So i decide to run gettxoutsetinfo and got following statistic which state there are currently 116.8 million UTXO.

Code:

{
  "height": 2810692,
  "bestblock": "000000000000009910d22b6f04ef25d718350863560ea24a85da54e666127e4f",
  "txouts": 116842453,
  "bogosize": 9425932147,
  "hash_serialized_3": "ba355559e0d1c5de59ec7e512a7ef374e0df01be653d60424db397cd84dae801",
  "total_amount": 20997039.32915585,
  "transactions": 64934575,
  "disk_size": 8364679205
}

it's not easy to obtain testnet3 coins these days.

There has always been faucets giving coins and the only complains I've seen so far have been from people who want large amounts and the worst part is that I have not seen any reasonable arguments for needing such amounts as opposed to using something like regtest.
So I wouldn't say "it is not easy to obtain testnet coins".

You're right, but mining tBTC become more challenging. Not only due to lower reward, but also due to higher hashrate/difficulty[1]. And i notice there are less tBTC faucet these days. At least, it's not as easy as few years ago.

[1] https://mempool.space/testnet/graphs/mining/hashrate-difficulty#all

[NotATether]

I'm also surprised chainstate folder has size 7.8GB. So i decide to run gettxoutsetinfo and got following statistic which state there are currently 116.8 million UTXO.

Code:

{
  "height": 2810692,
  "bestblock": "000000000000009910d22b6f04ef25d718350863560ea24a85da54e666127e4f",
  "txouts": 116842453,
  "bogosize": 9425932147,
  "hash_serialized_3": "ba355559e0d1c5de59ec7e512a7ef374e0df01be653d60424db397cd84dae801",
  "total_amount": 20997039.32915585,
  "transactions": 64934575,
  "disk_size": 8364679205
}

I think that is larger than even the bitcoin mainnet's chainstate, right? It was at 6GB or so the last time I checked its size.

But the fact that there is a ballooning UTXO set on testnet does not surprise me at all, as people rarely spend their coins after they are finish testing something. At best, they just return it to the faucet they got it from.

[ABCbits]

I'm also surprised chainstate folder has size 7.8GB. So i decide to run gettxoutsetinfo and got following statistic which state there are currently 116.8 million UTXO.
--snip--

I think that is larger than even the bitcoin mainnet's chainstate, right? It was at 6GB or so the last time I checked its size.

Mainnet's chainstate is still bigger at 11.3GB, with about 50% more total UTXO.

But the fact that there is a ballooning UTXO set on testnet does not surprise me at all, as people rarely spend their coins after they are finish testing something. At best, they just return it to the faucet they got it from.

You might be right, but i recall chainstate/UTXO was far smaller a year ago. I need to dig the data for a while to prove that though.

[pooya87]

You don't need a lot of testnet coins to test stuff but you do need enough to cover the fees for long enough that the tests can run for several thousand iterations. So we are looking at around 0.1 to 1 tBTC for each project.

I honestly never understood this and I've seen so many of those who ask for high amounts of tbtc say it. What are these tests that need to run "several thousands" times in a live network that can not be abstracted away using IoC design or something like that or even run in Regtest instead.

And also, there is some risk of replaying coinbase transactions between networks. Which means, that if you have some block 1,234,567 in the current testnet3, then you can migrate exactly the same coinbase transaction into testnet4.

But coinbase tx is just a tx. There is nothing special about it in this context ("replaying" between networks). You set the version and the outputs of that tx and you get the same exact tx.

Which means, that blocks from testnet3 can be copy-pasted into other networks, but not the other way around, because it would require a huge chain reorganization in testnet3.

No you can't. Because the difficulty is different and also any block in any network is positioned inside a chain. You can't take out a single block from the middle of that chain and place it on top of another chain.
For example block 1,234,567 in Testnet3 is referencing block 1,234,566 on Testnet3 if you take that and try to place it on top of for example Mainnet chain at 1,234,567 it is rejected because the 1,234,566 reference is not found.
This chain also goes back all the way to a hardcoded Genesis block that is unique to that specific chain to prevent exactly what you explained.

[vjudeu]

No you can't.

https://bitcointalk.org/index.php?topic=5479988.msg63437260#msg63437260

As you can see, we have transaction ID 0f3601a5da2f516fa9d3f80c9bf6e530f1afb0c90da73e8f8ad0630c5483afe5 in the mainnet, and exactly the same thing is repeated in regtest. And in general, transactions can be migrated in such way, while preserving "the chain of signatures".

Because the difficulty is different

Of course, that "replay" is not "effortless". You have to mine a valid block header. But the problem (or the feature) is, that such transactions can be migrated. Which means, that even if someone will destroy testnet3, then exactly the same transactions, with exactly the same transaction IDs, can be replicated in testnet4, if there will be enough incentive to do that (and because some people started assigning value to testnet3 coins, then it may happen).

You can't take out a single block from the middle of that chain and place it on top of another chain.

The only needed thing is to match the coinbase transaction. If that matches, then the whole chain of transactions is also matching. In general, if you recreate all UTXOs, used as inputs in a given transaction, then it can be replayed. Which also shows us, that if we want to replay something, then we have to trace that, only to the nearest coinbase transaction, because everything before that is just unsigned, and sent as fees, which can be taken from any transactions. The only requirement is to have matching amounts.

For example block 1,234,567 in Testnet3 is referencing block 1,234,566 on Testnet3 if you take that and try to place it on top of for example Mainnet chain at 1,234,567 it is rejected because the 1,234,566 reference is not found.

Of course, but note that the coinbase transaction does not contain the previous block hash. It contains only the number of the block. Which means, that such transactions can be replayed between unrelated networks.

This chain also goes back all the way to a hardcoded Genesis block that is unique to that specific chain to prevent exactly what you explained.

Of course, but the chain of block headers is the only unique thing per network. The chain of transactions, and the chain of signatures, can always be replayed. For example: you send 50 BTC in the mainnet block 1,234,567 as fees, and you also send 50 tBTC in the testnet3 block as fees. And then, if you mine both blocks in both networks, then you can use the same coinbase transaction, with the same amount, and reach identical transaction IDs in both networks. And since then, you will have a connection, similar to the one existing between BTC and BCH, where you have a lot of shared history. But here, it goes further, because you don't have to start from the same point, to meet in the same transaction. Because if something is sent as fees, then starting transactions can be different in both networks, but the final outcome could lead to identical transaction data.

[stwenhao]

But coinbase tx is just a tx. There is nothing special about it in this context

If you can replay transactions, then it means, that you can have two different chains, with two different histories, and they can end up with the same UTXO set. Or rather: a subset of their UTXO set can be identical in both. And the consequence is, that those coins will be connected, and if all of your transaction inputs are in that intersection, then they can exist in both networks. And the final consequence is "network migration", where you can set up the same UTXO set, and then just discard the original chain of block headers, and simply switch to the stronger one, because the whole set of data will be identical, so it will make no difference for the end user.

Also note that developers didn't get as huge hashrate in signet, as we have in testnet3. Which means, that if testnet4 will be almost the same as testnet3, then you will have a similar set of miners. And they may have a huge incentive, to just migrate the history, and get their tokens back in the new network. Which means, that a different set of rules is needed to prevent that, because if not, then you will have similar things in your mempool for testnet4, as you have in your mempool for testnet3.

[garlonicon]

I think we should come up with better mining rules that are not complicated like signet and not as vulnerable like testnet3.

I wonder, what do you think about using a subset of testnet3 for testing? I mean, you pick some kind of "challenge", similar to signet, and scan the whole chain of your choice (for example testnet3). And then, your UTXO set contains only a subset of the "real testnet3", matching that challenge. In that case, you can still test things, like in signet (for example if you pick your own signet challenge), but at the same time, you can import tests, provided by other people. And then, each node can have its own set of rules for accepting or rejecting transactions.

In that model, the network can be resetted at any time, by anyone, just by switching the challenge. But at the same time, you can pick and import any transactions at will, so if you will see anything interesting, that is done by someone else, then you can import it to your challenge, and get those UTXOs.

In general, I think such "Scanners" are worth exploring, because if you don't like something, that happens in testnet3, then it could also happen in testnet4, unless you will have some kind of "demurrage". And I think, that kind of "demurrage" is the most open for modifications, as it could be.

[bitmover]

I'm also surprised chainstate folder has size 7.8GB. So i decide to run gettxoutsetinfo and got following statistic which state there are currently 116.8 million UTXO.
--snip--

I think that is larger than even the bitcoin mainnet's chainstate, right? It was at 6GB or so the last time I checked its size.

Mainnet's chainstate is still bigger at 11.3GB, with about 50% more total UTXO.

Testnet still has way more blocks. I didn't knew about that.

At time of posting, testnet is on block 2.800 million, while mainnet is still 800 thousands..

https://mempool.space/testnet/block/00000000000000cf5c1d4fda088f2623c34643c7677ffa6d71e0b3e62214d39b

This is due to the fact that the difficulty resets to 1 regularly on testnet.

[Medusah]

Could anyone clarify this for me?  Why does testnet3 have halvings?  Given that the coins hold no value, there seems to be no rationale for imposing a supply cap.  It might be better to maintain it at 50 tBTC per block indefinitely, ensuring everyone always has an equal opportunity to earn a sufficient amount.  With halvings, it's only a matter of time before the network resets due to the absence of block rewards.

I think that is larger than even the bitcoin mainnet's chainstate, right? It was at 6GB or so the last time I checked its size.

That's correct.  Spamming on the mainnet is discouraged because... you're paying actual money.  In the testnet, you could execute a spam attack at no expense. 

[vjudeu]

Why does testnet3 have halvings?

Because it is as close to the mainnet, as possible. In the first version, even the Genesis Block was identical, so it was possible to literally overwrite the whole chain. Then, some small changes were introduced here and there, people switched to some new Genesis Block, and added the famous rule of resetting the difficulty, which caused blockstorms later.

Given that the coins hold no value, there seems to be no rationale for imposing a supply cap.

Ideally, the test network would have all rules, identical to the mainnet, but where you could experiment with worthless coins. And the main challenge is: what set of rules are needed, to make coins worthless, and to not departure from the mainnet by too much?

It might be better to maintain it at 50 tBTC per block indefinitely, ensuring everyone always has an equal opportunity to earn a sufficient amount.

In that case, how do you test halvings, if they would not be present in the test network? Not to mention, that if you have no halvings, then after block 420,000, the whole supply will reach exactly 21 million coins, which means, that there will be some potential, to trigger some rules from the Value Overflow Incident. Because that amount "21 million" is hard-coded in some places, and if you have bigger supply, then some tests may fail. And also, if you have too many coins, then if you have conversions between integers, and floating-point types, then your representation may not be exact, and you may lose some precision.

In the testnet, you could execute a spam attack at no expense.

And if you have no halvings, then you can always get a lot of fresh coins to spam the network at will. Because the amount you use, is correlated with the amount of data, which you can publicly share.

[Cricktor]

It's quite some months ago that I last ran a Core Testnet3 full node synced on one of my virtual machines. I remember some numbers like in the ballpark around 30GB for blocks, chainstate and indexes. Frankly, I'm shocked to see the current size easily doubled and more (can't remember from when my low numbers were).

And at the moment I also see blocks with nearly 4MWU size, insane! (OK, ok, if you want to test full blocks, they need to be filled to the limits.)

What's next? Inscribers abusing Testnet3 or other data spam dumped into the Testnet3 blockchain. WTF, this is pure madness!

[pooya87]

No you can't.

https://bitcointalk.org/index.php?topic=5479988.msg63437260#msg63437260

As you can see, we have transaction ID 0f3601a5da2f516fa9d3f80c9bf6e530f1afb0c90da73e8f8ad0630c5483afe5 in the mainnet, and exactly the same thing is repeated in regtest. And in general, transactions can be migrated in such way, while preserving "the chain of signatures".

The "can't" I said is the answer to this statement of yours:

Which means, that blocks from testnet3 can be copy-pasted into other networks

You are saying "blocks" not a sequence of transactions that start from a duplicated coinbase tx. That threw me off the issue you are raising.

~And they may have a huge incentive, to just migrate the history, and get their tokens back in the new network. Which means, that a different set of rules is needed to prevent that, because if not, then you will have similar things in your mempool for testnet4, as you have in your mempool for testnet3.

You are forgetting that the mess we are seeing is something that started recently and we are at block height 2.8 million. Only the coinbase tx of the same height can be duplicated, meaning you can NOT have coinbase of block #2.8 million as block #2 and create that "sequence" of UTXOs since the height is included inside the coinbase signature script.

If we migrate to version 4 (a brand new chain from block #1) and make the difficulty manipulation harder, it would take many years (50+) to get to block #2.8 million and by then nobody would even remember the said shittokens.