Fixing Testnet4: proposal

[BayAreaCoins]

I have completed the technical brief of the softfork: https://batmanbytes.github.io/testnet4-softfork/. I've tried to keep it clean, short and descriptive enough. This will be read primarily by miners, so we don't want to make it difficult to read by over-explaining the technical details.  

I believe the next step is direct outreach via email.

In theory, what if I or my service AltQuick.com don't get an email? (or perhaps we are lazy or don't care or whatever)

Nothing at all?  People keep testing?

[stwenhao]

what if I or my service AltQuick.com don't get an email?

Well, new blocks are compatible with the old network. But old blocks, mined by CPUs, are incompatible with new rules. Which means, that as long as ASICs will keep enforcing the new rules, then all blocks with minimal difficulty will be reorged in all nodes. And new nodes will accept only ASIC blocks after activation.

The only case I can see, where the chain would split, is if ASICs will build blocks on top of CPUs, which would detach the old nodes from the network. But even then, it will be a matter of hashrate, which means, that the new nodes will constantly try to build ASIC-only chain. And if it will be stronger, then old nodes will be later forced to accept it, because it will be seen as a chain reorganization on their side.

Nothing at all?  People keep testing?

Old nodes can be tricked into accepting a chain with CPU blocks for a while. But if the majority of ASICs will keep reorging them, then it will work.

Also, because of the changes in the mining code, people can start enforcing new rules today. The only thing, which would happen at block 151,200, is hard-rejecting CPU blocks. But if enough miners will upgrade here and now, then the network difficulty can start falling now, which would reduce the amount of min-difficulty blocks produced today. And then, if there would be a lot of ASIC blocks, produced every 10 minutes, then there will be no CPU blocks, long before block 151,200, if enough miners will upgrade.

[hmbdofficial]

Also, because of the changes in the mining code, people can start enforcing new rules today. The only thing, which would happen at block 151,200, is hard-rejecting CPU blocks. But if enough miners will upgrade here and now, then the network difficulty can start falling now, which would reduce the amount of min-difficulty blocks produced today. And then, if there would be a lot of ASIC blocks, produced every 10 minutes, then there will be no CPU blocks, long before block 151,200, if enough miners will upgrade.

I believe chain stability and reorg depth for old now and any other non- upgraded clients will be Accepted by the CPU mined as valid only if it has most cumulative work. But if the large portion of the hash rate still remains on old software the CPU chain could just pull ahead substantially in terms of of chainwork before enough ASICs switch over right? Why do I think it may cause double spending, and wallet confusion as well,because Reorging a deep chain is extremely disruptive. How do you handle that?

[stwenhao]

Reorging a deep chain is extremely disruptive. How do you handle that?

This is just testnet. Previous testnets had literally hundreds of reorged blocks.

it may cause double spending, and wallet confusion as well

Which is why exchanges like AltQuick require 100 confirmations. Also note, that the only reorged thing is related to CPU blocks, so regular users are unaffected. And coinbase transactions below 100 confirmations can be always reorged, even in mainnet, just because they are unspendable before reaching it.

In general, I think BlackHatCoiner handled it correctly. As long as it affects only testnet4, and nothing else, like mainnet, it should be safe to use. And it is definitely better than nothing, now it is just a matter of convincing miners to use it. If the difficulty will start dropping, then users will no longer have to wait an hour for ASIC blocks, and could get things confirmed every 10 minutes, as it should be.

But if the large portion of the hash rate still remains on old software the CPU chain could just pull ahead substantially in terms of of chainwork before enough ASICs switch over right?

Only ASIC blocks affect chainwork in a meaningful way. A single ASIC block could reorg millions of min-difficulty blocks. Also, splits are rarely 50/50, so it is more likely to see 10/90 split, or similar. And then, the question is: where the hashrate majority would be? Transactions made by users are only confirmed in ASIC blocks anyway, so reorgs will mainly affect only miners. It is very likely, that transactions made by users will land in both chains, so they will be confirmed everywhere.

[LoyceV]

The only case I can see, where the chain would split, is if ASICs will build blocks on top of CPUs, which would detach the old nodes from the network. But even then, it will be a matter of hashrate, which means, that the new nodes will constantly try to build ASIC-only chain. And if it will be stronger, then old nodes will be later forced to accept it, because it will be seen as a chain reorganization on their side.

Let's say "BHC's chain" gets a higher hash rate, and "old" nodes follow that chain. Does that mean that, say, 2 months from now, those nodes will think the latest block is weeks in the past, and all ASIC miners on that chain instantly produce thousands of CPU blocks each with 20 minutes in between on top of it?

Which is why exchanges like AltQuick require 100 confirmations.

If my scenario above is correct, that's going to be fun

[stwenhao]

Does that mean that, say, 2 months from now, those nodes will think the latest block is weeks in the past, and all miners on that chain produce thousands of CPU blocks each with 20 minutes in between on top of it?

No, because they will accept ASIC blocks, produced by the new chain.

In theory, the chain could split for a while. But later, a chain reorganization will just solve it. Old nodes will have some ASIC blocks, and some CPU blocks, while a new chain will have only ASIC blocks. If the new chain will reach a bigger chainwork, then it will overwrite the old chain.

Also note, that the code for mining things in a new way can be used today, without any soft-forks. The soft-fork part is only about marking CPU blocks as invalid. But mining ASIC blocks with 20 minutes delay can be done here and now, which would lower the difficulty.

[BlackHatCoiner]

In theory, what if I or my service AltQuick.com don't get an email? (or perhaps we are lazy or don't care or whatever)

If you do not upgrade, then your node's chain will periodically experience reorgs. You won't miss the soft fork chain. You will just consider the CPU miners' blocks valid until an ASIC miner sends a block with real difficulty which reorgs all of them.

For you specifically, you should upgrade your AltQuick's node for the user experience. If a user has deposited coins to your exchange, and his transaction reaches 10 confirmations, and an ASIC miner reorgs the past 10 blocks, then the transaction will be back to 0-conf or 1 confirmation. That is just bad UI.

Reorging does not mean replacing. If blocks 130,001 to 130,010 are min difficulty blocks, and an ASIC miner ignores them (because he is running the soft fork), he's mining block 130,001 with real difficulty, and once he does, those blocks will no longer be valid in the non-upgraded chain.

[stwenhao]

For you specifically, you should upgrade your AltQuick's node for the user experience. If a user has deposited coins to your exchange, and his transaction reaches 10 confirmations, and an ASIC miner reorgs the past 10 blocks, then the transaction will be back to 0-conf or 1 confirmation. That is just bad UI.

They require 100 confirmations, so they should be safe.

Reorging does not mean replacing.

From the perspective of a node, working on a minority chain, it looks exactly like that. But fortunately, only coinbase transactions are "replaced" in practice, everything else usually ends up being confirmed by all chains. In the worst case, some transactions could go back to zero confirmations, if some miner will decide to include some double-spends, but still: that miner would need to take part in that transaction. You have to sign your input in a different way, to actually make a double-spend, so if users are just sending transactions between themselves, then they will be safe even in that case.

[ertil]

It seems that your proposal was rejected, and people are going to make testnet5 from scratch: https://groups.google.com/g/bitcoindev/c/kGUMTxOvdJA

https://github.com/fjahr/bips/blob/bip-t5-draft/bip-XXXX.md

Rationale

Instead of starting a new Testnet, changing the rules of Testnet 4 was considered as well. The decision for a new network has two main reasons:

1. Deploying network changes is hard and typically takes a long time. This effort doesn't seem to be worth it for a Testnet that has the alternative option to start fresh, especially considering that the attackers exploiting Testnet 4's rules may use them to prevent a smooth deployment of new rules, including those of BIP 54.
2. Any fix to Testnet 4 that goes beyond a band-aid would require a hard fork, which brings its own implementation cost: adding hard-fork support to clients, handling the p2p layer correctly, etc. For a test network this isn't worth taking on, especially since a band-aid would only make Testnet 4 diverge further from mainnet.

Technically, you can still try to do that, if enough miners will upgrade, but if Bitcoin Core is going to drop support for testnet3 and testnet4 sooner or later, then they may become just some abandoned altcoins.

I didn't expect people to wake up, and suddenly try to launch testnet5, but here we are.

[BayAreaCoins]

It seems that your proposal was rejected, and people are going to make testnet5 from scratch: https://groups.google.com/g/bitcoindev/c/kGUMTxOvdJA

https://github.com/fjahr/bips/blob/bip-t5-draft/bip-XXXX.md

Rationale

Instead of starting a new Testnet, changing the rules of Testnet 4 was considered as well. The decision for a new network has two main reasons:

1. Deploying network changes is hard and typically takes a long time. This effort doesn't seem to be worth it for a Testnet that has the alternative option to start fresh, especially considering that the attackers exploiting Testnet 4's rules may use them to prevent a smooth deployment of new rules, including those of BIP 54.
2. Any fix to Testnet 4 that goes beyond a band-aid would require a hard fork, which brings its own implementation cost: adding hard-fork support to clients, handling the p2p layer correctly, etc. For a test network this isn't worth taking on, especially since a band-aid would only make Testnet 4 diverge further from mainnet.

Technically, you can still try to do that, if enough miners will upgrade, but if Bitcoin Core is going to drop support for testnet3 and testnet4 sooner or later, then they may become just some abandoned altcoins.

I didn't expect people to wake up, and suddenly try to launch testnet5, but here we are.

We will see if this actually gets done or if this is another January 1st missed date with no follow up messages.


Honestly, I think this is the best call.  

They fucked Testnet4 up beyond belief. Perhaps 5 will do better, technically speaking.

This is exciting and good news for Bitcoin Testnet & Bitcoin IMO.

(Yes, AltQuick likely will be listing Testnet5.)

Testnet4 maybe a great option for people when Testnet5 market cap exceeds BCH. har har

Interestingly, Testnet3 has traded over 0.014 btc this week and Testnet4 only saw a small amount of sales action from this news... Testnet is just a different beast.

She's a honey badger.

It will not, using a mainnnet block hash in the Pubkey field acts as an anti-premine-commitment.

I was tickled shitless to read this... seems to be the right path IMO.


They only care about it because of the little insider BIP 54 shit they got going on this week and their last testing (Citrea) left testnet development is shambles, but they premined v4, so they didn't feel the sting most of y'all did... It's not for strangers, it's for them.  But whatever, a fix is a fix.  I'm glad they are doing it with what appears to be taste and thought.

This launch looks and feels much better.  At least we got some adults' attention.

Keep in mind, anything that can happen in Testnet *could* happen with Bitcoin.

Also, the mining network is going to spike and drop.  People testing next-gen miners will use this network to test their future Bitcoin-dominating miners; having the network freeze up will bring attention to this.  Greg noticed ASIC BOOST due to Testnet.  This should actually discourage value as well from normies, but really, it's all neckbeards as is in and around Testnet.

Public mining pools will exist and thrive in v5 as well, which is really good for everyone, I suspect.  Now people will actually be able to mine dust with their computer on a public pool.  A small, worthless miner will be able to team up with other worthless miners with someone who has a big miner and no patience.  Everyone will get dust on what I suspect will eventually be 20% of the blocks.

It's going to be cool.


stwenhao, I assume you are going come back to mining Testnet when it's "fair" again?

[stwenhao]

stwenhao, I assume you are going come back to mining Testnet when it's "fair" again?

I guess not, because if it would be instantly traded, then it would quickly defeat the whole point of having a testnet. For now, I focus more on regtest, because many things can be tested there, including Proof of Work scripts. And because you wouldn't list a network, which is intended to be running locally, because it is too unstable to gain any value. You wouldn't dare to list something, where there are six blocks per second, where people could produce new blocks faster, than you would be able to validate them, and where everyone would try to constantly reorg the whole chain, to get any coins at all. To play with regtest safely, you would need to build more safety walls, to not endlessly burn all of your resources without any reason, and definitely 100 confirmations wouldn't be enough, because it would take around 17 seconds to reach it, so you probably won't touch it, which means it is, and would be worthless, and I can safely deploy my ideas there.

Also note, that you are talking to a person, who uses only CPUs for mining. And it is sufficient to develop things. Satoshi also started with CPUs, and later, he left the network, where GPUs, ASICs, and the rest of the community pushed everything forward, and made it more serious. I am more like a developer, than a miner, so I think more about designing a better payment system, and improving what we have, rather than about putting Proof of Work into it, beyond what I can get on CPUs alone.

Who knows, maybe I will give it a shot, and join some mining pool, to move some satoshis around. But I am more focused on pushing things, which I already pushed on-chain, which is a Proof of Work inside a Script. It works today, and it can be used everywhere, including mainnet. I still think Bitcoin needs sidechains, but I don't like the way Paul Sztorc is trying to launch it, in a hard-fork way, where soft-forks, and no-forks are technically possible; I also don't understand, why his coin won't use Merged Mining with BTC, or other SHA-256 chains.

To sum up, now I am playing with regtest, the only network, that is truly worthless, where things can be safely tested, and then deployed to any chain, which copy-pasted Bitcoin Core's code. I think by listing signet, you just proved, that Bitcoin Core developers are becoming Altcoin Core developers accidentally, by launching new testnets, so a different approach should be invented, or people should just stick with regtest, and check things locally.

[LoyceV]

Will Testnet5 start with the same low difficulty again, so ASIC miners can run through many difficulty adjustments in minutes, or will it start with a much higher difficulty to prevent that?

Correct me if I'm wrong in this estimate:
Difficulty 1: 2016 blocks in minutes
Difficulty 4: 2016 blocks in minutes
Difficulty 16: 2016 blocks in minutes
Difficulty 64: 2016 blocks in minutes
Difficulty 1024: 2016 blocks in minutes
Difficulty 4096: 2016 blocks in minutes
Difficulty 16384: 2016 blocks in 20 minutes
Difficulty 65536: 2016 blocks in 1.5 hours
Difficulty 1M: 2016 blocks in 6 hours
Difficulty 16M: 2016 blocks in 1 day
Difficulty 64M: 2016 blocks in 4 days
Difficulty 200M: 2016 blocks in 2 weeks
The first 22k blocks and 1M testcoins will be mined within a week. It may not be a premine, but it could just as well be one. Anyone actually using testnet for testing will still need a faucet.

[stwenhao]

Will Testnet5 start with the same low difficulty again

You have that answer in a new BIP: https://github.com/fjahr/bips/blob/bip-t5-draft/bip-XXXX.md

Other parameters (Difficulty: 0x1d00ffff, Version: 1) should match Testnet 4.

Which means, that 20k, maybe 30k blocks, are going to be mined quite fast, and then it will slow down. And whoever will be the creator, will suddenly get a lot of coins, if that creator would have any ASIC, or would be able to do the same thing, that it was in testnet4, where only those, who could build it from the source code, could mine the first 40k blocks in practice, because it was not included into Bitcoin Core before.

Actually, I wonder if that kind of barrier is the most efficient one: if building the client, or doing something technical is needed, to join a given network, then it has very low usage, which allows accumulating a lot of coins by the creator, which is kinda equivalent to premine.

But anyway, Bitcoin was not so popular in 2009. Almost everyone, who joined the project, already saw a chain with many blocks. But of course, the starting difficulty at that time, was picked more honestly, when Satoshi started with 40-bit zeroes, and lowered it to 32-bit zeroes, which is why the Genesis Block has more work, than the next blocks, even if the chainwork is still counted as minimal.

However, all of that matters only, if it would be traded. If not, then it really doesn't matter, who will get the first 20k, 30k, or 40k blocks. And because testnet creators don't care about making it "honest for trading", then it is, what it is.

[BayAreaCoins]

Ya, it's a type of premine if it isn't announced and dropped.

Which, meh, whatever.  I don't think any of us expect a drop date, but it would be super cool.

It doesn't matter if they trade or not; life will go on.

At least it is being fixed and will benefit Bitcoin.