Time for another testnet reset?

[Gavin Andresen]

I'm starting this thread to brainstorm rule changes for -testnet and talk about a possible testnet reset for the next release.

The testnet isn't currently usable, because hashing power on it is unpredictable.  Difficulty shoots up because somebody decides to throw lots of machines at it, then they leave and it takes MONTHS for difficulty to drift back down.

Here's a shot at hair-brained rules for the testnet:

+ Fix the difficulty at 1.0, no matter how many people are mining.

+ Clients reject new blocks with timestamps more than 1 minute different from their time (implies:  you have to have an accurate system clock to play with the other nodes on the testnet, otherwise you'll be on your own fork).

+ Clients reject new blocks if their timestamp is less than 2 minutes after the previous best-block's timestamp.

+ Clients prefer to build on blocks that include more memory-pool transactions, instead of taking first-block-they-receive.

Goals behind those rules:
  Always easy to mine
  Limit block-chain growth (max 1 new block every 2 minutes)

So: could a griefer make life miserable for the rest of us under those rules?  What would happen if there were five or six people with a fair bit of hashing power all trying to mine as fast as possible on testnet?  (lots of block chain splits, I think, but that's probably just fine on testnet)

[1715392684]

1715392684

[1715392684]

1715392684

[1715392684]

1715392684

[1715392684]

1715392684

[1715392684]

1715392684

[BTCurious]

Ignore me if this is a stupid idea, but:

If the difficulty is the problem, can't you just increase the adjustment speed of the difficulty, and leave the rest the same? This would keep the testnet concept nearly entirely the same as the prod net, which makes it as useful as possible.

[dree12]

What about adjusting the difficulty without cap every block based on the speed of the last 6 or so blocks? Then, even if a griefer shot the difficulty up, it wouldn't take much time at all for it to come down.

[BeeCee1]

Having different rules make it seem more like an entirely different network than a testnet, it also makes it less effective as a test platform.  Would it be possible to automatically reset it once a week?

[EhVedadoOAnonimato]

Adjusting difficulty more frequently is probably a better idea.
Imagine someone trying to write code for a new miner or something else that needs to react to changes in the difficulty factor. A fixed difficulty network isn't that helpful in such case.

[gmaxwell]

Having different rules make it seem more like an entirely different network than a testnet, it also makes it less effective as a test platform.  Would it be possible to automatically reset it once a week?

I think this is a better point... make testnet self-reset once a month.  This will also reduce some of the incentives that people had for doing a lot of mining on it... e.g. amassing a lot of testcoin to sell to people.

Here is how I might do it... create a new genesis, include in it a bunch of testnet btc (10 million?) transferred to the fountain.

Have an extra testnet-only rule for the second block— which says that its timestamp can't be more than a month old. (obviously the normal 2 hour future check still applies).  Add a little code to punt the blocks and transactions when the expiration happens. (instead of trying to restore them to the chain)

(the fountain should mine a block every cycle and merge the input from that block before paying out any fountain coin in order to prevent people from replaying old fountain payouts on the reset— e.g. the fountain's initial bounty should only ever be spent combined with an input which will be lost on reset)

So then once a month, all on its own.. all of testnet would fork and begin again from block 1.

If this was done along with adjusting the target time from two weeks to two days (7x faster than bitcoin, who cares if there are lot of splits)... testnet would be a pretty good target for testing without being gratuitously different from bitcoin or attractive as an alt-chain for non-testing purposes.

This would also helpfully trigger deep deconfirmations of transactions— allowing people to test behavior which is otherwise tricky to test.

[btc_artist]

Having different rules make it seem more like an entirely different network than a testnet, it also makes it less effective as a test platform.  Would it be possible to automatically reset it once a week?

I am by no means an expert, but this sounds like a good idea to me.

[nibor]

I normally use testnet-in-a-box. It would be good as part of the reset to update this with a VERY low difficulty.

[Gavin Andresen]

Auto-reset once a month...  very interesting idea.

It'd make a mess of peoples' testnet wallets-- they'd be full of orphaned 0-confirmation transactions.  But deleting your testnet wallet once a month wouldn't be that big a burden, and that should prevent people from trading testnet coins as if they were worth something.

Somebody who wanted to be annoying could still drive up difficulty after every reset and make life miserable for anybody testing their new exchange or merchant software, though.

The problem with just doing more frequent difficulty adjustments is somebody with lots of hashing power can still over-write huge parts of the chain whenever they like. I suppose you could argue that bitcoin services should be written so that they can handle suddenly getting a 600-block-long chain-reorg... but that just does NOT happen on the real bitcoin network.

More hare-brained thoughts: could automatic block-chain lock-ins for the testnet be implemented somehow?  Fetch a block depth/hash pair from a website somebody volunteers to create (auto-updated once a day...) ?

[etotheipi]

I was just on the testnet the other day, trying to test my software.  I found it convenient to get coins from the faucet, except that no one was mining which meant that I couldn't get any confirmations.  A couple hours later I got a burst of a couple blocks, and then nothing.  I felt like the real issue is that retargets on testnet go by block-count -- the network may be on target to go back to difficulty <=1, but it still has to accumulate the 2016 blocks to get there, which could take months.

One solution is to force difficulty adjustments by time or block-count, and with short intervals such as (1 day || 256 blocks).   Of course, this could create problems for those who are testing retarget-code, since the testnet would be operating under different rules... but I think all other aspects would work out fine, and difficulty movements would be very fluid.

Personally, I don't really like resetting the blockchain every month... there's a lot of good stuff in the testnet blockchain I want to look at (like some crazy scripts), and I think people who are developing software over the course of a couple months, enjoy having a static base of data to be testing on.  If I'm trying to debug a problem with my code that happens on block 23,522, and the testnet resets, I'm going to be mildly annoyed.  It's not the end of the world, but I think 1 month is too short a cycle.

[Maged]

The core problem here might be that we only have one testnet. The current testnet is used to test miner code, test new Bitcoin code, test for exploits, and test Bitcoin integration with businesses' fulfillment system, among other reasons. Many of those things have completely different needs. What's really needed are separate testnets, of which each one would be specially designed based on the needs of its users.

For example, we could have a core development testnet. It would use standard Bitcoin rules, except with the IsStandard check disabled (as it is now on the testnet). It might also use time-based retargeting, instead of block-based. You might even reduce the block target time to 30 seconds instead of 10 minutes.

Next, a stress-test/exploit network. It would have standard Bitcoin rules. Resets would be done regularly. There might even be different flavors of this testnet. For example, one flavor might set the block maturation time to 1-3 blocks.

Finally, a merchant testnet. Standard Bitcoin rules, except that IsStandard is ENFORCED on the block-acceptance level (to discourage mixing core development with merchant development) and difficulty retargets happen every few hundred blocks. Block target time would remain the same, to be realistic. Initial merchant testing could still use the core dev testnet to speed things up, though. This would be the testnet that merchants would use to stage final testing.

[EhVedadoOAnonimato]

Finally, a merchant testnet. Standard Bitcoin rules, except that IsStandard is ENFORCED on the block-acceptance level (to discourage mixing core development with merchant development)

What about merchants who want to use custom scripts?
No need to enforce standard transactions...

[Maged]

Finally, a merchant testnet. Standard Bitcoin rules, except that IsStandard is ENFORCED on the block-acceptance level (to discourage mixing core development with merchant development)

What about merchants who want to use custom scripts?
No need to enforce standard transactions...

The idea is to mimic the real network as much as possible. I guess that on that point, it'd be best to just use regular IsStandard checks. My only fear is that by not block-enforcing it, merchants would get a false sense that non-standard transactions would actually get confirmed in a reasonable amount of time on Mainnet. That being said, I think that you are partly right, in that it should just use standard IsStandard checks, like mainnet.

[pc]

I'd like to confirm the idea that there are a lot of reasons one might want a testnet, and I think a lot of them make for mutually exculsive desires. Testing miners and pool software and the like wants difficulty to change in some testable manner, and testing clients and merchant integration wants ridiculously low difficulty pretty much all the time. I've found testnet-in-a-box to be very useful when I've tried to test changes to the bitcoin client, but one needs to be careful to not accidentally integrate it with the "real" testnet.

I'm not really sure on the solution, or what the rules should be specifically, but we definitely need multiple testnets in some fashion.

[piotr_n]

Why don't you just make the genesis block/tcp port/irc channe/whatever configurable?
I.e. via a config file.

Then with one binary there could be many testnets described by the config file and everybody would choose the one he needs.
Or start his own one, when he needs.

[SomeoneWeird]

Why don't you just make the genesis block/tcp port/irc channe/whatever configurable?
I.e. via a config file.

Then with one binary there could be many testnets described by the config file and everybody would choose the one he needs.
Or start his own one, when he needs.

http://sourceforge.net/projects/bitcoin/files/Bitcoin/testnet-in-a-box/

[midnightmagic]

Hey, don't destroy all my testnet coins a second time!

The only reason people don't consistently mine on testnet was because of the reset the first time. People were mining on testnet, legitimately, and blocks were semi-regular. But then the first reset came in, and all that effort was destroyed. In my case, it was about a month of multi-GH mining effort that just went up in a puff of smoke. The first time, the reason was apparently that the difficulty was too high for low-end mining testing, plus, and this is IMO, people who weren't in on the testnet OTC market didn't like to see testnet coins for sale (which was making testnet coins a competing currency.)

Complaining about people not consistently mining on testnet this time around is kind of irritating given the first testnet reset was reset even in the face of opposition from people who were actually using it (and consistently mining in it.)

ಠ_ಠ

[etotheipi]

Hey, don't destroy all my testnet coins a second time!

The only reason people don't consistently mine on testnet was because of the reset the first time. People were mining on testnet, legitimately, and blocks were semi-regular. But then the first reset came in, and all that effort was destroyed. In my case, it was about a month of multi-GH mining effort that just went up in a puff of smoke. The first time, the reason was apparently that the difficulty was too high for low-end mining testing, plus, and this is IMO, people who weren't in on the testnet OTC market didn't like to see testnet coins for sale (which was making testnet coins a competing currency.)

Complaining about people not consistently mining on testnet this time around is kind of irritating given the first testnet reset was reset even in the face of opposition from people who were actually using it (and consistently mining in it.)

I see a philosophical dilemma here.  The testnet was never intended to produce anything of value.  Strictly speaking, it's used for "offline" testing in a "realistic" environment.  From that perspective, the devs are only interested in what's best for the developer community in terms of what to do with the testnet.

On the other hand, if a clique of people latch onto the testnet and its coins, and start treating it as if it has value -- well then it's no different than BTC itself, and the devs should take into account the fact that they are about to destroy something "of value."  As such, a lot of people--those who perceive value in the testnet coins--would be peeved at a testnet reset...

So where does reality fit into all of this?  Well, the developers need a testing network that isn't the real network.  I think it had been made clear before that was the purpose of the testnet, so assigning value to it was a "mistake" of that clique, not the developers "destroying" it.  We shouldn't be forcing the devs to go out of their way to create a Testnet2 just to accommodate the clique of people who attached themselves to the first one.  

And for the record, I'm not so much complaining about people not mining consistently... it's that no one is mining right now, and I would do it myself if the difficulty wasn't so high.  But, I think it's silly for me spend 24 hours of CPU compute time just to mine one testnet block, which doesn't have any value to me...

[piotr_n]

the goal of the testnet is to be able to use an alternative blockchain with the same binary.
the current approach creates the issue when you want to switch your tests to a different block chain, yet not to the expensive one
first of all, you need to modify the client itself - second, you mess with other people's objectives by doing this.

so an external config file, defining the testchain, would be much less intrusive and future friendly.
just replace the "-testnet" switch with an optional config file that would define the genesis block.

[Gavin Andresen]

In my case, it was about a month of multi-GH mining effort that just went up in a puff of smoke.

What were you testing that you needed to dedicate about a month of multi-GH mining effort?

[BTCurious]

The widespread adoption or merged mining for bitcoin/testnet would solve the problem of volatile difficulty.

But not for people wanting to test mining. Or including weird scripts. Or excluding weird scripts. Or maybe the people sending weird scripts will hardly ever get them accepted.

[Disposition]

Three proposed changes to simulate testnet to what the Future for Bitcoin will look like.

• Fixed Difficulty/Fast Adjusted Difficulty as in the future I do not suspect difficulty to change too much. e.g. controlled environment.
  
• No BTC generated per block like in the future, block are mined purely for quickly confirming/verifying transactions.
• 21 Million testnet BTC generated in Genesis Block, emulating the future of circulation, transfer to faucet.

This will make testing much faster,
one can fire up miner, point to testnet,
request coins from faucet, proceed to test,
send coins back, shut off miner.

[Gavin Andresen]

In the spirit of "do the simplest possible thing that works..."  I think I see a very simple thing that will work.

The simple additional rule for the testnet:

If current_block_timestamp - previous_block_timestamp > 20 minutes:
  block difficulty = minimum difficulty

And that's it.

If mining is proceeding normally and most blocks are produced in less than 20 minutes, then the rules are exactly the same as the main network.

But if somebody has driven difficulty way up, then the new rule acts as a safety valve, ensuring that new blocks are created at least once every 20-something minutes or so.  After a month of "safety valve blocks" the difficulty would be calculated as normal, and would get cut in (approximately) half.

This does make the testnet block chain more susceptible to forks -- somebody with a bunch of hashing power can pretty easily invalidate a long chain of 20-minute, min-difficulty blocks if they want to. And there is likely to be a flurry min-difficulty blocks produced/announced every 20 minutes. But that could be considered a feature (test your block-chain-reorganization code!).

[SomeoneWeird]

In the spirit of "do the simplest possible thing that works..."  I think I see a very simple thing that will work.

The simple additional rule for the testnet:

If current_block_timestamp - previous_block_timestamp > 20 minutes:
  block difficulty = minimum difficulty

And that's it.

If mining is proceeding normally and most blocks are produced in less than 20 minutes, then the rules are exactly the same as the main network.

But if somebody has driven difficulty way up, then the new rule acts as a safety valve, ensuring that new blocks are created at least once every 20-something minutes or so.  After a month of "safety valve blocks" the difficulty would be calculated as normal, and would get cut in (approximately) half.

This does make the testnet block chain more susceptible to forks -- somebody with a bunch of hashing power can pretty easily invalidate a long chain of 20-minute, min-difficulty blocks if they want to. And there is likely to be a flurry min-difficulty blocks produced/announced every 20 minutes. But that could be considered a feature (test your block-chain-reorganization code!).

+1

[jojkaart]

In the spirit of "do the simplest possible thing that works..."  I think I see a very simple thing that will work.

The simple additional rule for the testnet:

If current_block_timestamp - previous_block_timestamp > 20 minutes:
  block difficulty = minimum difficulty

And that's it.

If mining is proceeding normally and most blocks are produced in less than 20 minutes, then the rules are exactly the same as the main network.

But if somebody has driven difficulty way up, then the new rule acts as a safety valve, ensuring that new blocks are created at least once every 20-something minutes or so.  After a month of "safety valve blocks" the difficulty would be calculated as normal, and would get cut in (approximately) half.

This does make the testnet block chain more susceptible to forks -- somebody with a bunch of hashing power can pretty easily invalidate a long chain of 20-minute, min-difficulty blocks if they want to. And there is likely to be a flurry min-difficulty blocks produced/announced every 20 minutes. But that could be considered a feature (test your block-chain-reorganization code!).

How about, instead of sharp min difficulty, start scaling the difficulty down smoothly from, maybe 15 minutes and have it reach min difficulty at 20 minutes from last block. This would also mean that miners should remember the best block they've made so far. It might end up being the one that gets accepted.

However, this sounds like it would be exploitable since the timestamp for a block can be quite a bit in the past.

[Gavin Andresen]

How about, instead of sharp min difficulty, start scaling the difficulty down smoothly from, maybe 15 minutes and have it reach min difficulty at 20 minutes from last block.

Why?  What problem would that solve?

[jojkaart]

How about, instead of sharp min difficulty, start scaling the difficulty down smoothly from, maybe 15 minutes and have it reach min difficulty at 20 minutes from last block.

Why?  What problem would that solve?

If you intend this feature to stay with testnet, what you suggested is probably fine. However, dropping difficulty sharply to the minimum would result in quite a flood of blocks competing with each other. Scaling the difficulty down slowly would reduce that a lot.

[pc]

And just to be clear, this rule will fork the test blockchain between those running on the current or prior versions of the client, and those running on whatever version this change comes out in, right?

[Gavin Andresen]

And just to be clear, this rule will fork the test blockchain between those running on the current or prior versions of the client, and those running on whatever version this change comes out in, right?

Yes.

Any opinions on whether to do a hard reset (new genesis block with the new rules) or a split ("new rules apply to blocks with timestamps after XYZ") ?


RE: scaling down difficulty:  good idea if we were thinking of implementing something like this for the main network.  But I'd want to give everybody lots of time to think long and hard about potential hacks/exploits/unintended consequences if this was being considered for the main network-- sudden drops in hashing power is NOT a problem for the main network, so I don't think it is worth considering now.

[dree12]

Is there a way to make sure people don't just modify timestamps to get extra coins? I'm sure I'm missing something, but I can't quite see what it is.

[BTCurious]

to get extra coins?

This is the testnet we're talking about… coins on the testnet have no value…

[Gavin Andresen]

Yes, who cares if you get extra testnet coins?

But... if somebody wanted to be annoying, they'd pre-generate as long a min-difficulty coinbase-only-transaction chain as the block timestamp rules allowed, and constantly broadcast those blocks.  Just to prevent transactions from getting confirmed.

To prevent that...

Testnet could prefer to build on blocks with more transactions from the memory pool over blocks with fewer transactions from the memory pool (that's not a bad rule for main net, either; might be worth considering if it works well for testnet). The rule now is "build on first valid block seen".

And "discourage" (refuse to directly build on or relay) blocks with timestamps in the future.

[BTCurious]

Is testnet trolling really an issue at the moment? I could imagine no one will really bother, since the worst thing you'll cause us mild  annoyance. Testing a feature for mainnet may be a worthy goal though.

About the suggestion itself: I believe it would open the doors for a new kind of attack. If you have ten prevent of the hashing power, but include loads of phoney transactions in your generated blocks, you might still take over the blockchain, depending on the exact implementation.

[Gavin Andresen]

https://github.com/bitcoin/bitcoin/pull/686

Testnet difficulty calculation changes, to take effect Jan 1 2012

Allow mining of min-difficulty blocks if 20 minutes have gone by without mining a regular-difficulty block.
Normal rules apply every 2016 blocks, though, so there may be a very-slow-to-confirm block at the difficulty-adjustment blocks (once per month, assuming testnet is in it's normal "difficulty too high for number of people mining it" state).

This will almost certainly cause a testnet blockchain split after Jan 1. After pulling I'll update the Testnet Faucet, I'll ask theymos if he can update the testnet block explorer bitcoind.

I didn't implement any "shun blocks from the future" or "prefer blocks with more memory-pool transactions", I want to see how well the simplest-thing-that-might-possibly-work solution works first.