Time for another testnet reset?

[Gavin Andresen]

I'm starting this thread to brainstorm rule changes for -testnet and talk about a possible testnet reset for the next release.

The testnet isn't currently usable, because hashing power on it is unpredictable.  Difficulty shoots up because somebody decides to throw lots of machines at it, then they leave and it takes MONTHS for difficulty to drift back down.

Here's a shot at hair-brained rules for the testnet:

+ Fix the difficulty at 1.0, no matter how many people are mining.

+ Clients reject new blocks with timestamps more than 1 minute different from their time (implies:  you have to have an accurate system clock to play with the other nodes on the testnet, otherwise you'll be on your own fork).

+ Clients reject new blocks if their timestamp is less than 2 minutes after the previous best-block's timestamp.

+ Clients prefer to build on blocks that include more memory-pool transactions, instead of taking first-block-they-receive.

Goals behind those rules:
  Always easy to mine
  Limit block-chain growth (max 1 new block every 2 minutes)

So: could a griefer make life miserable for the rest of us under those rules?  What would happen if there were five or six people with a fair bit of hashing power all trying to mine as fast as possible on testnet?  (lots of block chain splits, I think, but that's probably just fine on testnet)

[BTCurious]

Ignore me if this is a stupid idea, but:

If the difficulty is the problem, can't you just increase the adjustment speed of the difficulty, and leave the rest the same? This would keep the testnet concept nearly entirely the same as the prod net, which makes it as useful as possible.

[dree12]

What about adjusting the difficulty without cap every block based on the speed of the last 6 or so blocks? Then, even if a griefer shot the difficulty up, it wouldn't take much time at all for it to come down.

[BeeCee1]

Having different rules make it seem more like an entirely different network than a testnet, it also makes it less effective as a test platform.  Would it be possible to automatically reset it once a week?

[EhVedadoOAnonimato]

Adjusting difficulty more frequently is probably a better idea.
Imagine someone trying to write code for a new miner or something else that needs to react to changes in the difficulty factor. A fixed difficulty network isn't that helpful in such case.

[gmaxwell]

Having different rules make it seem more like an entirely different network than a testnet, it also makes it less effective as a test platform.  Would it be possible to automatically reset it once a week?

I think this is a better point... make testnet self-reset once a month.  This will also reduce some of the incentives that people had for doing a lot of mining on it... e.g. amassing a lot of testcoin to sell to people.

Here is how I might do it... create a new genesis, include in it a bunch of testnet btc (10 million?) transferred to the fountain.

Have an extra testnet-only rule for the second block— which says that its timestamp can't be more than a month old. (obviously the normal 2 hour future check still applies).  Add a little code to punt the blocks and transactions when the expiration happens. (instead of trying to restore them to the chain)

(the fountain should mine a block every cycle and merge the input from that block before paying out any fountain coin in order to prevent people from replaying old fountain payouts on the reset— e.g. the fountain's initial bounty should only ever be spent combined with an input which will be lost on reset)

So then once a month, all on its own.. all of testnet would fork and begin again from block 1.

If this was done along with adjusting the target time from two weeks to two days (7x faster than bitcoin, who cares if there are lot of splits)... testnet would be a pretty good target for testing without being gratuitously different from bitcoin or attractive as an alt-chain for non-testing purposes.

This would also helpfully trigger deep deconfirmations of transactions— allowing people to test behavior which is otherwise tricky to test.

[btc_artist]

Having different rules make it seem more like an entirely different network than a testnet, it also makes it less effective as a test platform.  Would it be possible to automatically reset it once a week?

I am by no means an expert, but this sounds like a good idea to me.

[nibor]

I normally use testnet-in-a-box. It would be good as part of the reset to update this with a VERY low difficulty.

[Gavin Andresen]

Auto-reset once a month...  very interesting idea.

It'd make a mess of peoples' testnet wallets-- they'd be full of orphaned 0-confirmation transactions.  But deleting your testnet wallet once a month wouldn't be that big a burden, and that should prevent people from trading testnet coins as if they were worth something.

Somebody who wanted to be annoying could still drive up difficulty after every reset and make life miserable for anybody testing their new exchange or merchant software, though.

The problem with just doing more frequent difficulty adjustments is somebody with lots of hashing power can still over-write huge parts of the chain whenever they like. I suppose you could argue that bitcoin services should be written so that they can handle suddenly getting a 600-block-long chain-reorg... but that just does NOT happen on the real bitcoin network.

More hare-brained thoughts: could automatic block-chain lock-ins for the testnet be implemented somehow?  Fetch a block depth/hash pair from a website somebody volunteers to create (auto-updated once a day...) ?

[etotheipi]

I was just on the testnet the other day, trying to test my software.  I found it convenient to get coins from the faucet, except that no one was mining which meant that I couldn't get any confirmations.  A couple hours later I got a burst of a couple blocks, and then nothing.  I felt like the real issue is that retargets on testnet go by block-count -- the network may be on target to go back to difficulty <=1, but it still has to accumulate the 2016 blocks to get there, which could take months.

One solution is to force difficulty adjustments by time or block-count, and with short intervals such as (1 day || 256 blocks).   Of course, this could create problems for those who are testing retarget-code, since the testnet would be operating under different rules... but I think all other aspects would work out fine, and difficulty movements would be very fluid.

Personally, I don't really like resetting the blockchain every month... there's a lot of good stuff in the testnet blockchain I want to look at (like some crazy scripts), and I think people who are developing software over the course of a couple months, enjoy having a static base of data to be testing on.  If I'm trying to debug a problem with my code that happens on block 23,522, and the testnet resets, I'm going to be mildly annoyed.  It's not the end of the world, but I think 1 month is too short a cycle.

[Maged]

The core problem here might be that we only have one testnet. The current testnet is used to test miner code, test new Bitcoin code, test for exploits, and test Bitcoin integration with businesses' fulfillment system, among other reasons. Many of those things have completely different needs. What's really needed are separate testnets, of which each one would be specially designed based on the needs of its users.

For example, we could have a core development testnet. It would use standard Bitcoin rules, except with the IsStandard check disabled (as it is now on the testnet). It might also use time-based retargeting, instead of block-based. You might even reduce the block target time to 30 seconds instead of 10 minutes.

Next, a stress-test/exploit network. It would have standard Bitcoin rules. Resets would be done regularly. There might even be different flavors of this testnet. For example, one flavor might set the block maturation time to 1-3 blocks.

Finally, a merchant testnet. Standard Bitcoin rules, except that IsStandard is ENFORCED on the block-acceptance level (to discourage mixing core development with merchant development) and difficulty retargets happen every few hundred blocks. Block target time would remain the same, to be realistic. Initial merchant testing could still use the core dev testnet to speed things up, though. This would be the testnet that merchants would use to stage final testing.

[EhVedadoOAnonimato]

Finally, a merchant testnet. Standard Bitcoin rules, except that IsStandard is ENFORCED on the block-acceptance level (to discourage mixing core development with merchant development)

What about merchants who want to use custom scripts?
No need to enforce standard transactions...

[Maged]

Finally, a merchant testnet. Standard Bitcoin rules, except that IsStandard is ENFORCED on the block-acceptance level (to discourage mixing core development with merchant development)

What about merchants who want to use custom scripts?
No need to enforce standard transactions...

The idea is to mimic the real network as much as possible. I guess that on that point, it'd be best to just use regular IsStandard checks. My only fear is that by not block-enforcing it, merchants would get a false sense that non-standard transactions would actually get confirmed in a reasonable amount of time on Mainnet. That being said, I think that you are partly right, in that it should just use standard IsStandard checks, like mainnet.

[pc]

I'd like to confirm the idea that there are a lot of reasons one might want a testnet, and I think a lot of them make for mutually exculsive desires. Testing miners and pool software and the like wants difficulty to change in some testable manner, and testing clients and merchant integration wants ridiculously low difficulty pretty much all the time. I've found testnet-in-a-box to be very useful when I've tried to test changes to the bitcoin client, but one needs to be careful to not accidentally integrate it with the "real" testnet.

I'm not really sure on the solution, or what the rules should be specifically, but we definitely need multiple testnets in some fashion.

[piotr_n]

Why don't you just make the genesis block/tcp port/irc channe/whatever configurable?
I.e. via a config file.

Then with one binary there could be many testnets described by the config file and everybody would choose the one he needs.
Or start his own one, when he needs.

[SomeoneWeird]

Why don't you just make the genesis block/tcp port/irc channe/whatever configurable?
I.e. via a config file.

Then with one binary there could be many testnets described by the config file and everybody would choose the one he needs.
Or start his own one, when he needs.

http://sourceforge.net/projects/bitcoin/files/Bitcoin/testnet-in-a-box/

[midnightmagic]

Hey, don't destroy all my testnet coins a second time!

The only reason people don't consistently mine on testnet was because of the reset the first time. People were mining on testnet, legitimately, and blocks were semi-regular. But then the first reset came in, and all that effort was destroyed. In my case, it was about a month of multi-GH mining effort that just went up in a puff of smoke. The first time, the reason was apparently that the difficulty was too high for low-end mining testing, plus, and this is IMO, people who weren't in on the testnet OTC market didn't like to see testnet coins for sale (which was making testnet coins a competing currency.)

Complaining about people not consistently mining on testnet this time around is kind of irritating given the first testnet reset was reset even in the face of opposition from people who were actually using it (and consistently mining in it.)

ಠ_ಠ

[etotheipi]

Hey, don't destroy all my testnet coins a second time!

The only reason people don't consistently mine on testnet was because of the reset the first time. People were mining on testnet, legitimately, and blocks were semi-regular. But then the first reset came in, and all that effort was destroyed. In my case, it was about a month of multi-GH mining effort that just went up in a puff of smoke. The first time, the reason was apparently that the difficulty was too high for low-end mining testing, plus, and this is IMO, people who weren't in on the testnet OTC market didn't like to see testnet coins for sale (which was making testnet coins a competing currency.)

Complaining about people not consistently mining on testnet this time around is kind of irritating given the first testnet reset was reset even in the face of opposition from people who were actually using it (and consistently mining in it.)

I see a philosophical dilemma here.  The testnet was never intended to produce anything of value.  Strictly speaking, it's used for "offline" testing in a "realistic" environment.  From that perspective, the devs are only interested in what's best for the developer community in terms of what to do with the testnet.

On the other hand, if a clique of people latch onto the testnet and its coins, and start treating it as if it has value -- well then it's no different than BTC itself, and the devs should take into account the fact that they are about to destroy something "of value."  As such, a lot of people--those who perceive value in the testnet coins--would be peeved at a testnet reset...

So where does reality fit into all of this?  Well, the developers need a testing network that isn't the real network.  I think it had been made clear before that was the purpose of the testnet, so assigning value to it was a "mistake" of that clique, not the developers "destroying" it.  We shouldn't be forcing the devs to go out of their way to create a Testnet2 just to accommodate the clique of people who attached themselves to the first one.  

And for the record, I'm not so much complaining about people not mining consistently... it's that no one is mining right now, and I would do it myself if the difficulty wasn't so high.  But, I think it's silly for me spend 24 hours of CPU compute time just to mine one testnet block, which doesn't have any value to me...

[piotr_n]

the goal of the testnet is to be able to use an alternative blockchain with the same binary.
the current approach creates the issue when you want to switch your tests to a different block chain, yet not to the expensive one
first of all, you need to modify the client itself - second, you mess with other people's objectives by doing this.

so an external config file, defining the testchain, would be much less intrusive and future friendly.
just replace the "-testnet" switch with an optional config file that would define the genesis block.

[Gavin Andresen]

In my case, it was about a month of multi-GH mining effort that just went up in a puff of smoke.

What were you testing that you needed to dedicate about a month of multi-GH mining effort?