Proof-of-work difficulty increasing

[satoshi]

We had our first automatic adjustment of the proof-of-work difficulty on 30 Dec 2009.  

The minimum difficulty is 32 zero bits, so even if only one person was running a node, the difficulty doesn't get any easier than that.  For most of last year, we were hovering below the minimum.  On 30 Dec we broke above it and the algorithm adjusted to more difficulty.  It's been getting more difficult at each adjustment since then.

The adjustment on 04 Feb took it up from 1.34 times last year's difficulty to 1.82 times more difficult than last year.  That means you generate only 55% as many coins for the same amount of work.

The difficulty adjusts proportionally to the total effort across the network.  If the number of nodes doubles, the difficulty will also double, returning the total generated to the target rate.

For those technically inclined, the proof-of-work difficulty can be seen by searching on "target:" in debug.log.  It's a 256-bit unsigned hex number, which the SHA-256 value has to be less than to successfully generate a block.  It gets adjusted every 2016 blocks, typically two weeks.  That's when it prints "GetNextWorkRequired RETARGET" in debug.log.

minimum    00000000ffff0000000000000000000000000000000000000000000000000000
30/12/2009 00000000d86a0000000000000000000000000000000000000000000000000000
11/01/2010 00000000c4280000000000000000000000000000000000000000000000000000
25/01/2010 00000000be710000000000000000000000000000000000000000000000000000
04/02/2010 000000008cc30000000000000000000000000000000000000000000000000000
14/02/2010 0000000065465700000000000000000000000000000000000000000000000000
24/02/2010 0000000043b3e500000000000000000000000000000000000000000000000000
08/03/2010 00000000387f6f00000000000000000000000000000000000000000000000000
21/03/2010 0000000038137500000000000000000000000000000000000000000000000000
01/04/2010 000000002a111500000000000000000000000000000000000000000000000000
12/04/2010 0000000020bca700000000000000000000000000000000000000000000000000
21/04/2010 0000000016546f00000000000000000000000000000000000000000000000000
04/05/2010 0000000013ec5300000000000000000000000000000000000000000000000000
19/05/2010 00000000159c2400000000000000000000000000000000000000000000000000
29/05/2010 000000000f675c00000000000000000000000000000000000000000000000000
11/06/2010 000000000eba6400000000000000000000000000000000000000000000000000
24/06/2010 000000000d314200000000000000000000000000000000000000000000000000
06/07/2010 000000000ae49300000000000000000000000000000000000000000000000000
13/07/2010 0000000005a3f400000000000000000000000000000000000000000000000000
16/07/2010 000000000168fd00000000000000000000000000000000000000000000000000
27/07/2010 00000000010c5a00000000000000000000000000000000000000000000000000
05/08/2010 0000000000ba1800000000000000000000000000000000000000000000000000
15/08/2010 0000000000800e00000000000000000000000000000000000000000000000000
26/08/2010 0000000000692000000000000000000000000000000000000000000000000000

date, difficulty factor, % change
2009           1.00
30/12/2009     1.18   +18%
11/01/2010     1.31   +11%
25/01/2010     1.34    +2%
04/02/2010     1.82   +36%
14/02/2010     2.53   +39%
24/02/2010     3.78   +49%
08/03/2010     4.53   +20%
21/03/2010     4.57    +9%
01/04/2010     6.09   +33%
12/04/2010     7.82   +28%
21/04/2010    11.46   +47%
04/05/2010    12.85   +12%
19/05/2010    11.85    -8%
29/05/2010    16.62   +40%
11/06/2010    17.38    +5%
24/06/2010    19.41   +12%
06/07/2010    23.50   +21%
13/07/2010    45.38   +93%
16/07/2010   181.54  +300%
27/07/2010   244.21   +35%
05/08/2010   352.17   +44%
15/08/2010   511.77   +45%
26/08/2010   623.39   +22%

[satoshi]

14/02/2010 0000000065465700000000000000000000000000000000000000000000000000

2009        1.00
30/12/2009  1.18   +18%
11/01/2010  1.31   +11%
25/01/2010  1.34    +2%
04/02/2010  1.82   +36%
14/02/2010  2.53   +39%

Another big jump in difficulty yesterday from 1.82 times to 2.53 times, a 39% increase since 10 days ago.  It was 10 days apart not 14 because more nodes joined and generated the 2016 blocks in less time.

[Suggester]

[Edit: I later found that I was generating quite a bit more than that, just didn't realize it because of the "matures in xx more blocks" concept. I still think it will be a major headache when the difficulty significantly increases though. I apologize for my silliness ]

Satoshi, I figured it will take my modern core 2 duo about 20 hours of nonstop work to create ฿50.00! With older PCs it will take forever. People like to feel that they "own" something as soon as possible, is there a way to make the generation more divisible? So say, instead of making ฿50 every 20 hours, make ฿5 every 2 hours?

I don't know if that means reducing the block size or reducing the 120-block threshold to say 12-block only or what, but because the difficulty is increasing I can imagine that a year from now the situation will be even worse (3+ weeks until you see the first spendable coins!) and we better find a solution for this ASAP.

[Sabunir]

I would like to comment that as of late, it seems almost as if I am generating nearly no Bitcoins. Indeed, my rate of acquisition seems to be greater than ten times slower. If I cannot stay online for about fourteen consecutive hours (very hard to do on a satellite connection!), I actually get nothing at all.

How this exactly relates to the difficulty adjustments is beyond my knowledge; I offer this feedback as a kind of "field report".

[theymos]

I generated 5 blocks today on my Pentium processor. Two of them were within 3 minutes of each other.

I have noticed some slowdown since the adjustment, but I still generate a lot of coins. My computer is off while I'm sleeping, and BitCoin bootstraps quickly when I turn it back on. Do you guys-who-are-having-trouble have the BitCoin port open?

[Sabunir]

My port is open, both in my software and hardware firewall. My router is handling it appropriately. Perhaps it has to do with my connection's very high latency (2000ms or more on average) and/or my high packet loss (sometimes up to 10% loss)?

[satoshi]

Satoshi, I figured it will take my modern core 2 duo about 20 hours of nonstop work to create ฿50.00! With older PCs it will take forever. People like to feel that they "own" something as soon as possible, is there a way to make the generation more divisible? So say, instead of making ฿50 every 20 hours, make ฿5 every 2 hours?

I thought about that but there wasn't a practical way to do smaller increments.  The frequency of block generation is balanced between confirming transactions as fast as possible and the latency of the network.

The algorithm aims for an average of 6 blocks per hour.  If it was 5 bc and 60 per hour, there would be 10 times as many blocks and the initial block download would take 10 times as long.  It wouldn't work anyway because that would be only 1 minute average between blocks, too close to the broadcast latency when the network gets larger.

[Suggester]

If I cannot stay online for about fourteen consecutive hours (very hard to do on a satellite connection!), I actually get nothing at all.

Can Satoshi confirm whether the computations your machine had made carries on if the session was interrupted, or do you need to start all over if you disconnected before generating at least one block? If it carries on, maybe a little meter indicating the % left until your block completes can be a nice addition so people would have some hope (actually, it will be a nice addition anyway whether the computations get carried on after disconnection or not!)

I generated 5 blocks today on my Pentium processor. Two of them were within 3 minutes of each other.

Ok, I just realized that I didn't understand how Bitcoin worked to begin with. The blocks get generated anyway whether you're generating coins or not. The average amount of creation conformed what I observed before (120/20 hrs, or 6/hr). This has got absolutely nothing to do with your CPU power, it's  constant for all practical purposes. The CPU power determines the "transactions" that get created and "matures in xx blocks". My head just got a bit bigger now

This also means theymos that there was probably a coincidence or error for your 3-minute interval observation!

[satoshi]

. Perhaps it has to do with my connection's very high latency (2000ms or more on average)

2 seconds of latency in both directions should reduce your generation success by less than 1%.

and/or my high packet loss (sometimes up to 10% loss)?

Probably OK, but I'm not sure.  The protocol is designed to resync to the next message, and messages get re-requested from all the other nodes you're connected to until received.  If you miss a block, it'll also keep requesting it every time another blocks comes in and it sees there's a gap.  Before the original release I did a test dropping 1 out of 4 random messages under heavy load until I could run it overnight without any nodes getting stuck.

[Sabunir]

How do you adjust this difficulty, anyway? (Administrating a decentralized system?) And what would prevent an attacker from setting the difficulty very low or very high to interfere with the system?

[NewLibertyStandard]

How do you adjust this difficulty, anyway? (Administrating a decentralized system?) And what would prevent an attacker from setting the difficulty very low or very high to interfere with the system?

My understanding is that every Bitcoin client has the same algorithm (formula) built into it to automatically adjust the difficulty every so many blocks. Not only that, but I think that Bitcoin will not accept blocks generated at a different difficulty, so if a modified Bitcoin client tried to send out more easily generated blocks, all the authentic clients would reject the fake blocks.

[satoshi]

The automatic adjustment happened earlier today.

24/02/2010 0000000043b3e500000000000000000000000000000000000000000000000000

24/02/2010  3.78  +49%

I updated the first post.

[Suggester]

How do you adjust this difficulty, anyway? (Administrating a decentralized system?) And what would prevent an attacker from setting the difficulty very low or very high to interfere with the system?

My understanding is that every Bitcoin client has the same algorithm (formula) built into it to automatically adjust the difficulty every so many blocks.

Then how is it dependent on how many CPU's are connected to the whole network?

Not only that, but I think that Bitcoin will not accept blocks generated at a different difficulty, so if a modified Bitcoin client tried to send out more easily generated blocks, all the authentic clients would reject the fake blocks.

We need Satoshi to confirm that because clients accept blocks generated at easier difficulties all the time whenever the PoW's difficulty increases.

[satoshi]

The formula is based on the time it takes to generate 2016 blocks.  The difficulty is multiplied by 14/(actual days taken).  For instance, this time it took 9.4 days, so the calculation was 14/9.4 = 1.49.  Previous difficulty 2.53 * 1.49 = 3.78, a 49% increase.

I don't know what you're talking about accepting easier difficulties.

[Suggester]

I don't know what you're talking about accepting easier difficulties.

We were essentially discussing Sabunir's question about what prevents someone from messing with the program's source code to adjust block-generating difficulty to be very easy, then make a network on his own and create a, say, 50,000-block proof-of-work within seconds then finally propagate it across the real network to steal "votes" towards his new fake blocks as technically, his proof would be "the longest". So is there a way to verify how much work was actually put into a given PoW (for eg. how many zero's are at the beginning of each hash or something)?

It wouldn't work anyway because that would be only 1 minute average between blocks, too close to the broadcast latency when the network gets larger.

Since we're at it, what's the approximate time for proof-of-work propagation across a network of about 100,000 machines? Is there a way to optimize connections so that broadcasting is done via a pyramid-form to minimize the needed time? For example, the block creator sends it to 10 nodes, then those 10 send it to a 100 provided that none of those 100 were among the original 11, then those 100 tell a 1000 provided that none of those 1000 were among the original 111, etc to save time.

[Legion]

This overclocked i7 still hasn't generated any keys after 8 hours...

[NewLibertyStandard]

This overclocked i7 still hasn't generated any keys after 8 hours...

It may take longer than 8 hours to generate a block.

Have you previously generated bitcoins? Are the number of blocks listed at the bottom of Bitcoin greater than 42650? Those need to download before it can start generating coins. How many connections are listed at the bottom of Bitcoin? Did you click Options > Generate Coins? How much CPU does your process viewer show that Bitcoin is using? Is your Internet connection steady? I had problems when I tried sharing Internet from my smartphone to my computer.

[Legion]

This overclocked i7 still hasn't generated any keys after 8 hours...

It may take longer than 8 hours to generate a block.

Have you previously generated bitcoins? Are the number of blocks listed at the bottom of Bitcoin greater than 42650? Those need to download before it can start generating coins. How many connections are listed at the bottom of Bitcoin? Did you click Options > Generate Coins? How much CPU does your process viewer show that Bitcoin is using? Is your Internet connection steady? I had problems when I tried sharing Internet from my smartphone to my computer.

No, but..42663 blocks..8 connections..and yes, generating.  Bitcoin uses 50-80 CPU..but it only has access to two cores until I bump the VM it is in to 4 cores..Operating over tor by the way.

[NewLibertyStandard]

I think that no bitcoins generated in 8 hours from within a VM utilizing two modern cores is probably not unusual. Keep it running for a few days and I expect that you'll generate more than a few packs of bitcoins.

[Legion]

I wonder what I could generate with all eight threads...