Bitcoin Forum
December 04, 2016, 10:26:35 PM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 [5] 6 »  All
  Print  
Author Topic: Difficulty adjustment needs modifying  (Read 9308 times)
kano
Legendary
*
Offline Offline

Activity: 1918


Linux since 1997 RedHat 4


View Profile
October 06, 2011, 12:31:58 PM
 #81

OK luke-jr I shouldn't have agreed with you about the time in a block.

I just ran another block table generation (not at the link but these are in the old link anyway) and it shows this:

148111   04:04:32 5-Oct-2011 UTC   0x1a09ee5d (1689334.4045971)   -73m 00s   9m 38.15s   +3.64%
148110   05:17:32 5-Oct-2011 UTC   0x1a09ee5d (1689334.4045971)   84m 01s   10m 01.40s   -0.23%
148109   03:53:31 5-Oct-2011 UTC   0x1a09ee5d (1689334.4045971)   13m 25s   10m 02.45s   -0.41%

Yes of course 148110 is Eligius - more than a whole hour?!?

If you look through my table probably every time you find a block with a negative time it is after an Eligius block
True for the first 3 and easy to check the block look at the coinbase for the word Eligius, or a whole bunch of outs in the generation transaction.

If the diff change occurred on block 148110, that one block would affect the calculation by extending the average by more than 2 seconds thus making the difficulty lower (yeah I said higher - fixed Smiley ...

I guess this is what was being referred to before?

Pool: https://kano.is BTC: 1KanoiBupPiZfkwqB7rfLXAzPnoTshAVmb
CKPool and CGMiner developer, IRC FreeNode #ckpool and #cgminer kanoi
Help keep Bitcoin secure by mining on pools with Stratum, the best protocol to mine Bitcoins with ASIC hardware
1480890395
Hero Member
*
Offline Offline

Posts: 1480890395

View Profile Personal Message (Offline)

Ignore
1480890395
Reply with quote  #2

1480890395
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480890395
Hero Member
*
Offline Offline

Posts: 1480890395

View Profile Personal Message (Offline)

Ignore
1480890395
Reply with quote  #2

1480890395
Report to moderator
1480890395
Hero Member
*
Offline Offline

Posts: 1480890395

View Profile Personal Message (Offline)

Ignore
1480890395
Reply with quote  #2

1480890395
Report to moderator
1480890395
Hero Member
*
Offline Offline

Posts: 1480890395

View Profile Personal Message (Offline)

Ignore
1480890395
Reply with quote  #2

1480890395
Report to moderator
kano
Legendary
*
Offline Offline

Activity: 1918


Linux since 1997 RedHat 4


View Profile
October 11, 2011, 06:09:54 AM
 #82

Updated diffcalc.php -> http://tradebtc.net/diffcalc.html (N.B. that's a copy of when I just ran it - it takes 3min to run)
Hmm firstly, 2weeks is ~17 hours from now but still 300 blocks to go ...

Counting back to last change the new diff would be -11.48%

However:
 432 blocks (~3 days) is -16.78%
 144 blocks (~1 day) is -25.48%

(and -84 blocks is -30.37% <- yeah I chose that one on purpose - the highest one close to 100)

Again, I really think this needs to be addressed BEFORE it happens ... and if this past week is any indication of the near future ...

Pool: https://kano.is BTC: 1KanoiBupPiZfkwqB7rfLXAzPnoTshAVmb
CKPool and CGMiner developer, IRC FreeNode #ckpool and #cgminer kanoi
Help keep Bitcoin secure by mining on pools with Stratum, the best protocol to mine Bitcoins with ASIC hardware
sadpandatech
Hero Member
*****
Offline Offline

Activity: 504



View Profile
October 11, 2011, 01:25:24 PM
 #83

Updated diffcalc.php -> http://tradebtc.net/diffcalc.html (N.B. that's a copy of when I just ran it - it takes 3min to run)
Hmm firstly, 2weeks is ~17 hours from now but still 300 blocks to go ...

Counting back to last change the new diff would be -11.48%

However:
 432 blocks (~3 days) is -16.78%
 144 blocks (~1 day) is -25.48%

(and -84 blocks is -30.37% <- yeah I chose that one on purpose - the highest one close to 100)

Again, I really think this needs to be addressed BEFORE it happens ... and if this past week is any indication of the near future ...

 After looking at the top 5 pools(not counting 'other') today to get average user counts it was pretty obvious that a HUGE chunk of the hash power is in a very small percentage of hands. That said, I am in very strong aggreeance with you on this!

  Hopefully we will get lucky and those that may decide to switch off will do so during the last 1/4 of a 2016 chunk before a diff change. That would have the least amount of negative impact versus the more likely scenario that they would switch off right after dif change.  Undecided

If you're not excited by the idea of being an early adopter 'now', then you should come back in three or four years and either tell us "Told you it'd never work!" or join what should, by then, be a much more stable and easier-to-use system. - GA
It is being worked on by smart people. -DamienBlack
Transisto
Donator
Legendary
*
Offline Offline

Activity: 1624



View Profile WWW
October 12, 2011, 05:49:08 AM
 #84

Updated diffcalc.php -> http://tradebtc.net/diffcalc.html (N.B. that's a copy of when I just ran it - it takes 3min to run)
Hmm firstly, 2weeks is ~17 hours from now but still 300 blocks to go ...

Counting back to last change the new diff would be -11.48%

However:
 432 blocks (~3 days) is -16.78%
 144 blocks (~1 day) is -25.48%

(and -84 blocks is -30.37% <- yeah I chose that one on purpose - the highest one close to 100)

Again, I really think this needs to be addressed BEFORE it happens ... and if this past week is any indication of the near future ...
At what point do we start to care ? If 85% of hashing power was to disappear overnight it would take an hours on average between blocks and we would have 3 month to explain to the user-base that it is not the end of the world yet.

If BTCs were to drop at 10c usd we would still have a fairly large % of population with free electricity that would not stop mining.


After looking at the top 5 pools(not counting 'other') today to get average user counts it was pretty obvious that a HUGE chunk of the hash power is in a very small percentage of hands. That said, I am in very strong aggreeance with you on this...
Pool centralization does not affect transaction speed nor difficulty change. If a big pool was to disappear it wouldn't take half a day for 2/3 of it's miners to move elsewhere.
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2086



View Profile
October 12, 2011, 05:54:19 AM
 #85

If BTCs were to drop at 10c usd we would still have a fairly large % of population with free electricity that would not stop mining.
Very few people have free electricity.

Transisto
Donator
Legendary
*
Offline Offline

Activity: 1624



View Profile WWW
October 12, 2011, 05:58:16 AM
 #86

If BTCs were to drop at 10c usd we would still have a fairly large % of population with free electricity that would not stop mining.
Very few people have free electricity.
Free electricity for mining is about using generated heat for other purposes.  

AKA. Winter

Quote
Updated diffcalc.php -> http://tradebtc.net/diffcalc.html (N.B. that's a copy of when I just ran it - it takes 3min to run)
@Kano , Very nice stats, Shouldn't your average restart at zero after adjustments ? , block 1802   147167
kano
Legendary
*
Offline Offline

Activity: 1918


Linux since 1997 RedHat 4


View Profile
October 12, 2011, 11:19:26 AM
 #87

If BTCs were to drop at 10c usd we would still have a fairly large % of population with free electricity that would not stop mining.
Very few people have free electricity.
Free electricity for mining is about using generated heat for other purposes.  

AKA. Winter

Quote
Updated diffcalc.php -> http://tradebtc.net/diffcalc.html (N.B. that's a copy of when I just ran it - it takes 3min to run)
@Kano , Very nice stats, Shouldn't your average restart at zero after adjustments ? , block 1802   147167
Well the extra data after the adjustment is somewhat meaningless since it is a different difficulty,
but I decided to go with the simple calculation of just do 2017 blocks and mark 3 of interest.
In this case when we are close to the end, the data before the diff adjustment is pretty useless, but soon after a diff adjustment, it is worth looking at.
Anyway, only really those three % values are of real interest, the rest is just here to work out the numbers or run on a bit after the end.

I've just run it again (and updated it at the link)

Back to last difficulty: -12.67%
-432 blocks (~3 days): -23.06%
-144 blocks (~1 day): -37.09%

Interesting that -144 seems to be the lower limit and it's rising a bit since then
(though it's hard to tell for sure since the figures get very unreliable as you get closer to 1)

But earlier today I had some scary numbers:
back to last diff=-12.77% ~3days=-24.61% ~1day=-34.91% worst around 100 is -99=-51.84%

Pool: https://kano.is BTC: 1KanoiBupPiZfkwqB7rfLXAzPnoTshAVmb
CKPool and CGMiner developer, IRC FreeNode #ckpool and #cgminer kanoi
Help keep Bitcoin secure by mining on pools with Stratum, the best protocol to mine Bitcoins with ASIC hardware
sadpandatech
Hero Member
*****
Offline Offline

Activity: 504



View Profile
October 12, 2011, 11:56:45 AM
 #88

After looking at the top 5 pools(not counting 'other') today to get average user counts it was pretty obvious that a HUGE chunk of the hash power is in a very small percentage of hands. That said, I am in very strong aggreeance with you on this...
Pool centralization does not affect transaction speed nor difficulty change. If a big pool was to disappear it wouldn't take half a day for 2/3 of it's miners to move elsewhere.



My point had nothing to do with the pools themselves. I was refering to the individul users who hold the majority of the hash power. If even a small number of them decide to flip the switch, it will hurt.....

And yes, many will keep going no matter what. I am one of those. But how much hash power do 'we' hold?

If you're not excited by the idea of being an early adopter 'now', then you should come back in three or four years and either tell us "Told you it'd never work!" or join what should, by then, be a much more stable and easier-to-use system. - GA
It is being worked on by smart people. -DamienBlack
kano
Legendary
*
Offline Offline

Activity: 1918


Linux since 1997 RedHat 4


View Profile
October 12, 2011, 11:19:40 PM
 #89

Hmm - more than 24 hours late on the diff change and still more than 100 blocks to go Sad
(so that means a guaranteed drop of at least 11%)

... and DeepBit died about an hour ago ...
(Edit: and Slush and some of BTCGuild)

I wonder what will happen to the trailing end of this difficulty ...
(start time was "22:40:40 27-Sep-2011 UTC")

Pool: https://kano.is BTC: 1KanoiBupPiZfkwqB7rfLXAzPnoTshAVmb
CKPool and CGMiner developer, IRC FreeNode #ckpool and #cgminer kanoi
Help keep Bitcoin secure by mining on pools with Stratum, the best protocol to mine Bitcoins with ASIC hardware
sadpandatech
Hero Member
*****
Offline Offline

Activity: 504



View Profile
October 13, 2011, 01:35:25 AM
 #90

Hmm - more than 24 hours late on the diff change and still more than 100 blocks to go Sad
(so that means a guaranteed drop of at least 11%)

... and DeepBit died about an hour ago ...
(Edit: and Slush and some of BTCGuild)

I wonder what will happen to the trailing end of this difficulty ...
(start time was "22:40:40 27-Sep-2011 UTC")


Aye, showing about 12% here now. I would think with so few blocks left the effect will be very minimal. Mainly due to the affected pool ops working their butts off to counter the attacks.

If you're not excited by the idea of being an early adopter 'now', then you should come back in three or four years and either tell us "Told you it'd never work!" or join what should, by then, be a much more stable and easier-to-use system. - GA
It is being worked on by smart people. -DamienBlack
kano
Legendary
*
Offline Offline

Activity: 1918


Linux since 1997 RedHat 4


View Profile
October 13, 2011, 02:10:33 AM
 #91

Hmm - more than 24 hours late on the diff change and still more than 100 blocks to go Sad
(so that means a guaranteed drop of at least 11%)

... and DeepBit died about an hour ago ...
(Edit: and Slush and some of BTCGuild)

I wonder what will happen to the trailing end of this difficulty ...
(start time was "22:40:40 27-Sep-2011 UTC")


Aye, showing about 12% here now. I would think with so few blocks left the effect will be very minimal. Mainly due to the affected pool ops working their butts off to counter the attacks.
Hmm - but I was thinking more: how long it might extend out for the remaining ~100 blocks.

Pool: https://kano.is BTC: 1KanoiBupPiZfkwqB7rfLXAzPnoTshAVmb
CKPool and CGMiner developer, IRC FreeNode #ckpool and #cgminer kanoi
Help keep Bitcoin secure by mining on pools with Stratum, the best protocol to mine Bitcoins with ASIC hardware
sadpandatech
Hero Member
*****
Offline Offline

Activity: 504



View Profile
October 13, 2011, 03:10:40 AM
 #92


Aye, showing about 12% here now. I would think with so few blocks left the effect will be very minimal. Mainly due to the affected pool ops working their butts off to counter the attacks.
Hmm - but I was thinking more: how long it might extend out for the remaining ~100 blocks.

Aye, I caught your drift, m8. sittin at 13% now. ;p 149184 13/10/2011 22:55   the time is about what it was a few days ago when all the large pools had 24h+ of super bad luck. It has the same effect.

Judging by the last 10 blocks being back up a bit from a few hours ago (btc guild atleast mining again), I think the attacks are losing some steam. 

  The only thing they will accomplish doing such petty things to multiple places will be getting their botnet busted quicker than usual by 'the man'.

If you're not excited by the idea of being an early adopter 'now', then you should come back in three or four years and either tell us "Told you it'd never work!" or join what should, by then, be a much more stable and easier-to-use system. - GA
It is being worked on by smart people. -DamienBlack
kano
Legendary
*
Offline Offline

Activity: 1918


Linux since 1997 RedHat 4


View Profile
October 13, 2011, 08:03:25 PM
 #93

I've just run it again (and updated it at the link http://tradebtc.net/diffcalc.php )

Back to last difficulty: -14.91%
-432 blocks (~3 days): -31.78%
-144 blocks (~1 day): -45.75%

Worst figure around -100 blocks: -82 blocks: -59.94%

So I guess the following means
Nope, bitcoin still has the off-by-1, as for now it's deemed "too big to fail" and fixing it would mean a backwards-incompatible change to the blockchain.

First: sorry for conflating the off-by-1 and the asymmetric adjustment issues.  As I said, I haven't taken the time to consider changing the bitcoin difficulty adjustment algorithm (too big a change for too little benefit, in my opinion; if we lose 90% of hashing power in a month then there that is a sign something much more serious is wrong with bitcoin than the difficulty algorithm; ...
there is ALMOST something seriously wrong with bitcoin now ...
(1 day worth of hashing suggests we might be half way to 90%)

Pool: https://kano.is BTC: 1KanoiBupPiZfkwqB7rfLXAzPnoTshAVmb
CKPool and CGMiner developer, IRC FreeNode #ckpool and #cgminer kanoi
Help keep Bitcoin secure by mining on pools with Stratum, the best protocol to mine Bitcoins with ASIC hardware
sadpandatech
Hero Member
*****
Offline Offline

Activity: 504



View Profile
October 13, 2011, 09:51:45 PM
 #94

I've just run it again (and updated it at the link http://tradebtc.net/diffcalc.php )

Back to last difficulty: -14.91%
-432 blocks (~3 days): -31.78%
-144 blocks (~1 day): -45.75%

Worst figure around -100 blocks: -82 blocks: -59.94%

So I guess the following means
in my opinion; if we lose 90% of hashing power in a month then there that is a sign something much more serious is wrong with bitcoin than the difficulty algorithm; ...
there is ALMOST something seriously wrong with bitcoin now ...
(1 day worth of hashing suggests we might be half way to 90%)

  I sure hope you don't really mean that, Mr. Anderson.  I like to take a slightly more optimistic stance on it and attribute such a loss in hashing power in the near future to 'something much more serious wrong with the present state of the community at large', rather than Bitcoin itself. You and everyone else who have put in so much time and effort to build what is in concept nearly flawless certainly deserve more optimism. I'd like to think a massive loss in hash any time soon would be more of a 'weeding' out of all the crap. And potentially, could afford many who do see the value of the protocol a more stable environment to rebuild in.. just my 2 bitcents worth.


  Yea, kano, we are down to 14% drop now and if the current little trend continues into the next difficulty, Ddosing aside, we are looking at a 25-40% for the next one. Which is fine save for the potential for it being a bloody 4 week+ change time. :/  There is however another variable that may or may not enter the equation. That is our lil botnet buddies, if they decide to actually find a place to nest and mine. They are capable of about 3T worth on their own from best I can be calcumaltin'..  Interesting timess ahead, none the less.


  Again, I must agree with Kano that atleast some discussion should be had amongst those that know more than I about the 'What If' scenario of losing a massive chunk of hash early on in a dif change......
 

If you're not excited by the idea of being an early adopter 'now', then you should come back in three or four years and either tell us "Told you it'd never work!" or join what should, by then, be a much more stable and easier-to-use system. - GA
It is being worked on by smart people. -DamienBlack
kano
Legendary
*
Offline Offline

Activity: 1918


Linux since 1997 RedHat 4


View Profile
October 24, 2011, 03:19:01 AM
 #95

OK I've been thinking about this more again.

I think the simplest solution, without the issues of having 2 different calculations interfering with each other, is to make the 2016 block diff recalc a 432 block diff recalc
Yep drop it from ~14 days to ~3 days.

Points:

If some bot net can mess up the diff recalc for 1.5 days of a short 3 day diff recalc
I really can't see them having all that much more of a problem also reaching 7 days of a 14 day diff recalc.
It's 4.7 times longer - not even an order of magnitude more.
It also means, however, that the amount of time the current difficulty will be extended due to a bot difficulty attack is quite obviously smaller for a smaller diff recalc

The difficulty is not adjusted based on the current network, it's based on the past network - since we cannot know what the current network really is.
It also is an estimate simply calculated from the past block generation rate.
This guarantees it will always be late to change.
Is that there by desire, or due to consideration of the issue with calculating it?

The last diff change extended 2 days longer that the expected 14 days.
I wonder how long this current one will go ...

If some statistics pro wants to work out and compare the expected standard deviation of 432 blocks vs 2016 blocks that would probably simplify a yes/no opinion of this.

Pool: https://kano.is BTC: 1KanoiBupPiZfkwqB7rfLXAzPnoTshAVmb
CKPool and CGMiner developer, IRC FreeNode #ckpool and #cgminer kanoi
Help keep Bitcoin secure by mining on pools with Stratum, the best protocol to mine Bitcoins with ASIC hardware
sadpandatech
Hero Member
*****
Offline Offline

Activity: 504



View Profile
October 24, 2011, 05:01:02 AM
 #96

If some statistics pro wants to work out and compare the expected standard deviation of 432 blocks vs 2016 blocks that would probably simplify a yes/no opinion of this.

  432 would put it pretty high up on the curve there. Maybe 432 blocks sooner verses every 432 would keep us from being too far off. Its not as big of an issue coming back below 1mil difficulty. I think the problem lies, and probably what the thinking is for wanting it where it is, would be for the 10mil+ diff where a few points would make a much bigger impact on the deviation.

From Maaku's post on page 1;

If you're not excited by the idea of being an early adopter 'now', then you should come back in three or four years and either tell us "Told you it'd never work!" or join what should, by then, be a much more stable and easier-to-use system. - GA
It is being worked on by smart people. -DamienBlack
kano
Legendary
*
Offline Offline

Activity: 1918


Linux since 1997 RedHat 4


View Profile
October 24, 2011, 09:57:09 AM
 #97

Oh forgot about that Smiley

Anyway, again the problem with the current is it isn't accurate at all - it's based on the past 2 weeks which means it is always wrong except when the network hash rate is unchanging.
Even though 3 days has a high (but not extreme) SD, the actual result over time shouldn't be that bad IMO
... and again in situations like now, it will speed up the diff change and help retard the major negative effects the 14 day delay is currently having

Pool: https://kano.is BTC: 1KanoiBupPiZfkwqB7rfLXAzPnoTshAVmb
CKPool and CGMiner developer, IRC FreeNode #ckpool and #cgminer kanoi
Help keep Bitcoin secure by mining on pools with Stratum, the best protocol to mine Bitcoins with ASIC hardware
Transisto
Donator
Legendary
*
Offline Offline

Activity: 1624



View Profile WWW
October 24, 2011, 07:02:07 PM
 #98

kano, What problem are your trying to solve exactly ?

Estimating future difficulty adjustment or increasing re-target frequency ?
kano
Legendary
*
Offline Offline

Activity: 1918


Linux since 1997 RedHat 4


View Profile
October 24, 2011, 07:28:37 PM
 #99

The fact that the difficulty retarget is slow and late (and the result is wrong since it represents the previous 2 weeks not the current environment) and directly promotes the recent BTC economic climate.

If BTC climate it decreasing, it's slow to decrease the difficulty (and can be very slow to decrease it) and thus promotes making the decreasing environment worse.

Pool: https://kano.is BTC: 1KanoiBupPiZfkwqB7rfLXAzPnoTshAVmb
CKPool and CGMiner developer, IRC FreeNode #ckpool and #cgminer kanoi
Help keep Bitcoin secure by mining on pools with Stratum, the best protocol to mine Bitcoins with ASIC hardware
sadpandatech
Hero Member
*****
Offline Offline

Activity: 504



View Profile
October 24, 2011, 07:43:02 PM
 #100

kano, What problem are your trying to solve exactly ?

Estimating future difficulty adjustment or increasing re-target frequency ?

  Both.

  Yes, Kano I can definetly agree its always off anyhows. But one needs to consider the costs verses the benfits of coding such a change. And the potential issues a change in the dataset being used could have. Using less than ~1600 blocks starts to have a pretty big error rate when at high difficulty(10mil+). This, as you pointed out is more caused by the fact we are looking at past blocks for the data set and not so much by the deviation. The only true fix would have to be able to see the future. its just not possible.

  One possible solution that I was pondering, that would atleast make it a little more 'current' rate friendly would be to keep the same 2016 formula intact and simply adjust the weight given to the last 144 blocks mined to make the retarget more accurate by focusing more on the present hash rate. Now, the issue here would be the ability to game that. Not as much as only using 144 blocks would, but it is still 'hackable'. I believe this could be solved by using a 'Heartbeat Adjustment' to the last 144 current weight. I.e., we'd take the largest, consistent block reporters and adjust the next retarget by the % they are above or below it for x number of blocks. We would adjust how much weight each one of them gave to the 'HA' based on uptime. So if one were to be heavy ddos victems their last block set would count for only %N uptime into the adjustment.

  I know this idea would be much more useful with some example formulas, which I will gladly draw up real quick if anyone is really interested in my ramblings. Otherwise I am sure the better math minds here can see it in their heads like I can. *And are probably facepalming at me. ;p

  Cheers

If you're not excited by the idea of being an early adopter 'now', then you should come back in three or four years and either tell us "Told you it'd never work!" or join what should, by then, be a much more stable and easier-to-use system. - GA
It is being worked on by smart people. -DamienBlack
Pages: « 1 2 3 4 [5] 6 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!