Bitcoin Forum
December 04, 2016, 08:29:25 PM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1] 2  All
  Print  
Author Topic: Bitcoin scraped pool luck data/graphics  (Read 8089 times)
topheroly
Newbie
*
Offline Offline

Activity: 21



View Profile
August 01, 2011, 02:18:31 AM
 #1

Hello all, I've decided to share something that I think would be interesting to this community.  Over this past weekend I wrote a script that scrapes the biggest pools for blocks (deepbit/btcguild/slush/mtred/btcmine for now) and uses that information to get a running estimate of the pool's luck.  I know that some pools delay their stats so this information comes from the reported blocks when they are gathered.  Another key thing to note, when the difficulty switches, the first tick of information from this switch will be off, bare that in mind.  Otherwise all other information should be accurate.

About the graphic:
The images are just plots of luck as time goes on.  The largeness of the points themselves indicate how long of a time length for that estimate.  The smallest points are 6 hr windows, the second being 12 hrs and the largest dots are 24 hr time window estimates of luck.  There is a horizontal dotted rule added as a reference line as well as a fit to the data (the solid black line that is very thin).  Around the fit, there is a 95% confidence region.

The second image is simply a histogram of the luck data collected in the past 5 days (the limit I'm holding this graphic plotting region).  From this you can see roughly how each pool is doing overall. You can find this image in l0ss.net/images under the histogram png.  I moved it because it didn't fit with the other ones.

Site is now hosted here, updated daily at 5pm central:
http://www.l0ss.net/

For the inclined:
 I wrote this completely in R http://www.r-project.org/ and rastered the graphics using the package ggplot2 http://had.co.nz/ggplot2/.  The formula used to calculate luck is

z <- (x/y)
if z < 1
    luck <- (1 - z)*(-1)
if z > 1
    luck <- (z - 1)


x == Actual Blocks found within Time Frame
y == Estimated Blocks found within Time Frame

To calculate estimated blocks, use the formula (difficulty * 2^32)/hashrate.  These numbers are also grabbed from each site when the script is run.


Future work to do on this script:
1 - get more data from pools that are sized around ~250 GHash/s or more currently.
2 - write in a difficulty indicator on the graphs
3 - have a separate long luck running graph for > 30 days

If you have questions about any of this please let me know and I will do my best to answer them.
1480883365
Hero Member
*
Offline Offline

Posts: 1480883365

View Profile Personal Message (Offline)

Ignore
1480883365
Reply with quote  #2

1480883365
Report to moderator
1480883365
Hero Member
*
Offline Offline

Posts: 1480883365

View Profile Personal Message (Offline)

Ignore
1480883365
Reply with quote  #2

1480883365
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1480883365
Hero Member
*
Offline Offline

Posts: 1480883365

View Profile Personal Message (Offline)

Ignore
1480883365
Reply with quote  #2

1480883365
Report to moderator
1480883365
Hero Member
*
Offline Offline

Posts: 1480883365

View Profile Personal Message (Offline)

Ignore
1480883365
Reply with quote  #2

1480883365
Report to moderator
1480883365
Hero Member
*
Offline Offline

Posts: 1480883365

View Profile Personal Message (Offline)

Ignore
1480883365
Reply with quote  #2

1480883365
Report to moderator
grod
Full Member
***
Offline Offline

Activity: 154


View Profile
August 01, 2011, 02:37:37 AM
 #2

These figures make sense since BTCguild and slush are both part of a popular pool hopping script.

Hoppers join BTCguild for fresh blocks, stay for the "lucky" blocks, leave for the "unlucky" ones which means the short blocks get the most computing power and more "lucky" blocks get found than "unlucky" ones, though the "unlucky" ones take disproportionately longer to solve.

Slush's pool pays better for people joining late for "unlucky" blocks, so people hop out to fresh blocks in other pools and hop back in to slush otherwise, meaning fewer computing resources get used to find "lucky" blocks there.
topheroly
Newbie
*
Offline Offline

Activity: 21



View Profile
August 01, 2011, 02:45:38 AM
 #3

I forgot to mention that you can see a very cool effect of variance reduction among the pools.  If memory serves right, variance goes off as 1/sqrt(n) where n is the sample size, in our case number of miners.  Thus we see deepbit having the sharpest peak of the pools followed by btcguild and then finally slush.
Clipse
Hero Member
*****
Offline Offline

Activity: 504


View Profile
August 01, 2011, 02:52:28 AM
 #4

These figures make sense since BTCguild and slush are both part of a popular pool hopping script.

Hoppers join BTCguild for fresh blocks, stay for the "lucky" blocks, leave for the "unlucky" ones which means the short blocks get the most computing power and more "lucky" blocks get found than "unlucky" ones, though the "unlucky" ones take disproportionately longer to solve.

Slush's pool pays better for people joining late for "unlucky" blocks, so people hop out to fresh blocks in other pools and hop back in to slush otherwise, meaning fewer computing resources get used to find "lucky" blocks there.

WOW, wOOOw and WooooooW !!!

You have absolutely no understanding of how hopper works, let me try and educate you.

1.) BTCguild stats delayed by 1hr, effectively hoppers dont know when a fresh block starts.
2.) "lucky" blocks? Say what? Hoppers have no idea if the current running block will be long or short block, they cant magically know to stay because its going to magically be a lucky block.

You should really get a better understanding but with the logic you just posted I dont think any proper understanding will help you.

...In the land of the stale, the man with one share is king... >> Clipse

We pay miners at 130% PPS | Signup here : Bonus PPS Pool (Please read OP to understand the current process)
Sukrim
Legendary
*
Offline Offline

Activity: 1848


View Profile
August 01, 2011, 03:10:59 AM
 #5

Exactly - these stats are nice but have 0 prediction value for the future!

Could you do a month, quarter, year and alltime stats overview too? A week is a bit short...

https://bitfinex.com <-- leveraged trading of BTCUSD, LTCUSD and LTCBTC (long and short) - 10% discount on fees for the first 30 days with this refcode: x5K9YtL3Zb
Mail me at Bitmessage: BM-BbiHiVv5qh858ULsyRDtpRrG9WjXN3xf
Meatpile
Sr. Member
****
Offline Offline

Activity: 277


View Profile
August 01, 2011, 03:31:25 AM
 #6

Those are very interesting graphs

Even more interesting is all the people who will make "intelligent claims" on why this graph of luck is shaped the way it is.
topheroly
Newbie
*
Offline Offline

Activity: 21



View Profile
August 01, 2011, 03:33:34 AM
 #7

If anything I wanted to show that luck ... is luck.  It's all basic probability, anytime makes some silly claim about biased pool luck, they need only look to this image now.  If there was something fishy going on, we would be able to see it.  I also have updated my to-do list, including adding a long-running graph and stats, etc.
Sukrim
Legendary
*
Offline Offline

Activity: 1848


View Profile
August 01, 2011, 04:03:46 AM
 #8

If there was something fishy going on, we would be able to see it.
Would it really show up, if a pool operator would steal (with a random function) every 300th block for example?

https://bitfinex.com <-- leveraged trading of BTCUSD, LTCUSD and LTCBTC (long and short) - 10% discount on fees for the first 30 days with this refcode: x5K9YtL3Zb
Mail me at Bitmessage: BM-BbiHiVv5qh858ULsyRDtpRrG9WjXN3xf
topheroly
Newbie
*
Offline Offline

Activity: 21



View Profile
August 01, 2011, 04:07:14 AM
 #9

Sukrim, good point.  I guess if they took a small block duration out then no, It would be very hard to identify.  I guess what I was more eluding to were errors on the pool's fault, like a consistently running bad luck due to some software error in pushpool or what have you.  I guess we'll see how useful this information really is in the future.
Meatpile
Sr. Member
****
Offline Offline

Activity: 277


View Profile
August 01, 2011, 04:12:12 AM
 #10

I would assume this graphing would only show systemic problems, Like if someone figured out how to cheat the system as a whole.

At which point they wouldn't be a member of a pool i would assume... So besides aesthetically interesting I sort of question the usefulness of the graphs for identifying such a purpose.  (now if you could graph every single individual on the network wowee)
grod
Full Member
***
Offline Offline

Activity: 154


View Profile
August 01, 2011, 06:38:49 AM
 #11

These figures make sense since BTCguild and slush are both part of a popular pool hopping script.

Hoppers join BTCguild for fresh blocks, stay for the "lucky" blocks, leave for the "unlucky" ones which means the short blocks get the most computing power and more "lucky" blocks get found than "unlucky" ones, though the "unlucky" ones take disproportionately longer to solve.

Slush's pool pays better for people joining late for "unlucky" blocks, so people hop out to fresh blocks in other pools and hop back in to slush otherwise, meaning fewer computing resources get used to find "lucky" blocks there.

WOW, wOOOw and WooooooW !!!

You have absolutely no understanding of how hopper works, let me try and educate you.

1.) BTCguild stats delayed by 1hr, effectively hoppers dont know when a fresh block starts.
2.) "lucky" blocks? Say what? Hoppers have no idea if the current running block will be long or short block, they cant magically know to stay because its going to magically be a lucky block.

You should really get a better understanding but with the logic you just posted I dont think any proper understanding will help you.

Why does pool hopping work?  Because working on the freshest blocks in proportional pools pays more than working on stale blocks till the cows come home. 

Think about the effect of mining power being added only when the pool starts on a new block.  If that block is 'lucky' (read: finishes before 43.5% of difficulty shares) then the pool starts on a new block.  If the block is NOT 'lucky', the mining power leaves, and the 'unlucky' block gets mined for a longer period.   The pool has the most processing power ONLY for 'lucky' or short blocks, meaning the pool has higher odds of having MORE of them.  And fewer 'unlucky' blocks which take far longer to process. 

In other workds, the 'luck' did not change, but the speed at which 'lucky' and 'unlucky' blocks get solved did affecting the distribution over time.

Since it is pointless to be in a PPLNS pool early on, hoppers will be mining in proportional pools.  Once those are all stale they switch to a PPLNS pool.  Since nobody has announced a solved block by definition they're joining a stale or 'unlucky' block, where their payout is not affected as much by all the other work.  So the reverse happens - less mining power during 'lucky' blocks means they take longer, so they appear less frequently in the distribution.

As far as not knowing when a fresh block starts at BTC -- that is a difficult problem, you got me there.  It seems like there was no 1 hour delay in crediting my account for solved blocks when I was donating 2.5%, but that could have changed since I last mined there.

Clipse
Hero Member
*****
Offline Offline

Activity: 504


View Profile
August 01, 2011, 07:29:48 AM
 #12

These figures make sense since BTCguild and slush are both part of a popular pool hopping script.

Hoppers join BTCguild for fresh blocks, stay for the "lucky" blocks, leave for the "unlucky" ones which means the short blocks get the most computing power and more "lucky" blocks get found than "unlucky" ones, though the "unlucky" ones take disproportionately longer to solve.

Slush's pool pays better for people joining late for "unlucky" blocks, so people hop out to fresh blocks in other pools and hop back in to slush otherwise, meaning fewer computing resources get used to find "lucky" blocks there.

WOW, wOOOw and WooooooW !!!

You have absolutely no understanding of how hopper works, let me try and educate you.

1.) BTCguild stats delayed by 1hr, effectively hoppers dont know when a fresh block starts.
2.) "lucky" blocks? Say what? Hoppers have no idea if the current running block will be long or short block, they cant magically know to stay because its going to magically be a lucky block.

You should really get a better understanding but with the logic you just posted I dont think any proper understanding will help you.

Why does pool hopping work?  Because working on the freshest blocks in proportional pools pays more than working on stale blocks till the cows come home. 

Think about the effect of mining power being added only when the pool starts on a new block.  If that block is 'lucky' (read: finishes before 43.5% of difficulty shares) then the pool starts on a new block.  If the block is NOT 'lucky', the mining power leaves, and the 'unlucky' block gets mined for a longer period.   The pool has the most processing power ONLY for 'lucky' or short blocks, meaning the pool has higher odds of having MORE of them.  And fewer 'unlucky' blocks which take far longer to process. 

In other workds, the 'luck' did not change, but the speed at which 'lucky' and 'unlucky' blocks get solved did affecting the distribution over time.

Since it is pointless to be in a PPLNS pool early on, hoppers will be mining in proportional pools.  Once those are all stale they switch to a PPLNS pool.  Since nobody has announced a solved block by definition they're joining a stale or 'unlucky' block, where their payout is not affected as much by all the other work.  So the reverse happens - less mining power during 'lucky' blocks means they take longer, so they appear less frequently in the distribution.

As far as not knowing when a fresh block starts at BTC -- that is a difficult problem, you got me there.  It seems like there was no 1 hour delay in crediting my account for solved blocks when I was donating 2.5%, but that could have changed since I last mined there.



Again I will add, you sir have no clue.

...In the land of the stale, the man with one share is king... >> Clipse

We pay miners at 130% PPS | Signup here : Bonus PPS Pool (Please read OP to understand the current process)
Sukrim
Legendary
*
Offline Offline

Activity: 1848


View Profile
August 01, 2011, 09:36:50 AM
 #13

Again I will add, you sir have no clue.

I second that. Read a bit more about statistics + true random events. What you say would only (maybe) be true in a pool that pays everyone for the last N minutes of mining work, not for the shares.

Oh and I throw in the 9 second block recently at slush for good measure.

https://bitfinex.com <-- leveraged trading of BTCUSD, LTCUSD and LTCBTC (long and short) - 10% discount on fees for the first 30 days with this refcode: x5K9YtL3Zb
Mail me at Bitmessage: BM-BbiHiVv5qh858ULsyRDtpRrG9WjXN3xf
grod
Full Member
***
Offline Offline

Activity: 154


View Profile
August 02, 2011, 03:15:52 AM
 #14

You are right, there is nothing to see here. 

Correlation does not imply causality, and just because btcguild and slush were part of bithopper's default config and deepbit wasn't, combined with rise in popularity of pool hopping mapping roughly to the 'luck' growth trend is just a series of coincidences.

However, if it is by some miracle that is not a series of happy coincidences but instead some interaction I'm not smart enough to figure out then it's just that much more reason to run a pool hopper.

topheroly
Newbie
*
Offline Offline

Activity: 21



View Profile
August 04, 2011, 08:33:03 AM
 #15

Made a site, hosted, and looks better.  Check it out.  Updated to-do list.

http://www.l0ss.net/
topheroly
Newbie
*
Offline Offline

Activity: 21



View Profile
August 05, 2011, 02:35:17 AM
 #16

Script now includes mtred and btcmine in addition to deepbit, btcguild and slush.
TraderTimm
Legendary
*
Offline Offline

Activity: 1652



View Profile
August 05, 2011, 02:44:20 PM
 #17

Very nice, certainly useful when it comes to the solo-vs-pooled mining arguments. Good job!

fortitudinem multis - catenum regit omnia
topheroly
Newbie
*
Offline Offline

Activity: 21



View Profile
August 06, 2011, 01:00:14 AM
 #18

Script now generates some basic descriptive statistics on the sidebar.
Clipse
Hero Member
*****
Offline Offline

Activity: 504


View Profile
August 06, 2011, 01:35:34 AM
 #19

You are right, there is nothing to see here.  

Correlation does not imply causality, and just because btcguild and slush were part of bithopper's default config and deepbit wasn't, combined with rise in popularity of pool hopping mapping roughly to the 'luck' growth trend is just a series of coincidences.

However, if it is by some miracle that is not a series of happy coincidences but instead some interaction I'm not smart enough to figure out then it's just that much more reason to run a pool hopper.



Hey guess what, I and many others have other means of hopping even deepbit. So again your theory is loaded with cow maneure.

Of course there is no reason not to run a pool hopper, the question is will you drop your faith of being against it or join it making you a hypocrit like so many others? :p

...In the land of the stale, the man with one share is king... >> Clipse

We pay miners at 130% PPS | Signup here : Bonus PPS Pool (Please read OP to understand the current process)
Clipse
Hero Member
*****
Offline Offline

Activity: 504


View Profile
August 06, 2011, 01:37:32 AM
 #20

Script now generates some basic descriptive statistics on the sidebar.

Btw , love the new site layout with the graphs, good job.

...In the land of the stale, the man with one share is king... >> Clipse

We pay miners at 130% PPS | Signup here : Bonus PPS Pool (Please read OP to understand the current process)
Pages: [1] 2  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!