Eligius: 0% Fee BTC, 105% PPS NMC, No registration, CPPSRB

[zefir]

I need some help to correctly interpret the pool statistics.

To my understanding, the round luck expresses the ratio of spent shares to solve a block. While intuitive as immediate value, it is not suitable for intuitive statistical analyses. Take e.g. the following series of luck values (real ones starting from block 338694):

• 137.70%
• 3971.90%
• 18.20%
• 393.70%

Looks not so bad at first sight - makes one think the sub 20% one is well compensated by the lucky ones. But truth is, you need to average over the reciprocal values to get the combined luck of a series, formally:
luck(n₁..n_k) = 1 / [(1/n₁ + 1/n₂ + ... + 1/n_k) / k]

For the above example we get the series luck calculated as
1 / [(0.726 + 0.025 + 5.495 + 0.254)/4] = 1 / [6.5/4] = 61.54%

If my assumption is correct, then this was a quite bad luck series.

Talking variance: shouldn't the average over a longer series be close to 100%? I manually typed the round lucks for the last 100 blocks into a spreadsheet for closer inspection. The averaged luck over this series (covering last ~3 weeks of mining) is 78.74% - which I found shocking low since variance should be leveled out quite well. I disbelieve this number that much that I assume my interpretation is just wrong.

Anyone with better understanding willing to comment?

@wizkid: are those numbers at http://eligius.st/~wizkid057/newstats/blocks.php available in CSV or JSON format?


Thanks

[1715005754]

1715005754

[1715005754]

1715005754

[1715005754]

1715005754

[1715005754]

1715005754

[MrTeal]

I need some help to correctly interpret the pool statistics.

To my understanding, the round luck expresses the ratio of spent shares to solve a block. While intuitive as immediate value, it is not suitable for intuitive statistical analyses. Take e.g. the following series of luck values (real ones starting from block 338694):

• 137.70%
• 3971.90%
• 18.20%
• 393.70%

Looks not so bad at first sight - makes one think the sub 20% one is well compensated by the lucky ones. But truth is, you need to average over the reciprocal values to get the combined luck of a series, formally:
luck(n₁..n_k) = 1 / [(1/n₁ + 1/n₂ + ... + 1/n_k) / k]

For the above example we get the series luck calculated as
1 / [(0.726 + 0.025 + 5.495 + 0.254)/4] = 1 / [6.5/4] = 61.54%

If my assumption is correct, then this was a quite bad luck series.

Talking variance: shouldn't the average over a longer series be close to 100%? I manually typed the round lucks for the last 100 blocks into a spreadsheet for closer inspection. The averaged luck over this series (covering last ~3 weeks of mining) is 78.74% - which I found shocking low since variance should be leveled out quite well. I disbelieve this number that much that I assume my interpretation is just wrong.

Anyone with better understanding willing to comment?

@wizkid: are those numbers at http://eligius.st/~wizkid057/newstats/blocks.php available in CSV or JSON format?


Thanks

That sounds about right. According to organofcorti's blog, our average time to find a block over the last 3 weeks (Jan 04-24 inclusive) is 1.2x, 1.2x and 1.42x the expected value, which is 78.5%. It's been a bad start to the year.

[lanfeusst]

I need some help to correctly interpret the pool statistics.

To my understanding, the round luck expresses the ratio of spent shares to solve a block. While intuitive as immediate value, it is not suitable for intuitive statistical analyses. Take e.g. the following series of luck values (real ones starting from block 338694):

• 137.70%
• 3971.90%
• 18.20%
• 393.70%

Looks not so bad at first sight - makes one think the sub 20% one is well compensated by the lucky ones. But truth is, you need to average over the reciprocal values to get the combined luck of a series, formally:
luck(n₁..n_k) = 1 / [(1/n₁ + 1/n₂ + ... + 1/n_k) / k]

For the above example we get the series luck calculated as
1 / [(0.726 + 0.025 + 5.495 + 0.254)/4] = 1 / [6.5/4] = 61.54%

If my assumption is correct, then this was a quite bad luck series.

Talking variance: shouldn't the average over a longer series be close to 100%? I manually typed the round lucks for the last 100 blocks into a spreadsheet for closer inspection. The averaged luck over this series (covering last ~3 weeks of mining) is 78.74% - which I found shocking low since variance should be leveled out quite well. I disbelieve this number that much that I assume my interpretation is just wrong.

Anyone with better understanding willing to comment?

@wizkid: are those numbers at http://eligius.st/~wizkid057/newstats/blocks.php available in CSV or JSON format?


Thanks

After making the math for last 100 blocks :

Code:

cat Block\ List\ -\ Eligius\ Pool\ Statistics.html |egrep % | awk '{print $8}' | cut -d % -f 1 | cut -d '>' -f 2 | tr -d , | sed 1d | head -n 100 | awk '{SUM+=1/$1} END {print NR/SUM}'

78.8334% luck

For last 200 :

Code:

cat Block\ List\ -\ Eligius\ Pool\ Statistics.html |egrep % | awk '{print $8}' | cut -d % -f 1 | cut -d '>' -f 2 | tr -d , | sed 1d | head -n 200 | awk '{SUM+=1/$1} END {print NR/SUM}'

82.7441% luck

For last 500 :

Code:

cat Block\ List\ -\ Eligius\ Pool\ Statistics.html |egrep % | awk '{print $8}' | cut -d % -f 1 | cut -d '>' -f 2 | tr -d , | sed 1d | head -n 500 | awk '{SUM+=1/$1} END {print NR/SUM}'

90.384% luck

For last 1000 :

Code:

cat Block\ List\ -\ Eligius\ Pool\ Statistics.html |egrep % | awk '{print $8}' | cut -d % -f 1 | cut -d '>' -f 2 | tr -d , | sed 1d | head -n 1000 | awk '{SUM+=1/$1} END {print NR/SUM}'

93.1394% luck

For last 3000 :

Code:

cat Block\ List\ -\ Eligius\ Pool\ Statistics.html |egrep % | awk '{print $8}' | cut -d % -f 1 | cut -d '>' -f 2 | tr -d , | sed 1d | head -n 3000 | awk '{SUM+=1/$1} END {print NR/SUM}'

94.1135% luck

For everything (8870):

Code:

cat Block\ List\ -\ Eligius\ Pool\ Statistics.html |egrep % | awk '{print $8}' | cut -d % -f 1 | cut -d '>' -f 2 | tr -d , | sed 1d | awk '{SUM+=1/$1} END {print NR/SUM}'

95.989% luck (The pool is then equivalent for a loyal miner to a 4% fee PPS)



Orphan rate is 198/(8870+198) = 2.18% (Quite high if you ask me)
The total luck rate for the pool lifetime with orphan is :
 95.989*(8870+198)/8870 = 98.13 %

Which leaves 1.87 % of withholding hashing power for the pool lifetime assuming it is a long enough time frame (which I think it is)

[Grix]

That sounds about right. According to organofcorti's blog, our average time to find a block over the last 3 weeks (Jan 04-24 inclusive) is 1.2x, 1.2x and 1.42x the expected value, which is 78.5%. It's been a bad start to the year.

It's been about equally bad since about the start of December.

[MrTeal]

I've noticed something interesting with the stats today, and I'm not sure what to make of it.
I have two Antminer S5s, one running at 387.5MHz and one at 400MHz, each with their own worker. I unplugged one (the 400MHz) about 13 hours ago, and I wanted to see what the 12 hour stats looked like. When I first checked, both workers had the same 12hr average down to the hundredth of a GH/s. I though it was a little weird but just figured there was still some of the off time in there and it was a coincidence, so I waited for the next update. The 12 hr average diverged at the next update a little, but at this most recent update they are back to displaying not only the same hashrate, but exactly the same number of accepted shares.
Worker 135   12 Hours   1,301.04 Gh/s   13086208  <-The 387.5MHz one, device shows 1278GH/s
Worker 136   12 Hours   1,301.04 Gh/s   13086208  <-The 400MHz one, device shows 1319GH/s

I could see them matching up once, but having the exact same number of accepted shares twice in an hour seems extremely unlikely.

[Luke-Jr]

I've noticed something interesting with the stats today, and I'm not sure what to make of it.
I have two Antminer S5s, one running at 387.5MHz and one at 400MHz, each with their own worker. I unplugged one (the 400MHz) about 13 hours ago, and I wanted to see what the 12 hour stats looked like. When I first checked, both workers had the same 12hr average down to the hundredth of a GH/s. I though it was a little weird but just figured there was still some of the off time in there and it was a coincidence, so I waited for the next update. The 12 hr average diverged at the next update a little, but at this most recent update they are back to displaying not only the same hashrate, but exactly the same number of accepted shares.
Worker 135   12 Hours   1,301.04 Gh/s   13086208  <-The 387.5MHz one, device shows 1278GH/s
Worker 136   12 Hours   1,301.04 Gh/s   13086208  <-The 400MHz one, device shows 1319GH/s

I could see them matching up once, but having the exact same number of accepted shares twice in an hour seems extremely unlikely.

I think the S5 has a bug where it only sends up to N shares per period of time, which would explain this behaviour.

[BCwinning]

I've noticed something interesting with the stats today, and I'm not sure what to make of it.
I have two Antminer S5s, one running at 387.5MHz and one at 400MHz, each with their own worker. I unplugged one (the 400MHz) about 13 hours ago, and I wanted to see what the 12 hour stats looked like. When I first checked, both workers had the same 12hr average down to the hundredth of a GH/s. I though it was a little weird but just figured there was still some of the off time in there and it was a coincidence, so I waited for the next update. The 12 hr average diverged at the next update a little, but at this most recent update they are back to displaying not only the same hashrate, but exactly the same number of accepted shares.
Worker 135   12 Hours   1,301.04 Gh/s   13086208  <-The 387.5MHz one, device shows 1278GH/s
Worker 136   12 Hours   1,301.04 Gh/s   13086208  <-The 400MHz one, device shows 1319GH/s

I could see them matching up once, but having the exact same number of accepted shares twice in an hour seems extremely unlikely.

I think the S5 has a bug where it only sends up to N shares per period of time, which would explain this behaviour.

Why do you think that? Have you brought this to bitmain's attention or?

[Moria843]

Sixteen bitcoin miners and what do you get?
Another day older and deeper in debt
St. Peter don’t you call me cause I can’t go,
I owe my soul to the electric company store

[MrTeal]

Alright, test one.
I switched the 400MHz unit to a different worker (test) and let it run for 50 minutes.
test      
12 Hours   88.90 Gh/s   894208
3 Hours   355.61 Gh/s   894208
22.5 Minutes   710.20 Gh/s   223232



There will be a little bit of a difference in the shares since I don't have access to the box right now, so I refreshed the minerstatus tab and then restarted cgminer, but there was probably a second or so of delay there. Interestingly, (2^48/65535)*916307/(50*60) gives 1312GH/s. The actual accepted shares gives (2^48/65535)*693682/(50*60)=1279GH/s

I'm rerunning the test at 412.5 to see if the difference between poolside hashrate and displayed hashrate increases.

Edit: Reran the tests.

test3      
12 Hours   175.59 Gh/s   1766144
3 Hours   702.36 Gh/s   1766144
22.5 Minutes   487.04 Gh/s   153088


There doesn't seem to be a limit that I'm easily able to reach on the number of shares/min that I'm able to reach.

[zefir]

That sounds about right. According to organofcorti's blog, our average time to find a block over the last 3 weeks (Jan 04-24 inclusive) is 1.2x, 1.2x and 1.42x the expected value, which is 78.5%. It's been a bad start to the year.

Thank you for pointing me to that. What is interesting to note is that recently all major evaluated pools ran into bad luck. Looking back around one year, we see how bad luck and good luck pools were par - which is what one would expect. Today, all of the pools are significantly unlucky (over 1.2). How probable it can be that a set of pools worth 50PH are unlucky - individually and as a set? Variance aside, there has been a monotonic down-trend in the luck over the past year - unless I knew better I'd suspect either a systematic error in how the statistics are collected or something very fishy is going on. I'll try to get some feedback on that from organofcorti before starting conspiracy.

Which leaves 1.87 % of withholding hashing power for the pool lifetime assuming it is a long enough time frame (which I think it is)

Thanks man (or should I say: bash-guru ),

that was exactly what I was looking for. Kudos for instead of asking for CSV data, extract it from the website (small fix: your command for the last 3000 has a typo).

Your results show exactly what I suspected my feelings are cheating me: while the all-time average seems ok, the luck is monotonically getting worse. Compare any series of the last N blocks to the previous one to see what I mean. Must be missing something...

[sorry2xs]

Really this started to get worse about three months ago when the FBI released a statement on the tv networks that they were washing their hands with Bitcoin, and it does smell rotten Fish!

[Biodom]

That sounds about right. According to organofcorti's blog, our average time to find a block over the last 3 weeks (Jan 04-24 inclusive) is 1.2x, 1.2x and 1.42x the expected value, which is 78.5%. It's been a bad start to the year.

Thank you for pointing me to that. What is interesting to note is that recently all major evaluated pools ran into bad luck. Looking back around one year, we see how bad luck and good luck pools were par - which is what one would expect. Today, all of the pools are significantly unlucky (over 1.2). How probable it can be that a set of pools worth 50PH are unlucky - individually and as a set? Variance aside, there has been a monotonic down-trend in the luck over the past year - unless I knew better I'd suspect either a systematic error in how the statistics are collected or something very fishy is going on. I'll try to get some feedback on that from organofcorti before starting conspiracy.

Which leaves 1.87 % of withholding hashing power for the pool lifetime assuming it is a long enough time frame (which I think it is)

Thanks man (or should I say: bash-guru ),

that was exactly what I was looking for. Kudos for instead of asking for CSV data, extract it from the website (small fix: your command for the last 3000 has a typo).

Your results show exactly what I suspected my feelings are cheating me: while the all-time average seems ok, the luck is monotonically getting worse. Compare any series of the last N blocks to the previous one to see what I mean. Must be missing something...

yes, nobody complained repeatedly when it was going up and down, but since december, it is going either down or stays normal (there is no UP). there are many periods of underreporting, but very few (if any) of outperforming.

[kano]

That sounds about right. According to organofcorti's blog, our average time to find a block over the last 3 weeks (Jan 04-24 inclusive) is 1.2x, 1.2x and 1.42x the expected value, which is 78.5%. It's been a bad start to the year.

Thank you for pointing me to that. What is interesting to note is that recently all major evaluated pools ran into bad luck. Looking back around one year, we see how bad luck and good luck pools were par - which is what one would expect. Today, all of the pools are significantly unlucky (over 1.2). How probable it can be that a set of pools worth 50PH are unlucky - individually and as a set? Variance aside, there has been a monotonic down-trend in the luck over the past year - unless I knew better I'd suspect either a systematic error in how the statistics are collected or something very fishy is going on. I'll try to get some feedback on that from organofcorti before starting conspiracy.

...

"all" pools ... no.

[RealMalatesta]

Am I the only one not seeing the last two blocks in my earnings?

[fryarminer]

Sixteen bitcoin miners and what do you get?
Another day older and deeper in debt
St. Peter don’t you call me cause I can’t go,
I owe my soul to the electric company store

LOL!!!!

[wizkid057]

Am I the only one not seeing the last two blocks in my earnings?

Everything looks fine on this end.  Submit a ticket if you think there is an issue with your specific account, but I see nothing wrong here.

You guys know there is an API to pull block data in machine readable formats, right?  You don't have to parse the stats pages...

http://eligius.st/~gateway/pool-apis

http://eligius.st/~wizkid057/newstats/api.php?cmd=getblocks&limit=10&format=json&sortby=time&offset=0

http://eligius.st/~wizkid057/newstats/api.php?cmd=getblocks&limit=10&format=csv&sortby=time&offset=0&csvastext=1

http://eligius.st/~wizkid057/newstats/api.php?cmd=getblocks&limit=100&format=csv&sortby=time&offset=0&showpretty=0&csvastext=1

Tons of options.

[zefir]

[...]
http://eligius.st/~wizkid057/newstats/api.php?cmd=getblocks&limit=100&format=csv&sortby=time&offset=0&showpretty=0&csvastext=1

Tons of options.

Thanks, worked - one just needs to RTFM  .

This is how the luck developed since eligius started mining, averaged over 300 and 1000 blocks:


The feeling that luck got awfully bad over the past two moths is not cheating us. At the same time, it was worse 10 months ago. Let's hope we saw the bottom.

[lanfeusst]

Nice chart, nothing unusual here I think but the fact that a lot of majors pools have bad luck for 1 month now is a little worrying... No reason for all the network to be unlucky for a long time.
Need to wait a little more time to confirm there is a problem

[sorry2xs]

are the stats page down 

[kano]

Nice chart, nothing unusual here I think but the fact that a lot of majors pools have bad luck for 1 month now is a little worrying... No reason for all the network to be unlucky for a long time.
Need to wait a little more time to confirm there is a problem

Got some actual stats to point out "a lot of majors pools have bad luck for 1 month now"