Bitcoin Forum

Bitcoin => Pools => Topic started by: organofcorti on February 24, 2012, 08:39:20 AM



Title: Neighbourhood Pool Watch
Post by: organofcorti on February 24, 2012, 08:39:20 AM
Current post:
Early detection of unintentional block withholding (http://organofcorti.blogspot.com/2016/02/detecting-unintentional-block.html)

Some previous posts:
FAQ: Bitcoin mining and the luck statistic (http://organofcorti.blogspot.com/2015/07/faq-bitcoin-mining-and-luck-statistic.html)

Measures of network hashrate centralisation (http://organofcorti.blogspot.com/2014/12/measures-of-network-hashrate.html)

Neighbourhood Pool Watch 16.9 Another short post on mapping the historical hashrate distribution (http://organofcorti.blogspot.com/2014/08/169-another-short-post-on-mapping.html)

Neighbourhood Pool Watch 16.7 Satoshi's hashrate (http://organofcorti.blogspot.com/2014/08/167-satoshis-hashrate.html)

Neighbourhood Pool Watch 18.1 Who owns 1HTM4TYSXF5yZKLpco6MTUUNfSBCiiwGsU ? (http://organofcorti.blogspot.com/2014/03/181-who-owns-1htm4tysxf5yzklpco6mtuunfs.html)

Neighbourhood Pool Watch 9.18 Gridseed - should you mine bitcoin or litecoin? (http://organofcorti.blogspot.com/2014/03/918-gridseed-should-you-mine-bitcoin-or.html)

Neighbourhood Pool Watch  16.3 The unknown network hashrate. (http://organofcorti.blogspot.com/2014/01/163-unknown-network-hashrate.html)

Neighbourhood Pool Watch  17 Goodbye HHTT. The coinbase will never be as strange again. (http://organofcorti.blogspot.com/2014/01/17-goodbye-hhtt-coinbase-will-never-be.html)

Neighbourhood Pool Watch  16.2 A short post on mapping the historical pool hashrate (http://organofcorti.blogspot.com/2013/11/162-short-post-on-mapping-historical.html)

Neighbourhood Pool Watch 16.1 The network: Orphaned blocks part 1 (http://organofcorti.blogspot.com/2013/10/161-network-orphaned-blocks-part-1.html)

Neighbourhood Pool Watch 12.4 Miner hashrate distributions 28th May 2013 (http://organofcorti.blogspot.com/2013/05/124-miner-hashrate-distributions-28th.html)

Neighbourhood Pool Watch 13.1: BitMinter and "luck". (http://organofcorti.blogspot.com/2013/05/131-bitminter-and-luck.html)

Neighbourhood Pool Watch 12.2 Pool and network miner hashrate distributions. (http://organofcorti.blogspot.com/2013/05/122-pool-and-network-miner-hashrate_3.html)

Neighbourhood Pool Watch 12.1 The pre-ASIC network hashrate distribution. (http://organofcorti.blogspot.com/2013/04/121-pre-asic-network-hashrate.html)

Neighbourhood Pool Watch 8.2 BitcoinPool: The great anomaly hunt. (http://organofcorti.blogspot.com/2013/01/82-bitcoinpool-great-anomaly-hunt.html)

Neighbourhood Pool Watch 9.4 ASIC choices update 24th December (http://organofcorti.blogspot.com/2012/12/94-asic-choices-update-24th-december.html)

Neighbourhood Pool Watch 5.2 p2Pool: Achieving expectation (http://organofcorti.blogspot.com/2012/11/52-p2pool-achieving-expectations.html) Start here (https://bitcointalk.org/index.php?topic=66026.msg1347445#msg1347445)

Neighbourhood Pool Watch 9.3 More on ASIC choices (http://organofcorti.blogspot.com/2012/11/93-more-on-asic-choices.html)

Neighbourhood Pool Watch 9.2 ASIC choices update 2nd November (http://organofcorti.blogspot.com/2012/11/92-asic-choices-update-2nd-november.html)   Starts here (http://[url=https://bitcointalk.org/index.php?topic=66026.msg1312759#msg1312759)

Neighbourhood Pool Watch 9.1 ASIC choices (http://organofcorti.blogspot.com/2012/11/91-asic-choices.html)   Starts here (http://[url=https://bitcointalk.org/index.php?topic=66026.msg1310725#msg1310725)

Neighbourhood Pool Watch 8.1 BitcoinPool: Another Bitclockers? (http://organofcorti.blogspot.com/2012/10/81-bitcoinpool-another-bitclockers.html)   Starts here (https://bitcointalk.org/index.php?topic=66026.msg1306759#msg1306759)

Neighbourhood Pool Watch 7.1 Variable pool difficulty (http://organofcorti.blogspot.com/2012/10/71-variable-pool-difficulty.html)  Starts here (https://bitcointalk.org/index.php?topic=66026.msg1262071#msg1262071)

Neighbourhood Pool Watch 4.3 Slush's score method and miner earnings part 2 (http://organofcorti.blogspot.com/2012/09/43-slushs-score-method-and-miner.html)   Starts here (https://bitcointalk.org/index.php?topic=66026.msg1164772#msg1164772)

Neighbourhood Pool Watch 6.1 Bitlc.net - Pool hopping and unfulfilled promise (http://organofcorti.blogspot.com/2012/08/61-bitlcnet-pool-hopping-and.html)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg1136938#msg1136938)

Neighbourhood Pool Watch 4.2 Slush's score method and miner earnings part 1 (http://organofcorti.blogspot.com/2012/08/42-slushs-score-method-and-miner.html)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg1093764#msg1093764)

Neighbourhood Pool Watch 5.1 p2Pool - bad luck or flawed? (http://organofcorti.blogspot.com/2012/06/51-p2pool-bad-luck-or-flawed.html)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg974729#msg974729)

Neighbourhood Pool Watch 4.1 Slush's pool (http://organofcorti.blogspot.com/2012/04/41-slushs-pool.html)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg853421#msg853421)

Neighbourhood Pool Watch 3.2 The risks of SMPPS update (http://organofcorti.blogspot.com/2012/04/32-risks-of-smpps.html)    Update (https://bitcointalk.org/index.php?topic=66026.msg831928#msg853421)

Neighbourhood Pool Watch 3.1 The Rise and Fall of Arsbitcoin.com (http://organofcorti.blogspot.com/2012/03/31-rise-and-fall-of-arsbitcoincom.html)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg823955#msg823955)

Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping' (http://organofcorti.blogspot.com/2012/03/22-deepbit-and-pool-hopping.html)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg802847#msg802847)

Neighbourhood Pool Watch 2.1 Deepbit (http://organofcorti.blogspot.com/2012/03/21-deepbit.html)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg795366#msg795366)

Neighbourhood Pool Watch 1.5: Bitclockers.com summary (http://hoppersden.info/entries/32-Neighbourhood-Pool-Watch-1.5-Bitclockers.com-summary)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg779543#msg779543)

Neighbourhood Pool Watch 1: No-one should mine at Bitclockers (http://hoppersden.info/entries/31-Neighbourhood-Pool-Watch-1-No-one-should-mine-at-Bitclockers)


###################################################################
Neighbourhood Pool Watch 1: No-one should mine at Bitclockers (http://hoppersden.info/entries/31-Neighbourhood-Pool-Watch-1-No-one-should-mine-at-Bitclockers) is the first in a new installment of blogs from me that have nothing to do with strategic mining - er pool hopping, and everything to do with ensuring miners are safe from dishonest pool operators and cowboys.

The first to be covered is Bitclockers.com

I gave Bitclockers.com an opportunity to explain themselves on the bitclockers.com thread (https://bitcointalk.org/index.php?topic=10127.msg760406#msg760406) with no answer to my concerns. So I've decided to post what I know about how they are underpaying their fulltime miners.

From the post:
Quote
if Bitclockers.com are reporting block solve times accurately, they are underpaying their miners by about 21%. Since they have claimed they actually are reporting round lengths correctly, the only conclusion we can draw is that fulltime miners at Bitclockers.com are receiving only 79% of what they should be paid. Who's to know how long they have been doing this?

and
Quote
So here's my first rule in choosing a mining pool:

organofcorti's first rule of Bitcoin mining : Do not mine at a pool that publishes false statistics.

A corollary of this is:

Do not mine at Bitclockers.com until they disclose their actual records.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: RoloTonyBrownTown on February 24, 2012, 09:10:37 AM
Well done Organ.   They've dug themselves into a hole here.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: martychubbs on February 24, 2012, 05:50:34 PM
Well done Organ.   They've dug themselves into a hole here.

Way to Bust 'em...miners "you need to hide your kids,hide your wife and hide your husband cuz they rapen anybody out here"

http://www.youtube.com/watch?v=EzNhaLUT520 (http://www.youtube.com/watch?v=EzNhaLUT520)


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: DiabloD3 on February 24, 2012, 06:33:53 PM
Here's yet another reason why everyone should use p2pool.

Damned straight.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: DeathAndTaxes on February 24, 2012, 06:36:21 PM
Here's yet another reason why everyone should use p2pool.

Damned straight.

Yeah I remember that one block where the p2pool operator faked the stats to steal from us miners, it pissed us off so much that we ...

er wait that never happened, it couldn't happen.  :)


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: V2-V3 on February 26, 2012, 09:11:09 AM
Bitclockers does its best to protect its users, to protect the pool from hoppers bitclockers uses anti hopper protections to mitigate the hoppers bonuses over other miners in the pool

With the Hopping bonus mitigated Hoppers are paid 100% for the shares they mined. NON-Hoppers are not effected and payouts remain 100% of shares mined.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on February 26, 2012, 09:23:22 AM
Bitclockers does its best to protect its users, to protect the pool from hoppers bitclockers uses anti hopper protections to mitigate the hoppers bonuses over other miners in the pool

With the Hopping bonus mitigated Hoppers are paid 100% for the shares they mined. NON-Hoppers are not effected and payouts remain 100% of shares mined.

I'm not interested in your hopper protection - focused explanations. All I ever wanted to know from the start is why (until today edit: and since round 179) you never had a block under 0.3*D. Did you keep some for your self? No. Then if you've been respecting block boundaries you've been underpaying.

I proved without a doubt that you've been misreporting block lengths, which means  you've probably been stealing from your fulltime miners.

How about you publish the actual block lengths from rounds 179 to 504? If you can't do that you should get the scammer label.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Starlightbreaker on February 26, 2012, 09:32:23 AM
Bitclockers does its best to protect its users, to protect the pool from hoppers bitclockers uses anti hopper protections to mitigate the hoppers bonuses over other miners in the pool

With the Hopping bonus mitigated Hoppers are paid 100% for the shares they mined. NON-Hoppers are not effected and payouts remain 100% of shares mined.
then why my average daily income is 1.6-1.8 where i can get around 2.2 to 2.3 on bitlc, when i tried full time on your pool, without hopping?

don't tell me "it's the variance"


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: DiabloD3 on February 26, 2012, 10:38:06 AM
Bitclockers does its best to protect its users, to protect the pool from hoppers bitclockers uses anti hopper protections to mitigate the hoppers bonuses over other miners in the pool

With the Hopping bonus mitigated Hoppers are paid 100% for the shares they mined. NON-Hoppers are not effected and payouts remain 100% of shares mined.
then why my average daily income is 1.6-1.8 where i can get around 2.2 to 2.3 on bitlc, when i tried full time on your pool, without hopping?

don't tell me "it's the variance"

Its the variance.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Starlightbreaker on February 26, 2012, 05:27:26 PM
Bitclockers does its best to protect its users, to protect the pool from hoppers bitclockers uses anti hopper protections to mitigate the hoppers bonuses over other miners in the pool

With the Hopping bonus mitigated Hoppers are paid 100% for the shares they mined. NON-Hoppers are not effected and payouts remain 100% of shares mined.
then why my average daily income is 1.6-1.8 where i can get around 2.2 to 2.3 on bitlc, when i tried full time on your pool, without hopping?

don't tell me "it's the variance"

Its the variance.
for couple of days straight?


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: DiabloD3 on February 26, 2012, 05:41:00 PM
Bitclockers does its best to protect its users, to protect the pool from hoppers bitclockers uses anti hopper protections to mitigate the hoppers bonuses over other miners in the pool

With the Hopping bonus mitigated Hoppers are paid 100% for the shares they mined. NON-Hoppers are not effected and payouts remain 100% of shares mined.
then why my average daily income is 1.6-1.8 where i can get around 2.2 to 2.3 on bitlc, when i tried full time on your pool, without hopping?

don't tell me "it's the variance"

Its the variance.
for couple of days straight?

Deepbit once went like a week without finding a block.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Clipse on February 26, 2012, 05:47:25 PM
Bitclockers does its best to protect its users, to protect the pool from hoppers bitclockers uses anti hopper protections to mitigate the hoppers bonuses over other miners in the pool

With the Hopping bonus mitigated Hoppers are paid 100% for the shares they mined. NON-Hoppers are not effected and payouts remain 100% of shares mined.
then why my average daily income is 1.6-1.8 where i can get around 2.2 to 2.3 on bitlc, when i tried full time on your pool, without hopping?

don't tell me "it's the variance"

Its the variance.
for couple of days straight?

Deepbit once went like a week without finding a block.

When did this happen? Back in the 80s ?


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: DeathAndTaxes on February 26, 2012, 05:49:58 PM
Bitclockers does its best to protect its users, to protect the pool from hoppers bitclockers uses anti hopper protections to mitigate the hoppers bonuses over other miners in the pool

With the Hopping bonus mitigated Hoppers are paid 100% for the shares they mined. NON-Hoppers are not effected and payouts remain 100% of shares mined.
then why my average daily income is 1.6-1.8 where i can get around 2.2 to 2.3 on bitlc, when i tried full time on your pool, without hopping?

don't tell me "it's the variance"

Its the variance.
for couple of days straight?

Deepbit once went like a week without finding a block.

When did this happen? Back in the 80s ?

LOLZ


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: bitlane on February 26, 2012, 07:25:21 PM
Well done Organ.   They've dug themselves into a hole here.

Way to Bust 'em...I discourage all miners to stay away from BC so "you need to hide your kids,hide your wife and hide your husband cuz they rapen anybody out here"

http://www.youtube.com/watch?v=EzNhaLUT520 (http://www.youtube.com/watch?v=EzNhaLUT520)


http://www.youtube.com/watch?v=JFWuuP-cyy4

;)


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: useful_idiot on February 27, 2012, 12:58:37 AM
Thanks for the heads up, I had a sneaking suspicion something has been up for the past while. Bitclockers- take some action to rectify this swiftly and immediately. One miners lose faith an trust in your operations it will forever be an uphill battle to regain this trust. I am not 100% sure there is foul play here, but please address the issue for the community.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: grue on February 27, 2012, 02:03:04 AM
When did this happen? Back in the 80s ?
time travel :o


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on February 27, 2012, 01:04:39 PM
Is there enough evidence to ask for a scammers tag? I have a massive amount of shares that were never paid for:(

How much evidence is required?


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Maged on February 28, 2012, 03:21:33 AM
Is there enough evidence to ask for a scammers tag? I have a massive amount of shares that were never paid for:(

How much evidence is required?
Overwhelming evidence.

Sorry, guys, but I'm just not seeing it.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on February 28, 2012, 06:55:41 AM
Is there enough evidence to ask for a scammers tag? I have a massive amount of shares that were never paid for:(

How much evidence is required?
Overwhelming evidence.

Sorry, guys, but I'm just not seeing it.

Probably my fault for not making it easier to follow - sorry.

1. We have overwhelming evidence that they haven't been reporting the shares in a round accurately. Without any explanation from them, this is bad. They could be hiding anything with this, including hiding short rounds.

2. They have stated that they are accurately reporting rounds as they occur. I provide an explanation here (http://hoppersden.info/entries/31-Neighbourhood-Pool-Watch-1-No-one-should-mine-at-Bitclockers) as to why this means (if they're being truthful about block reporting) they've probably been underpaying their fulltime miners by 21%.

3. Their first round under 0.3xD in over 300 rounds occurred just after I made their obfuscations public. That is unlikely to be a coincidence.

4. I've asked them several times to publish their accurate round lengths which they haven't done. In fact I haven't had any response to any allegation I've made. I have sound mathematical backing behind all of my statements. I'm overstating nothing, if anything I'm being conservative.

I'm aware that I'm probably not covering something that you need to know, but if you can post or pm me your requirements I'll know whether or not I can fulfill them without wasting any more time for either of us.

I don't mine with bitclockers and I don't have a horse in this race.



Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on February 28, 2012, 07:48:18 AM
I do have a horse and can give personal data.

Do you have timestamped shares and payments? Also did you happen to keep your own log of shares submitted (I'm guessing not but thought I'd ask).


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on February 28, 2012, 08:01:34 AM
Sorry, guys, but I'm just not seeing it.

Probably my fault for not making it easier to follow - sorry.

1. We have overwhelming evidence that they haven't been reporting the shares in a round accurately. Without any explanation from them, this is bad. They could be hiding anything with this, including hiding short rounds.

2. They have stated that they are accurately reporting rounds as they occur. I provide an explanation here (http://hoppersden.info/entries/31-Neighbourhood-Pool-Watch-1-No-one-should-mine-at-Bitclockers) as to why this means (if they're being truthful about block reporting) they've probably been underpaying their fulltime miners by 21%.

3. Their first round under 0.3xD in over 300 rounds occurred just after I made their obfuscations public. That is unlikely to be a coincidence.

4. I've asked them several times to publish their accurate round lengths which they haven't done. In fact I haven't had any response to any allegation I've made. I have sound mathematical backing behind all of my statements. I'm overstating nothing, if anything I'm being conservative.

I'm aware that I'm probably not covering something that you need to know, but if you can post or pm me your requirements I'll know whether or not I can fulfill them without wasting any more time for either of us.

I don't mine with bitclockers and I don't have a horse in this race.



Organofcorti I just sent you 1 BTC to support your efforts for Neighbourhood Pool Watch. I’m fucking sick of all of the careless disregard for any type of oversight to the pool system. I’m glad someone is finally doing something about the problem.

The worst thing we can do is invite people here with our great utopian promises of a better way and then steal from them using what should be the proof of work security system of the currency. It’s pathetic!



Thanks for the btc CornedBeefHash - I promise I'll give it a good home.

I agree that there needs to be some sort of pool oversight, but not in the ad-hoc way I've done it. It needs to be some sort of a council made up of pool ops and miners. I discussed this a bit with MobiusL (his idea) and I like it.

Also, don't forget we put Maged in a hard position here. In other cases of scamming it's usually clear cut and the evidence doesn't require a knowledge of statistical probability in order to follow it.

Assessing whether or not a pool has been acting in poor faith is much harder. Maged must also realise that this is a test case and probably doesn't want a situation where at any time in the future a miner gets pissed at a pool he can refer back to this and cause unnecessary problems.

I'm happy to go along with the process since I'm sure I'll be able to supply whatever proofs I need to.

Cheers again for the support - always good to hear.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on February 28, 2012, 08:37:30 AM
I do have a horse and can give personal data.

Do you have timestamped shares and payments? Also did you happen to keep your own log of shares submitted (I'm guessing not but thought I'd ask).

PM sent with info, 5BTC also sent thanks for your awesome work, more info will be sent to you as soon as I get it. Thanks.


Thanks for the support Chaang - a few more btc and I'll make it to Tongsai Bay ;)

More data? You know me so well.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Graet on February 28, 2012, 08:43:04 AM


Organofcorti I just sent you 1 BTC to support your efforts for Neighbourhood Pool Watch. I’m fucking sick of all of the careless disregard for any type of oversight to the pool system. I’m glad someone is finally doing something about the problem.

The worst thing we can do is invite people here with our great utopian promises of a better way and then steal from them using what should be the proof of work security system of the currency. It’s pathetic!

+1
sent 2btc


I had also suggesting something like to some others a while back "this I agree that there needs to be some sort of pool oversight, but not in the ad-hoc way I've done it. It needs to be some sort of a council made up of pool ops and miners. I discussed this a bit with MobiusL (his idea) and I like it." It is a good idea and I would be fully supportive - just *how* to audit pools might cause discussion ;)

I started a thread so poolops could state what they do with miners shares, it was pretty fail as most didnt post then it was taken offtopic by people that didnt understand what it was for.... I thought it could be a handy quick reference :/

Keep up the good work organofcorti :)

Cheers
Graeme




Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on February 28, 2012, 10:42:56 AM
+1
sent 2btc
Thanks for the support, Graeme. BTC helps me justify my 'hobby' to my SOP when I miss dinner with her parents ;)
Quote
I had also suggesting something like to some others a while back "this I agree that there needs to be some sort of pool oversight, but not in the ad-hoc way I've done it. It needs to be some sort of a council made up of pool ops and miners. I discussed this a bit with MobiusL (his idea) and I like it." It is a good idea and I would be fully supportive - just *how* to audit pools might cause discussion ;)
Published round lengths can be a good start. I have no idea how you'd then audit payments.

Quote
I started a thread so poolops could state what they do with miners shares, it was pretty fail as most didnt post then it was taken offtopic by people that didnt understand what it was for.... I thought it could be a handy quick reference :/


I was kind of hoping your thread (https://bitcointalk.org/index.php?topic=64400.0) would get more substance to it, but maybe some pool ops aren't just using hashes the way we think they are and aren't ready to talk about it yet - apart from Goat that is.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Gabi on February 28, 2012, 01:08:01 PM
Just move to p2p, problem solved  :P


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Starlightbreaker on February 28, 2012, 05:02:29 PM
unhoppable. :P


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Maged on February 29, 2012, 01:35:43 AM
Probably my fault for not making it easier to follow - sorry.

1. We have overwhelming evidence that they haven't been reporting the shares in a round accurately. Without any explanation from them, this is bad. They could be hiding anything with this, including hiding short rounds.
I agree. It is quite clear that the reporting isn't accurate.

2. They have stated that they are accurately reporting rounds as they occur. I provide an explanation here (http://hoppersden.info/entries/31-Neighbourhood-Pool-Watch-1-No-one-should-mine-at-Bitclockers) as to why this means (if they're being truthful about block reporting) they've probably been underpaying their fulltime miners by 21%.
They are likely lying. However, lying doesn't get people a scammer tag. It is only a problem when they lie to make extra money. However, I'm not seeing any evidence of that. In your link, you only explore case 1. What if, however, the reality was case 2? Also, since their statistics are delayed, what's stopping it from being a hybrid of truth and fiction? If a round is too short, they could pretend that it never happened. If it's too long, they can say it ended early. Done properly, this would equal out over time.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on February 29, 2012, 12:57:44 PM
They are likely lying. However, lying doesn't get people a scammer tag. It is only a problem when they lie to make extra money. However, I'm not seeing any evidence of that. In your link, you only explore case 1. What if, however, the reality was case 2? Also, since their statistics are delayed, what's stopping it from being a hybrid of truth and fiction? If a round is too short, they could pretend that it never happened. If it's too long, they can say it ended early. Done properly, this would equal out over time.

I had thought that accepting they were lying about accurately reporting blocks solved was more inflammatory than accepting they were telling the truth, so I covered what they said was correct. Hoist by their own petard, etc.

If they are lying about reporting block lengths they may be using a simple sort of averaging algorithm as you suggest, but as I discuss here (http://hoppersden.info/entries/29-How-to-hop-11-Bitclockers.com-amp-spurious-round-lengths-part-2) simple averaging algorithms don't match the histograms of Bitclockers.com round length data.

A much more complex algorithm that matches the Bitclockers.com round lengths frequency distributions are possible, but a much simpler and equivalent approach is to use a function on all rounds, not just shorter and longer ones.

I identified several transforms that match the empirical cdf and histograms of the bitclockers.com round length data and discussed them briefly in Neighbourhood Pool Watch1 (http://hoppersden.info/entries/31-Neighbourhood-Pool-Watch-1-No-one-should-mine-at-Bitclockers).

So, assuming they are lying to their miners about when (and possibly if) blocks are solved, we have at least two options to investigate:

1. Bitclockers.com are lying to their miners about when blocks are solved but not if blocks are solved.
In this case, they are faithfully recording all shares submitted and changing block lengths to suit themselves. In this case it is simple to show empirically that strategic miners are being underpaid. If this seems to you to be ok, keep in mind that they claim to be a proportional pool. Strategic miners at a proportional pool can expect a certain return on their shares. If they don't receive the expected amount over time, then the pool is not paying out proportionally.

Their "About us" page doesn't mention anything about "hopping protection", just that the pool is a proportional pool. From their website:
Quote
" ….by pooling your resources with the other users of BitClockers we can generate blocks much faster and thus pay you a proportionate amount for the work contributed by your video cards."
Quote
"The miners complete their shares and return the work back to the BitClockers Pool Server. The Pool then checks to make sure that the work is valid and credits the miner for its work. Once the entire block is finished the Pool looks at everyones share of the work and gives out 49 Bitcoins proportionally."
Quote
"All the features you'll expect a bitcoin pool to have. Works in proportional: You get a part of every solved block proportional to your part in pool hashing rate."

In these quotes they are certainly lying about how they remunerate any miner.

Neither did their threads here, or on their forum (as far as I can tell) mention this until I posted How to hop 10 (http://hoppersden.info/entries/28-How-to-hop-10-Bitclockers.com-amp-spurious-round-lengths-part-1). If strategic miners have been underpaid, then they have been lying to make extra money. Whether or not they redistribute that extra money to other miners is a moot point - if we can prove that some miners are targeted for underpayment with no obvious warning, then we can show that they lied for extra money. In the words of backburn (a bitclockers.com pool op):

If you think its OK to hop pools, than its OK for the pool to hop hoppers.

How can we show that they are underpaying strategic miners without relying on a particular transform or the Bitclockers.com pool op's say so? Well, can use their own data to show the difference between expected share values on Bitclockers.com and a real proportional pool. (http://hoppersden.info/entries/29-How-to-hop-11-Bitclockers.com-amp-spurious-round-lengths-part-2) But since I already did that how about looking at the difference between the expected payout per round for Bitclockers.com compared to a real proportional pool?

The graphs below show how the expected share values and expected earnings vary for bitclockers.com and an actual proportional pool. This data is based on the round lengths published for rounds 180 to 505. The top graph I've published before. The lower graph shows the expected amount a miner can earn by leaving a pool at a particular number of shares. As can be clearly seen, if a strategic miner leaves Bitclockers.com at 0.43xD (the point at which the expected share value drops to zero on a proportional pool) the strategic miner will earn about 0.6 less than expected. So for every 111 btc a strategic miner earns, he or she should have earned 170 btc (not including real world inefficiencies).

http://i.imgur.com/KxC5b.png



2. Bitclockers.com are lying to their miners about if and when blocks are solved.
Part 1. partly covers this. In addition not paying strategic miners their expected reward, Bitclockers.com may be withholding btc from all miners, for example from the very short rounds that they don't report. The possibility can be assessed fairly simply by comparing the distribution of means for bitclockers with other generated random data from distributions that have a similar density plot and cdf to the Bitclockers.com round lengths. See Neighbourhood Pool Watch1 (http://hoppersden.info/entries/31-Neighbourhood-Pool-Watch-1-No-one-should-mine-at-Bitclockers) part 1 for details.

Firstly, what does this bitclockers.com data say? The Bitclockers.com mean round length for rounds 180 to 505 is 1.0281. It took 325*1.0281=334.1 rounds of shares to generate 325 bitcoin rewards. That means that they could have taken 455 btc and we'd be none the wiser.

How likely is this to have happened?

Using the three most likely distributions and this R code (http://pastebin.com/qnVTX2Cr) I generated a million sets of a million generated random numbers for each of the three distributions. I then generated a million means from each set, and found their means of the means and the proportion of them below or equal to D.

From these sets I obtained the following:
Code:
mean(r.mat$rlnorm)
[1] 1.020986
> mean(r.mat$rllogis)
[1] 1.027112
> mean(r.mat$rgamma)
[1] 1.021985
Code:
length(subset(r.mat$rlnorm,r.mat$rlnorm<=1))/length(r.mat$rlnorm)
#[1] 0.1914
length(subset(r.mat$rllogis,r.mat$rllogis<=1))/length(r.mat$rllogis)
#[1] 0.1530
length(subset(r.mat$rgamma,r.mat$rgamma<=1))/length(r.mat$rgamma)
#[1] 0.1682

The means are very close to that for bitclockers.com, and the second group of results show that for round length distributions similar to Bitclockers.com's own, only 0.19 are at D (the bitcoin Difficulty). This means that the for these sets, the mean is much less likely to be D than it is to be larger than D. The very method of generating their fake data - whatever it is - allows a mean to be greater than D 71% of the time. This could easily include hiding btc. Rounds 180 to 505 are Sept 22nd to Feb 17th. So that's an extra bonus to the pool ops of about 90 btc/month in addition to the 2% tax. A nice little earner.

In conclusion, I can only prove malicious lying to gain coins in the first case.

But the second case is also important, and on the balance of probabilities could easily be true. Any time a pool does not disclose round statistics, it is too easy to 'fudge' the books and keep the a short round for themselves.

I hope this clears up any doubt for you.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Starlightbreaker on February 29, 2012, 03:43:56 PM
good job.

well, that means them fuckers have some 'splainin and payment to all of whoever mines there to do.

pitchforks, anyone?


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Maged on February 29, 2012, 05:42:17 PM
organofcorti, your problem is that you're assuming that the actual expected round length is supposed to be 1.0281 D. Instead, assume that it's 1.0 D. What is the probability that the measured round lengths would average to 1.0281 D by chance?

Also, I'd like to see some graphs that compare them to the other types of payout methods. The fact the they lied about being proportional means nothing to me. Any issue regarding that should be handled by a local board mod, such as DiabloD3. I'm just doing a scammer investigation, and so far, all I'm seeing is you bending statistics to give the results you're looking for - a common issue in the field of statistics.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: farfiman on March 01, 2012, 05:41:52 AM
All the evidence proves that HOPPERS are earning less than they expect?

Are the "regular" miners earning less?? If so why would they stay there?

I'm not saying that It's OK to play around with the statistics even if the cause
is OK( In their eyes). Rules are rules.  But I guess the non-hopping miners
don't care as long as they get the share of the pie they expect (as if there
were no hoppers)


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 01, 2012, 06:10:16 AM
organofcorti, your problem is that you're assuming that the actual expected round length is supposed to be 1.0281 D. Instead, assume that it's 1.0 D. What is the probability that the measured round lengths would average to 1.0281 D by chance?
You'd like to assume the population mean to be 1xD and estimate the probability of the round lengths averaging to 1.0281? Using a similar technique to that described earlier (using code similar to this) (http://pastebin.com/qnVTX2Cr) we get a probability of 0.8801 rounds being less than 1.0281, giving a probability of rounds being 1.0281 or longer at 1-0.8801=0.1199. This is even worse than the estimate I provided earlier.
Quote
Also, I'd like to see some graphs that compare them to the other types of payout methods.
I'm happy to do that. What sort of graphs did you mean?
Quote
The fact the they lied about being proportional means nothing to me. Any issue regarding that should be handled by a local board mod, such as DiabloD3.
It should, since they agreed to pay a certain amount to miners, and then did not pay them that amount.
Quote
I'm just doing a scammer investigation, and so far, all I'm seeing is you bending statistics to give the results you're looking for - a common issue in the field of statistics.
If you'd like to provide examples I hope I'll be able to explain why I wasn't bending statistics and regain your trust.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 01, 2012, 06:17:53 AM
What I would like to see is a BOLD PRINT sticky thread at the top of the pools subsection with a rating system based on several criteria. The ratings can include the ability to hop that pool, possible misdeeds of the pool operator that can lower the rating and the names of pools and operators that have actually scammed (like A1).

This is a seriously good idea CBH. I think it would be extra hard works for the mods though. A council made up of respected bitcointalk forum members and poolops might be better doing the ground work, with  DiabloD3, gmaxwell, and MiningBuddy having the last say before anything is posted. I can see lots of disagreements arising. You'd need to have a cumulative rating the way they do in olympic gymnastics from various judges, i think.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 01, 2012, 06:20:07 AM
All the evidence proves that HOPPERS are earning less than they expect?

Are the "regular" miners earning less?? If so why would they stay there?

I'm not saying that It's OK to play around with the statistics even if the cause
is OK( In their eyes). Rules are rules.  But I guess the non-hopping miners
don't care as long as they get the share of the pie they expect (as if there
were no hoppers)

What it comes down to is that they agreed to pay miners a certain amount and didn't.



Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Maged on March 01, 2012, 06:55:03 AM
organofcorti, your problem is that you're assuming that the actual expected round length is supposed to be 1.0281 D. Instead, assume that it's 1.0 D. What is the probability that the measured round lengths would average to 1.0281 D by chance?
You'd like to assume the population mean to be 1xD and estimate the probability of the round lengths averaging to 1.0281? Using a similar technique to that described earlier (using code similar to this) (http://pastebin.com/qnVTX2Cr) we get a probability of 0.8801 rounds being less than 1.0281, giving a probability of rounds being 1.0281 or longer at 1-0.8801=0.1199. This is even worse than the estimate I provided earlier.
My statistics are a little rusty, but isn't that result not statistically significant?


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Starlightbreaker on March 01, 2012, 06:58:27 AM
organofcorti, your problem is that you're assuming that the actual expected round length is supposed to be 1.0281 D. Instead, assume that it's 1.0 D. What is the probability that the measured round lengths would average to 1.0281 D by chance?
You'd like to assume the population mean to be 1xD and estimate the probability of the round lengths averaging to 1.0281? Using a similar technique to that described earlier (using code similar to this) (http://pastebin.com/qnVTX2Cr) we get a probability of 0.8801 rounds being less than 1.0281, giving a probability of rounds being 1.0281 or longer at 1-0.8801=0.1199. This is even worse than the estimate I provided earlier.
My statistics are a little rusty, but isn't that result not statistically significant?
i don't think we're testing a hypothesis here. it's just a probability.

...unless organofcorti wants to do a series of real-time test with a similar sized prop pool.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 01, 2012, 07:02:53 AM
organofcorti, your problem is that you're assuming that the actual expected round length is supposed to be 1.0281 D. Instead, assume that it's 1.0 D. What is the probability that the measured round lengths would average to 1.0281 D by chance?
You'd like to assume the population mean to be 1xD and estimate the probability of the round lengths averaging to 1.0281? Using a similar technique to that described earlier (using code similar to this) (http://pastebin.com/qnVTX2Cr) we get a probability of 0.8801 rounds being less than 1.0281, giving a probability of rounds being 1.0281 or longer at 1-0.8801=0.1199. This is even worse than the estimate I provided earlier.
My statistics are a little rusty, but isn't that result not statistically significant?

It's not a p-value, but even so all it means is that you would expect round lengths means of this size to occur once in every nine sets of 325 rounds. You have to decide whether or not that's significant. As a final remark on this I wrote:

In conclusion, I can only prove malicious lying to gain coins in the first case.

But the second case is also important, and on the balance of probabilities could easily be true. Any time a pool does not disclose round statistics, it is too easy to 'fudge' the books and keep the a short round for themselves.
(the second case being what we just discussed)
So I wasn't claiming to prove Bitclockers.com were steaing blocks, just that it was more possible than for pools that post accurate records of round lengths. ( various edits because of my stupidity and feeling under the weather)


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 01, 2012, 07:05:53 AM
organofcorti, your problem is that you're assuming that the actual expected round length is supposed to be 1.0281 D. Instead, assume that it's 1.0 D. What is the probability that the measured round lengths would average to 1.0281 D by chance?
You'd like to assume the population mean to be 1xD and estimate the probability of the round lengths averaging to 1.0281? Using a similar technique to that described earlier (using code similar to this) (http://pastebin.com/qnVTX2Cr) we get a probability of 0.8801 rounds being less than 1.0281, giving a probability of rounds being 1.0281 or longer at 1-0.8801=0.1199. This is even worse than the estimate I provided earlier.
My statistics are a little rusty, but isn't that result not statistically significant?
i don't think we're testing a hypothesis here. it's just a probability.
yes, it's just based on the cdf.
Quote

...unless organofcorti wants to do a series of real-time test with a similar sized prop pool.

Can't do that either. If we assume the population mean is 1.0*D, we have to assume the distribution is similar to the Bitclockers.com distribution. Can't compare a geometrically distributed round with a bitclockers.com round - they aren't similar.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Starlightbreaker on March 01, 2012, 07:13:52 AM
gah.
didn't think of that.

which pretty much means, there's no way to test it, with only based on data gathered from bitclockers.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 01, 2012, 07:21:50 AM
gah.
didn't think of that.

which pretty much means, there's no way to test it, with only based on data gathered from bitclockers.


That's right. You can just make assumptions and do backtesting.

I'd like to make this plain: now that I have a bit more of an idea about what the 'scammer' tag means, the second case can't make any claims about scamming. It just makes clear that it is much more possible for Bitclockers.com to steal blocks, and that results so far do nothing to disprove the possibility.

The first case (that they have agreed to pay miners a certain amount and did not pay that amount) is really the only situation where lying to wrongfully obtain funds can be proven.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Starlightbreaker on March 01, 2012, 07:29:34 AM
so burden of proof now is on bitclockers. unless they prove they don't do anything shady, which i doubt, they have some explaining to do.

so far, they haven't really deny anything. only vague statements.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 01, 2012, 07:33:56 AM
so burden of proof now is on bitclockers. unless they prove they don't do anything shady, which i doubt, they have some explaining to do.
I think the burden of proof should be on every pool. It's just that what Bitclockers.com does is so obvious.
Quote
so far, they haven't really deny anything. only vague statements.
I know and it's frustrating. I was hoping to get them to open up and explain what they did and why. So far they haven't even acknowledged faking round length stats.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Starlightbreaker on March 01, 2012, 07:36:44 AM
send them an email.



....ohwait, they're not gonna reply, lolol.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: farfiman on March 01, 2012, 07:47:50 AM
All the evidence proves that HOPPERS are earning less than they expect?

Are the "regular" miners earning less?? If so why would they stay there?

I'm not saying that It's OK to play around with the statistics even if the cause
is OK( In their eyes). Rules are rules.  But I guess the non-hopping miners
don't care as long as they get the share of the pie they expect (as if there
were no hoppers)

What it comes down to is that they agreed to pay miners a certain amount and didn't.



So the answer to my question is yes.... hoppers only are hurt apparently


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Starlightbreaker on March 01, 2012, 07:50:25 AM
All the evidence proves that HOPPERS are earning less than they expect?

Are the "regular" miners earning less?? If so why would they stay there?

I'm not saying that It's OK to play around with the statistics even if the cause
is OK( In their eyes). Rules are rules.  But I guess the non-hopping miners
don't care as long as they get the share of the pie they expect (as if there
were no hoppers)

What it comes down to is that they agreed to pay miners a certain amount and didn't.



So the answer to my question is yes.... hoppers only are hurt apparently
what?

no.
both full-time miners and hoppers, to a degree.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 01, 2012, 07:58:27 AM
All the evidence proves that HOPPERS are earning less than they expect?

Are the "regular" miners earning less?? If so why would they stay there?

I'm not saying that It's OK to play around with the statistics even if the cause
is OK( In their eyes). Rules are rules.  But I guess the non-hopping miners
don't care as long as they get the share of the pie they expect (as if there
were no hoppers)

What it comes down to is that they agreed to pay miners a certain amount and didn't.



So the answer to my question is yes.... hoppers only are hurt apparently

Not quite. According to the promises of the Bitclockers payment method, the case for underpaying all miners on short rounds is the only one that can be proven. Would you be happy if a PPLNS pool you mined at decided to arbitrarily not pay some of your earnings?

But everything else i've written stands as is. I might not be able to prove Bitclockers are stealing blocks, just that it can't be disproven (from the data we have), and in terms of probability appears more likely than not. It would be easy for Bitclockers.com to disprove this, but they don't seem to want to. I hope it's not because they are unable to.






Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: farfiman on March 01, 2012, 08:06:58 AM


Not quite. According to the promises of the Bitclockers payment method, the case for underpaying all miners on short rounds is the only one that can be proven. Would you be happy if a PPLNS pool you mined at decided to arbitrarily not pay some of your earnings?

But everything else i've written stands as is. I might not be able to prove Bitclockers are stealing blocks, just that it can't be disproven (from the data we have), and in terms of probability appears more likely than not. It would be easy for Bitclockers.com to disprove this, but they don't seem to want to. I hope it's not because they are unable to.



ok,  I'll revise my statement..  hoppers are hurt more than others ( because they usually take a bigger amount of the pie)

 


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 01, 2012, 08:24:39 AM
It will never be possible to mine with some pools and be secure in the knowledge that you are not being robbed of your rightful share. You are spending your associated mining costs on the promise of a return that can be at the discretion of that pool and there isn’t a damn thing you can do about it.

I guess you should feel good if they give you anything!


So, this is probably a good time to start this:

Pools I have confidence in and would mine at (in no particular order)
  • Slush's pool (once they're on whatever hop proof payment method Slush decides on)
  • Ozcoin
  • Eclipse
Why Slush? I 've investigated them more than any other pool. I'd have noticed anything odd.
and generally, all three pools provide stats and are totally open. I also have a significant amount of trust for the pool ops of Ozcoin and Eclipse.

This doesn't mean I don't trust other pools. I wouldn't mine at deepbit for example because I don't want it larger than it is and because I don't need PPS because variance doesn't scare me. BTCGuild is also a fee-pay PPS. Bitlc is out for reasons that date back to multipool (ok so I can hold a grudge). Eligius and Ars are out because they're SMPPS.

I think that covers the main pools.

Thoughts?


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 01, 2012, 08:37:52 AM


Not quite. According to the promises of the Bitclockers payment method, the case for underpaying all miners on short rounds is the only one that can be proven. Would you be happy if a PPLNS pool you mined at decided to arbitrarily not pay some of your earnings?

But everything else i've written stands as is. I might not be able to prove Bitclockers are stealing blocks, just that it can't be disproven (from the data we have), and in terms of probability appears more likely than not. It would be easy for Bitclockers.com to disprove this, but they don't seem to want to. I hope it's not because they are unable to.



ok,  I'll revise my statement..  hoppers are hurt more than others ( because they usually take a bigger amount of the pie)

There is no pie. There is only bitcoin. There could be bitcoin pie though, I suppose. It would taste like victory!

To recap:

Everyone loses on short rounds, but strategic miners provably lose more than fulltime miners since strategic miners only stay for short rounds.

I don't know how much or even if fulltime miners' expected payouts are affected. It is possible or even probable that bitclockers have been withholding blocks rewards, but not provable because Bitclockers withhold real stats and produce fake stats.



Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: farfiman on March 01, 2012, 11:02:43 AM
If the tactics bitclockers used is not worth a scammers tag I will very soon take their methods to the lending area.

If it is okay for shares it should be okay for real money too. :/



We have enough of that in the lending section already.....

edit:

It seems to me that only the hoppers are mad because its bad for their system. I can understand that but
I guess the majority of the users have no problem with the "hidden" algorithm they use.

Just to be fair I admit that i have a "thing" for hopping and not a good one. When I first started mining I didn't have a clue
and was even happy when the hoppers  came to "help " on the small pools I used.   When I learned the truth it pissed me off.
Don't want to start another discussion on this since there have been so many already.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Mobius on March 01, 2012, 12:02:01 PM
If the tactics bitclockers used is not worth a scammers tag I will very soon take their methods to the lending area.

If it is okay for shares it should be okay for real money too. :/



We have enough of that in the lending section already.....

edit:

It seems to me that only the hoppers are mad because its bad for their system. I can understand that but
I guess the majority of the users have no problem with the "hidden" algorithm they use.

Just to be fair I admit that i have a "thing" for hopping and not a good one. When I first started mining I didn't have a clue
and was even happy when the hoppers  came to "help " on the small pools I used.   When I learned the truth it pissed me off.
Don't want to start another discussion on this since there have been so many already.

Who made you the spokesman, they owe all their miners 21% more than they paid. When they pay that out we can re-evaluate their position. Until then they deserve the "scammer tag" and all should avoid them.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: farfiman on March 01, 2012, 12:18:46 PM
If the tactics bitclockers used is not worth a scammers tag I will very soon take their methods to the lending area.

If it is okay for shares it should be okay for real money too. :/



We have enough of that in the lending section already.....

edit:

It seems to me that only the hoppers are mad because its bad for their system. I can understand that but
I guess the majority of the users have no problem with the "hidden" algorithm they use.

Just to be fair I admit that i have a "thing" for hopping and not a good one. When I first started mining I didn't have a clue
and was even happy when the hoppers  came to "help " on the small pools I used.   When I learned the truth it pissed me off.
Don't want to start another discussion on this since there have been so many already.

Who made you the spokesman, they owe all their miners 21% more than they paid. When they pay that out we can re-evaluate their position. Until then they deserve the "scammer tag" and all should avoid them.

I'm not anybody's spokesman and never even mined there.  All I'm saying is if they were paying 21% less to everybody then nobody would stay there.  The pools owner admits "not condoning hopping" etc etc.. so they took the law ( or the rules) and did whatever they did. I didn't say even once that it's right - just that it does protect somewhat  the full-time miners from the hopping. If the Pool owners ever come clean with the info we shall see.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Mobius on March 01, 2012, 01:01:06 PM
If the tactics bitclockers used is not worth a scammers tag I will very soon take their methods to the lending area.

If it is okay for shares it should be okay for real money too. :/



We have enough of that in the lending section already.....

edit:

It seems to me that only the hoppers are mad because its bad for their system. I can understand that but
I guess the majority of the users have no problem with the "hidden" algorithm they use.

Just to be fair I admit that i have a "thing" for hopping and not a good one. When I first started mining I didn't have a clue
and was even happy when the hoppers  came to "help " on the small pools I used.   When I learned the truth it pissed me off.
Don't want to start another discussion on this since there have been so many already.

Who made you the spokesman, they owe all their miners 21% more than they paid. When they pay that out we can re-evaluate their position. Until then they deserve the "scammer tag" and all should avoid them.

I'm not anybody's spokesman and never even mined there.  All I'm saying is if they were paying 21% less to everybody then nobody would stay there.  They Pools owner admits "not condoning hopping" etc etc.. so they took the law ( or the rules) and did whatever they did. I didn't say even once that it's right - just that it does protect somewhat  the full-time miners from the hopping. If the Pool owners ever come clean with the info we shall see.


It's been empirically shown that they have been lying and faking stats, the current analysis implicates them as having shorted ALL their miners (full time) at least 21%. This thread is not about hopping, it's about their lying and stealing from their miners and using hopping "prevention" as an excuse.

If you never mined there, why get involved in the conversation unless you have an alternative motive and/or are trying to promote your own individual agenda.

People continued to mine there because they started as legitimate pool, then they degraded to cheating their miners. Even now they are trying to rehabilitate their stats and information because they were CAUGHT. Their lack of significant response in IRC or in the forums indicates (in my opinion) their hope it will "blow over" and they can continue.

This is the problem with having "pools" and "exchanges" being unverified. People who mined at bitclockers will probably never get the funds that they were cheated/shortchanged. Yet there is no process to hold them accountable.

The principles of Bitclockers should go to jail for fraud, that will not happen at this point in time,  yet until the bitcoin community starts to enforce penalties against this cheating, or set standards, we will be looked upon as an amateur.

If we look at how exchanges now want us (miners) to ID ourselves, why should we not expect the same of exchanges and pools. Who are they? Should they be registered as a business, should they post a bond against fraud. Should there be some oversight council. We need to police ourselves before it is imposed on us by those who have no clue.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: farfiman on March 01, 2012, 01:12:00 PM
I have no agendas and again , never said its OK what they are doing.

I did say I have a "thing" about hopping and in this case it seems the
hoppers are pissed,  at least more than other miners.

So, maybe I commented on the subject  because of that...


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Mobius on March 01, 2012, 01:30:11 PM
I have no agendas and again , never said its OK what they are doing.

I did say I have a "thing" about hopping and in this case it seems the
hoppers are pissed,  at least more than other miners.

So, maybe I commented on the subject  because of that...


Whether you have a thing against hoppers or not, does not justify your interference, take that up in another thread,, I had bitcloakers as my backup, I was screwed by 21%. So if you have an issue against hoppers, do it in your own thread. This is about Bitcloaker, STEALING and LYING. Unless you have something productive (actual statistical evidence) go post in another thread and stop obscuring the purpose of this thread for your own ego.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Graet on March 01, 2012, 01:34:17 PM


This is the problem with having "pools" and "exchanges" being unverified.
Interesting, about 6 months ago I mentioned somewhere about miners trusting anonomous nicks with no other info :) - and was howled down in the community. oh well :)

If we look at how exchanges now want us (miners) to ID ourselves, why should we not expect the same of exchanges and pools. Who are they? Should they be registered as a business, should they post a bond against fraud. Should there be some oversight council. We need to police ourselves before it is imposed on us by those who have no clue.
Those of us that are here for the longterm I think are taking steps towards this, I know of at least one other pool that is a registered buisness...

Post a bond against fraud: hmm interesting - will you insure pools and exchanges against hacking (i know entirely different subject)... but lets be a bit real :)

There should be a group of  people overseeing this yes, it has been mentioned before in this thread and others.

Actually if people took more responsibility for themselves and did research on pools or exchanges before they use them would be less need for policing.

my 2s
Graeme Tee
Graet
Ozcoin Pooled Mining Pty Ltd
ACN: 152 509 272
^^^ (this info has been in the 1st post of our pool thread for about 7months)


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Mobius on March 01, 2012, 01:44:58 PM


This is the problem with having "pools" and "exchanges" being unverified.
Interesting, about 6 months ago I mentioned somewhere about miners trusting anonomous nicks with no other info :) - and was howled down in the community. oh well :)

If we look at how exchanges now want us (miners) to ID ourselves, why should we not expect the same of exchanges and pools. Who are they? Should they be registered as a business, should they post a bond against fraud. Should there be some oversight council. We need to police ourselves before it is imposed on us by those who have no clue.
Those of us that are here for the longterm I think are taking steps towards this, I know of at least one other pool that is a registered buisness...

Post a bond against fraud: hmm interesting - will you insure pools and exchanges against hacking (i know entirely different subject)... but lets be a bit real :)

There should be a group of  people overseeing this yes, it has been mentioned before in this thread and others.

Actually if people took more responsibility for themselves and did research on pools or exchanges before they use them would be less need for policing.

my 2s
Graeme Tee
Graet
Ozcoin Pooled Mining Pty Ltd
ACN: 152 509 272
^^^ (this info has been in the 1st post of our pool thread for about 7months)


Garet,
You are one of the few pools that I actually have full respect for, you have been completely open with your pool and it's progression.  The bond would be against the exchange/pool cheating people against their monetary investment. Hacking of a pool can be be covered by insurance against hacking(cost of business). Unfortunately, if we don't police our own neck of the woods,legislation will.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: farfiman on March 01, 2012, 02:32:21 PM
I have no agendas and again , never said its OK what they are doing.

I did say I have a "thing" about hopping and in this case it seems the
hoppers are pissed,  at least more than other miners.

So, maybe I commented on the subject  because of that...


I hop and I do have money involved in this however I would be just as upset and vocal even if I did not.

Organ has no money in this and has done all of the work and proved these guys are scammers.

Having this sort of thing in our community is bad and once it is exposed should be dealt with. I really do not think we should support this sort of thing or it will just spread.

anyway p2p for the win.

I agree - there is far too much bad going on in the forum and mining.   My comments weren't personal against you or any hopper -  was just - in my eyes- a bit of poetic justice...


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: martychubbs on March 01, 2012, 04:55:23 PM
BC went down, now that it is up, organ think anything has changed?

So, you think they slowed the initial hash rate and share count to have hoppers stay longer and/or delaying block starts?  So, would that buffer be spread out over the "block" (fake) or just kept by the op?  I can see you can hop it but it's like a even worse coinotron...get killed on the short rounds...not a good mix with other prop pools...

Also, did BC just have a short round? 


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Transisto on March 01, 2012, 05:21:04 PM
1.111~ sent,

Unfortunate that shady hopping countermeasure are being used to fake stats/payout,  when open and effective solution already exist.

Ps: statistically speaking,,, Some pool that charge 10% for PPS and 3% for prop..  They make it sound as if running a prop pool with 4Ths was a risky business to the operator. , it's not.  ... Just saying... see sig.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Starlightbreaker on March 01, 2012, 06:29:11 PM
BC went down, now that it is up, organ think anything has changed?

So, you think they slowed the initial hash rate and share count to have hoppers stay longer and/or delaying block starts?  So, would that buffer be spread out over the "block" (fake) or just kept by the op?  I can see you can hop it but it's like a even worse coinotron...get killed on the short rounds...not a good mix with other prop pools...

Also, did BC just have a short round?  
huh.

weird.

my daily payment is close to double what i normally get now.  ???
must be the short blocks.



....wait, they have two short blocks?


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: martychubbs on March 01, 2012, 08:39:19 PM
BC went down, now that it is up, organ think anything has changed?

So, you think they slowed the initial hash rate and share count to have hoppers stay longer and/or delaying block starts?  So, would that buffer be spread out over the "block" (fake) or just kept by the op?  I can see you can hop it but it's like a even worse coinotron...get killed on the short rounds...not a good mix with other prop pools...

Also, did BC just have a short round?  
huh.

weird.

my daily payment is close to double what i normally get now.  ???
must be the short blocks.



....wait, they have two short blocks?

I know, hope they're back to normal


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: DeathAndTaxes on March 01, 2012, 09:37:15 PM
....wait, they have two short blocks?

Yeah they got busted stealing miner's coins so they had to stop taking short blocks as profit. 


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: P4man on March 01, 2012, 10:14:30 PM
Pools I have confidence in and would mine at (in no particular order)
  • Slush's pool (once they're on whatever hop proof payment method Slush decides on)
  • Ozcoin
  • Eclipse
Why Slush? I 've investigated them more than any other pool. I'd have noticed anything odd.

Id certainly add bitminter to that list, and I have one concern with Slush; its not major, and I have no reason to distrust Slush, but I sure would like to see stats of shares submitted there. Must be the only pool that doesnt show it (expect for running round).

As for bitclockers, best I can tell their stats are still as fake as ever.
Maybe they thought faking a few short rounds would somehow be enough to disprove the mountain of evidence against them.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Starlightbreaker on March 01, 2012, 11:08:51 PM
....wait, they have two short blocks?

Yeah they got busted stealing miner's coins so they had to stop taking short blocks as profit. 
now they better give me back my coin they stole.
...and everybody elses.


i can safely assume we caught them red-handed.

it's interesting when your payout suddenly doubles after people call'em out.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 02, 2012, 01:41:07 PM
It was great to watch two forums members that I respect have at each other in such a reasoned manner. Discussions of that sort don't take place on bitcointalk.org often enough. However I realised with some information over in hoppersden.info and some here farfiman and Mobius each only had part of the information available.

With that in mind I've posted Neighbourhood Pool Watch 1.5: Bitclockers.com summary (http://hoppersden.info/entries/32-Neighbourhood-Pool-Watch-1.5-Bitclockers.com-summary). In it I reiterate what I've shown in both NHPW1 and here a couple of pages back (edit: make that 3 pages back) in one hopefully cogent post (I write 'hopefully' because I rushed it out before knives were drawn here, plus I had to cut bits out to get under the 10000 character limit.)

For those who can't be bothered reading the whole post, here's the tl;dr:
Quote
0. Introduction

There's been some confusion about exactly what is proven in regards to Bitclockers.com so I'm providing a summary on results so far with some results corrected.

My proofs can be divided into three categories:

1. Provable claim requiring no assumptions
2. Provable claim requiring one assumption
3. Claims that can't be proven but are likely on the balance of probabilities.

1. Provable claim requiring no assumptions

If Bitclockers.com are lying to their miners about when blocks are solved but not if blocks are solved .....it is simple to show empirically that strategic miners are being underpaid.

.........

2. Provable claim requiring one assumption

If Bitclockers.com are not lying to their miners about when blocks are solved or if blocks are solved, then we can prove that Bitclockers.com are underpaying all their miners by 21%, just using the available empirical data.

.........

3. Claims that can't be proven but are likely.

If Bitclockers.com are lying to their miners about if and when blocks are solved, not only are they not paying strategic miners their expected reward, Bitclockers.com may be withholding btc from all miners, for example from the very short rounds that they don't report. The possibility can be assessed fairly simply by examining the distribution of means for Bitclockers.com using the Central Limit Theorum.......So that's an extra bonus to the pool ops of more than 90 btc/month. This is a 2.81% addition to the 2% tax (about 65 btc/month), bringing it to a 4.81% tax overall.

.........

4. In conclusion
Case 1 is proven correct: By their own description of their payout method, Bitclockers.com is underpaying some miners and stealthily redistributing it to others.

Case 2 is only proven correct if the Bitclockers.com pool operators were being truthful when they stated that they reported blocks found as they occurred. Since they have recently been linking solved blocks to block explorer, this has become much more likely.

Case 3 is certainly possible, and on the balance of probabilities, even likely. If it is the case, all miners lose an additional 2.81% on top of the acknowledged 2% tax.

In conclusion, any time a pool does not disclose round statistics, it is too easy to 'fudge' the books and pay miners less than expected.


I hope this clears things up for everyone.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 02, 2012, 02:05:26 PM
1.111~ sent,

Thanks, Transisto, I do appreciate that. Full disclosure: the donations I've gotten all of which I'm grateful for (special mention to Goat - thanks mate) has been spent on making me a "Donator" to this forum. Some of the next donations I get - if any - will go to hoppersden.info admin 'myself'.

Quote
Unfortunate that shady hopping countermeasure are being used to fake stats/payout,  when open and effective solution already exist.

And irritating. Instead of moving to a system that is unhoppable, full time miners lose money (less than usual but still some) to strategic miners, all so they can keep the hashrate boost that strategic miners provide. And this is ignoring all the other ways they can and probably have ripped off miners so far.


Quote
Ps: statistically speaking,,, Some pool that charge 10% for PPS and 3% for prop..  They make it sound as if running a prop pool with 4Ths was a risky business to the operator. , it's not.  ... Just saying... see sig.

Well, I'm going to disagree with you there. A pool needs at least 7% to make PPS risk manageable. So really, they'd be taking 3% on top of the risk tax (which would be put aside for long rounds) plus 3% on prop. That's only 1% above what other pools take so I don't think it's too greedy. Man's gotta eat.




Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 02, 2012, 02:31:40 PM
BC went down, now that it is up, organ think anything has changed?

So, you think they slowed the initial hash rate and share count to have hoppers stay longer and/or delaying block starts?  So, would that buffer be spread out over the "block" (fake) or just kept by the op?  I can see you can hop it but it's like a even worse coinotron...get killed on the short rounds...not a good mix with other prop pools...

Also, did BC just have a short round?  
huh.

weird.

my daily payment is close to double what i normally get now.  ???
must be the short blocks.

....wait, they have two short blocks?

I know, hope they're back to normal

....wait, they have two short blocks?

Yeah they got busted stealing miner's coins so they had to stop taking short blocks as profit.  

It's a little odd, sure, but not impossible. They might have altered their algorith/transform function a little, but even using the transforms I previously identified (which might not be being used but do provide a guide), the equivalent geometric round shares at that D would be:
Code:
> qgeom(plnorm(0.1664707,-0.0627,0.4085),1/1376302.26)
[1] 15
> qgeom(plnorm(0.1565703,-0.0627,0.4085),1/1376302.26)
[1] 7
> qgeom(pgamma(0.1664707 ,6.080,5.949 ),1/1376302.26)
[1] 667
> qgeom(pgamma(0.1565703 ,6.080,5.949 ),1/1376302.26)
[1] 483
> qgeom(pllogis(0.1664707 ,4.3110,scale=1/1.0653),1/1376302.26)
[1] 794
> qgeom(pllogis(0.1565703 ,4.3110,scale=1/1.0653),1/1376302.26)
[1] 610

Round lengths as short as 7 shares or 700 shares do occur. Round lengths under one share do not. So, calculating the cdf at one share for the difficulty at that time, and using quantile functions to convert to a Bitclockers.com round length as a proportion of D, we get:
Code:
> pgeom(1,1/1376302.26)
[1] 1.453169e-06
> qllogis(1.453169e-06 ,4.3110,scale=1/1.0653)
[1] 0.04153282
> qlnorm(1.453169e-06,-0.0627,0.4085)
[1] 0.138985
> qgamma(1.453169e-06 ,6.080,5.949 )
[1] 0.05853246

So if we'd seen rounds between 0.04xD and 0.14xD, we could have been be pretty certain that they'd have changed their transform/algo. At the moment, I don't think we can say that. Looks possible though.

  


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 02, 2012, 03:00:09 PM
Pools I have confidence in and would mine at (in no particular order)
  • Slush's pool (once they're on whatever hop proof payment method Slush decides on)
  • Ozcoin
  • Eclipse
Why Slush? I 've investigated them more than any other pool. I'd have noticed anything odd.

Id certainly add bitminter to that list, and I have one concern with Slush; its not major, and I have no reason to distrust Slush, but I sure would like to see stats of shares submitted there. Must be the only pool that doesnt show it (expect for running round).

Good oh. I've put a poll on which might be a better way of indicating what we think (although online polls are subject to gaming, I suppose). Care to vote?


Title: Re: Neighbourhood Pool Watch 1.5: Bitclockers.com summary
Post by: organofcorti on March 02, 2012, 03:46:19 PM
I guess they wont be getting many votes then  ;D

p2Pool isn't a pool either but it seems to be leading the pack.



Title: Re: Neighbourhood Pool Watch 1.5: Bitclockers.com summary
Post by: DeathAndTaxes on March 02, 2012, 03:53:27 PM
I consider p2pool a pool.  It just happens to be a decentralized one.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: P4man on March 02, 2012, 03:55:00 PM
Good oh. I've put a poll on which might be a better way of indicating what we think (although online polls are subject to gaming, I suppose). Care to vote?

Perhaps you should have allowed multiple votes and rephrase the question to "what pools do you trust".  Being honest, if I understood what Ive read so far, is that p2pool is the most trustworthy bar none.
But I still voted for bitminter :).


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 02, 2012, 04:01:57 PM
Good oh. I've put a poll on which might be a better way of indicating what we think (although online polls are subject to gaming, I suppose). Care to vote?

Perhaps you should have allowed multiple votes and rephrase the question to "what pools do you trust".  Being honest, if I understood what Ive read so far, is that p2pool is the most trustworthy bar none.
But I still voted for bitminter :).

Good point. Unfortunately once a poll is started, multiple votes is one of the things that can't be changed. No doubt there'll be a next time. Right after a "Which pool do you trust the least?" poll.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Maged on March 02, 2012, 10:50:47 PM
Good oh. I've put a poll on which might be a better way of indicating what we think (although online polls are subject to gaming, I suppose). Care to vote?

Perhaps you should have allowed multiple votes and rephrase the question to "what pools do you trust".  Being honest, if I understood what Ive read so far, is that p2pool is the most trustworthy bar none.
But I still voted for bitminter :).

Good point. Unfortunately once a poll is started, multiple votes is one of the things that can't be changed. No doubt there'll be a next time. Right after a "Which pool do you trust the least?" poll.
You totally can change it. Just allow people to make multiple votes and allow them to edit their votes.


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 02, 2012, 11:11:06 PM
Good oh. I've put a poll on which might be a better way of indicating what we think (although online polls are subject to gaming, I suppose). Care to vote?

Perhaps you should have allowed multiple votes and rephrase the question to "what pools do you trust".  Being honest, if I understood what Ive read so far, is that p2pool is the most trustworthy bar none.
But I still voted for bitminter :).

Good point. Unfortunately once a poll is started, multiple votes is one of the things that can't be changed. No doubt there'll be a next time. Right after a "Which pool do you trust the least?" poll.
You totally can change it. Just allow people to make multiple votes and allow them to edit their votes.

Not seeing it, sorry.

http://i.imgur.com/SeY2v.png

Is there another option somewhere?


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: Maged on March 03, 2012, 01:44:47 AM
You totally can change it. Just allow people to make multiple votes and allow them to edit their votes.

Not seeing it, sorry.

Is there another option somewhere?
Oh, interesting. Mods apparently have more powers than normal people...

Poll updated.


Title: Re: Neighbourhood Pool Watch 1.5: Bitclockers.com summary
Post by: organofcorti on March 03, 2012, 01:48:04 AM
Thanks Maged.

Anyone who wants to update their vote or add votes, you'll have to remove your old vote first.


Title: Re: Neighbourhood Pool Watch 1.5: Bitclockers.com summary
Post by: K1773R on March 03, 2012, 03:24:53 AM
why can you vote for scammers? thats stupid! (bitclockers)


Title: Re: Neighbourhood Pool Watch 1.5: Bitclockers.com summary
Post by: organofcorti on March 03, 2012, 03:59:38 AM
why can you vote for scammers? thats stupid! (bitclockers)

To gauge the success of my posts, obviously.


Title: Re: Neighbourhood Pool Watch 1.5: Bitclockers.com summary
Post by: organofcorti on March 03, 2012, 05:22:54 AM
why can you vote for scammers? thats stupid! (bitclockers)

To gauge the success of my posts, obviously.

Wish my proxy is in there. The people who use it know exactly where the hashes are being sent:)

Sorry, reading your thread I didn't think 'Proj#2' was up any more. Is it? If it is I'll add it.


Title: Re: Neighbourhood Pool Watch 1.5: Bitclockers.com summary
Post by: Maged on March 03, 2012, 06:14:36 AM
You can change your vote.


Title: Re: Neighbourhood Pool Watch 1.5: Bitclockers.com summary
Post by: organofcorti on March 03, 2012, 06:15:51 AM
Should I add it, Goat?


Title: Re: Neighbourhood Pool Watch 1: No-one should mine at Bitclockers
Post by: organofcorti on March 03, 2012, 08:27:02 AM
organofcorti, your problem is that you're assuming that the actual expected round length is supposed to be 1.0281 D. Instead, assume that it's 1.0 D. What is the probability that the measured round lengths would average to 1.0281 D by chance?
You'd like to assume the population mean to be 1xD and estimate the probability of the round lengths averaging to 1.0281? Using a similar technique to that described earlier (using code similar to this) (http://pastebin.com/qnVTX2Cr) we get a probability of 0.8801 rounds being less than 1.0281, giving a probability of rounds being 1.0281 or longer at 1-0.8801=0.1199. This is even worse than the estimate I provided earlier.
My statistics are a little rusty, but isn't that result not statistically significant?
i don't think we're testing a hypothesis here. it's just a probability.
yes, it's just based on the cdf.
Quote

...unless organofcorti wants to do a series of real-time test with a similar sized prop pool.

Can't do that either. If we assume the population mean is 1.0*D, we have to assume the distribution is similar to the Bitclockers.com distribution. Can't compare a geometrically distributed round with a bitclockers.com round - they aren't similar.

Sorry guys, I was wrong in both cases. I forgot that using the CLT a) it doesn't matter what distributions the means are from and b you can assume normality so you could consider it a non-significant p-value. It still stands as a probability though, and 0.1316 is still low.








Title: Re: Neighbourhood Pool Watch 1.5: Bitclockers.com summary
Post by: Gabi on March 06, 2012, 04:07:52 PM
"trusting" p2pool is misleading. P2pool do not require trust, because it CANNOT scam you.


Title: Re: Neighbourhood Pool Watch 2.1 Deepbit
Post by: organofcorti on March 11, 2012, 01:25:41 PM
Neighbourhood Pool Watch 2.1 Deepbit (http://organofcorti.blogspot.com.au/2012/03/21-deepbit.html) is out (and yes I've moved the 'Neighbourhood Pool Watch' blog to Blogger).

From the post:

Quote
6. Conclusions

  • Comparisons of the empirical and expected statistical parameters for Deepbit's round lengths  show little difference.
  • Quantile and CDF comparisons likewise show expected results, and the Kolmogorov-Smirnov test indicates that the null hypothesis cannot be rejected.
  • Deepbit.net's current mean round length is slightly less than expected, but not improbably so.
  • A fulltime proportional miner can be quite confident that they will earn at least as much as a PPS miner within 11 days (at the current pool hashrate).

You can comment at the blog if you're logged in to gmail, or here if you don't have a gmail account (whaaa....?)

Edit: on second thoughts, I've left the commenting open to anyone - you won't have to log in.


Title: Re: Neighbourhood Pool Watch 2.1 Deepbit
Post by: ZPK on March 11, 2012, 02:33:45 PM
add pool.itzod.ru


Title: Re: Neighbourhood Pool Watch 2.1 Deepbit
Post by: organofcorti on March 12, 2012, 12:25:50 AM
add pool.itzod.ru

If you mean "please add pool.itzod.ru to the poll options", well maybe next time. I'm closing the poll soon, and it wouldn't be fair to the pool. I'll do another poll at some point.


Title: Re: Neighbourhood Pool Watch 1.5: Bitclockers.com summary
Post by: kano on March 12, 2012, 03:20:26 AM
"trusting" p2pool is misleading. P2pool do not require trust, because it CANNOT scam you.
Then again :)
Did you realise that by default you give 0.5% of your earnings to forrestv?
You must change that if you wish it to be a different value :)


Title: Re: Neighbourhood Pool Watch 1.5: Bitclockers.com summary
Post by: Clipse on March 12, 2012, 02:16:00 PM
"trusting" p2pool is misleading. P2pool do not require trust, because it CANNOT scam you.
Then again :)
Did you realise that by default you give 0.5% of your earnings to forrestv?
You must change that if you wish it to be a different value :)

Are you suggesting a voluntary donation that is switched on by default is a scam?

If so, at least provide me with instructions on disabling the fee on the three largest mining pools.  ;)

Oh and, RTFM!

Auto enabling a fee and calling it a voluntary donation isnt the same thing.

Voluntary suggests you informed a user and they then took steps to provide a donation, I wont be surprised if a bunch of p2pool users didnt even know about this 0.5% fee.


Title: Re: Neighbourhood Pool Watch 2.1 Deepbit
Post by: martychubbs on March 12, 2012, 02:53:53 PM
2.1 was more good reading! Thanks, very well done. ;)


Title: Re: Neighbourhood Pool Watch 2.1 Deepbit
Post by: organofcorti on March 13, 2012, 07:27:47 AM
2.1 was more good reading! Thanks, very well done. ;)

You're welcome, Marty. Next up looks at how strategic miners affect the payouts of fulltime miners on the prop payout side.


Title: Re: Neighbourhood Pool Watch 2.1 Deepbit
Post by: kano on March 13, 2012, 10:26:34 AM
2.1 was more good reading! Thanks, very well done. ;)

You're welcome, Marty. Next up looks at how strategic miners affect the payouts of fulltime miners on the prop payout side.
Hmm I've found 3 blocks on DeepBit prop and been paid about 30BTC ... ... :) 500% luck
(Yes that statement is true by the way)


Title: Re: Neighbourhood Pool Watch 2.1 Deepbit
Post by: organofcorti on March 13, 2012, 10:36:40 AM
2.1 was more good reading! Thanks, very well done. ;)

You're welcome, Marty. Next up looks at how strategic miners affect the payouts of fulltime miners on the prop payout side.
Hmm I've found 3 blocks on DeepBit prop and been paid about 30BTC ... ... :) 500% luck
(Yes that statement is true by the way)

I solved one block ever and mined 110 btc. Thats 220% over what I would have gotten solo. You would have gotten 150btc mining solo, so I'd call that bad luck. Still, it averages out for us all.


Title: Re: Neighbourhood Pool Watch 2.1 Deepbit
Post by: kano on March 13, 2012, 10:40:33 AM
2.1 was more good reading! Thanks, very well done. ;)

You're welcome, Marty. Next up looks at how strategic miners affect the payouts of fulltime miners on the prop payout side.
Hmm I've found 3 blocks on DeepBit prop and been paid about 30BTC ... ... :) 500% luck
(Yes that statement is true by the way)

I solved block ever and mined 110 btc. Thats 220% over what I would have gotten solo. You would have gotten 150btc mining solo, so I'd call that bad luck. Still, it averages out for us all.
My comment was actually in reply to your one I quoted in case you didn't realise :)


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: organofcorti on March 15, 2012, 12:47:19 PM
Neighbourhood Pool Watch 2.2 (http://organofcorti.blogspot.com.au/2012/03/22-deepbit-and-pool-hopping.html) is out now. This post shows how much strategic miners add to Deepbit.net's overall hashrate, and how much full time proportional payout miners lose in the process.

The post was inspired by P4man's initial work on the subject here (https://bitcointalk.org/index.php?topic=62975.0).

Quote
5. Conclusions

  • Strategic miners are successfully adding 450 Ghps to Deepbit.net's overall hashrate.
  • Due to the presence of strategic miners, full time miners can expect to earn only 94% of expected round earnings,not 97% (including fee).
  • This loss in income means that it actually takes over 30 days (at current pool hashrate) for fulltime proportional miners to be confident of earning the same as or more than PPS miners, rather than 11 days without strategic miners present.

organofcorti's Second Rule of Pooled Bitcoin Mining: Avoid proportional payouts unless you're mining strategically.

http://i.imgur.com/fATkV.png


Title: Re: Neighbourhood Pool Watch 2.1 Deepbit
Post by: organofcorti on March 15, 2012, 01:41:59 PM
2.1 was more good reading! Thanks, very well done. ;)

You're welcome, Marty. Next up looks at how strategic miners affect the payouts of fulltime miners on the prop payout side.
Hmm I've found 3 blocks on DeepBit prop and been paid about 30BTC ... ... :) 500% luck
(Yes that statement is true by the way)

I solved block ever and mined 110 btc. Thats 220% over what I would have gotten solo. You would have gotten 150btc mining solo, so I'd call that bad luck. Still, it averages out for us all.
My comment was actually in reply to your one I quoted in case you didn't realise :)

You can be a cryptic bugger, Kano ;)


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: P4man on March 15, 2012, 04:41:25 PM
Interesting read Organ.
I do wonder if perhaps you are not misinterpreting the data; when a round starts, you get fresh work. Even assuming zero delay from the pool, and zero network latency, only the fastest miners are going to be able to submit shares within seconds. Slower miners may take up to a minute or more before they submit their first share.

I wonder if that by itself couldnt cause both the drop at the beginning of the round, and the slow spike afterwards. After all, deepbit rounds are very short, and I wouldnt be surprised there a lot of slow and clueless miners on there.

As for the poll:

Bitclockers.com    - 2 (0.6%)

Will the one person besides the pool admin who voted for bitclockers please stand up?


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: organofcorti on March 17, 2012, 10:09:39 AM
Interesting read Organ.
I do wonder if perhaps you are not misinterpreting the data; when a round starts, you get fresh work. Even assuming zero delay from the pool, and zero network latency, only the fastest miners are going to be able to submit shares within seconds. Slower miners may take up to a minute or more before they submit their first share.

I wonder if that by itself couldnt cause both the drop at the beginning of the round, and the slow spike afterwards. After all, deepbit rounds are very short, and I wouldnt be surprised there a lot of slow and clueless miners on there.

No, I can see how you'd think that, but a group of poisson distributed variables with different means is still a poisson distributed variable with a new mean of arrivals per unit time that is the sum of the of the constituent mean arrivals per unit time.

Think of it this way. 10 miners with 1 Ghps will submit about 2.3 shares per second on average. So will one miner with 10Ghps.

The only thing I can put the slow hashrate increase from below normal is variable time to receive a long poll notice. Until such notice is received, any shares that are sent will be stale. Then there's the time taken for the trip back, which may not be the same as the trip there, due to variable servers on the way.

A log-normal distribution is a good fit for a combination of normally distributed variables that have a multiplicative effect on each other, so I chose it as a starting point for the model. It turned out to model that response pretty well. It's only the mean of ~ 0.7 seconds and deviation of ~1.7 seconds that I can't interpret.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: organofcorti on March 17, 2012, 11:21:26 AM
Poll voting is locked. Results published shortly.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: organofcorti on March 17, 2012, 12:54:59 PM
Here are the results for the "Which pools do you trust?" poll.

http://i.imgur.com/qciVA.png

Below is a bubble plot showing votes on the y axis and bubble size showing hashrate.

http://i.imgur.com/CO5VC.png

It seems that small pools vary significantly from the minimum to the maximum of the vote, and the larger pool are in the mid range. To try to account for hashrate, I divided all votes by the log of the hashrate to get a 'trust index'. I used log(hashrate) to try to account for the fact that novice miners that might not know much about trust tend to gravitate to larger pools first.

http://i.imgur.com/f1uRH.png

p2Pool is the clear winner, followed by EMC and Ozcoin. Well done to all in the top three pools - you've gained significantly more trust in the community than others.

Proviso: although data is accurate, it probably is not very reliable. It's self selecting: mostly english speakers and only those concerned about trust in pools. Further, sample size is small and voters could vote more than once which skew results further.





Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: organofcorti on March 18, 2012, 05:42:59 AM
New poll started. To complement the last one, this time it's: which pool do you trust the least?

add pool.itzod.ru

...and done.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: organofcorti on March 18, 2012, 09:08:40 AM
like the new pool, wonder why the only other voter does not trust slush, hmmm

Wasn't me. I forgot to vote after I posted the poll. I think it's pretty obvious which at least one of the pools I voted for was. Not Slush.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: organofcorti on March 18, 2012, 10:35:27 AM
Currently prop with delayed stats. I think they're changing to pplns soon, or may have already. If you're asking about how to hop them .... well, there's another thread for that.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: kano on March 18, 2012, 12:23:28 PM
Hmm - simplecoin.us still mines also (their last block was ... 171650)
I only know that coz they have their name in the coinbase :)

(at the moment there's 10 pools I can tell from looking at the block in the last 100 blocks - accounting for 68% of the 100)


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: DeepBit on March 18, 2012, 04:28:25 PM
Actually I think they vote against Eligius for taking down the Springcoin.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: kano on March 18, 2012, 09:56:16 PM
I guess you guys are right. His attitude lends to misbehavior of his pool. Based on past performance it’s better to not trust his pool at all.

Edit: I wish the people who voted down pools like p2pool and Ozcoin would speak up. I can see no reason for voting them down at all.
Actually - both of those pools are in direct contrast of trust.
P2Pool people say that you can't trust normal pools - so at least someone somewhere will probably say they don't trust each normal pool.
P2Pool, on the other hard, takes a 0.5% fee that most people don't realise until someone points it out to them and they can change it to zero - so there would certainly be a lack of trust by some due to that also.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: kano on March 19, 2012, 12:59:52 AM
...
I’ve never used p2pool so I had no idea it has a built in fee. Now, is that fee advertised somewhere or do you need to accidentally discover it in a setting and go out of your way to change it?

It's an option listed in the middle of the docs with all the other options.
Read if you are interested :)
That gives you a chance to decide if you think it is visible enough or not :D

Edit: hmm it's now 4 for P2Pool - maybe 4 people didn't realise it - or there are 4 other reasons?


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: grue on March 19, 2012, 01:22:06 AM
...
I’ve never used p2pool so I had no idea it has a built in fee. Now, is that fee advertised somewhere or do you need to accidentally discover it in a setting and go out of your way to change it?

It's a very small fee by default (0.5%), and you can always set it to 0 with "--give-author 0"


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: Nim on March 19, 2012, 04:40:01 AM
...
I’ve never used p2pool so I had no idea it has a built in fee. Now, is that fee advertised somewhere or do you need to accidentally discover it in a setting and go out of your way to change it?

It's an option listed in the middle of the docs with all the other options.
Read if you are interested :)
That gives you a chance to decide if you think it is visible enough or not :D

Edit: hmm it's now 4 for P2Pool - maybe 4 people didn't realise it - or there are 4 other reasons?
Middle of the docs? Who's going to read docs to join a mine? If I was joining a mine, I would expect the "fee" to be listed in the first post of the pool thread:

https://bitcointalk.org/index.php?topic=18313.msg231846#msg231846

Hmm. Nothing about it there. How about in the guide to setup p2pool?

https://bitcointalk.org/index.php?topic=18313.msg712967#msg712967

Nah, still nothing.

Oh wait, I remember seeing a link to a wiki. It must be there:

https://en.bitcoin.it/wiki/P2Pool

I scan the page down till I get to the heading beginning with Donating... thinking I hit the jackpot, but no. Surely something as important as a built-in fee in a pool that claims to be fee-free should be mentioned in a paragraph that talks about the reasoning behind it? Alas, I finally spot it hidden in the command line options, where very few people will ever notice, especially if they just follow the guides.

You guys are exploiting people much the same way that companies will exploit customers who buy products with rebates. They expect that a certain percentage won't know about the rebate, will forget about the rebate, or won't fill it out correctly. You're also taking advantage of people's inclination to do nothing when faced with a tough decision and hence take the default option. While it isn't truly hidden and has been talked about several times on here (with each time p2pool users surfacing saying that they never heard of it), anything less than front and center IS hidden when it comes to a fee, ESPECIALLY when one of your primary raison d'etre is no fees! You can claim that people should know about it and how to stop it. That the donation is worthy. That it is good intentioned. None of that matters.

There are only two options if you want to have an honest, reputable pool (which we all want). Advertise the built-in donation loudly and boldly, or make it opt-in rather than opt-out.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: organofcorti on March 19, 2012, 06:49:55 AM
I guess you guys are right. His attitude lends to misbehavior of his pool. Based on past performance it’s better to not trust his pool at all.

Edit: I wish the people who voted down pools like p2pool and Ozcoin would speak up. I can see no reason for voting them down at all.
Actually - both of those pools are in direct contrast of trust.
P2Pool people say that you can't trust normal pools - so at least someone somewhere will probably say they don't trust each normal pool.
P2Pool, on the other hard, takes a 0.5% fee that most people don't realise until someone points it out to them and they can change it to zero - so there would certainly be a lack of trust by some due to that also.

Kano, can you tell me why you think Ozcoin is "in direct contrast of trust"? It's one of the few pools I have trust in, so I'd like to hear an opposing view point.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: rjk on March 19, 2012, 01:46:25 PM
Kano, can you tell me why you think Ozcoin is "in direct contrast of trust"? It's one of the few pools I have trust in, so I'd like to hear an opposing view point.
I think he just means the literal definition of a pool, as such, and therefore all pools apply. If I am wrong, I would also like to hear an answer to this. Graeme runs an excellent pool.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: Nim on March 19, 2012, 05:00:19 PM
As I've said, I thought this was obvious. People should be auditing the P2Pool code before using it! Now they can't even be bothered to read the wiki?  :'(
I just want to let this sink in for a bit. You are one of the big proponents of p2pool. You would have everyone use it rather than other pools. And now you think that everyone should be auditing the code of p2pool before using it? So everyone who mines bitcoin (and any other cryptocoin that uses p2pool) should 1) have the programming ability to read and analyze the code and 2) the time to do so? Just think about that for a bit and tell me if you think that is a reasonable request.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: Nim on March 19, 2012, 05:29:50 PM
As I've said, I thought this was obvious. People should be auditing the P2Pool code before using it! Now they can't even be bothered to read the wiki?  :'(

I just want to let this sink in for a bit. You are one of the big proponents of p2pool. You would have everyone use it rather than other pools. And now you think that everyone should be auditing the code of p2pool before using it? So everyone who mines bitcoin (and any other cryptocoin that uses p2pool) should 1) have the programming ability to read and analyze the code and 2) the time to do so? Just think about that for a bit and tell me if you think that is a reasonable request.

I don't need to think about it. People need to be responsible for their actions. That's the entire reason I am a proponent for P2Pool. You are responsible for the blocks you create.

Let's say Gavin suddenly decides that Bitcoin is not going where he thinks it should and makes some major changes in the next client update. Do you think people should just update to the new client without auditing the code? Does it matter if you know how to audit the Bitcoin client code or not? Does it matter if you have the time or not?

People with the ability and time to audit the code will. And they will make forum posts, wiki entries, et cetera for those who can't.

My point was, you should audit the code before using it. If you can't, at least do some simple research like reading the wiki.
If you really think like that and are absolutely certain, then you should remove your p2pool guide. If someone is reading the code and studying the wiki, they won't need the guide. Guides are for people who don't do those things. In fact, your guide is enabling people to use p2pool that apparently shouldn't be. Don't be an enabler.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: P4man on March 19, 2012, 07:41:00 PM
I quit giving funds because of the .5% hidden fee :(    Project #2 was going to give 10% of the income to p2pool but I don't know now. I need to take some time and review it. I think this is a black make on p2pool :(

I like giving donations, I don't like hidden fees :(  How can I support something where people get money taken from then and not know it:(

I am the only one who feels like throwing up when reading this?

On one hand, we have ForrestV, who actually built something himself, from the ground up. Something thats incredibly innovative, (very) valuable to bitcoin and the community at large. Something that is open and opensource.

No secrets, no lies.

OTOH, we have a clueless Goat who has never done anything himself, but leaches off everyone else and thrives through secrecy and deceit. A goat that runs a mining business owned by his shareholders who are not being told where his farm is mining, who are not being told how much is truly being earned from pool hopping, who are not being told anything but who are paid dividends based only on fabricated numbers that dont come close to reflecting actual pool hopping profits.

That same Goat now cries foul because someone ignorant may not read the readme and therefore unwittingly donate  a small amount to someone deserving, yet he runs a hopping proxy, developed by people with the skills he lacks, a proxy that robs the poor and ignorant miners of prop pools from far more than 0.5%, yet he makes zero effort to educate those miners or protect them from their far larger losses because he pockets them, and not Forrestv.

So that hypocrite is now going to pretend being the miners advocate, judging what an honest (and optional) donation fee is for ForrestV's hard work, most (or all?) of which is redistributed among the miners?



Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: kano on March 20, 2012, 03:44:48 AM
Kano, can you tell me why you think Ozcoin is "in direct contrast of trust"? It's one of the few pools I have trust in, so I'd like to hear an opposing view point.
I think he just means the literal definition of a pool, as such, and therefore all pools apply. If I am wrong, I would also like to hear an answer to this. Graeme runs an excellent pool.
I was simply referring to the issue that MANY P2Pool acolytes say that normal pools are untrustworthy thus you should use P2Pool
So in that respect, P2Pool is in opposition to Ozcoin when it comes to trust.

I trust Ozcoin and Graeme probably more than any other pool around.
(I'm even mining on Ozcoin at the moment and have been 100% for the past 6 days)

Aside: The early on P2Pool evangelism turned me off P2Pool, so for myself it's unlikely I'll use P2Pool in the near future (and I have brought up reasons on the forum also - e.g. other issues relate to having no technical documentation, and IMO a lack of reliability and stability - also the 'accepted stales' issues I've brought up elsewhere)


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: kano on March 20, 2012, 03:48:32 AM
...
OTOH, we have a clueless Goat who has never done anything himself, but leaches off everyone else and thrives through secrecy and deceit. A goat that runs a mining business owned by his shareholders who are not being told where his farm is mining, who are not being told how much is truly being earned from pool hopping, who are not being told anything but who are paid dividends based only on fabricated numbers that dont come close to reflecting actual pool hopping profits.

That same Goat now cries foul because someone ignorant may not read the readme and therefore unwittingly donate  a small amount to someone deserving, yet he runs a hopping proxy, developed by people with the skills he lacks, a proxy that robs the poor and ignorant miners of prop pools from far more than 0.5%, yet he makes zero effort to educate those miners or protect them from their far larger losses because he pockets them, and not Forrestv.
...
I don't see how declaring someone else 'bad' makes you or P2Pool seem better.
It actually reflects poorly on both yourself and P2Pool that you need to use such a comparison.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: kano on March 20, 2012, 04:41:59 AM
It actually reflects poorly on both yourself and P2Pool that you need to use such a comparison.

You would let someone's diatribe against someone they've had a history with affect your judgement on the merits of the mining pool they happen to use?

That's a strange way to pick a Bitcoin mining pool.
No.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: kano on March 20, 2012, 04:47:06 AM
It actually reflects poorly on both yourself and P2Pool that you need to use such a comparison.

You would let someone's diatribe against someone they've had a history with affect your judgement on the merits of the mining pool they happen to use?

That's a strange way to pick a Bitcoin mining pool.
No.

Then how do his word reflect poorly on P2Pool?
Do I need to explain the English?
If English is not your first language I'll go to the trouble.

Otherwise - try reading it from the position of someone not enraptured with P2Pool :)


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: Inaba on March 20, 2012, 02:58:52 PM
Hey so who are the six that don't trust EMC?  I'd certainly like to address any issues with that, as I try to be as open as possible with everything I do with regards to the pool.  If there's something that is questionable, I will certainly address it.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: kano on March 20, 2012, 10:31:41 PM
Hey so who are the six that don't trust EMC?  I'd certainly like to address any issues with that, as I try to be as open as possible with everything I do with regards to the pool.  If there's something that is questionable, I will certainly address it.
Also, realise, as I mentioned further up and you can read elsewhere on the forum, P2Pools advocacy of saying that non-P2Pool pools are not trustworthy will obviously imply that those who agree with that will not trust any pool but P2Pool and thus any of them could choose any pool to list as untrustworthy.

Not long after I mentioned that I trusted Ozcoin, the number of people who selected Ozcoin jumped up :P (from 1 to 4)

Be interesting to see if I've just caused yours to jump up :P
It's currently 7 - maybe even your comment made it go up 1 more?

Edit: actually I was thinking about this advocacy yesterday and realised I had seen something similar here on the forum before.
SolidCoin

Anyway, I would also suggest that my comments in this thread have probably made this current poll meaningless ... ... ... ...

Edit2: it's gone up 2 more since I posted - hmm I wonder


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: m3ta on March 21, 2012, 12:39:46 AM

Previous posts:
Neighbourhood Pool Watch 2.1 Deepbit (http://organofcorti.blogspot.com.au/2012/03/21-deepbit.html)

Nicely done.
Except for

Quote
being honest with it's statistics

But because more than 90% of the universe is illiterate on the difference between "it's" and its", thumbs up anyway.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: organofcorti on March 22, 2012, 07:55:16 AM

Previous posts:
Neighbourhood Pool Watch 2.1 Deepbit (http://organofcorti.blogspot.com.au/2012/03/21-deepbit.html)

Nicely done.
Except for

Quote
being honest with it's statistics

But because more than 90% of the universe is illiterate on the difference between "it's" and its", thumbs up anyway.


Fixed! Thanks for reading so carefully. Feeling like being a proof reader, m3ta? I hate hassling Mobius all the time.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: organofcorti on March 22, 2012, 07:57:37 AM
Hey so who are the six that don't trust EMC?  I'd certainly like to address any issues with that, as I try to be as open as possible with everything I do with regards to the pool.  If there's something that is questionable, I will certainly address it.
Odds on it's just disgruntled miners with an axe to grind, or as Goat suggested other pool ops who can't DDOS anymore. I wouldn't worry about it - you and Ozcoin have great reps for openness and reliability.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: organofcorti on March 22, 2012, 08:03:45 AM
Hey so who are the six that don't trust EMC?  I'd certainly like to address any issues with that, as I try to be as open as possible with everything I do with regards to the pool.  If there's something that is questionable, I will certainly address it.
Also, realise, as I mentioned further up and you can read elsewhere on the forum, P2Pools advocacy of saying that non-P2Pool pools are not trustworthy will obviously imply that those who agree with that will not trust any pool but P2Pool and thus any of them could choose any pool to list as untrustworthy.

I don't think trust is a binary concept - you don't only 'completely trust' or 'completely distrust' anything or anyone. So p2Pool not requiring trust (if you've read the code) doesn't automatically mean you 'completely distrust' other options. So p2Pool advocates are being misleading, and if they are selecting random pools to distrust, or have selected 15 pools other than p2Pool to distrust, then they are being disingenuous or have a poor grasp of logic.

I'll normalise to the lowest distrust level anyway, so they won't have an effect unless they're targeting a particular pool out of spite.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: Brunic on March 22, 2012, 10:31:02 AM
Hey so who are the six that don't trust EMC?  I'd certainly like to address any issues with that, as I try to be as open as possible with everything I do with regards to the pool.  If there's something that is questionable, I will certainly address it.

Don't worry, your pool is great. For my part, I wanted to see the results of the poll, and voted for everybody so to not screw the % of the votes (well, except one pool, since you can only choose 15 on 16).


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: Clipse on March 22, 2012, 11:36:39 AM
Hey so who are the six that don't trust EMC?  I'd certainly like to address any issues with that, as I try to be as open as possible with everything I do with regards to the pool.  If there's something that is questionable, I will certainly address it.
Also, realise, as I mentioned further up and you can read elsewhere on the forum, P2Pools advocacy of saying that non-P2Pool pools are not trustworthy will obviously imply that those who agree with that will not trust any pool but P2Pool and thus any of them could choose any pool to list as untrustworthy.

I don't think trust is a binary concept - you don't only 'completely trust' or 'completely distrust' anything or anyone. So p2Pool not requiring trust (if you've read the code) doesn't automatically mean you 'completely distrust' other options. So p2Pool advocates are being misleading, and if they are selecting random pools to distrust, or have selected 15 pools other than p2Pool to distrust, then they are being disingenuous or have a poor grasp of logic.

I'll normalise to the lowest distrust level anyway, so they won't have an effect unless they're targeting a particular pool out of spite.

Some of these "distrust" votes do look fishy.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: organofcorti on March 22, 2012, 08:40:45 PM
Hey so who are the six that don't trust EMC?  I'd certainly like to address any issues with that, as I try to be as open as possible with everything I do with regards to the pool.  If there's something that is questionable, I will certainly address it.

Don't worry, your pool is great. For my part, I wanted to see the results of the poll, and voted for everybody so to not screw the % of the votes (well, except one pool, since you can only choose 15 on 16).

Results now available without voting. Hopefully no more donkey votes.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: organofcorti on March 28, 2012, 12:16:51 PM
Next post available: 3.1 The rise and fall of Arsbitcoin.com (http://organofcorti.blogspot.com.au/2012/03/31-rise-and-fall-of-arsbitcoincom.html)

From the post:

Quote
6. Conclusions.
  • Public perception of the pool operator and the pool is positive and miners there seem to trust the pool operator.
  • The pool operator is not publicly identifiable.
  • The pool has been very popular in the past with few complaints until problems began to occur in November 2011.
  • Arsbitcoin.com has a large negative buffer and a falling hashrate - two very big problems for an SMPPS pool.
  • The pool op, Burning Toad, is currently unable to keep the pool running smoothly, as evidenced by the Arsbitcoin.com thread on bitcointalk.org and the round length anomalies described in this post.
  • Next post will cover the problems with the SMPPS system.

I'm hoping for some debate on this one, from Arsbitcoin.com partisans and SMPPS fanciers alike.


Title: Re: Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping'
Post by: kano on March 28, 2012, 12:29:49 PM
...
I'm hoping for some debate on this one, from Arsbitcoin.com partisans and SMPPS fanciers alike.
So ... what am I missing if I say that a 'zero fee' SMPPS pool should just put 1.5% of each block into the buffer and pay 98.5% ?
i.e. the buffer loses out when you get an orphan, and the 'supposed' expected orphan rate is 1.5%


Title: Re: Neighbourhood Pool Watch 3.1 The Rise and Fall of Arsbitcoin.com
Post by: organofcorti on March 28, 2012, 12:32:07 PM
Thanks Kano. That will all be covered in the next post. BTW, Arsbitcoin.com records no orphaned/invalid blocks at all. I don't know if that's accurate or not. Where do you get your 1.5% stat from?


Title: Re: Neighbourhood Pool Watch 3.1 The Rise and Fall of Arsbitcoin.com
Post by: kano on March 28, 2012, 01:00:04 PM
Thanks Kano. That will all be covered in the next post. BTW, Arsbitcoin.com records no orphaned/invalid blocks at all. I don't know if that's accurate or not. Where do you get your 1.5% stat from?
Hmm good question :)
I'm sure I heard it indirectly from a Pool OP or 2 ... but I can't for the life of me remember who :P
I'll try a forum search ...
OK ABCProxy's web page says 1.5%
MintCondition also said it again in the ARS thread
and I think I remember seeing 1.0% written somewhere also ...
But any pool with a count of orphans can easily determine a % figure.


Title: Re: Neighbourhood Pool Watch 3.1 The Rise and Fall of Arsbitcoin.com
Post by: organofcorti on March 28, 2012, 01:15:46 PM
... if you trust their stats. Apparently arsbitcoin have had none at all, and it would be in ABC's interest to inflate orphaned block rates since their miners are paid regardless. Then again being a proxy pool, they'd probably follow all pool stats closely, so maybe its true.

Anyone have a citable source showing the average percentage of rounds orphaned?


Title: Re: Neighbourhood Pool Watch 3.1 The Rise and Fall of Arsbitcoin.com
Post by: Graet on March 28, 2012, 01:50:28 PM
173222    
Timestamp    2012-03-28 04:40:16
BTC Sent    9,755.133 BTC
N tx    67
Relayed By    Deepbit
   
Timestamp    2012-03-28 04:43:36
BTC Sent    9,742.691 BTC
N tx    72
Relayed By    Ars bitcoin

http://blockchain.info/orphaned-blocks
Ars had the second to last orphan on this page :)

was looking to see if they had stats on numbers of orphans


Title: Re: Neighbourhood Pool Watch 3.1 The Rise and Fall of Arsbitcoin.com
Post by: organofcorti on March 29, 2012, 11:07:34 AM
Thanks Graeme. At least we know they get orphans. I haven't found a definitive percentage figure yet though.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 02, 2012, 04:20:40 PM
New Neighbourhood Pool Watch : 3.2 The risks of SMPPS (http://organofcorti.blogspot.com.au/2012/04/32-risks-of-smpps.html)

I was going to call it "The failure of SMPPS", but Eligius is still going so I thought that a bit premature. Please note that the post is due in part to Meni Rosenfeld's paper "Analysis of pooled bitcoin mining reward systems". (https://bitcoil.co.il/pool_analysis.pdf)

Here's a bit from the post for the purpose of appetite-whetting:

Quote
5. Conclusions
  • The amount of time it takes for a miner to be paid is affected by the pool buffer:
    • A large negative buffer at will greatly increase the time it takes for a miner to be paid.
    • A buffer greater than zero minimises the time  it takes for a miner to be paid.
  • A strategic miner will only mine at an SMPPS pool when it has a positive buffer.
  • The probability of large negative buffer (that occurs some time before a a certain number of rounds) can be calculated and in the case of Arsbitcoin.com, a 1000 btc negative buffer before 1700 rounds is quite likely.
  • The more blocks a pool solves, the more likely it is for a an SMPPS pool to encounter so large a negative buffer that it will fail, leaving miners with much less than a PPS payout per share.

http://i.imgur.com/mCj18.png


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Graet on April 03, 2012, 03:02:18 AM
Ars severs have been down over 24hours now. A lot of  miners joining IRC with questions :/


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: kano on April 03, 2012, 05:56:54 AM
Can the discussion include DGM? ...

I've yet to find a valid reason why DGM, which attempts to be SMPPS, is any better.

With SMPPS you can see the buffer, with DGM, you cannot since there 'apparently is no buffer' (there is no spoon? :P)

I've had this discussion with Meni in his thread, but been unable to see a valid reason for saying one is better than the other without using jargon to hide something that is effectively the same.

... and as I said above, regarding the issue of the buffer running out, and well, it will run out for what reasons?
The answer seems to be orphans (what other reasons?)
... and that can be solved by having a variation that puts the small % of expected lost BTC per block into the buffer.
A variation of that could even adjust that % based on history.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Meni Rosenfeld on April 03, 2012, 06:58:24 AM
Can the discussion include DGM? ...

I've yet to find a valid reason why DGM, which attempts to be SMPPS, is any better.

With SMPPS you can see the buffer, with DGM, you cannot since there 'apparently is no buffer' (there is no spoon? :P)

I've had this discussion with Meni in his thread, but been unable to see a valid reason for saying one is better than the other without using jargon to hide something that is effectively the same.
WTF???

In SMPPS the buffer affects the attractiveness of mining. Low (highly negative) buffer = miners will leave.

In DGM the "buffer" does not affect the attractiveness of mining. Thus there is no reason to know it, it's a property of the pool operator, not of the pool. I don't tell you how many bitcoins I own either.

In SMPPS there is debt which can go arbitrarily high. If the pool collapses while it is still in debt to you you are screwed (and this will happen when everyone leaves due to low buffer).

In DGM the debt is bounded and so the pool can (and should) pay back the debt if it closes for any reason.

In SMPPS the variance of the buffer is 100%, in DGM it can be controlled with a parameter.

The claim that DGM "attempts to be SMPPS" is outrageous.

As I said before, I'm extremely weary of having to deal with your hostility and explaining the same thing over and over again multiple times.



Anyway, as long as I'm here:

You talk about the maturity time, but it should be emphasized that the value I'm analyzing is the time of average payback, not the time of being paid in full. For example, if you get 50% of the debt back in 1 round and the other 50% in 3 rounds, your maturity time is 2 rounds.

The second chart, giving the PDF of the pool debt after n rounds, looks completely wrong, I have no idea how you got that. The pool debt, assuming constant difficulty, is (X/D - n)*B, where X is the number of shares it took to get n blocks, which is distributed negative binomial. So it's really a shifted negative binomial, which doesn't have the cusp we see.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 03, 2012, 07:08:15 AM

... and as I said above, regarding the issue of the buffer running out, and well, it will run out for what reasons?
The answer seems to be orphans (what other reasons?)
... and that can be solved by having a variation that puts the small % of expected lost BTC per block into the buffer.
A variation of that could even adjust that % based on history.

I don't think you read the post. It explains why large negative buffers are quite likely the longer a pool goes on - without referring to orphaned blocks at all. Read it and get back to me.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: kano on April 03, 2012, 08:58:43 AM
The whole discussion about SMPPS is based on:

The assumption that this is true:
Quote
If the pool stays in a large negative buffer long enough, it is a likely that all miners will leave, and the pool will fail.
Which we ALREADY know to be FALSE.

Is there a reason to suggest to people to consider reading the rest when the reality is that arsbitcoin has proven it false?

If, on the other hand, a modified SMPPS that takes the expected pool loss into consideration (e.g. determine the % of blocks expected to be orphaned and determine that based on history after some period of time) and that % is put back into the buffer, then in the long term the pool will be expected to be even and thus people knowing that rather than listening to the WILD negative false assumptions that even such a modified pool would fail, would not be running and screaming that the sky is falling every time the buffer went negative.

However ...

Meni loves to say that "there is no buffer" so I take that to mean that when I look at a DGM pool that is paying more than 50BTC per block in a bad luck streak, that pool is in fact in serious trouble and about to fail?
Since there is no buffer, how can they pay more than 50BTC per block?


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Meni Rosenfeld on April 03, 2012, 10:20:36 AM
@kano I suggest you look up the word "Large" (http://en.wiktionary.org/wiki/large). Since you clearly have no interest in carrying out a reasonable discussion and would rather play dumb and ignore what your interlocutor says, I'll refrain from further responding to you.

For reference, here (https://bitcointalk.org/index.php?topic=39497.msg807444#msg807444) is my previous correspondence with kano about this issue.


I'll clarify one thing. In operator-risky methods like PPS, geometric method, DGM, the operator can either gain or lose on a round. These movements are in the operator's personal funds. Just like a riskless pool (like PPLNS) can collect fees for the service, so can a risky pool collect high or negative fees on a per-round basis. The variability in the operator's income is a risk he is willing to take in order to make the pool more attractive.

For his own personal accounting, the operator would do well to set aside a portion of his funds as the pool's "reserve". This makes sure that technically the funds have a place to come from, and allows a better handle on profits or loss. He could share information on his reserve if he wants. But this is still his own funds, which he can take away or add to as he pleases. The current reserve has no effect on the attractiveness, in terms of expected reward, variance and maturity time, of mining in the pool.

If the reserves run out, it is the operator's choice whether to allocate more of his funds to the pool, or to shut the pool down. Other than having to look for a new pool (which is not to be taken lightly), this has no ill effects on the miners - the debt to them, expressed as unrealized score, will be cashed out immediately based on the expected reward for it. Since the total possible debt is bounded (the bound can be controlled with parameters), the operator can simply set aside the necessary amount as an "emergency fund". There is no risk of growing an unbounded debt and then not being able to repay.

DGM settles good and bad luck on time and vents it as miner reward variance (or in the worst case, only likely with aggressive parameters, a clean bankruptcy). This allows it to continue in a sustainable steady state. SMPPS defers having to deal with luck indefinitely, until one day the debt grows so large it can no longer continue.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Inaba on April 03, 2012, 02:42:25 PM
Quote
If the pool stays in a large negative buffer long enough, it is a likely that all miners will leave, and the pool will fail.
Which we ALREADY know to be FALSE.

Wait... what?  How on earth do you come to this conclusion?  Arsbitcoin is a prime example of why it's true.  The buffer is negative, it's never going to be repaid, the miners who are owed money are screwed.  The few "hangers on" that are still around are the same people who buy lottery tickets, except in this case, they are hoping to win the lottery for the money they already earned, not extra income.

Quote
Meni loves to say that "there is no buffer" so I take that to mean that when I look at a DGM pool that is paying more than 50BTC per block in a bad luck streak, that pool is in fact in serious trouble and about to fail?
Since there is no buffer, how can they pay more than 50BTC per block?

Meni's comments aside, speaking from experience, I can tell you there is no meaningful "buffer."  As Meni has already explained, it comes out of the operators pocket - in my case, it comes out of my pocket. I refill my pocket on short blocks and pay it out on long blocks.  The total "buffer" that ever gets into the negative is no more than a few BTC at the worst of times.  If you consider 3 or 4 BTC to be a lot/large, well then I stand corrected, but to me, 1200 BTC is tending towards LARGE and 5 BTC is tending towards SMALL, especially when speaking of a pool buffer.

Those DGM pools paying substantially more than 50 BTC per block are doing so out of the operators pockets, not out of any "buffer."  Each round, you are paid what you earned, no buffering required.  I could close up EMC's shop today and cash everyone out for full payment and be right about even - specifically because I don't carry an arbitrary buffer.



Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 03, 2012, 02:47:52 PM
Anyway, as long as I'm here:

You talk about the maturity time, but it should be emphasized that the value I'm analyzing is the time of average payback, not the time of being paid in full. For example, if you get 50% of the debt back in 1 round and the other 50% in 3 rounds, your maturity time is 2 rounds.
I did gloss over that. I'll fix it tomoorow. Thanks for pointing it out.
Quote
The second chart, giving the PDF of the pool debt after n rounds, looks completely wrong, I have no idea how you got that. The pool debt, assuming constant difficulty, is (X/D - n)*B, where X is the number of shares it took to get n blocks, which is distributed negative binomial. So it's really a shifted negative binomial, which doesn't have the cusp we see.

I'll have to get back to you on that. I wrote that part a while ago, and I'll have to find my notes. It was strange, I puzzled over it for ages, but I did confirm it with a simulation. I'll post it to pastebin when I have a chance to tidy the code up (it's in R) so you can try it for yourself.

Thanks for your interest, Meni :)






Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 03, 2012, 02:53:21 PM
The whole discussion about SMPPS is based on:

The assumption that this is true:
Quote
If the pool stays in a large negative buffer long enough, it is a likely that all miners will leave, and the pool will fail.
Which we ALREADY know to be FALSE.

You're arguing about the the conclusion which I posted here without reading the materiel upon which it was based. Otherwise you'd try to prove your point instead of presenting it as a magical fait accompli.

Please read the actual post (http://organofcorti.blogspot.com.au/2012/04/32-risks-of-smpps.html), Kano. I think you mean well but if you aren't reading the post then I can't argue with you about it. You just come off seeming like a clever chatbot that repeats itself.



Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: kano on April 04, 2012, 12:43:56 AM
The whole discussion about SMPPS is based on:

The assumption that this is true:
Quote
If the pool stays in a large negative buffer long enough, it is a likely that all miners will leave, and the pool will fail.
Which we ALREADY know to be FALSE.

You're arguing about the the conclusion which I posted here without reading the materiel upon which it was based. Otherwise you'd try to prove your point instead of presenting it as a magical fait accompli.

Please read the actual post (http://organofcorti.blogspot.com.au/2012/04/32-risks-of-smpps.html), Kano. I think you mean well but if you aren't reading the post then I can't argue with you about it. You just come off seeming like a clever chatbot that repeats itself.


I've read the actual post twice already, ok 3 times now :P

The rest is based on the concept that over infinite time, during that infinite time, the value of the buffer can be any size - negative or positive - but taking that point to a specific case.

Which of course is true.
(though you didn't seem to bother to point out that it can be both positive and negative in the same way :P)

That is of course true of DGM also.
The DGM argument is that the pool will shutdown before that happens and the pool does not need to disclose the buffer.
I see that as a conflicting statement in itself.

Again you also make the assumption at the end
Quote
A strategic miner will only mine at an SMPPS pool when it has a positive buffer.
Which is in the least questionable as to it's relevance ...


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: kano on April 04, 2012, 01:19:43 AM
Quote
If the pool stays in a large negative buffer long enough, it is a likely that all miners will leave, and the pool will fail.
Which we ALREADY know to be FALSE.

Wait... what?  How on earth do you come to this conclusion?  Arsbitcoin is a prime example of why it's true.  The buffer is negative, it's never going to be repaid, the miners who are owed money are screwed.  The few "hangers on" that are still around are the same people who buy lottery tickets, except in this case, they are hoping to win the lottery for the money they already earned, not extra income.
Doesn't seem that way.
People keep mining there, the issue seems to be that BT is not maintaining the pool, not that the buffer is negative.
More people were mining on Ars with an approx -1000 buffer than were mining on EMC at some points.
The pool stopped so that's when mining stopped, not when the buffer got to -1000

Quote
Quote
Meni loves to say that "there is no buffer" so I take that to mean that when I look at a DGM pool that is paying more than 50BTC per block in a bad luck streak, that pool is in fact in serious trouble and about to fail?
Since there is no buffer, how can they pay more than 50BTC per block?

Meni's comments aside, speaking from experience, I can tell you there is no meaningful "buffer."  As Meni has already explained, it comes out of the operators pocket - in my case, it comes out of my pocket. I refill my pocket on short blocks and pay it out on long blocks.  The total "buffer" that ever gets into the negative is no more than a few BTC at the worst of times.  If you consider 3 or 4 BTC to be a lot/large, well then I stand corrected, but to me, 1200 BTC is tending towards LARGE and 5 BTC is tending towards SMALL, especially when speaking of a pool buffer.

Those DGM pools paying substantially more than 50 BTC per block are doing so out of the operators pockets, not out of any "buffer."  Each round, you are paid what you earned, no buffering required.  I could close up EMC's shop today and cash everyone out for full payment and be right about even - specifically because I don't carry an arbitrary buffer.
So what are the grounds when you would close EMC?
A few weeks back DeepBit had 26 blocks where the average shares per block (for 26 blocks) was ~200%
What debt for you would that represent with your EMC numbers?



Aside: there is also the argument about delaying payout due to a negative buffer, but on DGM full payout is always delayed, on SMPPS payout is only delayed when the buffer is negative.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Inaba on April 04, 2012, 02:25:43 AM
Full payout is never delayed, unless you count the "run-up" period (on EMC, it's 7 blocks), which is paid out on the run-down period (again, another approximately 7 blocks)... otherwise, you are paid in full each block.

As far as what grounds would I close EMC if the buffer were too negative?  Is that the question?  I would say -100 BTC, which would mean we would need a run of 200%+ blocks for over 200 blocks to even get vaguely close to -100BTC.  I just pull -100BTc out of my ass, BTW, just to prove a point.  I probably wouldn't even close it then, because I can cover that out of my pocket.  Although, if we did end up with a 200 block unlucky streak I would seriously be considering closing up shop, but not because of a negative buffer - I would pay whatever was negative out of my pocket to close up shop immediately on the 200th unlucky block.  Heck, if anyone was still even mining after the first 100 unlucky blocks I would be absolutely shocked.

My point is, it's virtually impossible with the proper parameters to have a negative buffer that has meaning... when we are talking about single digit BTC values, it's really meaningless in a pool context.

As far as people that keep mining on Ars is evidence that they don't understand the system.  Who understands how SMPPS works and is still mining there?  I'd like someone to speak up as to why you'd do such a thing?  There is nothing I can think of that would compel me to mine on a pool that has a negative buffer, where the best I can expect is to eventually, maybe, get what I am owed.  It makes absolutely no sense.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: cablepair on April 04, 2012, 02:29:21 AM
top TRUST WORTHY Pools
(not in any kind of order)

Slush
BTCGuild
Eclipse
Eligius
Ozcoin
P2pPool

if you voted that you do not trust any of these pools you are just plain clueless.



Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Graet on April 04, 2012, 02:38:55 AM
top TRUST WORTHY Pools
(not in any kind of order)

Slush
BTCGuild
Eclipse
Eligius

if you voted that you do not trust any of these pools you are just plain clueless.


I'm lost
p2Pool is the clear winner, followed by EMC and Ozcoin. Well done to all in the top three pools - you've gained significantly more trust in the community than others. from https://bitcointalk.org/index.php?topic=66026.msg806445#msg806445
or are you referring to the current "untrusted" poll?


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: cablepair on April 04, 2012, 02:48:17 AM
top TRUST WORTHY Pools
(not in any kind of order)

Slush
BTCGuild
Eclipse
Eligius
Ozcoin

if you voted that you do not trust any of these pools you are just plain clueless.


I'm lost
p2Pool is the clear winner, followed by EMC and Ozcoin. Well done to all in the top three pools - you've gained significantly more trust in the community than others. from https://bitcointalk.org/index.php?topic=66026.msg806445#msg806445
or are you referring to the current "untrusted" poll?

let me edit

this is MY OPINION - not result of any poll
(sure no one cares but here you go anyway)

and yes P2POOL goes without saying - but its not really a pool is it

ozcoin is trustworthy also graet ;)

I just dont understand why some of these people voted - does not make sense to me so I had to share my opinion

I mean seriously 15% think Slush is not trustworthy? Come on...

this poll is FUBAR
 



Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: kano on April 04, 2012, 03:00:02 AM
Full payout is never delayed, unless you count the "run-up" period (on EMC, it's 7 blocks), which is paid out on the run-down period (again, another approximately 7 blocks)... otherwise, you are paid in full each block.
Yes of course that is what I am referring to :)
Whatever that number of lead in blocks is, you will never be paid in full for those blocks until either after you stop mining on the pool, or if the pool closes down and pays out the debt in full.

Quote
As far as what grounds would I close EMC if the buffer were too negative?  Is that the question?  I would say -100 BTC, which would mean we would need a run of 200%+ blocks for over 200 blocks to even get vaguely close to -100BTC.  I just pull -100BTc out of my ass, BTW, just to prove a point.  I probably wouldn't even close it then, because I can cover that out of my pocket.  Although, if we did end up with a 200 block unlucky streak I would seriously be considering closing up shop, but not because of a negative buffer - I would pay whatever was negative out of my pocket to close up shop immediately on the 200th unlucky block.  Heck, if anyone was still even mining after the first 100 unlucky blocks I would be absolutely shocked.

My point is, it's virtually impossible with the proper parameters to have a negative buffer that has meaning... when we are talking about single digit BTC values, it's really meaningless in a pool context.

As far as people that keep mining on Ars is evidence that they don't understand the system.  Who understands how SMPPS works and is still mining there?  I'd like someone to speak up as to why you'd do such a thing?  There is nothing I can think of that would compel me to mine on a pool that has a negative buffer, where the best I can expect is to eventually, maybe, get what I am owed.  It makes absolutely no sense.

I'm just curious about how DMG works in reality rather than reading Meni's comments that certainly have an element of self bias in them :)

I mine on Ozcoin that is DGM, I'm not saying "throw it out, it's no good", however, but making comparisons that are dubious to promote DGM is not acceptable either.

Edit: though I could point out something else that everyone seems to be ignoring:
If someone mines on Ars and has an unpaid debt, they may consider it in their own interest to keep mining ...

Edit2: and if they had been paying 1% into the buffer to cover orphans, what would their balance be now :)


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: eleuthria on April 04, 2012, 03:25:43 AM
I just dont understand why some of these people voted - does not make sense to me so I had to share my opinion

I mean seriously 15% think Slush is not trustworthy? Come on...

this poll is FUBAR

To be fair, any poll in the Pools forum is pretty much just a test of how rabid the users of pools are.  The polls on this thread have been popularity contests (or how loud the haters were), rather than their trustworthiness (or lack thereof).  Even polls specifically for a pool on their own thread tend to give pretty awkward results that don't give an accurate representation of the overall pool users.

Examples:
  • Slush has 15 votes.  Not the largest number of votes, but very surprising.  Slush is the ORIGINAL mining pool.  The only bad news I've ever seen on Slush's pool was the Linode hack, which didn't impact users, and was not Slush's fault.
  • Deepbit has more "untrustworthy" votes than Slush & BTC Guild combined.  Yet their track record is spotless, and they are the second oldest (still running) pool.  Only bad thing Deepbit has going are the people convinced that Deepbit is a threat, which is a vocal minority.
  • P2Pool - This shouldn't have a single vote for being "untrustworthy".  It's a completely open source project that has no pool operator.  It shouldn't even be on the poll, since the entire point of p2pool is no trust is needed.

Those aren't the only ones I disagree with, but they're the 3 most obvious.  The two oldest pools who I have never seen do their users wrong, should not be getting remotely as many votes as they have.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: rjk on April 04, 2012, 03:33:08 AM
I just dont understand why some of these people voted - does not make sense to me so I had to share my opinion

I mean seriously 15% think Slush is not trustworthy? Come on...

this poll is FUBAR

To be fair, any poll in the Pools forum is pretty much just a test of how rabid the users of pools are.  The polls on this thread have been popularity contests (or how loud the haters were), rather than their trustworthiness (or lack thereof).  Even polls specifically for a pool on their own thread tend to give pretty awkward results that don't give an accurate representation of the overall pool users.
+1, pretty much. It's really difficult to get an accurate representative sample of users here.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 04, 2012, 03:14:58 PM

The second chart, giving the PDF of the pool debt after n rounds, looks completely wrong, I have no idea how you got that. The pool debt, assuming constant difficulty, is (X/D - n)*B, where X is the number of shares it took to get n blocks, which is distributed negative binomial. So it's really a shifted negative binomial, which doesn't have the cusp we see.

Meni, I couldn't find my notes so I've reproduced what I could from memory. I might have left out a step or two, but the results hold, and you can test them out if you like.

Let x1, x2, x3, …, xn be independent geometrically distributed random variables with the same probability, p.

Let Sj be the sum of x1 + x2 + x3 + ….+xj so that:

    S1 = x1
    S2 = S1 + x2
    S3 = S2 + x3
       …
    Sj = Sj-1 + xj
       …..
    Sn = Sn-1 + xn


Since x1, x2, x3, … ,xj …,xn are geometrically distributed, Sj is a random variable from a shifted negative binomial distribution with target number of successful trials = j, with density:

Pr(X=Sj) =  Γ(x+j)/(Γ(j) * Γ(x+1)) * p^j * (1-p)^x, so it follows:

Pr(X∈(S1, S2, S3, …, Sj, … ,Sn))
    = sum from 1 to n [Γ(x+j)/(Γ(j) * Γ(x+1)) * p^j * (1-p)^x]/n (1)

So the probability density of a cumulative sum of independent geometrically distributed random variables is given by (1) and tends toward a uniform distribution on [1,j/p].

Similarly, if x1, x2, x3, … ,xj …,xn are geometrically distributed with mean = 0, then Sj is a random variable from a shifted negative binomial distribution with target number of successful trials = j and mean = 0. Since the mean is 0, the component probability densities can be visualised as below.

http://i.imgur.com/cefNNl.png (http://imgur.com/cefNN)

The probability densities Pr(X=Sj) and Pr(X∈(S1, S2, S3, …, Sj, … ,Sn)):

Pr(X=Sj) =  Γ(x+(1-p)*j/p+j)/(Γ(j) * Γ(x+(1-p)*j/p+1)) * p^j * (1-p)^((x+(1-p)*j/p))

Pr(X∈(S1, S2, S3, …, Sj, … ,Sn))
    = sum from 1 to n [Γ(x+(1-p)*j/p+j)/(Γ(j) * Γ(x+(1-p)*j/p+1)) * p^j * (1-p)^((x+(1-p)*j/p))]/n (2)

Although (2) does not exist in a closed form, it can be calculated and charts of the distribution for n=100, n=1000 and n=10000 for p=1e-06 are shown below. This distribution does not seem to tend toward a known distribution for large n or very small p.

http://i.imgur.com/hyMu4l.png (http://imgur.com/hyMu4)


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 04, 2012, 03:29:37 PM
The whole discussion about SMPPS is based on:

The assumption that this is true:
Quote
If the pool stays in a large negative buffer long enough, it is a likely that all miners will leave, and the pool will fail.
Which we ALREADY know to be FALSE.

You're arguing about the the conclusion which I posted here without reading the materiel upon which it was based. Otherwise you'd try to prove your point instead of presenting it as a magical fait accompli.

Please read the actual post (http://organofcorti.blogspot.com.au/2012/04/32-risks-of-smpps.html), Kano. I think you mean well but if you aren't reading the post then I can't argue with you about it. You just come off seeming like a clever chatbot that repeats itself.


I've read the actual post twice already, ok 3 times now :P

The rest is based on the concept that over infinite time, during that infinite time, the value of the buffer can be any size - negative or positive - but taking that point to a specific case.

No, it's not. It's about how probabilities of the risk of an arbitrarily sized negative buffer occurring before n rounds can be calculated. It was to give miners a better idea of the risks involved with SMPPS.
Quote
Which of course is true.
(though you didn't seem to bother to point out that it can be both positive and negative in the same way :P)
True, but hardly useful when discussing the risks of losing shares. A positive buffer is not good for a pool in the way a negative buffer is bad for a pool. Maturity time won't get any less as the buffer becomes more positive, but maturity time will increase significantly as the buffer becomes more negative.

Quote
That is of course true of DGM also.
No, I don't think it is. I'll believe you if you can prove it using Meni's DGM functions though.
Quote
The DGM argument is that the pool will shutdown before that happens and the pool does not need to disclose the buffer.
I see that as a conflicting statement in itself.
As they say on wikipedia, "says who?".
Quote
Again you also make the assumption at the end
Quote
A strategic miner will only mine at an SMPPS pool when it has a positive buffer.
Which is in the least questionable as to it's relevance ...

It was very relevant. If some miners leave the pool when the buffer is negative, then the pool hashrate declines and while maturity time in terms of blocks will stay the same, with a reduced hashrate the number of days (or hours or weeks) until payment increases. The greater the increase, the fewer the number of miners that are prepared to wait so long for payment. Miners leave, the hashrate decreases again. And so on.

I'm sorry the post was misleading to you. I hope no one else interpreted it that way.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 04, 2012, 03:34:00 PM
I just dont understand why some of these people voted - does not make sense to me so I had to share my opinion

I mean seriously 15% think Slush is not trustworthy? Come on...

this poll is FUBAR

To be fair, any poll in the Pools forum is pretty much just a test of how rabid the users of pools are.  The polls on this thread have been popularity contests (or how loud the haters were), rather than their trustworthiness (or lack thereof).  Even polls specifically for a pool on their own thread tend to give pretty awkward results that don't give an accurate representation of the overall pool users.
+1, pretty much. It's really difficult to get an accurate representative sample of users here.

+1 here too. A page back I found out that some people were voting randomly just to see the results. Plus grudge holders have a reason to vote negatively even if it's unfair, or if they don't like the pool and aren't voting on trust but just to get bad ratings for a pool. So I'm sorry to all who voted, I'm killing the poll until I can find a better way to get an idea about trust. Maybe I just have to only phrase it positively, like the first one. Ideas?


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Meni Rosenfeld on April 04, 2012, 04:51:55 PM
Pr(X∈(S1, S2, S3, …, Sj, … ,Sn))
    = sum from 1 to n [Γ(x+j)/(Γ(j) * Γ(x+1)) * p^j * (1-p)^x]/n (1)
What is this quantity supposed to represent? It looks like you consider it to be meaningful but I just don't see it. Sn itself is the number of shares submitted in n rounds, isn't this what you're trying to analyze?

Also, if by the LHS you mean "probability that X is in the set {S1,..., Sn}", why would it be equal to the RHS?


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: eleuthria on April 04, 2012, 05:11:27 PM
+1 here too. A page back I found out that some people were voting randomly just to see the results. Plus grudge holders have a reason to vote negatively even if it's unfair, or if they don't like the pool and aren't voting on trust but just to get bad ratings for a pool. So I'm sorry to all who voted, I'm killing the poll until I can find a better way to get an idea about trust. Maybe I just have to only phrase it positively, like the first one. Ideas?

In my opinion, it doesn't matter how the poll is phrased because your sample size is a narrow subset of overall pool users.  You end up with the problem of vocal minorities being the primary respondents to your poll.  While in theory that shouldn't skew results, it's quite obvious in the last two polls [and this one especially] that you end up with slants that don't seem to reflect reality.

Similarly, all it takes is one pool mentioning the poll in their thread, their website, or their chatroom to completely swing the results of the poll in their favor.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: kano on April 04, 2012, 08:51:05 PM
...
Quote
Which of course is true.
(though you didn't seem to bother to point out that it can be both positive and negative in the same way :P)
True, but hardly useful when discussing the risks of losing shares. A positive buffer is not good for a pool in the way a negative buffer is bad for a pool. Maturity time won't get any less as the buffer becomes more positive, but maturity time will increase significantly as the buffer becomes more negative.

Quote
That is of course true of DGM also.
No, I don't think it is. I'll believe you if you can prove it using Meni's DGM functions though.
Quote
The DGM argument is that the pool will shutdown before that happens and the pool does not need to disclose the buffer.
I see that as a conflicting statement in itself.
As they say on wikipedia, "says who?".
...
Meni.

I'll find the posts he said this later if I really need to?


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 05, 2012, 03:50:05 AM
+1 here too. A page back I found out that some people were voting randomly just to see the results. Plus grudge holders have a reason to vote negatively even if it's unfair, or if they don't like the pool and aren't voting on trust but just to get bad ratings for a pool. So I'm sorry to all who voted, I'm killing the poll until I can find a better way to get an idea about trust. Maybe I just have to only phrase it positively, like the first one. Ideas?

In my opinion, it doesn't matter how the poll is phrased because your sample size is a narrow subset of overall pool users.  You end up with the problem of vocal minorities being the primary respondents to your poll.


Not just vocal minorities, but (hopefully) miners and pool ops who are concerned with promoting reliability and accountability for pools. I had hoped there'd be fewer barrows to push, but oh well.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 05, 2012, 03:52:49 AM
...
Quote
Which of course is true.
(though you didn't seem to bother to point out that it can be both positive and negative in the same way :P)
True, but hardly useful when discussing the risks of losing shares. A positive buffer is not good for a pool in the way a negative buffer is bad for a pool. Maturity time won't get any less as the buffer becomes more positive, but maturity time will increase significantly as the buffer becomes more negative.

Quote
That is of course true of DGM also.
No, I don't think it is. I'll believe you if you can prove it using Meni's DGM functions though.
Quote
The DGM argument is that the pool will shutdown before that happens and the pool does not need to disclose the buffer.
I see that as a conflicting statement in itself.
As they say on wikipedia, "says who?".
...
Meni.

I'll find the posts he said this later if I really need to?

When making a statement that relies on your interpretation of someone's else statements or even quoting them, a citation is vital to allow a reader to come to the same decision (or not) for themselves.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: kano on April 05, 2012, 04:19:16 AM
...
Quote
Which of course is true.
(though you didn't seem to bother to point out that it can be both positive and negative in the same way :P)
True, but hardly useful when discussing the risks of losing shares. A positive buffer is not good for a pool in the way a negative buffer is bad for a pool. Maturity time won't get any less as the buffer becomes more positive, but maturity time will increase significantly as the buffer becomes more negative.

Quote
That is of course true of DGM also.
No, I don't think it is. I'll believe you if you can prove it using Meni's DGM functions though.
Quote
The DGM argument is that the pool will shutdown before that happens and the pool does not need to disclose the buffer.
I see that as a conflicting statement in itself.
As they say on wikipedia, "says who?".
...
Meni.

I'll find the posts he said this later if I really need to?

When making a statement that relies on your interpretation of someone's else statements or even quoting them, a citation is vital to allow a reader to come to the same decision (or not) for themselves.
Top of the previous page here :P
https://bitcointalk.org/index.php?topic=66026.180

Then of course also our earlier discussion that he linked:
https://bitcointalk.org/index.php?topic=39497.msg807765#msg807765 ... etc. after that post


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Meni Rosenfeld on April 06, 2012, 07:57:02 AM
Organ, any news about the pdf issue (https://bitcointalk.org/index.php?topic=66026.msg835281#msg835281)?


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 06, 2012, 11:57:10 AM
Sorry, Meni, I wasn't ignoring you just wanted to make sure I had a clearer reply. Also looked for my notes again and failed. Further, realised I should be using the term "probability mass function' instead of 'probability density function' but I'll continue as I started for consistency.

It might be better if I just described what I'm proving, since I made some mistakes there which muddied the water.

The probability that Sn, the sum of n geometrically and identically distributed random variables (x1 to xn) will be some amount X is a negative binomially distributed variable where the probability density is = Γ(X+n)/(Γ(n) * Γ(X+1)) * p^n * (1-p)^X

Similarly for Sn-1, the probability that this sum will equal some variable X:
= Γ(X+(n-1))/(Γ(n-1) * Γ(X+1)) * p^(n-1) * (1-p)^X

and so on for all S.


The set (S1, S2, S3, ...,Sn) is the cumulative sum of the geometrically distributed variables. The probability that X is to be found somewhere in that set is given by the sum of all probabilities that X = some S divided by n:

Pr(X∈(S1, S2, S3, … ,Sn))
    = sum from 1 to n [Γ(X+n)/(Γ(n) * Γ(X+1)) * p^n * (1-p)^X]/n (1)

(note I've fixed it by removing the erroneous j's and changing x's to X's)

This is where my lack of notes fails me. I can't for the life of me remember where I came across this way of summing probabilities, or how I applied it. It doesn't appear to be the result of a convolution. Maybe you know?

Intuitively I see that, since the sum from x = -inf to +inf of each separate pdf = 1, and that the sum from x = -inf to +inf of the individual pdfs for S1 to Sn will be n, then dividing by n is 'normalising' (not the right word, but I hope you know what I mean) the probability to give (1), or if you like, providing the mean probability.

The mean of each separate pdf increases with increasing n and the 'normalised' probability approaches a uniform probability density from 0 to the mean of Sn as n gets very large.

(2) Simply defines the mean = 0 for each separate pdf and then follows the same logic.

Does this make sense to you? I remember the solution making complete sense to me at the time.

Interestingly, running simulations for uniformly randomly generated variables with a mean of 0 and negative binomially distributed variables with a mean of 0 and large n, give similar looking curves to each other, although on slightly different scales even with the same p.



Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Meni Rosenfeld on April 07, 2012, 09:14:56 PM
Sorry, Meni, I wasn't ignoring you just wanted to make sure I had a clearer reply.
Didn't think so, just wanted to make sure my query wasn't lost in the fold.

Also looked for my notes again and failed. Further, realised I should be using the term "probability mass function' instead of 'probability density function' but I'll continue as I started for consistency.
You can use the continuous approximation, and then you're talking about the pdf of the Erlang distribution.

The probability that Sn, the sum of n geometrically and identically distributed random variables (x1 to xn) will be some amount X is a negative binomially distributed variable where the probability density is = Γ(X+n)/(Γ(n) * Γ(X+1)) * p^n * (1-p)^X

Similarly for Sn-1, the probability that this sum will equal some variable X:
= Γ(X+(n-1))/(Γ(n-1) * Γ(X+1)) * p^(n-1) * (1-p)^X

and so on for all S.
No arguments there. (There could be off-by-one errors but that's immaterial.)

The set (S1, S2, S3, ...,Sn) is the cumulative sum of the geometrically distributed variables.The probability that X is to be found somewhere in that set is given by the sum of all probabilities that X = some S divided by n:

Pr(X∈(S1, S2, S3, … ,Sn))
    = sum from 1 to n [Γ(X+n)/(Γ(n) * Γ(X+1)) * p^n * (1-p)^X]/n (1)
Here is where things start to get horribly, horribly wrong. Both in what you're trying to get, and how you get it.

First, I'll say again: If what you want is the balance after n rounds (which is what the text implies), it is simply (n - Sn/D)*B. That's it. Sn is the number of shares the pool had to pay. Any other derivation is nonsense. So I can only assume you really are trying to figure out the probability that a balance of X will be reached at some point within the first n rounds. Even if this is the case, the above quantity is misguided.

Let's begin with this formula. It is absolutely, and obviously, wrong.

The event X∈(S1, S2, S3, … ,Sn) contains the even X=Sj for any j. Therefore its probability must be at least the probability that X=Sj, which is the pmf of Sj at point X. So the probability must be greater than all the pmfs, so it can't be equal to their average.

The one place where the average of pmfs is meaningful is when you have a mixture variable. For example, if you have a variable Sr which is generated by first choosing a uniform random integer j between 1 and n and then setting it equal to Sj, its pmf will be given by this formula. But such a variable has no relevance whatsoever to the problem at hand.

If you do want to find the probability that X will be in the set, you need to treat this as a union (logic or) of the equalities X=Sj, which can be expanded with the inclusion-exclusion principle. I don't know if the expansion has a nice closed form in this case, I think it's extremely messy because of the dependencies between the Sj's.

So now maybe you can understand why the quantity Pr(X∈(S1, S2, S3, … ,Sn)) is not meaningful. It is not a pmf of anything. The total area under this curve is more than 1, and you can't take areas under the curve for any insight. The events 0∈(S1, S2, S3, … ,Sn) and 1∈(S1, S2, S3, … ,Sn) are not mutually exclusive, so you can't disjunct them by simply adding the probabilities.

You need to define the problem as "what is the probability that the balance will reach X or lower at some point within the first n rounds" (taking into account, obviously, the shift of +B for every round). This is a problem that should be solvable with recursion, for either the exact problem or for an approximation for it.

This is where my lack of notes fails me. I can't for the life of me remember where I came across this way of summing probabilities, or how I applied it. It doesn't appear to be the result of a convolution. Maybe you know?

Intuitively I see that, since the sum from x = -inf to +inf of each separate pdf = 1, and that the sum from x = -inf to +inf of the individual pdfs for S1 to Sn will be n, then dividing by n is 'normalising' (not the right word, but I hope you know what I mean) the probability to give (1), or if you like, providing the mean probability.

The mean of each separate pdf increases with increasing n and the 'normalised' probability approaches a uniform probability density from 0 to the mean of Sn as n gets very large.

(2) Simply defines the mean = 0 for each separate pdf and then follows the same logic.

Does this make sense to you? I remember the solution making complete sense to me at the time.
Not in the least. I'm not following your train of thought because you're not explaining what, why and how you are trying to get.

Interestingly, running simulations for uniformly randomly generated variables with a mean of 0 and negative binomially distributed variables with a mean of 0 and large n, give similar looking curves to each other, although on slightly different scales even with the same p.
Probably the CLT at work.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 09, 2012, 12:49:31 PM
The set (S1, S2, S3, ...,Sn) is the cumulative sum of the geometrically distributed variables.The probability that X is to be found somewhere in that set is given by the sum of all probabilities that X = some S divided by n:

Pr(X∈(S1, S2, S3, … ,Sn))
    = sum from 1 to n [Γ(X+n)/(Γ(n) * Γ(X+1)) * p^n * (1-p)^X]/n (1)
Here is where things start to get horribly, horribly wrong. Both in what you're trying to get, and how you get it.

First, I'll say again: If what you want is the balance after n rounds (which is what the text implies), it is simply (n - Sn/D)*B. That's it. Sn is the number of shares the pool had to pay. Any other derivation is nonsense. So I can only assume you really are trying to figure out the probability that a balance of X will be reached at some point within the first n rounds. Even if this is the case, the above quantity is misguided.

Let's begin with this formula. It is absolutely, and obviously, wrong.

The event X∈(S1, S2, S3, … ,Sn) contains the even X=Sj for any j. Therefore its probability must be at least the probability that X=Sj, which is the pmf of Sj at point X. So the probability must be greater than all the pmfs, so it can't be equal to their average.

The one place where the average of pmfs is meaningful is when you have a mixture variable. For example, if you have a variable Sr which is generated by first choosing a uniform random integer j between 1 and n and then setting it equal to Sj, its pmf will be given by this formula. But such a variable has no relevance whatsoever to the problem at hand.

If you do want to find the probability that X will be in the set, you need to treat this as a union (logic or) of the equalities X=Sj, which can be expanded with the inclusion-exclusion principle. I don't know if the expansion has a nice closed form in this case, I think it's extremely messy because of the dependencies between the Sj's.
Quote
I'm not following your train of thought because you're not explaining what, why and how you are trying to get.

Yes, my aim was to find the probability that a cumulative sum or balance of X or lower will be reached before n rounds. Firstly to define a pdf, compare with simulated results, and then sum the pdf to a cdf.

Quote
So now maybe you can understand why the quantity Pr(X∈(S1, S2, S3, … ,Sn)) is not meaningful. It is not a pmf of anything. The total area under this curve is more than 1, and you can't take areas under the curve for any insight. The events 0∈(S1, S2, S3, … ,Sn) and 1∈(S1, S2, S3, … ,Sn) are not mutually exclusive, so you can't disjunct them by simply adding the probabilities.

After more reading (and learning) I have to say I agree with you. So I really don't know how the simulation matches the 'mean negative binomial pdf' (which you show isn't meaningful). An error in concept maybe? I wrote a  simpler simulation (http://pastebin.com/7ZBErb3d) with no special libraries required, commented, doesn't make shortcuts and should be possible to follow if you have time.

The sim works as follows: Take n geometrically distributed random variables with the same p, subtracting 1/p from each variable so the expectation of each variable is 0. The take the cumulative sum of the variables. For example let p = 10^-6 and n = 10:

Code:
Geometrically distributed random variables:
 [1] 1182417 3279310  482644  550187  404841  183692 1106328 1663533 1716093 5213483

Geometrically distributed random variables minus 10^6
 [1]  182417 2279310 -517356 -449813 -595159 -816308  106328  663533  716093 4213483

Cumulative sum of (geometrically distributed random variables minus 10^6)
 [1]  182417 2461727 1944371 1494558  899399   83091  189419  852952 1569045 5782528

Now make m iterations of the cumulative sums to n and plot the histogram or sample density for the entire m*n samples. For example, if m=10, you'd plot the density or histogram of:

Code:
         m=1      m=2      m=3      m=4      m=5      m=6      m=7     m=8      m=9     m=10
n=1  182417  -567342  -844019  -992762  -965404  -621776  -163689   34795  -790737   219609
n=2  2461727 -1328229 -1116118 -1352713 -1693465  -574916  -744127 -769836  -966007  -259975
n=3  1944371  -370917 -1951890 -1044311 -2358626 -1014358 -1149823  755121   125528 -1139511
n=4  1494558  1608634 -1645397    16903 -1972138 -1336607 -1674819  318716  -393838  1008137
n=5  899399   1333825  1530411   -96895 -2960904 -1578059 -2312004  299624 -1354225   897368
n=6  83091     414391  4826197     9793 -3182523 -2087554 -2619065 -374199 -1590228    91992
n=7  189419    568167  4014970  -681510  -372781 -2077967 -1813568 -159466 -1914558  -357282
n=8  852952    295560  3170795 -1481067  -239713 -2468778  -226886 -342762 -2101092 -1334599
n=9  1569045   683593  3718213 -2315456  1281720 -1220656  -684308   56256 -2411408  -913902
n=10 5782528   169347  2806338 -3215373  1084351 -1431807 -1526684 -254378 -3073730 -1742821
where n = the number of "fails" before each success, and m = the iteration number (the 'mth' time a sample of n was taken)

This gives a density plot or histogram which corresponds to the probability density of the distribution from which it came. If I then multiply the cumulative sums by B/D, I can then create a cdf and find, as you state, "the probability that the balance will reach X or lower at some point within the first n rounds".

So, at some point I somehow I came up with the supposed "mean negative binomial density" and wrote such a simulation which appeared to 'confirm' it. The sim I linked to plots the simulated cumulative sum densities against the "mean negative binomial pdf" for comparison, as below.

http://i.imgur.com/9lUKll.png (http://imgur.com/9lUKl)

If you click on the graph for a closer view, you'll see that the densities match very well.

I'm aware that random variable based simulations have limitations, but such a clear correlation is unlikely to be random. Since the "mean negative binomial pdf" is not just wrong but not meaningful, have I misunderstood what I was finding or looking for in the simulation?

Secondly, and possibly more interestingly, after your comment about the clt I looked for and found that for large n (larger than ~ 100) the simulated densities match skewed generalized error distributions quite well. Since I don't think I'd be able to use the inclusion-exclusion principle easily (tbh a the method to achieve a sum to a non-specific 'n' for this principle is beyond me), I'd like to be able to replace the error on the blog with a skewed GED instead as a model. However I'm unwilling to do more work on it until I can get a consensus (well, your advice :)) as to whether I've simulated the cumulative sum (balance) at all times before n rounds.

I know I'm asking a lot for you to go through the sim and look at the results. I wish I could say "I'll owe you one" but I'll just settle for being grateful for either a confirmation that the simulation works or a reason why it doesn't. Right now, I'm just confused.  ???

Thanks for your help.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Meni Rosenfeld on April 09, 2012, 01:36:47 PM
@organ: I don't really speak R, but as far as I can see, your simulation matches the mean pmf because you simulated the mixture variable I talked about rather than your target quantity. That's also evident in your verbal description of the simulation. But I think I finally understand the source of your confusion.

The variable we are interested in is M="the minimum balance over the next n rounds". Its cmf at point X is the probability that the minimum is at most X, which means that the balance reaches X or lower at some point. So once we find the pmf of M, we can calculate the chance of hitting any X by the area under the curve.

To simulate the pmf, we need of course to make many instantiations of the random process, and add +1 to the empirical weight of the resulting value of M. What is the value of M? The minimum among all the values of the (shifted) cumulative sum. But instead of adding +1 to the minimum, it seems you added +1 (or +1/n after normalization) to all the values in the cumulative sum. If this is fixed you'll get a proper simulation.

Instead of a generative simulation, you can also find the cmf by approximating the process as a simple random walk, and solving (numerically or, if possible, symbolically) the following recursion (in Mathematica syntax):
Code:
f[a_, n_] :=  f[a, n] = If[a < 0, 1, If[a >= n, 0, 0.5 (f[a + 1, n - 1] + f[a - 1, n - 1])]]

Using this, I find that f[20, 1000], the probability of going below -1000 BTC (20 blocks worth) in 1000 rounds, is approximately 50.67%.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 09, 2012, 03:03:02 PM
I'm going to make this quick because I'm being called to bed but you got me thinking and I wanted to get a post out in reply, so sorry if my response seems terse and not well thought out.

@organ: I don't really speak R, but as far as I can see, your simulation matches the mean pmf because you simulated the mixture variable I talked about rather than your target quantity. That's also evident in your verbal description of the simulation. But I think I finally understand the source of your confusion.

The variable we are interested in is M="the minimum balance over the next n rounds". Its cmf at point X is the probability that the minimum is at most X, which means that the balance reaches X or lower at some point. So once we find the pmf of M, we can calculate the chance of hitting any X by the area under the curve.

 "the minimum balance over the next n rounds" is what I derived in the second part of the section using Erdos and Kac's probability limit theorem I.

What I wanted to do as a first step was find the probability of any given balance or less before n rounds (after finding the area under the density curve), so that after the second derivation I could compare it to the probability of a given minimum occurring. Is this the mixture variable?

Quote
To simulate the pmf, we need of course to make many instantiations of the random process, and add +1 to the empirical weight of the resulting value of M. What is the value of M? The minimum among all the values of the (shifted) cumulative sum. But instead of adding +1 to the minimum, it seems you added +1 (or +1/n after normalization) to all the values in the cumulative sum. If this is fixed you'll get a proper simulation.
I'm not following this bit, even accounting for the aim of finding the minimum. I'm thinking in terms of the cumulative sums being:

[(Geom var1 - 1/p)], [(Geom var1 - 1/p) + (Geom var2 - 1/p)], [(Geom var1 - 1/p) + (Geom var2 - 1/p)+ (Geom var2 - 1/p)] , .......

where I've made the share-debit a (non intuitively) positive number. I'm subtracting 1/p (or D) each round. What did you mean by the +1 or +1/n? I think there's a step I'm missing.

Quote
Instead of a generative simulation, you can also find the cmf by approximating the process as a simple random walk, and solving (numerically or, if possible, symbolically) the following recursion (in Mathematica syntax):
Code:
f[a_, n_] :=  f[a, n] = If[a < 0, 1, If[a >= n, 0, 0.5 (f[a + 1, n - 1] + f[a - 1, n - 1])]]

Using this, I find that f[20, 1000], the probability of going below -1000 BTC (20 blocks worth) in 1000 rounds, is approximately 50.67%.

I'm going to have to spend a bit of time on this tomorrow since I only know enough mathematica to get by on wolframaplha when natural language doesn't work, but thanks for giving me something new to work with which looks to be much quicker than my current method.

Using the probability limits theorem, I find the probability of going below -1000 BTC in 1000 rounds as 52.71%, and not until 1000 BTC at 1050 rounds did it reach 50.66%. Do you think this an acceptable variation given the different methods used, or a problem for me?

Cheers again.



Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Meni Rosenfeld on April 09, 2012, 03:22:03 PM
@organ: I don't really speak R, but as far as I can see, your simulation matches the mean pmf because you simulated the mixture variable I talked about rather than your target quantity. That's also evident in your verbal description of the simulation. But I think I finally understand the source of your confusion.

The variable we are interested in is M="the minimum balance over the next n rounds". Its cmf at point X is the probability that the minimum is at most X, which means that the balance reaches X or lower at some point. So once we find the pmf of M, we can calculate the chance of hitting any X by the area under the curve.
"the minimum balance over the next n rounds" is what I derived in the second part of the section using Erdos and Kac's probability limit theorem I.

What I wanted to do as a first step was find the probability of any given balance or less before n rounds (after finding the area under the density curve), so that after the second derivation I could compare it to the probability of a given minimum occurring. Is this the mixture variable?
You should probably think about it some more. "Reaching a balance of X or less at any point" is equivalent to "the minimum balance is at most X", which is the cmf of M. If you find the pmf of M, you can use it to find the cmf. In any case I'm talking about balance rather than debt, but that's only a matter of flipping the sign.

where I've made the share-debit a (non intuitively) positive number. I'm subtracting 1/p (or D) each round. What did you mean by the +1 or +1/n? I think there's a step I'm missing.
It's just the tallying of the empirical pmf. If I wanted to simulate the probabilities of the results 1-6 on a die roll, I'd have variables num1,... , num6, and I'd add +1 to num1 whenever I roll the die and get a 1. Likewise, you need to tally the number of times each value of M appears. Unless you have a different approach to this?

Using the probability limits theorem, I find the probability of going below -1000 BTC in 1000 rounds as 52.71%, and not until 1000 BTC at 1050 rounds did it reach 50.66%. Do you think this an acceptable variation given the different methods used, or a problem for me?
It's within reason, the model solved with this recursion is not very accurate (and I'm guessing the application of the general theorem also has some error). Maybe a better estimation can be found with a simulation after all, with 100000 iterations of 1000 rounds I get about 50.8% chance of going below -1000 BTC. My code for this is:
Code:
M[] := Min[Accumulate[1 - RandomReal[ExponentialDistribution[1], {1000}]]];
N[Mean[Table[If[M[] <= -20, 1, 0], {1000000}]]]


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 10, 2012, 04:01:40 AM
You should probably think about it some more.
Yes, I'm going to try to explain what I mean in terms of a random walk, I think. But right now I've just got a flashing cursor in my head going "thinking.......thinking.....thinking....". I touch on it briefly below in terms of simulation though.

It's just the tallying of the empirical pmf. If I wanted to simulate the probabilities of the results 1-6 on a die roll, I'd have variables num1,... , num6, and I'd add +1 to num1 whenever I roll the die and get a 1. Likewise, you need to tally the number of times each value of M appears. Unless you have a different approach to this?
After looking at your simulation below, I'm doing much the same thing except in R. In the limit theorem part, the simulation does exactly the same thing, except instead of measuring a specific amount I plot the overall density.

However for the first part, with the cuspy sGED modelable density, I'm not taking a minimum of the cusum of 1000 rounds but collecting all the cusums. I end up with (in this case, and with the number of iterations you use below) a 1000*1000000 matrix of variables. I then plot the density of the variables.

So to emulate this on Mathematica, you would:
1. create a 1000 col *1000000 row array or matrix, name it MAT
2. put Accumulate[RandomReal[ExponentialDistribution[1], {1000}]] - 1] in the first row
3. Iterate step 2 for rows 2 to 1000000
4. not sure this is correct, but try SmoothHistogram(MAT, Automatic, "PDF") and SmoothHistogram(MAT, Automatic, "CDF"). You might need to convert the MAT to a vector first.
5. Repeat for 10, 50, or 100 rounds on a 10 or 50 or 100 column by 1000000 row array for the histogram at a varying number of rounds.

Maybe this makes clear what I'm recording in the simulation (if not the why to which I'll give some more thought).

Maybe a better estimation can be found with a simulation after all, with 100000 iterations of 1000 rounds I get about 50.8% chance of going below -1000 BTC. My code for this is:
Code:
M[] := Min[Accumulate[1 - RandomReal[ExponentialDistribution[1], {1000}]]];
N[Mean[Table[If[M[] <= -20, 1, 0], {1000000}]]]

Your results are very similar to the ones I get in simulation. I've noticed that for larger probabilities the simulation always diverges slightly from the limit theorem result. If you repeated the simulation with fewer rounds or for a less probable -50, the simulation results will agree with the limit theorem error function much more closely. Cool, huh?


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Meni Rosenfeld on April 10, 2012, 09:35:15 AM
Maybe this makes clear what I'm recording in the simulation (if not the why to which I'll give some more thought).
The what has been clear for some time.

As for the why - you only need to provide a clear, verbal, unambiguous description of what you are trying to find. In the second part that is fairly clear and as far as I can tell you have a simulation that serves the purpose (you get the cmf of M, from which you can obtain the pmf if you want). In the first part it looks like you just took a bunch of numbers and mashed them together for no reason. The solution as far as I can see is to just remove the first part.

EDIT: It's actually easy to give a verbal description of the first graphs. The one with the cusp is a graph of "the average % of rounds that end with a balance of exactly X", and the accumulated area under it is "the average % of rounds that end with a balance of at most X". (eg, if there are 1000 rounds and 300 of them end with a balance less than -1000 BTC, the value of the variable will be 0.3 - and the graph lists the average of this over all instantiations). Note that as described these are not a pmf and cmf, though they can be looked at as the pmf of cmf of the mixture variable earlier discussed. And again, what you're looking for is not the average % of rounds that end with at most X, but rather the probability that at least one round ends with at most X.

I've noticed that for larger probabilities the simulation always diverges slightly from the limit theorem result. If you repeated the simulation with fewer rounds or for a less probable -50, the simulation results will agree with the limit theorem error function much more closely. Cool, huh?
If I had to take a wild guess, I'd say it's because the tail of the exponential distribution (corresponding to bad luck) is more similar to normal than the head. If that's true, your observation will be different if you take the negative of each shifted geometric variable (so effectively, instead of looking at hitting a low value, you're looking at the probability to hit a high value).


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 11, 2012, 07:04:20 AM
Maybe this makes clear what I'm recording in the simulation (if not the why to which I'll give some more thought).
The what has been clear for some time.

As for the why - you only need to provide a clear, verbal, unambiguous description of what you are trying to find. In the second part that is fairly clear and as far as I can tell you have a simulation that serves the purpose (you get the cmf of M, from which you can obtain the pmf if you want). In the first part it looks like you just took a bunch of numbers and mashed them together for no reason. The solution as far as I can see is to just remove the first part.

EDIT: It's actually easy to give a verbal description of the first graphs. The one with the cusp is a graph of "the average % of rounds that end with a balance of exactly X", and the accumulated area under it is "the average % of rounds that end with a balance of at most X". (eg, if there are 1000 rounds and 300 of them end with a balance less than -1000 BTC, the value of the variable will be 0.3 - and the graph lists the average of this over all instantiations). Note that as described these are not a pmf and cmf, though they can be looked at as the pmf of cmf of the mixture variable earlier discussed. And again, what you're looking for is not the average % of rounds that end with at most X, but rather the probability that at least one round ends with at most X.

Exactly! I was glad of that edit. I was about to post a description of the same using a random walk, but you've done a much simpler and clearer job. So, it just remains for me to try to explain why I thought "the average % of rounds that end with a balance of exactly X" or "the average % of rounds that end with a balance of at most X" is useful to know.

As you've shown, a negative buffer at an SMPPS pool will contribute to the a delay in the amount of time (in terms of blocks solved) that it takes for a miner to be paid. So the following two results will be of interest:
1. The maximum a negative buffer can be before n rounds.
2. The percentage of n rounds that one can expect the buffer to be any arbitrary amount (or worse/better).

Number 1 is of interest because a significantly negative buffer could delay payments long enough to kill a pool.

I thought number 2 would be of interest because knowing that for example one can expect 300 rounds out of the first 1000 to have a negative buffer of 10 rounds or worse, a miner might not decide to mine at an SMPPS pool, since for 30% of the time the average maturity time will be of the order of the amount of time it takes to solve ten blocks or longer.

If I haven't made a wrong assumption there in 2, it also occurs to me that there will be an expected and non-zero maturity time, since an increasing positive buffer doesn't reduce the maturity time as an increasing negative buffer increases it.



Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Meni Rosenfeld on April 11, 2012, 07:38:41 AM
I thought number 2 would be of interest because knowing that for example one can expect 300 rounds out of the first 1000 to have a negative buffer of 10 rounds or worse, a miner might not decide to mine at an SMPPS pool, since for 30% of the time the average maturity time will be of the order of the amount of time it takes to solve ten blocks or longer.
I guess. Still doesn't seem like a very intuitive quantity to look at.

If I haven't made a wrong assumption there in 2, it also occurs to me that there will be an expected and non-zero maturity time, since an increasing positive buffer doesn't reduce the maturity time as an increasing negative buffer increases it.
Right, the maturity time for future rounds is nonnegative and can be positive, so it has a positive expectation. And that - the average maturity time over the next N rounds - may be of interest to some people.

For an optimal miner it is inconsequential, because he can just leave when the buffer is low. But for a miner that wants to sign up for an extended period of time, it is a useful quantity that can be compared to the maturity time of other pools. The pmf of this average can fairly easily be simulated, and is more useful than the current graphs and IMO can replace them (as long as it is very clear what they represent). I can't think of a closed form for the pmf, but maybe it can be approximated by modeling it as Brownian motion (http://en.wikipedia.org/wiki/Wiener_process).


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 12, 2012, 02:12:25 AM
I guess. Still doesn't seem like a very intuitive quantity to look at.


Well that's clear. It took a thread page to explain it, which you ended up doing anyway. No, I don't think it's a useful quantity to help explain anything to a miner who might use SMPPS. I do think mean maturity time will be much more useful.

The pmf of this average can fairly easily be simulated, and is more useful than the current graphs and IMO can replace them (as long as it is very clear what they represent). I can't think of a closed form for the pmf, but maybe it can be approximated by modeling it as Brownian motion (http://en.wikipedia.org/wiki/Wiener_process).

I got as far as Donskers theorem before I realised that the mean negative round time could be calculated using a shifted negative binomial distribution with mean = 0, if I could find some way of calculating the mean of the negative buffer half of the distribution.

Since the negative binomial distribution approaches a normal distribution for large , I modelled it using a half normal distribution for which a method of caluculating the mean was already available. Since the positive buffer half of the normal distribution has weight zero when calculating the mean maturity time, the mean calculated using the half normal distribution method is halved.

The results from n = 6 to n= 10000 are within +/- 2% compared to several repeat simulations I ran, so I'm happy with the accuracy of the half normal distribution method and I'd like to use that rather than present a simulation since if readers want to they can follow the logic which is harder with a simulation written in code they won't know.

What I'd like to know is - did I explain that clearly enough? Is it correct? I'm not sure about the best way to explain the weighted means [Edit wrt probability distributions], and I 'll have to go into a bit more detail, but I did want your optinion before I went ahead and replaced that section. Also, does the graph (below) of mean maturity time up to 1000 rounds seem likely to you? If it is, then 1000 rounds into an SMPPS pools life you can expect a mean of about 12 rounds wait to be paid your average earnings.

http://i.imgur.com/4yvZg.png (http://imgur.com/4yvZg)


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Meni Rosenfeld on April 12, 2012, 08:55:48 AM
Finding the expected maturity time at round N is easy enough - as you say with a normal approximation (equivalent to a Brownian motion model) it's simply sqrt (N/(2Pi)), and the graphs match that. I thought it would be interesting to look at the expectation of the average maturity time in rounds 1 to N, but that's harder so the former would suffice.

(Edit: Actually, since expectation is linear, the expected average over 1-N is just the average of the expectation over 1-N, which is roughly 2/3 of the expectation at N. The pmf, though, is probably difficult.)

I think the explanation could use some more detail.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: finway on April 12, 2012, 09:19:42 AM
Yeah, i still have 2 bitcoins unpaid on arsbitcoin, it seems i'll never get it back.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 13, 2012, 11:52:18 AM
Well, as Luke-jr (the inventor of the SMPPS payout method) would say:

Quote
That was never promised to become Bitcoins or retain any value. (https://bitcointalk.org/index.php?topic=18567.msg811992#msg811992)

But I'm sorry you had to lose coins to find that out. Even besides the SMPPS payout, the poor pool shutdown has affected many miners. Found a new place to mine yet?



Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: organofcorti on April 13, 2012, 02:29:06 PM
Finding the expected maturity time at round N is easy enough - as you say with a normal approximation (equivalent to a Brownian motion model) it's simply sqrt (N/(2Pi)), and the graphs match that. I thought it would be interesting to look at the expectation of the average maturity time in rounds 1 to N, but that's harder so the former would suffice.

(Edit: Actually, since expectation is linear, the expected average over 1-N is just the average of the expectation over 1-N, which is roughly 2/3 of the expectation at N. The pmf, though, is probably difficult.)

I'd not thought of looking at the expectation of the average maturity time over N. Does it make sense to graph it as a function of N? I want to use whichever measure will be the most intuitive to understand.

Quote
I think the explanation could use some more detail.

Yes, indeed.


Title: Re: Neighbourhood Pool Watch 3.2 The risks of SMPPS
Post by: Meni Rosenfeld on April 14, 2012, 04:59:30 PM
I'd not thought of looking at the expectation of the average maturity time over N. Does it make sense to graph it as a function of N? I want to use whichever measure will be the most intuitive to understand.
Yes. I guess "average maturity time at round N" will be simpler.


Title: Re: NPW 4.1 Slush's pool and 3.2 The risks of SMPPS is updated
Post by: organofcorti on April 16, 2012, 01:50:00 PM
Neighbourhood Pool Watch 4.1 Slush's pool (http://organofcorti.blogspot.com.au/2012/04/41-slushs-pool.html) is posted.
Quote
6. Conclusions

  • Slush has a good reputation for honesty, openness, and responsiveness to his miners enquiries and preferences.
    Slush's pool is generally trusted by the community.
  • The current payout method, although better than proportional payouts, is flawed and increases variance for miners and doesn't eliminate the possibility of strategic miners reducing the payout of full time miners.
  • Once the payout method is changed to Meni Rosenfeld's DGM, full time miners will not lose earnings to strategic miners. Variance in earnings can be controlled by this method, and can be minimised.

Also, the update to Neighbourhood Pool Watch 3.2 The Risks of SMPPS (http://organofcorti.blogspot.com.au/2012/04/32-risks-of-smpps.html) is also posted. Thanks Meni for your help and advice on correcting it.

    Quote
    5. Conclusions
    ......
    • Average maturity time is approximately sqrt(n/(2*pi)). Arsbitcoin.com's maturity time of approximately 1000 btc at 1700 rounds is not much more than the average at 1700 rounds, 822 btc.
    • A reduction in pool hashrate is never of benefit to miners.
    .....[/list]

    Please post any questions or comments you have either here or on the blog.


    Title: Re: NPW 4.1 Slush's pool and 3.2 The risks of SMPPS is updated
    Post by: organofcorti on April 18, 2012, 08:39:32 AM
    Thanks very much to the person that last donation was from!


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 19, 2012, 03:58:08 PM
    Neighbourhood Pool Watch 5.1 p2Pool - bad luck or flawed? (http://organofcorti.blogspot.com.au/2012/06/51-p2pool-bad-luck-or-flawed.html) is posted.

    Quote
    There have been concerns by miners on the bitcointalk.org forum thread for p2Pool that there has been significantly poor luck at the pool for an extended period, and that the number of orphaned blocks created by the pool is higher than it should be. In this post we will investigate the published pool statistics generally, and then specifically with regard to these two concerns.
    Quote
    7. Conclusions

    • Although p2Pool's round lengths appear normally distributed, they have a mean value of 1.104D, and are greater than the expected value of D with 95% confidence. The expected round lengths at p2Pool are about 10% more than expected. This is not "bad luck" but inherent in the pool.
    • Orphan production seems to have been increasing,  although with a small sample size this conclusion may be erroneous.
    • Orphan production rate does not appear to correlate with changes in P2pool hashrate.
    • If the usual orphan rate at other pools is 1.5%, and the mean round length remains approximately 1.1D, then miners will earn 11% less here than their expected share values.
    • If the orphan rate is normal, then miners are earning approximately 9% less than expected.
    • Variance is significantly larger than for pooled mining using difficulty 1 shares, but much less than for solo mining.
    • Donations to p2Pool over the past 5 months 131.13 btc, increasing miner earnings by an average of 0.52%
    • Overall, miner share values have been either 8.5% or 10.5% less than expected.
    • Given the decentralised nature of the pool, some proponents feel that this is an acceptable price.
    • DDoS attacks can also lead to downtime and loss of income for miners. This will not occur at p2Pool.
    • The concept of p2Pool is laudable, and I hope it continues to be developed and the round length problem solved.

    http://i.imgur.com/Gx9Wn.png (http://imgur.com/Gx9Wn)


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Meni Rosenfeld on June 19, 2012, 04:28:05 PM
    Neighbourhood Pool Watch 5.1 p2Pool - bad luck or flawed? (http://organofcorti.blogspot.com.au/2012/06/51-p2pool-bad-luck-or-flawed.html) is posted.

    Quote
    There have been concerns by miners on the bitcointalk.org forum thread for p2Pool that there has been significantly poor luck at the pool for an extended period, and that the number of orphaned blocks created by the pool is higher than it should be. In this post we will investigate the published pool statistics generally, and then specifically with regard to these two concerns.
    Quote
    7. Conclusions

    • Although p2Pool's round lengths appear normally distributed, they have a mean value of 1.104D, and are greater than the expected value of D with 95% confidence. The expected round lengths at p2Pool are about 10% more than expected. This is not "bad luck" but inherent in the pool.
    • Orphan production seems to have been increasing,  although with a small sample size this conclusion may be erroneous.
    • Orphan production rate does not appear to correlate with changes in P2pool hashrate.
    • If the usual orphan rate at other pools is 1.5%, and the mean round length remains approximately 1.1D, then miners will earn 11% less here than their expected share values.
    • If the orphan rate is normal, then miners are earning approximately 9% less than expected.
    • Variance is significantly larger than for pooled mining using difficulty 1 shares, but much less than for solo mining.
    • Donations to p2Pool over the past 5 months 131.13 btc, increasing miner earnings by an average of 0.52%
    • Overall, miner share values have been either 8.5% or 10.5% less than expected.
    • Given the decentralised nature of the pool, some proponents feel that this is an acceptable price.
    • DDoS attacks can also lead to downtime and loss of income for miners. This will not occur at p2Pool.
    • The concept of p2Pool is laudable, and I hope it continues to be developed and the round length problem solved.
    I didn't read the entire analysis but I disagree with your conclusion that round lengths are inherently longer, there is hardly enough evidence to rule out bad luck. Especially if there is no plausible explanation how can round length mean be higher.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 19, 2012, 04:37:03 PM
    I didn't read the entire analysis but I disagree with your conclusion that round lengths are inherently longer, there is hardly enough evidence to rule out bad luck. Especially if there is no plausible explanation how can round length mean be higher.

    There's enough evidence to show that the round length mean is greater than D to a 95% confidence level. It would have to be quite bad luck.

    But I agree there's no mechanism I know of that could plausibly cause such a result. But I'm not a p2Pool expert, and I'm hoping that such an expert will come on board and discuss it. The p2Pool share chain and dynamic difficulty work so differently to a normal pool that I really can't comment. I was thinking that some shares may be unaccounted for and not reported locally, or some such thing. Most likely improbable, and if someone can prove the impossibility then I will gladly edit the post.



    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Inaba on June 19, 2012, 04:39:46 PM
    Did you factor in the high stale rate compared to a normal pool? 


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 19, 2012, 04:49:55 PM
    Did you factor in the high stale rate compared to a normal pool? 

    DOA and orphan shares don't make it into the sharechain and so can't increase the numbers of shares in a round.

    One possibility is that the method used to estimate the equivalent difficulty 1 shares to solve a block using the pool hashrate is buggy. If for example the pool hashrate is actually 9% lower than reported, then the number of shares to solve a block would be D.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Meni Rosenfeld on June 19, 2012, 04:50:25 PM
    I didn't read the entire analysis but I disagree with your conclusion that round lengths are inherently longer, there is hardly enough evidence to rule out bad luck. Especially if there is no plausible explanation how can round length mean be higher.
    There's enough evidence to show that the round length mean is greater than D to a 95% confidence level. It would have to be quite bad luck.

    But I agree there's no mechanism I know of that could plausibly cause such a result. But I'm not a p2Pool expert, and I'm hoping that such an expert will come on board and discuss it. The p2Pool share chain and dynamic difficulty work so differently to a normal pool that I really can't comment. I was thinking that some shares may be unaccounted for and not reported locally, or some such thing. Most likely improbable, and if someone can prove the impossibility then I will gladly edit the post.
    That's why Bayesian statistics is superior to frequentist statistics. All the data gives us is about 10:1 likelihood ratio for 1.1D over 1D, which doesn't mean much when the prior probability of 1.1D is so low.

    One possibility is that the method used to estimate the equivalent difficulty 1 shares to solve a block using the pool hashrate is buggy. If for example the pool hashrate is actually 9% lower than reported, then the number of shares to solve a block would be D.
    That makes more sense. But even then, not much evidence.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 19, 2012, 05:06:23 PM
    That's why Bayesian statistics is superior to frequentist statistics. All the data gives us is about 10:1 likelihood ratio for 1.1D over 1D, which doesn't mean much when the prior probability of 1.1D is so low.
    10:1 likelihood ratio? Isn't it 20:1? But I get your point. Perhaps I was a little too rash in discussing what it might mean in terms of possible future miner earnings. But it does mean that either:

    1. p2Pool's had a very poor sort of luck.
    2. p2Pool's round length mean is larger than it should be.
    3. There's a problem with the reported pool statistics.

    I think it's worth pointing out the possibility of either 2 or 3, even in absence of a mechanism known to me. Many p2Pool miners are concerned about the pool's luck, and I've read posts by a few that have left the pool for that reason. If the p2Pool devs can come up with an explanation then it might save the pool losing even more miners.

    Edit: if there is a problem with the reported stats then p2Pool miners should be receiving on average the expected value for their hashes. Anyone from p2Pool how has been mining consistently care to share their earnings history and hashrate?


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Meni Rosenfeld on June 19, 2012, 05:39:36 PM
    10:1 likelihood ratio? Isn't it 20:1?
    According to http://p2pool.info/luck/, 3851 PH were calculated but 3491 PH worth of blocks were found. Assuming difficulty = 1.6M this means 560.4 blocks expected and 508 blocks found. So the likelihood ratio between poisson distributions with mean 560.4 and 508 for 508 found is (508/560.4)^508*exp(560.4-508) ~ 12.5.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Shadow383 on June 19, 2012, 05:46:35 PM
    Where are you getting 5% probability from? I'm seeing more like 1% here.

    Left p2pool a while ago, might go back to using it as a backup at least as/if it improves...


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Smoovious on June 20, 2012, 03:08:43 AM
    Um... perhaps something to consider...

    When you were evaluating the other pools, one variable that didn't come into play, was the propagation of the new block among the miners.

    Since with a centralized pool, long-polling delays aside, everyone is basically mining off of the same daemon.

    With P2Pool, this is not the case. We're effectively all solo-miners, with pooled payouts. When one node finds a block, it takes time to propagate, so for a period of time, you would have part of the pool still working on one block, while a growing portion of the pool is working on the new block as the found block gets distributed.

    As far as orphans go, a centralized pool is in itself, a single solo miner, albeit one with lots of connections, and bandwidth to distribute the new block efficiently. P2Pool, on the other hand, is a whole bunch of solo miners cooperating.

    So, one thought that occurred to me, is when comparing stats, shouldn't solo mining be considered the normal, with the centralized pools, being above normal, due to the efficiency they have in their miners being notified that they should be working on the next block all at once? If they weren't pooled, all of those miners, would be notified at different rates, which may have an affect on orphan rates.

    Perhaps, the orphans P2Pool have been experiencing were simply normal for solo mining...

    (I'm no mathematician compared to your level, so, I may not be quite clear on what I'm getting at...)

    -- Smoov


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Meni Rosenfeld on June 20, 2012, 03:43:51 AM
    @Smoovious: I think it's the other way around. A centralized pool should have more latency than solo because first the pool needs to learn about the block, then it has to long-poll you. When solo mining you just need to learn about the block.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: bitfoo on June 20, 2012, 03:53:18 AM
    @Smoovious: I think it's the other way around. A centralized pool should have more latency than solo because first the pool needs to learn about the block, then it has to long-poll you. When solo mining you just need to learn about the block.

    If I understand this right, you're talking about stale shares, not orphans. If I find a block on p2pool, but my bitcoind is only connected to 8 peers, it might take a while for the block I found to propagate through the network. Meanwhile Deepbit finds a block and sends it to 1000 different nodes all at once, orphaning my block.

    I don't know the workings of p2pool fully. Do they send newly found blocks to each other as well, to aid in propagating it?


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 20, 2012, 03:54:14 AM
    Where are you getting 5% probability from? I'm seeing more like 1% here.

    Left p2pool a while ago, might go back to using it as a backup at least as/if it improves...

    The 95% CI excluded a mean of D, median of ~ 0.693D, and an sd of ~ D. I didn't check the 99% CI. It may also exclude the expected values.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Meni Rosenfeld on June 20, 2012, 04:27:24 AM
    @Smoovious: I think it's the other way around. A centralized pool should have more latency than solo because first the pool needs to learn about the block, then it has to long-poll you. When solo mining you just need to learn about the block.

    If I understand this right, you're talking about stale shares, not orphans. If I find a block on p2pool, but my bitcoind is only connected to 8 peers, it might take a while for the block I found to propagate through the network. Meanwhile Deepbit finds a block and sends it to 1000 different nodes all at once, orphaning my block.
    Well, if a stale share happens to be a block, it is technically an orphan block, so this is still an increase in the orphan rate, depending on how you count it (maybe orphan blocks which are known to be orphaned when the pool receives them are not counted). What we care about here is the total rewards of the pool, not how they are distributed internally.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: bitfoo on June 20, 2012, 04:33:48 AM
    Well, if a stale share happens to be a block, it is technically an orphan block, so this is still an increase in the orphan rate, depending on how you count it (maybe orphan blocks which are known to be orphaned when the pool receives them are not counted).

    Agreed. So both situations actually affect p2pool negatively. Assuming that the average p2pool node has lower bandwidth and lower connectivity than a big pool, it will generally receive blocks and send out new blocks slower, resulting in more orphans overall.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Smoovious on June 20, 2012, 04:46:35 AM
    Well, if a stale share happens to be a block, it is technically an orphan block, so this is still an increase in the orphan rate, depending on how you count it (maybe orphan blocks which are known to be orphaned when the pool receives them are not counted).

    Agreed. So both situations actually affect p2pool negatively. Assuming that the average p2pool node has lower bandwidth and lower connectivity than a big pool, it will generally receive blocks and send out new blocks slower, resulting in more orphans overall.
    Yeah... I think... and, I wasn't around in the early days, not beginning to mine until the last quarter of last year, so, dunno about the early intentions of bitcoin...  but...

    When it was designed, I think the intention was for individual peers mining individually. Not sure if pooled mining was the intent from the start. (I'm sure I'll be corrected on this if I'm mistaken :) )

    So, a higher level of orphaned blocks, when scaled up to a lot of peers, may have been an expected metric, and considered the normal situation. Perhaps P2Pool's orphan rate, better reflects the way this would have been, without the centralization of mining?

    What the centralized pools did, was cut out the latency from when found blocks, propagated to the other people mining. Since they would inform the mining peers more or less, simultaneously, it would have the effect of near-instant propagation among a large number of peers. Since none of them would have to sanity-check the block against their own blockchain, avoiding the latency that would come with it while distributing it, and not having to pass it along to the next set of peers afterwards, you have a large segment of miners abandoning the current block and starting the new one, much sooner than would have been expected, had everyone been solo-mining. This alone, would have eliminated a portion of what would have been a normal orphan rate.

    All of those mining peers, on a centralized pool, would be considered collectively, a single bitcoin peer as far as the network is concerned. Take the top 75% of the hashpower on the bitcoin network, and you end up with, what... a few dozen bitcoin peers? Perhaps a little more? Certainly a tiny fraction of all of the individual bitcoin peers that are mining as a whole, by node count.

    I don't know how this could be reflected in useful data, but perhaps this has an affect on assumptions of expected levels of good blocks and orphans within a given period of time?

    I think what I'm getting down to, is... perhaps instead of looking to the average pool results to find normal, the pool results, should be considered above-normal instead of the baseline, which would be pure solo mining on a more distributed network?

    -- Smoov


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Smoovious on June 20, 2012, 05:09:09 AM
    @Smoovious: I think it's the other way around. A centralized pool should have more latency than solo because first the pool needs to learn about the block, then it has to long-poll you. When solo mining you just need to learn about the block.

    If I understand this right, you're talking about stale shares, not orphans. If I find a block on p2pool, but my bitcoind is only connected to 8 peers, it might take a while for the block I found to propagate through the network. Meanwhile Deepbit finds a block and sends it to 1000 different nodes all at once, orphaning my block.

    I don't know the workings of p2pool fully. Do they send newly found blocks to each other as well, to aid in propagating it?
    forrestv added some protocol changes to aid this very thing, but we're still dealing with a distributed network for the daemon's connectivity, and a separate distributed network for p2pool's connectivity. We basically have another level of notification, letting us know to work on a new block, but we still have the latency of a distributed network, getting that notification propagated. I think I am understanding it right tho, that p2pool's notification, will go much faster, as it deals with less data in the transmission, and less sanity-checking at each node before it gets passed on.

    The main difference, the way I see it tho, amounts to the way a centralized vs decentralized system handles how the data gets around. How much it is handled between hand-offs, and how many hand-offs between the finder and the rest of the pool.

    Take deepbit as an example... when one of their miners find a block, it is a very short path for the rest of the pool to be notified...

    miner1!deepbit!{miner2|...|minerA}

    With P2Pool and no centralized bitcoin, and with forrestv's protocol change from last week, we have 2 paths, but they can be pretty long.

    miner1!p2pool1!bitcoind!bitcoind2!bitcoind3!bitcoind4!p2pool2!miner2
    or
    miner1!p2pool1!p2pool2!p2pool3!p2pool4!p2pool5!miner2

    The paths the data needs to take, through either route, can be even longer. The route via bitcoind would be slower, and via p2pool would be faster, but both would be much slower than the deepbit path above.

    If I remember right, we (p2pool users), have orphaned one of our own blocks before. Something that couldn't happen at the centralized pools.

    -- Smoov


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: kano on June 20, 2012, 05:25:21 AM
    Did you factor in the high stale rate compared to a normal pool?  

    DOA and orphan shares don't make it into the sharechain and so can't increase the numbers of shares in a round.

    One possibility is that the method used to estimate the equivalent difficulty 1 shares to solve a block using the pool hashrate is buggy. If for example the pool hashrate is actually 9% lower than reported, then the number of shares to solve a block would be D.
    Although they don't affect the number of shares per block difficulty
    (which is really the only valid measurement)
    DOA and orphan shares reduce the 'calculated' hash rate of people using p2pool.

    P2Pool itself does apparently send out DOA and orphan shares to the bitcoind to ensure that shares that are no good for the P2Pool share-chain but are actually a valid BTC-block will still go out.

    My other suggestion (that I've said many times) of the cause of p2pool's "misfortune" is that a 'typical' p2pool is quite literally crap compared to a well tuned pool.
    Thus the delay between the miner finding a share and the share making it 'out onto the net' would of course be worse on average with p2pool users vs a well tuned, well connected, high performance pool server.
    This 2nd item would of course show up in terms of more orphans compared to other well configured pools.

    Recent BTC problems may hide this :P


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 20, 2012, 06:49:18 AM
    I included orphaned blocks in the average round length calculations, since it was the number of shares taken to create a valid block - so the number of orphaned blocks do not affect the round length / D average. They do still affect miner earnings though.

    Rate of orphan production for the first hundred blocks solved was 0%, for the next hundred 1%, for the third hundred 3%, the fourth hundred 6%, and for the first 35 blocks of the sixth hundred blocks, 8.5%. Although there are only 17 orphaned blocks altogether, is does seem that orphan production has increased. This could just be variance though - I have no idea what the mean or variance of the orphaned block distribution should be, and not enough data to even guess.


    Edit: and a quick thank you to the recent very generous donator, whoever you are. Much appreciated.
    Edit2: actually thanks to the last few donators, especially whomever sent exp(1)/10 btc - I got a laugh out of that ;)


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: check_status on June 22, 2012, 02:34:08 AM
    @organofcorti Why do you work so hard to show how bad p2pool is? Do you have a dog in the race?


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 22, 2012, 02:53:32 AM
    @organofcorti Why do you work so hard to show how bad p2pool is? Do you have a dog in the race?

    No. No dogs in any race. I just provide miners with information. I thought that would have been evident from the other posts on the blog and in this thread, though.

    I hope by "working hard" you don't mean "working too hard to a preconceived conclusion". I didn't have any. I'd thought that once I had a good look at the data it would be within certain limits that included normal but it didn't, within a 95% confidence range.

    Meni Rosenfeld points out that the conclusions could be incorrect, especially I think this statement:

    Quote
    This is not "bad luck" but inherent in the pool

    which is a conclusion one cannot come too really given that it's based on a 95% CI.

    So at the moment and before I edit the blog post, I'm looking for more p2Pool data from miners in order to try to get a better idea of what's happening.

    Finally, the last points of the conclusion are:

    Quote
    • Given the decentralised nature of the pool, some proponents feel that this is an acceptable price.
    • DDoS attacks can also lead to downtime and loss of income for miners. This will not occur at p2Pool.
    • The concept of p2Pool is laudable, and I hope it continues to be developed and the round length problem solved.

    which summarise my personal feelings about p2Pool nicely.

    I'm not "anti" any pool I blog about, but I find it's more useful to miners in general to analyse a pool where there seems to be a number of miners who are unhappy or have concerns (be they validated or not), rather than one that has happy miners.



    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Smoovious on June 22, 2012, 03:51:06 AM
    @organofcorti Why do you work so hard to show how bad p2pool is? Do you have a dog in the race?
    He was being fair.

    It is about the numbers, period.

    In fact, his analysis, was helpful in validating some of the concerns some of us have posted about long-term luck.

    Whether we like or dislike the results, fitting or not fitting our own conclusions, is irrelevant to the data.

    -- Smoov


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Meni Rosenfeld on June 22, 2012, 04:00:02 AM
    @organofcorti Why do you work so hard to provide objective, actionable information to people who would rather bury their head in the sand?

    Seriously though, check_status, your comment was completely out of line. OOC has empirically investigated many pools, in many cases he found inadequacies caused by negligence or malice of the operator.

    There was a legitimate concern raised about p2pool functioning properly. I learned about it myself some time ago and run some calculations, hoping and expecting to find it well within the acceptable range. What I found was that it is within it, but not well within it.

    OOC too was hoping to show how great p2pool is and that there is no problem despite the fear. What he found was that there is weak evidence suggesting a problem. What would you have him do? Hide the information? Lie about it? Or share it so that the relevant people can look into it and possibly fix it (whether it affects the rewards or just the displayed values)? Or even if it is in fact an insolvable problem, being open about it and giving people the chance to make their own decisions in full knowledge of the facts? In fact mechanically presenting the data despite his personal feelings about it is exemplary scientific integrity.

    In my day job, whenever I find an inconsistency in our data, people are ecstatic because it means there is a bug that, when fixed, will improve our data considerably. I don't get put down for working hard to show how bad our data is. You should adopt the same mentality.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Smoovious on June 22, 2012, 04:06:54 AM
    The plus side of his analysis, is, after taking into account the concerns we were already debating in the other thread, the rest of his data confirms that p2pool is being fair with its members when it comes to payouts, which is a good thing for p2pool's reputation.

    I also think it is good he ran the data when he did.

    Since forrestv has made some changes that should have a direct effect (fingers crossed) on reducing our orphan rate, we now have a 'before' analysis put together, which can be used to compare against p2pool's statistics, post-change, once there are enough data points to make a reasonable analysis, to see exactly how those changes have improved things (fingers still crossed).

    -- Smoov


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Graet on June 22, 2012, 06:16:49 AM
    @organofcorti Why do you work so hard to show how bad p2pool is? Do you have a dog in the race?
    organofcorti has worked long and hard to show miners the truth and break through misconceptions, not just about p2pool but also about hopping and exploitable payout systems.
    I respect and appreciate the work he has done for the Bitcoin community and miners in particular.

    I also see organofcorti's work constantly reviewed by Meni Rosenfeld a champion of fair mining and another member of the Bitcoin community I have great respect for.

    I do have a "dog in the race" but have no problem being scrutinised. Only those with something to hide would fear it.

    Best wishes
    Graeme Tee
    Ozcoin Pooled Mining Pty Ltd


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: check_status on June 22, 2012, 06:50:09 AM
    Quote from: organofcorti
    Given the decentralised nature of the pool, some proponents feel that this is an acceptable price.

    Let's disect this statement shall we:

    a decentralized pool = an acceptable price

    This connotes a sacrifice in some other area. According to the majority of organofcorti's posts, that is profitability. Your using words to paint an unprofitable picture about p2pool that doesn't exist, the blocks attest to this.

    Quote from: Meni Rosenfeld
    Seriously though, check_status, your comment was completely out of line. OOC has empirically investigated many pools, in many cases he found inadequacies caused by negligence or malice of the operator.
    It's not out of line at all. There is a preponderance of evidence of organofcorti's p2pool posts where he flat out says or implies p2pool is the least profitable pool. You are not a fan of p2pool either, some of your remarks have been aimed at having p2pool users mine at other non-p2pool pools. A hater in support of another hater, I'm shocked.  :o

    Do we need to get the blocks out to see how p2pool provides one of the highest PPS payouts per gigahash when compared against any pool.

    Most people who mine watch what is said and make decisions on what occurs in posts. Many lack knowledge of how to determine profitability of a given pool, they rely on the thoughts and comments of others whom they deem "smart" and therefore, can be swayed by "Inteligent BS".


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 22, 2012, 07:39:54 AM
    Quote from: organofcorti
    Given the decentralised nature of the pool, some proponents feel that this is an acceptable price.

    Let's disect this statement shall we:

    a decentralized pool = an acceptable price

    This connotes a sacrifice in some other area. According to the majority of organofcorti's posts, that is profitability. Your using words to paint an unprofitable picture about p2pool that doesn't exist, the blocks attest to this.

    According to my blog post, the sacrifice is profitability. As I already noted, this statement was stronger than warranted, and I will be changing it to a provable statement at some point soon.

    Quote from: Meni Rosenfeld
    Seriously though, check_status, your comment was completely out of line. OOC has empirically investigated many pools, in many cases he found inadequacies caused by negligence or malice of the operator.

    It's not out of line at all. There is a preponderance of evidence of organofcorti's p2pool posts where he flat out says or implies p2pool is the least profitable pool.
    Please provide a link to a post where I flat out say that p2Pool is the least profitable pool.

    You are not a fan of p2pool either, some of your remarks have been aimed at having p2pool users mine at other non-p2pool pools. A hater in support of another hater, I'm shocked.  :o
    I guess we both must be in the pay of the Big Pool conglomerates then, huh.

    Do we need to get the blocks out to see how p2pool provides one of the highest PPS payouts per gigahash when compared against any pool.
    Sigh. I already did. Which part of the blog post didn't you follow? I provide the table of the expected parameters ("Theoretical 1") and experimental statistics for p2Pool here to refresh your memory:

    http://i.imgur.com/aPmHX.png (http://imgur.com/aPmHX)

    The data I based it on is here:

    http://pastebin.com/g2ESh1Uf (http://pastebin.com/g2ESh1Uf)

    Highest PPS payout? More than 100%? If the pool is paying out based on the recorded pool Ghps, then it has paid out 0.91 * PPS so far. If the public data is wrong and the payout based on your local hashrate is in the expected range, then in the long term your local payout will approach PPS.

    Most people who mine watch what is said and make decisions on what occurs in posts. Many lack knowledge of how to determine profitability of a given pool, they rely on the thoughts and comments of others whom they deem "smart" and therefore, can be swayed by "Inteligent BS".

    I'm grateful to anyone who puts time in to reading and understanding my blog. If they can show me a fault in my work, then I'm even more grateful.

    Simply implying that I'm "smart" and the analysis "Inteligent BS" is not a constructive criticism. Read the blog and find a mistake that I haven't already owned up to. Otherwise you're just posting "BS". And not very intelligent BS, either.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Meni Rosenfeld on June 22, 2012, 07:40:53 AM
    There is a preponderance of evidence of organofcorti's p2pool posts where he flat out says or implies p2pool is the least profitable pool.
    Even if that's true, offering constructive criticism is not hating. Truly supporting something  is caring enough to try to improve it, nobody needs a yes man.

    You are not a fan of p2pool either
    I am a fan of p2pool, and I've spoken several times about its importance going forward.

    , some of your remarks have been aimed at having p2pool users mine at other non-p2pool pools.
    I don't recall ever asking/recommending p2pool miners to switch to a different pool, so this is purely your own speculation on what I might be aiming at.

    A few remarks I did make about p2pool:
    1. The reward system should be looked at to verify that it is truly hopping-proof, and not just "almost hopping-proof" like the naive variant of PPLNS.
    2. Some of its suggested advantages are a bit exaggerated, at least at this point - the network isn't going to crumble tomorrow without p2pool.
    3. The correct way to use p2pool is not by directly connecting to it, but with a PPS proxy.


    You are not a fan of p2pool either, some of your remarks have been aimed at having p2pool users mine at other non-p2pool pools. A hater in support of another hater, I'm shocked.  :o
    I guess we both must be in the pay of the Big Pool conglomerates then, huh.
    [joke] Must be Deepbit. They're also clever enough to hide this connection by paying us to speak ill of the proportional method. [/joke]


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: kano on June 22, 2012, 12:06:16 PM
    I am curious about where you got "Round shares" from :)

    Yes I am also glad to see that shares is the basis for the calculations which of course is what you would use (yes I'm stating the obvious) - but it seems that there are those who don't understand this and thus argue about the results due to not using those same results.

    My curiosity is simply that the share-chain throws away this information and thus I guess someone must have full logs of the share chain since day one to be able to provide that information.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 22, 2012, 12:18:08 PM
    I am curious about where you got "Round shares" from :)

    Yes I am also glad to see that shares is the basis for the calculations which of course is what you would use (yes I'm stating the obvious) - but it seems that there are those who don't understand this and thus argue about the results due to not using those same results.

    My curiosity is simply that the share-chain throws away this information and thus I guess someone must have full logs of the share chain since day one to be able to provide that information.

    It'a all here: http://p2pool.info/blocks

    Round shares are an estimation based on pool hashrate and time between timestamps. It's in order of block height, so sometimes if a round is short and timestamps inaccurate, you get "negative" round shares. This  happens infrequently and doesn't affect the round shares probability distribution significantly. It averages out the same - if you used total hashes and total rounds, you'd get the same mean round length.

    Play with the data, see what you get.



    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: kano on June 22, 2012, 12:46:36 PM
    I am curious about where you got "Round shares" from :)

    Yes I am also glad to see that shares is the basis for the calculations which of course is what you would use (yes I'm stating the obvious) - but it seems that there are those who don't understand this and thus argue about the results due to not using those same results.

    My curiosity is simply that the share-chain throws away this information and thus I guess someone must have full logs of the share chain since day one to be able to provide that information.

    It'a all here: http://p2pool.info/blocks

    Round shares are an estimation based on pool hashrate ...
    OK ... not the answer I expected.

    hashrate is an estimate based on the number of shares.

    Using someone else's calculation then reversing that back to the number of shares is a little risky IMO ...
    It assumes that the hashrate was indeed calculated correctly ...

    The reason for using the number of shares is that you can't really count shares incorrectly unless you are losing count.
    If you aren't counting them correctly, then the hash rate will also be wrong no matter how you calculate it.
    You may not have the count, but it you have the count of shares that is certainly better to use than a hashrate estimate converted back to a share count


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 22, 2012, 01:08:08 PM
    Round shares are an estimation based on pool hashrate ...
    OK ... not the answer I expected.

    hashrate is an estimate based on the number of shares.

    Using someone else's calculation then reversing that back to the number of shares is a little risky IMO ...
    It assumes that the hashrate was indeed calculated correctly ...
    Yes. Although I can show that the hashrate estimated shares are still consistent with a geometrically distributed probability function, so the if the hashrate estimated round shares are in error then it would likely be a constant error.

    Somewhere above I pointed out that if the global hashrate estimate is incorrect, then that could account for the longer than expected round hares. That's why I'm encouraging p2Pool users to PM me their stats. If they're earning the expected amount based on the shares submitted, then the results stand. If they earn on average 10% more than expected then the pool hashrate calculations, and hence calculated round shares are overestimated and and pool luck underestimated for some reason.

    The reason for using the number of shares is that you can't really count shares incorrectly unless you are losing count.
    If you aren't counting them correctly, then the hash rate will also be wrong no matter how you calculate it.
    You may not have the count, but it you have the count of shares that is certainly better to use than a hashrate estimate converted back to a share count

    I wouldn't have used the hashrate to shares conversion if I had a choice.

    Because p2Pool has variable difficulty shares, you have transform them back to hashes. I would have preferred having each share recorded and then tagged by difficulty and block number, but this data isn't available to me. So I have to trust that the round shares figure they provide are accurate (and the pool's had really bad luck), or show that they are not (and the pool's luck is different, maybe in the expected range).








    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: check_status on June 22, 2012, 02:39:18 PM
    PPS payout:
    86,400 seconds/day * MH/sec / 4294.967296 (megahashes/share) * PPS = Payout/day

    Finding expected PPS:
    Take Payout/day from Alloscomp's calculator for 1000 MH/s and place into Payout/day of PPS payout to solve expected PPS.

    To find your Payout per gigahash/s you will need, a pools avg. GH/s for the period your measuring, # of blocks divided by days.

    Finding the expected payout:
    According to Alloscomp's Bitcoin Calculator: http://www.alloscomp.com/bitcoin/calculator.php
    If the difficulty factor is 1583177.84744, and the MH/s is 1000, then your Payout per GH/s per day is ⊅0.64
    This will be the standard measure, the expected value, for comparing miner, pools or network payouts.

    Calculating the past 7 day PPS and payout/day for P2pool:

    P2pool 7 day average hashrate: 222 GH/s
    Previous 7 days total blocks from 6/14-6/20: 23

    23 blocks * 50 / 222 GH/s = 5.18018 / 7 = 7 day avg. Payout per GH/s or ⊅0.740026 per GH/s per day

    P2pool has been performing for the past 7 days 15.6% better than expected.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: kano on June 22, 2012, 03:03:06 PM
    That's nice but you actually expect to get 100%, not 115%

    It does show that it is possible that luck is now hitting 100% instead of the low 9x%
    However, 7 days is way to short to make any conclusions.

    This week you should expect to get 0.58255098 a day due to the difficulty rise to 1726566.5591935


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 22, 2012, 03:16:30 PM

    P2pool has been performing for the past 7 days 15.6% better than expected.

    And that's good news for p2Pool miners, but I couldn't base an analysis on just one week. I based it on all rounds since the start of the pool. On all the pool data available, not just the last week, the pool as a whole has solved 535 blocks with 590.83D shares. That's only 90.5% of expected, or 9.5% worse in your terms.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Aseras on June 22, 2012, 03:32:53 PM
    something else to keep in mind is that work lasts longer than 1 round, sometimes work lasts 2-5 blocks later. so the work is distributed, pps comparisons are somewhat skewed because of this. the pps is going to be lower, but has momentum. so if you quit, you still get paid for a while even though your hashrate is zero.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: rjk on June 22, 2012, 03:40:30 PM
    lol at poll:

    Quote
    Has anything you've read here or on the Neighbourhood Pool Watch blog changed your mining habits for the better?
    You may only select up to 15 options.
    Yes
    No
    I don't mine


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 22, 2012, 03:59:26 PM
    something else to keep in mind is that work lasts longer than 1 round, sometimes work lasts 2-5 blocks later. so the work is distributed, pps comparisons are somewhat skewed because of this. the pps is going to be lower, but has momentum. so if you quit, you still get paid for a while even though your hashrate is zero.

    Can you explain this in more detail? What work do you mean is lasting more than one round?


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 22, 2012, 04:02:15 PM
    lol at poll:

    Quote
    Has anything you've read here or on the Neighbourhood Pool Watch blog changed your mining habits for the better?
    You may only select up to 15 options.
    Yes
    No
    I don't mine

    Ah, yes, um, well you can select any number of options up to 15, as long as that number is also less than four. Yeah, that's it. ;)

    Mods, can we get a poll edit here?


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Smoovious on June 22, 2012, 07:55:54 PM
    something else to keep in mind is that work lasts longer than 1 round, sometimes work lasts 2-5 blocks later. so the work is distributed, pps comparisons are somewhat skewed because of this. the pps is going to be lower, but has momentum. so if you quit, you still get paid for a while even though your hashrate is zero.

    Can you explain this in more detail? What work do you mean is lasting more than one round?
    our shares are persistent over a period of time... I believe it is for 24 hours, or the expected amount of work it would take to find 3 blocks, whichever is lower. (better explained by others)

    So, if I stop mining, while my payout is 0.0377... the shares I have persist until they fall off the end of the chain in a FIFO method. If the pool finds 2 blocks, within the next 12 hours, I would get a payout for those 2 blocks, based on my shares that haven't fallen off of the share chain yet.

    Perhaps this is what the poster meant by work lasting longer than a round?

    -- Smoov


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: check_status on June 22, 2012, 10:50:13 PM
    P2pool's actual payout per gigahash per day for the period 8/24/11-9/23/11 or 31 days

    Average daily hashrate: 9 GH/s
    # Blocks found: 6

    6 * 50 / 9 = 33.33333 / 31 = ⊅1.075269 per gigahash per day

    Now all that is needed is the difficulty factor for the same period to calculate expected value, then compare.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 22, 2012, 11:39:17 PM
    Get the difficulty from theymos' baby:

    http://blockexplorer.com/q/nethash

    Post an update when you're done?


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: check_status on June 23, 2012, 12:32:23 AM
    Get the difficulty from theymos' baby:

    http://blockexplorer.com/q/nethash

    Post an update when you're done?
    Unfortunately, it isn't in n00bese, so I would need another source for the difficulty factor.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: rjk on June 23, 2012, 12:36:09 AM
    Get the difficulty from theymos' baby:

    http://blockexplorer.com/q/nethash

    Post an update when you're done?
    Unfortunately, it isn't in n00bese, so I would need another source for the difficulty factor.
    Difficulty is the fourth column from the right.



    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 23, 2012, 12:40:25 AM
    Get the difficulty from theymos' baby:

    http://blockexplorer.com/q/nethash

    Post an update when you're done?
    Unfortunately, it isn't in n00bese, so I would need another source for the difficulty factor.

    Just copy paste the page into notepad, save as "nethash.csv" and then open with Excel. It will open to a standard csv spread sheet.

    Otherwise just use my data here:

    http://pastebin.com/g2ESh1Uf

    which is also a .csv

    Check a few datapoints against the the data at p2Pool.info/blocks to satisfy yourself it's the same dataset as on p2Pool.info. Eaxch block is tagged with difficulty.



    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: check_status on June 25, 2012, 03:43:57 AM
    It's not out of line at all. There is a preponderance of evidence of organofcorti's p2pool posts where he flat out says or implies p2pool is the least profitable pool.
    Please provide a link to a post where I flat out say that p2Pool is the least profitable pool.

    Below are 2 examples out of many that show organofcorti talking about p2pool earnings in relation to other pools:
    Quote from: organofcorti
    you're earning less than at a PPS pool.
    and
    Quote from: organofcorti
    you should continue mining at p2Pool, despite the lower earnings.

    I have provided 2 samples, 7 day recent and a 31 day historical that show his assertions are nonsense. organofcorti's assessment of p2pool profitability is erroneous, designed to confound and dishearten the uninformed.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: kano on June 25, 2012, 03:58:13 AM
    It's not out of line at all. There is a preponderance of evidence of organofcorti's p2pool posts where he flat out says or implies p2pool is the least profitable pool.
    Please provide a link to a post where I flat out say that p2Pool is the least profitable pool.

    Below are 2 examples out of many that show organofcorti talking about p2pool earnings in relation to other pools:
    Quote from: organofcorti
    you're earning less than at a PPS pool.
    and
    Quote from: organofcorti
    you should continue mining at p2Pool, despite the lower earnings.

    I have provided 2 samples, 7 day recent and a 31 day historical that show his assertions are nonsense. organofcorti's assessment of p2pool profitability is erroneous, designed to confound and dishearten the uninformed.
    Since you feel the need to argue in a thread dealing with statistics ... but you are ignoring that fact ...
    Look up and learn all about these 2 words in detail and how they relate to statistics:
    Sample, Population.
    If you still disagree with the discussion about p2pool after that, then it just means you didn't understand what you read, so read it again.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 25, 2012, 07:44:00 AM
    Below are 2 examples out of many that show organofcorti talking about p2pool earnings in relation to other pools:
    Quote from: organofcorti
    you're earning less than at a PPS pool.
    and
    Quote from: organofcorti
    you should continue mining at p2Pool, despite the lower earnings.
    I have provided 2 samples, 7 day recent and a 31 day historical that show his assertions are nonsense. organofcorti's assessment of p2pool profitability is erroneous, designed to confound and dishearten the uninformed.


    check_status, I'm happy for criticism and if I'm wrong I'll admit it and change my results. That's what thinking people do: read, learn, and change their minds when the facts show them to be wrong. Unfortunately you are not answering an important question which is: Where am I wrong?

    As far as your quotes go: since you didn't provide links I'm providing the full statements for context. I assume these are the quotes you are referring to, since I couldn't find those exact statements.

    The pool has been exceedingly lucky since the 14th June, with a mean round length of 0.6975. This means you'll get paid much more than PPS.

    However, this is a reflection of the variance inherent in bitcoin mining. If you recorded your fortnightly average, you'd find it much less than PPS, since the average p2Pool roundlength for the last two weeks was 1.204.

    The average round length since the start of p2Pool is about 1.11. So, since the start of p2Pool, average miner earnings are less than PPS.


    P2pool have few BIG advantages over other pools, thats why I support it, not only for income.

    And that's the reason you should continue mining at p2Pool, despite the lower earnings. I have no doubt that the current round length problem will be fixed eventually, but at the moment you are paying a price for the p2Pool advantages.


    I haven't written anything there that I have not proven. If you have a problem with the math, point it out to me. I provided you links with p2Pools round history and also to the data I used in handy .csv format. Even using Excel I can't see a quick mean of round length / D taking more than 10 minutes. Did you complete your own analysis? What was the result?

    Kano brings up a good point. I'm talking about average earnings since the start of the pool. In fact I already explained this, as you already know from the first quote above.




    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 25, 2012, 08:57:27 AM
    I've made some changes to the post, along the lines of Meni's comments. Apart from a few small changes in the main text, I changed the first points of the conclusion to:

    Quote
    • Although p2Pool's round lengths appear normally distributed, they have a mean value of 1.104D, and are greater than the expected value of D with 95% confidence. This is could be very "bad luck" but we cannot discount the possibility of the greater round lengths being inherent in the pool.  However, as Meni Rosenfeld points out there is no known mechanism by which this could occur.
    • A possibility is that the mechanism by which the equivalent number of D 1 shares are estimated could be faulty. This could mean the actual round lengths could be closer to the expected round lengths, and miner payouts woud not have been much less (or any less) than expected.
    • Orphan production rate does not appear to correlate with changes in P2pool hashrate....
    • ........
    I think what I'm getting down to, is... perhaps instead of looking to the average pool results to find normal, the pool results, should be considered above-normal instead of the baseline, which would be pure solo mining on a more distributed network?

    -- Smoov


    Good idea. As soon as I have enough data, in the Weekly pool statistics (https://bitcointalk.org/index.php?topic=77000.0) thread I'm adding charts showing
    Code:
    [ % orphans / % network orphans ]
    for each for which pool I can get orphan data. I started to include % orphans for each pool in the weekly stats table in the last post.


    Finally, Miner Earnings Insurance for DeepBit (https://bitcointalk.org/index.php?topic=87990.0) is here! If you don't mine on DeepBit give it a read anyway. I will offer the same service to other smaller pools if there are miners who want it.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: flower1024 on June 25, 2012, 09:00:47 AM
    maybe many p2pool miners have the problem with "LOST CONTACT WITH BITCOIND FOR 1.3min"
    ^^ i think this could explain longer round lengths

    is there any solution for this problem? will bitcoin0.7 help? does bitcoin0.7 even work with p2pool?


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 25, 2012, 09:10:44 AM
    maybe many p2pool miners have the problem with "LOST CONTACT WITH BITCOIND FOR 1.3min"
    ^^ i think this could explain longer round lengths

    is there any solution for this problem? will bitcoin0.7 help? does bitcoin0.7 even work with p2pool?

    I'm not sure how this would lead to longer round lengths - as far as I know, the equivalent D 1 shares are calculated using the pool hashrate, which is in turn calculated using the submitted variable difficulty shares.

    I'm probably wrong though - can you explain the above in a bit more detail?

    As far as solutions to the "LOST CONTACT WITH BITCOIND" problem goes, I'll leave that to the p2Pool community - I really don't know, sorry.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: flower1024 on June 25, 2012, 09:13:17 AM
    maybe many p2pool miners have the problem with "LOST CONTACT WITH BITCOIND FOR 1.3min"
    ^^ i think this could explain longer round lengths

    is there any solution for this problem? will bitcoin0.7 help? does bitcoin0.7 even work with p2pool?

    I'm not sure how this would lead to longer round lengths - as far as I know, the equivalent D 1 shares are calculated using the pool hashrate, which is in turn calculated using the submitted variable difficulty shares.

    I'm probably wrong though - can you explain the above in a bit more detail?

    As far as solutions to the "LOST CONTACT WITH BITCOIND" problem goes, I'll leave that to the p2Pool community - I really don't know, sorry.

    i thought you meant "time" by "round length".
    but if this refers to submitted shares i just dont know.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on June 25, 2012, 09:17:19 AM
    i thought you meant "time" by "round length".
    but if this refers to submitted shares i just dont know.

    We really have to standardise notation, eh? Yes, I was referring to total round shares.

    For anyone who doesn't follow, if round duration was increased this would lead to a lower pool hashrate overall, and reduced earnings for miners - but not increased numbers of shares per round.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: rav3n_pl on June 25, 2012, 07:46:07 PM
    maybe many p2pool miners have the problem with "LOST CONTACT WITH BITCOIND FOR 1.3min"
    ^^ i think this could explain longer round lengths

    is there any solution for this problem? will bitcoin0.7 help? does bitcoin0.7 even work with p2pool?
    It is 0.6.2 issue, load 0.6.99 and it is gone.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: rjk on June 25, 2012, 08:12:56 PM
    maybe many p2pool miners have the problem with "LOST CONTACT WITH BITCOIND FOR 1.3min"
    ^^ i think this could explain longer round lengths

    is there any solution for this problem? will bitcoin0.7 help? does bitcoin0.7 even work with p2pool?
    It is 0.6.2 issue, load 0.6.99 and it is gone.

    0.6.3 has been released, I assume it also contains the fix.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: check_status on June 30, 2012, 10:12:08 PM
    ]Since you feel the need to argue in a thread dealing with statistics ... but you are ignoring that fact ...
    Look up and learn all about these 2 words in detail and how they relate to statistics:
    Sample, Population.
    I know that statistics can be skewed to promote a particular view. As a matter of fact, statistics classes teach about representing perspectives with results, in the way election campaigns do or marketers.
    What value is there for statistics when 5th or 6th grade math is sufficient to prove organofcorti's statements are erroneous.
    Using statistics to determine profitability for p2pool would be like trying to open a jar of jelly with a sledge hammer. Sure you can do it, but it is way more than is required to accomplish the task.



    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Smoovious on June 30, 2012, 11:15:16 PM
    ]Since you feel the need to argue in a thread dealing with statistics ... but you are ignoring that fact ...
    Look up and learn all about these 2 words in detail and how they relate to statistics:
    Sample, Population.
    I know that statistics can be skewed to promote a particular view. As a matter of fact, statistics classes teach about representing perspectives with results, in the way election campaigns do or marketers.
    What value is there for statistics when 5th or 6th grade math is sufficient to prove organofcorti's statements are erroneous.
    Using statistics to determine profitability for p2pool would be like trying to open a jar of jelly with a sledge hammer. Sure you can do it, but it is way more than is required to accomplish the task.


    Then actually prove them wrong...

    Nothing wrong with disagreeing with his analysis, but you have to counter it with analysis of your own, that you two can pick apart and justify. This is how a scientific method works. He has his theory, and now it is up to his dissenters, to try and tear it apart to prove it in error.

    So far, I haven't seen any comparable analysis from your side yet. So... put up or shut up?

    -- Smoov


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: rav3n_pl on June 30, 2012, 11:57:21 PM
    Smoov, you have exact same thing in your stats:
    week:
    Local rate reflected in shares: 152
    Local rate: 132 (+15%)
    Month: 144/130 (+10%)

    Looks to me, that "smaller" miners (under 500MH/s, maybe higher too) are "overscored".
    Because there is alot of small miners in pool it can affect global pool stats alot.
    Pool rate and "expected time" is calculated from share rate.
    If share rate vs "real" hash rate vs predicted time vs real time is overcalculated (probably counted from block to block) all stats presented on site are inaccurate.

    Maybe only flaw in P2pool luck is some odd underestimation?


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Smoovious on July 01, 2012, 12:10:56 AM
    There are times where I get no payouts on a block, because I haven't been able to get a share for more than 24 hours, too.

    Pointing at my own tiny contribution is useless for statistical comparison. The sample is way too small, and margin for error too great, to make any reasonable analysis based on a random sample (which pointing to one peer, as representative of the whole, isn't random). Additionally, my own stats aren't stable. I do not operate a dedicated mining rig, so my mining is competing with all of the other programs I'm using too. Some have a high diskio, some use the GPU more than others, and I occasionally turn off my BTC mining entirely for periods of time. My stats would not be a good choice for analysis, by themselves. No reliable 'control'.

    The original analysis took the entire run of p2pool into account, no sample. Can't get more accurate than a 100% sample, so now, it is about the interpretation and methodology of how the analysis was done.

    The dissenting opinion of the analysis already presented, hasn't offered their own analysis to have a debate with. Until that happens, there isn't any point in arguing for or against the original analysis.

    Give me facts... leave the spin to the politicians.

    -- Smoov


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on July 01, 2012, 11:31:21 AM
    I know that statistics can be skewed to promote a particular view.

    From what I've read of your posts, I don't think that's something you know - it's something you've been told. So I'm encouraging you to learn about statistical analyses. http://oli.cmu.edu/courses/free-open/statistics-course-details/ is good, and assumes no prior stats knowledge on your part.

    As far as skewing data goes, I did the same analysis as always. If there was a problem with it, other analyses would have shown the same problem. Also, forum members better at math than I am would certainly have let me know.

    I know you're emotional about this, but that doesn't excuse you from accusing me of falsifying data when you don't yet have the knowledge on which to base such a claim. Get some learning, then start showing me where the analysis is wrong.




    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on July 01, 2012, 11:09:38 PM
    @ all the p2Pool miners that posted me data:

    Thanks for all your help! Unfortunately, the data is insufficient and in some cases measures different things.

    As much as I would have liked to have written a follow up from an earnings point of view, there were too few who collected actual numerical data, and I had to make too many guesses with the public node data. I'm just not very confident about the result.

    If anyone wants to start collecting data now, I need:

    timestamp* |  block number   |   valid shares  submitted for block   |   local average earnings for block  | difficulty for block*

    * = preferred data but not as essential as the other data listed.


    If I can get at least half the pool hashrate doing this for a two months, I'll have data which will have a much narrower and useful 95% confidence interval.

    check_status, I'd like you to be involved. If I show you what to do with the data once collected you might gain more of an insight into probability and statistics, and you can be confident that the results are correct since you will have collected some of the data and performed the analysis yourself as well.





    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: check_status on July 02, 2012, 07:54:51 PM
    From what I've read of your posts, I don't think that's something you know - it's something you've been told.
    Actually, by Actuaries, one of which is a family member, who passed the first 4 tests without a retest, and does complex math in the head I would have trouble entering into a calculator. Yes it's true my math sk1llz are limited to some simple algebra at best and Boolean equations make me cross-eyed.

    So I'm encouraging you to learn about statistical analyses. http://oli.cmu.edu/courses/free-open/statistics-course-details/ is good, and assumes no prior stats knowledge on your part.
    I appreciate the learning link, this will be handy in the future.

    I know you're emotional about this, but that doesn't excuse you from accusing me of falsifying data when you don't yet have the knowledge on which to base such a claim.
    I speak for and to those who can't do complex math to solve a simple problem. You say emotional I say concerned, but I guess we are each attempting to portray a perspective.

    Quote from: smoovious
    So far, I haven't seen any comparable analysis from your side yet. So... put up or shut up?

    -- Smoov
    So then, a graph, from beginning to current, showing actual v. expected, would be the only counter point you would except, is that right?
    Considering my math skill level that would take a month at least, that includes the youtube videos I will need to watch to learn how to use OoCalc. ;)


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on July 02, 2012, 10:59:36 PM
    From what I've read of your posts, I don't think that's something you know - it's something you've been told.
    Actually, by Actuaries, one of which is a family member, who passed the first 4 tests without a retest, and does complex math in the head I would have trouble entering into a calculator. Yes it's true my math sk1llz are limited to some simple algebra at best and Boolean equations make me cross-eyed.
    Get your actuary family member to have a read the blog post (if they understand pooled bitcoin mining) - I'm sure there will be at least something in there they'll disagree with, and I would like the feed back. I don't know any actuaries. I'd thought they were a myth, something to scare baby statisticians with when they're naughty.

    I know you're emotional about this, but that doesn't excuse you from accusing me of falsifying data when you don't yet have the knowledge on which to base such a claim.
    I speak for and to those who can't do complex math to solve a simple problem. You say emotional I say concerned, but I guess we are each attempting to portray a perspective.
    You didn't answer my main question - are you still ok with so forcefully stating that I'm anti-p2Pool and that I have falsified or skewed data? If you suspect I have not been honest in my analysis, then read the blog post. If you come to something you don't follow, post a message here and I'll try to explain it or provide links. It'll be a good learning experience for lots of people who would like to understand the analyses better.

    Quote from: smoovious
    So far, I haven't seen any comparable analysis from your side yet. So... put up or shut up?
    -- Smoov
    So then, a graph, from beginning to current, showing actual v. expected, would be the only counter point you would except, is that right?
    Considering my math skill level that would take a month at least, that includes the youtube videos I will need to watch to learn how to use OoCalc. ;)
    Here's one I provided in the blogpost with a 100 block rolling average (smooths the graph):

    http://i.imgur.com/ZgYzO.png (http://imgur.com/ZgYzO)



    In case you're wondering what effect the rolling mean might have, here's the original data:

    http://i.imgur.com/KvPU5.png (http://imgur.com/KvPU5)



    Finally, if you're going to use anything for math software and you know even a little BASIC or python, I'd try R  (http://www.r-project.org).




    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: Smoovious on July 03, 2012, 08:46:52 AM
    Quote from: smoovious
    So far, I haven't seen any comparable analysis from your side yet. So... put up or shut up?

    -- Smoov
    So then, a graph, from beginning to current, showing actual v. expected, would be the only counter point you would except, is that right?
    Considering my math skill level that would take a month at least, that includes the youtube videos I will need to watch to learn how to use OoCalc. ;)
    No... I'm saying, that... when faced with being up against one man's opinion, which he backs up with the data, which he put a lot of work into collecting, and processing it into something meaningful, explaining what he is looking for, what to expect, and what it is actually showing, in his interpretation...

    ...that you're going to have to do better to counter his opinion than with the equivalent of, "No it's not."

    You're going to have to counter his effort, with actual effort of your own, which proves his conclusions in error.

    We have yes-it-is, no-it-isn't, oh-yes-it-is, oh-no-it's-not debates all day every day in IRC, and in the forums too.

    This isn't one of those debates. He stepped up. Your turn.

    -- Smoov


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: check_status on July 18, 2012, 05:49:59 AM
    Quote from: smoovious
    So far, I haven't seen any comparable analysis from your side yet. So... put up or shut up?

    -- Smoov
    So then, a graph, from beginning to current, showing actual v. expected, would be the only counter point you would except, is that right?
    No... I'm saying, that... when faced with being up against one man's opinion, which he backs up with the data, which he put a lot of work into collecting, and processing it into something meaningful, explaining what he is looking for, what to expect, and what it is actually showing, in his interpretation...

    ...that you're going to have to do better to counter his opinion than with the equivalent of, "No it's not."

    You're going to have to counter his effort, with actual effort of your own, which proves his conclusions in error.

    We have yes-it-is, no-it-isn't, oh-yes-it-is, oh-no-it's-not debates all day every day in IRC, and in the forums too.

    This isn't one of those debates. He stepped up. Your turn.

    -- Smoov
    You seemed to be confused. I have posted results and that's not enough for you. The only logical extension that would end the disagreement is a complete actual v. expected comparison, to which you reply 'NO'.  ::)
    I have 114 days of data sampled from various periods of time, beginning, middle, middle+ and recent; 4 separate periods. I did post results on this forum 2 here and once in the p2pool thread, none of my samples shows a profit loss by mining at p2pool. Can I reasonably conclude that a statement which says 'p2pool is not profitable' is erroneous?

    To get a length of string into 8 pieces, you only need to divide by 7. ;) 1 / 7 = 8 :D

    @organofcorti  If round shares are so chaotic why would you use them in your calculations when simpler values are available?


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on July 18, 2012, 06:02:02 AM
    You seemed to be confused. I have posted results and that's not enough for you. The only logical extension that would end the disagreement is a complete actual v. expected comparison, to which you reply 'NO'.  ::)
    I have 114 days of data sampled from various periods of time, beginning, middle, middle+ and recent; 4 separate periods. I did post results on this forum 2 here and once in the p2pool thread, none of my samples shows a profit loss by mining at p2pool. Can I reasonably conclude that a statement which says 'p2pool is not profitable' is erroneous?

    I haven't seen these results - can you repost here?

    Quote
    @organofcorti  If round shares are so chaotic why would you use them in your calculations when simpler values are available?

    Which simpler values would you be referring to? I can't see a way to work out if the mean pool round length is significantly above the expected mean or not. (EDIT: without using total round shares)

    I'm not sure you can say round shares are chaotic, and I was wrong if I wrote that somewhere. p2Pool round shares/D still appear geometrically distributed, as is clear from the blog post.






    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: kano on July 18, 2012, 06:09:24 AM
    ...
    @organofcorti  If round shares are so chaotic why would you use them in your calculations when simpler values are available?
    Shares are the only accurate way to measure pool performance.
    Any other choice is meaningless (including saying 'I earned more than PPS')


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: check_status on July 18, 2012, 07:03:08 AM
    For every gigahash the number of shares per day is a constant value, 20,116.5676 shares per day.
    1000 megahashes = 20,116.5676 shares per day.
    1000 megahashes = 838.190317 shares per hour.

    Once discovered, a block becomes a constant value.
    Hash rate for a given period needs to be averaged, (2weeks to coincide with difficulty?).

    BTC found per 2 weeks / average gigahashes per 2 weeks = (actual value paid) PPGH for that 2 week period
    versus
    Expected Payout, like that determined using a Bitcoin Calculator.
    or
    (current block reward) / (current difficulty) = Expected PPS

    It appears that complex statistical analysis is NOT necessary to determine p2pools profitability.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on July 18, 2012, 07:26:50 AM
    BTC found per 2 weeks / average gigahashes per 2 weeks = (actual value paid) PPGH for that 2 week period
    versus
    Expected Payout, like that determined using a Bitcoin Calculator.
    or
    (current block reward) / (current difficulty) = Expected PPS
    It appears that complex statistical analysis is NOT necessary to determine p2pools profitability.

    Appearances can be deceptive.

    You might not realise it, but you are actually proposing quite a "complex statistical analysis" here. It might not be hard to calculate results, but how do you determine the the confidence interval? Stating that the results are "close to expected" is not enough. You need to be able to provide at minimum a confidence interval and show that both theoretical and empirical results lie within it.

    I agree that this method would provide a more robust analysis than shares per round/D. The problem with an analysis such as you suggest is that you need to have consistent payment data for a large portion of the pool and for a large portion of the pool's history. I'm not sure that the payment results for one miner - even starting from the history of the pool - would be sufficient to produce results with a suitable confidence interval. I did attempt to do this in the weeks following my post, but even with a good many miners sending me data it wasn't sufficient to rule out either the expected payments or payments at 0.91*expected.

    I'd still like you to post your data here so I can have a look at it though.


    Edit: Perhaps a better way to explain this is by asking you exactly what you're trying to prove. You can't just prove that your results are close to expected, you have to prove that your results exclude what you're trying to disprove, to a selected confidence level - usually p<0.05 or p<0.01. Start by reading about the null hypothesis (http://en.wikipedia.org/wiki/Null_hypothesis). The link to the online stats & probability course I gave in a previous post devotes a whole section to it.

    I hope you realise I'm not being contrary for the sake of it, but simply pointing out that analyses of this type are harder than you  think if you want to be able to come to a provable conclusion.









    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: kano on July 18, 2012, 08:40:52 AM
    For every gigahash the number of shares per day is a constant value, 20,116.5676 shares per day.
    1000 megahashes = 20,116.5676 shares per day.
    1000 megahashes = 838.190317 shares per hour.
    ...
    The number of shares you state are simply the expected number of shares.
    Every hour you check, you will see a different value.

    Over time with a LARGE sample you would expect the numbers towards converge to the expected values, however, making a statement like "1000 megahashes = 838.190317 shares per hour" is simply naive.

    GH/s -> shares is a random function.

    No pool can supply total hashes accurately.
    Miner programs can not even supply that accurately to the pools that accept it.

    However, more importantly, it doesn't matter what the calculation of GH/s -> shares is since that is in no way relevant to the expected performance of a pool.

    The performance of a pool is simply to show that over an extended period of time, it will find close to an expected number of blocks given a number of shares supplied by miners.
    It is certainly not any issue at all whatsoever to a pool, if a miner supplies less shares per MH/s than the miner expects.
    In fact no pool would even give a damn about it.

    ... and to give a very specific example ... I have a faulty BFL that seems to only find shares with one of the two FPGAs in it.
    I send the work to it and it responds in the expected amount of time (~5.1s) every time, however it finds half the expected number of shares.
    Why would my pool give a damn about that?

    In fact EVERY pool would estimate my hash rate as ~half what my BFL is saying it is doing ... as EVERY pool should.

    Hash rate should NEVER be used to determine pool performance since there is no way to even gauge it accurately.
    The hash rates shown for p2pool are simply the share rates converted to a hash rate (and who knows if that has even been done correctly or not ...)


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on July 18, 2012, 08:49:01 AM
    Hash rate should NEVER be used to determine pool performance since there is no way to even gauge it accurately.
    The hash rates shown for p2pool are simply the share rates converted to a hash rate (and who knows if that has even been done correctly or not ...)

    Actually, it's the other way around I think - hashrates determine the shares per round. But otherwise I agree. I would prefer to have actual round length data, but that's not possible unfortunately and could be a source of error.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: rav3n_pl on July 18, 2012, 11:07:35 AM
    Diff1 share = 2^48/65535 hashes = 4`295`032`833 hashes
    If you have 4,295GH/s you get 1 diff1 share every 1 second

    1. How p2pool calculate miner hash rate?
    - it counts diff1 shares over time diff1 share/sec is 4,3GH so 1 share ever 10 second is 430Mh/s
    2. How p2pool calculate network hash rate?
    - it counts last shares in share chain over time and multiply it by current chain share diff (maybe every share, not sure)

    IF we have slight over calculation there (like I rounded 4.295 to 4.3) pool hash rate can be inaccurate reported and p2pool.info block ETA can be wrong!


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on July 18, 2012, 11:19:06 AM
    IF we have slight over calculation there (like I rounded 4.295 to 4.3) pool hash rate can be inaccurate reported and p2pool.info block ETA can be wrong!


    That's right. However, it would only be out by the rounding error - in your example -0.005/4.3 = - 0.11%. However, the mean round shares are +10% compared to expected - that's a big rounding error. There could be some other miscalculation though.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: kano on August 02, 2012, 02:33:22 AM
    And to wake up the thread before it gets old ... :)
    So ... if someone was to get some BFL MiniRigs ... or some time in the far distant future ... a bunch of ASIC ...
    Wouldn't it be best for them to mine on a PPS pool and withhold blocks ... so they don't increase the difficulty and thus get more BTC ...


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: rampone on August 02, 2012, 03:08:11 AM
    Evil. PPS is so doomed...

    Has it been adressed, that p2pool includes lots of transactions in the block genrating a bigger size of block.

    The p2pool nodes are often/almost anytime behind home broadband. even with 1 mbit upstream a 130 kb block would take 8 seconds to propagate fully... could this explain the higher orphan rate?



    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: rav3n_pl on August 02, 2012, 08:16:31 AM
    The p2pool nodes are often/almost anytime behind home broadband. even with 1 mbit upstream a 130 kb block would take 8 seconds to propagate fully... could this explain the higher orphan rate?
    Thats why info about new block is spread between p2pool nodes :)


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: racerguy on August 02, 2012, 08:25:57 AM
    8secs would account for slightly over 1% orphans, if it's even that high.


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: zvs on August 02, 2012, 11:14:25 AM
    Evil. PPS is so doomed...

    Has it been adressed, that p2pool includes lots of transactions in the block genrating a bigger size of block.

    The p2pool nodes are often/almost anytime behind home broadband. even with 1 mbit upstream a 130 kb block would take 8 seconds to propagate fully... could this explain the higher orphan rate?



    it hasnt gotten an orphan since i started using it & set up my server (5.9.24.81).. about 3 weeks?


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on August 02, 2012, 11:37:39 AM
    Orphan rates for all pools are published in the Weekly Pool Stats thread. Orphans have been very low and round lengths shorter for p2Pool for a few weeks (I think).

    I hope that the luck increase is real and not just an artefact of the round length hashrate calculation method - everyone seeing an increase in earnings?


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on August 06, 2012, 02:18:09 PM
    And to wake up the thread before it gets old ... :)
    So ... if someone was to get some BFL MiniRigs ... or some time in the far distant future ... a bunch of ASIC ...
    Wouldn't it be best for them to mine on a PPS pool and withhold blocks ... so they don't increase the difficulty and thus get more BTC ...

    Sorry I missed this Kano. It's an interesting point. The block witholding attack has been mentioned before, but not in the context of someone with a large hashrate not wanting to affect difficulty. The miner would have to have a large proportion of the network hashrate to affect difficulty by himself, plus he'd have to hide the fact by using multiple miners on multiple pools.

    I think in this circumstance you everyone but the network would notice a problem. Say the attacker has 5% of the network hashrate and doesn't want to affect difficulty. He has to spread the hashrate over several workers (or one worker would look suspiciously unlucky) and at several pools (or one pool would look suspiciously unlucky). Then all the PPS pools will start looking slightly unlucky.

    If you found that round lengths were larger than normal on average but the luck index (calculated as log(Percentage of network blocks solved / Percentage of network hashrate) )  was averaging at zero, you'd start to suspect this could be happening.

    Luckily I have some charts somewhere that might help ;)


    Title: Re: NPW 4.2 Slush's score method and miner earnings part 1
    Post by: organofcorti on August 12, 2012, 12:07:32 AM
    4.2 Slush's score method and miner earnings part 1 is posted. This will be a three part series explaining the effects of the exponentially scored method on fulltime miner earnings.

    Quote
    5. Conclusions
    • Expected share values decrease with increasing total shares submitted until reaching a steady state.
    • The current steady state expected share value is about 0.957 and is reached shortly after the "hop point"
    • Increasing difficulty or decreasing 'c' or pool hashrate reduces the hop point and thus the pool hoppers potential profits; the converse is also true.
    • The theoretical maximum expected earnings loss is much lower than for a standard proportional pool; currently about 4.1% for Slush's pool as opposed to 43.4% for a standard proportional pool.

    http://3.bp.blogspot.com/-jHaj3W_6VNU/UCZDwdfpycI/AAAAAAAACMg/IcsLJUdJp90/s1600/npw4.2.plot.7.png


    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: organofcorti on August 27, 2012, 01:50:30 PM
    6.1 Bitlc.net - Pool hopping and unfulfilled promise (http://organofcorti.blogspot.com.au/2012/08/61-bitlcnet-pool-hopping-and.html) reports on a pool that started well, stagnated and then become one of the last few prop pools. What went wrong. And more to the point, who still mines there?


    Quote
    7. Conclusions

    * Bitlc.net began with a good service and great ideas. Unfortunately the pool has stagnated with few recent updates and an unfair reward method.
    * Bitlc.net could be a much more successful pool if the reward method was changed and if monetised in such a way that Jine was able to spend more time on the pool or delegate someone to manage it.
    * Fulltime miners at Bitlc.net currently lose approximately 30% of their expected earnings to pool hoppers. By comparison, at Deepbit.net the current percentage loss of earnings for fulltime proportionally rewarded miners to pool hoppers is 1.9%.
    * Mining at a 5% fee PPS pool would earning the fulltime miners at Bitlc.net 35% more than they earn at Bitlc.net currently.

    http://1.bp.blogspot.com/-EFFxt4t5k3k/UDjKCK7ioAI/AAAAAAAACUA/u-zTNkMQQ6s/s1600/npw6.chart10.png


    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: Shadow383 on August 28, 2012, 10:06:35 AM
    Very much in agreement regarding bitlc. That's been one of the pools my proxy has hopped for quite a long time now - the unfortunate fact of the matter is that whilst it's profitable to do we'll keep doing it.

    However, how did you come to your conclusion that only 6.7% of deepbit's hashrate pre-hop is pool hoppers? I was under the impression that ~500Ghash was hopping that pool, just like with Slush.


    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: organofcorti on August 28, 2012, 10:21:04 AM
    Very much in agreement regarding bitlc. That's been one of the pools my proxy has hopped for quite a long time now - the unfortunate fact of the matter is that whilst it's profitable to do we'll keep doing it.

    However, how did you come to your conclusion that only 6.7% of deepbit's hashrate pre-hop is pool hoppers? I was under the impression that ~500Ghash was hopping that pool, just like with Slush.

    It's an estimate, using the median average hashrate per round after 2*D as the fulltime miner hashrate, and  the median average hashrate per round after 2*D as (Fulltime + hopper) hashrate. It's close to results I've obtained in the past using more accurate estimations. I've noticed that the hop hashrate at Deepbit has been much less than when I posted NPW2.2.



    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: Shadow383 on August 28, 2012, 10:24:06 AM
    Very much in agreement regarding bitlc. That's been one of the pools my proxy has hopped for quite a long time now - the unfortunate fact of the matter is that whilst it's profitable to do we'll keep doing it.

    However, how did you come to your conclusion that only 6.7% of deepbit's hashrate pre-hop is pool hoppers? I was under the impression that ~500Ghash was hopping that pool, just like with Slush.

    It's an estimate, using the median average hashrate per round after 2*D as the fulltime miner hashrate, and  the median average hashrate per round after 2*D as (Fulltime + hopper) hashrate. It's close to results I've obtained in the past using more accurate estimations. I've noticed that the hop hashrate at Deepbit has been much less than when I posted NPW2.2.

    Interesting... There has been a bit of a ban wave on there lately, but mostly for larger hopping accounts (>5Gh/s or so). They don't seem to be IP banning yet.


    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: organofcorti on August 28, 2012, 10:57:03 AM
    I've noticed the hop rate dropped since mid April, so whatever is being done has been done since then.

    Edit: Thank you to my most recent anonymous donor.


    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: ShadesOfMarble on September 05, 2012, 01:27:00 PM
    I don't know if this pool is of interest because of its small hash rate: https://miner.ecki.net:444/

    But there seems to be serious evidence that the OP is cheating.
    After I found this post
    https://bitcointalk.org/index.php?topic=78391.msg874690#msg874690

    I made some investigations, i.e. clicked thru the "found blocks" page on the pool webpage and entered the block numbers on blockchain.info

    What I found is this:

    BTC 187,500 Distribution marrog 2012-07-04 20:43:02 537,341
    http://blockchain.info/block-index/242747/00000000000000461dcf498b2ba02f19dde9a98bb126de9d8e313b0bcdba5bfb
    => P2Pool


    BTC 188,332 Distribution glordglord 2012-07-10 04:10:01 966,959
    http://blockchain.info/block-index/244059/000000000000094b944cc807129a2e270753a565a3a8b37f609dc1e80a631e4c
    => P2Pool


    BTC 188,936 Distribution Mycom 2012-07-14 05:23:02 1,715,598
    http://blockchain.info/block-index/245155/00000000000001e4cee1f8f939b18d5248f367397ec5ec7ff8cf240cfb669ef8
    => OzCoin


    BTC 195,994 Distribution marrog 2012-08-28 05:32:05 7,313,184
    http://blockchain.info/block-index/269508/00000000000000e03ed033296e37cd676536ecbfdc600059dba12b25e2ca2fe3
    => BTC Guild


    So it's not the classic "faking round lengths" but the whole blocks are "faked".

    My question is: How reliable are the markers P2Pool, OzCoin and BTC Guild use? Is this strong enough evidence?

    I'm asking because I think you have way more experience in exposing cheating pool OPs.


    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: Graet on September 05, 2012, 01:53:31 PM
    not long after Ozcoin started I had a furious person join our IRC blaming us for "stealing" another pools block
    our software reported the wrong block number, we had the previous one.....

    since January all Ozcoin blocks have eco@ozco.in in the coinbase
    easily seen here http://blockchain.info/tx/cc8e9de906601a9f1537d83d0c86f48f76853cdae835d18141e1ca3186d9e62b/cc8e9de906601a9f1537d83d0c86f48f76853cdae835d18141e1ca3186d9e62b for example


    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: ShadesOfMarble on September 05, 2012, 02:00:24 PM
    So it could also be a bug in the pool software?


    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: organofcorti on September 05, 2012, 02:02:15 PM
    Interesting, but how would claiming blocks not theirs benefit the pool op? It's a PPLNS pool so he would have to pay extra to his miners. I can't quite see where the gain for the pool op is.

    I don't know how reliable the block claim markers are, and I'm not sure how accurate blockchain.info is.


    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: Graet on September 05, 2012, 02:03:58 PM
    So it could also be a bug in the pool software?
    would be my 1st guess :)
    I do see what you mean though, checked a couple of their blocks (including the Ozcoin one)


    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: ShadesOfMarble on September 05, 2012, 03:00:03 PM
    @Graet you were right. Block numbers are off. Blocknumber+1 is the correct one. So just a bug in the pool software :)


    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: Graet on September 05, 2012, 03:21:43 PM
    @Graet you were right. Block numbers are off. Blocknumber+1 is the correct one. So just a bug in the pool software :)
    good news :)


    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: organofcorti on September 05, 2012, 10:15:46 PM
    @Graet you were right. Block numbers are off. Blocknumber+1 is the correct one. So just a bug in the pool software :)

    Which ones are out by one - ecki.net - or the other pools?


    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: kano on September 05, 2012, 10:24:33 PM
    @Graet you were right. Block numbers are off. Blocknumber+1 is the correct one. So just a bug in the pool software :)

    Which ones are out by one - ecki.net - or the other pools?
    ecki.net or course ...
    As mentioned above you simply look at the coin base to identify a lot of pools.
    In fact around 50% of blocks can be identified directly by looking at the coinbase transaction - either they say the pool name or they are e.g. like p2pool you can identify it by the number of payouts and some of the addresses in there.


    Title: Re: NPW 6.1 Bitlc.net - Pool hopping and unfulfilled promise
    Post by: organofcorti on September 05, 2012, 10:28:03 PM
    @Graet you were right. Block numbers are off. Blocknumber+1 is the correct one. So just a bug in the pool software :)

    Which ones are out by one - ecki.net - or the other pools?
    ecki.net or course ...
    As mentioned above you simply look at the coin base to identify a lot of pools.
    In fact around 50% of blocks can be identified directly by looking at the coinbase transaction - either they say the pool name or they are e.g. like p2pool you can identify it by the number of payouts and some of the addresses in there.

    When I'm at work I have to deal with a locked down computer and internet - it's just easier to get other people to confirm things for me.


    Title: Re: NPW 4.3 Slush's score method and miner earnings part 2
    Post by: organofcorti on September 06, 2012, 02:23:04 PM
    New post : Neighbourhood Pool Watch 4.3 Slush's score method and miner earnings part 2 (http://organofcorti.blogspot.com.au/2012/09/43-slushs-score-method-and-miner.html)

    In this post I derive the actual current loss of earnings for fulltime miners at Slush's pool.

    Quote
    6. Conclusions
    • Slush's constant has changed to ~ 210. This has reduced the "hoppability" of the pool significantly. Currently, changing 'c' from 300 to 210 has reduced by half the maximum possible percent loss of earnings for fulltime miners.
    • The current loss experienced by fulltime miners is (as a proportion of the expected fee free PPS reward, B/D) for current D, pool hopper hashrate, full time miner hashrate and c is ~ 98.5% of PPS, where a standard proportional reward method would result in fulltime miners only earning ~ 82.8% of PPS.
    • Reducing Slush's constant 'c' makes pool hopping less profitable, but increases miner variance.

    http://4.bp.blogspot.com/-NWf3bs_2eFI/UEipmKW7csI/AAAAAAAACYQ/mGk4v1pdeXQ/s1600/npw4.3.plot.22012-09-06.png


    Title: Re: NPW 4.3 Slush's score method and miner earnings part 2
    Post by: emcg on September 11, 2012, 02:40:30 AM
    Thanks for the most recent topic as it was one I was waiting to read about.


    Title: Re: NPW 7.1 Variable pool difficulty
    Post by: organofcorti on October 10, 2012, 12:44:36 PM
    In this post (http://http//organofcorti.blogspot.com/2012/10/71-variable-pool-difficulty.html) I explain how to select either a particular number of submitted shares per minute or a particular pool difficulty to result in an acceptable variation in average hashrate.

    Quote
    4. Conclusions
    • Increasing average submitted shares per period only increases the standard deviation in submitted shares per period by the square root of the average submitted shares per period.
    • When difficulty varies constantly so shares per minute remains constant, the variation in average hashrate is not dependant on the miner's hashrate, but only on the number of shares per minute.
    • When difficulty is selected by the miner,  the variation in average hashrate is very much dependant on the miner's hashrate and the selected difficulty.


    http://3.bp.blogspot.com/-RwDc060-q6s/UHVXBo-cxLI/AAAAAAAACkk/I9i534G3v4c/s1600/npw7.1.chart1.png


    Title: Re: Neighbourhood Pool Watch 7.1 Variable pool difficulty
    Post by: Tittiez on October 11, 2012, 12:54:47 AM
    Current post:
    Neighbourhood Pool Watch 7.1 Variable pool difficulty (http://http://organofcorti.blogspot.com/2012/10/71-variable-pool-difficulty.html)  Starts here (https://bitcointalk.org/index.php?topic=66026.msg1262071#msg1262071)

    You accidentally a : in the link.


    Title: Re: Neighbourhood Pool Watch 7.1 Variable pool difficulty
    Post by: organofcorti on October 11, 2012, 01:16:59 AM
    Current post:
    Neighbourhood Pool Watch 7.1 Variable pool difficulty (http://http://organofcorti.blogspot.com/2012/10/71-variable-pool-difficulty.html)  Starts here (https://bitcointalk.org/index.php?topic=66026.msg1262071#msg1262071)

    You accidentally a : in the link.

    Oops! Fixed. Thanks for the sharp eye, Tittiez.


    Title: Re: Neighbourhood Pool Watch 8.1 BitcoinPool: Another Bitclockers?
    Post by: organofcorti on October 30, 2012, 10:36:04 AM
    In this installment of Neighborhood pool watch (http://organofcorti.blogspot.com/2012/10/81-bitcoinpool-another-bitclockers.html) I present part one of a three part series on BitcoinPool.

    From the post:

    Quote
    6. Discussion and conclusions

    I first noticed some anomalies in BitcoinPool's published data, and on 10th September 2012 contacted them to discuss my findings. Unfortunately I could not convince them that the data was anomalous in any way, and then Geebus simply stopped responding. I've not heard from them since and so I have decided to publish what I have.

    I hope it's clear that I am not inferring that the Geebus or FairUser have acted dishonestly, however I can say with 95% confidence that BitcoinPool's shares per round are much longer on average than expected, and we cannot reject that possibility that the shares per round do not belong to the expected statistical distribution. From this I can also say that Geebus and FairUser should be much more concerned about this than they appear.

    In the next post I will analyse in more detail where the anomalies lie, and hopefully provide sufficient information for Geebus and FairUser to look for possible problems in their pool. In a later post I will analyse the "pool hopping protection" that the pool features.

    http://2.bp.blogspot.com/-Edc_CK6xBOY/UI-TZtP69UI/AAAAAAAACtM/mvzhD3nePzs/s1600/npw8.1.chart3.png


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: organofcorti on November 01, 2012, 12:58:25 PM
    Another non-pool post, NPW 9.1 ASIC choices (http://organofcorti.blogspot.com.au/2012/11/91-asic-choices.html) is written to help miners determine which ASIC device will be best for them.

    Quote
    If my assumption that the average increase in D is 5% per difficulty period, then regardless of your differing mining environment it is plain that of the devices assessed the BFL SC Single will always be more profitable to run than any other miner and will have a longer profitable life time. It will also net more in the first year than the Avalon or the bASIC, and will usually pay for itself sooner than the other devices, depending on the local mining environment.

    If electricity is cheap or free and you don't think you'll start mining after D = 30 million, then the bASIC will pay for itself almost as quickly or slightly faster than the BFL SC Single, but won't net you as much in the first year. For a low starting difficulty and low electricity costs, the Avalon will net close to the BFL SC Single, but will take longer to pay for itself.

    http://1.bp.blogspot.com/-TzqkUhJuYGU/UJJtRlFDzTI/AAAAAAAACy8/omJ6EIi5rZk/s1600/npw9.1.chart3a.png


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: Bitinvestor on November 01, 2012, 01:06:52 PM
    Do you work for BSL? It is unknown how many watts the bASIC will use, but it won't use 300 watts. There have been hints in Tom's thread that it will be around 100 watts.


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: organofcorti on November 01, 2012, 01:15:17 PM
    Do you work for BSL? It is unknown how many watts the bASIC will use, but it won't use 300 watts. There have been hints in Tom's thread that it will be around 100 watts.

    No, if you had actually bothered read the post you'll note I explicitly write that I don't work for BFL.  If you can link to a better estimate on power usage then please do.

    Edit: I also explain how to do the calculations yourself and I've encouraged readers to do it for themselves. If you're not simply trolling, why not do your own calculations with your own estimates of power use? Further, I did note in the post that I will update the post as and when new and updated information becomes available.

    tl;dr: Please read the linked post before commenting.


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: Graet on November 01, 2012, 01:31:15 PM
    Do you work for BSL? It is unknown how many watts the bASIC will use, but it won't use 300 watts. There have been hints in Tom's thread that it will be around 100 watts.

    No, if you had actually bothered read the post you'll note I explicitly write that I don't work for BFL.  If you can link to a better estimate on power usage then please do.

    Edit: I also explain how to do the calculations yourself and I've encouraged readers to do it for themselves. If you're not simply trolling, why not do your own calculations with your own estimates of power use? Further, I did note in the post that I will update the post as and when new and updated information becomes available.

    tl;dr: Please read the linked post before commenting.
    I don't think its officially announced - so much spam and offtopic in bASIC thread I cant sift through it all.
    but Tom is selling 1000W PSUs to run "10" bASIC from - broadly suggesting that they will draw under 100W each

    great work on the write up, and yep, people with updated info should plug it in rather than throw accusations :0
    cheers
    graet


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: organofcorti on November 01, 2012, 01:41:36 PM
    Do you work for BSL? It is unknown how many watts the bASIC will use, but it won't use 300 watts. There have been hints in Tom's thread that it will be around 100 watts.

    No, if you had actually bothered read the post you'll note I explicitly write that I don't work for BFL.  If you can link to a better estimate on power usage then please do.

    Edit: I also explain how to do the calculations yourself and I've encouraged readers to do it for themselves. If you're not simply trolling, why not do your own calculations with your own estimates of power use? Further, I did note in the post that I will update the post as and when new and updated information becomes available.

    tl;dr: Please read the linked post before commenting.
    I don't think its officially announced - so much spam and offtopic in bASIC thread I cant sift through it all.
    but Tom is selling 1000W PSUs to run "10" bASIC from - broadly suggesting that they will draw under 100W each

    great work on the write up, and yep, people with updated info should plug it in rather than throw accusations :0
    cheers
    graet

    Glad I'm not the only one to give up trying to follow that thread. I think I'll wait until I get a clear indication from manufacturers of the power use specs before I write an updated post.

    But for readers of this thread who did not read the post, the whole point of the post was not just to compare products (although I did that in the conclusion). It was also to explain how a reader can do the same estimations I did. It's much easier to understand the results if you minimise complications and assumptions.


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: Bitinvestor on November 01, 2012, 01:47:28 PM
    people with updated info should plug it in rather than throw accusations :0

    Give me some time, I just had lunch. Tom posted some photos of his new place and on one of them the caption read "bASIC's will do 54Gh/s at under 100watts". I saw it myself but it has since been removed. I take it as an unofficial estimate.

    holy bovine

    Quote
    bASIC's will do 54Gh/s at under 100watts
    http://www.flickr.com/photos/88427916@N07/8075593408/in/photostream


    awww, tom edited the description...now i just sound crazy. at least psilan can back me up :P (i do have a screenshot though).

    In addition to that, the bASIC can be powered by a molex plug and someone stated that the maximum wattage a molex plug can supply is somewhere around 120-130 watts (I can't remember the exact number). Hope that helps.


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: organofcorti on November 01, 2012, 01:56:39 PM
    people with updated info should plug it in rather than throw accusations :0

    Give me some time, I just had lunch. Tom posted some photos of his new place and on one of them the caption read "bASIC's will do 54Gh/s at under 100watts". I saw it myself but it has since been removed. I take it as an unofficial estimate.

    holy bovine

    Quote
    bASIC's will do 54Gh/s at under 100watts
    http://www.flickr.com/photos/88427916@N07/8075593408/in/photostream


    awww, tom edited the description...now i just sound crazy. at least psilan can back me up :P (i do have a screenshot though).

    In addition to that, the bASIC can be powered by a molex plug and someone stated that the maximum wattage a molex plug can supply is somewhere around 120-130 watts (I can't remember the exact number). Hope that helps.

    Thanks for that. I'll update the post (with your maximum figure, 130 W - I prefer to use a conservative figure) during the next seven days assuming no other estimates come to light in that time. An official estimate - or one guaranteed not to change for at least a month - would be nice though.

    If anyone else has any power usage, hashrate, and cost estimates for the other ASIC devices for which I couldn't determine estimates, I'd be grateful if you'd post them here with a link.


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: Aseras on November 01, 2012, 05:59:41 PM
    Aren't you forgetting that the BFL SC and bASIC are going to need a PC of some sort, which is going to add at least another 100 watts?

    The avalon is a complete standalone unit.


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: MrTeal on November 01, 2012, 06:14:51 PM
    Aren't you forgetting that the BFL SC and bASIC are going to need a PC of some sort, which is going to add at least another 100 watts?

    The avalon is a complete standalone unit.

    1) It is dead simple to build a computer that uses less than 100W. A normal Ivy Bridge desktop can consume 60W at idle which is about the load that mining puts on the system. Small linux systems could mine using single digits watts.
    2) There's no requirement that you can run only one unit per computer. Several Singles or bASIC units could be run per computer, or even a couple minirigs.


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: DeepBit on November 01, 2012, 08:25:17 PM
    Isn't it a bit too early to evaluate ASIC parameters ? :)


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: organofcorti on November 01, 2012, 09:08:37 PM
    Aren't you forgetting that the BFL SC and bASIC are going to need a PC of some sort, which is going to add at least another 100 watts?

    The avalon is a complete standalone unit.

    No, I didn't forget that. The blog post does state my assumptions most clearly.


    I didn't know that Avalon was stand alone - and that will make a difference. Thanks for letting me know. Do you have a link to confirm this?


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: organofcorti on November 01, 2012, 09:13:48 PM
    Isn't it a bit too early to evaluate ASIC parameters ? :)

    What, because no device has even shipped yet? Yes, it's definitely too early to come to a final decision on which is best.

    But the post does address how to assess them so when there are shipped products. The biggest problem I'd noted was a lack of a simple, systematic and easy to follow method of assessment, and the post addresses that.

    So, any word on your product lines' specs yet?


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: DeepBit on November 01, 2012, 09:33:11 PM
    Isn't it a bit too early to evaluate ASIC parameters ? :)
    What, because no device has even shipped yet? Yes, it's definitely too early to come to a final decision on which is best.

    But the post does address how to assess them so when there are shipped products. The biggest problem I'd noted was a lack of a simple, systematic and easy to follow method of assessment, and the post addresses that.

    So, and word on your product lines' specs yet?
    What I mean is that no one knows their exact power requirements yet, which is very important.
    No, I don't know exact power figures on our product too, yet.


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: organofcorti on November 01, 2012, 10:02:48 PM
    Isn't it a bit too early to evaluate ASIC parameters ? :)
    What, because no device has even shipped yet? Yes, it's definitely too early to come to a final decision on which is best.

    But the post does address how to assess them so when there are shipped products. The biggest problem I'd noted was a lack of a simple, systematic and easy to follow method of assessment, and the post addresses that.

    So, and word on your product lines' specs yet?
    What I mean is that no one knows their exact power requirements yet, which is very important.
    No, I don't know exact power figures on our product too, yet.

    Yes, I knew what you meant, and I agree it's very important. I hope readers consider the blog post more as a primer on how to determine which ASIC will be best for their mining environment.


    Title: Re: Neighbourhood Pool Watch 9.2 ASIC choices update 2nd November
    Post by: organofcorti on November 02, 2012, 11:16:56 AM
    I've written an updated post with the suggested specifications:

    http://organofcorti.blogspot.com/2012/11/92-asic-choices-update-2nd-november.html

    I've attempted to stick to consensus, so I wont update again until there's new published information about device specs or I've made another transcription error.

    http://3.bp.blogspot.com/-U3Ftfyzmjpo/UJOqB9fbkNI/AAAAAAAAC8o/KGoLRyHDK44/s1600/npw9.1.chart3b+2012-11-02+.png


    Title: Re: Neighbourhood Pool Watch 9.2 ASIC choices update 2nd November
    Post by: kano on November 05, 2012, 08:55:28 AM
    Hmm ... I think it needs a big red watermark across it saying ... these numbers are fictional - don't take any notice of it.


    Title: Re: Neighbourhood Pool Watch 9.2 ASIC choices update 2nd November
    Post by: organofcorti on November 05, 2012, 08:58:05 AM
    Hmm ... I think it needs a big red watermark across it saying ... these numbers are fictional - don't take any notice of it.

    Hmmm ... I think kano didn't read the blog post:

    Quote
    Although the initial reason for the first ASIC choices post was to show miners some of the calculations that could be used to help determine which ASIC would be the right choice for their mining environment, most attention was focussed on the charts I used to illustrate the calculations. I'm kicking myself for not having used fake ASIC names and specifications but having used real devices I have a responsibility to ensure the specifications used are as correct as consensus opinion make them.


    Title: Re: Neighbourhood Pool Watch 9.2 ASIC choices update 2nd November
    Post by: kano on November 05, 2012, 04:40:58 PM
    Hmm ... I think it needs a big red watermark across it saying ... these numbers are fictional - don't take any notice of it.

    Hmmm ... I think kano didn't read the blog post:

    Quote
    Although the initial reason for the first ASIC choices post was to show miners some of the calculations that could be used to help determine which ASIC would be the right choice for their mining environment, most attention was focussed on the charts I used to illustrate the calculations. I'm kicking myself for not having used fake ASIC names and specifications but having used real devices I have a responsibility to ensure the specifications used are as correct as consensus opinion make them.
    Yep I saw the pretty picture and that was more than enough info ... until I saw the numbers ... that I know a bit about :D


    Title: Re: Neighbourhood Pool Watch 9.2 ASIC choices update 2nd November
    Post by: organofcorti on November 05, 2012, 08:35:53 PM
    Hmm ... I think it needs a big red watermark across it saying ... these numbers are fictional - don't take any notice of it.

    Hmmm ... I think kano didn't read the blog post:

    Quote
    Although the initial reason for the first ASIC choices post was to show miners some of the calculations that could be used to help determine which ASIC would be the right choice for their mining environment, most attention was focussed on the charts I used to illustrate the calculations. I'm kicking myself for not having used fake ASIC names and specifications but having used real devices I have a responsibility to ensure the specifications used are as correct as consensus opinion make them.
    Yep I saw the pretty picture and that was more than enough info ... until I saw the numbers ... that I know a bit about :D

    Alrighty then - if you're not talking about the hashrate / price / power consumption figures (which are used as examples to explain how others can make these calculations for themselves) what numbers are you talking about? If I've derived something incorrectly I'd like to know. Stop being so mysterious.


    Title: Re: Neighbourhood Pool Watch 9.3 More on ASIC choices
    Post by: organofcorti on November 06, 2012, 11:40:42 PM
    Yep I saw the pretty picture and that was more than enough info ...

    I heard you like pretty charts, so this one is dedicated to you ;)

    Quote
    Chart 1: (new) In order to read this group of charts, find the intersection of a percentage ROI and number of difficulty periods (eg. % ROI after one year is at ~ 26 difficulty periods). The colour of the tile is an indicator of the exchange rate required to meet this %ROI after the given number of difficulty periods. The faint white line along the middle of each plot indicates the break even point. Click on a chart for enlargement.

    http://4.bp.blogspot.com/-BDCNFtGwYnw/UJkcWy_WCxI/AAAAAAAADIA/Nrdp1JwFTPs/s1600/npw9.3.chart.avalon.rate+2012-11-07+.png



    More charts and more detail at http://organofcorti.blogspot.com.au/2012/11/93-more-on-asic-choices.html


    Title: Re: Neighbourhood Pool Watch 9.3 More on ASIC choices
    Post by: kano on November 07, 2012, 02:10:56 AM
    Yep I saw the pretty picture and that was more than enough info ...

    I heard you like pretty charts, so this one is dedicated to you ;)

    Quote
    Chart 1: (new) In order to read this group of charts, find the intersection of a percentage ROI and number of difficulty periods (eg. % ROI after one year is at ~ 26 difficulty periods). The colour of the tile is an indicator of the exchange rate required to meet this %ROI after the given number of difficulty periods. The faint white line along the middle of each plot indicates the break even point. Click on a chart for enlargement.

    http://4.bp.blogspot.com/-BDCNFtGwYnw/UJkcWy_WCxI/AAAAAAAADIA/Nrdp1JwFTPs/s1600/npw9.3.chart.avalon.rate+2012-11-07+.png



    More charts and more detail at http://organofcorti.blogspot.com.au/2012/11/93-more-on-asic-choices.html
    Awww pretty :)

    BFL=65nm was announced today - so the BFL figures before sound like they should be reliable at least ...


    Title: Re: Neighbourhood Pool Watch 9.3 More on ASIC choices
    Post by: organofcorti on November 07, 2012, 02:48:55 AM

    ............

    More charts and more detail at http://organofcorti.blogspot.com.au/2012/11/93-more-on-asic-choices.html
    Awww pretty :)

    BFL=65nm was announced today - so the BFL figures before sound like they should be reliable at least ...


    Aha! It was the device specs you were querying, not the results. Glad we cleared that up. Also glad you liked the pretty Kanoplot.


    Title: Re: Neighbourhood Pool Watch 9.1 ASIC choices
    Post by: Aseras on November 07, 2012, 04:08:21 PM
    Aren't you forgetting that the BFL SC and bASIC are going to need a PC of some sort, which is going to add at least another 100 watts?

    The avalon is a complete standalone unit.

    No, I didn't forget that. The blog post does state my assumptions most clearly.


    I didn't know that Avalon was stand alone - and that will make a difference. Thanks for letting me know. Do you have a link to confirm this?

    It's always been advertised as such

    http://store.avalon-asic.com/

    http://forum.bitsyn.com/viewtopic.php?id=5

    Quote
    Avalon self-contained unit >= 60GH/s guaranteed.
    pre-order price: $1,299.
    note: host PC NOT required. LAN/WIFI connection required.


    Title: Re: Neighbourhood Pool Watch 9.3 More on ASIC choices
    Post by: zvs on November 07, 2012, 04:28:09 PM
    OK, so the thing here is to hope that none of these ppl making these ASICs mine anything for themselves, and you're one of the few that gets them in the "first week" of availability?

    otherwise, why buy one?  wouldnt it be wiser to just use that money to buy bitcoins instead?   if they retain their value, you're out nothing.  if they retain their value but difficulty goes up with asic, you're waiting some years?   

    wouldn't seem very logical for me that if bitcoins were to increase in value (making mining more profitable), that ASICs would also increase in value.  added competition over time should result in reduced costs


    Title: Re: Neighbourhood Pool Watch 9.3 More on ASIC choices
    Post by: kano on November 07, 2012, 08:24:11 PM
    /me looks into his crystal ball
     ... future prediction! :P ASIC price per MH/s will drop over time.

    Also, look at the pretty pictures ... they do actually help you find when you will make a profit ...

    and yes you do have the right idea :) Don't buy ASIC - so everyone who has ASIC gets paid more due to the lower difficulty :)

    and lastly, at the gold fields, the shovel makers get all the gold :D


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: rav3n_pl on November 13, 2012, 03:56:08 PM
    Neighbourhood Pool Watch 5.1 p2Pool - bad luck or flawed? (http://organofcorti.blogspot.com.au/2012/06/51-p2pool-bad-luck-or-flawed.html) is posted.
    Looks like it need update, or another chapter :)
    http://p2pool.info/luck/
    All-time luck is back to 100%, bugs that possibly cause bad luck/orphans are fixed IMO.
    Long live P2pool! :D


    Title: Re: NPW 5.1 p2Pool - bad luck or flawed?
    Post by: organofcorti on November 14, 2012, 12:54:57 AM
    Neighbourhood Pool Watch 5.1 p2Pool - bad luck or flawed? (http://organofcorti.blogspot.com/2012/06/51-p2pool-bad-luck-or-flawed.html) is posted.
    Looks like it need update, or another chapter :)
    http://p2pool.info/luck/
    All-time luck is back to 100%, bugs that possibly cause bad luck/orphans are fixed IMO.
    Long live P2pool! :D


    I've been meaning to do follow ups for all posts, I just haven't had the time. Rest assured, I'll get on to it at some point. It would be interesting to see if there was some point where there's an obvious change in mean/orphans and relate it to a particular patch.


    Title: Re: Neighbourhood Pool Watch 9.3 More on ASIC choices
    Post by: kano on November 14, 2012, 02:16:45 AM
    Got a (somewhat self-serving) suggestion for a "pool" watch subject :D

    Transactions per block, kb per block, BTC fees per block, BTC fees per txn (and of course network % blocks per whatever range)

    With all this crap going on about what is good for BTC, clearly this will show what is (at least IMO) good for BTC.
    IMO good for BTC is blocks that confirm transactions - since no txn's means no one is using BTC which means bye-bye BTC :P

    I'd go as far as suggesting that pools that are near the top of this report (except for network %) should get the most miners and others with much lower stats should get fewer/less - but of course that is just IMO

    Anyway, it is mostly all there in the blockchain - the only problems are the issues of determining who owns half the blocks
    (~50% of them either name who they are or are obviously p2pool)
    The rest I guess you have to trust the pools to say which are their blocks (e.g. deepbit)

    I would also suspect that many of the unknowns would be at the bottom of the ranking ...


    Title: Re: Neighbourhood Pool Watch 9.3 More on ASIC choices
    Post by: organofcorti on November 14, 2012, 02:34:28 AM
    Got a (somewhat self-serving) suggestion for a "pool" watch subject :D

    Transactions per block, kb per block, BTC fees per block, BTC fees per txn (and of course network % blocks per whatever range)

    With all this crap going on about what is good for BTC, clearly this will show what is (at least IMO) good for BTC.
    IMO good for BTC is blocks that confirm transactions - since no txn's means no one is using BTC which means bye-bye BTC :P

    I'd go as far as suggesting that pools that are near the top of this report (except for network %) should get the most miners and others with much lower stats should get fewer/less - but of course that is just IMO

    Anyway, it is mostly all there in the blockchain - the only problems are the issues of determining who owns half the blocks
    (~50% of them either name who they are or are obviously p2pool)
    The rest I guess you have to trust the pools to say which are their blocks (e.g. deepbit)

    I would also suspect that many of the unknowns would be at the bottom of the ranking ...

    Something like I did here (https://bitcointalk.org/index.php?topic=88302.msg984376#msg984376)? Yes, I've wanted to do a more general statistical analysis of blockchain variables, however getting the data is problematic. I can't seem to get any blockchain browser to compile on my macs. Do you have an alternative source?


    Title: Re: Neighbourhood Pool Watch 9.3 More on ASIC choices
    Post by: kano on November 14, 2012, 03:30:22 AM
    No
    I just write all my own php to talk to bitcoind
    But to identify ~50% of blocks you just look in the coinbase transaction:

    The strings I used to look for were:
    Code:
    'slush' => 'Slush',
    'BTC Guild' => 'BTC Guild',
    'ozco.in' => 'OzCoin',
    'EclipseMC' => 'EclipseMC',
    'Eligius' => 'Eligius',
    'simplecoin.us' => 'SimpleCoin', // 171650
    'BitLC' => 'BitCL', // 171638
    'BitMinter' => 'BitMinter', // 171617
    'nmcbit.com' => 'NMCBit', // 172774
    'bitclockers' => 'BitClockers', // 174478
    'mkalinin.ru' => 'Mkalinin', // 174585
    'Triplemining.com' => 'Triplemining', // 175144
    'MaxBTC' => 'MaxBTC', // 175144
    'CoinLab' => 'CoinLab' // 197765
    You can see how old the info is :)

    As for the other 50% - the problem there is you have to go to the pools (like deepbit) to get the info.


    Title: Re: Neighbourhood Pool Watch 9.3 More on ASIC choices
    Post by: K1773R on November 14, 2012, 09:55:52 AM
    No
    I just write all my own php to talk to bitcoind
    But to identify ~50% of blocks you just look in the coinbase transaction:

    The strings I used to look for were:
    Code:
    'slush' => 'Slush',
    'BTC Guild' => 'BTC Guild',
    'ozco.in' => 'OzCoin',
    'EclipseMC' => 'EclipseMC',
    'Eligius' => 'Eligius',
    'simplecoin.us' => 'SimpleCoin', // 171650
    'BitLC' => 'BitCL', // 171638
    'BitMinter' => 'BitMinter', // 171617
    'nmcbit.com' => 'NMCBit', // 172774
    'bitclockers' => 'BitClockers', // 174478
    'mkalinin.ru' => 'Mkalinin', // 174585
    'Triplemining.com' => 'Triplemining', // 175144
    'MaxBTC' => 'MaxBTC', // 175144
    'CoinLab' => 'CoinLab' // 197765
    You can see how old the info is :)

    As for the other 50% - the problem there is you have to go to the pools (like deepbit) to get the info.
    care to share? :)


    Title: Re: Neighbourhood Pool Watch 9.3 More on ASIC choices
    Post by: organofcorti on November 14, 2012, 11:29:19 AM
    No
    I just write all my own php to talk to bitcoind
    But to identify ~50% of blocks you just look in the coinbase transaction:

    The strings I used to look for were:
    Code:
    'slush' => 'Slush',
    'BTC Guild' => 'BTC Guild',
    'ozco.in' => 'OzCoin',
    'EclipseMC' => 'EclipseMC',
    'Eligius' => 'Eligius',
    'simplecoin.us' => 'SimpleCoin', // 171650
    'BitLC' => 'BitCL', // 171638
    'BitMinter' => 'BitMinter', // 171617
    'nmcbit.com' => 'NMCBit', // 172774
    'bitclockers' => 'BitClockers', // 174478
    'mkalinin.ru' => 'Mkalinin', // 174585
    'Triplemining.com' => 'Triplemining', // 175144
    'MaxBTC' => 'MaxBTC', // 175144
    'CoinLab' => 'CoinLab' // 197765
    You can see how old the info is :)

    As for the other 50% - the problem there is you have to go to the pools (like deepbit) to get the info.

    Ah well. If you do manage to get the data together let me know and I'll help you with analysis.


    Title: NPW 5.2 p2Pool: Achieving expectation.
    Post by: organofcorti on November 20, 2012, 02:26:04 PM
    Neighbourhood Pool Watch 5.1 p2Pool - bad luck or flawed? (http://organofcorti.blogspot.com.au/2012/06/51-p2pool-bad-luck-or-flawed.html) is posted.
    Looks like it need update, or another chapter :)
    http://p2pool.info/luck/
    All-time luck is back to 100%, bugs that possibly cause bad luck/orphans are fixed IMO.
    Long live P2pool! :D


    Since rav3n_pl asked so nicely, this post is an update to the original p2Pool post.

    It answers some questions miners were asking and isn't able to answer other questions they weren't asking.

    Quote
    7. Discussion and conclusions
    7. Discussion and conclusions
    It is quite clear from this analysis that p2Pool's run of bad luck - in terms of difficulty 1 shares per round / D and orphaned block production - is over, and I hope I've dispelled the myth that an increased pool hashrate increase orphans and kills the pool's luck. I'll leave interested readers to provide an explanation of the odd correlation between orphaned block production, shares per round / D and certain time periods  - please do comment below if you have any suggestions yourself.

    • All of the sample statistics fit well in the expected range, and we can't reject the hypothesis that p2Pool's shares per round / D are exponentially distributed.
    • There is no correlation between pool luck and pool hashrate, or orphaned block production and pool hashrate, as some have suggested.
    • Since solved block 500, the cumulative mean shares per round / D and the cumulative orphans produced per block seem to have both increased and decreased at the same time.
    • My guess is that the change in cumulative mean shares per round / D is a coincidence, and the change in orphaned block production is related somehow to changes in the p2Pool client.


    I hope this post goes some way to encouraging more miners to test p2Pool for themselves.

    Enjoy!


    http://3.bp.blogspot.com/-YP3wwLozBbc/UKthB9kotVI/AAAAAAAADnQ/GAEf22THY5c/s1600/npw5.2chart4.2012-11-20.png


    Title: Re: Neighbourhood Pool Watch 5.2 p2Pool: Achieving expectation
    Post by: organofcorti on November 22, 2012, 01:08:54 PM
    I'm normally not the kind of person to go "I told you so"-ing all over the place, however:

    @organofcorti Why do you work so hard to show how bad p2pool is? Do you have a dog in the race?

    Quote from: organofcorti
    Given the decentralised nature of the pool, some proponents feel that this is an acceptable price.

    Let's disect this statement shall we:

    a decentralized pool = an acceptable price

    This connotes a sacrifice in some other area. According to the majority of organofcorti's posts, that is profitability. Your using words to paint an unprofitable picture about p2pool that doesn't exist, the blocks attest to this.

    It's not out of line at all. There is a preponderance of evidence of organofcorti's p2pool posts where he flat out says or implies p2pool is the least profitable pool.
    Please provide a link to a post where I flat out say that p2Pool is the least profitable pool.

    Below are 2 examples out of many that show organofcorti talking about p2pool earnings in relation to other pools:
    Quote from: organofcorti
    you're earning less than at a PPS pool.
    and
    Quote from: organofcorti
    you should continue mining at p2Pool, despite the lower earnings.

    I have provided 2 samples, 7 day recent and a 31 day historical that show his assertions are nonsense. organofcorti's assessment of p2pool profitability is erroneous, designed to confound and dishearten the uninformed.

    ......
    It appears that complex statistical analysis is NOT necessary to determine p2pools profitability.


    Any chance of an apology, check_status? :)


    Title: Re: Neighbourhood Pool Watch 8.2 BitcoinPool: The great anomaly hunt.
    Post by: organofcorti on January 10, 2013, 02:06:35 PM
    I finally got around to finishing part 2 of the BitcoinPool series: Neighbourhood Pool Watch 8.2 BitcoinPool: The great anomaly hunt. (http://organofcorti.blogspot.com/2013/01/82-bitcoinpool-great-anomaly-hunt.html)

    Lots of weirdness happening at this pool, and it seems that the BitcoinPool miners are suffering:

    Quote
    6. Discussion and summary.
    What has the great anomaly hunt revealed? Some strange and improbable anomalies, certainly. For a start, It is extremely improbable that the statistics of rounds 480 onward are distributed as they should be.

    To answer my initial questions:

    Did the anomalous rounds occur equally at all times, or only for a particular epoch of time?
    Rounds 1 to 479 have very different statistics to those from 480 onward
    Did the anomalous rounds occur equallyfor all sized rounds, or for some particular sized rounds only?
    For rounds 480, the number of rounds having shares per round between 0 to 0.11 x D, and 0.36 to 0.51 x D are improbably small, and the number of rounds having shares per round greater than 2.3 x D are improbably many.
    These facts do not really help us explain what the cause of the anomalies are. I have not been able to determine the way this could occur - either accidentally or on purpose. For example:
    Block withholding attack (see Meni Rosenfeld's "Analysis of Bitcoin Pooled Mining Reward Systems" for details): A "sabotage" attack is unlikely - the anomaly has been occurred at BitcoinPool for nearly 18 months (at the time of writing), and I find it unlikely that anyone would be able to keep up this kind "no reward" of attack for so long. Also, to what end? If I'm the first analyst to find this, it's not an obvious problem for the pool. A "lie in wait" attack might be more likely, but I'm not sure how to calculate how profitable it would be for BitcoinPool's piecemeal reward function. Certainly possible though.
    From pool op Geebus:
    "The efficiency of any given miner, if not returning a share that could have potentially have had the answer to that block due to not hashing it before another entity in the bitcoin network solves a block, and long polling triggers, could change the outcome. "
     Although this might seem intuitive to some, the efficiency of an arbitrary miner cannot affect the number of shares required for the pool to solve a block. Only valid shares received by the pool are recorded; non shares that are not sent by low efficiency miners are not recorded.

    Pool operator malfeasance (someone with access to the pool server and data base): I think this is unlikely. If, for example, round with few shares are not being recorded and simply being added to the next round, then we would expect to see a spike in the next largest sized round, then a slight dip for the next largest shares per round /D bin, and slightly more than expected for the remainder of larger shares per round /D bins. The histogram shows that this is not the case. Applying some sort of function to the result of the number of shares per round (as was almost certainly the case for bitclockers.com before they became a PPS pool) would not have had such an apparently discontinuous effect on the pools shares per round / D distribution.
    The most likely of the explanations that I've been able to analyse (and some I couldn't) seems to be a "lie in wait" attack. Even this is quite unlikely though - the hashrate required to make such an attack usefully profitable is quite high, and the pool's total percentage of the network is less than 0.2%.

    So I'm not sure what the mechanism behind the anomalies is, or whether it's an internal or external attack, or if perhaps the shares are not being recorded properly in some strange way that targets only specific sized rounds. I am however quite confident that this is not just a period of bad luck for BitcoinPool - there are just too many extremely improbable anomalies for that to have occurred.


    Until Geebus and FairUser can investigate and solve this problem, I'd avoid mining at BitcoinPool.com - the last 18 months history of the pool suggest you'll earn an average of only 80% of your expected mining reward.




    Title: Re: Neighbourhood Pool Watch 8.2 BitcoinPool: The great anomaly hunt.
    Post by: rav3n_pl on January 10, 2013, 05:40:21 PM
    20% fee pool last 18 moths? WOW!


    Title: Re: Neighbourhood Pool Watch 8.2 BitcoinPool: The great anomaly hunt.
    Post by: K1773R on January 10, 2013, 08:39:28 PM
    20% fee pool last 18 moths? WOW!
    deepbit version 2.0?


    Title: Re: Neighbourhood Pool Watch 8.2 BitcoinPool: The great anomaly hunt.
    Post by: organofcorti on January 11, 2013, 07:57:12 AM
    20% fee pool last 18 moths? WOW!

    I thought this post would get a bit more traction to be honest. There was quite an uproar when I found that Bitclockers.com was either underpaying their miners by ~21% or incorrectly reporting their stats to avoid paying pool hoppers. BitcoinPool's data shows that miners there have definitely been paid  ~ 80% PPS over the last 18 months - of that there is no doubt. It's also extremely unlikely that it's just bad luck - the data isn't even distributed as it should be, aside from the higher mean shares per round /Difficulty.

    Maybe it's just that in a low hashrate pool there aren't many miners, and maybe they don't read this board.

    The how and why of the glaring anomaly I don't know, and I don't think the pool ops are ripping their miners off - the data doesn't seem to support that.  I've made the anomalies I found as clear as I could and I've provided the dataset so hopefully someone a bit more cunning than I can figure out what the problem is.


    Title: Re: Neighbourhood Pool Watch 8.2 BitcoinPool: The great anomaly hunt.
    Post by: organofcorti on January 11, 2013, 07:58:26 AM
    20% fee pool last 18 moths? WOW!
    deepbit version 2.0?

    Well, it's not really a pool fee - BitcoinPool only take a donation. But on average, miners there have been paid 20% less than PPS since July 2011.


    Title: Re: Neighbourhood Pool Watch 12.1 The pre-ASIC network hashrate distribution
    Post by: organofcorti on April 10, 2013, 01:10:05 PM
    New post:

    Neighbourhood Pool Watch 12.1 The pre-ASIC network hashrate distribution. (http://organofcorti.blogspot.com/2013/04/121-pre-asic-network-hashrate.html)

    Quote
    5. Conclusions:
    • Hashrates are distributes amongst contributors as a Pareto II random variable.
    • Hashrate ownership percentages follow the Pareto Principle: ~ 20 percent of pool accounts contribute ~ 80% of the hashrate.
    • Minimum hashrates apparently vary by groups - four pools have hashrate averages per contributor approaching 2Ghps, and some have averages less than a quarter of that.

    6. Next:
    The reason for creating a sample of fulltime miners also had another purpose - to allow an estimation of the post-ASIC network hashrate, when the next plateau in hashrate will occur. I will do just that and also  explain how you can estimate this for yourself in the next post 12.2 Predicting post-ASIC network hashrates.


    http://3.bp.blogspot.com/-XZe7sFinWVQ/UWQaH6l6MHI/AAAAAAAAHUk/pRD0MxvRtB4/s1600/npw12.1.violinplot.2013-04-09.png (http://organofcorti.blogspot.com/2013/04/121-pre-asic-network-hashrate.html)


    Title: Re: Neighbourhood Pool Watch 12.2 12.2 Pool and network miner hashrate distributions
    Post by: organofcorti on May 03, 2013, 01:27:04 PM
    New post:

    Neighbourhood Pool Watch 12.2 Pool and network miner hashrate distributions. (http://organofcorti.blogspot.com/2013/05/122-pool-and-network-miner-hashrate_3.html)

    Quote
    5. Conclusions.
    • Most miners still have sub 1Ghps mining tools.
    • A visibly large number of user accounts are at ~65Ghps, likely as a result of using an Avalon ASIC mining tool.
      • Pre-ASIC, hashrates were continuously distributed from small to large, leading to a very smooth and rounded density plot. Post-ASIC, the distribution becomes quite lumpy - since Avalon ASICs were the only ones available up until this point, large amounts of hashrate exist in the "hips", at integer multiples of 65 Ghps or thereabouts.
      • The distribution of users accounts and user account hashrates is very much in flux, and given the combined distributions "lumpiness" it's unlikely that a Pareto II distribution will model the current user account hashrate distribution very well.
    http://1.bp.blogspot.com/-ulhkIvjfX7o/UYOqy3TjSzI/AAAAAAAAHtU/74bmnBtSXR8/s1600/npw12.2.eCDF.violin1.2013-05-03.png (http://organofcorti.blogspot.com/2013/05/122-pool-and-network-miner-hashrate_3.html)


    http://3.bp.blogspot.com/-Mnpb59rZRNE/UYOqy4YqVhI/AAAAAAAAHtY/CyN0eFVPX2s/s1600/npw12.2.eCDF.violin2.2013-05-03.png (http://organofcorti.blogspot.com/2013/05/122-pool-and-network-miner-hashrate_3.html)


    Title: Re: Neighbourhood Pool Watch 13.1: BitMinter and "luck".
    Post by: organofcorti on May 09, 2013, 11:00:53 PM
    New post:

    Neighbourhood Pool Watch 13.1: BitMinter and "luck". (http://organofcorti.blogspot.com/2013/05/131-bitminter-and-luck.html)


    Quote

    9. Conclusions.

    •   BitMinter's "luck" both over the history of the pool and over the last two months is not unusual. While this does not prove that the pool is running normally, there is no evidence that it is not running normally.
    •   Shares per round / Difficulty are distributed as expected for pooled bitcoin mining.
    •   The claim that miners at BitMinter have earned 10% less than expected over the past month are untrue.
    •   When transaction fee earnings and merged mining are taken into account, earnings are greater than for a fee free PPS pool that doesn't merged mine or include transaction fees. Miners can be confident that even in when recent luck has been poor, they're still earning more than at a PPS pool that doesn't pay transaction fees and doesn't merged mine.

    I think there are several reasons that concerns arose about BitMinter's luck.
    1.   There was a large influx of new miners who have not yet learned all there is to know about pooled bitcoin mining, and many older miners who have had to answer these questions repeatedly have started to avoid answering the questions, since from previous experience these explanations become long and time consuming. Perhaps I'll write a post on "luck" and pooled bitcoin mining  - then new miners could simply be directed to it when the "luck" questions arise.
    2.   Many miners seem unaware of the extra income from merged mining and transaction fees.
    3.   BitMinter's charts are also part of the problem In order for miners to intuitively understand whether or not statistics are unusual, confidence intervals must be provided. Without that guide, miners have no way to judge how bad or good "luck" has been over a given period of time. To my knowledge there are no public pools that that provide this data, so BitMinter is not the only pool with this problem. Another smaller issue is that on charts that do not provide time axis dates, miners find it hard to judge over how long a period of time luck has been either good or bad.




    http://3.bp.blogspot.com/-moeeJmunoy8/UYtq3rhmlUI/AAAAAAAAH2w/p-e3sweWOdA/s1600/npw12.1chart5.2013-05-09.png (http://organofcorti.blogspot.com/2013/05/131-bitminter-and-luck.htm)


    Title: Re: Neighbourhood Pool Watch 13.1: BitMinter and "luck".
    Post by: kano on May 10, 2013, 01:08:28 AM
    Meh - I always shudder when people use numbers other than shares/difficulty
    Anything else is effectively either derived from that (usually with added inaccuracies) or estimated (and even more inaccurate).
    I was wondering about that in the BitMinter thread for a while ... until you chimed in with the results :)


    Title: Re: Neighbourhood Pool Watch 13.1: BitMinter and "luck".
    Post by: organofcorti on May 10, 2013, 02:00:32 PM
    Meh - I always shudder when people use numbers other than shares/difficulty
    Anything else is effectively either derived from that (usually with added inaccuracies) or estimated (and even more inaccurate).

    Shares/difficulty is definitely my preferred metric, and the one with which I have the most experience. But there is a another metric not directly derived from shares/difficulty, and that is payments per share (for PPLNS pools). Payments per share is Poisson distributed with a mean of N, so it's just as amenable to analysis as shares/difficulty.

    But I get what you mean - there are a bunch of ways people have tried to figure out luck, most of them assuming that the variables are normally distributed which they are emphatically not.

    Figuring out probabilities is not hard once you know what you're doing. I'll write a post explaining how to do it using Wolfram Alpha if I get a chance - then no one has to listen to "Argh my pool paid out 10% less than normal over the last month!", since they can figure out the truth of the statement and if true how probable it is. Then it's just a matter of getting used to interpreting the results.


    I was wondering about that in the BitMinter thread for a while ... until you chimed in with the results :)

    .... and now it's gone all quiet over there ;)



    Title: Re: Neighbourhood Pool Watch 12.4 Miner hashrate distributions 28th May 2013
    Post by: organofcorti on May 29, 2013, 11:22:02 AM
    New post, and a new way to visualise changes in the user hashrate distribution. There's and excerpt below, but there's lots more at http://organofcorti.blogspot.com/2013/05/124-miner-hashrate-distributions-28th.html

    Quote
    0. Introduction
    Although the violin plots used previously were quite good at showing trends, they weren't so good for showing clear changes to total pool hashrates and total numbers of miners, or how particular amounts of hashrate changed over time. As well as this, the violins were not a good method to present an open ended chronological series.

    So instead I grouped user hashrates and plotted by time:

    The number of user accounts in that group.
    The total hashrate attributable to that group.
    All the charts in this post represent this, using various colours to represent various groups of hashrate. Think of them as a layer cake, building upwards to a total number of miners or total hashrate and to the right over time.

    In doing this I gained new insight into how users hashrates were changing, and also noticed a problem with Itzod's miner count - it seemed to top out at 500 miners. On investigation I realised this was a problem with the way I'd been scraping the data which is now fixed.



    1. Change in miner count and grouped hashrates by pool.
    Each coloured layer in the charts below represents either the number of miners (left plot) or the total hashrate contributed by that group (right plot) - ASICMiner solo will of course only ever have a miner count of 1.

    Some trends are evident:
    Nearly all miners have a hashrate less than 5 Ghps, but user accounts with more than 5Ghps contribute far more to the total hashrate of each pool.
    The amount of very low hashrate miners at BitMinter has decreased and the amount of hashrate contributed by high hashrate miners has increased significantly, and the same is true for BTCGuild.
    The number of low hashrate miners at Eligius has increased.
    At Ozcoin and HHTT, miners with hashrates greater than 5 Ghps make up more of the user base than at other pools.
    1 - 4.99 Ghps miners at P2Pool, Itzod and Polmine contribute more to the total hashrate than any other group.
    I'll leave it to the reader to find the other trends in visible in the charts.

    It should be noted that the data from the end of January to mid April is interpolated. I have no data point in that time frame. After that point data points are weekly (or every few days).


    http://1.bp.blogspot.com/-3wOD6itS8B8/UaTAgGnpk2I/AAAAAAAAIKQ/psa6aCOfCL4/s1600/npw12.4.chart1a-2013-05-28.png (http://organofcorti.blogspot.com/2013/05/124-miner-hashrate-distributions-28th.html)
    http://3.bp.blogspot.com/-sRcAAeoOwD4/UaTA5fiWVWI/AAAAAAAAIKY/De-Vx0QNUe8/s1600/npw12.4.chart1b-2013-05-28.png (http://organofcorti.blogspot.com/2013/05/124-miner-hashrate-distributions-28th.html)


    Title: Re: Neighbourhood Pool Watch 12.4 Miner hashrate distributions 28th May 2013
    Post by: organofcorti on October 22, 2013, 12:27:23 PM
    New post: Neighbourhood Pool Watch 16.1 The network: Orphaned blocks part 1

    http://organofcorti.blogspot.com/2013/10/161-network-orphaned-blocks-part-1.html

    It's just part 1, I hope to get the rest out soon.

    Quote
    0. Introduction
    Most miners become fascinated with orphaned blocks at some point - What are they? This can be simply answered, and also answered simply. But then the questions escalate (or at least mine did): Why do orphaned blocks occur? Can they be minimised or predicted? Which pool has the fewest? What factors most influence the number of orphans produced by the network?  Miners then find very few answers and mostly forget about orphaned blocks (until they lose income to an orphan race), putting them in the same category as forces of nature.

    http://1.bp.blogspot.com/-Og-IY9WS9kc/UmUyY0wAwoI/AAAAAAAAKHA/xCc_sQQE_G4/s1600/plot0.2013-10-22.png


    Title: Re: Neighbourhood Pool Watch 16.2 Mapping the historical pool hashrate
    Post by: organofcorti on November 20, 2013, 01:15:48 PM
    New post: 16.2 A short post on mapping the historical pool hashrate (http://organofcorti.blogspot.com/2013/11/162-short-post-on-mapping-historical.html)

    Except below:

    Quote
    0. Introduction

    For a while I've been trying to find better ways to present pool hashrate distributions. Pie charts are OK - most people understand them (although as you should all be aware, pie charts have severe problems and can mislead you), but I wanted something that a) was not misleading and b) could show proportions of the network attributable to specific pools. I also wanted to show cumulative data, rather than the plots I usually use to show the historical hashrate distribution.

    .....





    Title: Re: Neighbourhood Pool Watch 16.2 Mapping the historical pool hashrate
    Post by: ok on November 23, 2013, 05:51:44 PM
    Very good quality posts on this thread, keep it up.


    Title: Re: Neighbourhood Pool Watch 16.2 Mapping the historical pool hashrate
    Post by: eleuthria on November 23, 2013, 06:54:44 PM
    Sorry I can't give you the May 2011 through ~October 2011 speeds for BTC Guild.  All that information was wiped when the pool switched from free proportional to PPS :(.

    One request/suggestion:  Perhaps a graph which only shows a smaller subset of pools?  It's virtually impossible to distinguish them when their colors are so similar.


    Title: Re: Neighbourhood Pool Watch 16.2 Mapping the historical pool hashrate
    Post by: organofcorti on November 24, 2013, 06:06:10 AM
    Sorry I can't give you the May 2011 through ~October 2011 speeds for BTC Guild.  All that information was wiped when the pool switched from free proportional to PPS :(.

    I thought this would be a problem, so the charts don't show hashrate - just blocks per week and then blocks per week as a percentage of total of network blocks per week. Do you have block history going back before that?


     chart doesn't show hashrate

    One request/suggestion:  Perhaps a graph which only shows a smaller subset of pools?  It's virtually impossible to distinguish them when their colors are so similar.

    I did try that, but then you lose a feel for how much of the network the pools represent, and how many smaller pools there are, when they started and when they died etc.

    What I'd like to do is learn how to make it interactive. I'd like to be able to mouse over the chart, the whole band corresponding to the colour over which you've moused gets highlighted and the pool name appears. I think this could be done using JS (which I've never used) and some JS charting tools (that I've never used), so there's a big learning curve ahead of me.

    Very good quality posts on this thread, keep it up.

    Cheers!


    Title: Re: Neighbourhood Pool Watch 16.2 Mapping the historical pool hashrate
    Post by: eleuthria on November 24, 2013, 07:46:30 AM
    Sorry I can't give you the May 2011 through ~October 2011 speeds for BTC Guild.  All that information was wiped when the pool switched from free proportional to PPS :(.

    I thought this would be a problem, so the charts don't show hashrate - just blocks per week and then blocks per week as a percentage of total of network blocks per week. Do you have block history going back before that?

    Unfortunately, BTC Guild history pre-October 2011 is pretty much lost :(.  It was only about 5 months old, and I was still learning as I went along back then.  I didn't quite realize importance of keeping all that just for historical accuracy back then, after only being involved in BTC for half a year, paired with the long sustained decline in value after the 2011 bubble popped.  We didn't start signing the coinbase until PoolServerJ was released.


    Title: Re: Neighbourhood Pool Watch 16.2 Mapping the historical pool hashrate
    Post by: organofcorti on November 24, 2013, 08:46:47 AM
    Sorry I can't give you the May 2011 through ~October 2011 speeds for BTC Guild.  All that information was wiped when the pool switched from free proportional to PPS :(.

    I thought this would be a problem, so the charts don't show hashrate - just blocks per week and then blocks per week as a percentage of total of network blocks per week. Do you have block history going back before that?

    Unfortunately, BTC Guild history pre-October 2011 is pretty much lost :(.  It was only about 5 months old, and I was still learning as I went along back then.  I didn't quite realize importance of keeping all that just for historical accuracy back then, after only being involved in BTC for half a year, paired with the long sustained decline in value after the 2011 bubble popped.  We didn't start signing the coinbase until PoolServerJ was released.

    When was PoolServerJ released? The early data I have is from coinbase signatures only - do you have much data before that? (I can't load up the dataset atm, and I can't tell from the chart - there might be a month I don't have).


    Title: Re: Neighbourhood Pool Watch 17 Goodbye HHTT. The coinbase will never be as ......
    Post by: organofcorti on January 02, 2014, 07:27:21 AM
    New post:

    17 Goodbye HHTT. The coinbase will never be as strange again. (http://organofcorti.blogspot.com/2014/01/17-goodbye-hhtt-coinbase-will-never-be.html)

    Here's the whole thing in full (although with most of the links missing)

    Quote
    NOTE: There will be no Weekly Pool and Network Statistics this week, as I'm going to having a communications-free week. Instead, I've written a short post commemorating one of the strangest and most fun pools of the last couple of years.

    0. Introduction

    Recently, fireduck (the pool operator of the HHTT bitcoin pool), announced that he would be closing the pool on New Year's Day. This made me a little sad, since it was a great pool which had some innovative fee mechanisms, and was one of the first pools to take variable share submission difficulty seriously. The open source stratum server he developed has been used by at least one other pool (just grep "SOCK" in the coinbase and you'll see which one) and saw very little downtime.

    From my (non-mining) point of view, however, the best thing about HHTT was the sporadic and weird coinbase messages. I had been planning a post about my favourite bitcoin blockchain coinbase messages, but as a remembrance of all things HHTT I decided just to focus on theirs.

    The chart below shows HHTT's percentage of weekly network blocks solved, and if you click on the image you can read the coinbase messages that were added at that point in time. If you haven't got a magnifying glass handy (since Blogger seems to compress the chart somewhat) then HHTT's unicode coinbase messages, along with the block height of the block with which they were included, are here: http://bitbin.it/VOXWstm9. I will leave it up to the reader to translate the Japanese keyboard emoticons (such as β•―Β°β–‘Β°οΌ‰β•―οΈ΅ β”»β” β” ) . It should also be noted that block 229417 message appears to be not a penis, but an ASCII sword. Finally, "Finders keepers" had half a coin in it and was found not long after the message appeared. More like that, I say!

    So long and thanks for all the weirdness, fireduck. I had lots of fun wondering what weird message you'd come up with, and was more than a bit disappointed that I hadn't seen any recently. There should be more creativity and fun in the blockchain!

    http://4.bp.blogspot.com/-cIzv8HI1_vA/UsUPCVtLZyI/AAAAAAAAKq8/Ey5s7h_rLK8/s1600/npw17.png

    organofcorti.blogspot.com is a reader supported blog:
    12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r



    Title: Re: Neighbourhood Pool Watch 17 Goodbye HHTT. The coinbase will never be as ......
    Post by: organofcorti on January 02, 2014, 07:34:58 AM
    ....
    The funny thing is, my hate is from being a pool hopper yet i supported you more than almost all!

    Is this addressed to me, fireduck, or are you proposing your own coinbase message? I'll admit you have me stumped, since I thought we were chums.


    Title: Re: Neighbourhood Pool Watch 17 Goodbye HHTT. The coinbase will never be as ......
    Post by: organofcorti on January 02, 2014, 07:45:32 AM
    ....
    The funny thing is, my hate is from being a pool hopper yet i supported you more than almost all!

    Is this addressed to me, fireduck, or are you proposing your own coinbase message? I'll admit you have me stumped, since I thought we were chums.

    We are chums. I respect you more than almost anyone in BTC land. I am under attack from gmaxwell right now so I am a bit unpolite. Sorry.

    I figured I'd just misunderstood your meaning. I should have put a ;) in there somewhere.

    So, my two most favourite firebrands in a duel ;) Or something more serious?

    Can you post a link?


    Title: Re: Neighbourhood Pool Watch 16.3 The unknown network hashrate
    Post by: organofcorti on January 13, 2014, 03:47:38 PM
    Here's a treat for all you tinfoil hat wearers worried about the "unknown" hashrate owners:

    16.3 The unknown network hashrate. (http://organofcorti.blogspot.com/2014/01/163-unknown-network-hashrate.html)

    Quote
    0. Introduction
    When miners start looking at the weekly pie charts, they often start to wonder "Who is this "unknown"? What is their nefarious purpose?". If they start to read the daily pie charts from blockchain.info and notice "unknown" at 10 to 20 %, they may even start to become concerned.

    My recent work on providing unified block attribution history (last week's blocks here: http://bitbin.it/3iIbl6tA) has made it simple to find a bit more about the "unknown".

    .....

    http://4.bp.blogspot.com/-SVAa9PaTxZM/UtQAkkZM5zI/AAAAAAAALZQ/u0wDe2QBPoM/s1600/genAddressPlot2.2014-01-14.png (http://organofcorti.blogspot.com/2014/01/163-unknown-network-hashrate.html)

    .....

    2. I hate not knowing! So here's some bounties.
    I think most of these will end up being pools, a few early adopters of FPGAs and ASICs, and some of the recent ones will be businesses that don't sign the coinbase.


    To get the ball rolling, I'll pay 0.005 btc for the first readers to identify any of the addresses in the table below, on the conditions that they can be identified conclusively (proof required).

    .....


    Title: Re: Neighbourhood Pool Watch 9.18 Gridseed - should you mine bitcoin or litecoin?
    Post by: organofcorti on March 15, 2014, 02:49:37 PM
    If you just bought a Gridseed and were wondering whether or not it might be more profitable to mine bitcoin, then this post is for you.

    http://organofcorti.blogspot.com/2014/03/918-gridseed-should-you-mine-bitcoin-or.html

    Quote
    0. Introduction

    It has been a while since I last mined, and when I heard that Gridseed ASICs could mine both bitcoin and litecoin, I decided to throw caution to the wind, ignore the fact that I may never get a return on my investment, and purchase a batch of ten through seeds over at bitcointalk.org (in case you're in the market for a Gridseed miner, I can recommend seeds: he delivered within ten days, and when I had problems setting the miners up he had an engineer help me out remotely)

    These ASICs can mine either bitcoin or litecoin or both simultaneously, although the point of simultaneous ltc and btc mining eludes me - surely one would simply mine whichever coin was most profitable at the time?  When mining bitcoin (SHA256 mode) one device can produce 8 Ghps at 53W; mining litecoin (scrypt mode) one device can produce 300 Mhps at 7W.

    So, which is more profitable to mine with a Gridseed miner - bitcoin or litecoin?

    http://3.bp.blogspot.com/-KC2WJBlXrbE/UyRJnTFkbCI/AAAAAAAALq0/wGzamJZyJu4/s1600/gridseed2.png (http://organofcorti.blogspot.com/2014/03/918-gridseed-should-you-mine-bitcoin-or.html)


    Title: Re: Neighbourhood Pool Watch 18.1 Who owns 1HTM4TYSXF5yZKLpco6MTUUNfSBCiiwGsU ?
    Post by: organofcorti on March 17, 2014, 09:36:19 AM
    Neighbourhood Pool Watch 18.1 Who owns 1HTM4TYSXF5yZKLpco6MTUUNfSBCiiwGsU ? (http://organofcorti.blogspot.com/2014/03/181-who-owns-1htm4tysxf5yzklpco6mtuunfs.html)

    Quote
    Monday 17th March, 2014

    Essential reading:
    Wikipedia entry on "Graph theory"

    Other posts covering similar topics: 
    16.3 The unknown network hashrate
    16.4 Updated unknown network hashrate

    Other interesting links:
    Bitcoin Transaction Graph Analysis
    Quantitative Analysis of the Full Bitcoin Transaction Graph
    Do the rich get richer? An empirical analysis of the Bitcoin transaction network
    An Analysis of Anonymity in the Bitcoin System
    An Analysis of Anonymity in Bitcoin Using P2P Network Traffic

    0. Introduction
    A while back in posts 16.3 and 16.4 I pointed out some of the generation addresses from sources of unknown bitcoin network hashes. My aim was not to somehow "out" smaller solominers but to attempt to find out if some of the already known hash sources might be trying to hide some of the blocks they solve in an effort to appear less threatening to the network. If some large pool was to hide some of the blocks it solves, then some miners would be losing income, and the pool could approach 50% of the network without anyone knowing.

    A pool that was hiding some of its solved blocks would appear to have slightly poor luck over a long period. It is not hard to detect poor luck over a long period, but you wouldn't be looking for it without a reason. And even if you did, all you could say was that public hashrate contributor 'X' had statistically unlikely luck.

    I wanted an indicator that was a little more clear than that; and if there was no proof tying the new sources of hashes to a pool, then I hoped to have enough information to identify some of the new hash contributors (as I did for last week for KNC) or set some bounties (as I did in posts  16.3 and 16.4).


    http://2.bp.blogspot.com/-ZPyZmxyuG2E/UyWWotoldFI/AAAAAAAALsA/5lIM3O715-s/s1600/followGen1.png (http://organofcorti.blogspot.com/2014/03/181-who-owns-1htm4tysxf5yzklpco6mtuunfs.html)


    Title: Re: Neighbourhood Pool Watch 18.1 Who owns 1HTM4TYSXF5yZKLpco6MTUUNfSBCiiwGsU ?
    Post by: abracadabra on March 17, 2014, 01:23:31 PM
    OoC,  have you managed to determine which, if any of these "unknown" addresses, are Bitmain's? They claimed that their products were/are powering 20% of the network.


    Title: Re: Neighbourhood Pool Watch 18.1 Who owns 1HTM4TYSXF5yZKLpco6MTUUNfSBCiiwGsU ?
    Post by: organofcorti on March 17, 2014, 01:44:27 PM
    OoC,  have you managed to determine which, if any of these "unknown" addresses, are Bitmain's? They claimed that their products were/are powering 20% of the network.

    No, I have no idea who owns the addresses - but if you have some addresses you think they might be I can try to find out the shortest distance between them.


    Title: Re: Neighbourhood Pool Watch 9.18 Gridseed - should you mine bitcoin or litecoin?
    Post by: kano on March 26, 2014, 04:11:54 AM
    If you just bought a Gridseed and were wondering whether or not it might be more profitable to make less loss if you mine bitcoin, then this post is for you.

    http://organofcorti.blogspot.com/2014/03/918-gridseed-should-you-mine-bitcoin-or.html

    FTFY


    Title: Re: Neighbourhood Pool Watch 9.18 Gridseed - should you mine bitcoin or litecoin?
    Post by: organofcorti on March 26, 2014, 04:20:54 AM
    If you just bought a Gridseed and were wondering whether or not it might be more profitable to make less loss if you mine bitcoin, then this post is for you.

    http://organofcorti.blogspot.com/2014/03/918-gridseed-should-you-mine-bitcoin-or.html

    FTFY

    You can (only just) mine BTC at a profit with these things, even at 35c per kWh. That should hold true for a few months yet.

    I'm not assessing the net profit (after purchase) though, which might be what you're thinking. I did say " I decided to throw caution to the wind, ignore the fact that I may never get a return on my investment, and purchase a batch of ten"



    Title: Re: Neighbourhood Pool Watch 9.18 Gridseed - should you mine bitcoin or litecoin?
    Post by: kano on March 26, 2014, 04:22:46 AM
    If you just bought a Gridseed and were wondering whether or not it might be more profitable to make less loss if you mine bitcoin, then this post is for you.

    http://organofcorti.blogspot.com/2014/03/918-gridseed-should-you-mine-bitcoin-or.html

    FTFY

    You can (only just) mine BTC at a profit with these things, even at 35c per kWh. That should hold true for a few months yet.

    I'm not assessing the net profit (after purchase) though, which might be what you're thinking. I did say " I decided to throw caution to the wind, ignore the fact that I may will never get a positive return on my investment, and purchase a batch of ten"

    FTFY

    Edit: https://bitcointalk.org/index.php?topic=456691.msg5281847#msg5281847 :)


    Title: Re: Neighbourhood Pool Watch 9.18 Gridseed - should you mine bitcoin or litecoin?
    Post by: organofcorti on March 26, 2014, 04:27:06 AM
    If you just bought a Gridseed and were wondering whether or not it might be more profitable to make less loss if you mine bitcoin, then this post is for you.

    http://organofcorti.blogspot.com/2014/03/918-gridseed-should-you-mine-bitcoin-or.html

    FTFY

    You can (only just) mine BTC at a profit with these things, even at 35c per kWh. That should hold true for a few months yet.

    I'm not assessing the net profit (after purchase) though, which might be what you're thinking. I did say " I decided to throw caution to the wind, ignore the fact that I may will never get a positive return on my investment, and purchase a batch of ten"

    I may have FTFY

    Edit: https://bitcointalk.org/index.php?topic=456691.msg5281847#msg5281847 :)

    FTFY


    Title: Re: Neighbourhood Pool Watch 16.7 Satoshi's hashrate and mining hardware.
    Post by: organofcorti on August 13, 2014, 05:08:12 PM
    Neighbourhood Pool Watch 16.7 Satoshi's hashrate (http://organofcorti.blogspot.com/2014/08/167-satoshis-hashrate.html)

    This post is an investigation into Satoshis' hashrate and asks the question - did he have a hashrate plan? Some excerpts:

    Quote
    0. Introduction
    There are many reasons I enjoy working at coinometrics.com, and having access to data I wouldn't otherwise have is chief among them. Recently, Jonathan Levin had been discussing whether or not Satoshi's coins had been moved (which was in the news at the time) and Martin Harrigan of QuantaBytes (who had recently started working with us) was able to provide the sort of data that we needed to assign blocks to Satoshi.

    With that information I was able to do what I'd wanted to do since I first started mining - estimate Satoshi's hashrate. At the time I didn't realise that I would also be provided with a small window into the great man's thinking - his plan for managing his personal hashrate.

    http://3.bp.blogspot.com/-1moTKwfI18U/U-tDPnzhIxI/AAAAAAAAAa8/76WLA2FKDU4/s1600/satoshisHashrate2014-08-13.png

    Quote
    5. Summary
        * Satoshi initially followed a plan of reducing the hashrate by 1.7 Mhps every five months, but a month after the second such drop abandoned this method in favour of a continuously decreasing hashrate.
        * This implies two things: Satoshi had a hashrate plan, and that initially Satoshi had very coarse-grained control of his hashrate, but then was able to achieve a very fine grained control of the hashrate.
        * These insights may help investigators determine the sort of hardware Satoshi used, and how much control he would have had over the early client or his PC / cloud infrastructure.


    Title: Re: Neighbourhood Pool Watch 16.7 Satoshi's hashrate and mining hardware.
    Post by: organofcorti on August 22, 2014, 12:44:33 PM
    Neighbourhood Pool Watch 16.9 Another short post on mapping the historical hashrate distribution (http://organofcorti.blogspot.com/2014/08/169-another-short-post-on-mapping.html)


    Follows on from previous work referenced in the post.

    ]http://2.bp.blogspot.com/-rZ4bqF1yP8g/U_cyKr2QRKI/AAAAAAAAAd4/vtp_7_8A8q0/s1600/PoolPercentageNetworkBlocks-2014-08-22.png (http://organofcorti.blogspot.com/2014/08/169-another-short-post-on-mapping.html)


    1. Some important points
    It should be noted that some of the pre-2012 attributions will have errors. Some of these are:

        * Slush's pool blocks are missing from late 2010 to early 2012.
        * Some BTC Guild blocks are missing in late 2011.
        * Attributions for some of the very early miners may be incomplete.
        * The methods used attribute blocks to Satoshi Nakamoto are outlined in Satoshi's hashrate and A little more on Satoshi's blocks.



    Title: Re: Neighbourhood Pool Watch 16.9 Another short post on mapping the historical ....
    Post by: btcash on November 10, 2014, 09:46:01 AM
    Your latest blog post made me curious about F2Pool hashrate distribution. So I checked the last 1000 blocks for F2Pool signatures.

    276 blocks found by 155 different miner.
    Top five miner account for ~ 29% of the pools hashing power:
    daqi4ant: 21 - 7.61%
    f2poolscant: 19 - 6.88%
    qq123456: 19 - 6.88%
    avhb: 12 - 4.35%
    jua005: 9 - 3.26%

    I know this is not fully accurate but percentages stay roughly the same if I go back 3000 blocks.

    BTW: You are doing a great job!


    Title: Re: Neighbourhood Pool Watch 16.9 Another short post on mapping the historical ....
    Post by: organofcorti on November 11, 2014, 12:32:43 PM
    Your latest blog post made me curious about F2Pool hashrate distribution. So I checked the last 1000 blocks for F2Pool signatures.

    276 blocks found by 155 different miner.
    Top five miner account for ~ 29% of the pools hashing power:
    daqi4ant: 21 - 7.61%
    f2poolscant: 19 - 6.88%
    qq123456: 19 - 6.88%
    avhb: 12 - 4.35%
    jua005: 9 - 3.26%

    I know this is not fully accurate but percentages stay roughly the same if I go back 3000 blocks.

    BTW: You are doing a great job!

    Interesting - that could be a good way to get a basic idea of the hashrate distribution but the averaging time would be variable. Full marks for coming up with the plan though!


    Title: Re: Neighbourhood Pool Watch: Measures of network hashrate centralisation
    Post by: organofcorti on December 12, 2014, 01:06:01 AM
    http://organofcorti.blogspot.com.au/2014/12/measures-of-network-hashrate.html is a new post that I thought you all might find interesting:

    Quote
    0. Introduction
    I think we're all aware there is significant concern that if block makers (pools and large solo mining farms) are able to create significant proportions of the network's blocks in an arbitrary time period, then they could cause problems for the network.

    For example:
    50% of the network or more:  a 51% attack becomes a real possibility. This can be performed by one entity, or a colluding group.
    25% of the network or more:  cartel-style selfish mining and double spend attacks are both possible.

    ...
    ...
    ...
    ...

    6. Summary
    Available methods of centralisation measurement include:
          * The proportion of the network controlled by the largest or a group of the larger block creating entities;
          * The Gini coefficient and the Theil index;
          * The bitcoin centralisation index.

    * Of these, the last three are likely most useful to measure general inequality in the network hashrate distribution.
    * The Theil index and the bitcoin centralisation index are the easiest to measure; the Gini coefficient is more widely known.
    * None of the measures suggest a long term increase in centralisation over time; the Gini coefficient and Theil index both indicate a decrease in centralisation since the start of the year.

    http://4.bp.blogspot.com/-nu0xcTydNGo/VImICZ_TzXI/AAAAAAAAA3k/kDu0HmcdFvI/s1600/centralisation_plot6_2014-12-11.png



    Title: Re: Neighbourhood Pool Watch: FAQ - Bitcoin mining and the luck statistic
    Post by: organofcorti on July 05, 2015, 01:41:29 AM
    So, if you have questions about luck in bitcoin mining, you might find this post helpful:

    FAQ: Bitcoin mining and the luck statistic (http://organofcorti.blogspot.com/2015/07/faq-bitcoin-mining-and-luck-statistic.html)

    Quote
    8. Summary

    If I have to boil this information down to its important elements:
     # Variance in income reduces as a function of number of blocks solved.
     # Variance in income is not a function of time.
     # Learn how to plan for bad luck, and to check that your pool's luck is not impossibly bad.

    I hope I've made this a bit less tricky for you - post in the comments if I haven't been clear or need to add in more information. I expect this post to be a long term work in progress.


    http://2.bp.blogspot.com/-O9aCgxYNgLs/VZP8xW6MTBI/AAAAAAAADA8/21K7oEX1Vic/s1600/badLuckChart_2015-07-01.png


    Title: Re: Neighbourhood Pool Watch
    Post by: organofcorti on February 15, 2016, 02:34:36 PM
    New post!

    Early detection of unintentional block withholding (http://organofcorti.blogspot.com/2016/02/detecting-unintentional-block.html)

    Quote
    5. Summary

    By examining the numerical values of each block hash returned by a miner, a mining pool operator can detect unintentional block withholding a lower tailed probability of exp(-10) or 0.000045, after only (max difficulty work returned) / (network difficulty) shares have been accepted by the pool.
    Early detection of unintentional block withholding can represent a significant saving. At the current network difficulty, a pool that uses early detection to find a miner with equipment unable to return work greater than 2^32 will lose 0.3 of a block reward instead of ten block rewards.


    https://3.bp.blogspot.com/-bEIVG-j1ZLo/VsHJGlVSqJI/AAAAAAAAFTU/9_LWeZG7ibU/s1600/plot1_2016-02-15.png


    Title: Re: Neighbourhood Pool Watch
    Post by: rav3n_pl on February 15, 2016, 05:20:15 PM
    Are you planning some analysis for block version/Core version?


    Title: Re: Neighbourhood Pool Watch
    Post by: organofcorti on February 15, 2016, 08:28:01 PM
    Are you planning some analysis for block version/Core version?


    The block withholding check is applicable to any bitcoin version, or any altcoin too.


    Title: Re: Neighbourhood Pool Watch
    Post by: kano on February 15, 2016, 11:52:10 PM
    Well - yes that's assuming unintentional is only 32-bit related.

    Of course there could be other reasons, but yep thanks for this OOC - it seems blatantly obvious once you point it out but yeah never thought of it before.

    Fortunately I have full share history of my pool since it started :)
    Time to do 20TB of share searching (sdiff is part of the share log) ... and of course make it permanently record them ...


    Title: Re: Neighbourhood Pool Watch
    Post by: organofcorti on February 16, 2016, 02:03:40 AM
    Well - yes that's assuming unintentional is only 32-bit related.

    Of course there could be other reasons, but yep thanks for this OOC - it seems blatantly obvious once you point it out but yeah never thought of it before.

    Fortunately I have full share history of my pool since it started :)
    Time to do 20TB of share searching (sdiff is part of the share log) ... and of course make it permanently record them ...

    sdiff is the pool difficulty (minimum accepted work difficulty) or the work difficulty? You'll need both of them.

    Let me know what you find? That much data is going to be a very good test of the methodology.



    Title: Re: Neighbourhood Pool Watch
    Post by: kano on February 16, 2016, 01:36:12 PM
    Well - yes that's assuming unintentional is only 32-bit related.

    Of course there could be other reasons, but yep thanks for this OOC - it seems blatantly obvious once you point it out but yeah never thought of it before.

    Fortunately I have full share history of my pool since it started :)
    Time to do 20TB of share searching (sdiff is part of the share log) ... and of course make it permanently record them ...

    sdiff is the pool difficulty (minimum accepted work difficulty) or the work difficulty? You'll need both of them.

    Let me know what you find? That much data is going to be a very good test of the methodology.


    Yes both of course (sdiff is the share's difficulty, not the pool requested difficulty)

    Might be a few days yet :)


    Title: Re: Neighbourhood Pool Watch
    Post by: Cryptonomist on August 20, 2017, 02:32:22 PM
    Hello,

    @organofcorti: I'm currently writing a thesis on the performance of p2pool compared to other centralized mining pools (bitcoin mining). One part of it is the collection of data on the performance of the p2pool network. I've read your blog posts '5.1 p2pool - Bad luck or flawed?' and '5.2 p2pool: Achieving expectation'. I found them very informative and helpfull. However, I wonder whether you can give me some advice on how to collect the data? I'm fairly new to p2pool. I've been running a small-scale bitcoin mining operation for the last couple of months. I've found the logs that produce information about the hash rate, stale rate, etc... for my local node and the network. I just wonder were I can find all the data necessary to construct a dataset like you did, but now for the period 2017-2018.

    Thank you.