Bitcoin Forum

Current post:
Early detection of unintentional block withholding (http://organofcorti.blogspot.com/2016/02/detecting-unintentional-block.html)

Some previous posts:
FAQ: Bitcoin mining and the luck statistic (http://organofcorti.blogspot.com/2015/07/faq-bitcoin-mining-and-luck-statistic.html)

Measures of network hashrate centralisation (http://organofcorti.blogspot.com/2014/12/measures-of-network-hashrate.html)

Neighbourhood Pool Watch 16.9 Another short post on mapping the historical hashrate distribution (http://organofcorti.blogspot.com/2014/08/169-another-short-post-on-mapping.html)

Neighbourhood Pool Watch 16.7 Satoshi's hashrate (http://organofcorti.blogspot.com/2014/08/167-satoshis-hashrate.html)

Neighbourhood Pool Watch 18.1 Who owns 1HTM4TYSXF5yZKLpco6MTUUNfSBCiiwGsU ? (http://organofcorti.blogspot.com/2014/03/181-who-owns-1htm4tysxf5yzklpco6mtuunfs.html)

Neighbourhood Pool Watch 9.18 Gridseed - should you mine bitcoin or litecoin? (http://organofcorti.blogspot.com/2014/03/918-gridseed-should-you-mine-bitcoin-or.html)

Neighbourhood Pool Watch  16.3 The unknown network hashrate. (http://organofcorti.blogspot.com/2014/01/163-unknown-network-hashrate.html)

Neighbourhood Pool Watch  17 Goodbye HHTT. The coinbase will never be as strange again. (http://organofcorti.blogspot.com/2014/01/17-goodbye-hhtt-coinbase-will-never-be.html)

Neighbourhood Pool Watch  16.2 A short post on mapping the historical pool hashrate (http://organofcorti.blogspot.com/2013/11/162-short-post-on-mapping-historical.html)

Neighbourhood Pool Watch 16.1 The network: Orphaned blocks part 1 (http://organofcorti.blogspot.com/2013/10/161-network-orphaned-blocks-part-1.html)

Neighbourhood Pool Watch 12.4 Miner hashrate distributions 28th May 2013 (http://organofcorti.blogspot.com/2013/05/124-miner-hashrate-distributions-28th.html)

Neighbourhood Pool Watch 13.1: BitMinter and "luck". (http://organofcorti.blogspot.com/2013/05/131-bitminter-and-luck.html)

Neighbourhood Pool Watch 12.2 Pool and network miner hashrate distributions. (http://organofcorti.blogspot.com/2013/05/122-pool-and-network-miner-hashrate_3.html)

Neighbourhood Pool Watch 12.1 The pre-ASIC network hashrate distribution. (http://organofcorti.blogspot.com/2013/04/121-pre-asic-network-hashrate.html)

Neighbourhood Pool Watch 8.2 BitcoinPool: The great anomaly hunt. (http://organofcorti.blogspot.com/2013/01/82-bitcoinpool-great-anomaly-hunt.html)

Neighbourhood Pool Watch 9.4 ASIC choices update 24th December (http://organofcorti.blogspot.com/2012/12/94-asic-choices-update-24th-december.html)

Neighbourhood Pool Watch 5.2 p2Pool: Achieving expectation (http://organofcorti.blogspot.com/2012/11/52-p2pool-achieving-expectations.html) Start here (https://bitcointalk.org/index.php?topic=66026.msg1347445#msg1347445)

Neighbourhood Pool Watch 9.3 More on ASIC choices (http://organofcorti.blogspot.com/2012/11/93-more-on-asic-choices.html)

Neighbourhood Pool Watch 9.2 ASIC choices update 2nd November (http://organofcorti.blogspot.com/2012/11/92-asic-choices-update-2nd-november.html)   Starts here (http://[url=https://bitcointalk.org/index.php?topic=66026.msg1312759#msg1312759)

Neighbourhood Pool Watch 9.1 ASIC choices (http://organofcorti.blogspot.com/2012/11/91-asic-choices.html)   Starts here (http://[url=https://bitcointalk.org/index.php?topic=66026.msg1310725#msg1310725)

Neighbourhood Pool Watch 8.1 BitcoinPool: Another Bitclockers? (http://organofcorti.blogspot.com/2012/10/81-bitcoinpool-another-bitclockers.html)   Starts here (https://bitcointalk.org/index.php?topic=66026.msg1306759#msg1306759)

Neighbourhood Pool Watch 7.1 Variable pool difficulty (http://organofcorti.blogspot.com/2012/10/71-variable-pool-difficulty.html)  Starts here (https://bitcointalk.org/index.php?topic=66026.msg1262071#msg1262071)

Neighbourhood Pool Watch 4.3 Slush's score method and miner earnings part 2 (http://organofcorti.blogspot.com/2012/09/43-slushs-score-method-and-miner.html)   Starts here (https://bitcointalk.org/index.php?topic=66026.msg1164772#msg1164772)

Neighbourhood Pool Watch 6.1 Bitlc.net - Pool hopping and unfulfilled promise (http://organofcorti.blogspot.com/2012/08/61-bitlcnet-pool-hopping-and.html)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg1136938#msg1136938)

Neighbourhood Pool Watch 4.2 Slush's score method and miner earnings part 1 (http://organofcorti.blogspot.com/2012/08/42-slushs-score-method-and-miner.html)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg1093764#msg1093764)

Neighbourhood Pool Watch 5.1 p2Pool - bad luck or flawed? (http://organofcorti.blogspot.com/2012/06/51-p2pool-bad-luck-or-flawed.html)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg974729#msg974729)

Neighbourhood Pool Watch 4.1 Slush's pool (http://organofcorti.blogspot.com/2012/04/41-slushs-pool.html)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg853421#msg853421)

Neighbourhood Pool Watch 3.2 The risks of SMPPS update (http://organofcorti.blogspot.com/2012/04/32-risks-of-smpps.html)    Update (https://bitcointalk.org/index.php?topic=66026.msg831928#msg853421)

Neighbourhood Pool Watch 3.1 The Rise and Fall of Arsbitcoin.com (http://organofcorti.blogspot.com/2012/03/31-rise-and-fall-of-arsbitcoincom.html)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg823955#msg823955)

Neighbourhood Pool Watch 2.2 Deepbit and 'pool hopping' (http://organofcorti.blogspot.com/2012/03/22-deepbit-and-pool-hopping.html)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg802847#msg802847)

Neighbourhood Pool Watch 2.1 Deepbit (http://organofcorti.blogspot.com/2012/03/21-deepbit.html)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg795366#msg795366)

Neighbourhood Pool Watch 1.5: Bitclockers.com summary (http://hoppersden.info/entries/32-Neighbourhood-Pool-Watch-1.5-Bitclockers.com-summary)    Starts here (https://bitcointalk.org/index.php?topic=66026.msg779543#msg779543)

Neighbourhood Pool Watch 1: No-one should mine at Bitclockers (http://hoppersden.info/entries/31-Neighbourhood-Pool-Watch-1-No-one-should-mine-at-Bitclockers)


###################################################################
Neighbourhood Pool Watch 1: No-one should mine at Bitclockers (http://hoppersden.info/entries/31-Neighbourhood-Pool-Watch-1-No-one-should-mine-at-Bitclockers) is the first in a new installment of blogs from me that have nothing to do with strategic mining - er pool hopping, and everything to do with ensuring miners are safe from dishonest pool operators and cowboys.

The first to be covered is Bitclockers.com

I gave Bitclockers.com an opportunity to explain themselves on the bitclockers.com thread (https://bitcointalk.org/index.php?topic=10127.msg760406#msg760406) with no answer to my concerns. So I've decided to post what I know about how they are underpaying their fulltime miners.

From the post:

and

Well done Organ.   They've dug themselves into a hole here.

Way to Bust 'em...miners "you need to hide your kids,hide your wife and hide your husband cuz they rapen anybody out here"

http://www.youtube.com/watch?v=EzNhaLUT520 (http://www.youtube.com/watch?v=EzNhaLUT520)

Damned straight.

Yeah I remember that one block where the p2pool operator faked the stats to steal from us miners, it pissed us off so much that we ...

er wait that never happened, it couldn't happen.  :)

Bitclockers does its best to protect its users, to protect the pool from hoppers bitclockers uses anti hopper protections to mitigate the hoppers bonuses over other miners in the pool

With the Hopping bonus mitigated Hoppers are paid 100% for the shares they mined. NON-Hoppers are not effected and payouts remain 100% of shares mined.

I'm not interested in your hopper protection - focused explanations. All I ever wanted to know from the start is why (until today edit: and since round 179) you never had a block under 0.3*D. Did you keep some for your self? No. Then if you've been respecting block boundaries you've been underpaying.

I proved without a doubt that you've been misreporting block lengths, which means  you've probably been stealing from your fulltime miners.

How about you publish the actual block lengths from rounds 179 to 504? If you can't do that you should get the scammer label.

then why my average daily income is 1.6-1.8 where i can get around 2.2 to 2.3 on bitlc, when i tried full time on your pool, without hopping?

don't tell me "it's the variance"

Its the variance.

for couple of days straight?

Deepbit once went like a week without finding a block.

When did this happen? Back in the 80s ?

LOLZ

http://www.youtube.com/watch?v=JFWuuP-cyy4

;)

Thanks for the heads up, I had a sneaking suspicion something has been up for the past while. Bitclockers- take some action to rectify this swiftly and immediately. One miners lose faith an trust in your operations it will forever be an uphill battle to regain this trust. I am not 100% sure there is foul play here, but please address the issue for the community.

time travel :o

How much evidence is required?

Overwhelming evidence.

Sorry, guys, but I'm just not seeing it.

Probably my fault for not making it easier to follow - sorry.

1. We have overwhelming evidence that they haven't been reporting the shares in a round accurately. Without any explanation from them, this is bad. They could be hiding anything with this, including hiding short rounds.

2. They have stated that they are accurately reporting rounds as they occur. I provide an explanation here (http://hoppersden.info/entries/31-Neighbourhood-Pool-Watch-1-No-one-should-mine-at-Bitclockers) as to why this means (if they're being truthful about block reporting) they've probably been underpaying their fulltime miners by 21%.

3. Their first round under 0.3xD in over 300 rounds occurred just after I made their obfuscations public. That is unlikely to be a coincidence.

4. I've asked them several times to publish their accurate round lengths which they haven't done. In fact I haven't had any response to any allegation I've made. I have sound mathematical backing behind all of my statements. I'm overstating nothing, if anything I'm being conservative.

I'm aware that I'm probably not covering something that you need to know, but if you can post or pm me your requirements I'll know whether or not I can fulfill them without wasting any more time for either of us.

I don't mine with bitclockers and I don't have a horse in this race.

Do you have timestamped shares and payments? Also did you happen to keep your own log of shares submitted (I'm guessing not but thought I'd ask).

Thanks for the btc CornedBeefHash - I promise I'll give it a good home.

I agree that there needs to be some sort of pool oversight, but not in the ad-hoc way I've done it. It needs to be some sort of a council made up of pool ops and miners. I discussed this a bit with MobiusL (his idea) and I like it.

Also, don't forget we put Maged in a hard position here. In other cases of scamming it's usually clear cut and the evidence doesn't require a knowledge of statistical probability in order to follow it.

Assessing whether or not a pool has been acting in poor faith is much harder. Maged must also realise that this is a test case and probably doesn't want a situation where at any time in the future a miner gets pissed at a pool he can refer back to this and cause unnecessary problems.

I'm happy to go along with the process since I'm sure I'll be able to supply whatever proofs I need to.

Cheers again for the support - always good to hear.

Thanks for the support Chaang - a few more btc and I'll make it to Tongsai Bay ;)

More data? You know me so well.

+1
sent 2btc


I had also suggesting something like to some others a while back "this I agree that there needs to be some sort of pool oversight, but not in the ad-hoc way I've done it. It needs to be some sort of a council made up of pool ops and miners. I discussed this a bit with MobiusL (his idea) and I like it." It is a good idea and I would be fully supportive - just *how* to audit pools might cause discussion ;)

I started a thread so poolops could state what they do with miners shares, it was pretty fail as most didnt post then it was taken offtopic by people that didnt understand what it was for.... I thought it could be a handy quick reference :/

Keep up the good work organofcorti :)

Cheers
Graeme

Thanks for the support, Graeme. BTC helps me justify my 'hobby' to my SOP when I miss dinner with her parents ;)
Published round lengths can be a good start. I have no idea how you'd then audit payments.


I was kind of hoping your thread (https://bitcointalk.org/index.php?topic=64400.0) would get more substance to it, but maybe some pool ops aren't just using hashes the way we think they are and aren't ready to talk about it yet - apart from Goat that is.

Just move to p2p, problem solved  :P

unhoppable. :P

I agree. It is quite clear that the reporting isn't accurate.

They are likely lying. However, lying doesn't get people a scammer tag. It is only a problem when they lie to make extra money. However, I'm not seeing any evidence of that. In your link, you only explore case 1. What if, however, the reality was case 2? Also, since their statistics are delayed, what's stopping it from being a hybrid of truth and fiction? If a round is too short, they could pretend that it never happened. If it's too long, they can say it ended early. Done properly, this would equal out over time.

I had thought that accepting they were lying about accurately reporting blocks solved was more inflammatory than accepting they were telling the truth, so I covered what they said was correct. Hoist by their own petard, etc.

If they are lying about reporting block lengths they may be using a simple sort of averaging algorithm as you suggest, but as I discuss here (http://hoppersden.info/entries/29-How-to-hop-11-Bitclockers.com-amp-spurious-round-lengths-part-2) simple averaging algorithms don't match the histograms of Bitclockers.com round length data.

A much more complex algorithm that matches the Bitclockers.com round lengths frequency distributions are possible, but a much simpler and equivalent approach is to use a function on all rounds, not just shorter and longer ones.

I identified several transforms that match the empirical cdf and histograms of the bitclockers.com round length data and discussed them briefly in Neighbourhood Pool Watch1 (http://hoppersden.info/entries/31-Neighbourhood-Pool-Watch-1-No-one-should-mine-at-Bitclockers).

So, assuming they are lying to their miners about when (and possibly if) blocks are solved, we have at least two options to investigate:

1. Bitclockers.com are lying to their miners about when blocks are solved but not if blocks are solved.
In this case, they are faithfully recording all shares submitted and changing block lengths to suit themselves. In this case it is simple to show empirically that strategic miners are being underpaid. If this seems to you to be ok, keep in mind that they claim to be a proportional pool. Strategic miners at a proportional pool can expect a certain return on their shares. If they don't receive the expected amount over time, then the pool is not paying out proportionally.

Their "About us" page doesn't mention anything about "hopping protection", just that the pool is a proportional pool. From their website:

In these quotes they are certainly lying about how they remunerate any miner.

Neither did their threads here, or on their forum (as far as I can tell) mention this until I posted How to hop 10 (http://hoppersden.info/entries/28-How-to-hop-10-Bitclockers.com-amp-spurious-round-lengths-part-1). If strategic miners have been underpaid, then they have been lying to make extra money. Whether or not they redistribute that extra money to other miners is a moot point - if we can prove that some miners are targeted for underpayment with no obvious warning, then we can show that they lied for extra money. In the words of backburn (a bitclockers.com pool op):


How can we show that they are underpaying strategic miners without relying on a particular transform or the Bitclockers.com pool op's say so? Well, can use their own data to show the difference between expected share values on Bitclockers.com and a real proportional pool. (http://hoppersden.info/entries/29-How-to-hop-11-Bitclockers.com-amp-spurious-round-lengths-part-2) But since I already did that how about looking at the difference between the expected payout per round for Bitclockers.com compared to a real proportional pool?

The graphs below show how the expected share values and expected earnings vary for bitclockers.com and an actual proportional pool. This data is based on the round lengths published for rounds 180 to 505. The top graph I've published before. The lower graph shows the expected amount a miner can earn by leaving a pool at a particular number of shares. As can be clearly seen, if a strategic miner leaves Bitclockers.com at 0.43xD (the point at which the expected share value drops to zero on a proportional pool) the strategic miner will earn about 0.6 less than expected. So for every 111 btc a strategic miner earns, he or she should have earned 170 btc (not including real world inefficiencies).

https://i.imgur.com/KxC5b.png



2. Bitclockers.com are lying to their miners about if and when blocks are solved.
Part 1. partly covers this. In addition not paying strategic miners their expected reward, Bitclockers.com may be withholding btc from all miners, for example from the very short rounds that they don't report. The possibility can be assessed fairly simply by comparing the distribution of means for bitclockers with other generated random data from distributions that have a similar density plot and cdf to the Bitclockers.com round lengths. See Neighbourhood Pool Watch1 (http://hoppersden.info/entries/31-Neighbourhood-Pool-Watch-1-No-one-should-mine-at-Bitclockers) part 1 for details.

Firstly, what does this bitclockers.com data say? The Bitclockers.com mean round length for rounds 180 to 505 is 1.0281. It took 325*1.0281=334.1 rounds of shares to generate 325 bitcoin rewards. That means that they could have taken 455 btc and we'd be none the wiser.

How likely is this to have happened?

Using the three most likely distributions and this R code (http://pastebin.com/qnVTX2Cr) I generated a million sets of a million generated random numbers for each of the three distributions. I then generated a million means from each set, and found their means of the means and the proportion of them below or equal to D.

From these sets I obtained the following:

The means are very close to that for bitclockers.com, and the second group of results show that for round length distributions similar to Bitclockers.com's own, only 0.19 are at D (the bitcoin Difficulty). This means that the for these sets, the mean is much less likely to be D than it is to be larger than D. The very method of generating their fake data - whatever it is - allows a mean to be greater than D 71% of the time. This could easily include hiding btc. Rounds 180 to 505 are Sept 22nd to Feb 17th. So that's an extra bonus to the pool ops of about 90 btc/month in addition to the 2% tax. A nice little earner.

In conclusion, I can only prove malicious lying to gain coins in the first case.

But the second case is also important, and on the balance of probabilities could easily be true. Any time a pool does not disclose round statistics, it is too easy to 'fudge' the books and keep the a short round for themselves.

I hope this clears up any doubt for you.

good job.

well, that means them fuckers have some 'splainin and payment to all of whoever mines there to do.

pitchforks, anyone?

organofcorti, your problem is that you're assuming that the actual expected round length is supposed to be 1.0281 D. Instead, assume that it's 1.0 D. What is the probability that the measured round lengths would average to 1.0281 D by chance?

Also, I'd like to see some graphs that compare them to the other types of payout methods. The fact the they lied about being proportional means nothing to me. Any issue regarding that should be handled by a local board mod, such as DiabloD3. I'm just doing a scammer investigation, and so far, all I'm seeing is you bending statistics to give the results you're looking for - a common issue in the field of statistics.

All the evidence proves that HOPPERS are earning less than they expect?

Are the "regular" miners earning less?? If so why would they stay there?

I'm not saying that It's OK to play around with the statistics even if the cause
is OK( In their eyes). Rules are rules.  But I guess the non-hopping miners
don't care as long as they get the share of the pie they expect (as if there
were no hoppers)

You'd like to assume the population mean to be 1xD and estimate the probability of the round lengths averaging to 1.0281? Using a similar technique to that described earlier (using code similar to this) (http://pastebin.com/qnVTX2Cr) we get a probability of 0.8801 rounds being less than 1.0281, giving a probability of rounds being 1.0281 or longer at 1-0.8801=0.1199. This is even worse than the estimate I provided earlier.
I'm happy to do that. What sort of graphs did you mean? It should, since they agreed to pay a certain amount to miners, and then did not pay them that amount.If you'd like to provide examples I hope I'll be able to explain why I wasn't bending statistics and regain your trust.

This is a seriously good idea CBH. I think it would be extra hard works for the mods though. A council made up of respected bitcointalk forum members and poolops might be better doing the ground work, with  DiabloD3, gmaxwell, and MiningBuddy having the last say before anything is posted. I can see lots of disagreements arising. You'd need to have a cumulative rating the way they do in olympic gymnastics from various judges, i think.

What it comes down to is that they agreed to pay miners a certain amount and didn't.

My statistics are a little rusty, but isn't that result not statistically significant?

i don't think we're testing a hypothesis here. it's just a probability.

...unless organofcorti wants to do a series of real-time test with a similar sized prop pool.

It's not a p-value, but even so all it means is that you would expect round lengths means of this size to occur once in every nine sets of 325 rounds. You have to decide whether or not that's significant. As a final remark on this I wrote:

(the second case being what we just discussed)
So I wasn't claiming to prove Bitclockers.com were steaing blocks, just that it was more possible than for pools that post accurate records of round lengths. ( various edits because of my stupidity and feeling under the weather)

yes, it's just based on the cdf.
Can't do that either. If we assume the population mean is 1.0*D, we have to assume the distribution is similar to the Bitclockers.com distribution. Can't compare a geometrically distributed round with a bitclockers.com round - they aren't similar.

gah.
didn't think of that.

which pretty much means, there's no way to test it, with only based on data gathered from bitclockers.

That's right. You can just make assumptions and do backtesting.

I'd like to make this plain: now that I have a bit more of an idea about what the 'scammer' tag means, the second case can't make any claims about scamming. It just makes clear that it is much more possible for Bitclockers.com to steal blocks, and that results so far do nothing to disprove the possibility.

The first case (that they have agreed to pay miners a certain amount and did not pay that amount) is really the only situation where lying to wrongfully obtain funds can be proven.

so burden of proof now is on bitclockers. unless they prove they don't do anything shady, which i doubt, they have some explaining to do.

so far, they haven't really deny anything. only vague statements.

I think the burden of proof should be on every pool. It's just that what Bitclockers.com does is so obvious.
I know and it's frustrating. I was hoping to get them to open up and explain what they did and why. So far they haven't even acknowledged faking round length stats.

send them an email.



....ohwait, they're not gonna reply, lolol.

So the answer to my question is yes.... hoppers only are hurt apparently

what?

no.
both full-time miners and hoppers, to a degree.

Not quite. According to the promises of the Bitclockers payment method, the case for underpaying all miners on short rounds is the only one that can be proven. Would you be happy if a PPLNS pool you mined at decided to arbitrarily not pay some of your earnings?

But everything else i've written stands as is. I might not be able to prove Bitclockers are stealing blocks, just that it can't be disproven (from the data we have), and in terms of probability appears more likely than not. It would be easy for Bitclockers.com to disprove this, but they don't seem to want to. I hope it's not because they are unable to.

ok,  I'll revise my statement..  hoppers are hurt more than others ( because they usually take a bigger amount of the pie)

 

So, this is probably a good time to start this:

Pools I have confidence in and would mine at (in no particular order)

• Slush's pool (once they're on whatever hop proof payment method Slush decides on)
• Ozcoin
• Eclipse

Why Slush? I 've investigated them more than any other pool. I'd have noticed anything odd.
and generally, all three pools provide stats and are totally open. I also have a significant amount of trust for the pool ops of Ozcoin and Eclipse.

This doesn't mean I don't trust other pools. I wouldn't mine at deepbit for example because I don't want it larger than it is and because I don't need PPS because variance doesn't scare me. BTCGuild is also a fee-pay PPS. Bitlc is out for reasons that date back to multipool (ok so I can hold a grudge). Eligius and Ars are out because they're SMPPS.

I think that covers the main pools.

Thoughts?

There is no pie. There is only bitcoin. There could be bitcoin pie though, I suppose. It would taste like victory!

To recap:

Everyone loses on short rounds, but strategic miners provably lose more than fulltime miners since strategic miners only stay for short rounds.

I don't know how much or even if fulltime miners' expected payouts are affected. It is possible or even probable that bitclockers have been withholding blocks rewards, but not provable because Bitclockers withhold real stats and produce fake stats.

We have enough of that in the lending section already.....

edit:

It seems to me that only the hoppers are mad because its bad for their system. I can understand that but
I guess the majority of the users have no problem with the "hidden" algorithm they use.

Just to be fair I admit that i have a "thing" for hopping and not a good one. When I first started mining I didn't have a clue
and was even happy when the hoppers  came to "help " on the small pools I used.   When I learned the truth it pissed me off.
Don't want to start another discussion on this since there have been so many already.

Who made you the spokesman, they owe all their miners 21% more than they paid. When they pay that out we can re-evaluate their position. Until then they deserve the "scammer tag" and all should avoid them.

I'm not anybody's spokesman and never even mined there.  All I'm saying is if they were paying 21% less to everybody then nobody would stay there.  The pools owner admits "not condoning hopping" etc etc.. so they took the law ( or the rules) and did whatever they did. I didn't say even once that it's right - just that it does protect somewhat  the full-time miners from the hopping. If the Pool owners ever come clean with the info we shall see.

It's been empirically shown that they have been lying and faking stats, the current analysis implicates them as having shorted ALL their miners (full time) at least 21%. This thread is not about hopping, it's about their lying and stealing from their miners and using hopping "prevention" as an excuse.

If you never mined there, why get involved in the conversation unless you have an alternative motive and/or are trying to promote your own individual agenda.

People continued to mine there because they started as legitimate pool, then they degraded to cheating their miners. Even now they are trying to rehabilitate their stats and information because they were CAUGHT. Their lack of significant response in IRC or in the forums indicates (in my opinion) their hope it will "blow over" and they can continue.

This is the problem with having "pools" and "exchanges" being unverified. People who mined at bitclockers will probably never get the funds that they were cheated/shortchanged. Yet there is no process to hold them accountable.

The principles of Bitclockers should go to jail for fraud, that will not happen at this point in time,  yet until the bitcoin community starts to enforce penalties against this cheating, or set standards, we will be looked upon as an amateur.

If we look at how exchanges now want us (miners) to ID ourselves, why should we not expect the same of exchanges and pools. Who are they? Should they be registered as a business, should they post a bond against fraud. Should there be some oversight council. We need to police ourselves before it is imposed on us by those who have no clue.

I have no agendas and again , never said its OK what they are doing.

I did say I have a "thing" about hopping and in this case it seems the
hoppers are pissed,  at least more than other miners.

So, maybe I commented on the subject  because of that...

Whether you have a thing against hoppers or not, does not justify your interference, take that up in another thread,, I had bitcloakers as my backup, I was screwed by 21%. So if you have an issue against hoppers, do it in your own thread. This is about Bitcloaker, STEALING and LYING. Unless you have something productive (actual statistical evidence) go post in another thread and stop obscuring the purpose of this thread for your own ego.

Interesting, about 6 months ago I mentioned somewhere about miners trusting anonomous nicks with no other info :) - and was howled down in the community. oh well :)

Those of us that are here for the longterm I think are taking steps towards this, I know of at least one other pool that is a registered buisness...

Post a bond against fraud: hmm interesting - will you insure pools and exchanges against hacking (i know entirely different subject)... but lets be a bit real :)

There should be a group of  people overseeing this yes, it has been mentioned before in this thread and others.

Actually if people took more responsibility for themselves and did research on pools or exchanges before they use them would be less need for policing.

my 2s
Graeme Tee
Graet
Ozcoin Pooled Mining Pty Ltd
ACN: 152 509 272
^^^ (this info has been in the 1st post of our pool thread for about 7months)

Garet,
You are one of the few pools that I actually have full respect for, you have been completely open with your pool and it's progression.  The bond would be against the exchange/pool cheating people against their monetary investment. Hacking of a pool can be be covered by insurance against hacking(cost of business). Unfortunately, if we don't police our own neck of the woods,legislation will.

I agree - there is far too much bad going on in the forum and mining.   My comments weren't personal against you or any hopper -  was just - in my eyes- a bit of poetic justice...

BC went down, now that it is up, organ think anything has changed?

So, you think they slowed the initial hash rate and share count to have hoppers stay longer and/or delaying block starts?  So, would that buffer be spread out over the "block" (fake) or just kept by the op?  I can see you can hop it but it's like a even worse coinotron...get killed on the short rounds...not a good mix with other prop pools...

Also, did BC just have a short round? 

1.111~ sent,

Unfortunate that shady hopping countermeasure are being used to fake stats/payout,  when open and effective solution already exist.

Ps: statistically speaking,,, Some pool that charge 10% for PPS and 3% for prop..  They make it sound as if running a prop pool with 4Ths was a risky business to the operator. , it's not.  ... Just saying... see sig.

huh.

weird.

my daily payment is close to double what i normally get now.  ???
must be the short blocks.



....wait, they have two short blocks?

I know, hope they're back to normal

Yeah they got busted stealing miner's coins so they had to stop taking short blocks as profit. 

Id certainly add bitminter to that list, and I have one concern with Slush; its not major, and I have no reason to distrust Slush, but I sure would like to see stats of shares submitted there. Must be the only pool that doesnt show it (expect for running round).

As for bitclockers, best I can tell their stats are still as fake as ever.
Maybe they thought faking a few short rounds would somehow be enough to disprove the mountain of evidence against them.

now they better give me back my coin they stole.
...and everybody elses.


i can safely assume we caught them red-handed.

it's interesting when your payout suddenly doubles after people call'em out.

It was great to watch two forums members that I respect have at each other in such a reasoned manner. Discussions of that sort don't take place on bitcointalk.org often enough. However I realised with some information over in hoppersden.info and some here farfiman and Mobius each only had part of the information available.

With that in mind I've posted Neighbourhood Pool Watch 1.5: Bitclockers.com summary (http://hoppersden.info/entries/32-Neighbourhood-Pool-Watch-1.5-Bitclockers.com-summary). In it I reiterate what I've shown in both NHPW1 and here a couple of pages back (edit: make that 3 pages back) in one hopefully cogent post (I write 'hopefully' because I rushed it out before knives were drawn here, plus I had to cut bits out to get under the 10000 character limit.)

For those who can't be bothered reading the whole post, here's the tl;dr:

I hope this clears things up for everyone.

Thanks, Transisto, I do appreciate that. Full disclosure: the donations I've gotten all of which I'm grateful for (special mention to Goat - thanks mate) has been spent on making me a "Donator" to this forum. Some of the next donations I get - if any - will go to hoppersden.info admin 'myself'.


And irritating. Instead of moving to a system that is unhoppable, full time miners lose money (less than usual but still some) to strategic miners, all so they can keep the hashrate boost that strategic miners provide. And this is ignoring all the other ways they can and probably have ripped off miners so far.



Well, I'm going to disagree with you there. A pool needs at least 7% to make PPS risk manageable. So really, they'd be taking 3% on top of the risk tax (which would be put aside for long rounds) plus 3% on prop. That's only 1% above what other pools take so I don't think it's too greedy. Man's gotta eat.

It's a little odd, sure, but not impossible. They might have altered their algorith/transform function a little, but even using the transforms I previously identified (which might not be being used but do provide a guide), the equivalent geometric round shares at that D would be:

Round lengths as short as 7 shares or 700 shares do occur. Round lengths under one share do not. So, calculating the cdf at one share for the difficulty at that time, and using quantile functions to convert to a Bitclockers.com round length as a proportion of D, we get:

So if we'd seen rounds between 0.04xD and 0.14xD, we could have been be pretty certain that they'd have changed their transform/algo. At the moment, I don't think we can say that. Looks possible though.

  

Good oh. I've put a poll on which might be a better way of indicating what we think (although online polls are subject to gaming, I suppose). Care to vote?

I guess they wont be getting many votes then  ;D

p2Pool isn't a pool either but it seems to be leading the pack.

I consider p2pool a pool.  It just happens to be a decentralized one.

Perhaps you should have allowed multiple votes and rephrase the question to "what pools do you trust".  Being honest, if I understood what Ive read so far, is that p2pool is the most trustworthy bar none.
But I still voted for bitminter :).

Good point. Unfortunately once a poll is started, multiple votes is one of the things that can't be changed. No doubt there'll be a next time. Right after a "Which pool do you trust the least?" poll.

You totally can change it. Just allow people to make multiple votes and allow them to edit their votes.

Not seeing it, sorry.

https://i.imgur.com/SeY2v.png

Is there another option somewhere?

Oh, interesting. Mods apparently have more powers than normal people...

Poll updated.

Thanks Maged.

Anyone who wants to update their vote or add votes, you'll have to remove your old vote first.

why can you vote for scammers? thats stupid! (bitclockers)

To gauge the success of my posts, obviously.

Sorry, reading your thread I didn't think 'Proj#2' was up any more. Is it? If it is I'll add it.

You can change your vote.

Should I add it, Goat?

Sorry guys, I was wrong in both cases. I forgot that using the CLT a) it doesn't matter what distributions the means are from and b you can assume normality so you could consider it a non-significant p-value. It still stands as a probability though, and 0.1316 is still low.

"trusting" p2pool is misleading. P2pool do not require trust, because it CANNOT scam you.

Neighbourhood Pool Watch 2.1 Deepbit (http://organofcorti.blogspot.com.au/2012/03/21-deepbit.html) is out (and yes I've moved the 'Neighbourhood Pool Watch' blog to Blogger).

From the post:


You can comment at the blog if you're logged in to gmail, or here if you don't have a gmail account (whaaa....?)

Edit: on second thoughts, I've left the commenting open to anyone - you won't have to log in.

add pool.itzod.ru

If you mean "please add pool.itzod.ru to the poll options", well maybe next time. I'm closing the poll soon, and it wouldn't be fair to the pool. I'll do another poll at some point.

Then again :)
Did you realise that by default you give 0.5% of your earnings to forrestv?
You must change that if you wish it to be a different value :)

Auto enabling a fee and calling it a voluntary donation isnt the same thing.

Voluntary suggests you informed a user and they then took steps to provide a donation, I wont be surprised if a bunch of p2pool users didnt even know about this 0.5% fee.

2.1 was more good reading! Thanks, very well done. ;)

You're welcome, Marty. Next up looks at how strategic miners affect the payouts of fulltime miners on the prop payout side.

Hmm I've found 3 blocks on DeepBit prop and been paid about 30BTC ... ... :) 500% luck
(Yes that statement is true by the way)

I solved one block ever and mined 110 btc. Thats 220% over what I would have gotten solo. You would have gotten 150btc mining solo, so I'd call that bad luck. Still, it averages out for us all.

My comment was actually in reply to your one I quoted in case you didn't realise :)

Neighbourhood Pool Watch 2.2 (http://organofcorti.blogspot.com.au/2012/03/22-deepbit-and-pool-hopping.html) is out now. This post shows how much strategic miners add to Deepbit.net's overall hashrate, and how much full time proportional payout miners lose in the process.

The post was inspired by P4man's initial work on the subject here (https://bitcointalk.org/index.php?topic=62975.0).


https://i.imgur.com/fATkV.png

You can be a cryptic bugger, Kano ;)

Interesting read Organ.
I do wonder if perhaps you are not misinterpreting the data; when a round starts, you get fresh work. Even assuming zero delay from the pool, and zero network latency, only the fastest miners are going to be able to submit shares within seconds. Slower miners may take up to a minute or more before they submit their first share.

I wonder if that by itself couldnt cause both the drop at the beginning of the round, and the slow spike afterwards. After all, deepbit rounds are very short, and I wouldnt be surprised there a lot of slow and clueless miners on there.

As for the poll:

Bitclockers.com    - 2 (0.6%)

Will the one person besides the pool admin who voted for bitclockers please stand up?

No, I can see how you'd think that, but a group of poisson distributed variables with different means is still a poisson distributed variable with a new mean of arrivals per unit time that is the sum of the of the constituent mean arrivals per unit time.

Think of it this way. 10 miners with 1 Ghps will submit about 2.3 shares per second on average. So will one miner with 10Ghps.

The only thing I can put the slow hashrate increase from below normal is variable time to receive a long poll notice. Until such notice is received, any shares that are sent will be stale. Then there's the time taken for the trip back, which may not be the same as the trip there, due to variable servers on the way.

A log-normal distribution is a good fit for a combination of normally distributed variables that have a multiplicative effect on each other, so I chose it as a starting point for the model. It turned out to model that response pretty well. It's only the mean of ~ 0.7 seconds and deviation of ~1.7 seconds that I can't interpret.

Poll voting is locked. Results published shortly.

Here are the results for the "Which pools do you trust?" poll.

https://i.imgur.com/qciVA.png

Below is a bubble plot showing votes on the y axis and bubble size showing hashrate.

https://i.imgur.com/CO5VC.png

It seems that small pools vary significantly from the minimum to the maximum of the vote, and the larger pool are in the mid range. To try to account for hashrate, I divided all votes by the log of the hashrate to get a 'trust index'. I used log(hashrate) to try to account for the fact that novice miners that might not know much about trust tend to gravitate to larger pools first.

https://i.imgur.com/f1uRH.png

p2Pool is the clear winner, followed by EMC and Ozcoin. Well done to all in the top three pools - you've gained significantly more trust in the community than others.

Proviso: although data is accurate, it probably is not very reliable. It's self selecting: mostly english speakers and only those concerned about trust in pools. Further, sample size is small and voters could vote more than once which skew results further.

New poll started. To complement the last one, this time it's: which pool do you trust the least?


...and done.

Wasn't me. I forgot to vote after I posted the poll. I think it's pretty obvious which at least one of the pools I voted for was. Not Slush.

Currently prop with delayed stats. I think they're changing to pplns soon, or may have already. If you're asking about how to hop them .... well, there's another thread for that.

Hmm - simplecoin.us still mines also (their last block was ... 171650)
I only know that coz they have their name in the coinbase :)

(at the moment there's 10 pools I can tell from looking at the block in the last 100 blocks - accounting for 68% of the 100)

Actually I think they vote against Eligius for taking down the Springcoin.

Actually - both of those pools are in direct contrast of trust.
P2Pool people say that you can't trust normal pools - so at least someone somewhere will probably say they don't trust each normal pool.
P2Pool, on the other hard, takes a 0.5% fee that most people don't realise until someone points it out to them and they can change it to zero - so there would certainly be a lack of trust by some due to that also.

It's an option listed in the middle of the docs with all the other options.
Read if you are interested :)
That gives you a chance to decide if you think it is visible enough or not :D

Edit: hmm it's now 4 for P2Pool - maybe 4 people didn't realise it - or there are 4 other reasons?

It's a very small fee by default (0.5%), and you can always set it to 0 with "--give-author 0"

Middle of the docs? Who's going to read docs to join a mine? If I was joining a mine, I would expect the "fee" to be listed in the first post of the pool thread:

https://bitcointalk.org/index.php?topic=18313.msg231846#msg231846

Hmm. Nothing about it there. How about in the guide to setup p2pool?

https://bitcointalk.org/index.php?topic=18313.msg712967#msg712967

Nah, still nothing.

Oh wait, I remember seeing a link to a wiki. It must be there:

https://en.bitcoin.it/wiki/P2Pool

I scan the page down till I get to the heading beginning with Donating... thinking I hit the jackpot, but no. Surely something as important as a built-in fee in a pool that claims to be fee-free should be mentioned in a paragraph that talks about the reasoning behind it? Alas, I finally spot it hidden in the command line options, where very few people will ever notice, especially if they just follow the guides.

You guys are exploiting people much the same way that companies will exploit customers who buy products with rebates. They expect that a certain percentage won't know about the rebate, will forget about the rebate, or won't fill it out correctly. You're also taking advantage of people's inclination to do nothing when faced with a tough decision and hence take the default option. While it isn't truly hidden and has been talked about several times on here (with each time p2pool users surfacing saying that they never heard of it), anything less than front and center IS hidden when it comes to a fee, ESPECIALLY when one of your primary raison d'etre is no fees! You can claim that people should know about it and how to stop it. That the donation is worthy. That it is good intentioned. None of that matters.

There are only two options if you want to have an honest, reputable pool (which we all want). Advertise the built-in donation loudly and boldly, or make it opt-in rather than opt-out.

Kano, can you tell me why you think Ozcoin is "in direct contrast of trust"? It's one of the few pools I have trust in, so I'd like to hear an opposing view point.

I think he just means the literal definition of a pool, as such, and therefore all pools apply. If I am wrong, I would also like to hear an answer to this. Graeme runs an excellent pool.

I just want to let this sink in for a bit. You are one of the big proponents of p2pool. You would have everyone use it rather than other pools. And now you think that everyone should be auditing the code of p2pool before using it? So everyone who mines bitcoin (and any other cryptocoin that uses p2pool) should 1) have the programming ability to read and analyze the code and 2) the time to do so? Just think about that for a bit and tell me if you think that is a reasonable request.

If you really think like that and are absolutely certain, then you should remove your p2pool guide. If someone is reading the code and studying the wiki, they won't need the guide. Guides are for people who don't do those things. In fact, your guide is enabling people to use p2pool that apparently shouldn't be. Don't be an enabler.

I am the only one who feels like throwing up when reading this?

On one hand, we have ForrestV, who actually built something himself, from the ground up. Something thats incredibly innovative, (very) valuable to bitcoin and the community at large. Something that is open and opensource.

No secrets, no lies.

OTOH, we have a clueless Goat who has never done anything himself, but leaches off everyone else and thrives through secrecy and deceit. A goat that runs a mining business owned by his shareholders who are not being told where his farm is mining, who are not being told how much is truly being earned from pool hopping, who are not being told anything but who are paid dividends based only on fabricated numbers that dont come close to reflecting actual pool hopping profits.

That same Goat now cries foul because someone ignorant may not read the readme and therefore unwittingly donate  a small amount to someone deserving, yet he runs a hopping proxy, developed by people with the skills he lacks, a proxy that robs the poor and ignorant miners of prop pools from far more than 0.5%, yet he makes zero effort to educate those miners or protect them from their far larger losses because he pockets them, and not Forrestv.

So that hypocrite is now going to pretend being the miners advocate, judging what an honest (and optional) donation fee is for ForrestV's hard work, most (or all?) of which is redistributed among the miners?

I was simply referring to the issue that MANY P2Pool acolytes say that normal pools are untrustworthy thus you should use P2Pool
So in that respect, P2Pool is in opposition to Ozcoin when it comes to trust.

I trust Ozcoin and Graeme probably more than any other pool around.
(I'm even mining on Ozcoin at the moment and have been 100% for the past 6 days)

Aside: The early on P2Pool evangelism turned me off P2Pool, so for myself it's unlikely I'll use P2Pool in the near future (and I have brought up reasons on the forum also - e.g. other issues relate to having no technical documentation, and IMO a lack of reliability and stability - also the 'accepted stales' issues I've brought up elsewhere)

I don't see how declaring someone else 'bad' makes you or P2Pool seem better.
It actually reflects poorly on both yourself and P2Pool that you need to use such a comparison.

No.

Do I need to explain the English?
If English is not your first language I'll go to the trouble.

Otherwise - try reading it from the position of someone not enraptured with P2Pool :)

Hey so who are the six that don't trust EMC?  I'd certainly like to address any issues with that, as I try to be as open as possible with everything I do with regards to the pool.  If there's something that is questionable, I will certainly address it.

Also, realise, as I mentioned further up and you can read elsewhere on the forum, P2Pools advocacy of saying that non-P2Pool pools are not trustworthy will obviously imply that those who agree with that will not trust any pool but P2Pool and thus any of them could choose any pool to list as untrustworthy.

Not long after I mentioned that I trusted Ozcoin, the number of people who selected Ozcoin jumped up :P (from 1 to 4)

Be interesting to see if I've just caused yours to jump up :P
It's currently 7 - maybe even your comment made it go up 1 more?

Edit: actually I was thinking about this advocacy yesterday and realised I had seen something similar here on the forum before.
SolidCoin

Anyway, I would also suggest that my comments in this thread have probably made this current poll meaningless ... ... ... ...

Edit2: it's gone up 2 more since I posted - hmm I wonder

Nicely done.
Except for


But because more than 90% of the universe is illiterate on the difference between "it's" and its", thumbs up anyway.

Fixed! Thanks for reading so carefully. Feeling like being a proof reader, m3ta? I hate hassling Mobius all the time.

Odds on it's just disgruntled miners with an axe to grind, or as Goat suggested other pool ops who can't DDOS anymore. I wouldn't worry about it - you and Ozcoin have great reps for openness and reliability.

I don't think trust is a binary concept - you don't only 'completely trust' or 'completely distrust' anything or anyone. So p2Pool not requiring trust (if you've read the code) doesn't automatically mean you 'completely distrust' other options. So p2Pool advocates are being misleading, and if they are selecting random pools to distrust, or have selected 15 pools other than p2Pool to distrust, then they are being disingenuous or have a poor grasp of logic.

I'll normalise to the lowest distrust level anyway, so they won't have an effect unless they're targeting a particular pool out of spite.

Don't worry, your pool is great. For my part, I wanted to see the results of the poll, and voted for everybody so to not screw the % of the votes (well, except one pool, since you can only choose 15 on 16).

Some of these "distrust" votes do look fishy.

Results now available without voting. Hopefully no more donkey votes.

Next post available: 3.1 The rise and fall of Arsbitcoin.com (http://organofcorti.blogspot.com.au/2012/03/31-rise-and-fall-of-arsbitcoincom.html)

From the post:


I'm hoping for some debate on this one, from Arsbitcoin.com partisans and SMPPS fanciers alike.

So ... what am I missing if I say that a 'zero fee' SMPPS pool should just put 1.5% of each block into the buffer and pay 98.5% ?
i.e. the buffer loses out when you get an orphan, and the 'supposed' expected orphan rate is 1.5%

Thanks Kano. That will all be covered in the next post. BTW, Arsbitcoin.com records no orphaned/invalid blocks at all. I don't know if that's accurate or not. Where do you get your 1.5% stat from?

Hmm good question :)
I'm sure I heard it indirectly from a Pool OP or 2 ... but I can't for the life of me remember who :P
I'll try a forum search ...
OK ABCProxy's web page says 1.5%
MintCondition also said it again in the ARS thread
and I think I remember seeing 1.0% written somewhere also ...
But any pool with a count of orphans can easily determine a % figure.

... if you trust their stats. Apparently arsbitcoin have had none at all, and it would be in ABC's interest to inflate orphaned block rates since their miners are paid regardless. Then again being a proxy pool, they'd probably follow all pool stats closely, so maybe its true.

Anyone have a citable source showing the average percentage of rounds orphaned?

173222    
Timestamp    2012-03-28 04:40:16
BTC Sent    9,755.133 BTC
N tx    67
Relayed By    Deepbit
   
Timestamp    2012-03-28 04:43:36
BTC Sent    9,742.691 BTC
N tx    72
Relayed By    Ars bitcoin

http://blockchain.info/orphaned-blocks
Ars had the second to last orphan on this page :)

was looking to see if they had stats on numbers of orphans

Thanks Graeme. At least we know they get orphans. I haven't found a definitive percentage figure yet though.

New Neighbourhood Pool Watch : 3.2 The risks of SMPPS (http://organofcorti.blogspot.com.au/2012/04/32-risks-of-smpps.html)

I was going to call it "The failure of SMPPS", but Eligius is still going so I thought that a bit premature. Please note that the post is due in part to Meni Rosenfeld's paper "Analysis of pooled bitcoin mining reward systems". (https://bitcoil.co.il/pool_analysis.pdf)

Here's a bit from the post for the purpose of appetite-whetting:


https://i.imgur.com/mCj18.png

Ars severs have been down over 24hours now. A lot of  miners joining IRC with questions :/

Can the discussion include DGM? ...

I've yet to find a valid reason why DGM, which attempts to be SMPPS, is any better.

With SMPPS you can see the buffer, with DGM, you cannot since there 'apparently is no buffer' (there is no spoon? :P)

I've had this discussion with Meni in his thread, but been unable to see a valid reason for saying one is better than the other without using jargon to hide something that is effectively the same.

... and as I said above, regarding the issue of the buffer running out, and well, it will run out for what reasons?
The answer seems to be orphans (what other reasons?)
... and that can be solved by having a variation that puts the small % of expected lost BTC per block into the buffer.
A variation of that could even adjust that % based on history.

WTF???

In SMPPS the buffer affects the attractiveness of mining. Low (highly negative) buffer = miners will leave.

In DGM the "buffer" does not affect the attractiveness of mining. Thus there is no reason to know it, it's a property of the pool operator, not of the pool. I don't tell you how many bitcoins I own either.

In SMPPS there is debt which can go arbitrarily high. If the pool collapses while it is still in debt to you you are screwed (and this will happen when everyone leaves due to low buffer).

In DGM the debt is bounded and so the pool can (and should) pay back the debt if it closes for any reason.

In SMPPS the variance of the buffer is 100%, in DGM it can be controlled with a parameter.

The claim that DGM "attempts to be SMPPS" is outrageous.

As I said before, I'm extremely weary of having to deal with your hostility and explaining the same thing over and over again multiple times.



Anyway, as long as I'm here:

You talk about the maturity time, but it should be emphasized that the value I'm analyzing is the time of average payback, not the time of being paid in full. For example, if you get 50% of the debt back in 1 round and the other 50% in 3 rounds, your maturity time is 2 rounds.

The second chart, giving the PDF of the pool debt after n rounds, looks completely wrong, I have no idea how you got that. The pool debt, assuming constant difficulty, is (X/D - n)*B, where X is the number of shares it took to get n blocks, which is distributed negative binomial. So it's really a shifted negative binomial, which doesn't have the cusp we see.

I don't think you read the post. It explains why large negative buffers are quite likely the longer a pool goes on - without referring to orphaned blocks at all. Read it and get back to me.

The whole discussion about SMPPS is based on:

The assumption that this is true:
Which we ALREADY know to be FALSE.

Is there a reason to suggest to people to consider reading the rest when the reality is that arsbitcoin has proven it false?

If, on the other hand, a modified SMPPS that takes the expected pool loss into consideration (e.g. determine the % of blocks expected to be orphaned and determine that based on history after some period of time) and that % is put back into the buffer, then in the long term the pool will be expected to be even and thus people knowing that rather than listening to the WILD negative false assumptions that even such a modified pool would fail, would not be running and screaming that the sky is falling every time the buffer went negative.

However ...

Meni loves to say that "there is no buffer" so I take that to mean that when I look at a DGM pool that is paying more than 50BTC per block in a bad luck streak, that pool is in fact in serious trouble and about to fail?
Since there is no buffer, how can they pay more than 50BTC per block?

@kano I suggest you look up the word "Large" (http://en.wiktionary.org/wiki/large). Since you clearly have no interest in carrying out a reasonable discussion and would rather play dumb and ignore what your interlocutor says, I'll refrain from further responding to you.

For reference, here (https://bitcointalk.org/index.php?topic=39497.msg807444#msg807444) is my previous correspondence with kano about this issue.


I'll clarify one thing. In operator-risky methods like PPS, geometric method, DGM, the operator can either gain or lose on a round. These movements are in the operator's personal funds. Just like a riskless pool (like PPLNS) can collect fees for the service, so can a risky pool collect high or negative fees on a per-round basis. The variability in the operator's income is a risk he is willing to take in order to make the pool more attractive.

For his own personal accounting, the operator would do well to set aside a portion of his funds as the pool's "reserve". This makes sure that technically the funds have a place to come from, and allows a better handle on profits or loss. He could share information on his reserve if he wants. But this is still his own funds, which he can take away or add to as he pleases. The current reserve has no effect on the attractiveness, in terms of expected reward, variance and maturity time, of mining in the pool.

If the reserves run out, it is the operator's choice whether to allocate more of his funds to the pool, or to shut the pool down. Other than having to look for a new pool (which is not to be taken lightly), this has no ill effects on the miners - the debt to them, expressed as unrealized score, will be cashed out immediately based on the expected reward for it. Since the total possible debt is bounded (the bound can be controlled with parameters), the operator can simply set aside the necessary amount as an "emergency fund". There is no risk of growing an unbounded debt and then not being able to repay.

DGM settles good and bad luck on time and vents it as miner reward variance (or in the worst case, only likely with aggressive parameters, a clean bankruptcy). This allows it to continue in a sustainable steady state. SMPPS defers having to deal with luck indefinitely, until one day the debt grows so large it can no longer continue.

Wait... what?  How on earth do you come to this conclusion?  Arsbitcoin is a prime example of why it's true.  The buffer is negative, it's never going to be repaid, the miners who are owed money are screwed.  The few "hangers on" that are still around are the same people who buy lottery tickets, except in this case, they are hoping to win the lottery for the money they already earned, not extra income.


Meni's comments aside, speaking from experience, I can tell you there is no meaningful "buffer."  As Meni has already explained, it comes out of the operators pocket - in my case, it comes out of my pocket. I refill my pocket on short blocks and pay it out on long blocks.  The total "buffer" that ever gets into the negative is no more than a few BTC at the worst of times.  If you consider 3 or 4 BTC to be a lot/large, well then I stand corrected, but to me, 1200 BTC is tending towards LARGE and 5 BTC is tending towards SMALL, especially when speaking of a pool buffer.

Those DGM pools paying substantially more than 50 BTC per block are doing so out of the operators pockets, not out of any "buffer."  Each round, you are paid what you earned, no buffering required.  I could close up EMC's shop today and cash everyone out for full payment and be right about even - specifically because I don't carry an arbitrary buffer.

I did gloss over that. I'll fix it tomoorow. Thanks for pointing it out.

I'll have to get back to you on that. I wrote that part a while ago, and I'll have to find my notes. It was strange, I puzzled over it for ages, but I did confirm it with a simulation. I'll post it to pastebin when I have a chance to tidy the code up (it's in R) so you can try it for yourself.

Thanks for your interest, Meni :)

You're arguing about the the conclusion which I posted here without reading the materiel upon which it was based. Otherwise you'd try to prove your point instead of presenting it as a magical fait accompli.

Please read the actual post (http://organofcorti.blogspot.com.au/2012/04/32-risks-of-smpps.html), Kano. I think you mean well but if you aren't reading the post then I can't argue with you about it. You just come off seeming like a clever chatbot that repeats itself.

I've read the actual post twice already, ok 3 times now :P

The rest is based on the concept that over infinite time, during that infinite time, the value of the buffer can be any size - negative or positive - but taking that point to a specific case.

Which of course is true.
(though you didn't seem to bother to point out that it can be both positive and negative in the same way :P)

That is of course true of DGM also.
The DGM argument is that the pool will shutdown before that happens and the pool does not need to disclose the buffer.
I see that as a conflicting statement in itself.

Again you also make the assumption at the end
Which is in the least questionable as to it's relevance ...

Doesn't seem that way.
People keep mining there, the issue seems to be that BT is not maintaining the pool, not that the buffer is negative.
More people were mining on Ars with an approx -1000 buffer than were mining on EMC at some points.
The pool stopped so that's when mining stopped, not when the buffer got to -1000

So what are the grounds when you would close EMC?
A few weeks back DeepBit had 26 blocks where the average shares per block (for 26 blocks) was ~200%
What debt for you would that represent with your EMC numbers?



Aside: there is also the argument about delaying payout due to a negative buffer, but on DGM full payout is always delayed, on SMPPS payout is only delayed when the buffer is negative.

Full payout is never delayed, unless you count the "run-up" period (on EMC, it's 7 blocks), which is paid out on the run-down period (again, another approximately 7 blocks)... otherwise, you are paid in full each block.

As far as what grounds would I close EMC if the buffer were too negative?  Is that the question?  I would say -100 BTC, which would mean we would need a run of 200%+ blocks for over 200 blocks to even get vaguely close to -100BTC.  I just pull -100BTc out of my ass, BTW, just to prove a point.  I probably wouldn't even close it then, because I can cover that out of my pocket.  Although, if we did end up with a 200 block unlucky streak I would seriously be considering closing up shop, but not because of a negative buffer - I would pay whatever was negative out of my pocket to close up shop immediately on the 200th unlucky block.  Heck, if anyone was still even mining after the first 100 unlucky blocks I would be absolutely shocked.

My point is, it's virtually impossible with the proper parameters to have a negative buffer that has meaning... when we are talking about single digit BTC values, it's really meaningless in a pool context.

As far as people that keep mining on Ars is evidence that they don't understand the system.  Who understands how SMPPS works and is still mining there?  I'd like someone to speak up as to why you'd do such a thing?  There is nothing I can think of that would compel me to mine on a pool that has a negative buffer, where the best I can expect is to eventually, maybe, get what I am owed.  It makes absolutely no sense.

top TRUST WORTHY Pools
(not in any kind of order)

Slush
BTCGuild
Eclipse
Eligius
Ozcoin
P2pPool

if you voted that you do not trust any of these pools you are just plain clueless.

I'm lost
p2Pool is the clear winner, followed by EMC and Ozcoin. Well done to all in the top three pools - you've gained significantly more trust in the community than others. from https://bitcointalk.org/index.php?topic=66026.msg806445#msg806445
or are you referring to the current "untrusted" poll?

let me edit

this is MY OPINION - not result of any poll
(sure no one cares but here you go anyway)

and yes P2POOL goes without saying - but its not really a pool is it

ozcoin is trustworthy also graet ;)

I just dont understand why some of these people voted - does not make sense to me so I had to share my opinion

I mean seriously 15% think Slush is not trustworthy? Come on...

this poll is FUBAR
 

Yes of course that is what I am referring to :)
Whatever that number of lead in blocks is, you will never be paid in full for those blocks until either after you stop mining on the pool, or if the pool closes down and pays out the debt in full.

I'm just curious about how DMG works in reality rather than reading Meni's comments that certainly have an element of self bias in them :)

I mine on Ozcoin that is DGM, I'm not saying "throw it out, it's no good", however, but making comparisons that are dubious to promote DGM is not acceptable either.

Edit: though I could point out something else that everyone seems to be ignoring:
If someone mines on Ars and has an unpaid debt, they may consider it in their own interest to keep mining ...

Edit2: and if they had been paying 1% into the buffer to cover orphans, what would their balance be now :)

To be fair, any poll in the Pools forum is pretty much just a test of how rabid the users of pools are.  The polls on this thread have been popularity contests (or how loud the haters were), rather than their trustworthiness (or lack thereof).  Even polls specifically for a pool on their own thread tend to give pretty awkward results that don't give an accurate representation of the overall pool users.

Examples:

• Slush has 15 votes.  Not the largest number of votes, but very surprising.  Slush is the ORIGINAL mining pool.  The only bad news I've ever seen on Slush's pool was the Linode hack, which didn't impact users, and was not Slush's fault.
• Deepbit has more "untrustworthy" votes than Slush & BTC Guild combined.  Yet their track record is spotless, and they are the second oldest (still running) pool.  Only bad thing Deepbit has going are the people convinced that Deepbit is a threat, which is a vocal minority.
• P2Pool - This shouldn't have a single vote for being "untrustworthy".  It's a completely open source project that has no pool operator.  It shouldn't even be on the poll, since the entire point of p2pool is no trust is needed.

Those aren't the only ones I disagree with, but they're the 3 most obvious.  The two oldest pools who I have never seen do their users wrong, should not be getting remotely as many votes as they have.

+1, pretty much. It's really difficult to get an accurate representative sample of users here.

Meni, I couldn't find my notes so I've reproduced what I could from memory. I might have left out a step or two, but the results hold, and you can test them out if you like.

Let x1, x2, x3, …, xn be independent geometrically distributed random variables with the same probability, p.

Let Sj be the sum of x1 + x2 + x3 + ….+xj so that:

    S1 = x1
    S2 = S1 + x2
    S3 = S2 + x3
       …
    Sj = Sj-1 + xj
       …..
    Sn = Sn-1 + xn


Since x1, x2, x3, … ,xj …,xn are geometrically distributed, Sj is a random variable from a shifted negative binomial distribution with target number of successful trials = j, with density:

Pr(X=Sj) =  Γ(x+j)/(Γ(j) * Γ(x+1)) * p^j * (1-p)^x, so it follows:

Pr(X∈(S1, S2, S3, …, Sj, … ,Sn))
    = sum from 1 to n [Γ(x+j)/(Γ(j) * Γ(x+1)) * p^j * (1-p)^x]/n (1)

So the probability density of a cumulative sum of independent geometrically distributed random variables is given by (1) and tends toward a uniform distribution on [1,j/p].

Similarly, if x1, x2, x3, … ,xj …,xn are geometrically distributed with mean = 0, then Sj is a random variable from a shifted negative binomial distribution with target number of successful trials = j and mean = 0. Since the mean is 0, the component probability densities can be visualised as below.

https://i.imgur.com/cefNNl.png (http://imgur.com/cefNN)

The probability densities Pr(X=Sj) and Pr(X∈(S1, S2, S3, …, Sj, … ,Sn)):

Pr(X=Sj) =  Γ(x+(1-p)*j/p+j)/(Γ(j) * Γ(x+(1-p)*j/p+1)) * p^j * (1-p)^((x+(1-p)*j/p))

Pr(X∈(S1, S2, S3, …, Sj, … ,Sn))
    = sum from 1 to n [Γ(x+(1-p)*j/p+j)/(Γ(j) * Γ(x+(1-p)*j/p+1)) * p^j * (1-p)^((x+(1-p)*j/p))]/n (2)

Although (2) does not exist in a closed form, it can be calculated and charts of the distribution for n=100, n=1000 and n=10000 for p=1e-06 are shown below. This distribution does not seem to tend toward a known distribution for large n or very small p.

https://i.imgur.com/hyMu4l.png (http://imgur.com/hyMu4)

No, it's not. It's about how probabilities of the risk of an arbitrarily sized negative buffer occurring before n rounds can be calculated. It was to give miners a better idea of the risks involved with SMPPS.
True, but hardly useful when discussing the risks of losing shares. A positive buffer is not good for a pool in the way a negative buffer is bad for a pool. Maturity time won't get any less as the buffer becomes more positive, but maturity time will increase significantly as the buffer becomes more negative.

No, I don't think it is. I'll believe you if you can prove it using Meni's DGM functions though.
As they say on wikipedia, "says who?".

It was very relevant. If some miners leave the pool when the buffer is negative, then the pool hashrate declines and while maturity time in terms of blocks will stay the same, with a reduced hashrate the number of days (or hours or weeks) until payment increases. The greater the increase, the fewer the number of miners that are prepared to wait so long for payment. Miners leave, the hashrate decreases again. And so on.

I'm sorry the post was misleading to you. I hope no one else interpreted it that way.

+1 here too. A page back I found out that some people were voting randomly just to see the results. Plus grudge holders have a reason to vote negatively even if it's unfair, or if they don't like the pool and aren't voting on trust but just to get bad ratings for a pool. So I'm sorry to all who voted, I'm killing the poll until I can find a better way to get an idea about trust. Maybe I just have to only phrase it positively, like the first one. Ideas?

What is this quantity supposed to represent? It looks like you consider it to be meaningful but I just don't see it. Sn itself is the number of shares submitted in n rounds, isn't this what you're trying to analyze?

Also, if by the LHS you mean "probability that X is in the set {S1,..., Sn}", why would it be equal to the RHS?

In my opinion, it doesn't matter how the poll is phrased because your sample size is a narrow subset of overall pool users.  You end up with the problem of vocal minorities being the primary respondents to your poll.  While in theory that shouldn't skew results, it's quite obvious in the last two polls [and this one especially] that you end up with slants that don't seem to reflect reality.

Similarly, all it takes is one pool mentioning the poll in their thread, their website, or their chatroom to completely swing the results of the poll in their favor.

Meni.

I'll find the posts he said this later if I really need to?

Not just vocal minorities, but (hopefully) miners and pool ops who are concerned with promoting reliability and accountability for pools. I had hoped there'd be fewer barrows to push, but oh well.

When making a statement that relies on your interpretation of someone's else statements or even quoting them, a citation is vital to allow a reader to come to the same decision (or not) for themselves.

Top of the previous page here :P
https://bitcointalk.org/index.php?topic=66026.180

Then of course also our earlier discussion that he linked:
https://bitcointalk.org/index.php?topic=39497.msg807765#msg807765 ... etc. after that post

Organ, any news about the pdf issue (https://bitcointalk.org/index.php?topic=66026.msg835281#msg835281)?

Sorry, Meni, I wasn't ignoring you just wanted to make sure I had a clearer reply. Also looked for my notes again and failed. Further, realised I should be using the term "probability mass function' instead of 'probability density function' but I'll continue as I started for consistency.

It might be better if I just described what I'm proving, since I made some mistakes there which muddied the water.

The probability that Sn, the sum of n geometrically and identically distributed random variables (x1 to xn) will be some amount X is a negative binomially distributed variable where the probability density is = Γ(X+n)/(Γ(n) * Γ(X+1)) * p^n * (1-p)^X

Similarly for Sn-1, the probability that this sum will equal some variable X:
= Γ(X+(n-1))/(Γ(n-1) * Γ(X+1)) * p^(n-1) * (1-p)^X

and so on for all S.


The set (S1, S2, S3, ...,Sn) is the cumulative sum of the geometrically distributed variables. The probability that X is to be found somewhere in that set is given by the sum of all probabilities that X = some S divided by n:

Pr(X∈(S1, S2, S3, … ,Sn))
    = sum from 1 to n [Γ(X+n)/(Γ(n) * Γ(X+1)) * p^n * (1-p)^X]/n (1)

(note I've fixed it by removing the erroneous j's and changing x's to X's)

This is where my lack of notes fails me. I can't for the life of me remember where I came across this way of summing probabilities, or how I applied it. It doesn't appear to be the result of a convolution. Maybe you know?

Intuitively I see that, since the sum from x = -inf to +inf of each separate pdf = 1, and that the sum from x = -inf to +inf of the individual pdfs for S1 to Sn will be n, then dividing by n is 'normalising' (not the right word, but I hope you know what I mean) the probability to give (1), or if you like, providing the mean probability.

The mean of each separate pdf increases with increasing n and the 'normalised' probability approaches a uniform probability density from 0 to the mean of Sn as n gets very large.

(2) Simply defines the mean = 0 for each separate pdf and then follows the same logic.

Does this make sense to you? I remember the solution making complete sense to me at the time.

Interestingly, running simulations for uniformly randomly generated variables with a mean of 0 and negative binomially distributed variables with a mean of 0 and large n, give similar looking curves to each other, although on slightly different scales even with the same p.

Didn't think so, just wanted to make sure my query wasn't lost in the fold.

You can use the continuous approximation, and then you're talking about the pdf of the Erlang distribution.

No arguments there. (There could be off-by-one errors but that's immaterial.)

Here is where things start to get horribly, horribly wrong. Both in what you're trying to get, and how you get it.

First, I'll say again: If what you want is the balance after n rounds (which is what the text implies), it is simply (n - Sn/D)*B. That's it. Sn is the number of shares the pool had to pay. Any other derivation is nonsense. So I can only assume you really are trying to figure out the probability that a balance of X will be reached at some point within the first n rounds. Even if this is the case, the above quantity is misguided.

Let's begin with this formula. It is absolutely, and obviously, wrong.

The event X∈(S1, S2, S3, … ,Sn) contains the even X=Sj for any j. Therefore its probability must be at least the probability that X=Sj, which is the pmf of Sj at point X. So the probability must be greater than all the pmfs, so it can't be equal to their average.

The one place where the average of pmfs is meaningful is when you have a mixture variable. For example, if you have a variable Sr which is generated by first choosing a uniform random integer j between 1 and n and then setting it equal to Sj, its pmf will be given by this formula. But such a variable has no relevance whatsoever to the problem at hand.

If you do want to find the probability that X will be in the set, you need to treat this as a union (logic or) of the equalities X=Sj, which can be expanded with the inclusion-exclusion principle. I don't know if the expansion has a nice closed form in this case, I think it's extremely messy because of the dependencies between the Sj's.

So now maybe you can understand why the quantity Pr(X∈(S1, S2, S3, … ,Sn)) is not meaningful. It is not a pmf of anything. The total area under this curve is more than 1, and you can't take areas under the curve for any insight. The events 0∈(S1, S2, S3, … ,Sn) and 1∈(S1, S2, S3, … ,Sn) are not mutually exclusive, so you can't disjunct them by simply adding the probabilities.

You need to define the problem as "what is the probability that the balance will reach X or lower at some point within the first n rounds" (taking into account, obviously, the shift of +B for every round). This is a problem that should be solvable with recursion, for either the exact problem or for an approximation for it.

Not in the least. I'm not following your train of thought because you're not explaining what, why and how you are trying to get.

Probably the CLT at work.

Yes, my aim was to find the probability that a cumulative sum or balance of X or lower will be reached before n rounds. Firstly to define a pdf, compare with simulated results, and then sum the pdf to a cdf.


After more reading (and learning) I have to say I agree with you. So I really don't know how the simulation matches the 'mean negative binomial pdf' (which you show isn't meaningful). An error in concept maybe? I wrote a  simpler simulation (http://pastebin.com/7ZBErb3d) with no special libraries required, commented, doesn't make shortcuts and should be possible to follow if you have time.

The sim works as follows: Take n geometrically distributed random variables with the same p, subtracting 1/p from each variable so the expectation of each variable is 0. The take the cumulative sum of the variables. For example let p = 10^-6 and n = 10:


Now make m iterations of the cumulative sums to n and plot the histogram or sample density for the entire m*n samples. For example, if m=10, you'd plot the density or histogram of:

where n = the number of "fails" before each success, and m = the iteration number (the 'mth' time a sample of n was taken)

This gives a density plot or histogram which corresponds to the probability density of the distribution from which it came. If I then multiply the cumulative sums by B/D, I can then create a cdf and find, as you state, "the probability that the balance will reach X or lower at some point within the first n rounds".

So, at some point I somehow I came up with the supposed "mean negative binomial density" and wrote such a simulation which appeared to 'confirm' it. The sim I linked to plots the simulated cumulative sum densities against the "mean negative binomial pdf" for comparison, as below.

https://i.imgur.com/9lUKll.png (http://imgur.com/9lUKl)

If you click on the graph for a closer view, you'll see that the densities match very well.

I'm aware that random variable based simulations have limitations, but such a clear correlation is unlikely to be random. Since the "mean negative binomial pdf" is not just wrong but not meaningful, have I misunderstood what I was finding or looking for in the simulation?

Secondly, and possibly more interestingly, after your comment about the clt I looked for and found that for large n (larger than ~ 100) the simulated densities match skewed generalized error distributions quite well. Since I don't think I'd be able to use the inclusion-exclusion principle easily (tbh a the method to achieve a sum to a non-specific 'n' for this principle is beyond me), I'd like to be able to replace the error on the blog with a skewed GED instead as a model. However I'm unwilling to do more work on it until I can get a consensus (well, your advice :)) as to whether I've simulated the cumulative sum (balance) at all times before n rounds.

I know I'm asking a lot for you to go through the sim and look at the results. I wish I could say "I'll owe you one" but I'll just settle for being grateful for either a confirmation that the simulation works or a reason why it doesn't. Right now, I'm just confused.  ???

Thanks for your help.

@organ: I don't really speak R, but as far as I can see, your simulation matches the mean pmf because you simulated the mixture variable I talked about rather than your target quantity. That's also evident in your verbal description of the simulation. But I think I finally understand the source of your confusion.

The variable we are interested in is M="the minimum balance over the next n rounds". Its cmf at point X is the probability that the minimum is at most X, which means that the balance reaches X or lower at some point. So once we find the pmf of M, we can calculate the chance of hitting any X by the area under the curve.

To simulate the pmf, we need of course to make many instantiations of the random process, and add +1 to the empirical weight of the resulting value of M. What is the value of M? The minimum among all the values of the (shifted) cumulative sum. But instead of adding +1 to the minimum, it seems you added +1 (or +1/n after normalization) to all the values in the cumulative sum. If this is fixed you'll get a proper simulation.

Instead of a generative simulation, you can also find the cmf by approximating the process as a simple random walk, and solving (numerically or, if possible, symbolically) the following recursion (in Mathematica syntax):

Using this, I find that f[20, 1000], the probability of going below -1000 BTC (20 blocks worth) in 1000 rounds, is approximately 50.67%.

I'm going to make this quick because I'm being called to bed but you got me thinking and I wanted to get a post out in reply, so sorry if my response seems terse and not well thought out.


 "the minimum balance over the next n rounds" is what I derived in the second part of the section using Erdos and Kac's probability limit theorem I.

What I wanted to do as a first step was find the probability of any given balance or less before n rounds (after finding the area under the density curve), so that after the second derivation I could compare it to the probability of a given minimum occurring. Is this the mixture variable?

I'm not following this bit, even accounting for the aim of finding the minimum. I'm thinking in terms of the cumulative sums being:

[(Geom var1 - 1/p)], [(Geom var1 - 1/p) + (Geom var2 - 1/p)], [(Geom var1 - 1/p) + (Geom var2 - 1/p)+ (Geom var2 - 1/p)] , .......

where I've made the share-debit a (non intuitively) positive number. I'm subtracting 1/p (or D) each round. What did you mean by the +1 or +1/n? I think there's a step I'm missing.


I'm going to have to spend a bit of time on this tomorrow since I only know enough mathematica to get by on wolframaplha when natural language doesn't work, but thanks for giving me something new to work with which looks to be much quicker than my current method.

Using the probability limits theorem, I find the probability of going below -1000 BTC in 1000 rounds as 52.71%, and not until 1000 BTC at 1050 rounds did it reach 50.66%. Do you think this an acceptable variation given the different methods used, or a problem for me?

Cheers again.

You should probably think about it some more. "Reaching a balance of X or less at any point" is equivalent to "the minimum balance is at most X", which is the cmf of M. If you find the pmf of M, you can use it to find the cmf. In any case I'm talking about balance rather than debt, but that's only a matter of flipping the sign.

It's just the tallying of the empirical pmf. If I wanted to simulate the probabilities of the results 1-6 on a die roll, I'd have variables num1,... , num6, and I'd add +1 to num1 whenever I roll the die and get a 1. Likewise, you need to tally the number of times each value of M appears. Unless you have a different approach to this?

It's within reason, the model solved with this recursion is not very accurate (and I'm guessing the application of the general theorem also has some error). Maybe a better estimation can be found with a simulation after all, with 100000 iterations of 1000 rounds I get about 50.8% chance of going below -1000 BTC. My code for this is:

Yes, I'm going to try to explain what I mean in terms of a random walk, I think. But right now I've just got a flashing cursor in my head going "thinking.......thinking.....thinking....". I touch on it briefly below in terms of simulation though.

After looking at your simulation below, I'm doing much the same thing except in R. In the limit theorem part, the simulation does exactly the same thing, except instead of measuring a specific amount I plot the overall density.

However for the first part, with the cuspy sGED modelable density, I'm not taking a minimum of the cusum of 1000 rounds but collecting all the cusums. I end up with (in this case, and with the number of iterations you use below) a 1000*1000000 matrix of variables. I then plot the density of the variables.

So to emulate this on Mathematica, you would:
1. create a 1000 col *1000000 row array or matrix, name it MAT
2. put Accumulate[RandomReal[ExponentialDistribution[1], {1000}]] - 1] in the first row
3. Iterate step 2 for rows 2 to 1000000
4. not sure this is correct, but try SmoothHistogram(MAT, Automatic, "PDF") and SmoothHistogram(MAT, Automatic, "CDF"). You might need to convert the MAT to a vector first.
5. Repeat for 10, 50, or 100 rounds on a 10 or 50 or 100 column by 1000000 row array for the histogram at a varying number of rounds.

Maybe this makes clear what I'm recording in the simulation (if not the why to which I'll give some more thought).


Your results are very similar to the ones I get in simulation. I've noticed that for larger probabilities the simulation always diverges slightly from the limit theorem result. If you repeated the simulation with fewer rounds or for a less probable -50, the simulation results will agree with the limit theorem error function much more closely. Cool, huh?

The what has been clear for some time.

As for the why - you only need to provide a clear, verbal, unambiguous description of what you are trying to find. In the second part that is fairly clear and as far as I can tell you have a simulation that serves the purpose (you get the cmf of M, from which you can obtain the pmf if you want). In the first part it looks like you just took a bunch of numbers and mashed them together for no reason. The solution as far as I can see is to just remove the first part.

EDIT: It's actually easy to give a verbal description of the first graphs. The one with the cusp is a graph of "the average % of rounds that end with a balance of exactly X", and the accumulated area under it is "the average % of rounds that end with a balance of at most X". (eg, if there are 1000 rounds and 300 of them end with a balance less than -1000 BTC, the value of the variable will be 0.3 - and the graph lists the average of this over all instantiations). Note that as described these are not a pmf and cmf, though they can be looked at as the pmf of cmf of the mixture variable earlier discussed. And again, what you're looking for is not the average % of rounds that end with at most X, but rather the probability that at least one round ends with at most X.

If I had to take a wild guess, I'd say it's because the tail of the exponential distribution (corresponding to bad luck) is more similar to normal than the head. If that's true, your observation will be different if you take the negative of each shifted geometric variable (so effectively, instead of looking at hitting a low value, you're looking at the probability to hit a high value).

Exactly! I was glad of that edit. I was about to post a description of the same using a random walk, but you've done a much simpler and clearer job. So, it just remains for me to try to explain why I thought "the average % of rounds that end with a balance of exactly X" or "the average % of rounds that end with a balance of at most X" is useful to know.

As you've shown, a negative buffer at an SMPPS pool will contribute to the a delay in the amount of time (in terms of blocks solved) that it takes for a miner to be paid. So the following two results will be of interest:
1. The maximum a negative buffer can be before n rounds.
2. The percentage of n rounds that one can expect the buffer to be any arbitrary amount (or worse/better).

Number 1 is of interest because a significantly negative buffer could delay payments long enough to kill a pool.

I thought number 2 would be of interest because knowing that for example one can expect 300 rounds out of the first 1000 to have a negative buffer of 10 rounds or worse, a miner might not decide to mine at an SMPPS pool, since for 30% of the time the average maturity time will be of the order of the amount of time it takes to solve ten blocks or longer.

If I haven't made a wrong assumption there in 2, it also occurs to me that there will be an expected and non-zero maturity time, since an increasing positive buffer doesn't reduce the maturity time as an increasing negative buffer increases it.

I guess. Still doesn't seem like a very intuitive quantity to look at.

Right, the maturity time for future rounds is nonnegative and can be positive, so it has a positive expectation. And that - the average maturity time over the next N rounds - may be of interest to some people.

For an optimal miner it is inconsequential, because he can just leave when the buffer is low. But for a miner that wants to sign up for an extended period of time, it is a useful quantity that can be compared to the maturity time of other pools. The pmf of this average can fairly easily be simulated, and is more useful than the current graphs and IMO can replace them (as long as it is very clear what they represent). I can't think of a closed form for the pmf, but maybe it can be approximated by modeling it as Brownian motion (http://en.wikipedia.org/wiki/Wiener_process).

Well that's clear. It took a thread page to explain it, which you ended up doing anyway. No, I don't think it's a useful quantity to help explain anything to a miner who might use SMPPS. I do think mean maturity time will be much more useful.


I got as far as Donskers theorem before I realised that the mean negative round time could be calculated using a shifted negative binomial distribution with mean = 0, if I could find some way of calculating the mean of the negative buffer half of the distribution.

Since the negative binomial distribution approaches a normal distribution for large , I modelled it using a half normal distribution for which a method of caluculating the mean was already available. Since the positive buffer half of the normal distribution has weight zero when calculating the mean maturity time, the mean calculated using the half normal distribution method is halved.

The results from n = 6 to n= 10000 are within +/- 2% compared to several repeat simulations I ran, so I'm happy with the accuracy of the half normal distribution method and I'd like to use that rather than present a simulation since if readers want to they can follow the logic which is harder with a simulation written in code they won't know.

What I'd like to know is - did I explain that clearly enough? Is it correct? I'm not sure about the best way to explain the weighted means [Edit wrt probability distributions], and I 'll have to go into a bit more detail, but I did want your optinion before I went ahead and replaced that section. Also, does the graph (below) of mean maturity time up to 1000 rounds seem likely to you? If it is, then 1000 rounds into an SMPPS pools life you can expect a mean of about 12 rounds wait to be paid your average earnings.

https://i.imgur.com/4yvZg.png (http://imgur.com/4yvZg)

Finding the expected maturity time at round N is easy enough - as you say with a normal approximation (equivalent to a Brownian motion model) it's simply sqrt (N/(2Pi)), and the graphs match that. I thought it would be interesting to look at the expectation of the average maturity time in rounds 1 to N, but that's harder so the former would suffice.

(Edit: Actually, since expectation is linear, the expected average over 1-N is just the average of the expectation over 1-N, which is roughly 2/3 of the expectation at N. The pmf, though, is probably difficult.)

I think the explanation could use some more detail.

Yeah, i still have 2 bitcoins unpaid on arsbitcoin, it seems i'll never get it back.

Well, as Luke-jr (the inventor of the SMPPS payout method) would say:


But I'm sorry you had to lose coins to find that out. Even besides the SMPPS payout, the poor pool shutdown has affected many miners. Found a new place to mine yet?

I'd not thought of looking at the expectation of the average maturity time over N. Does it make sense to graph it as a function of N? I want to use whichever measure will be the most intuitive to understand.


Yes, indeed.

Yes. I guess "average maturity time at round N" will be simpler.

Neighbourhood Pool Watch 4.1 Slush's pool (http://organofcorti.blogspot.com.au/2012/04/41-slushs-pool.html) is posted.

Also, the update to Neighbourhood Pool Watch 3.2 The Risks of SMPPS (http://organofcorti.blogspot.com.au/2012/04/32-risks-of-smpps.html) is also posted. Thanks Meni for your help and advice on correcting it.

Thanks very much to the person that last donation was from!

Neighbourhood Pool Watch 5.1 p2Pool - bad luck or flawed? (http://organofcorti.blogspot.com.au/2012/06/51-p2pool-bad-luck-or-flawed.html) is posted.


https://i.imgur.com/Gx9Wn.png (http://imgur.com/Gx9Wn)

I didn't read the entire analysis but I disagree with your conclusion that round lengths are inherently longer, there is hardly enough evidence to rule out bad luck. Especially if there is no plausible explanation how can round length mean be higher.

There's enough evidence to show that the round length mean is greater than D to a 95% confidence level. It would have to be quite bad luck.

But I agree there's no mechanism I know of that could plausibly cause such a result. But I'm not a p2Pool expert, and I'm hoping that such an expert will come on board and discuss it. The p2Pool share chain and dynamic difficulty work so differently to a normal pool that I really can't comment. I was thinking that some shares may be unaccounted for and not reported locally, or some such thing. Most likely improbable, and if someone can prove the impossibility then I will gladly edit the post.

Did you factor in the high stale rate compared to a normal pool? 

DOA and orphan shares don't make it into the sharechain and so can't increase the numbers of shares in a round.

One possibility is that the method used to estimate the equivalent difficulty 1 shares to solve a block using the pool hashrate is buggy. If for example the pool hashrate is actually 9% lower than reported, then the number of shares to solve a block would be D.

That's why Bayesian statistics is superior to frequentist statistics. All the data gives us is about 10:1 likelihood ratio for 1.1D over 1D, which doesn't mean much when the prior probability of 1.1D is so low.

That makes more sense. But even then, not much evidence.

10:1 likelihood ratio? Isn't it 20:1? But I get your point. Perhaps I was a little too rash in discussing what it might mean in terms of possible future miner earnings. But it does mean that either:

1. p2Pool's had a very poor sort of luck.
2. p2Pool's round length mean is larger than it should be.
3. There's a problem with the reported pool statistics.

I think it's worth pointing out the possibility of either 2 or 3, even in absence of a mechanism known to me. Many p2Pool miners are concerned about the pool's luck, and I've read posts by a few that have left the pool for that reason. If the p2Pool devs can come up with an explanation then it might save the pool losing even more miners.

Edit: if there is a problem with the reported stats then p2Pool miners should be receiving on average the expected value for their hashes. Anyone from p2Pool how has been mining consistently care to share their earnings history and hashrate?

According to http://p2pool.info/luck/, 3851 PH were calculated but 3491 PH worth of blocks were found. Assuming difficulty = 1.6M this means 560.4 blocks expected and 508 blocks found. So the likelihood ratio between poisson distributions with mean 560.4 and 508 for 508 found is (508/560.4)^508*exp(560.4-508) ~ 12.5.

Where are you getting 5% probability from? I'm seeing more like 1% here.

Left p2pool a while ago, might go back to using it as a backup at least as/if it improves...

Um... perhaps something to consider...

When you were evaluating the other pools, one variable that didn't come into play, was the propagation of the new block among the miners.

Since with a centralized pool, long-polling delays aside, everyone is basically mining off of the same daemon.

With P2Pool, this is not the case. We're effectively all solo-miners, with pooled payouts. When one node finds a block, it takes time to propagate, so for a period of time, you would have part of the pool still working on one block, while a growing portion of the pool is working on the new block as the found block gets distributed.

As far as orphans go, a centralized pool is in itself, a single solo miner, albeit one with lots of connections, and bandwidth to distribute the new block efficiently. P2Pool, on the other hand, is a whole bunch of solo miners cooperating.

So, one thought that occurred to me, is when comparing stats, shouldn't solo mining be considered the normal, with the centralized pools, being above normal, due to the efficiency they have in their miners being notified that they should be working on the next block all at once? If they weren't pooled, all of those miners, would be notified at different rates, which may have an affect on orphan rates.

Perhaps, the orphans P2Pool have been experiencing were simply normal for solo mining...

(I'm no mathematician compared to your level, so, I may not be quite clear on what I'm getting at...)

-- Smoov

@Smoovious: I think it's the other way around. A centralized pool should have more latency than solo because first the pool needs to learn about the block, then it has to long-poll you. When solo mining you just need to learn about the block.

If I understand this right, you're talking about stale shares, not orphans. If I find a block on p2pool, but my bitcoind is only connected to 8 peers, it might take a while for the block I found to propagate through the network. Meanwhile Deepbit finds a block and sends it to 1000 different nodes all at once, orphaning my block.

I don't know the workings of p2pool fully. Do they send newly found blocks to each other as well, to aid in propagating it?

The 95% CI excluded a mean of D, median of ~ 0.693D, and an sd of ~ D. I didn't check the 99% CI. It may also exclude the expected values.

Well, if a stale share happens to be a block, it is technically an orphan block, so this is still an increase in the orphan rate, depending on how you count it (maybe orphan blocks which are known to be orphaned when the pool receives them are not counted). What we care about here is the total rewards of the pool, not how they are distributed internally.

Agreed. So both situations actually affect p2pool negatively. Assuming that the average p2pool node has lower bandwidth and lower connectivity than a big pool, it will generally receive blocks and send out new blocks slower, resulting in more orphans overall.

Yeah... I think... and, I wasn't around in the early days, not beginning to mine until the last quarter of last year, so, dunno about the early intentions of bitcoin...  but...

When it was designed, I think the intention was for individual peers mining individually. Not sure if pooled mining was the intent from the start. (I'm sure I'll be corrected on this if I'm mistaken :) )

So, a higher level of orphaned blocks, when scaled up to a lot of peers, may have been an expected metric, and considered the normal situation. Perhaps P2Pool's orphan rate, better reflects the way this would have been, without the centralization of mining?

What the centralized pools did, was cut out the latency from when found blocks, propagated to the other people mining. Since they would inform the mining peers more or less, simultaneously, it would have the effect of near-instant propagation among a large number of peers. Since none of them would have to sanity-check the block against their own blockchain, avoiding the latency that would come with it while distributing it, and not having to pass it along to the next set of peers afterwards, you have a large segment of miners abandoning the current block and starting the new one, much sooner than would have been expected, had everyone been solo-mining. This alone, would have eliminated a portion of what would have been a normal orphan rate.

All of those mining peers, on a centralized pool, would be considered collectively, a single bitcoin peer as far as the network is concerned. Take the top 75% of the hashpower on the bitcoin network, and you end up with, what... a few dozen bitcoin peers? Perhaps a little more? Certainly a tiny fraction of all of the individual bitcoin peers that are mining as a whole, by node count.

I don't know how this could be reflected in useful data, but perhaps this has an affect on assumptions of expected levels of good blocks and orphans within a given period of time?

I think what I'm getting down to, is... perhaps instead of looking to the average pool results to find normal, the pool results, should be considered above-normal instead of the baseline, which would be pure solo mining on a more distributed network?

-- Smoov

forrestv added some protocol changes to aid this very thing, but we're still dealing with a distributed network for the daemon's connectivity, and a separate distributed network for p2pool's connectivity. We basically have another level of notification, letting us know to work on a new block, but we still have the latency of a distributed network, getting that notification propagated. I think I am understanding it right tho, that p2pool's notification, will go much faster, as it deals with less data in the transmission, and less sanity-checking at each node before it gets passed on.

The main difference, the way I see it tho, amounts to the way a centralized vs decentralized system handles how the data gets around. How much it is handled between hand-offs, and how many hand-offs between the finder and the rest of the pool.

Take deepbit as an example... when one of their miners find a block, it is a very short path for the rest of the pool to be notified...

miner1!deepbit!{miner2|...|minerA}

With P2Pool and no centralized bitcoin, and with forrestv's protocol change from last week, we have 2 paths, but they can be pretty long.

miner1!p2pool1!bitcoind!bitcoind2!bitcoind3!bitcoind4!p2pool2!miner2
or
miner1!p2pool1!p2pool2!p2pool3!p2pool4!p2pool5!miner2

The paths the data needs to take, through either route, can be even longer. The route via bitcoind would be slower, and via p2pool would be faster, but both would be much slower than the deepbit path above.

If I remember right, we (p2pool users), have orphaned one of our own blocks before. Something that couldn't happen at the centralized pools.

-- Smoov

Although they don't affect the number of shares per block difficulty
(which is really the only valid measurement)
DOA and orphan shares reduce the 'calculated' hash rate of people using p2pool.

P2Pool itself does apparently send out DOA and orphan shares to the bitcoind to ensure that shares that are no good for the P2Pool share-chain but are actually a valid BTC-block will still go out.

My other suggestion (that I've said many times) of the cause of p2pool's "misfortune" is that a 'typical' p2pool is quite literally crap compared to a well tuned pool.
Thus the delay between the miner finding a share and the share making it 'out onto the net' would of course be worse on average with p2pool users vs a well tuned, well connected, high performance pool server.
This 2nd item would of course show up in terms of more orphans compared to other well configured pools.

Recent BTC problems may hide this :P

I included orphaned blocks in the average round length calculations, since it was the number of shares taken to create a valid block - so the number of orphaned blocks do not affect the round length / D average. They do still affect miner earnings though.

Rate of orphan production for the first hundred blocks solved was 0%, for the next hundred 1%, for the third hundred 3%, the fourth hundred 6%, and for the first 35 blocks of the sixth hundred blocks, 8.5%. Although there are only 17 orphaned blocks altogether, is does seem that orphan production has increased. This could just be variance though - I have no idea what the mean or variance of the orphaned block distribution should be, and not enough data to even guess.


Edit: and a quick thank you to the recent very generous donator, whoever you are. Much appreciated.
Edit2: actually thanks to the last few donators, especially whomever sent exp(1)/10 btc - I got a laugh out of that ;)

@organofcorti Why do you work so hard to show how bad p2pool is? Do you have a dog in the race?

No. No dogs in any race. I just provide miners with information. I thought that would have been evident from the other posts on the blog and in this thread, though.

I hope by "working hard" you don't mean "working too hard to a preconceived conclusion". I didn't have any. I'd thought that once I had a good look at the data it would be within certain limits that included normal but it didn't, within a 95% confidence range.

Meni Rosenfeld points out that the conclusions could be incorrect, especially I think this statement:


which is a conclusion one cannot come too really given that it's based on a 95% CI.

So at the moment and before I edit the blog post, I'm looking for more p2Pool data from miners in order to try to get a better idea of what's happening.

Finally, the last points of the conclusion are:


which summarise my personal feelings about p2Pool nicely.

I'm not "anti" any pool I blog about, but I find it's more useful to miners in general to analyse a pool where there seems to be a number of miners who are unhappy or have concerns (be they validated or not), rather than one that has happy miners.

He was being fair.

It is about the numbers, period.

In fact, his analysis, was helpful in validating some of the concerns some of us have posted about long-term luck.

Whether we like or dislike the results, fitting or not fitting our own conclusions, is irrelevant to the data.

-- Smoov

@organofcorti Why do you work so hard to provide objective, actionable information to people who would rather bury their head in the sand?

Seriously though, check_status, your comment was completely out of line. OOC has empirically investigated many pools, in many cases he found inadequacies caused by negligence or malice of the operator.

There was a legitimate concern raised about p2pool functioning properly. I learned about it myself some time ago and run some calculations, hoping and expecting to find it well within the acceptable range. What I found was that it is within it, but not well within it.

OOC too was hoping to show how great p2pool is and that there is no problem despite the fear. What he found was that there is weak evidence suggesting a problem. What would you have him do? Hide the information? Lie about it? Or share it so that the relevant people can look into it and possibly fix it (whether it affects the rewards or just the displayed values)? Or even if it is in fact an insolvable problem, being open about it and giving people the chance to make their own decisions in full knowledge of the facts? In fact mechanically presenting the data despite his personal feelings about it is exemplary scientific integrity.

In my day job, whenever I find an inconsistency in our data, people are ecstatic because it means there is a bug that, when fixed, will improve our data considerably. I don't get put down for working hard to show how bad our data is. You should adopt the same mentality.

The plus side of his analysis, is, after taking into account the concerns we were already debating in the other thread, the rest of his data confirms that p2pool is being fair with its members when it comes to payouts, which is a good thing for p2pool's reputation.

I also think it is good he ran the data when he did.

Since forrestv has made some changes that should have a direct effect (fingers crossed) on reducing our orphan rate, we now have a 'before' analysis put together, which can be used to compare against p2pool's statistics, post-change, once there are enough data points to make a reasonable analysis, to see exactly how those changes have improved things (fingers still crossed).

-- Smoov

organofcorti has worked long and hard to show miners the truth and break through misconceptions, not just about p2pool but also about hopping and exploitable payout systems.
I respect and appreciate the work he has done for the Bitcoin community and miners in particular.

I also see organofcorti's work constantly reviewed by Meni Rosenfeld a champion of fair mining and another member of the Bitcoin community I have great respect for.

I do have a "dog in the race" but have no problem being scrutinised. Only those with something to hide would fear it.

Best wishes
Graeme Tee
Ozcoin Pooled Mining Pty Ltd

Let's disect this statement shall we:

a decentralized pool = an acceptable price

This connotes a sacrifice in some other area. According to the majority of organofcorti's posts, that is profitability. Your using words to paint an unprofitable picture about p2pool that doesn't exist, the blocks attest to this.

It's not out of line at all. There is a preponderance of evidence of organofcorti's p2pool posts where he flat out says or implies p2pool is the least profitable pool. You are not a fan of p2pool either, some of your remarks have been aimed at having p2pool users mine at other non-p2pool pools. A hater in support of another hater, I'm shocked.  :o

Do we need to get the blocks out to see how p2pool provides one of the highest PPS payouts per gigahash when compared against any pool.

Most people who mine watch what is said and make decisions on what occurs in posts. Many lack knowledge of how to determine profitability of a given pool, they rely on the thoughts and comments of others whom they deem "smart" and therefore, can be swayed by "Inteligent BS".

According to my blog post, the sacrifice is profitability. As I already noted, this statement was stronger than warranted, and I will be changing it to a provable statement at some point soon.

Please provide a link to a post where I flat out say that p2Pool is the least profitable pool.

I guess we both must be in the pay of the Big Pool conglomerates then, huh.

Sigh. I already did. Which part of the blog post didn't you follow? I provide the table of the expected parameters ("Theoretical 1") and experimental statistics for p2Pool here to refresh your memory:

https://i.imgur.com/aPmHX.png (http://imgur.com/aPmHX)

The data I based it on is here:

http://pastebin.com/g2ESh1Uf (http://pastebin.com/g2ESh1Uf)

Highest PPS payout? More than 100%? If the pool is paying out based on the recorded pool Ghps, then it has paid out 0.91 * PPS so far. If the public data is wrong and the payout based on your local hashrate is in the expected range, then in the long term your local payout will approach PPS.


I'm grateful to anyone who puts time in to reading and understanding my blog. If they can show me a fault in my work, then I'm even more grateful.

Simply implying that I'm "smart" and the analysis "Inteligent BS" is not a constructive criticism. Read the blog and find a mistake that I haven't already owned up to. Otherwise you're just posting "BS". And not very intelligent BS, either.

Even if that's true, offering constructive criticism is not hating. Truly supporting something  is caring enough to try to improve it, nobody needs a yes man.

I am a fan of p2pool, and I've spoken several times about its importance going forward.

I don't recall ever asking/recommending p2pool miners to switch to a different pool, so this is purely your own speculation on what I might be aiming at.

A few remarks I did make about p2pool:
1. The reward system should be looked at to verify that it is truly hopping-proof, and not just "almost hopping-proof" like the naive variant of PPLNS.
2. Some of its suggested advantages are a bit exaggerated, at least at this point - the network isn't going to crumble tomorrow without p2pool.
3. The correct way to use p2pool is not by directly connecting to it, but with a PPS proxy.


[joke] Must be Deepbit. They're also clever enough to hide this connection by paying us to speak ill of the proportional method. [/joke]

I am curious about where you got "Round shares" from :)

Yes I am also glad to see that shares is the basis for the calculations which of course is what you would use (yes I'm stating the obvious) - but it seems that there are those who don't understand this and thus argue about the results due to not using those same results.

My curiosity is simply that the share-chain throws away this information and thus I guess someone must have full logs of the share chain since day one to be able to provide that information.

It'a all here: http://p2pool.info/blocks

Round shares are an estimation based on pool hashrate and time between timestamps. It's in order of block height, so sometimes if a round is short and timestamps inaccurate, you get "negative" round shares. This  happens infrequently and doesn't affect the round shares probability distribution significantly. It averages out the same - if you used total hashes and total rounds, you'd get the same mean round length.

Play with the data, see what you get.

OK ... not the answer I expected.

hashrate is an estimate based on the number of shares.

Using someone else's calculation then reversing that back to the number of shares is a little risky IMO ...
It assumes that the hashrate was indeed calculated correctly ...

The reason for using the number of shares is that you can't really count shares incorrectly unless you are losing count.
If you aren't counting them correctly, then the hash rate will also be wrong no matter how you calculate it.
You may not have the count, but it you have the count of shares that is certainly better to use than a hashrate estimate converted back to a share count

Yes. Although I can show that the hashrate estimated shares are still consistent with a geometrically distributed probability function, so the if the hashrate estimated round shares are in error then it would likely be a constant error.

Somewhere above I pointed out that if the global hashrate estimate is incorrect, then that could account for the longer than expected round hares. That's why I'm encouraging p2Pool users to PM me their stats. If they're earning the expected amount based on the shares submitted, then the results stand. If they earn on average 10% more than expected then the pool hashrate calculations, and hence calculated round shares are overestimated and and pool luck underestimated for some reason.

I wouldn't have used the hashrate to shares conversion if I had a choice.

Because p2Pool has variable difficulty shares, you have transform them back to hashes. I would have preferred having each share recorded and then tagged by difficulty and block number, but this data isn't available to me. So I have to trust that the round shares figure they provide are accurate (and the pool's had really bad luck), or show that they are not (and the pool's luck is different, maybe in the expected range).

PPS payout:
86,400 seconds/day * MH/sec / 4294.967296 (megahashes/share) * PPS = Payout/day

Finding expected PPS:
Take Payout/day from Alloscomp's calculator for 1000 MH/s and place into Payout/day of PPS payout to solve expected PPS.

To find your Payout per gigahash/s you will need, a pools avg. GH/s for the period your measuring, # of blocks divided by days.

Finding the expected payout:
According to Alloscomp's Bitcoin Calculator: http://www.alloscomp.com/bitcoin/calculator.php
If the difficulty factor is 1583177.84744, and the MH/s is 1000, then your Payout per GH/s per day is ⊅0.64
This will be the standard measure, the expected value, for comparing miner, pools or network payouts.

Calculating the past 7 day PPS and payout/day for P2pool:

P2pool 7 day average hashrate: 222 GH/s
Previous 7 days total blocks from 6/14-6/20: 23

23 blocks * 50 / 222 GH/s = 5.18018 / 7 = 7 day avg. Payout per GH/s or ⊅0.740026 per GH/s per day

P2pool has been performing for the past 7 days 15.6% better than expected.

That's nice but you actually expect to get 100%, not 115%

It does show that it is possible that luck is now hitting 100% instead of the low 9x%
However, 7 days is way to short to make any conclusions.

This week you should expect to get 0.58255098 a day due to the difficulty rise to 1726566.5591935

And that's good news for p2Pool miners, but I couldn't base an analysis on just one week. I based it on all rounds since the start of the pool. On all the pool data available, not just the last week, the pool as a whole has solved 535 blocks with 590.83D shares. That's only 90.5% of expected, or 9.5% worse in your terms.

something else to keep in mind is that work lasts longer than 1 round, sometimes work lasts 2-5 blocks later. so the work is distributed, pps comparisons are somewhat skewed because of this. the pps is going to be lower, but has momentum. so if you quit, you still get paid for a while even though your hashrate is zero.

lol at poll:

Can you explain this in more detail? What work do you mean is lasting more than one round?

Ah, yes, um, well you can select any number of options up to 15, as long as that number is also less than four. Yeah, that's it. ;)

Mods, can we get a poll edit here?

our shares are persistent over a period of time... I believe it is for 24 hours, or the expected amount of work it would take to find 3 blocks, whichever is lower. (better explained by others)

So, if I stop mining, while my payout is 0.0377... the shares I have persist until they fall off the end of the chain in a FIFO method. If the pool finds 2 blocks, within the next 12 hours, I would get a payout for those 2 blocks, based on my shares that haven't fallen off of the share chain yet.

Perhaps this is what the poster meant by work lasting longer than a round?

-- Smoov

P2pool's actual payout per gigahash per day for the period 8/24/11-9/23/11 or 31 days

Average daily hashrate: 9 GH/s
# Blocks found: 6

6 * 50 / 9 = 33.33333 / 31 = ⊅1.075269 per gigahash per day

Now all that is needed is the difficulty factor for the same period to calculate expected value, then compare.

Get the difficulty from theymos' baby:

http://blockexplorer.com/q/nethash

Post an update when you're done?

Unfortunately, it isn't in n00bese, so I would need another source for the difficulty factor.

Difficulty is the fourth column from the right.

Just copy paste the page into notepad, save as "nethash.csv" and then open with Excel. It will open to a standard csv spread sheet.

Otherwise just use my data here:

http://pastebin.com/g2ESh1Uf

which is also a .csv

Check a few datapoints against the the data at p2Pool.info/blocks to satisfy yourself it's the same dataset as on p2Pool.info. Eaxch block is tagged with difficulty.

Below are 2 examples out of many that show organofcorti talking about p2pool earnings in relation to other pools:
and

I have provided 2 samples, 7 day recent and a 31 day historical that show his assertions are nonsense. organofcorti's assessment of p2pool profitability is erroneous, designed to confound and dishearten the uninformed.

Since you feel the need to argue in a thread dealing with statistics ... but you are ignoring that fact ...
Look up and learn all about these 2 words in detail and how they relate to statistics:
Sample, Population.
If you still disagree with the discussion about p2pool after that, then it just means you didn't understand what you read, so read it again.

check_status, I'm happy for criticism and if I'm wrong I'll admit it and change my results. That's what thinking people do: read, learn, and change their minds when the facts show them to be wrong. Unfortunately you are not answering an important question which is: Where am I wrong?

As far as your quotes go: since you didn't provide links I'm providing the full statements for context. I assume these are the quotes you are referring to, since I couldn't find those exact statements.





I haven't written anything there that I have not proven. If you have a problem with the math, point it out to me. I provided you links with p2Pools round history and also to the data I used in handy .csv format. Even using Excel I can't see a quick mean of round length / D taking more than 10 minutes. Did you complete your own analysis? What was the result?

Kano brings up a good point. I'm talking about average earnings since the start of the pool. In fact I already explained this, as you already know from the first quote above.

I've made some changes to the post, along the lines of Meni's comments. Apart from a few small changes in the main text, I changed the first points of the conclusion to:


Good idea. As soon as I have enough data, in the Weekly pool statistics (https://bitcointalk.org/index.php?topic=77000.0) thread I'm adding charts showing for each for which pool I can get orphan data. I started to include % orphans for each pool in the weekly stats table in the last post.


Finally, Miner Earnings Insurance for DeepBit (https://bitcointalk.org/index.php?topic=87990.0) is here! If you don't mine on DeepBit give it a read anyway. I will offer the same service to other smaller pools if there are miners who want it.

maybe many p2pool miners have the problem with "LOST CONTACT WITH BITCOIND FOR 1.3min"
^^ i think this could explain longer round lengths

is there any solution for this problem? will bitcoin0.7 help? does bitcoin0.7 even work with p2pool?

I'm not sure how this would lead to longer round lengths - as far as I know, the equivalent D 1 shares are calculated using the pool hashrate, which is in turn calculated using the submitted variable difficulty shares.

I'm probably wrong though - can you explain the above in a bit more detail?

As far as solutions to the "LOST CONTACT WITH BITCOIND" problem goes, I'll leave that to the p2Pool community - I really don't know, sorry.

i thought you meant "time" by "round length".
but if this refers to submitted shares i just dont know.

We really have to standardise notation, eh? Yes, I was referring to total round shares.

For anyone who doesn't follow, if round duration was increased this would lead to a lower pool hashrate overall, and reduced earnings for miners - but not increased numbers of shares per round.

It is 0.6.2 issue, load 0.6.99 and it is gone.

0.6.3 has been released, I assume it also contains the fix.

I know that statistics can be skewed to promote a particular view. As a matter of fact, statistics classes teach about representing perspectives with results, in the way election campaigns do or marketers.
What value is there for statistics when 5th or 6th grade math is sufficient to prove organofcorti's statements are erroneous.
Using statistics to determine profitability for p2pool would be like trying to open a jar of jelly with a sledge hammer. Sure you can do it, but it is way more than is required to accomplish the task.

Then actually prove them wrong...

Nothing wrong with disagreeing with his analysis, but you have to counter it with analysis of your own, that you two can pick apart and justify. This is how a scientific method works. He has his theory, and now it is up to his dissenters, to try and tear it apart to prove it in error.

So far, I haven't seen any comparable analysis from your side yet. So... put up or shut up?

-- Smoov

Smoov, you have exact same thing in your stats:
week:
Local rate reflected in shares: 152
Local rate: 132 (+15%)
Month: 144/130 (+10%)

Looks to me, that "smaller" miners (under 500MH/s, maybe higher too) are "overscored".
Because there is alot of small miners in pool it can affect global pool stats alot.
Pool rate and "expected time" is calculated from share rate.
If share rate vs "real" hash rate vs predicted time vs real time is overcalculated (probably counted from block to block) all stats presented on site are inaccurate.

Maybe only flaw in P2pool luck is some odd underestimation?

There are times where I get no payouts on a block, because I haven't been able to get a share for more than 24 hours, too.

Pointing at my own tiny contribution is useless for statistical comparison. The sample is way too small, and margin for error too great, to make any reasonable analysis based on a random sample (which pointing to one peer, as representative of the whole, isn't random). Additionally, my own stats aren't stable. I do not operate a dedicated mining rig, so my mining is competing with all of the other programs I'm using too. Some have a high diskio, some use the GPU more than others, and I occasionally turn off my BTC mining entirely for periods of time. My stats would not be a good choice for analysis, by themselves. No reliable 'control'.

The original analysis took the entire run of p2pool into account, no sample. Can't get more accurate than a 100% sample, so now, it is about the interpretation and methodology of how the analysis was done.

The dissenting opinion of the analysis already presented, hasn't offered their own analysis to have a debate with. Until that happens, there isn't any point in arguing for or against the original analysis.

Give me facts... leave the spin to the politicians.

-- Smoov

From what I've read of your posts, I don't think that's something you know - it's something you've been told. So I'm encouraging you to learn about statistical analyses. http://oli.cmu.edu/courses/free-open/statistics-course-details/ is good, and assumes no prior stats knowledge on your part.

As far as skewing data goes, I did the same analysis as always. If there was a problem with it, other analyses would have shown the same problem. Also, forum members better at math than I am would certainly have let me know.

I know you're emotional about this, but that doesn't excuse you from accusing me of falsifying data when you don't yet have the knowledge on which to base such a claim. Get some learning, then start showing me where the analysis is wrong.

@ all the p2Pool miners that posted me data:

Thanks for all your help! Unfortunately, the data is insufficient and in some cases measures different things.

As much as I would have liked to have written a follow up from an earnings point of view, there were too few who collected actual numerical data, and I had to make too many guesses with the public node data. I'm just not very confident about the result.

If anyone wants to start collecting data now, I need:

timestamp* |  block number   |   valid shares  submitted for block   |   local average earnings for block  | difficulty for block*

* = preferred data but not as essential as the other data listed.


If I can get at least half the pool hashrate doing this for a two months, I'll have data which will have a much narrower and useful 95% confidence interval.

check_status, I'd like you to be involved. If I show you what to do with the data once collected you might gain more of an insight into probability and statistics, and you can be confident that the results are correct since you will have collected some of the data and performed the analysis yourself as well.

Actually, by Actuaries, one of which is a family member, who passed the first 4 tests without a retest, and does complex math in the head I would have trouble entering into a calculator. Yes it's true my math sk1llz are limited to some simple algebra at best and Boolean equations make me cross-eyed.

I appreciate the learning link, this will be handy in the future.

I speak for and to those who can't do complex math to solve a simple problem. You say emotional I say concerned, but I guess we are each attempting to portray a perspective.

So then, a graph, from beginning to current, showing actual v. expected, would be the only counter point you would except, is that right?
Considering my math skill level that would take a month at least, that includes the youtube videos I will need to watch to learn how to use OoCalc. ;)

Get your actuary family member to have a read the blog post (if they understand pooled bitcoin mining) - I'm sure there will be at least something in there they'll disagree with, and I would like the feed back. I don't know any actuaries. I'd thought they were a myth, something to scare baby statisticians with when they're naughty.

You didn't answer my main question - are you still ok with so forcefully stating that I'm anti-p2Pool and that I have falsified or skewed data? If you suspect I have not been honest in my analysis, then read the blog post. If you come to something you don't follow, post a message here and I'll try to explain it or provide links. It'll be a good learning experience for lots of people who would like to understand the analyses better.

Here's one I provided in the blogpost with a 100 block rolling average (smooths the graph):

https://i.imgur.com/ZgYzO.png (http://imgur.com/ZgYzO)



In case you're wondering what effect the rolling mean might have, here's the original data:

https://i.imgur.com/KvPU5.png (http://imgur.com/KvPU5)



Finally, if you're going to use anything for math software and you know even a little BASIC or python, I'd try R  (http://www.r-project.org).

No... I'm saying, that... when faced with being up against one man's opinion, which he backs up with the data, which he put a lot of work into collecting, and processing it into something meaningful, explaining what he is looking for, what to expect, and what it is actually showing, in his interpretation...

...that you're going to have to do better to counter his opinion than with the equivalent of, "No it's not."

You're going to have to counter his effort, with actual effort of your own, which proves his conclusions in error.

We have yes-it-is, no-it-isn't, oh-yes-it-is, oh-no-it's-not debates all day every day in IRC, and in the forums too.

This isn't one of those debates. He stepped up. Your turn.

-- Smoov

You seemed to be confused. I have posted results and that's not enough for you. The only logical extension that would end the disagreement is a complete actual v. expected comparison, to which you reply 'NO'.  ::)
I have 114 days of data sampled from various periods of time, beginning, middle, middle+ and recent; 4 separate periods. I did post results on this forum 2 here and once in the p2pool thread, none of my samples shows a profit loss by mining at p2pool. Can I reasonably conclude that a statement which says 'p2pool is not profitable' is erroneous?

To get a length of string into 8 pieces, you only need to divide by 7. ;) 1 / 7 = 8 :D

@organofcorti  If round shares are so chaotic why would you use them in your calculations when simpler values are available?

I haven't seen these results - can you repost here?


Which simpler values would you be referring to? I can't see a way to work out if the mean pool round length is significantly above the expected mean or not. (EDIT: without using total round shares)

I'm not sure you can say round shares are chaotic, and I was wrong if I wrote that somewhere. p2Pool round shares/D still appear geometrically distributed, as is clear from the blog post.

Shares are the only accurate way to measure pool performance.
Any other choice is meaningless (including saying 'I earned more than PPS')

For every gigahash the number of shares per day is a constant value, 20,116.5676 shares per day.
1000 megahashes = 20,116.5676 shares per day.
1000 megahashes = 838.190317 shares per hour.

Once discovered, a block becomes a constant value.
Hash rate for a given period needs to be averaged, (2weeks to coincide with difficulty?).

BTC found per 2 weeks / average gigahashes per 2 weeks = (actual value paid) PPGH for that 2 week period
versus
Expected Payout, like that determined using a Bitcoin Calculator.
or
(current block reward) / (current difficulty) = Expected PPS

It appears that complex statistical analysis is NOT necessary to determine p2pools profitability.

Appearances can be deceptive.

You might not realise it, but you are actually proposing quite a "complex statistical analysis" here. It might not be hard to calculate results, but how do you determine the the confidence interval? Stating that the results are "close to expected" is not enough. You need to be able to provide at minimum a confidence interval and show that both theoretical and empirical results lie within it.

I agree that this method would provide a more robust analysis than shares per round/D. The problem with an analysis such as you suggest is that you need to have consistent payment data for a large portion of the pool and for a large portion of the pool's history. I'm not sure that the payment results for one miner - even starting from the history of the pool - would be sufficient to produce results with a suitable confidence interval. I did attempt to do this in the weeks following my post, but even with a good many miners sending me data it wasn't sufficient to rule out either the expected payments or payments at 0.91*expected.

I'd still like you to post your data here so I can have a look at it though.


Edit: Perhaps a better way to explain this is by asking you exactly what you're trying to prove. You can't just prove that your results are close to expected, you have to prove that your results exclude what you're trying to disprove, to a selected confidence level - usually p<0.05 or p<0.01. Start by reading about the null hypothesis (http://en.wikipedia.org/wiki/Null_hypothesis). The link to the online stats & probability course I gave in a previous post devotes a whole section to it.

I hope you realise I'm not being contrary for the sake of it, but simply pointing out that analyses of this type are harder than you  think if you want to be able to come to a provable conclusion.

The number of shares you state are simply the expected number of shares.
Every hour you check, you will see a different value.

Over time with a LARGE sample you would expect the numbers towards converge to the expected values, however, making a statement like "1000 megahashes = 838.190317 shares per hour" is simply naive.

GH/s -> shares is a random function.

No pool can supply total hashes accurately.
Miner programs can not even supply that accurately to the pools that accept it.

However, more importantly, it doesn't matter what the calculation of GH/s -> shares is since that is in no way relevant to the expected performance of a pool.

The performance of a pool is simply to show that over an extended period of time, it will find close to an expected number of blocks given a number of shares supplied by miners.
It is certainly not any issue at all whatsoever to a pool, if a miner supplies less shares per MH/s than the miner expects.
In fact no pool would even give a damn about it.

... and to give a very specific example ... I have a faulty BFL that seems to only find shares with one of the two FPGAs in it.
I send the work to it and it responds in the expected amount of time (~5.1s) every time, however it finds half the expected number of shares.
Why would my pool give a damn about that?

In fact EVERY pool would estimate my hash rate as ~half what my BFL is saying it is doing ... as EVERY pool should.

Hash rate should NEVER be used to determine pool performance since there is no way to even gauge it accurately.
The hash rates shown for p2pool are simply the share rates converted to a hash rate (and who knows if that has even been done correctly or not ...)

Actually, it's the other way around I think - hashrates determine the shares per round. But otherwise I agree. I would prefer to have actual round length data, but that's not possible unfortunately and could be a source of error.

Diff1 share = 2^48/65535 hashes = 4`295`032`833 hashes
If you have 4,295GH/s you get 1 diff1 share every 1 second

1. How p2pool calculate miner hash rate?
- it counts diff1 shares over time diff1 share/sec is 4,3GH so 1 share ever 10 second is 430Mh/s
2. How p2pool calculate network hash rate?
- it counts last shares in share chain over time and multiply it by current chain share diff (maybe every share, not sure)

IF we have slight over calculation there (like I rounded 4.295 to 4.3) pool hash rate can be inaccurate reported and p2pool.info block ETA can be wrong!

That's right. However, it would only be out by the rounding error - in your example -0.005/4.3 = - 0.11%. However, the mean round shares are +10% compared to expected - that's a big rounding error. There could be some other miscalculation though.

And to wake up the thread before it gets old ... :)
So ... if someone was to get some BFL MiniRigs ... or some time in the far distant future ... a bunch of ASIC ...
Wouldn't it be best for them to mine on a PPS pool and withhold blocks ... so they don't increase the difficulty and thus get more BTC ...

Evil. PPS is so doomed...

Has it been adressed, that p2pool includes lots of transactions in the block genrating a bigger size of block.

The p2pool nodes are often/almost anytime behind home broadband. even with 1 mbit upstream a 130 kb block would take 8 seconds to propagate fully... could this explain the higher orphan rate?

Thats why info about new block is spread between p2pool nodes :)

8secs would account for slightly over 1% orphans, if it's even that high.

it hasnt gotten an orphan since i started using it & set up my server (5.9.24.81).. about 3 weeks?

Orphan rates for all pools are published in the Weekly Pool Stats thread. Orphans have been very low and round lengths shorter for p2Pool for a few weeks (I think).

I hope that the luck increase is real and not just an artefact of the round length hashrate calculation method - everyone seeing an increase in earnings?

Sorry I missed this Kano. It's an interesting point. The block witholding attack has been mentioned before, but not in the context of someone with a large hashrate not wanting to affect difficulty. The miner would have to have a large proportion of the network hashrate to affect difficulty by himself, plus he'd have to hide the fact by using multiple miners on multiple pools.

I think in this circumstance you everyone but the network would notice a problem. Say the attacker has 5% of the network hashrate and doesn't want to affect difficulty. He has to spread the hashrate over several workers (or one worker would look suspiciously unlucky) and at several pools (or one pool would look suspiciously unlucky). Then all the PPS pools will start looking slightly unlucky.

If you found that round lengths were larger than normal on average but the luck index (calculated as log(Percentage of network blocks solved / Percentage of network hashrate) )  was averaging at zero, you'd start to suspect this could be happening.

Luckily I have some charts somewhere that might help ;)

4.2 Slush's score method and miner earnings part 1 is posted. This will be a three part series explaining the effects of the exponentially scored method on fulltime miner earnings.


http://3.bp.blogspot.com/-jHaj3W_6VNU/UCZDwdfpycI/AAAAAAAACMg/IcsLJUdJp90/s1600/npw4.2.plot.7.png

6.1 Bitlc.net - Pool hopping and unfulfilled promise (http://organofcorti.blogspot.com.au/2012/08/61-bitlcnet-pool-hopping-and.html) reports on a pool that started well, stagnated and then become one of the last few prop pools. What went wrong. And more to the point, who still mines there?


http://1.bp.blogspot.com/-EFFxt4t5k3k/UDjKCK7ioAI/AAAAAAAACUA/u-zTNkMQQ6s/s1600/npw6.chart10.png

Very much in agreement regarding bitlc. That's been one of the pools my proxy has hopped for quite a long time now - the unfortunate fact of the matter is that whilst it's profitable to do we'll keep doing it.

However, how did you come to your conclusion that only 6.7% of deepbit's hashrate pre-hop is pool hoppers? I was under the impression that ~500Ghash was hopping that pool, just like with Slush.

It's an estimate, using the median average hashrate per round after 2*D as the fulltime miner hashrate, and  the median average hashrate per round after 2*D as (Fulltime + hopper) hashrate. It's close to results I've obtained in the past using more accurate estimations. I've noticed that the hop hashrate at Deepbit has been much less than when I posted NPW2.2.

Interesting... There has been a bit of a ban wave on there lately, but mostly for larger hopping accounts (>5Gh/s or so). They don't seem to be IP banning yet.

I've noticed the hop rate dropped since mid April, so whatever is being done has been done since then.

Edit: Thank you to my most recent anonymous donor.

I don't know if this pool is of interest because of its small hash rate: https://miner.ecki.net:444/

But there seems to be serious evidence that the OP is cheating.
After I found this post
https://bitcointalk.org/index.php?topic=78391.msg874690#msg874690

I made some investigations, i.e. clicked thru the "found blocks" page on the pool webpage and entered the block numbers on blockchain.info

What I found is this:

BTC 187,500 Distribution marrog 2012-07-04 20:43:02 537,341
http://blockchain.info/block-index/242747/00000000000000461dcf498b2ba02f19dde9a98bb126de9d8e313b0bcdba5bfb
=> P2Pool


BTC 188,332 Distribution glordglord 2012-07-10 04:10:01 966,959
http://blockchain.info/block-index/244059/000000000000094b944cc807129a2e270753a565a3a8b37f609dc1e80a631e4c
=> P2Pool


BTC 188,936 Distribution Mycom 2012-07-14 05:23:02 1,715,598
http://blockchain.info/block-index/245155/00000000000001e4cee1f8f939b18d5248f367397ec5ec7ff8cf240cfb669ef8
=> OzCoin


BTC 195,994 Distribution marrog 2012-08-28 05:32:05 7,313,184
http://blockchain.info/block-index/269508/00000000000000e03ed033296e37cd676536ecbfdc600059dba12b25e2ca2fe3
=> BTC Guild


So it's not the classic "faking round lengths" but the whole blocks are "faked".

My question is: How reliable are the markers P2Pool, OzCoin and BTC Guild use? Is this strong enough evidence?

I'm asking because I think you have way more experience in exposing cheating pool OPs.

not long after Ozcoin started I had a furious person join our IRC blaming us for "stealing" another pools block
our software reported the wrong block number, we had the previous one.....

since January all Ozcoin blocks have eco@ozco.in in the coinbase
easily seen here http://blockchain.info/tx/cc8e9de906601a9f1537d83d0c86f48f76853cdae835d18141e1ca3186d9e62b/cc8e9de906601a9f1537d83d0c86f48f76853cdae835d18141e1ca3186d9e62b for example

So it could also be a bug in the pool software?

Interesting, but how would claiming blocks not theirs benefit the pool op? It's a PPLNS pool so he would have to pay extra to his miners. I can't quite see where the gain for the pool op is.

I don't know how reliable the block claim markers are, and I'm not sure how accurate blockchain.info is.

would be my 1st guess :)
I do see what you mean though, checked a couple of their blocks (including the Ozcoin one)

@Graet you were right. Block numbers are off. Blocknumber+1 is the correct one. So just a bug in the pool software :)

good news :)

Which ones are out by one - ecki.net - or the other pools?

ecki.net or course ...
As mentioned above you simply look at the coin base to identify a lot of pools.
In fact around 50% of blocks can be identified directly by looking at the coinbase transaction - either they say the pool name or they are e.g. like p2pool you can identify it by the number of payouts and some of the addresses in there.

When I'm at work I have to deal with a locked down computer and internet - it's just easier to get other people to confirm things for me.

New post : Neighbourhood Pool Watch 4.3 Slush's score method and miner earnings part 2 (http://organofcorti.blogspot.com.au/2012/09/43-slushs-score-method-and-miner.html)

In this post I derive the actual current loss of earnings for fulltime miners at Slush's pool.


http://4.bp.blogspot.com/-NWf3bs_2eFI/UEipmKW7csI/AAAAAAAACYQ/mGk4v1pdeXQ/s1600/npw4.3.plot.22012-09-06.png

Thanks for the most recent topic as it was one I was waiting to read about.

In this post (http://http//organofcorti.blogspot.com/2012/10/71-variable-pool-difficulty.html) I explain how to select either a particular number of submitted shares per minute or a particular pool difficulty to result in an acceptable variation in average hashrate.



http://3.bp.blogspot.com/-RwDc060-q6s/UHVXBo-cxLI/AAAAAAAACkk/I9i534G3v4c/s1600/npw7.1.chart1.png

You accidentally a : in the link.

Oops! Fixed. Thanks for the sharp eye, Tittiez.

In this installment of Neighborhood pool watch (http://organofcorti.blogspot.com/2012/10/81-bitcoinpool-another-bitclockers.html) I present part one of a three part series on BitcoinPool.

From the post:


http://2.bp.blogspot.com/-Edc_CK6xBOY/UI-TZtP69UI/AAAAAAAACtM/mvzhD3nePzs/s1600/npw8.1.chart3.png

Another non-pool post, NPW 9.1 ASIC choices (http://organofcorti.blogspot.com.au/2012/11/91-asic-choices.html) is written to help miners determine which ASIC device will be best for them.


http://1.bp.blogspot.com/-TzqkUhJuYGU/UJJtRlFDzTI/AAAAAAAACy8/omJ6EIi5rZk/s1600/npw9.1.chart3a.png

Do you work for BSL? It is unknown how many watts the bASIC will use, but it won't use 300 watts. There have been hints in Tom's thread that it will be around 100 watts.

No, if you had actually bothered read the post you'll note I explicitly write that I don't work for BFL.  If you can link to a better estimate on power usage then please do.

Edit: I also explain how to do the calculations yourself and I've encouraged readers to do it for themselves. If you're not simply trolling, why not do your own calculations with your own estimates of power use? Further, I did note in the post that I will update the post as and when new and updated information becomes available.

tl;dr: Please read the linked post before commenting.

I don't think its officially announced - so much spam and offtopic in bASIC thread I cant sift through it all.
but Tom is selling 1000W PSUs to run "10" bASIC from - broadly suggesting that they will draw under 100W each

great work on the write up, and yep, people with updated info should plug it in rather than throw accusations :0
cheers
graet

Glad I'm not the only one to give up trying to follow that thread. I think I'll wait until I get a clear indication from manufacturers of the power use specs before I write an updated post.

But for readers of this thread who did not read the post, the whole point of the post was not just to compare products (although I did that in the conclusion). It was also to explain how a reader can do the same estimations I did. It's much easier to understand the results if you minimise complications and assumptions.

Give me some time, I just had lunch. Tom posted some photos of his new place and on one of them the caption read "bASIC's will do 54Gh/s at under 100watts". I saw it myself but it has since been removed. I take it as an unofficial estimate.


In addition to that, the bASIC can be powered by a molex plug and someone stated that the maximum wattage a molex plug can supply is somewhere around 120-130 watts (I can't remember the exact number). Hope that helps.

Thanks for that. I'll update the post (with your maximum figure, 130 W - I prefer to use a conservative figure) during the next seven days assuming no other estimates come to light in that time. An official estimate - or one guaranteed not to change for at least a month - would be nice though.

If anyone else has any power usage, hashrate, and cost estimates for the other ASIC devices for which I couldn't determine estimates, I'd be grateful if you'd post them here with a link.

Aren't you forgetting that the BFL SC and bASIC are going to need a PC of some sort, which is going to add at least another 100 watts?

The avalon is a complete standalone unit.

1) It is dead simple to build a computer that uses less than 100W. A normal Ivy Bridge desktop can consume 60W at idle which is about the load that mining puts on the system. Small linux systems could mine using single digits watts.
2) There's no requirement that you can run only one unit per computer. Several Singles or bASIC units could be run per computer, or even a couple minirigs.

Isn't it a bit too early to evaluate ASIC parameters ? :)

No, I didn't forget that. The blog post does state my assumptions most clearly.


I didn't know that Avalon was stand alone - and that will make a difference. Thanks for letting me know. Do you have a link to confirm this?

What, because no device has even shipped yet? Yes, it's definitely too early to come to a final decision on which is best.

But the post does address how to assess them so when there are shipped products. The biggest problem I'd noted was a lack of a simple, systematic and easy to follow method of assessment, and the post addresses that.

So, any word on your product lines' specs yet?

What I mean is that no one knows their exact power requirements yet, which is very important.
No, I don't know exact power figures on our product too, yet.

Yes, I knew what you meant, and I agree it's very important. I hope readers consider the blog post more as a primer on how to determine which ASIC will be best for their mining environment.

I've written an updated post with the suggested specifications:

http://organofcorti.blogspot.com/2012/11/92-asic-choices-update-2nd-november.html

I've attempted to stick to consensus, so I wont update again until there's new published information about device specs or I've made another transcription error.

http://3.bp.blogspot.com/-U3Ftfyzmjpo/UJOqB9fbkNI/AAAAAAAAC8o/KGoLRyHDK44/s1600/npw9.1.chart3b+2012-11-02+.png

Hmm ... I think it needs a big red watermark across it saying ... these numbers are fictional - don't take any notice of it.

Hmmm ... I think kano didn't read the blog post:

Yep I saw the pretty picture and that was more than enough info ... until I saw the numbers ... that I know a bit about :D

Alrighty then - if you're not talking about the hashrate / price / power consumption figures (which are used as examples to explain how others can make these calculations for themselves) what numbers are you talking about? If I've derived something incorrectly I'd like to know. Stop being so mysterious.

I heard you like pretty charts, so this one is dedicated to you ;)


http://4.bp.blogspot.com/-BDCNFtGwYnw/UJkcWy_WCxI/AAAAAAAADIA/Nrdp1JwFTPs/s1600/npw9.3.chart.avalon.rate+2012-11-07+.png



More charts and more detail at http://organofcorti.blogspot.com.au/2012/11/93-more-on-asic-choices.html

Awww pretty :)

BFL=65nm was announced today - so the BFL figures before sound like they should be reliable at least ...

Aha! It was the device specs you were querying, not the results. Glad we cleared that up. Also glad you liked the pretty Kanoplot.

It's always been advertised as such

http://store.avalon-asic.com/

http://forum.bitsyn.com/viewtopic.php?id=5

OK, so the thing here is to hope that none of these ppl making these ASICs mine anything for themselves, and you're one of the few that gets them in the "first week" of availability?

otherwise, why buy one?  wouldnt it be wiser to just use that money to buy bitcoins instead?   if they retain their value, you're out nothing.  if they retain their value but difficulty goes up with asic, you're waiting some years?   

wouldn't seem very logical for me that if bitcoins were to increase in value (making mining more profitable), that ASICs would also increase in value.  added competition over time should result in reduced costs

/me looks into his crystal ball
 ... future prediction! :P ASIC price per MH/s will drop over time.

Also, look at the pretty pictures ... they do actually help you find when you will make a profit ...

and yes you do have the right idea :) Don't buy ASIC - so everyone who has ASIC gets paid more due to the lower difficulty :)

and lastly, at the gold fields, the shovel makers get all the gold :D

Looks like it need update, or another chapter :)
http://p2pool.info/luck/
All-time luck is back to 100%, bugs that possibly cause bad luck/orphans are fixed IMO.
Long live P2pool! :D

I've been meaning to do follow ups for all posts, I just haven't had the time. Rest assured, I'll get on to it at some point. It would be interesting to see if there was some point where there's an obvious change in mean/orphans and relate it to a particular patch.

Got a (somewhat self-serving) suggestion for a "pool" watch subject :D

Transactions per block, kb per block, BTC fees per block, BTC fees per txn (and of course network % blocks per whatever range)

With all this crap going on about what is good for BTC, clearly this will show what is (at least IMO) good for BTC.
IMO good for BTC is blocks that confirm transactions - since no txn's means no one is using BTC which means bye-bye BTC :P

I'd go as far as suggesting that pools that are near the top of this report (except for network %) should get the most miners and others with much lower stats should get fewer/less - but of course that is just IMO

Anyway, it is mostly all there in the blockchain - the only problems are the issues of determining who owns half the blocks
(~50% of them either name who they are or are obviously p2pool)
The rest I guess you have to trust the pools to say which are their blocks (e.g. deepbit)

I would also suspect that many of the unknowns would be at the bottom of the ranking ...

Something like I did here (https://bitcointalk.org/index.php?topic=88302.msg984376#msg984376)? Yes, I've wanted to do a more general statistical analysis of blockchain variables, however getting the data is problematic. I can't seem to get any blockchain browser to compile on my macs. Do you have an alternative source?

No
I just write all my own php to talk to bitcoind
But to identify ~50% of blocks you just look in the coinbase transaction:

The strings I used to look for were:
You can see how old the info is :)

As for the other 50% - the problem there is you have to go to the pools (like deepbit) to get the info.

care to share? :)

Ah well. If you do manage to get the data together let me know and I'll help you with analysis.

Since rav3n_pl asked so nicely, this post is an update to the original p2Pool post.

It answers some questions miners were asking and isn't able to answer other questions they weren't asking.


Enjoy!


http://3.bp.blogspot.com/-YP3wwLozBbc/UKthB9kotVI/AAAAAAAADnQ/GAEf22THY5c/s1600/npw5.2chart4.2012-11-20.png

I'm normally not the kind of person to go "I told you so"-ing all over the place, however:






Any chance of an apology, check_status? :)

I finally got around to finishing part 2 of the BitcoinPool series: Neighbourhood Pool Watch 8.2 BitcoinPool: The great anomaly hunt. (http://organofcorti.blogspot.com/2013/01/82-bitcoinpool-great-anomaly-hunt.html)

Lots of weirdness happening at this pool, and it seems that the BitcoinPool miners are suffering:

20% fee pool last 18 moths? WOW!

deepbit version 2.0?

I thought this post would get a bit more traction to be honest. There was quite an uproar when I found that Bitclockers.com was either underpaying their miners by ~21% or incorrectly reporting their stats to avoid paying pool hoppers. BitcoinPool's data shows that miners there have definitely been paid  ~ 80% PPS over the last 18 months - of that there is no doubt. It's also extremely unlikely that it's just bad luck - the data isn't even distributed as it should be, aside from the higher mean shares per round /Difficulty.

Maybe it's just that in a low hashrate pool there aren't many miners, and maybe they don't read this board.

The how and why of the glaring anomaly I don't know, and I don't think the pool ops are ripping their miners off - the data doesn't seem to support that.  I've made the anomalies I found as clear as I could and I've provided the dataset so hopefully someone a bit more cunning than I can figure out what the problem is.

Well, it's not really a pool fee - BitcoinPool only take a donation. But on average, miners there have been paid 20% less than PPS since July 2011.

New post:

Neighbourhood Pool Watch 12.1 The pre-ASIC network hashrate distribution. (http://organofcorti.blogspot.com/2013/04/121-pre-asic-network-hashrate.html)



http://3.bp.blogspot.com/-XZe7sFinWVQ/UWQaH6l6MHI/AAAAAAAAHUk/pRD0MxvRtB4/s1600/npw12.1.violinplot.2013-04-09.png (http://organofcorti.blogspot.com/2013/04/121-pre-asic-network-hashrate.html)

New post:

Neighbourhood Pool Watch 12.2 Pool and network miner hashrate distributions. (http://organofcorti.blogspot.com/2013/05/122-pool-and-network-miner-hashrate_3.html)

New post:

Neighbourhood Pool Watch 13.1: BitMinter and "luck". (http://organofcorti.blogspot.com/2013/05/131-bitminter-and-luck.html)




http://3.bp.blogspot.com/-moeeJmunoy8/UYtq3rhmlUI/AAAAAAAAH2w/p-e3sweWOdA/s1600/npw12.1chart5.2013-05-09.png (http://organofcorti.blogspot.com/2013/05/131-bitminter-and-luck.htm)

Meh - I always shudder when people use numbers other than shares/difficulty
Anything else is effectively either derived from that (usually with added inaccuracies) or estimated (and even more inaccurate).
I was wondering about that in the BitMinter thread for a while ... until you chimed in with the results :)

Shares/difficulty is definitely my preferred metric, and the one with which I have the most experience. But there is a another metric not directly derived from shares/difficulty, and that is payments per share (for PPLNS pools). Payments per share is Poisson distributed with a mean of N, so it's just as amenable to analysis as shares/difficulty.

But I get what you mean - there are a bunch of ways people have tried to figure out luck, most of them assuming that the variables are normally distributed which they are emphatically not.

Figuring out probabilities is not hard once you know what you're doing. I'll write a post explaining how to do it using Wolfram Alpha if I get a chance - then no one has to listen to "Argh my pool paid out 10% less than normal over the last month!", since they can figure out the truth of the statement and if true how probable it is. Then it's just a matter of getting used to interpreting the results.



.... and now it's gone all quiet over there ;)

New post, and a new way to visualise changes in the user hashrate distribution. There's and excerpt below, but there's lots more at http://organofcorti.blogspot.com/2013/05/124-miner-hashrate-distributions-28th.html


http://1.bp.blogspot.com/-3wOD6itS8B8/UaTAgGnpk2I/AAAAAAAAIKQ/psa6aCOfCL4/s1600/npw12.4.chart1a-2013-05-28.png (http://organofcorti.blogspot.com/2013/05/124-miner-hashrate-distributions-28th.html)
http://3.bp.blogspot.com/-sRcAAeoOwD4/UaTA5fiWVWI/AAAAAAAAIKY/De-Vx0QNUe8/s1600/npw12.4.chart1b-2013-05-28.png (http://organofcorti.blogspot.com/2013/05/124-miner-hashrate-distributions-28th.html)

New post: Neighbourhood Pool Watch 16.1 The network: Orphaned blocks part 1

http://organofcorti.blogspot.com/2013/10/161-network-orphaned-blocks-part-1.html

It's just part 1, I hope to get the rest out soon.


http://1.bp.blogspot.com/-Og-IY9WS9kc/UmUyY0wAwoI/AAAAAAAAKHA/xCc_sQQE_G4/s1600/plot0.2013-10-22.png

New post: 16.2 A short post on mapping the historical pool hashrate (http://organofcorti.blogspot.com/2013/11/162-short-post-on-mapping-historical.html)

Except below:

Very good quality posts on this thread, keep it up.

Sorry I can't give you the May 2011 through ~October 2011 speeds for BTC Guild.  All that information was wiped when the pool switched from free proportional to PPS :(.

One request/suggestion:  Perhaps a graph which only shows a smaller subset of pools?  It's virtually impossible to distinguish them when their colors are so similar.

I thought this would be a problem, so the charts don't show hashrate - just blocks per week and then blocks per week as a percentage of total of network blocks per week. Do you have block history going back before that?


 chart doesn't show hashrate


I did try that, but then you lose a feel for how much of the network the pools represent, and how many smaller pools there are, when they started and when they died etc.

What I'd like to do is learn how to make it interactive. I'd like to be able to mouse over the chart, the whole band corresponding to the colour over which you've moused gets highlighted and the pool name appears. I think this could be done using JS (which I've never used) and some JS charting tools (that I've never used), so there's a big learning curve ahead of me.


Cheers!

Unfortunately, BTC Guild history pre-October 2011 is pretty much lost :(.  It was only about 5 months old, and I was still learning as I went along back then.  I didn't quite realize importance of keeping all that just for historical accuracy back then, after only being involved in BTC for half a year, paired with the long sustained decline in value after the 2011 bubble popped.  We didn't start signing the coinbase until PoolServerJ was released.

When was PoolServerJ released? The early data I have is from coinbase signatures only - do you have much data before that? (I can't load up the dataset atm, and I can't tell from the chart - there might be a month I don't have).

New post:

17 Goodbye HHTT. The coinbase will never be as strange again. (http://organofcorti.blogspot.com/2014/01/17-goodbye-hhtt-coinbase-will-never-be.html)

Here's the whole thing in full (although with most of the links missing)

Is this addressed to me, fireduck, or are you proposing your own coinbase message? I'll admit you have me stumped, since I thought we were chums.

I figured I'd just misunderstood your meaning. I should have put a ;) in there somewhere.

So, my two most favourite firebrands in a duel ;) Or something more serious?

Can you post a link?

Here's a treat for all you tinfoil hat wearers worried about the "unknown" hashrate owners:

16.3 The unknown network hashrate. (http://organofcorti.blogspot.com/2014/01/163-unknown-network-hashrate.html)

If you just bought a Gridseed and were wondering whether or not it might be more profitable to mine bitcoin, then this post is for you.

http://organofcorti.blogspot.com/2014/03/918-gridseed-should-you-mine-bitcoin-or.html


http://3.bp.blogspot.com/-KC2WJBlXrbE/UyRJnTFkbCI/AAAAAAAALq0/wGzamJZyJu4/s1600/gridseed2.png (http://organofcorti.blogspot.com/2014/03/918-gridseed-should-you-mine-bitcoin-or.html)

Neighbourhood Pool Watch 18.1 Who owns 1HTM4TYSXF5yZKLpco6MTUUNfSBCiiwGsU ? (http://organofcorti.blogspot.com/2014/03/181-who-owns-1htm4tysxf5yzklpco6mtuunfs.html)


http://2.bp.blogspot.com/-ZPyZmxyuG2E/UyWWotoldFI/AAAAAAAALsA/5lIM3O715-s/s1600/followGen1.png (http://organofcorti.blogspot.com/2014/03/181-who-owns-1htm4tysxf5yzklpco6mtuunfs.html)

OoC,  have you managed to determine which, if any of these "unknown" addresses, are Bitmain's? They claimed that their products were/are powering 20% of the network.

No, I have no idea who owns the addresses - but if you have some addresses you think they might be I can try to find out the shortest distance between them.

FTFY

You can (only just) mine BTC at a profit with these things, even at 35c per kWh. That should hold true for a few months yet.

I'm not assessing the net profit (after purchase) though, which might be what you're thinking. I did say " I decided to throw caution to the wind, ignore the fact that I may never get a return on my investment, and purchase a batch of ten"

FTFY

Edit: https://bitcointalk.org/index.php?topic=456691.msg5281847#msg5281847 :)

FTFY