Fascinating what Michael Deletes [BTCguild 90% luck and dropping]

[Entropy-uc]

I thought these were pretty neutral points but apparently Michael has something to hide.

DPoS, I appreciate that you are genuinely concerned about this, but variance happens.

If you have any evidence beyond the de facto luck of the pool, I'd personally be delighted to see it, but claiming to be the only guy who can see the conspiracy while the math nerds politely nod and smile...

I say this without malice, but it reminds me of the scores of (to a person, bad) poker players I've met over the years who talk about how dealers screw them, how winners are just lucky, how they usually win when they sit in their lucky seat, how they never win with pocket aces, and a gillion other things that all come down to the reality that variance happens.

They don't understand it, and so the game must be rigged.

It's funny you mention poker.  I remember a friendly game once where I never seemed to get hands breaking the way you would expect from the odds.  So when the deal came around to me, I counted the deck.  There were only 46 cards in it!

Sometimes bad luck is bad luck.  But it never hurts to check.

Michael rakes ~$250k per month from this little game.  And his participants have been losing out on over $1.5M / month for the last 3+ months.  That calls for a LOT more diligence than a little hand waving about variance.  Bitcoin is a dirty business, with cheats and thieves always looking for a weakness to exploit.

I can think of half a dozen ways Michael could be testing for exploits that would explain the pools recent performance.  But he seems more interesting in collecting his rake and insulting anyone who asks reasonable questions.

It reminds me of how Deepbit disappeared from the pools list.  Tycho was too busy with hookers and blow to respond to his users, and in a year he went from the biggest pool, to non-existence.

[/quote]

It reminds me of how Deepbit disappeared from the pools list.

It's no wonder [Tycho], and others, have disappeared from these forums.  Let's accuse everyone else, with NO evidence, of malfeasance as well so that we have no responsible people left in the community.

Great idea.

No evidence of malfeasance?  Why use a $3 word when a 5 cent one will work?

Allowing your pool to leak $1.5M / month and disrespecting people who ask reasonable questions about the possibility that some actors could be cheating other participants in the pool isn't exactly malfeasance - it's simply a lack of diligence.

And I've seen no trend of losing responsible people in Bitcoin.   Tycho made his fortune and moved on, because he didn't care any more.  Michael seems quite happy taking in $250k per month and not addressing this issue.  If I were in his shoes I would be madly coding performance tests until I was 100% certain my pool wasn't being abused.  That would be the responsible thing to do.

[/quote]

[DPoS]

So it goes..   I tried to be the canary in the coal mine and then let the data over time speak for me but seems a few more months of this nonsense will have to pass for the sheep on that pool to get nervous

[dropt]

So it goes..   I tried to be the canary in the coal mine and then let the data over time speak for me but seems a few more months of this nonsense will have to pass for the sheep on that pool to get nervous

I was disappointed to see Entropy-uc quoted with the original and relevant posts deleted because I appreciate and respect his opinion, so I'm glad he's made this thread.

I've been watching and biting my nails, I saw the beginning of the poor streak coming so I switched everything to PPS and lo-and-behold, next day PPS is gone and all my equipment was forced back to PPLNS.  BTCGuild has been very good to me for the last two years so I've held on hoping, but I'm getting the itchy feeling it's time to move on.  I've never seen the luck this bad, especially not in terms of the 3M indicator.  

I don't believe that Eleuthria is acting maliciously, but I agree that something doesn't feel right.  There was a very definitive drop in the average, and block count has been absolutely atrocious since.  Believe me, you're not the only one that's noticed and been watching closer than usual.

[-ck]

If a pool is large enough, then one can mine on it even with PPLNS and execute a block withhold attack and take the small pay hit it amounts to for the sake of preventing network diff from rising (the risk is far greater with PPS). This is the only realistic attack that could be taking place on this large a pool. Pool ops are obliged to audit their larger miners. Even auditing miners cannot protect you from this though since all they need to do is spread their mining between many accounts or just keep creating accounts to never have a significant amount of shares to raise suspicion. [NOTE: I am not saying there is a problem with btcguild]. The only way to audit this is to actually send suspicious miners work that is guaranteed to be solved and see what they do.

It's the vexing question of what miners should do about pooled mining which has for more to be concerned about than this one issue that is currently troubling btcguild members.

Mining doesn't lend itself to solo mining unless you have mega hashpower, but the larger a pool is the more risk you expose yourself and the bitcoin network to. P2pool is the obvious next step, but if that gets large enough then it's not much different to solo mining at lower diff. Clusters of smaller p2pools that feed into larger p2pool like set ups are the next obvious step.

However the number of miners is diminishing, and the amount of hashpower with each mining group (like KnC's farms) gets larger by the minute, and they'll probably all just run their own solo operations with time. Pooled mining as we currently know it may end up just being an interesting blip on the history of bitcoin mining.

[Entropy-uc]

I would think you could detect someone performing a block withholding attack on the pool through a simple test:

First identify your 'unluckiest' participants.  Then when a block is solved, immediately send the work unit for the block to those miners.  If they consistently fail to return the winning solution you know they are ripping you off.

There are a number of other attacks that could be detected with similar approaches.

What's troubling to me is the massive asymmetry of risks right now.  The miners on BTCguild have over $50M in hardware investments that is depreciating at 25% / month .  I'd be shocked if Micheal has $50k in total in hardware to run the pool.

My personal theory has been mentioned already:

<Tin foil hat on>
Startup 42e1 spends $2M on design and masks to build out their $5M 2Ph/s mine only to discover that the design has a flaw that prevents it from finding nonces with difficulties over 4.2 billion.  It still hashes lots of results below that level, so instead of scrapping things they move all their machines off the internal pool to a big public pool.
<tin foil hat off>

There are many, many other reasons someone could be maliciously attacking a pool, and the stakes keep getting larger.  If you're operating a public pool and not actively developing defenses, you will be burned.

And personally if I was clearing $250k / month off my pool, I wouldn't care what little people think either, but I would pay somebody to care.

[-ck]

What you describe is not dissimilar to what I just said. However:

First identify your 'unluckiest' participants.  

Step one biggest problem already. Your miners need to have had more than 1 diff's change worth of shares to even begin calling them unlucky. So they need to submit 8 billion diff1 equivalent shares. The miners would need to be hundreds of terrahash before you even accumulate any significant data to even make this call, and then if they split their mining into multiple workers, you will never get 8billion shares like I said.

[eleuthria]

Yes, fascinating that I deleted the posts that contributed nothing constructive, and decided to imply that I simply don't care, or am too lazy to do anything, and also include a dollar figure for income that was pulled directly out of your ass as well.

What you describe is not dissimilar to what I just said. However:

First identify your 'unluckiest' participants. 

Step one biggest problem already. Your miners need to have had more than 1 diff's change worth of shares to even begin calling them unlucky. So they need to submit 8 billion diff1 equivalent shares. The miners would need to be hundreds of terrahash before you even accumulate any significant data to even make this call, and then if they split their mining into multiple workers, you will never get 8billion shares like I said.

This.  Additionally, you can't send a "prove you're not evil" work packet to miners in stratum.  Every miner has a unique ExtraNonce1, so they will never overlap their hash results.

[Entropy-uc]

What you describe is not dissimilar to what I just said. However:

First identify your 'unluckiest' participants.  

Step one biggest problem already. Your miners need to have had more than 1 diff's change worth of shares to even begin calling them unlucky. So they need to submit 8 billion diff1 equivalent shares. The miners would need to be hundreds of terrahash before you even accumulate any significant data to even make this call, and then if they split their mining into multiple workers, you will never get 8billion shares like I said.

This is true.  But wouldn't hundreds of users coming from the same IP block raise it's own set of flags?  I would expect that existing DDOS defenses would ban somebody trying that approach right away.

A pool might have a few IPs like that due to hosting providers but a few phone calls would validate those.  

[Entropy-uc]

Yes, fascinating that I deleted the posts that contributed nothing constructive, and decided to imply that I simply don't care, or am too lazy to do anything, and also include a dollar figure for income that was pulled directly out of your ass as well.

What you describe is not dissimilar to what I just said. However:

First identify your 'unluckiest' participants. 

Step one biggest problem already. Your miners need to have had more than 1 diff's change worth of shares to even begin calling them unlucky. So they need to submit 8 billion diff1 equivalent shares. The miners would need to be hundreds of terrahash before you even accumulate any significant data to even make this call, and then if they split their mining into multiple workers, you will never get 8billion shares like I said.

This.  Additionally, you can't send a "prove you're not evil" work packet to miners in stratum.  Every miner has a unique ExtraNonce1, so they will never overlap their hash results.

So what have you done to verify the pool isn't being exploited?

[Entropy-uc]

Yes, fascinating that I deleted the posts that contributed nothing constructive, and decided to imply that I simply don't care, or am too lazy to do anything, and also include a dollar figure for income that was pulled directly out of your ass as well.

What you describe is not dissimilar to what I just said. However:

First identify your 'unluckiest' participants.  

Step one biggest problem already. Your miners need to have had more than 1 diff's change worth of shares to even begin calling them unlucky. So they need to submit 8 billion diff1 equivalent shares. The miners would need to be hundreds of terrahash before you even accumulate any significant data to even make this call, and then if they split their mining into multiple workers, you will never get 8billion shares like I said.

This.  Additionally, you can't send a "prove you're not evil" work packet to miners in stratum.  Every miner has a unique ExtraNonce1, so they will never overlap their hash results.

So what have you done to verify the pool isn't being exploited?

Doing nothing is pretty much the definition of not caring wouldn't you say?

On a more serious note, if you are unable to actively detect miners performing a block withholding attack on the pool, then Con is right, the era of public pools is over.  Unscrupulous commercial miners will destroy the public pools to eliminate competition from small players.

[os2sam]

Unscrupulous commercial miners will destroy the public pools to eliminate competition from small players.

That would be counter productive as it would devalue bitcoin and destroy confidence in it.

[fs2k155]

Unscrupulous commercial miners will destroy the public pools to eliminate competition from small players.

That would be counter productive as it would devalue bitcoin and destroy confidence in it.

That assumes that the goal of the commercial farms is to make a profit in bitcoin. That there is no ulterior motive for preserving the status quo commercial banking and money transferring systems, or for bringing their own crypto currency to the market. "See, bitcoin is broken, but here we have this massive farm and will use it to back up and secure bitcoin2."

[DeathAndTaxes]

Someone (meni or was it gmaxwell) proposed a solution to this any many other attacks against pools.  Going from memory (and likely butchering it) it involved sending blinded work to the miner such that the miner would know if a hash produced a valid share but not if it produced a valid block.  The pool using information kept from the miner could check all returned work and confirm which ones produced valid blocks.  Thus there would be no withholding attack possible (not even against PPS).  It would require a hard fork but I could see it spun as a security measure to prevent disruption of the vital proof of work component.  Maybe I can find the paper.

Found the paper (annoyingly it doesn't support copying, so I can't quote the relevant section).
https://bitcoil.co.il/pool_analysis.pdf
See Section 6.3.2 Oblivious Shares

[eleuthria]

Someone (meni or was it gmaxwell) proposed a solution to this any many other attacks against pools.  Going from memory (and likely butchering it) it involved sending blinded work to the miner such that the miner would know if a hash produced a valid share but not if it produced a valid block.  The pool using information kept from the miner could check all returned work and confirm which ones produced valid blocks.  Thus there would be no withholding attack possible (not even against PPS).  It would require a hard fork but I could see it spun as a security measure to prevent disruption of the vital proof of work component.  Maybe I can find the paper.

Yes, I remember something about it, I believe it was Meni.  The problem is, as you said, it would require a hardfork.  Even if we had proof that this type of attack was happening to explicitly kill off the ability for miners to pool together without incurring huge losses, it would ever be implemented.

If it becomes widespread, the obvious solution would be going to invitation based pooling where miners can be vetted before receiving payments.  The problem is this would completely kill the ability for small miners to join a pool since they would be too slow to prove that they were going to provide valid blocks.

[Entropy-uc]

Someone (meni or was it gmaxwell) proposed a solution to this any many other attacks against pools.  Going from memory (and likely butchering it) it involved sending blinded work to the miner such that the miner would know if a hash produced a valid share but not if it produced a valid block.  The pool using information kept from the miner could check all returned work and confirm which ones produced valid blocks.  Thus there would be no withholding attack possible (not even against PPS).  It would require a hard fork but I could see it spun as a security measure to prevent disruption of the vital proof of work component.  Maybe I can find the paper.

Found the paper (annoyingly it doesn't support copying, so I can't quote the relevant section).
https://bitcoil.co.il/pool_analysis.pdf
See Section 6.3.2 Oblivious Shares

A hard fork is definitely a bridge too far - there would be too many competing options to ever develop consensus on a solution that requires a hard fork.

I do think stratum is broken if it isn't possible for a pool to verify that the hardware it is connected to is functional and operating honestly.

Given what has been said today, I am thinking a certain first generation design known for having many bugs may be broken by current difficulty.  If so, the large users of that hardware had to be aware of the issue and choose to cheat others rather than deal with the problem.  They would have seen a degradation in solo-mining performance as the cliff approached since the probability of a share >1B is dramatically larger than the probability of finding a share between 1B and 4.2B.

[-ck]

First and even second generation asics had no diff support within the chips whatsoever so they just produced a hash and it was up to the software to decide what to do with the result so it cannot be the hardware in the case of avalon 1/2, bfl etc. 3rd generation hardware almost all has diff support on chip or in firmware and that's the only place that the hardware itself could be responsible.

[Entropy-uc]

First and even second generation asics had no diff support within the chips whatsoever so they just produced a hash and it was up to the software to decide what to do with the result so it cannot be the hardware in the case of avalon 1/2, bfl etc. 3rd generation hardware almost all has diff support on chip or in firmware and that's the only place that the hardware itself could be responsible.

Not necessarily.  If there is a bug in the hashing engine itself it could be possible.  Think about the shift operations in SHA-256.  You could have an incorrect implementation that produced good hashes in the last 32 bits (all that is tested in 1st gen approaches) up until you roll into hashes with zeros all the way up to 64 plus bits.  Then think about what chips were famous for generating invalids at outrageous rates.  :-)

If something like this is going on the actual details of how it happens is likely to be vastly more complicated that what I am saying.  But with hardware implementations anything is possible.

[-ck]

First and even second generation asics had no diff support within the chips whatsoever so they just produced a hash and it was up to the software to decide what to do with the result so it cannot be the hardware in the case of avalon 1/2, bfl etc. 3rd generation hardware almost all has diff support on chip or in firmware and that's the only place that the hardware itself could be responsible.

Not necessarily.  If there is a bug in the hashing engine itself it could be possible.  Think about the shift operations in SHA-256.  You could have an incorrect implementation that produced good hashes in the last 32 bits (all that is tested in 1st gen approaches) up until you roll into hashes with zeros all the way up to 64 plus bits.  Then think about what chips were famous for generating invalids at outrageous rates.  :-)

If something like this is going on the actual details of how it happens is likely to be vastly more complicated that what I am saying.  But with hardware implementations anything is possible.

No sorry you don't understand sha256. It's a 32 bit operation on a fixed length string in mining and has NO concept of difficulty or zeroes.

[Entropy-uc]

First and even second generation asics had no diff support within the chips whatsoever so they just produced a hash and it was up to the software to decide what to do with the result so it cannot be the hardware in the case of avalon 1/2, bfl etc. 3rd generation hardware almost all has diff support on chip or in firmware and that's the only place that the hardware itself could be responsible.

Not necessarily.  If there is a bug in the hashing engine itself it could be possible.  Think about the shift operations in SHA-256.  You could have an incorrect implementation that produced good hashes in the last 32 bits (all that is tested in 1st gen approaches) up until you roll into hashes with zeros all the way up to 64 plus bits.  Then think about what chips were famous for generating invalids at outrageous rates.  :-)

If something like this is going on the actual details of how it happens is likely to be vastly more complicated that what I am saying.  But with hardware implementations anything is possible.

No sorry you don't understand sha256. It's a 32 bit operation on a fixed length string in mining and has NO concept of difficulty or zeroes.

Your failure to understand me doesn't mean I don't understand SHA-256.  I've implemented it in both software and hardware and my comments are based upon failure modes I worried about in the hardware implementation.

Let's just sit back and see what the next few days reveal.

[zvisha]

First and even second generation asics had no diff support within the chips whatsoever so they just produced a hash and it was up to the software to decide what to do with the result so it cannot be the hardware in the case of avalon 1/2, bfl etc. 3rd generation hardware almost all has diff support on chip or in firmware and that's the only place that the hardware itself could be responsible.

Not necessarily.  If there is a bug in the hashing engine itself it could be possible.  Think about the shift operations in SHA-256.  You could have an incorrect implementation that produced good hashes in the last 32 bits (all that is tested in 1st gen approaches) up until you roll into hashes with zeros all the way up to 64 plus bits.  Then think about what chips were famous for generating invalids at outrageous rates.  :-)

If something like this is going on the actual details of how it happens is likely to be vastly more complicated that what I am saying.  But with hardware implementations anything is possible.

No sorry you don't understand sha256. It's a 32 bit operation on a fixed length string in mining and has NO concept of difficulty or zeroes.

Your failure to understand me doesn't mean I don't understand SHA-256.  I've implemented it in both software and hardware and my comments are based upon failure modes I worried about in the hardware implementation.

Let's just sit back and see what the next few days reveal.

You can easily fix this in cg-miner/miner driver. Give lower difficulty (let's say 43 leading zeroes). Even on 5TH miner it will give you less then 1 win per second. Then drop unneeded results from HW. You will get all the shares, even if the HW doesn't know how to compare "high" zeroes.