how do pools work? / why are pools not a threat?

[giszmo]

Hi

as to my understanding, bitCoins success is based on the peer to peer structure and any entity owning 50% of computational power could cheat on the rest or stop the show for all. While pools are no threat in the latter sense as miners are no zombies tied to them against their will, I come to understand that pools really pose a threat in the former sense.

(Yes I know this must have been discussed somewhere in depth but i could not find it. It's mostly annoying me that all people suggest to join a pool based on the kh or Gh they do.)

My understanding of "mining" was that the miner may add into a block he creates *one* transaction for the reward and as many other transactions as he likes taken from all 0conf transaction that he sees. (input data) he then brute force tries adding salt (never read that but that's my conclusion) and if hash(prior block + input + time + ... + salt) < x he sends it out to the world. fool proof. simple. great.

As in slush pool's sticky post it is made very clear that in a pool situation it is not the miner who decides what gets into the block but the pool delegates the work to the miners. I read it such that not only does the pool dictate the reward (would make sense) but also transactions, time, prior block and the salt to try out.

My questions:
Am I about right in my analysis?
Does the miner know when it found a block?
Does the miner promote its finding only to the pool?
Does the miner do plausibility checks for the timestamp at least?
What if it is not plausible?
Do pools keep an eye on each other by comparing block chain data ...?

Thanx for pointers and answers ...

[1714044799]

1714044799

[1714044799]

1714044799

[giszmo]

Is there seriously nobody able or willing to debunk my accusations as stupid and paranoid?

If not, why is all the world still promoting to use pools? What's the big benefit of having a cash out every day over having one every month or two if the sum is the same taken you support fuzzy constructs that promise a profit where there should actually not be a profit if the system itself was not flawed.

Any way to deliver an additional profit should be considered cheating in a system that has as one of its most basic ideas to empower everybody to run part of the infrastructure and be rewarded for it at equal chance.

[RyNinDaCleM]

A while back I figured out that it would take me an average of 30 days to solve a block solo mining! I then realized that mining with a pool, I would make about 60 BTC in the same amount of time. Then you factor in variance. The average of "30 days" doesn't mean it WILL happen. Just on average. It could take 60 days. It could take 60 minutes. That is where pools come in. They remove *some* of the variance (not all) and you get many small payouts. Plus it has a psychological effect on the miner (person). If you sit there for 30 days watching your zero BTC balance never increasing, is kind of discouraging to say the least. However, in a pool, it is somewhat relieving to see that your work is "paying off". For smaller miners, this can be magnified, and they may never find a block solo.
As far as size of a pool, for choosing the "right" pool, I would have to say bigger is not necessarily better. Being part of DeepBit, the payouts are frequent but very small. A tiny pool takes way to long to find a block. So I aim for "mid-range"! 200-600GH/s! This way payouts are much larger, they come in a little slower, but not too infrequently. All and all, any method you choose, averages out in the long run.

[willphase]

The pool can't dictate the reward because it would have to control more than 50% of the total compute power and run a botched client that gives the pool more than 50btc power block. However if a pool did control more than 50% then this would be possible.  This is why it's better for the network to have more pools, just as it's better for the economy to have more exchanges.

bitcoin security is done by consensus. If you are the consensus then in theory you control the network.

Will

[giszmo]

A while back I figured out that it would take me an average of 30 days to solve a block solo mining! I then realized that mining with a pool, I would make about 60 BTC in the same amount of time.

If this is true, the pool gives you 20% more profit? How can that be without cheating? Have you been inaccurate with your numbers or did the pool actually cash out 20% more than solo mining?

The pool can't dictate the reward because it would have to control more than 50% of the total compute power and run a botched client that gives the pool more than 50btc power block.

The pool could increase the reward but that would be way to obvious for other clients. My client would (hopefully) not accept a block with a 51BTC reward so the block chain would split and stay split, wouldn't it?

Still by owning more than 50% the cartel of pools (no pool has more than 50% right now) would be able to just ignore the rest of the net's blocks, bring down the difficulty and double the pools drop rate by just taking the full share rather than the 51% it deserves. Or it just goes with +20% (see RyNinDaCleM's post) drop rate instead of +100% by not rejecting all but only some blocks of the rest. Less greedy. Less obvious.

[RyNinDaCleM]

In my post, the 60 BTC in 30 days was an approximation! Luckiness plays a role here too! By lucky, I mean the current difficulty is 877227. If a block is solved in less than 877227 shares, it is lucky. If more than 877227, it is an unlucky block! Many lucky blocks will pay more than many unlucky blocks! So, it's not that a pool can change the 50BTC reward, if the average "luck" is high, it will pay out more often than average! Right now, I am part of a smaller pool of about 300GH/s. we have been solving blocks a bit more on the lucky side, so I have been earning more per day than a BTC calculator would say I *should* be making. This is not cheating, nor do I poolhop (which can cheat some pools, if they have a proportional payout system). I guess I have just chose the right place to be, somewhat consistently enough, to maximize my benefit! When difficulty rises to a point that my hardware won't any longer keep up, even in a pool, I may start solo mining just to see if I can find 1 last block in any decent amount of time.

[Meni Rosenfeld]

as to my understanding, bitCoins success is based on the peer to peer structure and any entity owning 50% of computational power could cheat on the rest or stop the show for all. While pools are no threat in the latter sense as miners are no zombies tied to them against their will, I come to understand that pools really pose a threat in the former sense.

(Yes I know this must have been discussed somewhere in depth but i could not find it. It's mostly annoying me that all people suggest to join a pool based on the kh or Gh they do.)

Indeed it has, and solutions have been proposed. In fact "don't join deepbit because it's harmful for the network" is commonly heard.

As in slush pool's sticky post it is made very clear that in a pool situation it is not the miner who decides what gets into the block but the pool delegates the work to the miners. I read it such that not only does the pool dictate the reward (would make sense) but also transactions, time, prior block and the salt to try out.

That's how it works currently, yes. There are suggestions to allow the miner to choose what to put in the block.

Am I about right in my analysis?

More or less.

Does the miner know when it found a block?

Yes. Which is actually a problem since it opens up several types of attack. I suggested once to allow miners to know if they found a block only after the fact.

Does the miner promote its finding only to the pool?

Yes, the generation transaction in the block rewards the pool so there's no sense doing anything else with it. But the pool then broadcasts the block to the network.

Does the miner do plausibility checks for the timestamp at least?
What if it is not plausible?

I think the miner chooses the timestamp. And the timestamp is validated by nodes on the network before propagating the block.

Do pools keep an eye on each other by comparing block chain data ...?

A block is propagated in the network only if it is valid.

If not, why is all the world still promoting to use pools? What's the big benefit of having a cash out every day over having one every month or two if the sum is the same taken you support fuzzy constructs that promise a profit where there should actually not be a profit if the system itself was not flawed.

It's the same on average. But it's completely random. Suppose your payout should be $1000 per month. If you mine in a pool you get $1000. If you try to mine solo you could easily get $0, or $2K or $3K. And it's not like if you didn't find a block then it's waiting around the corner, for the next month you have exactly the same odds. Not only is this nerve-wrecking, but economically it is inferior since the utility of money is concave. Not to mention availability if you actually need to use the money. And this only gets worse as more people start to mine, instead of 50% of getting $1K you could be looking at 0.5% of getting $100K.

Variance is bad, and reducing it is not a luxury. Economics is not a zero-sum game, pools and miners can have a mutually beneficial arrangement - the pool takes some fees and the miner greatly reduces his variance.

[giszmo]

Does the miner know when it found a block?

Yes. Which is actually a problem since it opens up several types of attack. I suggested once to allow miners to know if they found a block only after the fact.

What attack could there possibly be? My hash only works for the exact block and modifying details doesn't work. I would only have two options: report the block to be found or not. How would the latter possibly be an attack? What am I missing?

Does the miner promote its finding only to the pool?

Yes, the generation transaction in the block rewards the pool so there's no sense doing anything else with it. But the pool then broadcasts the block to the network.

Well ... If the promotion of found blocks is to the discretion of the pool, only, it can pile up some blocks ahead of time to release them if the pool risks to lose the race for the longest chain but hold it back to leave the rest compute for the wrong chain.

Does the miner do plausibility checks for the timestamp at least?
What if it is not plausible?

I think the miner chooses the timestamp. And the timestamp is validated by nodes on the network before propagating the block.

How does the proof of work work if the miner chooses the timestamp. It must be part of the hashed data.

Do pools keep an eye on each other by comparing block chain data ...?

A block is propagated in the network only if it is valid.

Nah I meant rather if pools watch for dropped blocks as this would be a sign of cartel activity.

And this only gets worse as more people start to mine, instead of 50% of getting $1K you could be looking at 0.5% of getting $100K.

With 50% to get a block per month I claim people should be patient and rely on maths. With your projection I more or less get the picture. BTC becomes gambling more and more.

[fpgaminer]

I think the miner chooses the timestamp.

Most do not (if any). The pool just gives the miner 1024-bits + 256-bits of data, the miner chugs along changing 32-bits of it, until it runs out of possibilities for that 32-bit number (or it gets new work). Those 32-bits are called the nonce, by the way.

In my post, the 60 BTC in 30 days was an approximation!

Just to clear this up, you'd get 50 BTC in the same amount of time whether you mined at a pool or solo. On average. Of course, in reality, for pooled mining you're more likely to get less (compared to solo) due to stale shares, pool fees, and loss of generation fees.

The pool can't dictate the reward because it would have to control more than 50% of the total compute power and run a botched client that gives the pool more than 50btc power block. However if a pool did control more than 50% then this would be possible.

Controlling >50% of Bitcoin's hashing power does not let you changing the rules. Giving yourself more than 50BTC as part of the Generation Transaction would make that block invalid to all official clients. i.e. you've just made useless money that nobody will recognize as valid.

The clients enforce a set of rules on the blocks, regardless of who "controls" the network. So controlling the network doesn't actually gain you very much. You can read more about it on the wiki, which I will quote here:

An attacker that controls more than 50% of the network's computing power can, for the time that he is in control, exclude and modify the ordering of transactions. This allows him to:
Reverse transactions that he sends while he's in control
Prevent some or all transactions from gaining any confirmations
Prevent some or all other generators from getting any generations

The attacker can't:
Reverse other people's transactions
Prevent transactions from being sent at all (they'll show as 0/unconfirmed)
Change the number of coins generated per block
Create coins out of thin air
Send coins that never belonged to him

What attack could there possibly be? My hash only works for the exact block and modifying details doesn't work. I would only have two options: report the block to be found or not. How would the latter possibly be an attack? What am I missing?

If you're mining against a PPS pool, for example, you're getting paid immediately for every share you submit. Now, while doing that, withhold any shares you've found that would result in the pool getting a block. You're still getting paid the same amount of money (-1 share in a million), but the pool is getting less found blocks.

[relmeas]

the getwork command should be re-written to give to the miners full block data instead of just the header.

that way miners can verify that the block they are working on will give money to the pool and not some other wallet.

currently pool owners can easily cheat by giving the miners work that generates for some different address.

i dont claim some do that but it is very easy AND absolutely undetectable. sure you can in theory check nonces on each new block to see if your found successful nonce is in any of them (and check who they generate btc for) but this check is only possible if you do find a solution which is very very rare and I doubt some mining software is written with that function...

face it, when you can cheat and you're absolutely sure no one will ever catch you .. the temptation is too strong.

the thing is, client was not written with pool mining in mind .. it needs to be amended according to the current times.

[Meni Rosenfeld]

What attack could there possibly be? My hash only works for the exact block and modifying details doesn't work. I would only have two options: report the block to be found or not. How would the latter possibly be an attack? What am I missing?

If you're mining against a PPS pool, for example, you're getting paid immediately for every share you submit. Now, while doing that, withhold any shares you've found that would result in the pool getting a block. You're still getting paid the same amount of money (-1 share in a million), but the pool is getting less found blocks.

This, as well as others. See for example here.

Well ... If the promotion of found blocks is to the discretion of the pool, only, it can pile up some blocks ahead of time to release them if the pool risks to lose the race for the longest chain but hold it back to leave the rest compute for the wrong chain.
...
Nah I meant rather if pools watch for dropped blocks as this would be a sign of cartel activity.

It shouldn't be difficult for miners to verify that they're working on the latest block. And clients can detect when they suddenly get a whole new long branch. I don't think these tests have been implemented yet because we're just not there. The sudden media attention notwithstanding, everything is still in beta.

How does the proof of work work if the miner chooses the timestamp. It must be part of the hashed data.

The miner decides the nonce by itself, in theory there shouldn't be a problem to choose the timestamp too.

With 50% to get a block per month I claim people should be patient and rely on maths. With your projection I more or less get the picture. BTC becomes gambling more and more.

The math says that the utility of money is concave and hence variance is bad. Also when you don't get any payout for a long time it's difficult to know everything is working. The .5% per month can happen right now if you're using a low-end GPU. And we're not Vulcans, when I tried mining solo I couldn't handle the pressure, and I was at an average 10 blocks per month.

[Meni Rosenfeld]

the getwork command should be re-written to give to the miners full block data instead of just the header.

that way miners can verify that the block they are working on will give money to the pool and not some other wallet.

currently pool owners can easily cheat by giving the miners work that generates for some different address.

i dont claim some do that but it is very easy AND absolutely undetectable. sure you can in theory check nonces on each new block to see if your found successful nonce is in any of them (and check who they generate btc for) but this check is only possible if you do find a solution which is very very rare and I doubt some mining software is written with that function...

face it, when you can cheat and you're absolutely sure no one will ever catch you .. the temptation is too strong.

the thing is, client was not written with pool mining in mind .. it needs to be amended according to the current times.

You're confused about several things.

There's no "pool's address" which is the indication that the pool found a block. It uses whatever address it wants, possibly a different one per getwork request. It's up to the operator to distribute rewards when the pool finds a block.

The way to avoid cheating is for the miners to notice that when they find a block it is acknowledged by the pool (with my oblivious shares suggestion, this can only be done with some delay).

It's possible that having a fixed address for the pool can improve transparency but that's a separate change.

Also, in the future each block can be gigabytes or terabytes and it will be unfeasible for miners to download it.

[relmeas]

the getwork command should be re-written to give to the miners full block data instead of just the header.

that way miners can verify that the block they are working on will give money to the pool and not some other wallet.

currently pool owners can easily cheat by giving the miners work that generates for some different address.

i dont claim some do that but it is very easy AND absolutely undetectable. sure you can in theory check nonces on each new block to see if your found successful nonce is in any of them (and check who they generate btc for) but this check is only possible if you do find a solution which is very very rare and I doubt some mining software is written with that function...

face it, when you can cheat and you're absolutely sure no one will ever catch you .. the temptation is too strong.

the thing is, client was not written with pool mining in mind .. it needs to be amended according to the current times.

You're confused about several things.

There's no "pool's address" which is the indication that the pool found a block. It uses whatever address it wants, possibly a different one per getwork request. It's up to the operator to distribute rewards when the pool finds a block.

The way to avoid cheating is for the miners to notice that when they find a block it is acknowledged by the pool (with my oblivious shares suggestion, this can only be done with some delay).

but this is very rare and very few miners can do this check. it is very likely that even if some mining software would implement such feature, the ones that find the block won't have it installed. we need a method for any miner to check.

It's possible that having a fixed address for the pool can improve transparency but that's a separate change.

exactly, that was implied.

i think the new pool that would offer such transparency would gain followers since it's an actual feature.

Also, in the future each block can be gigabytes or terabytes and it will be unfeasible for miners to download it.

currently it's not a problem. Note that I do not propose to give out full block all the time, just very rarely. There could be separate command that requests full block (instead of just a header) that the miners would use once in a while to check what they are working on. Or simply publish full block via http. When there is 4 hour block on a 2 TH pool the miners can just look if its legit instead of just switching pools.

when blocks become too big for this it can always be disabled since some better mining method would probably be implemented by then.

[giszmo]

The miner decides the nonce by itself, in theory there shouldn't be a problem to choose the timestamp too.

I read that (some) pools trial the miners with payload they know contains a hit just to verify they actually do work. If that is the case as I assume and if changing the timestamp would change the result of such test, the pool *must* dictate the timestamp and thus can do unpublished blocks in advance.

[relmeas]

The miner decides the nonce by itself, in theory there shouldn't be a problem to choose the timestamp too.

I read that (some) pools trial the miners with payload they know contains a hit just to verify they actually do work.

can't this be used to detect attacking clients that want to withhold the winning blocks?

[Meni Rosenfeld]

I read that (some) pools trial the miners with payload they know contains a hit just to verify they actually do work.

I'm fairly sure no pool has done that in ages, verification of work is done using shares.

If that is the case as I assume and if changing the timestamp would change the result of such test, the pool *must* dictate the timestamp and thus can do unpublished blocks in advance.

It's unnecessary to forge the timestamp, having an old timestamp doesn't make a block invalid unless it's earlier than the blocks it builds on. It's the reference to the last block that matters. If a pool tries to concoct an alternative branch the miner can easily find out that the previous block he works on is not the latest block known to the network.

can't this be used to check attacking clients that want to withhold the wining blocks?

Possibly.

[giszmo]

can't this be used to detect attacking clients that want to withhold the winning blocks?

withhold the winning blocks??? what does that mean? i'm not a native english speaker but according to my dictionary "withhold" can mean "keep for them self" which is not possible as pool mining means the pool dictates the block (txs that get in, timestamp and addresses for the 50BTC reward) and the miner only finds a salt so the hash fits. Changing the data will make him fail on test data and get him kicked out of that pool.

"withhold" also means "hold back" which would not make sense neither as it would simply delay the next block from being found.

[relmeas]

can't this be used to detect attacking clients that want to withhold the winning blocks?

withhold the winning blocks??? what does that mean? i'm not a native english speaker but according to my dictionary "withhold" can mean "keep for them self" which is not possible as pool mining means the pool dictates the block (txs that get in, timestamp and addresses for the 50BTC reward) and the miner only finds a salt so the hash fits. Changing the data will make him fail on test data and get him kicked out of that pool.

"withhold" also means "hold back" which would not make sense neither as it would simply delay the next block from being found.

there are rumors that some pools may add miners to the competitor pool and not submit winning blocks so competitor would spend more time per block and eventually go bankrupt.

this check won't help anyway since the attacker miner can just check if the header is for some previous existing block and submit if this is detected.

another way to do it - remember all invalid blocks for current difficulty and use them. but those can also be collected...

[giszmo]

I read that (some) pools trial the miners with payload they know contains a hit just to verify they actually do work.

I'm fairly sure no pool has done that in ages, verification of work is done using shares.

Hmm ... didn't know pools work like that (=the miner has to provide solutions against a lower difficulty as proof of work.) but I think I remember having read that checking actually found blocks also was used in pools.

Anyway so if a miner has to proof work he can either waste time on tampered data resulting in a block creation maybe but that is called solo mining or he can try to find solutions for the pools data and eventually find solutions to the lower difficulty allowing him to proof work.

It's unnecessary to forge the timestamp, having an old timestamp doesn't make a block invalid unless it's earlier than the blocks it builds on. It's the reference to the last block that matters. If a pool tries to concoct an alternative branch the miner can easily find out that the previous block he works on is not the latest block known to the network.

Makes sense but old timestamps would rise suspicion. Also it's not true that the block chain only contains strictly growing time stamps. I've seen examples where this was not the case.

[giszmo]

there are rumors that some pools may add miners to the competitor pool and not submit winning blocks so competitor would spend more time per block and eventually go bankrupt.

Sorry but that sounds ridiculous - at first ...

So the scenario would be:
Pool (E)vil registers as a miner with Pool (G)ood. Now G assigns work to E. E assigns it on to its miners. E hands on the proof of work part to G. G still finds some blocks and distributes the reward among all according to their proof of work.
E would only need to shift as many miners over to G as there were miners there before to cut G's miners payouts in half. E's miners actually mining for G would get rewards based on E's policies that would be higher than G's miners.

Sounds like a plan for big pools that want to keep smaller pools at distance. Thanx for the inspiration
I love gitCoin's complexity in its simplicity.