p2pool - Decentralized, Absolutely DoS-Proof, Pool Hopping-Proof Pool [archival]

[lizthegrey]

Nice. Thank you for beating me to the punch, as I've been red taped. I'll go audit your code and give suggestions this weekend since I'm still unable to write code.

What license is your code under?

[1714004027]

1714004027

[1714004027]

1714004027

[1714004027]

1714004027

[1714004027]

1714004027

[forrestv]

GPLv3 (now). What exactly were you thinking of doing?

---

EDIT: Original (outdated) announcement: http://im.forre.st/pb/85341005.txt

[lizthegrey]

GPLv3 (now). What exactly were you thinking of doing?

(mentioned on IRC, but for the record, https://docs.google.com/document/d/1ciKH3M8WYS49ywz08beXtvpCm2wVGdzU7waKwcn_uaU/edit?hl=en_US&authkey=CJTqyOMF&pli=1 was what I had in mind)

[Artefact2]

This definitely seems interesting, but I have a few concerns that would make me think that the system is fubar :

The network finds, in average, 1 block every 10 minutes (and it's often less because the difficulty only resets every 2 weeks).
A block should be found, in average, after 600 of your shares. So that's pretty much 1 share per second to announce to all the miners !

That's an awful lot. If you want it to be decentralized, you can't just send your share to one server, you have to send it to everyone, and that P2P-style like propagation induces latency. And because there's a new share found, in average, every second, you're going to be computing outdated work most of the time. If we consider that it takes 500 milliseconds to announce a share to all the network (and that's a very optimistic estimate, most people don't have very high-quality internet connections), you're effectively going to waste 50% of your work time.

I could be wrong, though, if shares don't have to includes the hash of the "previous" share like blocks do. But, why call it a chain then ?

Also, you claim it to be mostly decentralized at the moment, yet you handle the payouts. How do you ensure the generated shares only give the 50 BTC to you ? (Ie, a malicious miner that finds a "winning" share could keep it for himself and take the 50 BTC). That needs an explanation in my opinion.

[lizthegrey]

The network finds, in average, 1 block every 10 minutes (and it's often less because the difficulty only resets every 2 weeks).
A block should be found, in average, after 600 of your shares. So that's pretty much 1 share per second to announce to all the miners !

Incorrect - this assumes a *single* p2pool represents all of bitcoin's hash network, rather than only a small fraction of the total hash power. If p2pool becomes popular enough, it can be split into multiple sub-pools.

Also, you claim it to be mostly decentralized at the moment, yet you handle the payouts. How do you ensure the generated shares only give the 50 BTC to you ? (Ie, a malicious miner that finds a "winning" share could keep it for himself and take the 50 BTC). That needs an explanation in my opinion.

Only the payouts that balance risk from short blocks to long blocks are managed centrally; the rest of the payouts are fully distributed. A malicious miner finding a winning share would have computed it already with the pool's payout proportions in mind; otherwise, his/her earlier shares would have been rejected by the other miners in the pool and would have no credit with them in the event that they found the block instead; therefore, malicious mining is equivalent to solo mining.

[Artefact2]

The network finds, in average, 1 block every 10 minutes (and it's often less because the difficulty only resets every 2 weeks).
A block should be found, in average, after 600 of your shares. So that's pretty much 1 share per second to announce to all the miners !

Incorrect - this assumes a *single* p2pool represents all of bitcoin's hash network, rather than only a small fraction of the total hash power. If p2pool becomes popular enough, it can be split into multiple sub-pools.

Also, you claim it to be mostly decentralized at the moment, yet you handle the payouts. How do you ensure the generated shares only give the 50 BTC to you ? (Ie, a malicious miner that finds a "winning" share could keep it for himself and take the 50 BTC). That needs an explanation in my opinion.

Only the payouts that balance risk from short blocks to long blocks are managed centrally; the rest of the payouts are fully distributed. A malicious miner finding a winning share would have computed it already with the pool's payout proportions in mind; otherwise, his/her earlier shares would have been rejected by the other miners in the pool and would have no credit with them in the event that they found the block instead; therefore, malicious mining is equivalent to solo mining.

About the first point : okay, I didn't consider that. This pool is still not very scalable though (even if it represents 10% of the network, that's 1 share every 10 seconds).

About the second point : seems complicated. How about simply putting the 599 previous shares in every found share, and when one is a "winner", pay all the previous 599 shares ? Regardless whether they are from a previous block or not. This is much simpler, allows complete decentralization and still pool-hop resistant.

[lizthegrey]

About the second point : seems complicated. How about simply putting the 599 previous shares in every found share, and when one is a "winner", pay all the previous 599 shares ? Regardless whether they are from a previous block or not. This is much simpler, allows complete decentralization and still pool-hop resistant.

This means that people will get double-paid if there are two blocks in short succession and some people won't get paid at all for their shares.

Another option might be to keep a list of the people who have not yet been paid for their shares, and always pay the oldest shares first.

[forrestv]

About the second point : seems complicated. How about simply putting the 599 previous shares in every found share, and when one is a "winner", pay all the previous 599 shares ? Regardless whether they are from a previous block or not. This is much simpler, allows complete decentralization and still pool-hop resistant.

This could work, but variance would be quite a bit higher. P2pool only keeps a chain of shares made since the last p2pool-generated-block in the blockchain (each share points to this last-generated-block). If p2pool looked past that block into the shares for the previous round, it would be vulnerable to somebody generating a block that looks like it was generated by p2pool, but with all the reward going to them, among other things. Then, any p2pool blocks in the next 600 shares would pay a chunk of their reward to the evil creator of that fake block.
With how it is currently, all a fake block does is reset the round, thereby increasing variance a bit. There is a constant fee that goes to me in every block in place to make this not worth it. (You'd have to pay a little less than 1btc for every time that you reset the p2pool round.)
Looking at previous rounds would be more complex - what happens if a node can't find shares from a previous round? (This could happen if an evil person generated a block referencing shares but didn't publish them)
I'm thinking about it and am going to try implementing it.

[forrestv]

Screenshot of mining on Windows (on the testnet, don't get too excited):


Larger: http://u.forre.st/u/quzzemof/p2pool.jpg

[twmz]

What about the fact that IP transactions are scheduled to be removed from the client soon?

[forrestv]

What about the fact that IP transactions are scheduled to be removed from the client soon?

I can change it to generate to addresses instead of pubkeys, but that's not optimal as they're harder to claim (larger scriptSig). Ideally there would be a way to get a pubkey via the RPC interface ... maybe a patch could make 'getnewaddress' return the pubkey along with the address.

Also, the protocol and the verification rules are designed to be very forward compatible and extensible. Somebody can begin using addresses instead of pubkeys in the existing network.

[gmaxwell]

If a block is found in the first 599 shares, the extra is paid to me. On average, 36% of the subsidy of a block will go to me as a result of this. This is a trade-off between minimizing the payment to some central entity and having lower variance in payouts. If the proportion going to me were reduced by half, variance would double. I believe that this is an acceptable compromise.

I think this is a beautiful system, but I have a suggested improvement.

If the block is found in the first 599 shares, it should be payed to a multisig escrow. Like this: https://github.com/bitcoin/bitcoin/pull/319

The keys in use could be held by either a collection of parties (reduced trust centralized) or, better,  by a large deterministic random subset of the P2P nodes.  E.g. Take all of the nodes who were paid in the last round and use the block hash of the last block to pick 21 of them (or all of them if there are fewer), and require >50% to sign the transaction releasing the escrow to the appropriate receivers.

I believe this would completely decentralize it while preserving the other properties of your system.

[jojkaart]

I like this idea for how to solve the problem of making it completely decentralized. The only misgiving I have about this system is that while it works for now, it'll eventually end up having too much variance as well. So, people would start pooling again. However, it would likely still have solved the problem of pools having too much power.

I'd love to have something like this integrated into the main client to replace solo mining but the scalability problems inherent in this version make this approach unsuitable for it.

- Joel

[gmaxwell]

I like this idea for how to solve the problem of making it completely decentralized. The only misgiving I have about this system is that while it works for now, it'll eventually end up having too much variance as well. So, people would start pooling again. However, it would likely still have solved the problem of pools having too much power.

I'd love to have something like this integrated into the main client to replace solo mining but the scalability problems inherent in this version make this approach unsuitable for it.

Hm? it reduces variance vs solo by an enormous factor. This might not be all that helpful for joe-random cpu-miner, but for someone with enough power that they might even think about running solo its _very_ attractive because it reduces the variance enough to make 'solo' perfectly reasonable without the negatives of the pool system— poor reliability, fees, cheating (by operators and other users), and power consolidation that endangers bitcoin.

The payout system implied by this makes it automatically not so great for aggregating lots of tiny users. Were you to increase the number of shares so that most slow users would get one every block then you'd end up with massively bloated blocks, and also a lot of people with micropayments that cost them more money to use as inputs.

Moreover, centralized pools themselves could be participants in this scheme, so smaller pool operators would have less variance disadvantage vs big ones, so even the pooling that remains could be more distributed and competitive.

 

[huayra.agera]

Hi I read through it all and it seems I got it to work, only that I'm generating "zero" shares. Is this up and running now? I can help in testing this. I too am looking at this as a very good idea/implementation of a pool. 

[Shevek]

Nothing is sent though port 9332.

I've launched locally p2pool, bitcoind (0.3.23) and cpuminer (I know it is useless; just to test p2pool).

p2pool and bitcoind inter-communicate well.

Miner ("cpuminer") doesn't get in touch with p2pool.

Code:

minerd -a cryptopp_asm32 -t 1 --syslog --url http://127.0.0.1:9332 --userpass dummy:dummy &

And the answer:

Code:

Jun 27 11:12:30 anarres cpuminer[32184]: 1 miner threads started, using SHA256 'cryptopp_asm32' algorithm.
Jun 27 11:12:59 anarres cpuminer[32184]: HTTP request failed: couldn't connect to host
Jun 27 11:12:59 anarres cpuminer[32184]: json_rpc_call failed, retry after 30 seconds

Any clues?

[Shevek]

Nothing is sent though port 9332.

OK.

It's simple a matter of waiting 'till "Got block..." messages finish. 

[Shevek]

Is the program actually used in the real net or only in the testnet?

I've used it for a while and all shares I've found were rejected as stale, with an error messages on terminal related to python code.

There were also messages about connected peers... but only one IP-address appeared (I don't remember: 71.?).

[forrestv]

There just aren't many users currently (only me at the moment). Can you post any Python error messages you see?

[Shevek]

There just aren't many users currently (only me at the moment). Can you post any Python error messages you see?

Code:

GOT SHARE! afe24d1fd5f252cceb1b781f683db4bdddc45da174456f5f0d2d16ef

Error processing data received from worker:
Traceback (most recent call last):
  File "main.py", line 399, in got_response
    p2p_share(share)
  File "main.py", line 245, in p2p_share
    res = chain.accept(share, net)
  File "main.py", line 55, in accept
    share2 = share.check(self, height, previous_share2, net) # raises exceptions
  File "/home/shevek/bitcoin/p2pool_2/p2pool.py", line 103, in check
    raise ValueError('not enough work!')
ValueError: not enough work!