[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[kano]

It requires sending the entire block template (potentially up to 1 MB) to miners every 10 seconds, which is definitely impractical for remote miners. An extension to GBT that allowed only sending the merkle root would avoid this, though.

Sending only the merkle root, whether it be via getwork, Stratum, or some new GBT extension, makes the pool centralized, and kinda defeats the entire point of p2pool... Perhaps it would make sense for "remote" p2pool servers to be the first to move forward with mandatory-miner-provides-the-transactions GBT?

If the protocol (Stratum) has the option to request the transaction list - as you already know - then why are you making this statement?
Are you arguing that GBT isn't centralised because you HAVE to receive the full transaction list?
That optionally receiving it isn't good enough? Seriously?

Yeah forrestv it seems that you need to take his hidden agenda comments with a grain of salt.
His agenda is to promote GBT no matter what problems there are with it, and to use dictionary arguments to avoid transparency.

[forrestv]

The earlier argument (^) about remote miners not working was a bit of a farce, as shown by the above numbers.

The thing to focus on in order to make sure things work is the timestamp rolling support in ASIC mining software. We need to ensure that it can roll ahead of the current time (and potentially backwards, though that would require an extension to getwork).
...

Sorry - I don't get the point of this comment.
Why do you need rolling the time?

Unless I've completely missed your point, you don't need to roll the time with Stratum or GBT, you roll a value in the coinbase instead, named (IMO badly) the secondary nonce instead of screwing with the block timestamp (which is a hack) unnecessary

P2Pool can't handle simply altering the coinbase script. P2Pool internally has a structure that describes how to recreate the coinbase transaction, including the coinbase script, but that can't be changed because the hash of that structure is stuck in the last txout. P2Pool uses it to create short, self-contained proofs of work by storing the SHA256 midstate with shares and computing the
generation transaction's hash using that midstate and the hash of that internal structure.

TL;DR: If the coinbase script is altered, the hash in the last txout of the generation transaction has to be changed too.

However, I discovered yesterday that Stratum is flexible enough to roll any data in the coinbase transaction at all, though, so rolling something in the last txout script is probably possible (though it would require a change to P2Pool's rules).

[kano]

Um - it's the actual coinbase field itself - the same place where you put all that other random data.
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?

[forrestv]

Um - it's the actual coinbase field itself - the same place where you put all that other random data.
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?

You could easily roll something in the coinbase script,  yes, but it wouldn't result in a valid p2pool share. The miner would need to be smart enough to update the data in the last txout, which is impractical.

[kano]

Um - it's the actual coinbase field itself - the same place where you put all that other random data.
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?

You could easily roll something in the coinbase script,  yes, but it wouldn't result in a valid p2pool share. The miner would need to be smart enough to update the data in the last txout, which is impractical.

But the software already has to do this every time a share changes (or a block changes) or new transactions are added.

Why can't it do it every time it completes a nonce range?

From the point of view of the merkle tree, it's only one side that needs to be recalculated (unless you are adding more transactions or using different transactions)

Again I don't see what I am missing here that makes this impossible.

[btharper]

Um - it's the actual coinbase field itself - the same place where you put all that other random data.
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?

You could easily roll something in the coinbase script,  yes, but it wouldn't result in a valid p2pool share. The miner would need to be smart enough to update the data in the last txout, which is impractical.

But the software already has to do this every time a share changes (or a block changes) or new transactions are added.

Why can't it do it every time it completes a nonce range?

From the point of view of the merkle tree, it's only one side that needs to be recalculated (unless you are adding more transactions or using different transactions)

Again I don't see what I am missing here that makes this impossible.

You're missing a semi-truck barreling down the road at you. The coinbase rolling that P2Pool uses is different because of the way p2pool payouts happen. While it may not be impossible having miners roll the coin base (and by extension, the whole merkle tree) may be harder. I don't know enough to elaborate but you seem to be completely ignoring what forestv is saying.

Given what else was said and assuming the other issues can be worked out stratum may be a better choice then, mostly because of bandwidth. GBT and Stratum both offer some modification to the coinbase (GBT through "coinbase/append" flag and Stratum through how the coinbase is always built to identify each miner).

Maybe it would be worth adding a "transaction/compact" flag to the GBT request and allow it to respond with just the merkle branches instead of the full transactions. If my logic isn't too far off that would close the gab between GBT and Stratum based on bandwidth usage.

[Luke-Jr]

Maybe it would be worth adding a "transaction/compact" flag to the GBT request and allow it to respond with just the merkle branches instead of the full transactions. If my logic isn't too far off that would close the gab between GBT and Stratum based on bandwidth usage.

Might as well just use Stratum in that case, as you've negated the main benefit of GBT (decentralization).
As long as we're talking p2pool though, might as well use another pool entirely, since decentralization is the main benefit of p2pool

[Krak]

Might as well just use Stratum in that case, as you've negated the main benefit of GBT (decentralization).
As long as we're talking p2pool though, might as well use another pool entirely, since decentralization is the main benefit of p2pool

But if you're just using Stratum for communication between p2pool and your mining software, centralization isn't a problem.

[btharper]

Maybe it would be worth adding a "transaction/compact" flag to the GBT request and allow it to respond with just the merkle branches instead of the full transactions. If my logic isn't too far off that would close the gab between GBT and Stratum based on bandwidth usage.

Might as well just use Stratum in that case, as you've negated the main benefit of GBT (decentralization).
As long as we're talking p2pool though, might as well use another pool entirely, since decentralization is the main benefit of p2pool

If you're mining on your own P2Pool instance it's not an issue anyway. If you're mining on someone else's P2Pool instance then you're already becoming more centralized.

It does defeat some of the ideological niceties of GBT, but it's still an optional flag (Stratum has similar from what I've heard, though it's off by default, instead of on by default with this proposed compact flag).

[Luke-Jr]

Might as well just use Stratum in that case, as you've negated the main benefit of GBT (decentralization).
As long as we're talking p2pool though, might as well use another pool entirely, since decentralization is the main benefit of p2pool

But if you're just using Stratum for communication between p2pool and your mining software, centralization isn't a problem.

In that case, neither is bandwidth

If you're mining on someone else's P2Pool instance then you're already becoming more centralized.

Not if it was using GBT. But obviously with p2pool's 10 second altchain blocks, the protocol used by GBT today is unreasonable.

It does defeat some of the ideological niceties of GBT, but it's still an optional flag (Stratum has similar from what I've heard, though it's off by default, instead of on by default with this proposed compact flag).

No, it's currently not possible to use GBT in a centralized setup. Stratum has recently added a similar feature, but it still has overhead for decentralized use.

[kano]

Um - it's the actual coinbase field itself - the same place where you put all that other random data.
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?

You could easily roll something in the coinbase script,  yes, but it wouldn't result in a valid p2pool share. The miner would need to be smart enough to update the data in the last txout, which is impractical.

But the software already has to do this every time a share changes (or a block changes) or new transactions are added.

Why can't it do it every time it completes a nonce range?

From the point of view of the merkle tree, it's only one side that needs to be recalculated (unless you are adding more transactions or using different transactions)

Again I don't see what I am missing here that makes this impossible.

You're missing a semi-truck barreling down the road at you. The coinbase rolling that P2Pool uses is different because of the way p2pool payouts happen. While it may not be impossible having miners roll the coin base (and by extension, the whole merkle tree) may be harder. I don't know enough to elaborate but you seem to be completely ignoring what forestv is saying.

Given what else was said and assuming the other issues can be worked out stratum may be a better choice then, mostly because of bandwidth. GBT and Stratum both offer some modification to the coinbase (GBT through "coinbase/append" flag and Stratum through how the coinbase is always built to identify each miner).

Maybe it would be worth adding a "transaction/compact" flag to the GBT request and allow it to respond with just the merkle branches instead of the full transactions. If my logic isn't too far off that would close the gab between GBT and Stratum based on bandwidth usage.

Ah nope I see the problem.

If you implement it in p2pool, p2pool itself is also adding roll-n-time to the miner on top of what it is doing so the miner doesn't have to get work every nonce range.

Stratum block building is done by the miner, in p2pool it is done by the p2pool software only - so yeah the miner wont know the p2pool rule to add to Stratum block building.

So results so far is that p2pool is forced to use the roll-n-time hack - it has no other way to give out long term work to the miner with it's current design.

... you could throw some BTC to ckolivas to request a change to cgminer to support the p2pool rule addition to the stratum generation process ...
... an optional flag that p2pool sends to say it needs to add that extra change to the coinbase transaction generation

(Edit: of course a larger nonce in the block header would solve that )

[tiberiandusk]

I'm running Win7 64bit with 12gb of RAM and a 3 core 3.0ghz processor and P2Pool is always losing connection to bitcoind. What can I do to keep bitcoin running properly?

edit I created a symbolic link to the Bitcoin folder in appdata and moved the blockchain stuff to a faster drive in my computer. I'll see if this helps.

I have linux VM w/2GB ram running two p2pools and 5 xCoind`s w/o any trouble
Try exclude bitcoin and p2pool directory form virus scanning.

Excluding from the virus scanner seems to have helped. It must have been rescanning the whole blockchain every time a new block was found.

[cabin]

What does "Punishing share for 'Block-stale detected!" mean? I gather it happens after a new block appears on the network.. but what is the punishment?

[TheHarbinger]

What does "Punishing share for 'Block-stale detected!" mean? I gather it happens after a new block appears on the network.. but what is the punishment?

Off with your head!

[stevegee58]

What does "Punishing share for 'Block-stale detected!" mean? I gather it happens after a new block appears on the network.. but what is the punishment?

You must spank her well.  And after you have spanked her, you may deal with her as you like.  And then, spank me.

[forrestv]

What does "Punishing share for 'Block-stale detected!" mean? I gather it happens after a new block appears on the network.. but what is the punishment?

First, this has always been done, but I've just recently made the message more visible. It used to be simply "Stale detected! (long hex number) < (long hex number)".

In order to punish the share, your node tries to orphan it by building on that share's parent instead of on top of it. If you find a share, you'll have made a fork, and other nodes will prefer your share to the original, since yours wasn't block-stale, resulting in the original share not being integrated into the chain.

[bitpop]

Did I miss something, can we use fpgas or asics with p2pool now?

[Krak]

Did I miss something, can we use fpgas or asics with p2pool now?

Anything except for the current BFL Singles. Minirigs and the new ASICs should work fine.

[bitpop]

Oh what changed?

[Krak]

Oh what changed?

Nothing. That's the way it's always been. Although ckolivas did have to add the --bfl-range flag to cgminer to get it working with Minirigs.