[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[lavajumper]

You're using non-standard UI?
can you share node's address?

Yes, it the ExtendedUI. I cloned p2pool from github git://github.com/forrestv/p2pool.git
then got the extended UI. The issue exists with both frontends.

Here's the link: http://sexcoin.lavajumper.com:9699

Over the last couple of hours some of the stats have started filling in.

There's another issue I'm seeing. Coins are not being dropped into the mining wallet. My miner is getting .5% even when its the only miner in the pool. I've chased down the address that they are going to, and I've never seen it before. Its never been on the pool.

If I need to clarify anything, let me know. I'm a bit frustrated at the moment.
Thanks for the help!

[twmz]

You're using non-standard UI?
can you share node's address?

Yes, it the ExtendedUI. I cloned p2pool from github git://github.com/forrestv/p2pool.git
then got the extended UI. The issue exists with both frontends.

Here's the link: http://sexcoin.lavajumper.com:9699

Over the last couple of hours some of the stats have started filling in.

There's another issue I'm seeing. Coins are not being dropped into the mining wallet. My miner is getting .5% even when its the only miner in the pool. I've chased down the address that they are going to, and I've never seen it before. Its never been on the pool.

If I need to clarify anything, let me know. I'm a bit frustrated at the moment.
Thanks for the help!

Data is not loading because there are javascript errors on the frontend.  In particular, the frontend is trying to load /web/best_share_hash and is getting a 404 error.  That is then cascading into several other javascript errors as a result.  I have no idea why /web/best_share_hash is erroring out.

For the payout issue, did you specify a payout address on the command line?  If not, p2pool will allocate a new address from the wallet it is connected to for things like any configured fee, etc.

[lavajumper]

Ok, I can look for the javascript errors.
best_share_hash is erroring out because the call was throwing python exceptions, so I commented out the call in python. Looks like I'll have to actually fix it

I specified an address on the command line, and it gets reported at startup, it is also in the 'cached-addresses' file. It still has a zero balance.

[kano]

Ok, I can look for the javascript errors.
best_share_hash is erroring out because the call was throwing python exceptions, so I commented out the call in python. Looks like I'll have to actually fix it

I specified an address on the command line, and it gets reported at startup, it is also in the 'cached-addresses' file. It still has a zero balance.

... and will have a zero balance until:
1) You get a share
2) P2Pool gets a block after 1)
3) The payment switches to generate (about 16 hours)

[lavajumper]

Ok, I can look for the javascript errors.
best_share_hash is erroring out because the call was throwing python exceptions, so I commented out the call in python. Looks like I'll have to actually fix it

I specified an address on the command line, and it gets reported at startup, it is also in the 'cached-addresses' file. It still has a zero balance.

... and will have a zero balance until:
1) You get a share
2) P2Pool gets a block after 1)
3) The payment switches to generate (about 16 hours)

Thanks for the heads up!
But it is an altcoin, 1 minute blocks, 60 confirms. The pool has pulled around 40 blocks in the last 3 hours. Checking the transactions on the client, they are not showing up as 'immature', and via the block explorer, the payments were sent to that strange address. The split was 95.5% going to that address, .5% going to the only miner on the pool.

I'll check it again in an hour, just to be sure. Also, if this is the wrong thread to post this, let me know where it should be posted.

[kano]

Nice, 2 more blocks last night while I was asleep ... I got one of them
(my first block in MANY months)

Edit: ... and I've spent some time now (since that post) working out why my block sux.
It is only 16k and only had 28 txn in it.
My bitcoind (default 0.8.3 settings for txns etc except: paytxfee=0.0005) on the other hand at the time had 1854 in it's mempool:

Code:

2013-07-25 16:25:43 CTxMemPool::accept() : accepted 7b1f0fd72e281abdb9b6724aa839b872550a3e7ff8de395dfbe20cd92777d4ae (poolsz 1854)
2013-07-25 16:25:43 ThreadRPCServer method=submitblock
.
.
2013-07-25 16:25:44 SetBestChain: new best=000000000000006e312482364ffc008b8fabad506fefde343a1acfc1f64ae982  height=248394  log2_work=70.874866  tx=21101483  date=2013-07-25 16:24:32 progress=0.999995
2013-07-25 16:25:44 ProcessBlock: ACCEPTED
.
.
2013-07-25 16:25:44 CTxMemPool::accept() : accepted c75dd649944be1e262c3c82fda612e29c1631b74b7563f0142e28486623c2217 (poolsz 1828)

... anyone care to explain that before I switch over to some other pool ... it really does look like it's p2pool itself that decided to produce that crap block ... unless I've misunderstood somewhere ... or are there hidden options in p2pool that I need to fix and the default settings suck?

[gyverlb]

Nice, 2 more blocks last night while I was asleep ... I got one of them
(my first block in MANY months)

Edit: ... and I've spent some time now (since that post) working out why my block sux.
It is only 16k and only had 28 txn in it.
My bitcoind (default 0.8.3 settings for txns etc except: paytxfee=0.0005) on the other hand at the time had 1854 in it's mempool:

Code:

2013-07-25 16:25:43 CTxMemPool::accept() : accepted 7b1f0fd72e281abdb9b6724aa839b872550a3e7ff8de395dfbe20cd92777d4ae (poolsz 1854)
2013-07-25 16:25:43 ThreadRPCServer method=submitblock
.
.
2013-07-25 16:25:44 SetBestChain: new best=000000000000006e312482364ffc008b8fabad506fefde343a1acfc1f64ae982  height=248394  log2_work=70.874866  tx=21101483  date=2013-07-25 16:24:32 progress=0.999995
2013-07-25 16:25:44 ProcessBlock: ACCEPTED
.
.
2013-07-25 16:25:44 CTxMemPool::accept() : accepted c75dd649944be1e262c3c82fda612e29c1631b74b7563f0142e28486623c2217 (poolsz 1828)

... anyone care to explain that before I switch over to some other pool ... it really does look like it's p2pool itself that decided to produce that crap block ... unless I've misunderstood somewhere ... or are there hidden options in p2pool that I need to fix and the default settings suck?

Just after a block is broadcasted on the bitcoin network there is less juicy txs on the network to include: if a block is found just after another one it may not have much tx fees.

Another rare cause for low income blocks : pools (not only p2pool) use a trick to avoid miners working on stale coinbases: when they detect a new block they immediately ask for (or maybe compute themselves) an empty coinbase which is really fast to compute and makes it possible for miners to do useful work as soon as possible. Then they ask for a "normal" coinbase which is slower.

You can check what your bitcoind includes in its coinbase with

Code:

bitcoind getmininginfo

: currentblocktx is the number of tx in the coinbase and is obviously lower than pooledtx (the number in your memory pool).

paytxfee only affects what your node includes in its own transactions. To make juicier blocks (to maximize profit and speed up transaction confirmations everyone on p2pool should probably use this if they don't have any out-of-the-ordinary setup):

Code:

mintxfee=0.00001
minrelaytxfee=0.00001
blockmaxsize=1000000

[kano]

...
... anyone care to explain that before I switch over to some other pool ... it really does look like it's p2pool itself that decided to produce that crap block ... unless I've misunderstood somewhere ... or are there hidden options in p2pool that I need to fix and the default settings suck?

Just after a block is broadcasted on the bitcoin network there is less juicy txs on the network to include: if a block is found just after another one it may not have much tx fees.

... the previous block was 86 seconds before it ...

Code:

2013-07-25 16:24:17 CTxMemPool::accept() : accepted 529cf3fc055a54a2c73dd04c9fddd948c92e955ebd4383faf3d71562a5164d18 (poolsz 1860)
2013-07-25 16:24:17 received block 000000000000006de7e7d91c59bbcbd7578b09a81a2241192c8accd98d076f6e
2013-07-25 16:24:17 Committing 2522 changed transactions to coin database...
2013-07-25 16:24:18 SetBestChain: new best=000000000000006de7e7d91c59bbcbd7578b09a81a2241192c8accd98d076f6e  height=248393  log2_work=70.874776  tx=21101455  date=2013-07-25 16:23:19 progress=0.999996
2013-07-25 16:24:18 ProcessBlock: ACCEPTED
.
.
2013-07-25 16:24:18 CTxMemPool::accept() : accepted e2d20ab0608ec065214f8c24ff2505ac4d6a2cd1a5e3904d61aff1bc80e2f39a (poolsz 1776)

Another rare cause for low income blocks : pools (not only p2pool) use a trick to avoid miners working on stale coinbases: when they detect a new block they immediately ask for (or maybe compute themselves) an empty coinbase which is really fast to compute and makes it possible for miners to do useful work as soon as possible. Then they ask for a "normal" coinbase which is slower.

It's not empty, so got nothing to do with it ... as you can easily have seen by looking at the 3 blocks mined in the last 24 hours without having to guess which is the block I found ... even though my block is listed there in my post

You can check what your bitcoind includes in its coinbase with

Code:

bitcoind getmininginfo

: currentblocktx is the number of tx in the coinbase and is obviously lower than pooledtx (the number in your memory pool).

Code:

# ./bitcoind getmininginfo
{
    "blocks" : 248451,
    "currentblocksize" : 77889,
    "currentblocktx" : 133,
    "difficulty" : 31256960.72776893,
    "errors" : "",
    "generate" : false,
    "genproclimit" : -1,
    "hashespersec" : 0,
    "pooledtx" : 1759,
    "testnet" : false
}
# 

Well that says something evil about bitcoind ...

paytxfee only affects what your node includes in its own transactions.

Yes I know

To make juicier blocks (to maximize profit and speed up transaction confirmations everyone on p2pool should probably use this if they don't have any out-of-the-ordinary setup):

Code:

mintxfee=0.00001
minrelaytxfee=0.00001
blockmaxsize=1000000

The defaults (which are my settings since I haven't changed them - I only upgraded from 0.7.0 to 0.8.3 on Jul-8 ) are:
mintxfee=10000 = 0.00010000
minrelaytxfee=10000 = 0.00010000
blockmaxsize=250000

... looks like I need to decrease those 2 'min' values to at least stop bitcoind from screwing over the BTC network ...
Imagine how bad it must be to the BTC network for people who INCREASE those values ...

Someone else I was discussing this with suggested that p2pool decides to drops transactions itself based on performance, it wont use all the txns given to it by bitcoind ... is that correct?
Interestingly related to that is that python sucks badly.
On my Quad core Q9300 2.5GH/s with 8GB of RAM, that is running p2pool, it sometimes hits 9x% CPU ... ouch ... but will never go beyond that since it is single threaded ... so a low power CPU will indeed suck badly in p2pool CPU performance during e.g. block changes

[gyverlb]

... the previous block was 86 seconds before it ...

So that was a fast block, probably expected to get 86 / 600 as much transaction fees as an average block.

It's not empty, so got nothing to do with it ... as you can easily have seen by looking at the 3 blocks mined in the last 24 hours without having to guess which is the block I found ... even though my block is listed there in my post

Probably but I'm lazy...

You can check what your bitcoind includes in its coinbase with

Code:

bitcoind getmininginfo

: currentblocktx is the number of tx in the coinbase and is obviously lower than pooledtx (the number in your memory pool).

Code:

# ./bitcoind getmininginfo
{
    "blocks" : 248451,
    "currentblocksize" : 77889,
    "currentblocktx" : 133,
    "difficulty" : 31256960.72776893,
    "errors" : "",
    "generate" : false,
    "genproclimit" : -1,
    "hashespersec" : 0,
    "pooledtx" : 1759,
    "testnet" : false
}
# 

Well that says something evil about bitcoind ...

Not really evil, txs that aren't included with default bitcoind settings pay below-minimum fees. Some pools obviously have more permissive settings and it probably pays off.

... looks like I need to decrease those 2 'min' values to at least stop bitcoind from screwing over the BTC network ...

Your bitcoind isn't really screwing anyone as it includes txs paying the minimum fees. TXs with lower fees are eventually included (older inputs get higher priority and there's provision for txs with no fees in a block), not giving a standard fee gets the expected outcome: slower processing by the Bitcoin network.

Someone else I was discussing this with suggested that p2pool decides to drops transactions itself based on performance, it wont use all the txns given to it by bitcoind ... is that correct?

I'll bet not. I didn't check the code but:

• p2pool logs match what you would expect when modifying bitcoind settings
• it's easier (and probably faster) to let bitcoind compute the coinbase instead of fiddling with it
• my node has the settings given in my previous mail which includes more txs than average and it is sending more data to other p2pool nodes than it receives, which matches the expectation that it processes more transaction and need to forward them to other nodes so that they can verify its shares

Interestingly related to that is that python sucks badly.
On my Quad core Q9300 2.5GH/s with 8GB of RAM, that is running p2pool, it sometimes hits 9x% CPU ... ouch ... but will never go beyond that since it is single threaded ... so a low power CPU will indeed suck badly in p2pool CPU performance during e.g. block changes

python doesn't suck badly, it just isn't designed for HPC. Bad CPU performance isn't necessarily a show stopper: as long as p2pool finishes to process data before it's needed or not delay the process waiting too much (almost everything happens asynchronously in p2pool) it's fine. Recent changes have helped in this regard: p2pool can prepare work before it's needed with the new protocol. So it could be at 100% CPU for a whole second preparing work (that's assuming a slow CPU with multiple miners using a different payout address) and only delay the workbase communication to miners by ~3%.

That said I actually don't like the language itself but that's a matter of taste.

[bicer]

...Bad CPU performance isn't necessarily a show stopper...

upgrade from Pentium D 915 2.8Ghz (dual core) to i7 3.2Ghz reduced GBT latency by 1s on unchanged OS (linux), p2pool, and bitcoind config.

... the previous block was 86 seconds before it ...

So that was a fast block, probably expected to get 86 / 600 as much transaction fees as an average block.

It's not empty, so got nothing to do with it ... as you can easily have seen by looking at the 3 blocks mined in the last 24 hours without having to guess which is the block I found ... even though my block is listed there in my post

Probably but I'm lazy...

You can check what your bitcoind includes in its coinbase with

Code:

bitcoind getmininginfo

: currentblocktx is the number of tx in the coinbase and is obviously lower than pooledtx (the number in your memory pool).

Code:

# ./bitcoind getmininginfo
{
    "blocks" : 248451,
    "currentblocksize" : 77889,
    "currentblocktx" : 133,
    "difficulty" : 31256960.72776893,
    "errors" : "",
    "generate" : false,
    "genproclimit" : -1,
    "hashespersec" : 0,
    "pooledtx" : 1759,
    "testnet" : false
}
# 

Well that says something evil about bitcoind ...

Not really evil, txs that aren't included with default bitcoind settings pay below-minimum fees. Some pools obviously have more permissive settings and it probably pays off.

... looks like I need to decrease those 2 'min' values to at least stop bitcoind from screwing over the BTC network ...

Your bitcoind isn't really screwing anyone as it includes txs paying the minimum fees. TXs with lower fees are eventually included (older inputs get higher priority and there's provision for txs with no fees in a block), not giving a standard fee gets the expected outcome: slower processing by the Bitcoin network.

Someone else I was discussing this with suggested that p2pool decides to drops transactions itself based on performance, it wont use all the txns given to it by bitcoind ... is that correct?

I'll bet not. I didn't check the code but:

• p2pool logs match what you would expect when modifying bitcoind settings
• it's easier (and probably faster) to let bitcoind compute the coinbase instead of fiddling with it
• my node has the settings given in my previous mail which includes more txs than average and it is sending more data to other p2pool nodes than it receives, which matches the expectation that it processes more transaction and need to forward them to other nodes so that they can verify its shares

Interestingly related to that is that python sucks badly.
On my Quad core Q9300 2.5GH/s with 8GB of RAM, that is running p2pool, it sometimes hits 9x% CPU ... ouch ... but will never go beyond that since it is single threaded ... so a low power CPU will indeed suck badly in p2pool CPU performance during e.g. block changes

python doesn't suck badly, it just isn't designed for HPC. Bad CPU performance isn't necessarily a show stopper: as long as p2pool finishes to process data before it's needed or not delay the process waiting too much (almost everything happens asynchronously in p2pool) it's fine. Recent changes have helped in this regard: p2pool can prepare work before it's needed with the new protocol. So it could be at 100% CPU for a whole second preparing work (that's assuming a slow CPU with multiple miners using a different payout address) and only delay the workbase communication to miners by ~3%.

That said I actually don't like the language itself but that's a matter of taste.

[gyverlb]

...Bad CPU performance isn't necessarily a show stopper...

upgrade from Pentium D 915 2.8Ghz (dual core) to i7 3.2Ghz reduced GBT latency by 1s on unchanged OS (linux), p2pool, and bitcoind config.

That's higher than I would have expected, are you using bitcoind 0.8.3? If not, you should upgrade.

GBT latency is latency in bitcoind not p2pool and it doesn't matter much as forrestv pointed out months ago and tests confirmed.

That said, on a dual core which is used for some other CPU intensive tasks, you may experience lower efficiency (you need to have free CPU time for p2pool *and* bitcoind). A quad-core will probably handle this without lowering efficiency (count one core for bitcoind, one for p2pool and other ones as needed for the CPU intensive tasks you want to run in parallel).

[HellDiverUK]

I'm starting to set up p2pool on my little spare machine.  Bitcoin-QT is using 98% CPU usage while downloading the blockchain.  Is this a bad sign for the future?  Do I abandon the idea of running p2pool on this machine, and run it off something with a bit more go?  Drive is a Samsung 840 250GB SSD, and it has 4GB RAM.

[rav3n_pl]

I'm starting to set up p2pool on my little spare machine.  Bitcoin-QT is using 98% CPU usage while downloading the blockchain. 

It is normal.

[HellDiverUK]

Got it up and running on the Celeron 847.  Runs like crap.    One core is basically running flat out for p2pool.exe all the time.

I think I'll swap the drive over to a faster machine (Pentium G620) when I get home, and see how it goes on there.

Initial impressions seem good though.

I might have missed it, but how do I get the fancier web page running?  The p2pool page is just a bunch of tables and stuff at the moment.  I don't see graphs or the likes?

Running OK now, it took a little while to settle down. 

I also found the graphs.  I'm running a browser on a Remote Desktop session to my p2p machine via my home PC via LogMeIn, which is reducing 1080p res to 1280x1024 for work PC.  Slow, laggy and tiny. 

[HellDiverUK]

Idk the celerons, atoms and i5s I've used are trash

I'm running p2pool/bitcoind/altcoinds/cgminer on an Intel Next Unit of Computing with a Celeron 847 @ 1.10GHz, 4GB RAM with 4x AsicMiner USB Eruptors. I had to move the blockchains (bitcoin, devcoin, namecoin, ixcoin) to an external USB hard drive (system drive is 32GB SSD). It took some tweaking, but my get work latency is ~0.2-0.3 and I see DOA of 1.5% and 101.5% efficiency. CPU load seems pretty low (~80-90% idle mostly). Seems fine to me. Biggest problem is that my share luck seems to be inversely proportional to p2pool's block luck.

The NUC is brilliant, I have one here as my HTPC (it's the i3 version).  I've got it in a Tranquil passive case.  Totally silent.

The 847 I have is on a MiniITX Gigabyte board.  Current specs are 4GB RAM, 250GB Samsung 840 SSD.  In the hour or so it's been running, it seems to be doing OK.  I'll give it a few days and see how it performs, I have plenty of spare hardware I can throw at it.  

I'm running my miners on a different machine, though that may change.  I might run the ASICMiners off this machine, too, if it can cope.  Leave my main PC to do it's thing.

[dc81]

i'm running bitcoind/litecoind and p2pool for each on an ODRIOD-U2 without any issues. it's a quad core 1.7ghz arm with 2 gb ram ~$120 shipped. it's pretty much like a pi on steroids.

also, i put in a pull request to change the links to blockchain.info instead of blockexplorer, which doesn't even seem to be working anymore.

[bitpop]

Man I want the i7 4770 haswell

[forrestv]

Nice, 2 more blocks last night while I was asleep ... I got one of them
(my first block in MANY months)

Edit: ... and I've spent some time now (since that post) working out why my block sux.
It is only 16k and only had 28 txn in it.
My bitcoind (default 0.8.3 settings for txns etc except: paytxfee=0.0005) on the other hand at the time had 1854 in it's mempool:

Code:

2013-07-25 16:25:43 CTxMemPool::accept() : accepted 7b1f0fd72e281abdb9b6724aa839b872550a3e7ff8de395dfbe20cd92777d4ae (poolsz 1854)
2013-07-25 16:25:43 ThreadRPCServer method=submitblock
.
.
2013-07-25 16:25:44 SetBestChain: new best=000000000000006e312482364ffc008b8fabad506fefde343a1acfc1f64ae982  height=248394  log2_work=70.874866  tx=21101483  date=2013-07-25 16:24:32 progress=0.999995
2013-07-25 16:25:44 ProcessBlock: ACCEPTED
.
.
2013-07-25 16:25:44 CTxMemPool::accept() : accepted c75dd649944be1e262c3c82fda612e29c1631b74b7563f0142e28486623c2217 (poolsz 1828)

... anyone care to explain that before I switch over to some other pool ... it really does look like it's p2pool itself that decided to produce that crap block ... unless I've misunderstood somewhere ... or are there hidden options in p2pool that I need to fix and the default settings suck?

P2Pool sometimes has to limit the number of transactions in a block because of per-share tx inclusion limits. Only 50kB of new transactions are allowed per-share, so it takes time the pool of allowed share to build up.

[gyverlb]

Nice, 2 more blocks last night while I was asleep ... I got one of them
(my first block in MANY months)

Edit: ... and I've spent some time now (since that post) working out why my block sux.
It is only 16k and only had 28 txn in it.
My bitcoind (default 0.8.3 settings for txns etc except: paytxfee=0.0005) on the other hand at the time had 1854 in it's mempool:

Code:

2013-07-25 16:25:43 CTxMemPool::accept() : accepted 7b1f0fd72e281abdb9b6724aa839b872550a3e7ff8de395dfbe20cd92777d4ae (poolsz 1854)
2013-07-25 16:25:43 ThreadRPCServer method=submitblock
.
.
2013-07-25 16:25:44 SetBestChain: new best=000000000000006e312482364ffc008b8fabad506fefde343a1acfc1f64ae982  height=248394  log2_work=70.874866  tx=21101483  date=2013-07-25 16:24:32 progress=0.999995
2013-07-25 16:25:44 ProcessBlock: ACCEPTED
.
.
2013-07-25 16:25:44 CTxMemPool::accept() : accepted c75dd649944be1e262c3c82fda612e29c1631b74b7563f0142e28486623c2217 (poolsz 1828)

... anyone care to explain that before I switch over to some other pool ... it really does look like it's p2pool itself that decided to produce that crap block ... unless I've misunderstood somewhere ... or are there hidden options in p2pool that I need to fix and the default settings suck?

P2Pool sometimes has to limit the number of transactions in a block because of per-share tx inclusion limits. Only 50kB of new transactions are allowed per-share, so it takes time the pool of allowed share to build up.

I didn't have a clue p2pool controlled the precise list of transactions in the coinbase. I assumed it was all or nothing as the expected block income as reported by p2pool was rising after a bitcoind restart (with more transactions being learned by bitcoind) and fees only briefly were 0 after a new block, rising sharply immediately after it and resuming a slow rise (as I assumed, transactions kept coming in).

Is the pool emptied at each new block? 50kB every share is 1MB/average block interval. This is not enough to build full blocks half the time if the pool is emptied though.

Could you take the time to explain how p2pool handles transactions in more details (and maybe paste it on the bitcoin wiki page for p2pool if you have the rights to do so)? One of many questions is how it behaves when nodes don't have the same bitcoind settings (meaning they don't process the same transactions).

[HellDiverUK]

I had a look at things when I got home, and I've set up a Pentium G620 (dual core 2.6GHz Sandy Bridge) to run both the p2pool and also it's running the Block Erupters.  It'll run the Jalapeno if it ever appears, too.   The Celeron was maybe a little too slow - might have been OK running Linux, but I don't have the patience to set it up that.

Pool is available now (hopefully) on http://847pool.no-ip.biz:31337 for web stats; port 8333 for mining.