whalingoutbox
Newbie
Offline
Activity: 10
Merit: 0
|
|
May 20, 2017, 08:47:40 PM |
|
Hello,
I'm still trying to understand how P2Pool works. How do I find a list of P2Pool servers? The idea is that I make a node and connect to a server, right? I can't quite visualize how all of this works, despite looking on Google for information visualizations. The image I saw on the Bitcoin wiki didn't make much sense, seemed too technical.
There are multiple servers, right?
Is there like one main server? Or do a bunch of nodes together make the server?
I read and read but it doesn't stick/make-sense. So, hopefully, I can talk to people about this.
|
|
|
|
in2tactics
|
|
May 20, 2017, 10:39:56 PM |
|
Hello,
I'm still trying to understand how P2Pool works. How do I find a list of P2Pool servers? The idea is that I make a node and connect to a server, right? I can't quite visualize how all of this works, despite looking on Google for information visualizations. The image I saw on the Bitcoin wiki didn't make much sense, seemed too technical.
There are multiple servers, right?
Is there like one main server? Or do a bunch of nodes together make the server?
I read and read but it doesn't stick/make-sense. So, hopefully, I can talk to people about this.
Each node is its own pool. p2pool is a collection of pools. You can either make your own node or use another.
|
Current HW: 2x Apollo, 2x Apollo BTC, 2x Apollo II Retired HW: 3x 2PAC, 3x Moonlander 2, 2x AntMiner S7-LN, 5x AntMiner U1, 2x ASICMiner Block Erupter Cube, 4x AntMiner S3, 4x AntMiner S1, GAW Black Widow, and ZeusMiner Thunder X6
|
|
|
frodocooper
|
|
May 21, 2017, 12:12:51 AM |
|
I'm still trying to understand how P2Pool works. How do I find a list of P2Pool servers? The idea is that I make a node and connect to a server, right? I can't quite visualize how all of this works, despite looking on Google for information visualizations. The image I saw on the Bitcoin wiki didn't make much sense, seemed too technical.
This P2Pool guide should be able to help. It's a lot more beginner-friendly than the Bitcoin wiki's article. And here is an up-to-date list of public P2Pool nodes that you can connect your miners to, if you decide to not run your own P2Pool node. There are multiple servers, right?
Yes, in a sense. There are multiple P2Pool nodes, and all of them make up the P2Pool network. It is very similar to how the Bitcoin network works, where multiple Bitcoin full nodes make up the Bitcoin network. Is there like one main server? Or do a bunch of nodes together make the server?
P2Pool is a decentralized pool, similar to how Bitcoin is a decentralized network. There is therefore no main or central server, nor nodes that make up a main or central server. There is only the network of P2Pool nodes. In other words, try to look at it according to what in2tactics said: each P2Pool node is its own pool. The P2Pool network connects these nodes together through the P2Pool sharechain, similar to how the Bitcoin network connects every Bitcoin full node together through the Bitcoin blockchain. P2Pool can therefore also be described as a collection of solo miners that pool block payouts and distribute them accordingly, since each P2Pool node is essentially doing its own thing. Contrast this to a traditional pool, where the central pool server dictates what its miners do.
|
|
|
|
jtoomim
|
|
May 21, 2017, 09:28:23 PM |
|
Quick reminder: the jtoomimnet 1mb_hardforked branch of p2pool uses about 2x as much RAM, since the share chain contains around 2x as many transactions. If you are using pypy, this means that your memory consumption may get up to 6 GB. If you're using CPython, it's a little under 1 GB. Please make sure that you have enough RAM in your nodes. If you don't have enough RAM, you will get massive swapping, and you might notice that bitcoind is unable to keep up with blocks and may fall behind, causing p2pool to stop working.
Lowering RAM usage is my next goal for p2pool.
|
Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power. http://Toom.im
|
|
|
tubexc
|
|
May 21, 2017, 11:41:22 PM |
|
Quick reminder: the jtoomimnet 1mb_hardforked branch of p2pool uses about 2x as much RAM, since the share chain contains around 2x as many transactions. If you are using pypy, this means that your memory consumption may get up to 6 GB. If you're using CPython, it's a little under 1 GB. Please make sure that you have enough RAM in your nodes. If you don't have enough RAM, you will get massive swapping, and you might notice that bitcoind is unable to keep up with blocks and may fall behind, causing p2pool to stop working.
Lowering RAM usage is my next goal for p2pool.
That's good. And what about the p2pool ltc node testing and then the merge into only one p2pool powerfull bitcoin sharechain?
|
|
|
|
jtoomim
|
|
May 22, 2017, 04:27:32 AM |
|
And what about the p2pool ltc node testing
LTC now requires SegWit, which requires veqtrus's PR. It would take some more work to get LTC working with both my code and veqtrus's. It would need to be a different alt for testing. At the moment, I'm more interested in the memory issues and the share size issues than in doing testing and merging. I'd also like to add some code that checks the system clock against NTP (when reachable) at startup in order to reduce the clock offset issues that people have been having, and I think that might be good to do before merging into p2pool master.
|
Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power. http://Toom.im
|
|
|
|
KorbinDallas
Newbie
Offline
Activity: 55
Merit: 0
|
|
May 25, 2017, 10:17:27 PM |
|
Queue the block dance guy ......
|
|
|
|
|
in2tactics
|
|
May 26, 2017, 12:30:29 PM |
|
Well, there was only 24 seconds between block 468172 and 468173.
|
Current HW: 2x Apollo, 2x Apollo BTC, 2x Apollo II Retired HW: 3x 2PAC, 3x Moonlander 2, 2x AntMiner S7-LN, 5x AntMiner U1, 2x ASICMiner Block Erupter Cube, 4x AntMiner S3, 4x AntMiner S1, GAW Black Widow, and ZeusMiner Thunder X6
|
|
|
|
jtoomim
|
|
May 26, 2017, 06:53:12 PM Last edit: May 26, 2017, 07:28:39 PM by jtoomim |
|
It's more an issue of processing and propagation latency than mempool size. P2pool (along with most other pools) has code in it to generate empty blocks when the pool has heard of a new block header on the network, but has not yet gotten a block template from bitcoind. bitcoind can't generate a new block template until after it has fully downloaded and verified the previous block, since otherwise it might include transactions twice or include double-spends. Since downloading a block header is faster than downloading and verifying a full block, there can sometimes be a substantial amount of time in between where the best you can do is mine an empty block. The idea is that if you have a miner hashing on something, it's better to hash on an empty block at the right height than a full block at the wrong height (which would result in an orphan race). This condition should only last for a few seconds, though. The 23 actual seconds that elapsed seems excessive. It's possible that I need to do something to improve block propagation latency on that node. It's also possible that p2pool didn't poll getblocktemplate frequently enough or something. If this happens when blocks are found in the first 30 15 seconds of mining, that would correspond to a 5% 2.5% reduction in average transactions included. That seems undesirable, but not excessive. The alternative -- mining on the wrong block height for up to 15 seconds -- sounds worse. I wish I had time to look into this more and improve performance, but at the moment I don't. Edit: It appears that the timestamps on the blocks are wrong. This is pretty common, as generally timestamps are set when the stratum job is issued to the miner, not when the miner actually finds the block. Here's what shows up in my bitcoind logs: 2017-05-26 05:03:32 UpdateTip: new best=0000000000000000019c0b9124dd971ca31d59deac39f23cac484495895eb2af height=468 172 log2_work=86.489404 tx=226059198 date=2017-05-26 05:02:56 progress=1.000000 cache=144.7MiB(44861tx) 2017-05-26 05:03:32 CreateNewBlock(): total size 999899 txs: 2197 fees: 296309213 sigops 7618 2017-05-26 05:03:35 Acceptable block: ver:20000000 time:1495775000 size: 1623 Tx:1 Sig:41 2017-05-26 05:03:35 UpdateTip: new best=00000000000000000117d37aba3cd8d9d1811f4f719834f5b8f71c78f8c504d7 height=468173 log2_work=86.489438 tx=226059199 date=2017-05-26 05:03:20 progress=1.000000 cache=144.7MiB(44877tx)
Then, in my p2pool logs: 2017-05-25 22:03:20.093931 Skipping from block 156cb130cb9ee78a01dd6d5ed4f144fb05c222f117829e7 to block 19c0b9124dd971ca31d59deac39f23cac484495895eb2af! 2017-05-25 22:03:20.102054 New work for 1PXxBrUbWUMZemAQknEqDTSHKaKmVmhJCK! Diff: 65681.48 Share diff: 9893593.66 Block value: 12.50 BTC (0 tx, 0 kB) 2017-05-25 22:03:34.275890 Generating a share with 998819 bytes (348901 new) and 2197 transactions (678 new) 2017-05-25 22:03:34.316686 New work for 1PXxBrUbWUMZemAQknEqDTSHKaKmVmhJCK! Diff: 69877.30 Share diff: 9893593.66 Block value: 15.46 BTC (2197 tx, 999 kB) 2017-05-25 22:03:34.384701 Generating a share with 998819 bytes (348901 new) and 2197 transactions (678 new) 2017-05-25 22:03:34.788256 2017-05-25 22:03:34.788319 GOT BLOCK FROM MINER! Passing to bitcoind! https://blockchain.info/block/00000000000000000117d37aba3cd8d9d1811f4f719834f5b8f71c78f8c504d7 2017-05-25 22:03:34.788340 2017-05-25 22:03:34.791621 GOT SHARE! 1EyWF5ZQ9BHxbLAKuFj2MfQT9daE1sVsTx f8c504d7 prev 91af17f0 age 14.68s DEAD ON ARRIVAL 2017-05-25 22:03:34.802307
It's worth noting that this was a DOA share that happened to be a block. P2pool learned of the block at height 468172 at 3m20s, bitcoind learned of it 12 seconds later at 3m32s and had work for p2pool around that time, but p2pool didn't have it bundled into a stratum job until 3m34.7s. While p2pool was bundling the stratum job, a miner returned the block from the previous stratum job, and p2pool didn't process it until after it finished assigning the new work because p2pool is single-threaded. This means that the block was found at some point between 3m32s and 3m34.7s. This node was running CPython, not pypy, because that server is running low on RAM. This may have contributed to the slow (2.7s) processing of new work, and increases the likelihood of a DOA share. So in this case, we're talking about a 15 second time window, not 30 or 24. Of those 15 seconds, 12 seconds were due to bitcoind being a bit slow, and 3 seconds were due to p2pool being slow. Overall, I think finding an empty block in this scenario is a reasonable outcome, and do not consider this to be a priority for optimization right now.
|
Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power. http://Toom.im
|
|
|
kano
Legendary
Offline
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
|
|
May 26, 2017, 10:40:40 PM Last edit: May 27, 2017, 01:00:58 AM by kano |
|
It's more an issue of processing and propagation latency than mempool size. P2pool (along with most other pools) has code in it to generate empty blocks when the pool has heard of a new block header on the network, but has not yet gotten a block template from bitcoind. ...
This is based on the fact that LukeJr is a crappy coder and came up with the idea that a block change with transactions is slow, so do not put transaction in the first work. This is of course bullshit.
|
|
|
|
jtoomim
|
|
May 27, 2017, 02:49:29 AM |
|
I'm having trouble following your assertion. Is 14.68 seconds not slow to you? Or is it bullshit that it's so slow? Or is it bullshit to mine transaction-less blocks during those 14.68 seconds? Or is it just bullshit because Luke-jr was involved?
|
Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power. http://Toom.im
|
|
|
kano
Legendary
Offline
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
|
|
May 27, 2017, 05:33:35 AM |
|
I'm having trouble following your assertion. Is 14.68 seconds not slow to you? Or is it bullshit that it's so slow? Or is it bullshit to mine transaction-less blocks during those 14.68 seconds? Or is it just bullshit because Luke-jr was involved?
Takes me less than 1 second - so I guess you have the same problem as LukeJr. (less than 1 second includes: from block arrives, processed, work generated, new work sent out to miners)
|
|
|
|
jtoomim
|
|
May 27, 2017, 06:16:03 AM |
|
Takes me less than 1 second - so I guess you have the same problem as LukeJr. (less than 1 second includes: from block arrives, processed, work generated, new work sent out to miners)
We're not counting the same way. In my example above, it took 12 seconds (after p2pool received the header) for the block to arrive, less than one second for the block to be validated and a new block template to be pushed to p2pool, and 2.7 seconds (because CPython p2pool is slow) for p2pool to issue new work to the miners. You're not counting what took 12 seconds in the case above.
|
Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power. http://Toom.im
|
|
|
kano
Legendary
Offline
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
|
|
May 27, 2017, 09:22:49 AM |
|
Takes me less than 1 second - so I guess you have the same problem as LukeJr. (less than 1 second includes: from block arrives, processed, work generated, new work sent out to miners)
We're not counting the same way. In my example above, it took 12 seconds (after p2pool received the header) for the block to arrive, less than one second for the block to be validated and a new block template to be pushed to p2pool, and 2.7 seconds (because CPython p2pool is slow) for p2pool to issue new work to the miners. You're not counting what took 12 seconds in the case above. Thus you are saying there's some problem with p2pool receiving blocks. That 12 seconds doesn't exist for me. Edit: here's a link to make you worry more about p2pool ... https://poolbench.antminer.link/
|
|
|
|
HeroC
Legendary
Offline
Activity: 858
Merit: 1000
|
|
May 27, 2017, 03:11:18 PM |
|
Takes me less than 1 second - so I guess you have the same problem as LukeJr. (less than 1 second includes: from block arrives, processed, work generated, new work sent out to miners)
We're not counting the same way. In my example above, it took 12 seconds (after p2pool received the header) for the block to arrive, less than one second for the block to be validated and a new block template to be pushed to p2pool, and 2.7 seconds (because CPython p2pool is slow) for p2pool to issue new work to the miners. You're not counting what took 12 seconds in the case above. Thus you are saying there's some problem with p2pool receiving blocks. That 12 seconds doesn't exist for me. Edit: here's a link to make you worry more about p2pool ... https://poolbench.antminer.link/Maybe it's just my internet, but that link you sent never actually loaded for me. It's just been loading for like a minute. EDIT: Another minute later it loaded, nevermind
|
|
|
|
jtoomim
|
|
May 27, 2017, 11:29:49 PM |
|
Thus you are saying there's some problem with p2pool receiving blocks. That 12 seconds doesn't exist for me.
No, the 12 seconds was due to my bitcoind process, not p2pool, as you would know if you had read all of my original post. I'm not running Falcon or FIBRE on my node, just theblumatt's old relay network, which is probably why it took 12 seconds. If you think it is bullshit that it took 12 seconds to download that block, then I'm inclined to agree with you. If you think that it's bullshit that p2pool choose to mine an empty block during those 12+2.7 seconds, then I'm inclined to disagree with you.
|
Hosting bitcoin miners for $65 to $80/kW/month on clean, cheap hydro power. http://Toom.im
|
|
|
kano
Legendary
Offline
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
|
|
May 27, 2017, 11:32:49 PM |
|
Thus you are saying there's some problem with p2pool receiving blocks. That 12 seconds doesn't exist for me.
No, the 12 seconds was due to my bitcoind process, not p2pool, as you would know if you had read all of my original post. I'm not running Falcon or FIBRE on my node, just theblumatt's old relay network, which is probably why it took 12 seconds. If you think it is bullshit that it took 12 seconds to download that block, then I'm inclined to agree with you. If you think that it's bullshit that p2pool choose to mine an empty block during those 12+2.7 seconds, then I'm inclined to disagree with you. No, I'm saying there is no reason to mine empty blocks due to the first work generated on a block change. If your stats say otherwise, then you need to fix your stats, not mine empty blocks to overcome a crappy setup or crappy code.
|
|
|
|
|