 Show Posts
|
|
Pages: [1] 2 »
|
Hello Is there any way to make p2pool mode modern so it can use multithreading and multi cores etc?
Rewrite it from scratch in another programming language. Python is inherently single-threaded. https://wiki.python.org/moin/GlobalInterpreterLockIt would be far easier and more effective to remove most of the transaction-handling code from p2pool, and make share objects contain only the merkle root plus the coinbase transaction (and the merkle path thereto). However, I have not had the time to do that, and nobody else seems to be interested in doing it. Even though it would be much easier, it's not super easy to do. Note: Bitcoin Core is *also* mostly single-threaded, although not quite to the same extent as p2pool. Wouldn't it be possible to rewrite the code and compile it with cython? This avoids the GIL. It also easily allows to mix python with C/C++ code, which might be useful for some parts of the code, and it would avoid rewriting everything in another language. However, I don't think twisted can be used by cython (I might be wrong on this), so this would need to be replaced with something different to handle the asynchronous networking (which might be quite a lot of work). I've been running the 1mb_segwit code for quite some time now. It seems that the optimalisations done by jtoomim and the use of pypy has reduced the single core load quite considerably compared to the main code. So, although some people still have problems, it alleviated in my opinion the single thread bottleneck considerably. @jtoomim: what parts of the transaction handling can be removed in your opinion? |
|
|
|
Cryptonomist, your node is handing out a difficulty of around 300k to your miners. The expected number of hashes needed to find a 300k diff pseudoshare is 300000*2**48/0xffff = 1.28 PH. With a hashrate of 80 GH/s, it's expected that would take about 4.47 hours per pseudoshare. Since p2pool needs to switch jobs every 30 seconds or so, that means that 99.82% of the time, p2pool would switch jobs before your miner found a (pseudo)share. I suggest you manually set a lower pseudoshare difficulty (e.g. 256, for about 14 sec per) by putting address+256 as your stratum username. This will only affect your stats, not your revenue. You can also put address+256/32768 to set your share difficulty target to either 32768 or the network's minimum share difficulty (whichever is higher) if you want to have a better chance of mining an actual share. Ideally, I would tweak my p2pool code so that it would adjust pseudoshare difficulty more quickly and intelligently to miners with different hashrates. However, that's pretty low on my priority list right now, since it does not affect revenue or resource requirements at all. Hello, @jtoomim: It was indeed just a difficulty issue. I've lowered the difficulty and now I get the local statistics in the p2pool output. Everything seems to work as expected. So thank you for the info. |
|
|
|
Cryptonomist, your CPU is faster than bitlock1's, and is probably fast enough to run jtoomimnet p2pool with pypy and with an acceptable (but not excellent) DOA+orphan rate, though of course faster would be better.
Edit: based on PMs, it looks like Cryptonomist's setup is working okay. jtoomimnet will often choose higher pseudoshare difficulty, especially on startup, and Cryptonomist is using USB miners with a very low hashrate (~80 GH/s), so he's only finding one stratum share (p2pool pseudoshare) every 15 minutes or so. But it's working.
Hello, So i'm no mining again on my own p2pool node using jtoomim's fork. However, there are still some things that aren't very clear to me. I've been running my miners and node now for a couple of hours, however, the p2pool output is still the following: 2017-12-11 13:57:56.476565 Peer sent entire transaction c714882643dfb355152cffe5ec79754a64c73610fb9c41edaccc8e4ca641c6b7 that was already received
2017-12-11 13:57:58.721058 Peer sent entire transaction c714882643dfb355152cffe5ec79754a64c73610fb9c41edaccc8e4ca641c6b7 that was already received
2017-12-11 13:57:59.927420 Peer sent entire transaction c714882643dfb355152cffe5ec79754a64c73610fb9c41edaccc8e4ca641c6b7 that was already received
2017-12-11 13:58:04.081785 Peer sent entire transaction 5fc8f4ccf5ccd68c4a0ca57a30fbca580061a75873b0bed4cbc9e8a441e5e64a that was already received
2017-12-11 13:58:05.791001 Peer sent entire transaction 5fc8f4ccf5ccd68c4a0ca57a30fbca580061a75873b0bed4cbc9e8a441e5e64a that was already received
2017-12-11 13:58:06.049836 Peer sent entire transaction 5fc8f4ccf5ccd68c4a0ca57a30fbca580061a75873b0bed4cbc9e8a441e5e64a that was already received
2017-12-11 13:58:06.285570 Peer sent entire transaction 5fc8f4ccf5ccd68c4a0ca57a30fbca580061a75873b0bed4cbc9e8a441e5e64a that was already received
2017-12-11 13:58:06.726747 Peer sent entire transaction 5fc8f4ccf5ccd68c4a0ca57a30fbca580061a75873b0bed4cbc9e8a441e5e64a that was already received
2017-12-11 13:58:12.124851 Peer sent entire transaction 5fc8f4ccf5ccd68c4a0ca57a30fbca580061a75873b0bed4cbc9e8a441e5e64a that was already received
2017-12-11 13:58:14.679518 Peer sent entire transaction 5fc8f4ccf5ccd68c4a0ca57a30fbca580061a75873b0bed4cbc9e8a441e5e64a that was already received
2017-12-11 13:58:14.902474 P2Pool: 17628 shares in chain (16240 verified/17632 total) Peers: 10 (4 incoming)
2017-12-11 13:58:14.902849 Local: 0H/s in last 0.0 seconds Local dead on arrival: ??? Expected time to share: ???
2017-12-11 13:58:14.903110 Shares: 0 (0 orphan, 0 dead) Stale rate: ??? Efficiency: ??? Current payout: (0.0000)=0.0000 BTC
2017-12-11 13:58:14.903413 Pool: 7173TH/s Stale rate: 3.2% Expected time to block: 11.0 days
2017-12-11 13:58:15.408087 Peer sent entire transaction 5fc8f4ccf5ccd68c4a0ca57a30fbca580061a75873b0bed4cbc9e8a441e5e64a that was already received
2017-12-11 13:58:18.516933 Peer sent entire transaction bea3223c82a3c08b75c9d51b7ce7a3bb1931d3dec355903a0d214ad50e569393 that was already received
2017-12-11 13:58:19.591504 Peer sent entire transaction 521b0e85fece2a4af6b84f9cd7a9a57b310a9a1b3a0349f74308f07ed33bf196 that was already received
2017-12-11 13:58:21.115704 Peer sent entire transaction 7f7e0343783c01bd435e6c2291d2c3ff662e7f2f011c20e63cda9753b5ef5e91 that was already received
2017-12-11 13:58:21.277624 Peer sent entire transaction 7f7e0343783c01bd435e6c2291d2c3ff662e7f2f011c20e63cda9753b5ef5e91 that was already received
2017-12-11 13:58:21.414326 Peer sent entire transaction 7f7e0343783c01bd435e6c2291d2c3ff662e7f2f011c20e63cda9753b5ef5e91 that was already received
2017-12-11 13:58:22.602042 Peer sent entire transaction 7f7e0343783c01bd435e6c2291d2c3ff662e7f2f011c20e63cda9753b5ef5e91 that was already received
2017-12-11 13:58:29.465659 Peer sent entire transaction eefc6be2c096c3f0a4b5a2f40b3db71dee318b094db1c4669560495b1f60a742 that was already received
2017-12-11 13:58:30.497847 Peer sent entire transaction 7f7e0343783c01bd435e6c2291d2c3ff662e7f2f011c20e63cda9753b5ef5e91 that was already received
2017-12-11 13:58:30.743429 Peer sent entire transaction eefc6be2c096c3f0a4b5a2f40b3db71dee318b094db1c4669560495b1f60a742 that was already received
2017-12-11 13:58:35.010332 Peer sent entire transaction eefc6be2c096c3f0a4b5a2f40b3db71dee318b094db1c4669560495b1f60a742 that was already received
2017-12-11 13:58:36.765627 Peer sent entire transaction eefc6be2c096c3f0a4b5a2f40b3db71dee318b094db1c4669560495b1f60a742 that was already received
2017-12-11 13:58:36.815290 Peer sent entire transaction eefc6be2c096c3f0a4b5a2f40b3db71dee318b094db1c4669560495b1f60a742 that was already received
2017-12-11 13:58:38.326642 Peer sent entire transaction eefc6be2c096c3f0a4b5a2f40b3db71dee318b094db1c4669560495b1f60a742 that was already received
2017-12-11 13:58:39.066717 Peer sent entire transaction eefc6be2c096c3f0a4b5a2f40b3db71dee318b094db1c4669560495b1f60a742 that was already received
2017-12-11 13:58:43.359680 Generating a share with 1009297 bytes (289609 new) and 572 transactions (281 new)
2017-12-11 13:58:43.454053 Warning: Previous share's timestamp is 205 seconds old.
2017-12-11 13:58:43.454353 Make sure your system clock is accurate, and ensure that you're connected to decent peers.
2017-12-11 13:58:43.454537 If your clock is more than 300 seconds behind, it can result in orphaned shares.
2017-12-11 13:58:43.454685 (It's also possible that this share is just taking a long time to mine.)
2017-12-11 13:58:43.510764 New work for xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx Diff: 346418.14 Share diff: 49058729.28 Block value: 13.78 BTC (572 tx, 1009 kB)
2017-12-11 13:58:43.999881 Generating a share with 1009297 bytes (289609 new) and 572 transactions (281 new)
2017-12-11 13:58:44.095464 Warning: Previous share's timestamp is 206 seconds old.
2017-12-11 13:58:44.095825 Make sure your system clock is accurate, and ensure that you're connected to decent peers.
2017-12-11 13:58:44.095977 If your clock is more than 300 seconds behind, it can result in orphaned shares.
2017-12-11 13:58:44.096125 (It's also possible that this share is just taking a long time to mine.)
2017-12-11 13:58:44.226306 Peer sent entire transaction c1200722f49b5865e71c72509ba7378a720b04597208b3ea84dcc2d2736ed1d0 that was already received
2017-12-11 13:58:44.325999 Peer sent entire transaction 896452b94905c8bab594c088a3ae17f99a286f3e0d4514c31ab8ae52e84cfe23 that was already received
2017-12-11 13:58:44.494081 Peer sent entire transaction 688583e03e9043bbd6566c9ef5da8f0550d8a8b960f0b9239a94e4fc99887c5e that was already received
2017-12-11 13:58:44.671330 Peer sent entire transaction 896452b94905c8bab594c088a3ae17f99a286f3e0d4514c31ab8ae52e84cfe23 that was already received
2017-12-11 13:58:44.911456 Peer sent entire transaction 688583e03e9043bbd6566c9ef5da8f0550d8a8b960f0b9239a94e4fc99887c5e that was already received
2017-12-11 13:58:45.033646 P2Pool: 17628 shares in chain (16240 verified/17632 total) Peers: 10 (4 incoming)
2017-12-11 13:58:45.033913 Local: 0H/s in last 0.0 seconds Local dead on arrival: ??? Expected time to share: ???
2017-12-11 13:58:45.034058 Shares: 0 (0 orphan, 0 dead) Stale rate: ??? Efficiency: ??? Current payout: (0.0000)=0.0000 BTC
2017-12-11 13:58:45.034204 Pool: 7173TH/s Stale rate: 3.2% Expected time to block: 11.0 days
2017-12-11 13:58:45.092149 Peer sent entire transaction 688583e03e9043bbd6566c9ef5da8f0550d8a8b960f0b9239a94e4fc99887c5e that was already received
2017-12-11 13:58:53.621809 Peer sent entire transaction 688583e03e9043bbd6566c9ef5da8f0550d8a8b960f0b9239a94e4fc99887c5e that was already received
2017-12-11 13:58:53.957365 Peer sent entire transaction 688583e03e9043bbd6566c9ef5da8f0550d8a8b960f0b9239a94e4fc99887c5e that was already received
2017-12-11 13:58:57.317096 Peer sent entire transaction a94cf7b5a6d446b901783e8168141bd5699b18b099d6315621441251b7c4659b that was already received
2017-12-11 13:58:57.330907 Peer sent entire transaction 97d75c8a745d8b8593d328e3f4150a243f75cf3531c7f4c67fef447841500d87 that was already received
2017-12-11 13:58:57.527107 Peer sent entire transaction 97d75c8a745d8b8593d328e3f4150a243f75cf3531c7f4c67fef447841500d87 that was already received
2017-12-11 13:58:57.970848 Peer sent entire transaction 97d75c8a745d8b8593d328e3f4150a243f75cf3531c7f4c67fef447841500d87 that was already received
2017-12-11 13:58:58.373802 Peer sent entire transaction 97d75c8a745d8b8593d328e3f4150a243f75cf3531c7f4c67fef447841500d87 that was already received
2017-12-11 13:58:58.617500 Peer sent entire transaction 97d75c8a745d8b8593d328e3f4150a243f75cf3531c7f4c67fef447841500d87 that was already received So as you can see, although my miners are pointed to my node at 127.0.0.1:9332, the local rate is still 0H/s. My miners are very slow, but still I think that their hash rate should be visible. Or am I missing something? Also, it is strange that my cgminer now again gives 0 accepted and 5815374 rejected. This is something I find strange. The output of cgminer is: cgminer version 4.10.0 - Started: [2017-12-11 11:12:05.150]
--------------------------------------------------------------------------------
(5s):105.8G (1m):96.54G (5m):93.58G (15m):90.14G (avg):95.91Gh/s
A:0 R:5815374 HW:263 WU:1334.1/m
Connected to 127.0.0.1 diff 329K with stratum as user xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Block: 8ee4d35a... Diff:1.59T Started: [14:10:53.954] Best share: 226K
--------------------------------------------------------------------------------
[U]SB management [P]ool management [S]ettings [D]isplay options [Q]uit
0: BSC 10000197: COMPAC-1 150.00MHz HW:0 | 7.307G / 8.277Gh/s WU:115.7/m
1: BSD 10013137: COMPAC-2 200.00MHz HW:262 | 21.02G / 21.96Gh/s WU:301.0/m
2: BSD 10013099: COMPAC-2 200.00MHz HW:1 | 34.09G / 21.93Gh/s WU:306.4/m
3: BSD 10013031: COMPAC-2 200.00MHz HW:0 | 28.43G / 21.79Gh/s WU:304.4/m
4: BSD 10013138: COMPAC-2 200.00MHz HW:0 | 21.81G / 21.95Gh/s WU:306.7/m [2017-12-11 13:52:13.774] Stratum from pool 0 requested work restart
[2017-12-11 13:52:27.996] Stratum from pool 0 requested work restart
[2017-12-11 13:52:40.520] Pool 0 difficulty changed to 342703.6
[2017-12-11 13:52:40.521] Stratum from pool 0 requested work restart
[2017-12-11 13:53:40.111] Pool 0 difficulty changed to 338314.9
[2017-12-11 13:53:40.111] Stratum from pool 0 requested work restart
[2017-12-11 13:53:42.105] Stratum from pool 0 requested work restart
[2017-12-11 13:54:00.967] Pool 0 difficulty changed to 337301.9
[2017-12-11 13:54:00.967] Stratum from pool 0 requested work restart
[2017-12-11 13:54:22.709] Pool 0 difficulty changed to 335591.2
[2017-12-11 13:54:22.710] Stratum from pool 0 requested work restart
[2017-12-11 13:54:23.999] Stratum from pool 0 requested work restart
[2017-12-11 13:54:53.832] Pool 0 difficulty changed to 333706.6
[2017-12-11 13:54:53.832] Stratum from pool 0 requested work restart
[2017-12-11 13:55:18.669] Pool 0 difficulty changed to 332825.1
[2017-12-11 13:55:18.669] Stratum from pool 0 requested work restart
[2017-12-11 13:55:41.308] Pool 0 difficulty changed to 329907.0
[2017-12-11 13:55:41.308] Stratum from pool 0 requested work restart
[2017-12-11 13:57:11.395] Stratum connection to pool 0 interrupted
[2017-12-11 13:57:12.424] Pool 0 difficulty changed to 322413.0
[2017-12-11 13:57:12.428] Stratum from pool 0 requested work restart
[2017-12-11 13:57:13.008] Pool 0 difficulty changed to 329907.0
[2017-12-11 13:57:13.008] Stratum from pool 0 requested work restart
[2017-12-11 13:57:13.008] Rejected untracked stratum share from pool 0
[2017-12-11 13:58:43.051] Stratum connection to pool 0 interrupted
[2017-12-11 13:58:44.164] Pool 0 difficulty changed to 346418.1
[2017-12-11 13:58:44.169] Stratum from pool 0 detected new block at height 498741
[2017-12-11 13:58:44.256] Stratum from pool 0 requested work restart
[2017-12-11 13:58:44.262] Rejected untracked stratum share from pool 0
[2017-12-11 13:59:04.957] Pool 0 difficulty changed to 345313.8
[2017-12-11 13:59:04.957] Stratum from pool 0 requested work restart
[2017-12-11 13:59:49.833] Pool 0 difficulty changed to 349850.7
[2017-12-11 13:59:49.833] Stratum from pool 0 detected new block at height 498742
[2017-12-11 14:00:26.001] Pool 0 difficulty changed to 344780.4
[2017-12-11 14:00:26.002] Stratum from pool 0 requested work restart
[2017-12-11 14:00:27.293] Stratum from pool 0 requested work restart
[2017-12-11 14:00:44.426] Pool 0 difficulty changed to 342663.0
[2017-12-11 14:00:44.427] Stratum from pool 0 requested work restart
[2017-12-11 14:00:53.575] Pool 0 difficulty changed to 341213.7
[2017-12-11 14:00:53.575] Stratum from pool 0 requested work restart
[2017-12-11 14:00:55.052] Stratum from pool 0 requested work restart
[2017-12-11 14:02:04.548] Pool 0 difficulty changed to 333298.2
[2017-12-11 14:02:04.548] Stratum from pool 0 requested work restart
[2017-12-11 14:02:26.943] Pool 0 difficulty changed to 330923.5
[2017-12-11 14:02:26.943] Stratum from pool 0 requested work restart So it detects the new blocks. But the work is always restarted, and the difficulty adjusted. Under normal operation I should see a lot of accepted, which is here not the case. I understand that the difficulty is higher on the jtoomim fork, but even then I should have seen some accepted in the last couple of hours. We know (jtoomim confirmed this) that my miners work, so it must be that something is wrong with my p2pool node or with the connection between my miners and my p2pool node. However, the output of p2pool doesn't seem abnormal. It looks like a normal operating p2pool node, except for the timestamp error (this error was only temporary). Also, cgminer doesn't give any error messages. It's just that I don't know what is going wrong, since that I use exactly the same setup like the one i was using for mining on the main p2pool. So does anyone have a suggestion of what I can try next? Or is this purely because my miners are to slow? Thank you in advance. |
|
|
|
https://en.bitcoin.it/wiki/P2Pool#StalesCould anybody be so kind to explain in more details, what does this mean? On P2Pool stales refer to shares which can't make it into the sharechain. Because the sharechain is 20 times faster than the Bitcoin chain many stales are common and expected. However, because the payout is PPLNS only your stale rate relative to other nodes is relevant; the absolute rate is not. What's PPLNS got to do with it? Hello Rabinovitch, The reason is in my opinion the following. PPLNS is pay per last n shares. So lets assume that the last 200 shares result in a payment to the nodes that mined the shares. Lets assume that A, B, C and D mine and lets also assume they have the same hash rate. Further we assume that their stale rate is 0 (so each miner has a stale rate of 0). In this case each miner has 50 shares (this is not necessarily true in reality because mining is a random process) in the share chain. Lets also assume for simplicity that every share leads to an equal payment. In that case each miner gets a payment when a block is found that is equal to 1/4th of the block reward. Now lets assume that A, B, C and D mine, but they all have a stale rate of 10%. This means that 10% of the shares of the network are lost. This means that to generate a chain of 200 shares, the network takes 10% more time than in the previous situation. Since they all have a 10% stale rate, they all wast an equal amount of work, so in the end the share chain of 200 shares will again contain 50 shares of each node. This means also that they have right on 1/4th of the block reward. Now lets assume again that A, B, C and D mine, and they again all have the same hash power. But now A, B, C have a stale rate of 0, and D has a 50% stale rate. Now there is a difference in the amount of shares each node will have in the share chain of 200 shares length. A, B, and C will have 57.14 shares in the share chain, and D will only have 28.57 shares in the share chain. This means that D will get much less reward than A, B and C if a block is found. The share for D decreases, but the share of A, B and C increases. So the relevance of PPLNS is that it pays a reward to a miner according to the number of shares it has in the share chain. The relative stale rate has an impact on the distribution of the reward, because it determines how many shares each miner will have in the share chain. Of course the examples here are simplifications of the real algorithm. PPLNS comes in many different forms. You can find a more detailed explanation of PPLNS at https://bitcointalk.org/index.php?topic=39832. I hope this is an answer to your question. |
|
|
|
Hello, I've recently switched from the main p2pool to jtoomim's fork, due to issues with cpu load. P2Pool works, and I can mine, but over time my node start creating huge amounts of orphans. An analysis of the problem showed that the issue was the cpu load. It's nearly always at 100%. I also saw the same messages that other people on this forum had, namely that the connection with bitcoind was lost. It is indeed so that this starts to happen when the cpu utilization is at 100% for a very long time. I must say that this became only quite recently an issue for me. I've been using p2pool for over a year now, but it is only the last couple of weeks that the cpu utilization became an issue. At first I used just python to run p2pool, and this was sufficient. Recently I switched to pypy as an attempt to alleviated the problem with the cpu load. This diminished the cpu load slightly, however not sufficiently, and my node still gets a lot of orphan shares (the stale rate is enormous: between 30 and 50%). So at that point I decided to switch to jtoomim's fork because of the optimalizations. However I experience some problems. Well actually I haven't been able to mine at all with jtoomim's fork. Somehow my miners don't mine. I know the miners work and so does the setup because I can mine 'perfectly' on the main P2pool. But when using jtoomim's fork all the work of my miners get rejected. I also see the lights flashing on the usb miners, so they are active. Cgminer gives me the following: [2017-12-09 12:37:24.975] Pool 0 difficulty changed to 318767.1
[2017-12-09 12:37:24.976] Stratum from pool 0 requested work restart
[2017-12-09 12:38:00.753] Pool 0 difficulty changed to 316516.4
[2017-12-09 12:38:00.753] Stratum from pool 0 requested work restart
[2017-12-09 12:38:53.471] Pool 0 difficulty changed to 311201.8
[2017-12-09 12:38:53.471] Stratum from pool 0 requested work restart
[2017-12-09 12:39:21.296] Pool 0 difficulty changed to 308553.9
[2017-12-09 12:39:21.297] Stratum from pool 0 requested work restart
[2017-12-09 12:39:44.927] Pool 0 difficulty changed to 305991.7
[2017-12-09 12:39:44.927] Stratum from pool 0 requested work restart
[2017-12-09 12:39:46.181] Stratum from pool 0 requested work restart
[2017-12-09 12:40:08.425] Pool 0 difficulty changed to 304497.7
[2017-12-09 12:40:08.426] Stratum from pool 0 requested work restart
[2017-12-09 12:40:08.426] Stratum from pool 0 requested work restart
[2017-12-09 12:40:10.956] Stratum from pool 0 requested work restart
[2017-12-09 12:40:13.911] Stratum from pool 0 requested work restart
[2017-12-09 12:40:28.796] Pool 0 difficulty changed to 303616.9
[2017-12-09 12:40:28.796] Stratum from pool 0 detected new block at height 498385
[2017-12-09 12:40:41.587] Pool 0 difficulty changed to 301658.7
[2017-12-09 12:40:41.587] Stratum from pool 0 requested work restart
[2017-12-09 12:40:49.738] Pool 0 difficulty changed to 301019.8
[2017-12-09 12:40:49.738] Stratum from pool 0 requested work restart
[2017-12-09 12:40:52.193] Pool 0 difficulty changed to 331502.0
[2017-12-09 12:40:52.193] Stratum from pool 0 requested work restart
[2017-12-09 12:41:01.276] Stratum from pool 0 requested work restart
[2017-12-09 12:41:05.722] Stratum from pool 0 detected new block at height 498386
[2017-12-09 12:41:07.698] Pool 0 difficulty changed to 344825.9
[2017-12-09 12:41:07.698] Stratum from pool 0 requested work restart
[2017-12-09 12:41:39.337] Pool 0 difficulty changed to 343068.6
[2017-12-09 12:41:39.337] Stratum from pool 0 requested work restart
[2017-12-09 12:42:02.765] Pool 0 difficulty changed to 338019.6
[2017-12-09 12:42:02.765] Stratum from pool 0 requested work restart
[2017-12-09 12:42:02.765] Stratum from pool 0 requested work restart
[2017-12-09 12:42:20.844] Pool 0 difficulty changed to 336936.5
[2017-12-09 12:42:20.845] Stratum from pool 0 requested work restart
[2017-12-09 12:42:53.143] Pool 0 difficulty changed to 335038.0
[2017-12-09 12:42:53.144] Stratum from pool 0 requested work restart
[2017-12-09 12:43:07.531] Pool 0 difficulty changed to 334145.2
[2017-12-09 12:43:07.531] Stratum from pool 0 requested work restart
[2017-12-09 12:43:11.495] Stratum from pool 0 requested work restart
[2017-12-09 12:43:27.773] Pool 0 difficulty changed to 331792.9
[2017-12-09 12:43:27.773] Stratum from pool 0 requested work restart
[2017-12-09 12:43:46.130] Pool 0 difficulty changed to 330480.7
[2017-12-09 12:43:46.130] Stratum from pool 0 requested work restart
[2017-12-09 12:44:45.373] Pool 0 difficulty changed to 322995.3
[2017-12-09 12:44:45.373] Stratum from pool 0 requested work restart
[2017-12-09 12:44:48.620] Stratum from pool 0 detected new block at height 498387 That is all I get. Also accepted = 0 and rejected = a lot. If I analyze the output of p2pool, I see the following: 2017-12-09 12:46:42.790786 Pool: 5909TH/s Stale rate: 4.8% Expected time to block: 13.4 days
2017-12-09 12:46:44.605121 Peer sent entire transaction 7f7e027bf783f74f0004f84945fa0ad4e811dea0f77653ed26646facef7ab012 that was already received
2017-12-09 12:46:50.004147 Peer sent entire transaction b1dbf883f527584830abcd8cc2ed37db0bfc7f0e9cd0a6ea73558577a1641214 that was already received
2017-12-09 12:46:51.680249 Peer sent entire transaction b1dbf883f527584830abcd8cc2ed37db0bfc7f0e9cd0a6ea73558577a1641214 that was already received
2017-12-09 12:46:54.528876 Peer sent entire transaction e21cbec550ca962e08a9e630d9192e1ff3813ab158b5d9882ed8c0a406128614 that was already received
2017-12-09 12:46:55.393046 Peer sent entire transaction b1dbf883f527584830abcd8cc2ed37db0bfc7f0e9cd0a6ea73558577a1641214 that was already received
2017-12-09 12:47:00.150376 Peer sent entire transaction b1dbf883f527584830abcd8cc2ed37db0bfc7f0e9cd0a6ea73558577a1641214 that was already received
2017-12-09 12:47:00.306119 Peer sent entire transaction 24c5c38682bfe425d1317d6b56d83ef1e3b04f732450f8b7f8131ad90842fa39 that was already received
2017-12-09 12:47:04.560185 Peer sent entire transaction ad2f5f371325641b5f8e5db13c2d62826104414beaef2f26e987c0f9baa36149 that was already received
2017-12-09 12:47:04.951282 Peer sent entire transaction 24c5c38682bfe425d1317d6b56d83ef1e3b04f732450f8b7f8131ad90842fa39 that was already received
2017-12-09 12:47:08.196591 Peer sent entire transaction 24c5c38682bfe425d1317d6b56d83ef1e3b04f732450f8b7f8131ad90842fa39 that was already received
2017-12-09 12:47:10.547186 Peer sent entire transaction 24c5c38682bfe425d1317d6b56d83ef1e3b04f732450f8b7f8131ad90842fa39 that was already received
2017-12-09 12:47:10.624559 Peer sent entire transaction 24c5c38682bfe425d1317d6b56d83ef1e3b04f732450f8b7f8131ad90842fa39 that was already received
2017-12-09 12:47:12.794002 P2Pool: 18894 shares in chain (10338 verified/18895 total) Peers: 7 (1 incoming)
2017-12-09 12:47:12.794440 Local: 0H/s in last 0.0 seconds Local dead on arrival: ??? Expected time to share: ???
2017-12-09 12:47:12.794901 Shares: 0 (0 orphan, 0 dead) Stale rate: ??? Efficiency: ??? Current payout: (0.0000)=0.0000 BTC
2017-12-09 12:47:12.795191 Pool: 5909TH/s Stale rate: 4.8% Expected time to block: 13.4 days
2017-12-09 12:47:14.620279 Peer sent entire transaction 55b649247f4f1c3401dff5ab149f1363821c6c26b39e0e9344b9e6c04b968277 that was already received
2017-12-09 12:47:16.179862 Peer sent entire transaction 55b649247f4f1c3401dff5ab149f1363821c6c26b39e0e9344b9e6c04b968277 that was already received
2017-12-09 12:47:20.354942 Peer sent entire transaction 55b649247f4f1c3401dff5ab149f1363821c6c26b39e0e9344b9e6c04b968277 that was already received
2017-12-09 12:47:21.211953 Peer sent entire transaction 28bd8ec0eeef7fcfc6ef95bbb13d5249eb2d6bcdef964815dd1f13534e36c152 that was already received
2017-12-09 12:47:24.414647 Peer sent entire transaction 55b649247f4f1c3401dff5ab149f1363821c6c26b39e0e9344b9e6c04b968277 that was already received
2017-12-09 12:47:25.638982 Peer sent entire transaction 55b649247f4f1c3401dff5ab149f1363821c6c26b39e0e9344b9e6c04b968277 that was already received
2017-12-09 12:47:25.852640 Peer sent entire transaction 55b649247f4f1c3401dff5ab149f1363821c6c26b39e0e9344b9e6c04b968277 that was already received
2017-12-09 12:47:31.968817 Peer sent entire transaction 91418671d582f9ca897299ecbada2a1b89048e84850fb8b449e36fb0cb64e0f3 that was already received
2017-12-09 12:47:32.112685 Peer sent entire transaction 91418671d582f9ca897299ecbada2a1b89048e84850fb8b449e36fb0cb64e0f3 that was already received
2017-12-09 12:47:35.474336 Peer sent entire transaction 91418671d582f9ca897299ecbada2a1b89048e84850fb8b449e36fb0cb64e0f3 that was already received
2017-12-09 12:47:36.768029 Peer sent entire transaction 4390d4b9df73c20f53e187f88f1d1a90bef6e58a9c4abf9fb0528b2287f56f88 that was already received
2017-12-09 12:47:40.935720 Peer sent entire transaction 958abe7a189bcc7669e8de0eec0895b11bcc21a4a74ad25380d55e7237baec5e that was already received
2017-12-09 12:47:41.006484 Peer sent entire transaction 958abe7a189bcc7669e8de0eec0895b11bcc21a4a74ad25380d55e7237baec5e that was already received
2017-12-09 12:47:41.180621 Peer sent entire transaction 958abe7a189bcc7669e8de0eec0895b11bcc21a4a74ad25380d55e7237baec5e that was already received
2017-12-09 12:47:42.799063 P2Pool: 18894 shares in chain (10338 verified/18895 total) Peers: 7 (1 incoming)
2017-12-09 12:47:42.799495 Local: 0H/s in last 0.0 seconds Local dead on arrival: ??? Expected time to share: ???
2017-12-09 12:47:42.799797 Shares: 0 (0 orphan, 0 dead) Stale rate: ??? Efficiency: ??? Current payout: (0.0000)=0.0000 BTC
2017-12-09 12:47:42.800125 Pool: 5909TH/s Stale rate: 4.8% Expected time to block: 13.4 days
2017-12-09 12:47:46.797353 Peer sent entire transaction 958abe7a189bcc7669e8de0eec0895b11bcc21a4a74ad25380d55e7237baec5e that was already received
2017-12-09 12:47:47.811957 Peer sent entire transaction 8ae74c9c3367fbbcf954e1f3dc085fa4dcca70dc075b016585fbf4d092327271 that was already received
2017-12-09 12:47:50.377209 Peer sent entire transaction 8ae74c9c3367fbbcf954e1f3dc085fa4dcca70dc075b016585fbf4d092327271 that was already received
2017-12-09 12:47:52.139107 Peer sent entire transaction 7906c45bfb1ac2d8cc68c1a112ec00f419f6c6f97d0dbdb9c472b680295f9916 that was already received
2017-12-09 12:47:55.676708 Generating a share with 1030867 bytes (796675 new) and 1749 transactions (1203 new)
2017-12-09 12:47:55.868625 New work for xxxxxxxxxxxxxxxxxxxxxxxxxx Diff: 331431.13 Share diff: 47521705.54 Block value: 14.40 BTC (1749 tx, 1031 kB)
2017-12-09 12:47:56.641314 Peer sent entire transaction ddc0747ebc82a8648afc7e25c820bd5a66cd624bf4dbf37a66ba9f2a1fce7444 that was already received
2017-12-09 12:47:56.650497 Peer sent entire transaction ddc0747ebc82a8648afc7e25c820bd5a66cd624bf4dbf37a66ba9f2a1fce7444 that was already received
2017-12-09 12:47:57.388120 Peer sent entire transaction 8ae74c9c3367fbbcf954e1f3dc085fa4dcca70dc075b016585fbf4d092327271 that was already received
2017-12-09 12:48:03.519249 Peer sent entire transaction 6ef6ca8a2e71117f951852a8054aaade92253bd3bf3edcd5c9540866aeb76875 that was already received
2017-12-09 12:48:03.780748 Peer sent entire transaction 3c23a8842d731c695162989c8aaeaaca8168a7829121c0136a405368d38e9c5b that was already received
2017-12-09 12:48:05.673745 Peer sent entire transaction 3c23a8842d731c695162989c8aaeaaca8168a7829121c0136a405368d38e9c5b that was already received
2017-12-09 12:48:11.758827 Peer sent entire transaction 3c23a8842d731c695162989c8aaeaaca8168a7829121c0136a405368d38e9c5b that was already received
2017-12-09 12:48:11.833870 Peer sent entire transaction 3c23a8842d731c695162989c8aaeaaca8168a7829121c0136a405368d38e9c5b that was already received
2017-12-09 12:48:12.804277 P2Pool: 18895 shares in chain (10339 verified/18896 total) Peers: 7 (1 incoming)
2017-12-09 12:48:12.804664 Local: 0H/s in last 0.0 seconds Local dead on arrival: ??? Expected time to share: ???
2017-12-09 12:48:12.804912 Shares: 0 (0 orphan, 0 dead) Stale rate: ??? Efficiency: ??? Current payout: (0.0000)=0.0000 BTC
2017-12-09 12:48:12.805183 Pool: 6173TH/s Stale rate: 4.8% Expected time to block: 12.8 days
2017-12-09 12:48:13.162434 Peer sent entire transaction 3c23a8842d731c695162989c8aaeaaca8168a7829121c0136a405368d38e9c5b that was already received
2017-12-09 12:48:19.125554 Peer sent entire transaction e396580068983ef8e3ce328047b8beaa0c9b69a51a86c6df687e3b5519b8bafc that was already received
2017-12-09 12:48:19.265750 Peer sent entire transaction e396580068983ef8e3ce328047b8beaa0c9b69a51a86c6df687e3b5519b8bafc that was already received
2017-12-09 12:48:20.927594 Peer sent entire transaction e396580068983ef8e3ce328047b8beaa0c9b69a51a86c6df687e3b5519b8bafc that was already received It keeps on doing that. I see that new work is send to the miners. Also the rest of the output makes me believe that the P2Pool node is running as it should. The cgminer command I use to run cgminer and connect to p2pool is the following: screen -d -m -S cgminer ~/git/vthoang/cgminer/cgminer -o stratum+tcp://127.0.0.1:9332 -u xxxxxxxxxxxxxxxxxxxxxxxxxxxx -p xxxxxxxxxxxxx --suggest-diff 32 --gekko-compac-freq 150 --gekko-2pac-freq 200 Can someone explain to me what i'm doing wrong? Thank you Extra info: My setup is the following: * Processor: i3 M350 @2.77Ghz (so it is not a powerful processor) * RAM: 8GB * OS: ubuntu server 16.04 LTS * mining software: cgminer 4.10 * miners: a couple of gekkoscience compac and gekkoscience 2Pac |
|
|
|
|
Hello all,
47 minutes ago p2pool main net found a block. According to blockchain.info it contains 1749 transactions, its size is 971.35 kb and its weight is 3626.63 kb.
|
|
|
|
Sorry for the slow response, Cryptonomist. 1) Is it correct to assume that the instance of the Tracker class in p2pool/util/forrest.py is the actual share chain?
That's one part of the relevant code. The actual tracker for the share chain is the OkayTracker class which inherits from forest.tracker, as instantiated as node.tracker. I think OkayTracker's code is more relevant and interesting. By the way, it's forest.py with one 'r' (as in a bunch of trees), not forrest.py (as in the author's name). 2) Can someone suggest a way to get the time between the "Time first seen" and the addition to the share chain.
I would suggest printing out the difference between the time first seen and time.time() at the end of data.py:OkayTracker.attempt_verify(). That seems like useful information for everyone. If you put it under the --bench switch and submit it as a PR to 1mb_segwit, I'd likely merge it. Don't worry about it if you're not good with git/github, as it's not a big deal either way. 3) the flow of a share between the moment the node detects its existence and the final addition of it to the share chain is not very clear to me.
Yeah, that code is kinda spaghettified. It might help to insert a "raise" somewhere and then run it so you can get a printout of the stack trace at that point. Quick from-memory version: the stuff in data.py (the BaseShare class and its child classes) gets called during share object instantiation and deserialization. When p2p.py receives a serialized share over the wire, it deserializes it and turns it into an object, then asks the node object in node.py what to do with it. node.py then passes it along to the node.tracker object and asks the tracker if it fits in the share chain; if it does, then node.tracker adds it to node.tracker.verified, and the next time node.tracker.think() is run (which is probably immediately afterward), node.tracker may choose to use that new share for constructing work to be sent to miners. This causes work.py:get_work() to generate a new stratum job (using data.py:*Share.generate_transaction() to make a coinbase transaction stub and block header) which gets passed via bitcoin/worker_interface.py and bitcoin/stratum.py to the mining hardware. 4a) Under the rules in the main p2pool network the shares are 100kb. So after 300 seconds on average the shares will have a total of 1mb transactions, and after 600 seconds on average the bitcoin blocks would be 2mb. Is this correct?
Sorta. It's a limit of 100 kB of new transactions. Less than 100 kB of new transactions can be added per share. The serialized size of the share is much lower than this, since the transactions are referred to by hash instead of as the full transaction; the serialized size of the candidate block that the share represents is much larger than this, and includes the old (reused) transactions as well as the new ones. If the transaction that puts it over 100 kB is 50 kB in size, and has 51 kB of new transactions preceding it, then only 51 kB of transactions get added. If some of the old transactions from previous shares have been removed from the block template and replaced with other transactions, then those old transactions don't get included in the new share and your share (and candidate block) size goes down. In practice, the candidate block sizes grow slower than 100 kB per share. I haven't checked very thoroughly how much slower, but in the one instance that I followed carefully it took around 25 shares to get to 1 MB instead of 10 shares. 4b) The hash of the header of the bitcoin block contains the merkle tree of the transactions the block contains. ... How can the transactions of several shares be added to get for example after 300 seconds 1 mb of transactions in a bitcoin block.
The hash of a share *is equal to* the hash of the corresponding bitcoin block header. The share structure includes a hash of all p2pool-specific metadata embedded into the coinbase transaction (search for gentx in data.py). The share has two different serializations: the long serialization (which is exactly equal to the block serialization, and which only includes the hash of the share-specific metadata), and the short serialization (which includes the block header plus the share-specific metadata such as the list of hashes of new transactions, the 2-byte or 3-byte reference links for the old transactions, the share difficulty, timestamps, etc.). Any synced p2pool node can recreate the long serialization from the short serialization, data in the last 200 shares of the share chain, and the hash:full_tx map in the node's known_txs_var. The transactions aren't "added". If a transaction has been included in one of the last 200 shares in the share chain, then a share can reference that share using the number of steps back in the share chain (1 byte) and the index of the transaction within that share (1 or 2 bytes). These transactions -- "old" transactions -- do not count toward the 100 kB limit. If a transaction has not been included before, then the share will reference this transaction using its full 32-byte hash, and counts its full size (e.g. 448 bytes) against the 100 kB limit. Both types of references are committed into the metadata hash in the gentx, so both are immutable and determined at the time the stratum job is sent to the mining hardware. https://github.com/jtoomim/p2pool/blob/9692d6e8f9980b057ae67e8970353be3411fe0fe/p2pool/data.py#L156My code currently has a soft limit of 1000 kB (instead of 100 kB or 50 kB) on new transactions per share, but unlike the p2ool master branch, this is not enforced at the consensus layer, so anyone can modify their code to exceed this limit without consequences from should_punish_reason(). Thank you for your reply. It's most helpfull. I will put the code on github once I've implemented and tested it. |
|
|
|
|
Hello,
I'm still going through the p2pool code. I have however a few questions about how p2pool works.
1) Is it correct to assume that the instance of the Tracker class in p2pool/util/forrest.py is the actual share chain? It seems to keep track of all the shares in the share chain. Or is this assumption wrong?
2) As jtoomim pointed out, the "Time first seen" on the p2pool web page comes from self.time_seen = time.time() of the class BaseShare in p2pool/data.py. I wonder how much time passes between this time_seen, and the addition of the share to the share chain. I'm trying to put some timers in the code, but I'm not very successful in my attempts. Can someone suggest a way to get the time between the "Time first seen" and the addition to the share chain. I would like to modify the p2pool client and run it on several computers to get an estimate of the time I need to add to "Time first seen" to get the approximate time it takes to update the share chain.
3) Another problem I'm facing, but which is related to the problem I have in question 2, is that the flow of a share between the moment the node detects its existence and the final addition of it to the share chain is not very clear to me. My image of the process is for the moment the following:
* the main function in p2pool/main.py creates an instance of class P2PNode from p2pool/node.py.
* the instance of P2PNode handles the p2pool shares and the bitcoin blocks, through functions like handle_shares, handle_get_shares, etc... This part I understand reasonable well I think. For example in the method handle_shares it adds the shares that were given as parameter to the method to the tracker if the tracker didn't know them already. I was able to add some lines to the code that return the time it takes to process the new shares.
My problem is that I can't for the moment connect the methods in P2PNode to the time function in BaseShare (and the classes that inherite from this class Share and NewShare). I don't understand how the processes in P2PNode are chronologically speaking arranged versus the time function in BaseShare. Would someone be able to clarify this to me? It would be a great help.
4) My last question is related to a discussion between jtoomim and veqtrus. From the forum posts I could understand that the share size is relevant to the size of the bitcoin blocks found by p2pool. Apparently the sum of the size of the transactions in the different shares tells us something about the expected size of the bitcoin block if a block is found by the p2pool network. So on average a share is found every 30 seconds. A bitcoin block is found on average every 10 minutes. Under the rules in the main p2pool network the shares are 100kb. So after 300 seconds on average the shares will have a total of 1mb transactions, and after 600 seconds on average the bitcoin blocks would be 2mb. Is this correct?
There is however something I don't understand. The hash of the header of the bitcoin block contains the merkle tree of the transactions the block contains. So these transactions can't be changed without changing the merkle tree, and the block hash. Now p2pool creates its own shares, which have also a hash, that can potentially be a hash that is accepted by the bitcoin network. This hash needs to have sufficient zeros in front to be considered as a block in the bitcoin network. So the share hash contains I suppose also a merkle tree of the transactions. This means I think that those transactions can't be changed either. My question is now, how can this work? How can the transactions of several shares be added to get for example after 300 seconds 1 mb of transactions in a bitcoin block. If transactions are added, then the merkle tree changes, and also the share hash. So the work done by the miners would be lost. Of course I know that I'm missing something, but can someone explain me how it works, and where in the code I can find it?
Thank you in advance
|
|
|
|
I haven't been on p2pool-bitcointalk for a while, but I remember reading through the posts every day, and jtoomin, I believe had the version of p2pool to run. On p2pool.org, it shows it has been 83 days since the last block, so are they not running jtoomin's fork?
Is there a list somewhere of pools that are running the fork? Sorry if this has been answered somewhere, but I looked through a bunch of pages without finding any info.
Thanks,
Dave
Hello, There are a few websites that give the results from a node scanner. For example http://p2pool.jir.dk/bitcoin/ gives publicly reachable nodes in the P2pool main network and in the jtoomim network. Nodes with a version number 17 or 16 are nodes of the main network. Nodes with a version number of 15.0-33 are nodes from the jtoomim network. Another useful site is https://p2pool.coinpool.pw/. This site also gives the publicly reachable nodes for both networks. P2pool.org has also a node scanner, but this one only shows results for the p2pool main network. All the information shown on the p2pool.org website is currently that of the p2pool main network and not of the jtoomim network. If you want to know the ip address of nodes that are not publicly reachable, the only way to do that is run your own node and look to which nodes it makes a connection. I hope this answers your question. |
|
|
|
1) On the web interface of my node I find the following in relation to the shares: "Best share", "Verified heads", "Heads", "Verified tails", "Tails". I'm not sure I fully understand what they mean.
P2pool has a share chain that is very much like the Bitcoin blockchain, with a few differences. The tip of a chain (a block/share with no children) is known as a head. The root of a chain is known as the genesis block in Bitcoin (the block with no parent), but in p2pool it's known as a tail. P2pool's tails *do* have parents, unlike bitcoin, and consequently have a height greater than 1; however, they're called tails in p2pool because their parents have been pruned off from the node's copy of the share chain, so as far as your node is concerned, they have no living parents. The best share is always a head. It's usually the head with the most work behind it, but not always. The rules for determining the head are a little complicated, and are different in different versions of p2pool. *I suppose "best share" is the last share added to the sharechain (I'm I right about this???).
Not necessarily, but usually. Just because a share is recent doesn't mean that it is placed at the tip of the chain. *"Verified heads" -- Other heads that are or will be orphaned or so?
Yes, precisely. *What is the difference between "Verified heads" and "heads"?
*What are the "Verified Tails" and "Tails"?
I'm not sure. I think unverified heads or tails are shares that have not yet been connected to the share chain, such as for a share that is was just downloaded during initial sync but whose ancestors have not yet been downloaded. However, I haven't verified that hypothesis, as I have not yet needed to work on that part of the code. *I suppose "Time first seen" gives the time that my node received a Inv ... Or is it the time when my node has completely downloaded the share?
Completely downloaded and mostly deserialized. *"Peer first received from" gives then probably the node that has send my node the Inv containing the share.
Yes, but it's a sendShares or share_reply message. *"Timestamp" is the timestamp from the node that has found the share I guess. But this timestamp probably depends on the accuracy of that nodes clock, and can in theory deviate from reality?
Correct. In the current p2pool master, the timestamp is required to be between 1 second after the previous share's timestamp and 59 seconds after the previous share's timestamp. Since the timestamps are used for minimum share difficulty adjustments, this can result in a situation where the share difficulty is very slow to adjust downwards after a rapid decrease in the network hashrate. In the jtoomimnet fork, I got rid of this timestamp clipping, and replaced it with a rule that rejects shares that are timestamped to come from the future in order to prevent share difficulty manipulation. Keep in mind that the timestamp is the time at which the mining job was sent to the hardware, not the time at which the hardware returned the nonce value. 3) Say for example that I want to know the time it takes my node to receive the latest share of the sharechain (so I want to know the time it takes my node to converge to the consensus sharechain). I probably can use the timestamp of the share gegenerated by the node that found the share, as it will be very close to the moment that it sends it to the rest of the network.
Nope, it's not close. It will usually be off by around 5-50 seconds. But what you can do is use the "Peer first received from" line to browse to the info page for that share on the peer's address (you may have to guess the port number), and repeat, until eventually you get to a peer that says "Peer first received from: Null" for that share, and then you can use the "Time first seen" shown by that ndoe (or just the node that you got the share from, if you're lazy). When I've done this measurement in the past, it's usually around 1 second per hop for nodes running CPython with 100 ms network latency, and around half that for nodes running pypy. 4) If my reasoning in point 3 is correct, I just need to find the time when my node is finished downloading the new share, and the share is added to the local sharechain. Is there a way to access the sharechain of my node directly through a log file or something? I know that p2pool creates logs in the directory /p2pool/data/bitcoin/, and occasionaly I find references to shares downloaded and stuff like that in the output. But is there another file for the sharechain, that contains time data of when a new share is downloaded and added to the share chain?
If you run my code (e.g. 1mb_segwit), you can install the rfoo package for python, then run run_p2pool.py with the --rconsole command line option, and then run the rconsole command in a separate terminal. This will give you a python interpreter that effectively runs inside the p2pool process, and allows you access to all of the variables and functions of p2pool while it's running. The share objects can be found in the gnode.tracker.verified object. 1mb_segwit also now has a --bench command-line option, which I think you would find interesting. You could also maybe parse the log in data/bitcoin/log. That's just the stdout output of p2pool for (usually) the last few days. Ok great. Thank you for the clarifications. I'll try the methodology you suggest and see if I can get the information I need. |
|
|
|
|
Hello,
I've got some questions related to P2Pool:
1) On the web interface of my node I find the following in relation to the shares: "Best share", "Verified heads", "Heads", "Verified tails", "Tails". I'm not sure I fully understand what they mean.
*I suppose "best share" is the last share added to the sharechain (I'm I right about this???).
*"Verified heads" contains several shares. I see that one of these shares is the one mentioned in "best share". So what are the other shares mentioned? Other heads that are or will be orphaned or so?
*What is the difference between "Verified heads" and "heads"?
*What are the "Verified Tails" and "Tails"?
2) When I look to the "Best share" share, then I see stuff like "Time first seen", "Peer first received from", "Timestamp".
*I suppose "Time first seen" gives the time that my node received a Inv (or the p2pool equivalent) from another node containing the hash of the share. Is this correct? Or is it the time when my node has completely downloaded the share?
*"Peer first received from" gives then probably the node that has send my node the Inv containing the share.
*"Timestamp" is the timestamp from the node that has found the share I guess. But this timestamp probably depends on the accuracy of that nodes clock, and can in theory deviate from reality?
3) Say for example that I want to know the time it takes my node to receive the latest share of the sharechain (so I want to know the time it takes my node to converge to the consensus sharechain). I probably can use the timestamp of the share gegenerated by the node that found the share, as it will be very close to the moment that it sends it to the rest of the network. So if I can also get the time that my node has downloaded the share, and restarts its mining work on top of the new share, I can in principal determine the time that it took the share to propagate from the node that has found the share to my node, and as such I can determine the "dwell time" of my node. Is this correct?
4) If my reasoning in point 3 is correct, I just need to find the time when my node is finished downloading the new share, and the share is added to the local sharechain. Is there a way to access the sharechain of my node directly through a log file or something? I know that p2pool creates logs in the directory /p2pool/data/bitcoin/, and occasionaly I find references to shares downloaded and stuff like that in the output. But is there another file for the sharechain, that contains time data of when a new share is downloaded and added to the share chain?
Thank you in advance
|
|
|
|
It seems that the p2pool master branch is back in business  . Currently >7000 TH. However, it still fluctuates a lot during the day. A few hours ago only 400 TH. |
|
|
|
How I can see fees for my node? Workers are in there, but I dont see money. ^)
Hello, I suppose you want to know your mining reward. You should be able to find it in the web interface of your P2pool node reachable through http://YOUR_P2POOL_HOST:9332/. It should give you statistics about the payout you would get if the P2pool network finds a block now (see under "Payouts if a block were found NOW:"). If you use the default bitcoin address for your payouts, then normally you should find a plot under the link "graphs". If you use a different address it will show 0. You can also find this information when you go to the web interface of a public node. Normally they should also list the "Payouts if a block were found NOW:". Pick a public node that follows the P2pool share chain you use for mining (jtoomim branch or master branch). You will only find a quantity different from 0 under "Payouts if a block were found NOW:" if your miners have found at least 1 share in the last 3 days in the share chain. Also, the "Payouts if a block were found NOW:" quantities are potential payouts. They only become real when the P2pool network finds a block that is accepted in the Bitcoin blockchain. So it may take a while before you get a real payout. I hope this answers your question. You can find more information on http://p2pool.in/ and http://p2pool.org/. |
|
|
|
Actually it's a simple CDF calculation that shows how bad it is to limit block sizes to taking an exorbitant 5 minutes to get to 1MB ... 5 minutes is a 50% block on an expected 10 minute network. So the CDF is 0.39346934028737 or 1 in 1.6 blocks (60.65%) will be over 5 minutes or reversing that ... 1 in 2.54 blocks (39.3%) will be under 5 minutes ... yeah that's a big % of blocks, not "albeit a small one" at all. I'm often surprised at how little maths people understand about Bitcoin when they are considered experts in it ... and/or spending money mining on it ... Note of course, this last comment is directed at someone else not you ...  can you show me the algebraic formula so I can understand it better .... Hello, The Bitcoin mining process can be modelled as a Poisson process. From this we can calculate the probabilistic interarrival times (the time between two events, here finding of two blocks), which are exponentially distributed with lambda = 1/600. The probability of getting interarrival times (IAT) larger than 5 min or 300 sec is: Pr{IAT > 300} = e^(-300/600) ≈ 0.60653 = 60.65%. So the probability of getting two blocks less than 300 sec from each other is: Pr{IAT < 300} = 1 - Pr{IAT > 300} ≈ 0.39347 = 39.34%. This is the probability of the Bitcoin network finding a block in less than 300 sec from the previous one. We can off course do the same for P2pool. The probability that the P2pool network finds a block less than 300 sec from the other is (assuming P2Pool has 0.1% of the total hashing power of the Bitcoin network, which is more than it currently has): lambda is now 1/600000. Pr{IAT < 300} = 1 - e^(-300/600000) ≈ 0.0005 = 0.05%. (This is an exercise I did for my self. It may contain errors related to how P2pool works. If I made errors please make me aware of them so I can learn from it.). We can also find the expected number of KBs that P2pool can add to the block if P2pool finds a block after 5 min, assuming 100 KB per share, and no difficulty change of the dynamic difficulty adjustment mechanism, and after the last found block by the network, P2Pool had to start again from scratch, meaning 0 KBs). As explained above the finding of shares is also a Poisson process. The probability to find no shares in an interval of length 300 sec is: Pr(N(300) = 0) = (((300/30)^0)/0!)*e^(-300/30) ≈ 0.00005. In this case P2pool will add 0 KBs to a block. We can continue this until an ridiculous and irrelevant number of shares, so I will truncate it at 20 shares: Pr(N(300) = 1) ≈ 0.00045 100KB Pr(N(300) = 2) ≈ 0.00227 200KB Pr(N(300) = 3) ≈ 0.00757 300KB ... Pr(N(300) = 9) ≈ 0.12511 900KB Pr(N(300) = 10) ≈ 0.12511 1000KB Pr(N(300) = 11) ≈ 0.11374 1100KB ... Pr(N(300) = 19) ≈ 0.00373 1900KB Pr(N(300) = 20) ≈ 0.00187 2000KB From these results we can calculate the weighted average: 996.55KB. So if a Bitcoin block is found by P2pool 300 sec after the previous than the expected amount of KBs in that block is 996.55KB. I hope this clarifies a thing or two. Or if you find errors please make me aware of them so I can learn from it. |
|
|
|
okay thank you.. is there a possibility to combine the two branches again? it seems to be very uselesss to split this very small pool in two even smaller, separate networks..
In 3 months jtoomimnet will fork off to segwit2x anyway so it seems to be pointless. Ok that is true, however I believe that jtoomim is planning to let his chain automatically split in 3 months. One part would be following segwit2x, and the other part segwit. In that case it is still possible to end up with two separate P2pool networks that actually follow the same Bitcoin rules. It depends all on how much miners are following jtoomim but do not support segwit2x. To be honest I have no idea how many people are in that case. Also, I don't know if jtoomim is still working on this automatic split. Just a hypothetical question (I'm not yet far enough in understanding the code, so this might be gibberish): In case the split of jtoomim's network happens, would it not be possible to merge the two versions that follow segwit, so that the reasons that some miners have to follow jtoomim even though they only support segwit is partially alleviated? |
|
|
|
so i tried https://github.com/jtoomim/p2pool - it worked ( although the invalid share rate was higher, sometimes 100% cpu usage.. ) now i switched back to https://github.com/p2pool/p2pool/ . but here the network is only 0.5 PH big ( 3 PH in the other version ) so am i an the wrong network, or are miners just frightened of the transition? will soemone join me? Hello, The network hash rate for the main branch (forrestv-veqtrus version) is indeed currently only 0.5 PH. It fluctuates a bit throughout the day. Since the split between the jtoomim and main branch the hash power in the main branch has decreased drastically, however, it is now again a bit higher compared to a few days ago. Before the split it was around >2PH on average (this fluctuated also a lot) if I remember it correctly, but even then P2pool was a very small pool. The reason of the decline (this is just my assessment of the situation) is on the one hand the fact that the network has split, and on the other hand the confusion in the P2pool community during the transition to segWit, and the not so straightforward way (through github) to get the segWit compatible version of P2pool main branch. This has now changed. Forrestv has updated the links on http://p2pool.in/ and veqtrus has merged his segWit compatible p2pool with the main branch on github. I expect that this will result over time in a higher hash rate for the p2pool main branch (I hope). |
|
|
|
Hello to all,
Just because you currently do not have issues with mainnet's memory consumption doesn't mean that others don't or won't. Not everyone has the luxury of having 24 GB of RAM to play with.
I just want to state for the record and clarity that you don't need anywhere near 24GB of RAM to run v17. I run it using an old used laptop with 4GB RAM and an i3 processor (2nd generation I believe). Bitcoind, p2pool and cgminer are running on the same system. I did have to do a bit of fine tuning like running Ubuntu Server, limit the connections to bitcoind to 35, etc... but nothing much out of the ordinary. I've been running v17 for two days non-stop. I did not find any errors in the logs that can be related to the laptop running out of memory (or not that I've noticed) (i actually haven't found any real problems at all). Everything has been working as it should. My stale rate is for the moment a bit on the high side ~15.4% but if I check the logs I see that it has been quite a lot lower a few hours ago (even lower than the stale rate of the network). In general it is close or lower than the stale rate of the network. But I'll keep an eye on it. The memory consumption by the relevant processes is for the moment as follows (this data comes from 'top'): 1) python (which is the p2pool node): 53.2%, 2) bitcoind: 29.9%, 3) cgminer: 0.2%, which totals to 83.3%. This changes a bit over time but not much. The CPU usage is at most around 70% for all three processes, but it is much more erratic than the memory consumption. It never stays for a long time at 70%. However, my laptop runs into trouble if I use http://(MY_LOCAL_IP_ADDRESS):9332/. I've noticed that if i use this once or a few times, a few hours later my node runs into trouble. P2pool freezes and my computer is constantly busy doing stuff. Normally the freezing stops after a few minutes, but in general it starts again a bit later. I need to kill p2pool and restart it to get again working properly. Apparently when I use http://(MY_LOCAL_IP_ADDRESS):9332/, the memory consumption of P2pool slowly increases until it is over 2GB and then my laptop runs into trouble. This evolution takes up a few hours. If i don't use http://(MY_LOCAL_IP_ADDRESS):9332/ then the memory consumption of P2pool remains on a constant level. If I don't use http://(MY_LOCAL_IP_ADDRESS):9332/, and just check the node status through the command line i don't have any problems at all. I have been running V16 for weeks without interruptions. Has anyone experienced this before? All in all I think we can say that running v17 or P2pool in general is also a real possibility for people with less powerful hardware and people on a budget. Of course if you want to run a public P2pool node you will need more powerful equipment. |
|
|
|
Hello all, I'm looking for technical information about P2pool. My goal is to understand the source code of P2pool, so that I get a better understanding of how the pool works. I've already read https://en.bitcoin.it/wiki/P2Pool, and quite a lot of posts on this forum, but now I'm looking for information related to how for example shares are constructed, how they are transmitted, how the difficulty is adjusted, etc... I've been looking so far to the code on github, using the bitcoin wiki as a guide, but I'm wondering whether there is some sort of guide available that contains more information? I'm familiar with Python and programming in general, so that is not an issue. Any suggestions on the best way to tackle this? Thank you. |
|
|
|
Just throwing it out there, would anyone be interested in a Bitcoin fork that has faster blocks (Let's say 10 second blocks). Block size 0.2MB. Effectively increasing block size to 12MB per 10 minute
Advantages:
* Time to 1st confirmation will be a few seconds on average, good enough for even buying coffee and actually getting a confirmation, instead of relying on completely insecure 0-conf or 3rd party service
* Much better tx/sec capacity obviously
* Better user experience overall, it will just feel fluid for all tx, anyone that has used a sub-10 sec altcoin should know the feeling. Everything is just fluid
Disadvantages:
* Potentially bigger blockchain size obviously, but that's not really a big problem unless we are doing VISA level transaction volume
* Orphans for everyone, but that's fair
I agree that a increase in the block creation rate would be useful. It would allow for much quicker settlement of transactions, which I consider very important if we want that Bitcoin becomes more mainstream. However I see a couple of problems: The problems associated with much higher block creation rates have been thoroughly studied. It causes, like mentioned before, an increase in the orphan block rate and so also a decrease in the efficiency of the network, which makes the network more vulnerable to attacks. More computing power is wasted on orphans, so the attacker needs less computing power to have a reasonable chance of success. However, solutions have been suggested for this problem. One of it is GHOST, which chooses the chain to mine on top of differently. An interesting article is Sompolinsky, Zohar "Fast money grows on trees, not chains". A more accessible article is of Vitalik Buterin "Toward a 12-second block time". These changes allow for very high block frequencies. A different problem is of course that the higher block frequency will make it more resource intensive to run a node in my opinion. Not only will the block chain increase rapidly in size, also the bandwidth usage and probably CPU usage will increase. This will result in fewer people running nodes, which will lead to more centralization. This is in my opinion something that we really need to avoid. So i think we would also need some kind of compression method, so that this burden is decreased, and kept within the range of ordinary users. Even an increase to 0.2 MB per 10 seconds or 12MB per 10 minutes will in my opinion be a problem. Next to that there is in my opinion also an implementation issue. It would require a massive rewriting of the Bitcoin code and will be a hard fork. It would in my opinion be much more difficult to implement this than a "simple" increase of the block size like segwit2x, etc... suggest. This makes it in my opinion practically impossible to do. A simpler solution would be to increase the transaction processing by using sidechains or something like that. For example there would be a chain that starts with a few relevant bitcoin UTXO's which serve as the basis for the sidechain. Transactions are done quickly on the sidechain, which has a much higher block creation rate (for example every 10 seconds). This can be done by using GHOST and small blocks. Once every few hours a settlement is done on the Bitcoin blockchain on the basis of the end result of the balances on the sidechain. A bit like the Lightning network but using a blockchain. |
|
|
|
can I run p2pool in the background, if yes then how? (Linux version)
Have user interface?
I suppose you want to run the process without the need to keep open the terminal? In that case you can use "screen". For example "screen -d -m -S p2pool ~/p2pool/run_p2pool.py". This will run the process in a seperate screen with the name p2pool. You can switch to the screen by using "screen -r p2pool", and detach from it using first Ctrl + A and subsequently Ctrl + D. Don't use Ctrl + C when you have retached the screen because that will terminate the process. If you want a GUI or so, I don't think there is one (I might be wrong. I run mine on Ubuntu Server so I only use command line). If you want statistics of your miners and stuff like that, you can get those from within your browser by going to http://(ip adress of node on local network):9332/. More info can be found on http://p2pool.in/I hope this answers your question. |
|
|
|
|
