[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[jtoomim]

I'm looking into the memory consumption issue now. Here's some data when running CPython:

Of the first 100 shares in the share chain, the average memory consumption (using pympler.asizeof) is 57 kB per share, with most shares between 20 kB and 100 kB.

Of this 57.3 kB average, 43.5 kB on average is taken up in share_info.transaction_hash_refs. See the background paragraph at the end if you want to know what that does. Each of these ints takes 24 bytes of RAM in CPython, even though we only need 1 or 2 bytes for each one based on the size of each int. It might be possible to encode these numbers as an array of uint8 and/or uint16 integers instead of regular python integers, and reduce the CPython memory consumption of this data by about 10-15x, or of the full share size by a little more than 3x, without any loss of functionality.

On the other hand, the list of new transaction hashes averaged 7 kB per share. These are 32 byte integers, but they take 64 bytes of Python memory. We might be able to save some memory here by using an array of strings (or just one really long string), but that would be inconvenient, and would only save about 2x memory on that variable, so it's probably not worth it.

Another option is to just forget all of the transaction data after the share is >200 shares away from the head of the chain, and reload it from disk if it's requested. This will probably be more work, and might open up some DoS vulnerabilities if I'm not careful with the code (since someone could mine a share that requires us to reload 200 shares off disk, and share parsing is hella slow right now), but would probably be able to reduce memory consumption by around 20x if done well.

I can't do the same tests on pypy, since sys.getsizeof() and asizeof() don't work on pypy, unfortunately.


Background on how p2pool's share format handles transactions: For each transaction in a share/block, p2pool will check to see if that transaction hash was encoded into one of the 200 previous shares in the share chain. P2pool then encodes that in share_info.transaction_hash_refs by referring to the share number (where 0 is this share, 1 is this share's parent, 2 is the grandparent, etc) and the index of that transaction in that share's share_info.new_transaction_hashes. If the transaction hash wasn't in a previous share, then p2pool also sticks that transaction hash into share_info.new_transaction_hashes. When sent over the network, these numbers are encoded as var_ints, so it's usually 1 byte for the share index, and 1-2 bytes for the transaction index, plus 32 bytes for each of the new hashes.

Edit: Yeah, the array of uint16 thing seems to work pretty well, at least on CPython. 2170 MB -> 489 MB = 4.44x reduction.


Edit 2: Seems to benefit pypy (5.7.1) even more. 5310 MB -> 785 MB = 6.76x reduction. Now I just need to make sure the new code can successfully mine shares...

Edit 3: Seems to mine shares just fine. Neat. https://github.com/jtoomim/p2pool/commits/lowmem for now, but I'll merge it into 1mb_hardforked once it's been tested for more than a few minutes.

[1714221498]

1714221498

[1714221498]

1714221498

[1714221498]

1714221498

[windpath]

... reduce the CPython memory consumption of this data by about 10-15x, or of the full share size by a little more than 3x, without any loss of functionality.

That's a pretty significant improvement, awesome!!

[veqtrus]

Most of p2pool's performance issues could be solved by switching to a statically typed language...

[jtoomim]

About half would be solved by switching to something like C or Rust. We could rewrite the whole codebase to solve those problems, sure. We could also profile the code to find those problems and tweak a few lines or rewrite a module in C using Cython or CFFI. Personally, the latter approach sounds easier and more fun to me, so that's what I've been doing. Switching to pypy makes a big difference, so at this point it might be enough to just publish binaries (or source-based installation instructions) that use pypy.

The other half of the performance problems are from algorithmic/design issues or network bandwidth and latency, and are easier to solve in Python.

[Rabinovitch]

https://github.com/jtoomim/p2pool/tree/1mb_hardforked has new code. This should fix the sync issues that people have been having. It also includes a few bootstrap nodes to make it easier for people to connect to the jtoomimnet p2pool.

To set up and run my fork, you will need to do the regular steps for installing p2pool, except that instead of getting the regular github.com/p2pool/p2pool repository, you will do:

Code:

git clone https://github.com/jtoomim/p2pool
cd p2pool
git checkout 1mb_hardforked

If anyone continues to have trouble connecting to jtoomimnet, please let me know.

Why not to screw any smooth and nice frontend to your p2pool fork by default? Everyone who deploy a node for mining will then set some other frontend instead of current ugly default one...

[tubexc]

P2pool seems to be alive again !!! 

http://imgur.com/3LBaNYA

[justdigging]

Is something negative happening to the pool?

My hash rate is good, but my expected payout
has been dropping the couple Days

My node uptime reset from 45+ days to 10 hours
and we're down a couple users.

Efficiency seems to have gone up, from low 80s
To low 90s.

Looks like some of the pool stats are down,
TH - expected payout....

I'm looking to switch nodes.

I'm new to mining, would appreciate any advice/insight.

Thanks!

[windpath]

Is something negative happening to the pool?

My hash rate is good, but my expected payout
has been dropping the couple Days

My node uptime reset from 45+ days to 10 hours
and we're down a couple users.

Efficiency seems to have gone up, from low 80s
To low 90s.

Looks like some of the pool stats are down,
TH - expected payout....

I'm looking to switch nodes.

I'm new to mining, would appreciate any advice/insight.

Thanks!

As the overal pool hashrate increases your expected payout for a given hashrate will decrease, but we are also expected to find blocks more frequently so in the end your payout will be about the same...

[jtoomim]

Why not to screw any smooth and nice frontend to your p2pool fork by default?

I'm open to the idea. The ugly one has additional information that is useful for debugging, so it hasn't been a priority for me. If someone wants to make a PR, I'd strongly consider it.

[Duce]

Is there something I am missing about the Avalon 741 setup and P2P as it is not getting the amount of shares to reflect the payout as others with the same hash rate. I know shares and luck come into play but there is such a difference I feel the need to ask. My node is not forked, running version 16.0-4-gde1be30. I ran a node in the 2012-2014 timeframe so I had some catching up to do and read through the pages from where I left off. I see the old relay has now changed, there is a forked P2P version running, and some of the configuration options used in the past are not optimal for the current environment.

The Avalon 741 has the new driver and MM updates. If it is one of those issues where it is just not compatible with the quick "start-drop-start" environment of P2P that will be a shame as I enjoy this rather than conventional pools.

Any suggestions/help would be appreciated. Yes, it is good to be back. I see some of the old players are still around as well.

[justdigging]

stratum+tcp://p2pool.org:9332 - seems to have been down/dead/frozen for nearly an hour.

I've been posting how there has seemed to be some issues with the Node I was at, or
maybe it's P2Pool itself?

i was connected here: stratum+tcp://p2pool.org:9332

I just switched over to: p2pool () ‐ node dashboard - SegWit - Eurasian server
                                 82.200.205.30:9332

Anyone else experiencing/know anything?

[justdigging]

(apologies to the veteran miners, i'm still new here)

I mentioned it earlier that I had understood that PING Response Time does matter in the
sense of being able to pickup/report back work performed.

I had +/- 100ms to the node I was on, before it went dead, so i stayed there.

I found another node at 30MS, which I was advised NOT to mine there.

The next best performance, and with more than 5 users, is responding between 300-500,
the 2nd & 3rd address  responds at around 200ms.

I know it doesn't have anything to do with hashing, but I understood the faster you can
pickup and report back, was better.

[sawa]

I just switched over to: p2pool () ‐ node dashboard - SegWit - Eurasian server
                                 82.200.205.30:9332

This is my node. More info & complete list of nodes are in first message here: https://bitcointalk.org/index.php?topic=852083.0

[justdigging]

I just switched over to: p2pool () ‐ node dashboard - SegWit - Eurasian server
                                 82.200.205.30:9332

This is my node. More info & complete list of nodes are in first message here: https://bitcointalk.org/index.php?topic=852083.0

Thank you, will do!

[frodocooper]

Is there something I am missing about the Avalon 741 setup and P2P as it is not getting the amount of shares to reflect the payout as others with the same hash rate. I know shares and luck come into play but there is such a difference I feel the need to ask...

How much of a difference are we talking about?

The Avalon 741 has the new driver and MM updates. If it is one of those issues where it is just not compatible with the quick "start-drop-start" environment of P2P that will be a shame as I enjoy this rather than conventional pools.

Check if one, or some, of your 741's are dropping off after a while of mining. I had been struggling with an issue where both of my 741's would detach from the Controller at random, and only one 741 would reattach about a minute later. This beta 7411706-3e3ab60 MM firmware fixes that bug. The latest stable 7411706-3162860 MM firmware doesn't.

Edit: If you haven't already, try running your 741's with the "--failover-only" option. And be sure that both your node's and Controller's clocks are set correctly and have their NTP clients enabled and synced to "pool.ntp.org". In some rare and random cases, this, and setting both clocks to UTC, might lower your DOA rates.

[Duce]

Is there something I am missing about the Avalon 741 setup and P2P as it is not getting the amount of shares to reflect the payout as others with the same hash rate. I know shares and luck come into play but there is such a difference I feel the need to ask...

How much of a difference are we talking about?

The Avalon 741 has the new driver and MM updates. If it is one of those issues where it is just not compatible with the quick "start-drop-start" environment of P2P that will be a shame as I enjoy this rather than conventional pools.

Check if one, or some, of your 741's are dropping off after a while of mining. I had been struggling with an issue where both of my 741's would detach from the Controller at random, and only one 741 would reattach about a minute later. This beta 7411706-3e3ab60 MM firmware fixes that bug. The latest stable 7411706-3162860 MM firmware doesn't.

Edit: If you haven't already, try running your 741's with the "--failover-only" option. And be sure that both your node's and Controller's clocks are set correctly and have their NTP clients enabled and synced to "pool.ntp.org". In some rare and random cases, this, and setting both clocks to UTC, might lower your DOA rates.

Windpath's active tab had me ~3TH for one 741

Interesting about the MM Firmware, I will try that.

I do failover only but the NTP client I used was Colorado, I will change them. Thanks for the info, is there a 741 topic I missed?

[windpath]

I'm unable to get p2pool.org up and running again remotely, ill be able to access the server in the morning... I hope everyone has solid backup p2pool nodes in their config

[justdigging]

I'm unable to get p2pool.org up and running again remotely, ill be able to access the server in the morning... I hope everyone has solid backup p2pool nodes in their config

Good luck with that!

I had been on there for nearly 2 weeks

What's the status of the Node Version?

Also what is the Node Fee?  It shows at 0% on the Node Scanner page, but on the 'node page' it was 1%+1%

thanks!

[frodocooper]

Windpath's active tab had me ~3TH for one 741

Ah, then the difference is most likely due to luck and variance. Windpath's miner stats calculate your hashrate based on how many accepted shares — weighted by share difficulty — you currently have in the P2Pool sharechain relative to the entire pool. It is quite normal for Windpath's stats to differ from your actual miners' hashrate, since Windpath's stats are heavily influenced by your miners' luck and the pool's total hashrate. For example, if a large miner suddenly dumps a significant amount of hashrate on P2Pool and gobbles up a large amount of accepted shares within a short amount of time, expect to see everyone else's hashrate on Windpath's stats drop significantly while the stats readjust to compensate for the large miner's presence. Similarly, when the large miner leaves P2Pool, expect to see your hashrate on Windpath's stats remain at the low end while your miners catch up on the additional shares now available in the large miner's absence.

P2Pool's total hashrate — mainnet, at least — has been fluctuating a lot recently. So the differences you see in Windpath's stats should not be of much concern. What matters most, and what you should be monitoring closely, is your efficiency relative to the pool's.

I'd wager that your miners' P2Pool efficiency is a little bit on the low end. Have a look at this guide to see where else you might be able to eke out a little bit more efficiency from your miners and your P2Pool node.

Interesting about the MM Firmware, I will try that.

I do failover only but the NTP client I used was Colorado, I will change them. Thanks for the info, is there a 741 topic I missed?

No. The bug fix was in response to a support ticket that I opened with Canaan. It's still in beta, though, which is probably why there hasn't been any official word from Canaan regarding the fix. But then again, Canaan doesn't have an official thread or account on bitcointalk, as far as I know, nor do they usually announce the release of new software or firmware.

There is an unofficial Avalon A7 thread here.

[Duce]

^^frodocooper-Yeah I get the variance on P2P but this happened before we ramped up last week that is why I started to dig a bit. All good info you have though.
Soi I had read the A7 thread, I guess as there is not a problem with the Avalon units there is not the need for specific threads as with other units.