Block on jtoomimnet (Bitcoin 1MB chain): https://blockchain.info/block-index/1611649

Mempool wasn't full, so only 504 kB. Previous and subsequent blocks from other pools were also not full.

Sorry this took so long. I haven't fully tested this to make sure it works on testnet and can make segwit blocks, but if people can help review it and maybe help test it, that would be nice.

https://github.com/jtoomim/p2pool/commits/1mb_segwit

Segwit compatibility stuff starts with the July 19th commits.

For the UAHF, I have an idea for how to deal with it. Each time we receive a share, we can query our bitcoind to see whether that share's previousblockhash refers to a valid block hash in our bitcoind's blockchain (via the getblockheader RPC). If it is, we accept the share and memoize that hash. If it isn't, we ignore that share. This should allow people to mine on either blockchain without needing to worry about people getting valid shares while mining on the other blockchain.

Unfortunately, I'm very busy right now with datacenter stuff, so I don't know when I'll have a chance to implement this.

Correct, jtoomimnet finally found a block. It's been 2 months 2 days since the last one.

SixOfFive: I would like to have p2pool support both branches of the hard fork, but making p2pool fork-aware will take a bit of work. I've been really busy recently with an 8 MW datacenter project we are now starting, so I haven't had as much time to dedicate to p2pool as I'd like, but I'll try to get it working eventually. It probably will not be ready for the launch date, though.

SegWit support is in beta on both mainnet and jtoomimnet. I need to finish some testing on the new code ASAP. I should probably just push it to my github so that others can help test it. I'll try to at least do that much today.

It's not a simple version check for p2pool, unfortunately, because the duration of the version check is very specific, and it requires a copy of the blockchain (which p2pool does not have) in order to know what that duration is.

-ck, the question is whether p2pool should allow BIP91-incompatible shares, not blocks. I'm fine with people mining BIP91-incompatible blocks if they want to, but it's not clear that they should get rewarded by other p2poolers for their folly.

The point I was trying to make is that btc1 and other BIP91 clients will blacklist certain blocks, and the current version of p2pool will ignore and violate that blacklisting. This needs to be fixed. P2pool should not assume that blocks that it hears about over the network and not via bitcoind are valid when chainsplits are occurring.

Basically, the tip that p2pool sees will often be the wrong tip. P2pool needs to trust its bitcoind for the next month.

Making sure that btc1/bitcoind always has the best block is a separate issue. All btc1 miners need to ensure that they have used addnode or one of Matt Corallo's relay networks to connect to other btc1 miners, but that is not something that p2pool can control.

This is a complicated question. Currently, there's an incompatibility between p2pool and BIP91 in that p2pool will mine a transaction-free block on top of the longest chain seen if p2pool hears about a new block but bitcoind does not inform p2pool about this block. While I think this is the correct behavior during normal operation, it is clearly incorrect behavior during chainsplit conditions, so it will need to be disabled for the next month or so. I will be including that change in the new branch I'm working on that merges SegWit and large block generation capability.

I am not convinced that this is the correct action to take. P2pool generally does not validate the blocks that other nodes are trying to mine except to ensure that the payouts are fair. I'm not sure that we should start now. On one hand, it serves as an added incentive for people to ensure that they are BIP91 compatible. On the other hand, it disincentivizes miners using p2pool as a tool to vote their conscience. I tend to believe that the latter objective is generally more important, but I'm not sure in this particular case. It may be that people would want to exploit p2pool to mine minority-chain blocks while still getting paid on the majority chain, which could be a problem. Perhaps the best solution is just to fork the share chain again, and let anyone who wants to disobey BIP91 do so on their own minority sharechain. However, that approach would require the most work of any.

That would actually be a difficult rule, since p2pool does not store a copy of the blockchain anywhere, and is unable to evaluate whether 750 of the last 1000 blocks have it set.

Using a web browser, navigate to your node's status page, and click on one of the links in My Shares. Then look for a section that looks like this:

(I recently switched my node over to btc1 too.)

The bitcoind version you're using (Unlimited/Core/btc1/etc) will include a version bits suggestion which gets passed to your p2pool node in the getblocktemplate response. Depending on which version of the p2pool code you're using, your p2pool node may either use the version bits it receives from bitcoind, or it may override it (see https://github.com/jtoomim/p2pool/blob/lowmem/p2pool/work.py#L386 for an example -- I've never liked that max() line, but haven't changed it in my branch yet). If you want to force particular version bits even though the bitcoind you're using doesn't make those version bits, you can do so by editing that line in your p2pool source code. Each p2pool node can use different version bits, so the final block version bits that you see depends on which p2pool node actually mined the block.

I just finished a beta version of a merge of veqtrus's SegWit code with my 1mb_hardforked+lowmem code. I've also added a couple of things that should improve the likelihood of it working with altcoins. I'm testing with Bitcoin right now. I'll try to have the testing done and the Bitcoin code ready for others to use within a couple of days, and then I'll work on the altcoin stuff and test it with Litecoin.

The blocker on merging is still testing with altcoins. If someone can set up a node for a non-litecoin alt using the 1mb_hardforked or lowmem branches and mine to it for a while, and maybe verify that it forks cleanly to v17 and then v32 shares, that would speed things up a lot.

A better analogy would be using the bittorrent protocol to download different files. Each of the different p2pool forks represents a different share chain, a different set of users, and a different history.

If you switch share chains, you don't "lose" your work. If a block is found on the chain you left within ~3 days of you leaving, you get paid for the work you did on the old chain. The math works out such that if two p2pool chains are equally efficient, then no matter which chain you mine on, the expected value of your payout remains the same regardless of how much time you spend on each chain and how often you switch between chains.

(Of course, the two chains are not equally efficient. jtoomimnet has about 2x higher transaction fees, so your expected revenue will be higher on jtoomimnet than on mainnet. Recently, jtoomimnet's expected revenue has been about 10% higher than mainnet.)

Been there, done that.

I think the proper response to our little flamewar is to bring popcorn.

veqtrus makes some good technical comments and observations at times, and I value his contributions. However, when he trolls me on how my fork is gonna doom p2pool to a centralized future, I think the proper response is to troll him back, laugh it off, and move on.

I'm not very good at trolling, though. Maybe I should ask Kano for a lesson.

New shares are already as big as they're going to get in the absence of explicitly malicious behavior or Bitcoin blocksize increases.

Do you mean in the sense that nobody has attacked the p2pool network with large, difficult-to-propagate shares, as an adjunct to doing a selfish mining attack, because somehow that's easier than just renting a bunch of hashrate on Nicehash and 51% attacking p2pool?

Or intentionally creating large shares with the intent of forcing low-bandwidth miners off of p2pool and onto other pools because somehow there's an incentive for someone to want to do that?

Keep in mind that with a 1 MB block size limit, the only way to have shares that require 1 MB of network traffic is to create block templates that use transactions that nobody has heard of before (i.e. transactions you created yourself). This also requires hacking p2pool so that it does not notify peers of transactions when they're first seen (i.e. disabling the have_tx p2p message). Normally, shares require only 2 to 32 bytes per transaction when transmitted regardless of the transaction size, but the 50 kB/1 MB limits apply to the total summed size of the transactions (e.g. 500 bytes).

Even to the extent these attacks are possible, they wouldn't be very effective. If you want to DoS someone, there are much easier ways than mining big shares.

Maybe because my fork has lower CPU usage, lower memory usage, lower orphan rates, better fairness, and substantially higher revenue from tx fees? And maybe because your premise of that causing centralization is, I dunno, wrong?

Looks like another nicehash miner with high DOA rates, but on mainnet this time. It's interesting to see how they drive the orphan rates down even as they drive the DOA rates up.

User-configurable thresholds aren't hard. Each share you mine could contain a committed number representing your desired payment threshold. The current active threshold for each user is the threshold specified in the most recent share mined by that user.

While I've thought about the minimum payout problem and possible solutions, I don't intend to invest any time implementing them in the near future.

Sorry Sergio, but that's not a cool idea. For one, it doesn't actually help the fairness problem at all, which is due to DOA/orphan rates being unequal between miners. SDL's proposal only helps with the minimum viable payout size and transaction fees, which is not a problem that we really have with one block a week.

Besides, there are much simpler ways to fix that problem. P2pool could easily encode a balance for each user into the share structure, and only make payouts for users whose balance (including this block) exceeds some threshold. Alternately, for a stateless system, if a user is owed 0.001 BTC and the minimum payout is 0.0015 BTC, we could pay out 2*.001=0.002 BTC each time a block is mined if (parent_hash % 2) == 0. Replace 2 with n for even smaller amounts owed.

Does the amount of network traffic used by p2pool bother anyone? I have some ideas for how to reduce the amount of traffic (without improving propagation performance), but the traffic doesn't bother me, so I haven't made implementing it a priority.

The other project I have in mind is trying to improve fairness independent of performance by using some sort of share DAG (e.g. with uncles) or one of the other ideas mentioned in https://bitcointalk.org/index.php?topic=18313.msg19248232#msg19248232. That's going to require far more work than the network traffic reduction, though.

Alternately, I could work on polishing the current code a bit, merging into the main p2pool repo, and organizing the community to upgrade. If I do that before the fairness work, that means that we have to do another big upgrade later.

Thoughts?

I'm open to the idea. The ugly one has additional information that is useful for debugging, so it hasn't been a priority for me. If someone wants to make a PR, I'd strongly consider it.

About half would be solved by switching to something like C or Rust. We could rewrite the whole codebase to solve those problems, sure. We could also profile the code to find those problems and tweak a few lines or rewrite a module in C using Cython or CFFI. Personally, the latter approach sounds easier and more fun to me, so that's what I've been doing. Switching to pypy makes a big difference, so at this point it might be enough to just publish binaries (or source-based installation instructions) that use pypy.

The other half of the performance problems are from algorithmic/design issues or network bandwidth and latency, and are easier to solve in Python.

I'm looking into the memory consumption issue now. Here's some data when running CPython:

Of the first 100 shares in the share chain, the average memory consumption (using pympler.asizeof) is 57 kB per share, with most shares between 20 kB and 100 kB.

Of this 57.3 kB average, 43.5 kB on average is taken up in share_info.transaction_hash_refs. See the background paragraph at the end if you want to know what that does. Each of these ints takes 24 bytes of RAM in CPython, even though we only need 1 or 2 bytes for each one based on the size of each int. It might be possible to encode these numbers as an array of uint8 and/or uint16 integers instead of regular python integers, and reduce the CPython memory consumption of this data by about 10-15x, or of the full share size by a little more than 3x, without any loss of functionality.

On the other hand, the list of new transaction hashes averaged 7 kB per share. These are 32 byte integers, but they take 64 bytes of Python memory. We might be able to save some memory here by using an array of strings (or just one really long string), but that would be inconvenient, and would only save about 2x memory on that variable, so it's probably not worth it.

Another option is to just forget all of the transaction data after the share is >200 shares away from the head of the chain, and reload it from disk if it's requested. This will probably be more work, and might open up some DoS vulnerabilities if I'm not careful with the code (since someone could mine a share that requires us to reload 200 shares off disk, and share parsing is hella slow right now), but would probably be able to reduce memory consumption by around 20x if done well.

I can't do the same tests on pypy, since sys.getsizeof() and asizeof() don't work on pypy, unfortunately.


Background on how p2pool's share format handles transactions: For each transaction in a share/block, p2pool will check to see if that transaction hash was encoded into one of the 200 previous shares in the share chain. P2pool then encodes that in share_info.transaction_hash_refs by referring to the share number (where 0 is this share, 1 is this share's parent, 2 is the grandparent, etc) and the index of that transaction in that share's share_info.new_transaction_hashes. If the transaction hash wasn't in a previous share, then p2pool also sticks that transaction hash into share_info.new_transaction_hashes. When sent over the network, these numbers are encoded as var_ints, so it's usually 1 byte for the share index, and 1-2 bytes for the transaction index, plus 32 bytes for each of the new hashes.

Edit: Yeah, the array of uint16 thing seems to work pretty well, at least on CPython. 2170 MB -> 489 MB = 4.44x reduction.


Edit 2: Seems to benefit pypy (5.7.1) even more. 5310 MB -> 785 MB = 6.76x reduction. Now I just need to make sure the new code can successfully mine shares...

Edit 3: Seems to mine shares just fine. Neat. https://github.com/jtoomim/p2pool/commits/lowmem for now, but I'll merge it into 1mb_hardforked once it's been tested for more than a few minutes.