smoothrunnings, if you're planning on mining Bitcoin with p2pool, I strongly advise using pypy instead of regular Python. Mining with regular Python will usually result in CPU overload, which will reduce your revenue by 20-40%. This is true both for jtoomimnet and for mainnet.

Instructions for installing pypy on WSL on Windows can be found here: https://bitcointalk.org/index.php?topic=18313.msg21025074#msg21025074

Python is inherently single-threaded. https://wiki.python.org/moin/GlobalInterpreterLock

It would be far easier and more effective to remove most of the transaction-handling code from p2pool, and make share objects contain only the merkle root plus the coinbase transaction (and the merkle path thereto). However, I have not had the time to do that, and nobody else seems to be interested in doing it. Even though it would be much easier, it's not super easy to do.

Note: Bitcoin Core is *also* mostly single-threaded, although not quite to the same extent as p2pool.

"Per core" is not meaningful here. Python can only use one core at a time, and it's bouncing around between all four cores. If it's using about 20% of each core, that means that 80% of the time it is using at least one core, which is too much.

Your getblocktemplate latency graph shows the same thing. It's taking about 2 seconds for p2pool/python to process the GBT results returned by bitcoind, which indicates that p2pool has a short backlog of tasks for it to perform that is delaying the GBT processing.

Reloading the webpage is sluggish (~2 second delay), which indicates the same thing.

You're running into CPU load issues. Try pypy and/or a CPU with faster single-threaded performance. Pypy is likely to make a bigger difference than a CPU upgrade. Reducing your peer count can also help.

There are two likely causes: One, maybe your mining hardware is not compatible with the frequent work changes that p2pool requires. Two, maybe your node is slow.

1) What type of miner are you using? If you point it to someone else's node which is known to be working well (e.g. ml.toom.im:9332), what DOA rate do you get?

2) You'll have better results if you use pypy. That's easiest to do if you're running Linux, but you may also be able to get it to work using the Windows Subsystem for Linux:

https://bitcointalk.org/index.php?topic=18313.msg21025074#msg21025074

You can also check your CPU usage to see if it ever gets pegged at 25% (i.e. 1 full core used).

P.S.: The Bitcoin Cash p2pool has different chain parameters that results in less work switching, so you will see better performance and efficiency there. However, it doesn't have much hashrate yet, so it might be a while before it finds a block.

P.P.S.: A higher dbcache setting would be a good idea, though it won't solve your problem. Also, blockmaxweight=4000000 is safe on 1mb_segwit.

The 3 PH/s spike on mainnet is over. It was probably just someone renting on Nicehash for an hour or so. Mainnet is back to ~1 PH/s.

It is not possible to "merge" the two networks. They are separate share chains. What is possible is for people to switch chains so that everyone is on the same chain.

No, it's not. At this particular moment, I see:

16.72565364 BTC/block on mainnet in 1681 transactions
17.48822478 BTC/block on jtoomimnet in 2026 transactions

You have to look at the actual shares being mined to see the difference. The "Current block value" metric on a node's front page shows the value before mainnet's transaction trimming takes place.

Something to keep in mind is that the bug/design flaw on mainnet that forces small blocks is intermittent. It shows up when a Bitcoin block has been recently published, and disappears a few minutes later. Just because they show the same thing at one moment doesn't mean it will always be the same.

Both 32 bit and 64 bit versions of Python should work. I don't think I've tested with 32 bit versions before, though.

ml.toom.im:9332 and ml.toom.im:9334 are available for public use. Zero fee.

None yet. Still need to get altcoins working with my code, or verify that they work. I have not had any time to work on p2pool stuff recently, alas.

Roughly, this: set PERSIST=False in p2pool/networks/(networkname).py or p2pool/bitcoin/networks/(networkname).py, then run it with the maximum peers set to 0.

Let me know if you have trouble and I'll give more specific instructions.

Lack of big and medium miners, mostly.

Your phrasing is mostly identical to frodocooper's. You use this sentence

Which is word-for-word identical to frodocooper's post. https://bitcointalk.org/index.php?topic=18313.msg18761195#msg18761195

You then went on to say
Whereas frodocooper's post said

Frodocooper's account was 4 days old at the time he made that post. The jennaya account is 6 days old. The jennaya account also has recent posts offering "Social Media Campaigne ideas" from India and asking "Do you need best Internet marketing policy?"

What is going on here? Are you both reading from the same script? Is this two accounts belonging to the same person, who is copy-pasting this concern-troll message everywhere as part of a PR campaign? Or is there some other explanation for the similarity in these two sets of messages? Did jennaya just read through this thread's history and decide to plagiarize frodocooper's post as jennaya's own?

Curiouser and curiouser.

Map auth.minerlink.com to 127.0.0.1 on your LAN's DNS server.

https://bitcointalk.org/index.php?topic=18313.msg18761421#msg18761421

Given current fees on the Bitcoin network, in order for donations to be spendable, you need to give more than the fee it would cost to spend it. Currently, that's about 500 sat/byte * 200 bytes = 0.001 BTC per user, minimum. Given how many users there are and the hashrate distribution, a donation smaller than about 0.1 BTC would end up mostly wasted on fees.

This metric is bugged. For example, my big node lists "Expected time to share: 1.2 seconds", which is impossible to sustain, as the protocol-level minimum share difficulty will adjust upward until the network as a whole creates one share every 30 seconds.

A difficulty of 1520000 should result in an average of 1 share every 1520000 *2**48/0xffff = 6.53 PH. With a hashrate of 20 TH/s, that should take on average 5.44 hours.

On my to-do list is changing a few things with the difficulty-setting and share-orphaning algorithms to improve both fairness and also to assign large miners higher-difficulty shares. This should substantially improve the number of shares small miners mine. It will not increase expected revenue for small miners, though; it will only reduce the variance.

The averaging period that determines your revenue per block is approximately 3 days long, so you are expected to have around 14 shares in the share chain at steady state, and so your variance will be around 1/sqrt(14)~=27%.

Correct. On the other hand, if you find 2x as many shares as the expectation, then you get 2x the payout.

Yes, after about 3 days you will be back to ground zero. However, if p2pool finds a block 1.5 days after you stop mining, then you will still get paid at about 50% of your normal payout level. Your expected payout ramps up slowly over 3 days, and it ramps down slowly over 3 days.

Number of shares found on this node since it was (re)started.

That sounds weird. Perhaps you have your worker name/address set incorrectly? If that's the case, then you will end up mining to that node's default address, which usually belongs to the node operator. Your worker name needs to be a Bitcoin address starting with a 1 (no segwit or multisig addresses) and must not have an _ or ., though you may use + or / to set the pseudoshare or actual share difficulty.

The initial share difficulty on p2pool/jtoomim is by default higher due to a bug I ran into with nodes with very high hashrate. If you have low hashrate (e.g. less than 3TH/s), it can be a good idea to manually set the difficulty by adding something like +512/65536 after your Bitcoin address in the stratum username field.

https://bitcointalk.org/index.php?topic=18313.msg26151135#msg26151135

Share difficulty is adjusted based on hashrate. Also, the interface you're looking at will always display the most recent 20 shares no matter how many total there are.

Also, the node running at p2pool.org is overloaded and has poor performance (efficiency at 70%), so it is recommended that you run your own node or at least use a node with a faster CPU.

Check your .litecoin/debug.log file and search for the block's hash. If it was invalid, the debug.log should say so. If it was valid, your debug.log should say something else, like UpdateTip (if you mined this block before your local node had heard of any other blocks). Feel free to post part of the log if you want help analyzing it. `grep -C 20 <hash> ~/.litecoin/debug.log` will find all lines that are within 20 lines of a line containing <hash>.

Once my code is working and tested with altcoins (e.g. Litecoin), I will submit a pull request and ask forrestv to merge it in.

Are you using the most recent version of 1mb_segwit? Can you post the log output in code tags?

Amazing. On jtoomimnet BTC, transaction fees are almost equal to the block subsidy:

Mainnet BTC, for comparison:

mainnet p2pool will reject the last share mined in each block interval. For Litecoin, a share is mined roughly every 30 seconds and a block is mined roughly every 150 seconds, so this causes 30/150 = 20% of shares to get orphaned. P2pool will report "punishing share" and "block-stale detected!" when this happens. This effect happens to everybody almost equally, so it does not cause imbalance. The mining that is done in the last 30 seconds of each block interval can still be used to mine an actual block and pay all p2pool users, so it's not lost hashrate; it's merely unaccounted hashrate. However, if you are a large enough miner, you can mine another share on top of that when it happens to you and prevent your hashing from being unaccounted like everyone else's, so this mechanism *is* a centralization/selfish mining risk.

I eliminated this mechanism in jtoomimnet:

https://github.com/jtoomim/p2pool/commit/b57a4ff93e58c0702aa2481c164517e7290c8d43
https://bitcointalk.org/index.php?topic=18313.msg18580559#msg18580559

It's far less likely to be an intentional attack than it is to be plain performance problems. The litecoin network has seen higher transaction volume recently, and that results in higher p2pool CPU load. If your CPU is not fast enough, it will take several seconds to process a share and issue new work to miners. Any work returned from your miners during that time will be considered DOA (dead) shares. Once you find a share, you have to send it to all of your peers, and they have to send it to all of theirs, etc. While the share is being propagated, there's a chance that someone else will mine a share that will out-propagate yours, which can cause your shares to be orphaned. Furthermore, each time mainnet p2pool changes blocks, the most recent share gets orphaned unless the person who mines the next share is the same as the person who mined that share. (This third effect was removed on jtoomimnet.) These three effects make p2pool biased towards miners with a lot of hashrate, and that bias increases as transaction volume increases. The first two biases are substantially reduced if node operators use CPUs with high single-threaded performance (e.g. Core i7 4790k, which runs at 4.4 GHz) and if they use pypy.

I've put in a lot of work on p2pool's code on jtoomimnet to reduce the CPU load of p2pool and reduce these biases, but I have not yet managed to get jtoomimnet working for Litecoin. It might be ready now, actually -- the most recent issue I had was the port conflict with Bitcoin Cash, which I fixed, but I haven't tested it since. However, the improvements I've made so far do not eliminate the problem, and probably only reduce it by about 50% or so. In the long term, there are a few major design changes I can make to p2pool to nearly completely eliminate this issue, but I have not had enough time to implement one yet.

The most important metric is your node's efficiency rating. If your efficiency is above 100%, you're gaining extra revenue at someone else's expense due to the average miner having more DOA and orphan shares than you. If it's below 100%, you're losing revenue because you have more DOAs and orphans than average. If your efficiency is 90%, for example, then you're effectively donating 10% of your potential revenue and hashrate to other miners who have better node hardware and maybe more hashrate than you. If you're not okay with that, then you should either upgrade your node hardware or (ugh) switch to a centralized pool.

The data you've described suggests that the new large miners are poorly configured (e.g. running on slow CPUs using CPython), and are losing revenue because of that. If that's the case, other miners are probably benefiting from their incompetence.

No, the smartest attack would not be to submit mass invalid shares purposefully. Attacks would probably take the form of submitting valid shares, but ignoring valid shares submitted by users other than the selfish miner. If this is the case, then the node that is selfishly mining will show a low-normal orphan and DOA rate, and everyone else will see normal DOA rates but high orphan rates.