[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[frodocooper]

I consider 4 Mbps to be a reasonable minimum HW spec for p2pool

Yes, this is indeed a reasonable spec for a private pool.

With the way you're stubbornly blocking reasonable solutions from being implemented in mainnet P2Pool — solutions that arguably go a long way in solving many of P2Pool's teething problems, with benefits that far outweigh the risks, and where even kano (who has more to gain from mainnet P2Pool remaining in its current artificially anemic state, since he can all the more persuade current and potential mainnet P2Pool miners to mine with his pool instead) agrees that it would be for P2Pool's greater good — mainnet P2Pool has become your private pool, veqtrus.

[veqtrus]

veqtrus, is it your opinion that people should be able to mine on p2pool via ISDN?

No, 4 Mbps upload bandwidth is nowhere close to ISDN.

[veqtrus]

With the way you're stubbornly blocking reasonable solutions from being implemented in mainnet P2Pool — solutions that arguably go a long way in solving many of P2Pool's teething problems, with benefits that far outweigh the risks, and where even kano (who has more to gain from mainnet P2Pool remaining in its current artificially anemic state, since he can all the more persuade current and potential mainnet P2Pool miners to mine with his pool instead) agrees that it would be for P2Pool's greater good — mainnet P2Pool has become your private pool, veqtrus.

I'm not in control of the main p2pool repo. ForrestV can merge what he wants. It is not my job to tell him to merge stuff I don't agree with.

[jtoomim]

veqtrus, is it your opinion that people should be able to mine on p2pool via ISDN?

No, 4 Mbps upload bandwidth is nowhere close to ISDN.

ISDN is 16 KB/s symmetric. The 50-100 kB share limit in adversarial conditions would be around 3-7 seconds on ISDN. Your advocacy of a 100 kB share limit seems to me like you're trying to protect people mining with ISDN, or maybe 1st-gen ADSL. You know, the speeds we had in 1999.

By the way, the relevant metric for adversarial conditions is download bandwidth, not upload.

You don't have to upload other people's shares in a timely fashion. You only need to upload your own shares quickly. Adversarial conditions don't apply to the shares you upload yourself. The existing code will send the transactions in a getblocktemplate request to peers as soon as you start mining on them. The peer will usually have received all of those transactions before you find the share, so you usually only have to upload the hashes for those transactions. For a share that adds 100 kB of transactions, that's expected to be around 8 kB of hashes. Again, ISDN spec.

However, an adversarial entity could modify their p2pool code to NOT send transactions out in advance. Doing so would substantially increase their orphan rate, of course, but let's say they want to do it anyway for selfish mining purposes. In this case, they could make a share with 100 kB of completely new transactions, and they would make you download that 100+8 kB ASAP so you could switch to mining on top of it. Until you get their share from them, your mining would be competing with theirs, and you would possibly end up orphaning their share (or they yours). That might take 650 ms if you have 1.5 Mbps ADSL, or 6.75 seconds if you have ISDN (suboptimal, but potentially tolerable for short periods).

[veqtrus]

You are assuming that a user's connection will be dedicated to p2pool, where at least bitcoind is required for p2pool to even function, let alone using the internet for other purposes.

[jtoomim]

I'm just trying to figure out what the min HW spec you envision is. You haven't said it yourself, so I'm reduced to guessing.

I've been clear about the HW min spec that I envision. My assumption is that almost nobody is going to try to run p2pool with < 1 Mbit/s of available bandwidth. My assumption is that anybody who tries to run p2pool will have at least 4 Mbit/s of downstream bandwidth and 1 Mbit/s of upstream bandwidth, and that it is okay to use up to 0.5 Mbit/s sustained in each direction during normal operation. My guess is that most users will have about 10x that amount of bandwidth available.

My conclusion is that it's not a good idea to cripple p2pool's revenue capability for everyone just in case someone wants to use p2pool without meeting that reasonable HW minimum.

By the way, I metered one of my bitcoind processes. With 8 peers, I'm getting around 2.2 kB/s up, 3.4 kB/s down. It might be possible to keep a Bitcoin full node synced with just a 56k modem. If anyone else is curious: sudo iftop -B -f "port 8333". That assumes all of your peers are on that port for either the source or the destination, as mine are. You can verify that manually with bitcoin-cli getpeerinfo. This is with btc1 1.14.5.

let alone using the internet for other purposes.

I don't think anybody should be trying to use the internet nowadays with ISDN or <1Mbps ADSL in the first place. 4 Mbps is about what I consider to be the min HW spec for browsing the internet. Since jtoomimnet p2pool would only use 4 Mbit/s during adversarial conditions for a second or two at a time, I don't think it would significantly hamper the user experience for browsing the web.

[-ck]

By the way, I metered one of my bitcoind processes. With 8 peers, I'm getting around 2.2 kB/s up, 3.4 kB/s down. It might be possible to keep a Bitcoin full node synced with just a 56k modem. If anyone else is curious: sudo iftop -B -f "port 8333". That assumes all of your peers are on that port for either the source or the destination, as mine are. You can verify that manually with bitcoin-cli getpeerinfo. This is with btc1 1.14.5.

The network is actually surprisingly quiet at the moment with mempools actually draining close to empty. This is very different to what happens during spam transaction floods. Best to plan for the worst case scenario.

[jtoomim]

By the way, I metered one of my bitcoind processes. With 8 peers, I'm getting around 2.2 kB/s up, 3.4 kB/s down. It might be possible to keep a Bitcoin full node synced with just a 56k modem. If anyone else is curious: sudo iftop -B -f "port 8333". That assumes all of your peers are on that port for either the source or the destination, as mine are. You can verify that manually with bitcoin-cli getpeerinfo. This is with btc1 1.14.5.

The network is actually surprisingly quiet at the moment with mempools actually draining close to empty. This is very different to what happens during spam transaction floods. Best to plan for the worst case scenario.

The numbers I collected appear to correspond to about 3 tps. The biggest spike I see on https://blockchain.info/charts/transactions-per-second?timespan=1year is 22 tps. Scaling linearly, that would predict 16 kB/s up, 25 kB/s down during the peak of a spam attack for bitcoind. Given that watching a 240p video on Youtube uses around 87.5 kB/s (or 250 kB/s for 480p), I really don't think it's significant. If your internet connection is fast enough to watch movies on Youtube, then you can run p2pool and bitcoind under adversarial conditions.

[Blue Bear]

at least you two are being polite ...

[Blue Bear]

As I understand it what makes p2pool DOS resistant is the fact that there are many nodes running. To be effective a DOS attack would require most of the nodes to be overwhelmed simultaneously. An attacker would have to determine the most effective nodes to attack to disrupt the network. If the network is adaptive enough it will reach out to link with other nodes that are not under attack maintaining integrity of the network. A Centralized Pool which only has one or two access points would be easily over come in a DOS attack because the attack can be concentrated to limited access points. The more spread out things are harder it is to disrupt. so arguing that consolidation is the best option is an faulty tactic as demonstrated by the Japanese at Pearl Harbor. If you both want to consolidate under your banners your both wacked. The more nodes running the better.

I think you are mostly arguing semantics.

Wake up and smell the coffee. If people here wanted to be in a centralized pool they would already be in one.

BB

[jtoomim]

Nobody is arguing for a centralized pool.

I want p2pool to be profitable enough that people will want to mine on it. I believe that if p2pool is more profitable than most pools and also easy to use, then we will get more people using it and will see an increase in node counts and miner counts. To that end, I have been working hard to improve expected revenue and fairness, and to reduce HW requirements by improving CPU, RAM, and (soonish) network efficiency.

Veqtrus wants p2pool to sacrifice profitability in order to reduce worst-case bandwidth requirements. He thinks that p2pool will work better if p2pool can be used (albeit with lower revenue) during adversarial conditions even on very slow internet connections, such as 1st gen ADSL.

During typical usage, my code uses about 18% more bandwidth than p2pool master (33.4 kB/s vs 28.3 kB/s, sum of UL and DL, averaged over the last week, from http://crypto.mine.nu:9332/static/classic/graphs.html?Week and http://crypto.mine.nu:9330/static/classic/graphs.html?Week), and gets about 10% to 20% more revenue. I think that's a pretty awesome tradeoff.

Anyway, Blue Bear, I made some changes a week ago that improves the CPU usage of jtoomimnet. It's possible that you might be able to use it on CPython now without pegging your CPU at 100%. I would be interested to know if that's the case or not. If it's not, then I have some other changes in mind that could help some more. If you're not too busy, could you give it a try and let me know how it goes?

https://github.com/jtoomim/p2pool/tree/1mb_segwit

[frodocooper]

Anyway, Blue Bear, I made some changes a week ago that improves the CPU usage of jtoomimnet. It's possible that you might be able to use it on CPython now without pegging your CPU at 100%. I would be interested to know if that's the case or not. If it's not, then I have some other changes in mind that could help some more. If you're not too busy, could you give it a try and let me know how it goes?

https://github.com/jtoomim/p2pool/tree/1mb_segwit

I'm not Blue Bear, but for what it's worth, I'd like to say that commit f223000 managed to lower 1mb_segwit's memory requirements to the point where I'm now able to run my 1mb_segwit node with PyPy on a system with only 3.75 GB of RAM, and still have enough memory left to comfortably allocate 700 MB to Bitcoin Core's UTXO cache. In the last five days of continuous running, my 1mb_segwit node's memory consumption never exceeded 1.50 GB, mostly settling at the 1.45 GB mark. That's equivalent to my old lowmem node being run with CPython.

And what a difference PyPy makes. There has been a remarkable increase in performance across the board, most noticeable in the much lower orphan rate (3.5% on PyPy compared to roughly 7% on CPython) and in the roughly 0.2 s reduction in average GetBlockTemplate latencies. The only limiting factor in my P2Pool setup now is my AvalonMiner 741's. If it weren't for their sluggishness in keeping up with P2Pool's frequent work restarts, I'd most probably be enjoying low single-digit DOA percentages, instead of the 10%-15% I'm currently stuck with (although, to be fair, it is still a reasonably healthy 5% decrease when compared to running 1mb_segwit with CPython).

[jtoomim]

htle0006, we've had influxes of 15 PH/s from Nicehash renters in the past, and that worked okay. Mining from Nicehash is not recommended with p2pool, since the high latency added by Nicehash will increase DOA rates for whomever is doing the Nicehash mining. I know of some changes I can make to the code to reduce this problem, but I haven't had the time to implement them yet.

If someone were to jump in with 150 PH/s of miners that they physically control, I think it would work fine. There is currently a mechanism in place that will increase the difficulty of shares submitted by a single node to try to target 1/30th of the pool's total shares from a single node. While the code does not achieve that goal, it does successfully increase the share difficulty dramatically for large miners. I haven't done the math for 150 PH/s specifically, but my intuition says that the total variance experienced by small miners would be much less with 150 PH/s total hashrate than with 1.5 PH/s total hashrate. Let me know if you want me to try to do the math to model it. It would probably take an hour or two to do that.

Right now, about 95% of the total reward variance comes from the infrequency with which p2pool finds a block. If we got to 150 PH/s, we'd find multiple blocks in a day, and that variance would go way down, but the variance due to the difficulty of finding a share would go up a bit. My intuition says that with 150 PH/s, variance for small miners would be down to about 20% of what it is currently.

In terms of performance scalability, the code should be able to handle 150 PH/s just fine, especially if it's only a small number of mining addresses. The major scalability problems in p2pool are currently mostly with transaction volume and node count. A minor scalability problem is the number of unique mining addresses using a single node. The number of actual pieces of mining hardware (e.g. S9s) on a single node is basically irrelevant, as handing out copies of a single cached stratum job to separate machines is effectively free. As share difficulty scales with hashrate, the number of shares found would also not be an issue.

[Blue Bear]

good answer ...

basically it would benefit the pool as a whole if we had that much hashing power in the pool as we would find block more frequently and would be rewarded based on the percentage of work done. A large miner would be facing a higher difficulty than a small miner so the small miner would have a greater chance of getting shares. Frequency of payouts would be greater and is where the greatest benefit would be even if the pay outs are smaller. In the end it would even out or be greater than what the pool is able to get right now with the hash rate being so low.

BB

[kano]

good answer ...

basically it would benefit the pool as a whole if we had that much hashing power in the pool as we would find block more frequently and would be rewarded based on the percentage of work done. A large miner would be facing a higher difficulty than a small miner so the small miner would have a greater chance of getting shares. Frequency of payouts would be greater and is where the greatest benefit would be even if the pay outs are smaller. In the end it would even out or be greater than what the pool is able to get right now with the hash rate being so low.

BB

As long as the large miner is nice to everyone else and sets a much higher difficulty ...

[jtoomim]

htle0006, sounds good. Keep me updated of your progress. Quick question: will your farm be inside China? If so, I should do some network optimization work to help with crossing the Great Firewall.

As long as the large miner is nice to everyone else and sets a much higher difficulty ...

Rather, as long as the large miner isn't mean to everyone else by manually overriding the current difficulty code. If that happened, I'd either make it a consensus rule or I'd turn the share chain into a share DAG so that it didn't matter.

[Blue Bear]

does anyone here know about spondoolies sp10's?

I am trying to connect using putty and it keeps rejecting the password which I have set in the web interface.

[jtoomim]

It's a different password. The default ssh login and password for SP10s is root/root, even though the default web login/pass is admin/admin.

On the other hand, the default ssh login and password for Antminers is root/admin, even though the default web login/pass is root/root.

I've found that quite annoying over the years.

[jtoomim]

If China makes sense, by all means go ahead and set up a DC there. Don't worry about it too much. I know how to write networking code to cross the firewall without performance loss. I just need to know when it's necessary so that I have enough time to do it.

That said, USA and Iceland are both reasonable choices. Are you aiming for the Pacific NW? If so, we might be neighbors.

[windpath]

Lightning's Next Act: Decentralizing Bitcoin Mining?

https://www.coindesk.com/hub-and-spoke-could-bitcoins-lightning-network-decentralize-mining/