[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[frodocooper]

That is possible, but I would deem that worthy of a warning at most. I suspect that that misconfiguration is not common enough to merit the programming time.

Perhaps an addition to P2Pool's setup instructions, then, alongside the instructions for setting up bitcoind's RPC server.

Update: I restarted my node two hours ago. Memory consumption 15 minutes after startup was recorded as 1.84 GB. Memory consumption then stabilized and remains at 1.85 GB as of 20 minutes ago (i.e., just over two hours after the restart).

Update 2: Interestingly, memory consumption is now flatlining at 1.84 GB, four hours after the restart.

[1714221182]

1714221182

[1714221182]

1714221182

[1714221182]

1714221182

[1714221182]

1714221182

[1714221182]

1714221182

[jtoomim]

I'm not seeing higher memory consumption on 1mb_segwit, but I am seeing what appears to be excessive CPU usage and longer GBT times plus very high DOA rates (~5%). I just restarted the server with profiling enabled and will look into it further.

veqtrus used some dynamic share format code that would either create a share with witness data or without depending on the runtime value of the VERSION attribute instead of putting the segwit code in a different class, and that might be messing up pypy's JIT compilation somehow. Or it could be something else. We'll find out soon enough.

Edit: Profiling shows that the performance issue seems to be in p2p.py:update_remote_view_of_my_known_txs(). This sounds like the performance issue that I addressed a couple years ago in the jtoomim_performance branch. My guess is that veqtrus's code updated the known_txs var somewhere using assignment (which is O(n^2) versus transactions per second) instead of inserts and deletions (which are O(n)).

[Meuh6879]

Interesting, the regular 1,6 PH/s ...

[jtoomim]

has switched over to jtoomimnet.

Code:

git clone https://github.com/jtoomim/p2pool.git
cd p2pool
git checkout lowmem

Edit: Nope, we just got a short-term nicehash renter or something.

[sawa]

I've switched over my nodes to jtoomimnet.
Now both http://crypto.office-on-the.net:9334 and http://crypto.office-on-the.net:9332 work in one network

Who was mining at port 9334 can connect your asics to port 9332, if you do not want to pay me 1% fee - there's 0% fee.
It is better to switch to http://crypto.mine.nu:9334 or http://crypto.mine.nu:9332 - this is a faster channel to the same server where crypto.office-on-the.net is

[Xantus]

sorry to bugging you people, but i do not understand wath you are doing here.

next few Days would Sergwit be aktivated on Bitcoin network, around 22th Aug as i think. Do we have an problem with that with old p2pool node ?


And if i look on my P2Pool Shares of my own small P2Pool node there would be mined an Block Version 536870914. that is the same as post before from "sawa"
Mining on Jtoomim ... right ? - signaling Sergwit al the time ...

wath you talking about an 1MB sergwit fork ? - does that mean the Sergwit aktivation on 22th Aug ?

would be nice if someone could explain me these things 

[jtoomim]

We lost power at our datacenter this morning. When our p2pool servers automaticallhy restarted after we regained power, they ran the branch of p2pool I happened to have checked out on my nodes, which was 1mb_segwit. This was not intentional, as I meant to run lowmem. We mined a bunch of v33 shares, and you will start to see a message saying:

Code:

Warning: A MAJORITY OF SHARES CONTAIN A VOTE FOR AN UNSUPPORTED SHARE IMPLEMENTATION! (v33 with 71% support) An upgrade is likely necessary. Check http://p2pool.forre.st/ for more information.

if you're on jtoomimnet. Please disregard this message for now. We will be switching to 1mb_segwit eventually in order to support SegWit/SegWit2x, but not quite yet, as there are still some performance bugs at least in the 1mb_segwit branch.

[jtoomim]

Xantus:

The old p2pool code (https://github.com/p2pool/p2pool master) does not support SegWit. There are two branches of code that do support SegWit. One is veqtrus's SegWit PR, and the other is my 1mb_segwit code, which is derived from veqtrus's code.

The main difference between veqtrus's version and my version is that my version is based on the jtoomimnet hardfork, which I made a few months ago to solve a problem with p2pool being unable to create 999 kB blocks. The jtoomimnet code handles higher transaction volumes than the mainnet p2pool code, but it also contains several performance optimizations, so overall resource requirements are about the same for the two networks. The jtoomimnet code also contains some changes to the protocol that reduce orphan rates, which makes jtoomimnet more fair to large and small miners. The creation of jtoomimnet required changing consensus rules of p2pool, so it had to be a hardfork. I wrote the code in such a way to leave the original p2pool functioning normally, so only those who wanted to use my upgrades needed to switch their code. Thus, we currently have two networks.

I want to do some more testing and optimization on the 1mb_segwit branch first, but I intend to switch jtoomimnet over to 1mb_segwit before SegWit activates.

[veqtrus]

Edit: Profiling shows that the performance issue seems to be in p2p.py:update_remote_view_of_my_known_txs(). This sounds like the performance issue that I addressed a couple years ago in the jtoomim_performance branch. My guess is that veqtrus's code updated the known_txs var somewhere using assignment (which is O(n^2) versus transactions per second) instead of inserts and deletions (which are O(n)).

My code doesn't modify that variable, only reads it.

[veqtrus]

The issue is that p2pool will refuse to mine if a softfork is detected which is not in the p2pool/network/(coinname).py:SOFTFORKS-REQUIRED list. It may be best to remove that check entirely, as it seems to be explicitly anti-forwards-compatible, which sounds like it may be a bad idea. Still thinking it over.

GBT indicates which forks require changes to mining software by prepending an "!" to their names. Therefore p2pool should stop mining when such fork is detected. Currently Core doesn't treat segwit as required since a miner can mine old-style blocks so if "Fail providing work on unknown rule activation" were to be backported it would still work.

[Xantus]

okey, thanks for you explainment !

so i will change my p2pool node to jtoomim branche befor 22th aug.
But i would like to wait for our test ;-)

hope you post it here if finish. and wich is the right github adress ... (i dont like github :-p)

[jtoomim]

I think I've solved the performance issue with 1mb_segwit. It looks like the PR that veqtrus put on my github to correct what he perceived as a couple of omissions on my part for the old (2014?) jtoomim_performance code had an error. Specifically, it called VariableDict.transitioned.happened() on the known_txs variable whenever a transaction was added or deleted from known_txs variable. The purpose of the jtoomim_performance code was to only call transitioned.happened() when a full assignment was done on known_txs var, since transitioned.happened() calls subsequent code that reviews the full contents of known_txs -- O(n) versus 'mempool' size-- instead of just the added or removed items -- O(1) versus 'mempool' size. Even worse, the place that veqtrus added the call to transitioned.happened() causes this O(n) operation to happen once for every transaction that's added and once for every transaction that's removed, instead of once per batch of transactions added and removed as per the p2pool master code, so in many circumstances that PR would perform around 10x worse than the original p2pool master code.

This bug should not affect veqtrus's base segwit PR on the p2pool/p2pool github, but it likely will affect the branch veqtrus has been working on in his own github repo that merges some performance fixes (plus this performance regression masquerading as a code refactor) into his segwit branch.

I reverted that code from 1mb_segwit and pushed the changes to github, and am now running it on ml.toom.im:9332, and it seems to be performing as well as the lowmem branch. I will likely switch all of my nodes over to 1mb_segwit before the eclipse.

[Duce]

I tried to run the new 1mb_segwit version but I get  Coin daemon too old! Upgrade! I am running Bitcoin Core 14.2. Does this have something to do with -rpcserialversion which I should not have to set as the default should be 1?

The lowmem version works fine.

[jtoomim]

Duce, that's not a bug in p2pool. It's because Bitcoin Core 14.2 does not support Segwit2x, which was enacted by 80-90% of miners about 2 weeks ago.  All versions of Bitcoin Core are about to become incompatible with the majority of the Bitcoin hashrate, as they do not support Segwit2x. If you continue to use Bitcoin Core, it is expected that you will be on the minority fork of a chainsplit in 2-3 months, when the 2x part of SegWit2x is enabled. I recommend that you either switch to btc1 or lobby Bitcoin Core to change their policy regarding SegWit2x.

P2pool does not currently have any functionality to handle chainsplits gracefully. If you are aware of the SegWit2x issue and wish to continue to mine with Bitcoin Core even though doing so will support a chainsplit, let me know and I will consider adding code to p2pool to make the p2pool sharechain split in parallel to the Bitcoin chain.

[veqtrus]

Duce, that's not a bug in p2pool. It's because Bitcoin Core 14.2 does not support Segwit2x, which was enacted by 80-90% of miners about 2 weeks ago.  All versions of Bitcoin Core are about to become incompatible with the majority of the Bitcoin hashrate, as they do not support Segwit2x. If you continue to use Bitcoin Core, it is expected that you will be on the minority fork of a chainsplit in 2-3 months, when the 2x part of SegWit2x is enabled. I recommend that you either switch to btc1 or lobby Bitcoin Core to change their policy regarding SegWit2x.

P2pool does not currently have any functionality to handle chainsplits gracefully. If you are aware of the SegWit2x issue and wish to continue to mine with Bitcoin Core even though doing so will support a chainsplit, let me know and I will consider adding code to p2pool to make the p2pool sharechain split in parallel to the Bitcoin chain.

Bitcoin Core is also not compatible with Bitcoin Cash and all the other altcoins out there. "2x" will be an altcoin and therefore is irrelevant to the Bitcoin P2Pool network. Then again you are a BU supporter which is on par with your P2Pool fork attempt.

[jtoomim]

veqtrus, what I personally prefer or what you personally prefer is irrelevant. SegWit2x has 92.76% support among miners at the moment, well above the needed threshold of 80% for activation. The only reason SegWit is being enabled is because it is part of SegWit2x. P2pool needs to support the majority blockchain.

Supporting minority blockchains like Bitcoin Core or Bitcoin Cash is desirable if it can be implemented in a way that does not conflict with support for the majority chain. Until that code is written, p2pool should consider minority chains to be unsupported.

You don't like 2x. I don't like SegWit. It doesn't matter. The majority has spoken.

https://coin.dance/blocks/proposals

If you wish to maintain your own no-2x fork of p2pool with a separate share chain, that would be appropriate.

[Xantus]

 
Hey People, i downloaded from Github the jtoomim_preformance branche by chosing "Download zip" to test it.
But, i am a Windows user, unable to compile it...

is there an easy way ? - or should i easy wait until our tests are finished ?

[jtoomim]

jtoomim_performance is an obsolete branch from 3 years ago and is not supported.

To run p2pool from source, you need to install python 2.7, twisted, zope.interface, and a couple other things. There are some instructions on the github page for what you need to install:

https://github.com/jtoomim/p2pool/tree/1mb_segwit

Code:

Windows:

Install Python 2.7: http://www.python.org/getit/
Install Twisted: http://twistedmatrix.com/trac/wiki/Downloads
Install Zope.Interface: http://pypi.python.org/pypi/zope.interface/3.8.0
Install python win32 api: http://sourceforge.net/projects/pywin32/files/pywin32/Build%20218/
Install python win32 api wmi wrapper: https://pypi.python.org/pypi/WMI/#downloads
Unzip the files into C:\Python27\Lib\site-packages

I've never actually run it from source on Windows, so beyond those instructions I can't help you. Maybe someone else has done it?

[Xantus]

hey, it seams to run using phyton to exicute ...
so i downloading the share ..
well, http://131.117.146.223:9332 -> could that be right ?  Version 1 ?

seams it dont work  
now it lost al of the 6 connections, and wath does that mean ? :

Code:

Share check failed: 000000000000005a15a2a9eef3379f7c1e0c063774fff9f728b38dd9e7efeb17 -> 00000000000000609b18d98d223dffbb34b3a176f9969d0ff847f8ce0ba7fb40

Code:

2017-08-14 22:47:20.920000 P2Pool: 0 shares in chain (0 verified/16590 total) Peers: 0 (0 incoming)
2017-08-14 22:47:20.920000  Local: 0H/s in last 0.0 seconds Local dead on arrival: ??? Expected time to share: ???

Code:

Unhandled Error
Traceback (most recent call last):
  File "C:\Python27\p2pool-1mb_segwit\p2pool-1mb_segwit\p2pool\main.py", line 685, in run
    reactor.run()
  File "C:\Python27\lib\site-packages\twisted\internet\base.py", line 1243, in run
    self.mainLoop()
  File "C:\Python27\lib\site-packages\twisted\internet\base.py", line 1252, in mainLoop
    self.runUntilCurrent()
  File "C:\Python27\lib\site-packages\twisted\internet\base.py", line 878, in runUntilCurrent
    call.func(*call.args, **call.kw)
--- <exception caught here> ---
  File "C:\Python27\p2pool-1mb_segwit\p2pool-1mb_segwit\p2pool\bitcoin\stratum.py", line 38, in _send_work
    x, got_response = self.wb.get_work(*self.wb.preprocess_request('' if self.username is None else self.username))
  File "C:\Python27\p2pool-1mb_segwit\p2pool-1mb_segwit\p2pool\work.py", line 212, in preprocess_request
    raise jsonrpc.Error_for_code(-12345)(u'p2pool is not connected to any peers')
p2pool.util.jsonrpc.NarrowError: -12345 p2pool is not connected to any peers

Share check failed: 00000000000000609b18d98d223dffbb34b3a176f9969d0ff847f8ce0ba7fb40 -> 0000000000000122d9e5182a7c37cb23400ae2995c13d62497671cd017b19394
Traceback (most recent call last):
  File "C:\Python27\p2pool-1mb_segwit\p2pool-1mb_segwit\p2pool\util\deferral.py", line 233, in it
    res = gen.send(cur) # external code is run here
  File "C:\Python27\p2pool-1mb_segwit\p2pool-1mb_segwit\p2pool\util\deferral.py", line 284, in _worker
    self.func(*self.args, **self.kwargs)
  File "C:\Python27\p2pool-1mb_segwit\p2pool-1mb_segwit\p2pool\node.py", line 309, in clean_tracker
    best, desired, decorated_heads, bad_peer_addresses = self.tracker.think(self.get_height_rel_highest, self.bitcoind_work.value['previous_block'], self.bitcoind_work.value['bits'], self.known_txs_var.value)
  File "C:\Python27\p2pool-1mb_segwit\p2pool-1mb_segwit\p2pool\data.py", line 554, in think
    if self.attempt_verify(share):
--- <exception caught here> ---
  File "C:\Python27\p2pool-1mb_segwit\p2pool-1mb_segwit\p2pool\data.py", line 532, in attempt_verify
    share.check(self)
  File "C:\Python27\p2pool-1mb_segwit\p2pool-1mb_segwit\p2pool\data.py", line 377, in check
    if self.timestamp > int(time.mktime(time.gmtime()) - time.mktime(time.gmtime(0))) + 300:
exceptions.OverflowError: mktime argument out of range

so i go to sleep  , gn8 2all

[veqtrus]

veqtrus, what I personally prefer or what you personally prefer is irrelevant. SegWit2x has 92.76% support among miners at the moment, well above the needed threshold of 80% for activation. The only reason SegWit is being enabled is because it is part of SegWit2x. P2pool needs to support the majority blockchain.

Supporting minority blockchains like Bitcoin Core or Bitcoin Cash is desirable if it can be implemented in a way that does not conflict with support for the majority chain. Until that code is written, p2pool should consider minority chains to be unsupported.

You don't like 2x. I don't like SegWit. It doesn't matter. The majority has spoken.

https://coin.dance/blocks/proposals

If you wish to maintain your own no-2x fork of p2pool with a separate share chain, that would be appropriate.

I have no problem with your fork other than it is misleading. Bitcoin isn't defined by miners but by the economic majority which hasn't reached consensus on 2x. Therefore labeling your fork as "Bitcoin P2Pool" is misleading.

Also where is this majority support among miners you speak of?

Block versions on main P2Pool