[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[frodocooper]

As a new miner in this pool how do I know which fork I am in?

If you cloned P2Pool from https://github.com/p2pool/p2pool, then you are on the main P2Pool branch.

If you cloned P2Pool from https://github.com/jtoomim/p2pool/tree/1mb_hardforked, https://github.com/jtoomim/p2pool/tree/lowmem, or https://github.com/jtoomim/p2pool/tree/1mb_segwit, then you are on jtoomim's fork.

Also, the protocol version number on the main P2Pool branch is 16, while the protocol version number on jtoomim's fork is 32. You may find this by clicking on any share under the "My Shares" section of your P2Pool node's web frontend.

What is this all about?

2017-08-02 16:03:15.482000 Punishing share for 'Block-stale detected! height(5435887388abccffb3a282c994c986d6a893cd8a0f94bc) < height(13b061b122ac7c84e3fb81ca6c62554ae4e2aea3b26e30e)

Please refer to https://bitcointalk.org/index.php?topic=18313.msg18580559#msg18580559.

[1714053831]

1714053831

[1714053831]

1714053831

[1714053831]

1714053831

[jtoomim]

Sorry this took so long. I haven't fully tested this to make sure it works on testnet and can make segwit blocks, but if people can help review it and maybe help test it, that would be nice.

https://github.com/jtoomim/p2pool/commits/1mb_segwit

Segwit compatibility stuff starts with the July 19th commits.

For the UAHF, I have an idea for how to deal with it. Each time we receive a share, we can query our bitcoind to see whether that share's previousblockhash refers to a valid block hash in our bitcoind's blockchain (via the getblockheader RPC). If it is, we accept the share and memoize that hash. If it isn't, we ignore that share. This should allow people to mine on either blockchain without needing to worry about people getting valid shares while mining on the other blockchain.

Unfortunately, I'm very busy right now with datacenter stuff, so I don't know when I'll have a chance to implement this.

[windpath]

Sorry this took so long. I haven't fully tested this to make sure it works on testnet and can make segwit blocks, but if people can help review it and maybe help test it, that would be nice.

https://github.com/jtoomim/p2pool/commits/1mb_segwit

In order to expedite this as much as possible, p2pool.org is now running this code on Mainnet, cuurently downloading the share chain.

[veqtrus]

Sorry this took so long. I haven't fully tested this to make sure it works on testnet and can make segwit blocks, but if people can help review it and maybe help test it, that would be nice.

https://github.com/jtoomim/p2pool/commits/1mb_segwit

In order to expedite this as much as possible, p2pool.org is now running this code on Mainnet, cuurently downloading the share chain.

I posted the segwit PR in November last year. Jtoomim has now included it without modifications (except for the version number change) in his 1MB fork. It has been running on vertcoin for months with great success.

Why are you testing jtoomim's fork now when the exact same code has been available since last year?

[jtoomim]

Block on jtoomimnet (Bitcoin 1MB chain): https://blockchain.info/block-index/1611649

Mempool wasn't full, so only 504 kB. Previous and subsequent blocks from other pools were also not full.

[Duce]

Block on jtoomimnet (Bitcoin 1MB chain): https://blockchain.info/block-index/1611649

Mempool wasn't full, so only 504 kB. Previous and subsequent blocks from other pools were also not full.

On the fork with approximately 1.4PH? This is the same that Windpath is now running correct? I had just changed my node this morning but I don't see it on either.

[jtoomim]

For some reason, it's not listed on the node dashboards in the "Blocks found in the last day" section, but it's a valid block. If you plug in the block/share's hash into share.html, you can see it. Example:

http://ml.toom.im:9332/static/share.html#00000000000000000068e942f0cc1a00684c039c2c3a729bed92729e2e72f14c

[Blue Bear]

did the pool go down?

[Blue Bear]

Did something happen ... cause I seem to have lost all my shares ...

[frodocooper]

Did something happen ... cause I seem to have lost all my shares ...

P2Pool.org is now running jtoomim's fork of P2Pool. The stats page on P2Pool.org is therefore now tracking jtoomimnet's stats, not mainnet's stats.

If you're mining on mainnet, your miner stats will no longer appear on P2Pool.org's stats page as long as windpath continues running jtoomim's fork. This does not mean your shares on mainnet have been lost, just that P2Pool.org is no longer keeping track of mainnet and its miners' stats. If you're looking for your miners' mainnet stats, you would have to look at the web frontend of another mainnet P2Pool node.

If your miner is pointed at P2Pool.org's node, you would now be mining on jtoomimnet. It would take some time for your miner to ramp up to its expected amount of shares on jtoomimnet, while your mainnet shares would start to fall off the mainnet sharechain. This would, in theory and in general, have limited impact on your expected earnings during your miners' transition from mainnet to jtoomimnet.

(I'm assuming that you are not running and mining at your own P2Pool node, and that you were relying on P2Pool.org's stats page for your miners' stats.)

Sorry this took so long. I haven't fully tested this to make sure it works on testnet and can make segwit blocks, but if people can help review it and maybe help test it, that would be nice.

https://github.com/jtoomim/p2pool/commits/1mb_segwit

In order to expedite this as much as possible, p2pool.org is now running this code on Mainnet, cuurently downloading the share chain.

[Blue Bear]

Actually I am mining my own node. It appears to have been knocked off when the switch happened and took me a minute to get it back on line.
This leads to the question: If I am running my own node, with 6 outbound connections and 0 inbound connections, am I still connected and part of the pool or am I on my own now?

[windpath]

Actually I am mining my own node. It appears to have been knocked off when the switch happened and took me a minute to get it back on line.
This leads to the question: If I am running my own node, with 6 outbound connections and 0 inbound connections, am I still connected and part of the pool or am I on my own now?

Yes, it can take a long time to get inbound connections.

[jtoomim]

The p2pool code will refuse to mine if your node is not connected to any peers.

[windpath]

The p2pool code will refuse to mine if your node is not connected to any peers.

"6 outbound connections"

[sawa]

jtoomim, hello. I'd like to try your https://github.com/jtoomim/p2pool/tree/lowmem branch for working with altcoins. But there's a problem - workers cannot connect, there's an empty sharechain. Can you please look what to do with it? I've already wrote about it here:
https://bitcointalk.org/index.php?topic=18313.msg19772037#msg19772037
https://bitcointalk.org/index.php?topic=18313.msg19786200#msg19786200

[jtoomim]

sawa, I added some changes in the 1mb_segwit branch that I hope will address the altcoin issues. Can you try with that? The 1mb_segwit branch's commits are a superset of the lowmem branch.

[sawa]

sawa, I added some changes in the 1mb_segwit branch that I hope will address the altcoin issues. Can you try with that? The 1mb_segwit branch's commits are a superset of the lowmem branch.

This does not work.

Code:

git clone https://github.com/jtoomim/p2pool p2pool-emd
cd p2pool-emd
git checkout 1mb_hardforked

Run and get an error

Code:

Unhandled Error
Traceback (most recent call last):
  File "/opt/scrypt/p2pool-emd/p2pool/main.py", line 669, in run
    reactor.run()
  File "/usr/local/lib/pypy2.7/dist-packages/twisted/internet/base.py", line 1243, in run
    self.mainLoop()
  File "/usr/local/lib/pypy2.7/dist-packages/twisted/internet/base.py", line 1252, in mainLoop
    self.runUntilCurrent()
  File "/usr/local/lib/pypy2.7/dist-packages/twisted/internet/base.py", line 878, in runUntilCurrent
    call.func(*call.args, **call.kw)
--- <exception caught here> ---
  File "/opt/scrypt/p2pool-emd/p2pool/bitcoin/stratum.py", line 38, in _send_work
    x, got_response = self.wb.get_work(*self.wb.preprocess_request('' if self.username is None else self.username))
  File "/opt/scrypt/p2pool-emd/p2pool/bitcoin/worker_interface.py", line 129, in get_work
    x, handler = self._inner.get_work(*args)
  File "/opt/scrypt/p2pool-emd/p2pool/work.py", line 334, in get_work
    base_subsidy=self.node.net.PARENT.SUBSIDY_FUNC(self.current_work.value['height']),
  File "/opt/scrypt/p2pool-emd/p2pool/data.py", line 202, in generate_transaction
    if desired_timestamp > previous_share.timestamp + 180:
exceptions.AttributeError: 'NoneType' object has no attribute 'timestamp'

This will not work even with bitcoin. I tried to make a solo pool.
 If I clear the sharechain, change the values of IDENTIFIER, PREFIX, make empty BOOTSTRAP_ADDRS
And change PERSIST = True in PERSIST = False
In this case, the worker can not connect to the pool

[jtoomim]

You're using the wrong branch. The 1mb_segwit branch has the altcoin and empty sharechain fixes. The 1mb_hardforked branch does not.

Note: 1mb_segwit will work on the same sharechain as 1mb_hardforked and 1mb_lowmem. 1mb_segwit will vote for v33 shares, but until v33 happens it should be compatible with the other jtoomimnet branches. The v33 upgrade will be the same as the hardforks that forrestv has done in the past for upgrading p2pool to be compatible with bitcoin softforks (e.g. OP_CLTV).

[frodocooper]

Actually I am mining my own node. It appears to have been knocked off when the switch happened and took me a minute to get it back on line.
This leads to the question: If I am running my own node, with 6 outbound connections and 0 inbound connections, am I still connected and part of the pool or am I on my own now?

As long as your P2Pool node has at least one connection to another node, inbound or outbound, you are connected to the P2Pool network and your node remains a part of the pool.

And as long as the P2Pool network is online and active (i.e., there are other active P2Pool nodes on the network), your accepted shares are safe even though your own node has lost its copy of the sharechain. The other active P2Pool nodes have your accepted shares in their copies of the sharechain. That is, unless through some catastrophic failure the entire P2Pool network loses your accepted shares or loses all copies of the sharechain, which is next to impossible under normal circumstances.

[sawa]

You're using the wrong branch. The 1mb_segwit branch has the altcoin and empty sharechain fixes. The 1mb_hardforked branch does not.

Note: 1mb_segwit will work on the same sharechain as 1mb_hardforked and 1mb_lowmem. 1mb_segwit will vote for v33 shares, but until v33 happens it should be compatible with the other jtoomimnet branches. The v33 upgrade will be the same as the hardforks that forrestv has done in the past for upgrading p2pool to be compatible with bitcoin softforks (e.g. OP_CLTV).

I tested 1mb_segwit for 'ltc_scrypt' POW_FUNC

Code:

/usr/bin/pypy -O -E /opt/scrypt/p2pool-emd/run_p2pool.py --net emerald --address Ewwbnzi6sxwLL6XbtDoV2Sj9C8JUPQ1sks --give-author 0 --no-bugreport -w 12125 -f 1 --bitcoind-address 127.0.0.1 --bitcoind-p2p-port 12127 --bitcoind-rpc-port 12128 user pass
2017-08-06 13:45:55.953718 p2pool (version 15.0-56-gf79f584-dirty)
2017-08-06 13:45:55.953841 
2017-08-06 13:45:55.953936 Testing bitcoind RPC connection to 'http://127.0.0.1:12128/' with username 'user'...
2017-08-06 13:45:55.971389     ...success!
2017-08-06 13:45:55.971572     Current block hash: 526bdb3d4f26a5e89f9c32276c34b076c97eaf871195ffa5e8ad707aae016c23
2017-08-06 13:45:55.971686     Current block height: 5693321
2017-08-06 13:45:55.971820 
2017-08-06 13:45:55.971940 Testing bitcoind P2P connection to '127.0.0.1:12127'...
2017-08-06 13:45:55.993385     ...success!
2017-08-06 13:45:55.993583 
2017-08-06 13:45:55.993736 Determining payout address...
2017-08-06 13:45:55.994010     ...success! Payout address: Ewwbnzi6sxwLL6XbtDoV2Sj9C8JUPQ1sks
2017-08-06 13:45:55.994112 
2017-08-06 13:45:55.994236 Loading shares...
2017-08-06 13:45:57.582201     1000
2017-08-06 13:45:58.598017     2000
2017-08-06 13:45:59.447829     3000
2017-08-06 13:46:00.314925     4000
2017-08-06 13:46:01.230490     5000
2017-08-06 13:46:01.987326     6000
2017-08-06 13:46:02.760611     7000
2017-08-06 13:46:03.688368     8000
2017-08-06 13:46:04.437252     9000
2017-08-06 13:46:05.186910     10000
2017-08-06 13:46:06.182604     11000
2017-08-06 13:46:06.913339     12000
2017-08-06 13:46:07.649518     13000
2017-08-06 13:46:08.382123     14000
2017-08-06 13:46:09.446092     15000
2017-08-06 13:46:10.166873     16000
2017-08-06 13:46:10.904880     17000
2017-08-06 13:46:11.644394     18000
2017-08-06 13:46:12.380946     19000
2017-08-06 13:46:13.129931     20000
2017-08-06 13:46:14.362359     21000
2017-08-06 13:46:15.105757     22000
2017-08-06 13:46:15.861376     23000
2017-08-06 13:46:16.599423     24000
2017-08-06 13:46:17.352762     25000
2017-08-06 13:46:17.877206 Share loading took 21.883 seconds
2017-08-06 13:46:17.877367     ...done loading 25700 shares (25701 verified)!
2017-08-06 13:46:17.877429 
2017-08-06 13:46:17.877493 Initializing work...
Error while processing Event callbacks:
Traceback (most recent call last):
  File "/opt/scrypt/p2pool-emd/p2pool/util/deferral.py", line 284, in _worker
    self.func(*self.args, **self.kwargs)
  File "/opt/scrypt/p2pool-emd/p2pool/node.py", line 330, in clean_tracker
    self.tracker.verified.remove(share_hash)
  File "/opt/scrypt/p2pool-emd/p2pool/util/forest.py", line 361, in remove
    Tracker.remove(self, item_hash)
  File "/opt/scrypt/p2pool-emd/p2pool/util/forest.py", line 330, in remove
    self.removed.happened(item)
--- <exception caught here> ---
  File "/opt/scrypt/p2pool-emd/p2pool/util/variable.py", line 42, in happened
    func(*event)
  File "/opt/scrypt/p2pool-emd/p2pool/util/variable.py", line 19, in <lambda>
    watch_id = self.watch(lambda *args: func(obj_ref(), *args))
  File "/opt/scrypt/p2pool-emd/p2pool/util/forest.py", line 15, in <lambda>
    self.tracker.removed.watch_weakref(self, lambda self, item: self.forget_item(item.hash))
exceptions.AttributeError: 'NoneType' object has no attribute 'forget_item'

=== Cut the same errors ===

2017-08-06 13:46:19.305269     ...success!
2017-08-06 13:46:19.305451 
2017-08-06 13:46:19.305524 Joining p2pool network using port 46858...
2017-08-06 13:46:19.320109     ...success!
2017-08-06 13:46:19.320245 
2017-08-06 13:46:19.324959 Listening for workers on '' port 12125...
2017-08-06 13:46:20.732464     ...success!
2017-08-06 13:46:20.732607 
2017-08-06 13:46:20.732715 Started successfully!
2017-08-06 13:46:20.732836 Go to http://127.0.0.1:12125/ to view graphs and statistics!
2017-08-06 13:46:20.732949 Donating 0.0% of work towards P2Pool's development. Please donate to encourage further development of P2Pool!
2017-08-06 13:46:20.733023 
2017-08-06 13:46:20.828401 Peer 127.0.0.1:42234 says protocol version is 3300, client version 15.0-56-gf79f584-dirty
2017-08-06 13:46:20.828615 Peer 127.0.0.1:42234 misbehaving, will drop and ban. Reason: was connected to self
2017-08-06 13:46:20.828706 Bad peer banned: ('127.0.0.1', 42234)
2017-08-06 13:46:20.829542 Peer 127.0.0.1:46858 says protocol version is 3300, client version 15.0-56-gf79f584-dirty
2017-08-06 13:46:20.829665 Peer 0.0.0.0:46858 misbehaving, will drop and ban. Reason: was connected to self
2017-08-06 13:46:20.829779 Bad peer banned: (u'0.0.0.0', 46858)
2017-08-06 13:46:24.550886 Warning: Previous share's timestamp is 1019207 seconds old.
2017-08-06 13:46:24.551057 Make sure your system clock is accurate, and ensure that you're connected to decent peers.
2017-08-06 13:46:24.551142 If your clock is more than 300 seconds behind, it can result in orphaned shares.
2017-08-06 13:46:24.551223 (It's also possible that this share is just taking a long time to mine.)

=== Cut the same errors ===

2017-08-06 13:46:32.377668 Worker EpKjxiEMuxEQje4hB9f7tkZzxG2QP4TC2F submitted share with hash > target:
2017-08-06 13:46:32.377942     Hash:   b2648e10e08feb7ac8712d83e8a6b5cf7a76728de6c638e19bf723bb3923f9f
2017-08-06 13:46:32.378118     Target: 145b892ce5d6c97c00000000000000000000000000000000000000000000
=== Cut the same errors ===
2017-08-06 13:46:38.416586 Worker EpKjxiEMuxEQje4hB9f7tkZzxG2QP4TC2F submitted share with hash > target:
2017-08-06 13:46:38.417083     Hash:   1342a5a68d0bd4e707df41cc726d718cb5d81e0f16f096da22bde4befb197ead
2017-08-06 13:46:38.417122     Target: 145b892ce5d6c97c00000000000000000000000000000000000000000000
Error in DeferredResource handler:
Traceback (most recent call last):
  File "/opt/scrypt/p2pool-emd/p2pool/util/deferred_resource.py", line 24, in render
    defer.maybeDeferred(resource.Resource.render, self, request).addCallbacks(finish, finish_error)
  File "/usr/local/lib/pypy2.7/dist-packages/twisted/internet/defer.py", line 150, in maybeDeferred
    result = f(*args, **kw)
  File "/usr/local/lib/pypy2.7/dist-packages/twisted/web/resource.py", line 250, in render
    return m(request)
  File "/usr/local/lib/pypy2.7/dist-packages/twisted/internet/defer.py", line 1532, in unwindGenerator
    return _inlineCallbacks(None, gen, Deferred())
--- <exception caught here> ---
  File "/usr/local/lib/pypy2.7/dist-packages/twisted/internet/defer.py", line 1386, in _inlineCallbacks
    result = g.send(result)
  File "/opt/scrypt/p2pool-emd/p2pool/web.py", line 198, in render_GET
    res = yield self.func(*self.args)
  File "/opt/scrypt/p2pool-emd/p2pool/web.py", line 179, in get_local_stats
    warnings=p2pool_data.get_warnings(node.tracker, node.best_share_var.value, node.net, bitcoind_getinfo_var.value, node.bitcoind_work.value),
  File "/opt/scrypt/p2pool-emd/p2pool/data.py", line 723, in get_warnings
    if bitcoind_getinfo['errors'] != '':
exceptions.KeyError: 'errors'

2017-08-06 13:46:38.671323 Worker EwSjj3pdzF8qji9uqgqfcu5EjDorjf2h4s submitted share with hash > target:
2017-08-06 13:46:38.671412     Hash:   79d1a99c6c11888c6ca8ad834a4f3616959b34b14ab7385ed5e3d43e6593840
2017-08-06 13:46:38.671440     Target: 145b892ce5d6c97c00000000000000000000000000000000000000000000
=== Cut the same errors ===
2017-08-06 13:46:38.882973 Worker EpKjxiEMuxEQje4hB9f7tkZzxG2QP4TC2F submitted share with hash > target:
2017-08-06 13:46:38.883040     Hash:   13d915870608c1281b6fa897a865951abe9746d9452bac2a1a074dba3ac6be57
2017-08-06 13:46:38.883059     Target: 145b892ce5d6c97c00000000000000000000000000000000000000000000
=== Cut the same errors ===

Error in DeferredResource handler:
Traceback (most recent call last):
  File "/opt/scrypt/p2pool-emd/p2pool/util/deferred_resource.py", line 24, in render
    defer.maybeDeferred(resource.Resource.render, self, request).addCallbacks(finish, finish_error)
  File "/usr/local/lib/pypy2.7/dist-packages/twisted/internet/defer.py", line 150, in maybeDeferred
    result = f(*args, **kw)
  File "/usr/local/lib/pypy2.7/dist-packages/twisted/web/resource.py", line 250, in render
    return m(request)
  File "/usr/local/lib/pypy2.7/dist-packages/twisted/internet/defer.py", line 1532, in unwindGenerator
    return _inlineCallbacks(None, gen, Deferred())
--- <exception caught here> ---
  File "/usr/local/lib/pypy2.7/dist-packages/twisted/internet/defer.py", line 1386, in _inlineCallbacks
    result = g.send(result)
  File "/opt/scrypt/p2pool-emd/p2pool/web.py", line 198, in render_GET
    res = yield self.func(*self.args)
  File "/opt/scrypt/p2pool-emd/p2pool/web.py", line 179, in get_local_stats
    warnings=p2pool_data.get_warnings(node.tracker, node.best_share_var.value, node.net, bitcoind_getinfo_var.value, node.bitcoind_work.value),
  File "/opt/scrypt/p2pool-emd/p2pool/data.py", line 723, in get_warnings
    if bitcoind_getinfo['errors'] != '':
exceptions.KeyError: 'errors'

2017-08-06 13:46:39.792308 Worker EpKjxiEMuxEQje4hB9f7tkZzxG2QP4TC2F submitted share with hash > target:
2017-08-06 13:46:39.792373     Hash:   110ccbd08d89ec03fa0722e857610912b77fe1feb593c3d29651a34f2589bafd
2017-08-06 13:46:39.792401     Target: 145b892ce5d6c97c00000000000000000000000000000000000000000000
=== Cut the same errors ===
2017-08-06 13:46:41.182417 Worker EqRdrDBLpKCr2JZUZ4roTwNXT8QJQFiAFe submitted share with hash > target:
2017-08-06 13:46:41.182504     Hash:   6f4d4a028b2b11573a7fe92c37a45736c99765c43fce7de9081ebb99659fdf
2017-08-06 13:46:41.182532     Target: 145b892ce5d6c97c00000000000000000000000000000000000000000000

... and so on.
All workers submitted share with hash