[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[clown]

It has been attempted before.  I forget what it was called ... someone put up a server to do just that, complete with merged mining.  It was fairly popular until it was hacked.  I think you search the forums you can find it, I think the source might be available as well.

M

You'll have to enlighten me, I've searched long and hard before I made this post and didn't see anything. Care to drop a name?

[mdude77]

It has been attempted before.  I forget what it was called ... someone put up a server to do just that, complete with merged mining.  It was fairly popular until it was hacked.  I think you search the forums you can find it, I think the source might be available as well.

M

You'll have to enlighten me, I've searched long and hard and didn't see anything. Care to drop a name?

I just remembered the name.  Was p2pmining. 

https://bitcointalk.org/index.php?topic=66202.0

There wasn't any risk to the pool operator, to my knowledge.  The basic principle was the subpool had it's own share chain (or equiv - could manage with a properly designed database) to track the p2pmining users.  Each share was also submitted to the real chain under the p2pmining address.  When a payout occurred, it was divided among the users in the subpool based on their piece of the subchain.  This also allowed merged mining to occur at a sub pool level, instead of basically solo, which is what p2pool is now.

The downside, of course, was it was a mini centralized pool, subject to possible pool operator abuse and hacking.  Hacking is what brought it down..

M

[kjj]

I was under the impression that pseudo shares are not included in the share chain, only shares with diff above the p2pool diff are included.

Another question. Let's pretend for a moment the p2pool share difficulty is at 10. A miner submits a share with difficulty 20 and the share goes into the share chain. During payout, is the value of the share capped to the share difficulty at the time of submission (10)?

I'm trying to build a proxy pool that'd would be able to payout to miners on a pseudo share basis (much less variance for miners at higher risk to pool operator). And perhaps have a vardiff system based on the pseudo shares. Has anything like this been attempted before?

Miners can intentionally pick a higher difficulty for themselves.  These higher difficulty shares give a higher payout and free up space in the share window.

This was discussed to death a few pages back.

[bkminer]

I think that with your 5mbit upload the most you can handle is 10-15 bitcoind connections, more if you choose them carefully. Those random connections eats your upload bandwith - because theres allways someone downloading blockchain..

Thanks, nreal, I've adjusted the bitcoind connections down.  Also I'm on Time Warner Cable and they have a habit of shaping large outbound transfers, or any P2P traffic they see.  I'm trying to get my landlord to get FiOS but he doesn't seem interested.

[jedimstr]

I was under the impression that pseudo shares are not included in the share chain, only shares with diff above the p2pool diff are included.

Another question. Let's pretend for a moment the p2pool share difficulty is at 10. A miner submits a share with difficulty 20 and the share goes into the share chain. During payout, is the value of the share capped to the share difficulty at the time of submission (10)?

I'm trying to build a proxy pool that'd would be able to payout to miners on a pseudo share basis (much less variance for miners at higher risk to pool operator). And perhaps have a vardiff system based on the pseudo shares. Has anything like this been attempted before?

Miners can intentionally pick a higher difficulty for themselves.  These higher difficulty shares give a higher payout and free up space in the share window.

This was discussed to death a few pages back.

It would be best if the higher hashrate miners were to all set their min difficulties higher and thereby helping the rest of us, but having it as a voluntary action doesn't seem to be having much effect.

Anyway this could be enforced as part of VARDIFF for minimum diff shares for the big guns? Or if this is done already, can they be more aggressively applied?

[lenny_]

May be its time to do sudo apt-get update and sudo apt-get upgrade for your Ubuntu?

I have newest Xubuntu version, man. Fully updated.

Have you restarted your Ubuntu to see if bitcoind and the P2Pool server startup again?

I just restarted p2pool. It's running fine so far on http://lenny.dnsd.me:9332/static/
Will see how it goes.

[ikolubr]

May be its time to do sudo apt-get update and sudo apt-get upgrade for your Ubuntu?

I have newest Xubuntu version, man. Fully updated.

Have you restarted your Ubuntu to see if bitcoind and the P2Pool server startup again?

I just restarted p2pool. It's running fine so far on http://lenny.dnsd.me:9332/static/
Will see how it goes.

It is exactly what is happening with my node: bitcoin.breeba.com:3330
I deleted the data folder, restarted the server, after a while the error comes back and everything stales for few minutes. I don't know what to do.

Igor

[gyverlb]

[...]
Yes, I understand that.  I still want to see some numbers! You keep claiming you do better in p2pool than a conventional pool.  Let's see numbers to prove it!  We know theory and reality usually don't meet.

I don't compute these numbers anymore (at least not accurately) because:
- I read and understood the p2pool code,
- I validated them ages ago and have no reason to believe that p2pool behavior changed since (both because our income is still high enough and the code changes didn't introduce changes in the way the distribution works, stales are used, efficiency is computed, ... that could affect the basic principles),
- they are only one part of our above 100% PPS efficiency now (which makes isolating the p2pool's income and corresponding hashrate difficult, we simply use a 30-day sliding window and check the global income vs a theoretical 100+% PPS which is our target).

Reporting these numbers if they were available wouldn't even be of much interest: this would just say that our numbers match the expected p2pool behavior on our setup and you wouldn't even be able to verify that I didn't misrepresent them.
Either you trust my understanding of the system and my earlier experiment results and don't need recent numbers or you wouldn't have any reason to trust numbers I would provide (there's the possibility that I would fabricate good looking numbers hoping to lure you into a pool where your hashrate would work for me).
In my opinion you shouldn't trust anybody here (people luring others on "magical" pools looked like a sport on bitcointalk not long ago): if you can't use public data, your own understanding and/or your own experiments there's really no point in discussing this.

[mdude77]

[...]
Yes, I understand that.  I still want to see some numbers! You keep claiming you do better in p2pool than a conventional pool.  Let's see numbers to prove it!  We know theory and reality usually don't meet.

I don't compute these numbers anymore (at least not accurately) because:
- I read and understood the p2pool code,
- I validated them ages ago and have no reason to believe that p2pool behavior changed since (both because our income is still high enough and the code changes didn't introduce changes in the way the distribution works, stales are used, efficiency is computed, ... that could affect the basic principles),
- they are only one part of our above 100% PPS efficiency now (which makes isolating the p2pool's income and corresponding hashrate difficult, we simply use a 30-day sliding window and check the global income vs a theoretical 100+% PPS which is our target).

Reporting these numbers if they were available wouldn't even be of much interest: this would just say that our numbers match the expected p2pool behavior on our setup and you wouldn't even be able to verify that I didn't misrepresent them.
Either you trust my understanding of the system and my earlier experiment results and don't need recent numbers or you wouldn't have any reason to trust numbers I would provide (there's the possibility that I would fabricate good looking numbers hoping to lure you into a pool where your hashrate would work for me).
In my opinion you shouldn't trust anybody here (people luring others on "magical" pools looked like a sport on bitcointalk not long ago): if you can't use public data, your own understanding and/or your own experiments there's really no point in discussing this.

You're one of the few people who's number I'd assume is right.

But oh well, since you won't back it up, I see no reason to believe your claim that you come out ahead with p2pool would be accurate.

M

[bkminer]

It is exactly what is happening with my node: bitcoin.breeba.com:3330
I deleted the data folder, restarted the server, after a while the error comes back and everything stales for few minutes. I don't know what to do.

Igor

After I lowered my bitcoind connections to 20 when this error happens my GBT latency doesn't spike anymore but my hash rate drops to zero on the pool until it's over.  It's happening over and over now and I've checked and I'm on the latest code.

(also) My network rate never goes about about 60kBs on the graph and averages 12kBs)

-Rick

[smoothrunnings]

May be its time to do sudo apt-get update and sudo apt-get upgrade for your Ubuntu?

I have newest Xubuntu version, man. Fully updated.

Have you restarted your Ubuntu to see if bitcoind and the P2Pool server startup again?

I just restarted p2pool. It's running fine so far on http://lenny.dnsd.me:9332/static/
Will see how it goes.

You know it's funny what you experienced just happened to me 2 minutes ago. I am using openSUSE, I restarted my pool and everything is normal again, it looks like there might be a bug in P2Pool. I wish there was a system that could stop the server and restart it when it errors out like this.

My status if anyone cares can be found at http://p2pool.smoothrunnings.ca:9332/

[TierNolan]

What does this mean?

Punishing share for 'Block-stale detected! height(1118c0efab8b0d8005a2995d3bf2c94d3823c91859cd30799) < height(2bee57eb6cf6965b98b6f34b8118b8ad863c2dc039618f41) or 19026666 != 19026666'! Jumping from 0b46a971 to 974819b9!

Is to cover "late" shares?

[K1773R]

What does this mean?

Punishing share for 'Block-stale detected! height(1118c0efab8b0d8005a2995d3bf2c94d3823c91859cd30799) < height(2bee57eb6cf6965b98b6f34b8118b8ad863c2dc039618f41) or 19026666 != 19026666'! Jumping from 0b46a971 to 974819b9!

Is to cover "late" shares?

correct

[TierNolan]

I am getting this spam.  File corruption or something?

Code:

2014-01-19 19:22:51.214727 >   File "/home/miner1/p2pool/p2pool/util/memoize.py", line 28, in b
2014-01-19 19:22:51.214885 >     res = f(*args)
2014-01-19 19:22:51.215046 >   File "/home/miner1/p2pool/p2pool/util/skiplist.py", line 21, in __call__
2014-01-19 19:22:51.215200 >     self.skips[pos] = math.geometric(self.p), [(self.previous(pos), self.get_delta(pos))]
2014-01-19 19:22:51.215357 >   File "/home/miner1/p2pool/p2pool/util/forest.py", line 18, in previous
2014-01-19 19:22:51.215510 >     return self.tracker._delta_type.from_element(self.tracker.items[element]).tail
2014-01-19 19:22:51.215695 > exceptions.KeyError: None
2014-01-19 19:22:51.226186 > Share check failed:
2014-01-19 19:22:51.226606 > Traceback (most recent call last):
2014-01-19 19:22:51.226840 >   File "/home/miner1/p2pool/p2pool/util/deferral.py", line 233, in it
2014-01-19 19:22:51.227076 >     res = gen.send(cur) # external code is run here
2014-01-19 19:22:51.227285 >   File "/home/miner1/p2pool/p2pool/util/deferral.py", line 284, in _worker
2014-01-19 19:22:51.227495 >     self.func(*self.args, **self.kwargs)
2014-01-19 19:22:51.227727 >   File "/home/miner1/p2pool/p2pool/node.py", line 304, in clean_tracker
2014-01-19 19:22:51.227955 >     best, desired, decorated_heads = self.tracker.think(self.get_height_rel_highest, self.bitcoind_work.value['previous_block'], self.bitcoind_work.value['bits'], self.known_txs_var.value)
2014-01-19 19:22:51.228188 >   File "/home/miner1/p2pool/p2pool/data.py", line 452, in think
2014-01-19 19:22:51.228440 >     if self.attempt_verify(share):
2014-01-19 19:22:51.228659 > --- <exception caught here> ---
2014-01-19 19:22:51.228866 >   File "/home/miner1/p2pool/p2pool/data.py", line 431, in attempt_verify
2014-01-19 19:22:51.229093 >     share.check(self)
2014-01-19 19:22:51.229293 >   File "/home/miner1/p2pool/p2pool/data.py", line 317, in check
2014-01-19 19:22:51.229516 >     share_info, gentx, other_tx_hashes2, get_share = self.generate_transaction(tracker, self.share_info['share_data'], self.header['bits'].target, self.share_info['timestamp'], self.share_info['bits'].target, self.contents['ref_merkle_link'], [(h, None) for h in other_tx_hashes], self.net, last_txout_nonce=self.contents['last_txout_nonce'])
2014-01-19 19:22:51.229820 >   File "/home/miner1/p2pool/p2pool/data.py", line 162, in generate_transaction
2014-01-19 19:22:51.230052 >     65535*net.NEW_SPREAD*bitcoin_data.target_to_average_attempts(block_target),
2014-01-19 19:22:51.230262 >   File "/home/miner1/p2pool/p2pool/util/memoize.py", line 28, in b
2014-01-19 19:22:51.230498 >     res = f(*args)
2014-01-19 19:22:51.230725 >   File "/home/miner1/p2pool/p2pool/util/skiplist.py", line 21, in __call__
2014-01-19 19:22:51.230955 >     self.skips[pos] = math.geometric(self.p), [(self.previous(pos), self.get_delta(pos))]
2014-01-19 19:22:51.231171 >   File "/home/miner1/p2pool/p2pool/util/forest.py", line 18, in previous
2014-01-19 19:22:51.231380 >     return self.tracker._delta_type.from_element(self.tracker.items[element]).tail
2014-01-19 19:22:51.231589 > exceptions.KeyError: None
2014-01-19 19:22:51.241558 > Share check failed:
2014-01-19 19:22:51.241947 > Traceback (most recent call last):
2014-01-19 19:22:51.242221 >   File "/home/miner1/p2pool/p2pool/util/deferral.py", line 233, in it
2014-01-19 19:22:51.242466 >     res = gen.send(cur) # external code is run here
2014-01-19 19:22:51.242717 >   File "/home/miner1/p2pool/p2pool/util/deferral.py", line 284, in _worker
2014-01-19 19:22:51.242987 >     self.func(*self.args, **self.kwargs)
2014-01-19 19:22:51.243234 >   File "/home/miner1/p2pool/p2pool/node.py", line 304, in clean_tracker
2014-01-19 19:22:51.243521 >     best, desired, decorated_heads = self.tracker.think(self.get_height_rel_highest, self.bitcoind_work.value['previous_block'], self.bitcoind_work.value['bits'], self.known_txs_var.value)
2014-01-19 19:22:51.243855 >   File "/home/miner1/p2pool/p2pool/data.py", line 452, in think
2014-01-19 19:22:51.244122 >     if self.attempt_verify(share):
2014-01-19 19:22:51.244400 > --- <exception caught here> ---
2014-01-19 19:22:51.244653 >   File "/home/miner1/p2pool/p2pool/data.py", line 431, in attempt_verify
2014-01-19 19:22:51.244937 >     share.check(self)
2014-01-19 19:22:51.245229 >   File "/home/miner1/p2pool/p2pool/data.py", line 317, in check
2014-01-19 19:22:51.245493 >     share_info, gentx, other_tx_hashes2, get_share = self.generate_transaction(tracker, self.share_info['share_data'], self.header['bits'].target, self.share_info['timestamp'], self.share_info['bits'].target, self.contents['ref_merkle_link'], [(h, None) for h in other_tx_hashes], self.net, last_txout_nonce=self.contents['last_txout_nonce'])
2014-01-19 19:22:51.245820 >   File "/home/miner1/p2pool/p2pool/data.py", line 162, in generate_transaction
2014-01-19 19:22:51.246085 >     65535*net.NEW_SPREAD*bitcoin_data.target_to_average_attempts(block_target),
2014-01-19 19:22:51.246394 >   File "/home/miner1/p2pool/p2pool/util/memoize.py", line 28, in b
2014-01-19 19:22:51.246914 >     res = f(*args)
2014-01-19 19:22:51.247158 >   File "/home/miner1/p2pool/p2pool/util/skiplist.py", line 21, in __call__
2014-01-19 19:22:51.247407 >     self.skips[pos] = math.geometric(self.p), [(self.previous(pos), self.get_delta(pos))]
2014-01-19 19:22:51.248801 >   File "/home/miner1/p2pool/p2pool/util/forest.py", line 18, in previous
2014-01-19 19:22:51.254732 >     return self.tracker._delta_type.from_element(self.tracker.items[element]).tail
2014-01-19 19:22:51.255015 > exceptions.KeyError: None
2014-01-19 19:22:51.269010 > Share check failed:
2014-01-19 19:22:51.272994 > Traceback (most recent call last):
2014-01-19 19:22:51.273263 >   File "/home/miner1/p2pool/p2pool/util/deferral.py", line 233, in it
2014-01-19 19:22:51.273532 >     res = gen.send(cur) # external code is run here
2014-01-19 19:22:51.273801 >   File "/home/miner1/p2pool/p2pool/util/deferral.py", line 284, in _worker
2014-01-19 19:22:51.274067 >     self.func(*self.args, **self.kwargs)
2014-01-19 19:22:51.274313 >   File "/home/miner1/p2pool/p2pool/node.py", line 304, in clean_tracker
2014-01-19 19:22:51.274594 >     best, desired, decorated_heads = self.tracker.think(self.get_height_rel_highest, self.bitcoind_work.value['previous_block'], self.bitcoind_work.value['bits'], self.known_txs_var.value)
2014-01-19 19:22:51.274892 >   File "/home/miner1/p2pool/p2pool/data.py", line 452, in think
2014-01-19 19:22:51.275147 >     if self.attempt_verify(share):
2014-01-19 19:22:51.275400 > --- <exception caught here> ---
2014-01-19 19:22:51.275682 >   File "/home/miner1/p2pool/p2pool/data.py", line 431, in attempt_verify
2014-01-19 19:22:51.275944 >     share.check(self)
2014-01-19 19:22:51.276187 >   File "/home/miner1/p2pool/p2pool/data.py", line 317, in check
2014-01-19 19:22:51.276442 >     share_info, gentx, other_tx_hashes2, get_share = self.generate_transaction(tracker, self.share_info['share_data'], self.header['bits'].target, self.share_info['timestamp'], self.share_info['bits'].target, self.contents['ref_merkle_link'], [(h, None) for h in other_tx_hashes], self.net, last_txout_nonce=self.contents['last_txout_nonce'])
2014-01-19 19:22:51.276791 >   File "/home/miner1/p2pool/p2pool/data.py", line 162, in generate_transaction
2014-01-19 19:22:51.277054 >     65535*net.NEW_SPREAD*bitcoin_data.target_to_average_attempts(block_target),
2014-01-19 19:22:51.277306 >   File "/home/miner1/p2pool/p2pool/util/memoize.py", line 28, in b
2014-01-19 19:22:51.277561 >     res = f(*args)
2014-01-19 19:22:51.277820 >   File "/home/miner1/p2pool/p2pool/util/skiplist.py", line 21, in __call__
2014-01-19 19:22:51.278084 >     self.skips[pos] = math.geometric(self.p), [(self.previous(pos), self.get_delta(pos))]
2014-01-19 19:22:51.278357 >   File "/home/miner1/p2pool/p2pool/util/forest.py", line 18, in previous
2014-01-19 19:22:51.278616 >     return self.tracker._delta_type.from_element(self.tracker.items[element]).tail
2014-01-19 19:22:51.278884 > exceptions.KeyError: None

[bkminer]

What does this mean?

Punishing share for 'Block-stale detected! height(1118c0efab8b0d8005a2995d3bf2c94d3823c91859cd30799) < height(2bee57eb6cf6965b98b6f34b8118b8ad863c2dc039618f41) or 19026666 != 19026666'! Jumping from 0b46a971 to 974819b9!

Is to cover "late" shares?

If I'm not mistaken I believe it might be what's discussed in this issue: https://github.com/forrestv/p2pool/issues/80

[oldbushie]

I have the latest version off of git and the last 24 hours have been really rocky:
http://192.30.165.92:9332/static/graphs.html?Day

Rebooting my VPS (where I host the p2pool node to save my home router from being overloaded) and my miners (along with the laptop I connect to my miners) doesn't seem to have alleviated the situation, I keep getting periodic losses in connectivity.

I've alerted the VPS provider but it seems like there may be some madness from the recent p2pool update according to this thread.

[alanwgeorge]

I am getting this spam.  File corruption or something?

Code:

2014-01-19 19:22:51.214727 >   File "/home/miner1/p2pool/p2pool/util/memoize.py", line 28, in b
2014-01-19 19:22:51.214885 >     res = f(*args)
2014-01-19 19:22:51.215046 >   File "/home/miner1/p2pool/p2pool/util/skiplist.py", line 21, in __call__
2014-01-19 19:22:51.215200 >     self.skips[pos] = math.geometric(self.p), [(self.previous(pos), self.get_delta(pos))]
2014-01-19 19:22:51.215357 >   File "/home/miner1/p2pool/p2pool/util/forest.py", line 18, in previous
2014-01-19 19:22:51.215510 >     return self.tracker._delta_type.from_element(self.tracker.items[element]).tail
2014-01-19 19:22:51.215695 > exceptions.KeyError: None
2014-01-19 19:22:51.226186 > Share check failed:
2014-01-19 19:22:51.226606 > Traceback (most recent call last):
2014-01-19 19:22:51.226840 >   File "/home/miner1/p2pool/p2pool/util/deferral.py", line 233, in it
2014-01-19 19:22:51.227076 >     res = gen.send(cur) # external code is run here
2014-01-19 19:22:51.227285 >   File "/home/miner1/p2pool/p2pool/util/deferral.py", line 284, in _worker
2014-01-19 19:22:51.227495 >     self.func(*self.args, **self.kwargs)
2014-01-19 19:22:51.227727 >   File "/home/miner1/p2pool/p2pool/node.py", line 304, in clean_tracker
2014-01-19 19:22:51.227955 >     best, desired, decorated_heads = self.tracker.think(self.get_height_rel_highest, self.bitcoind_work.value['previous_block'], self.bitcoind_work.value['bits'], self.known_txs_var.value)
2014-01-19 19:22:51.228188 >   File "/home/miner1/p2pool/p2pool/data.py", line 452, in think
2014-01-19 19:22:51.228440 >     if self.attempt_verify(share):
2014-01-19 19:22:51.228659 > --- <exception caught here> ---
2014-01-19 19:22:51.228866 >   File "/home/miner1/p2pool/p2pool/data.py", line 431, in attempt_verify
2014-01-19 19:22:51.229093 >     share.check(self)
2014-01-19 19:22:51.229293 >   File "/home/miner1/p2pool/p2pool/data.py", line 317, in check
2014-01-19 19:22:51.229516 >     share_info, gentx, other_tx_hashes2, get_share = self.generate_transaction(tracker, self.share_info['share_data'], self.header['bits'].target, self.share_info['timestamp'], self.share_info['bits'].target, self.contents['ref_merkle_link'], [(h, None) for h in other_tx_hashes], self.net, last_txout_nonce=self.contents['last_txout_nonce'])
2014-01-19 19:22:51.229820 >   File "/home/miner1/p2pool/p2pool/data.py", line 162, in generate_transaction
2014-01-19 19:22:51.230052 >     65535*net.NEW_SPREAD*bitcoin_data.target_to_average_attempts(block_target),
2014-01-19 19:22:51.230262 >   File "/home/miner1/p2pool/p2pool/util/memoize.py", line 28, in b
2014-01-19 19:22:51.230498 >     res = f(*args)
2014-01-19 19:22:51.230725 >   File "/home/miner1/p2pool/p2pool/util/skiplist.py", line 21, in __call__
2014-01-19 19:22:51.230955 >     self.skips[pos] = math.geometric(self.p), [(self.previous(pos), self.get_delta(pos))]
2014-01-19 19:22:51.231171 >   File "/home/miner1/p2pool/p2pool/util/forest.py", line 18, in previous
2014-01-19 19:22:51.231380 >     return self.tracker._delta_type.from_element(self.tracker.items[element]).tail
2014-01-19 19:22:51.231589 > exceptions.KeyError: None
2014-01-19 19:22:51.241558 > Share check failed:
2014-01-19 19:22:51.241947 > Traceback (most recent call last):
2014-01-19 19:22:51.242221 >   File "/home/miner1/p2pool/p2pool/util/deferral.py", line 233, in it
2014-01-19 19:22:51.242466 >     res = gen.send(cur) # external code is run here
2014-01-19 19:22:51.242717 >   File "/home/miner1/p2pool/p2pool/util/deferral.py", line 284, in _worker
2014-01-19 19:22:51.242987 >     self.func(*self.args, **self.kwargs)
2014-01-19 19:22:51.243234 >   File "/home/miner1/p2pool/p2pool/node.py", line 304, in clean_tracker
2014-01-19 19:22:51.243521 >     best, desired, decorated_heads = self.tracker.think(self.get_height_rel_highest, self.bitcoind_work.value['previous_block'], self.bitcoind_work.value['bits'], self.known_txs_var.value)
2014-01-19 19:22:51.243855 >   File "/home/miner1/p2pool/p2pool/data.py", line 452, in think
2014-01-19 19:22:51.244122 >     if self.attempt_verify(share):
2014-01-19 19:22:51.244400 > --- <exception caught here> ---
2014-01-19 19:22:51.244653 >   File "/home/miner1/p2pool/p2pool/data.py", line 431, in attempt_verify
2014-01-19 19:22:51.244937 >     share.check(self)
2014-01-19 19:22:51.245229 >   File "/home/miner1/p2pool/p2pool/data.py", line 317, in check
2014-01-19 19:22:51.245493 >     share_info, gentx, other_tx_hashes2, get_share = self.generate_transaction(tracker, self.share_info['share_data'], self.header['bits'].target, self.share_info['timestamp'], self.share_info['bits'].target, self.contents['ref_merkle_link'], [(h, None) for h in other_tx_hashes], self.net, last_txout_nonce=self.contents['last_txout_nonce'])
2014-01-19 19:22:51.245820 >   File "/home/miner1/p2pool/p2pool/data.py", line 162, in generate_transaction
2014-01-19 19:22:51.246085 >     65535*net.NEW_SPREAD*bitcoin_data.target_to_average_attempts(block_target),
2014-01-19 19:22:51.246394 >   File "/home/miner1/p2pool/p2pool/util/memoize.py", line 28, in b
2014-01-19 19:22:51.246914 >     res = f(*args)
2014-01-19 19:22:51.247158 >   File "/home/miner1/p2pool/p2pool/util/skiplist.py", line 21, in __call__
2014-01-19 19:22:51.247407 >     self.skips[pos] = math.geometric(self.p), [(self.previous(pos), self.get_delta(pos))]
2014-01-19 19:22:51.248801 >   File "/home/miner1/p2pool/p2pool/util/forest.py", line 18, in previous
2014-01-19 19:22:51.254732 >     return self.tracker._delta_type.from_element(self.tracker.items[element]).tail
2014-01-19 19:22:51.255015 > exceptions.KeyError: None
2014-01-19 19:22:51.269010 > Share check failed:
2014-01-19 19:22:51.272994 > Traceback (most recent call last):
2014-01-19 19:22:51.273263 >   File "/home/miner1/p2pool/p2pool/util/deferral.py", line 233, in it
2014-01-19 19:22:51.273532 >     res = gen.send(cur) # external code is run here
2014-01-19 19:22:51.273801 >   File "/home/miner1/p2pool/p2pool/util/deferral.py", line 284, in _worker
2014-01-19 19:22:51.274067 >     self.func(*self.args, **self.kwargs)
2014-01-19 19:22:51.274313 >   File "/home/miner1/p2pool/p2pool/node.py", line 304, in clean_tracker
2014-01-19 19:22:51.274594 >     best, desired, decorated_heads = self.tracker.think(self.get_height_rel_highest, self.bitcoind_work.value['previous_block'], self.bitcoind_work.value['bits'], self.known_txs_var.value)
2014-01-19 19:22:51.274892 >   File "/home/miner1/p2pool/p2pool/data.py", line 452, in think
2014-01-19 19:22:51.275147 >     if self.attempt_verify(share):
2014-01-19 19:22:51.275400 > --- <exception caught here> ---
2014-01-19 19:22:51.275682 >   File "/home/miner1/p2pool/p2pool/data.py", line 431, in attempt_verify
2014-01-19 19:22:51.275944 >     share.check(self)
2014-01-19 19:22:51.276187 >   File "/home/miner1/p2pool/p2pool/data.py", line 317, in check
2014-01-19 19:22:51.276442 >     share_info, gentx, other_tx_hashes2, get_share = self.generate_transaction(tracker, self.share_info['share_data'], self.header['bits'].target, self.share_info['timestamp'], self.share_info['bits'].target, self.contents['ref_merkle_link'], [(h, None) for h in other_tx_hashes], self.net, last_txout_nonce=self.contents['last_txout_nonce'])
2014-01-19 19:22:51.276791 >   File "/home/miner1/p2pool/p2pool/data.py", line 162, in generate_transaction
2014-01-19 19:22:51.277054 >     65535*net.NEW_SPREAD*bitcoin_data.target_to_average_attempts(block_target),
2014-01-19 19:22:51.277306 >   File "/home/miner1/p2pool/p2pool/util/memoize.py", line 28, in b
2014-01-19 19:22:51.277561 >     res = f(*args)
2014-01-19 19:22:51.277820 >   File "/home/miner1/p2pool/p2pool/util/skiplist.py", line 21, in __call__
2014-01-19 19:22:51.278084 >     self.skips[pos] = math.geometric(self.p), [(self.previous(pos), self.get_delta(pos))]
2014-01-19 19:22:51.278357 >   File "/home/miner1/p2pool/p2pool/util/forest.py", line 18, in previous
2014-01-19 19:22:51.278616 >     return self.tracker._delta_type.from_element(self.tracker.items[element]).tail
2014-01-19 19:22:51.278884 > exceptions.KeyError: None

I am seeing same.

Code:

2014-01-19 12:34:09.334941 > --- <exception caught here> ---
2014-01-19 12:34:09.334967 >   File "/Users/ageo/src/p2pool/p2pool/data.py", line 431, in attempt_verify
2014-01-19 12:34:09.334994 >     share.check(self)
2014-01-19 12:34:09.335020 >   File "/Users/ageo/src/p2pool/p2pool/data.py", line 320, in check
2014-01-19 12:34:09.335047 >     raise ValueError('share_info invalid')
2014-01-19 12:34:09.335074 > exceptions.ValueError: share_info invalid
2014-01-19 12:34:09.359392 > Share check failed:
2014-01-19 12:34:09.359472 > Traceback (most recent call last):
2014-01-19 12:34:09.359516 >   File "/Users/ageo/src/p2pool/p2pool/util/deferral.py", line 233, in it
2014-01-19 12:34:09.359555 >     res = gen.send(cur) # external code is run here
2014-01-19 12:34:09.359603 >   File "/Users/ageo/src/p2pool/p2pool/util/deferral.py", line 284, in _worker
2014-01-19 12:34:09.359652 >     self.func(*self.args, **self.kwargs)
2014-01-19 12:34:09.359789 >   File "/Users/ageo/src/p2pool/p2pool/node.py", line 304, in clean_tracker
2014-01-19 12:34:09.359856 >     best, desired, decorated_heads = self.tracker.think(self.get_height_rel_highest, self.bitcoind_work.value['previous_block'], self.bitcoind_work.value['bits'], self.known_txs_var.value)
2014-01-19 12:34:09.359916 >   File "/Users/ageo/src/p2pool/p2pool/data.py", line 452, in think

In periods 5 to 10 minutes.  bfgminer fails over to secondary pool then returns once p2pool errors subside.

Any ideas as to cause?

Thanks,
-alan

EDIT: It's actually happening again as I type.  P2pool filled with the errors above and miners failing over.  1:19PM PST.  Anybody else seeing same?


EDIT 2: Looks like something local.

Code:

n' + ''.join(traceback.format_stack())
2014-01-19 13:16:21.051978 > ########################################
2014-01-19 13:16:21.052079 > >>> Warning: LOST CONTACT WITH BITCOIND for 4.6 minutes! Check that it isn't frozen or dead!
2014-01-19 13:16:21.052118 > ########################################
2014-01-19 13:16:21.052153 P2Pool: 17395 shares in chain (17600 verified/20953 total) Peers: 0 (0 incoming)

[forrestv]

There is currently a problem and a fix is being worked on. For now, refusing incoming connections (--max-conns 0) is a temporary solution. Expect more news in the next half hour.

[smoothrunnings]

There is currently a problem and a fix is being worked on. For now, refusing incoming connections (--max-conns 0) is a temporary solution. Expect more news in the next half hour.

Just as I suspected. I had my P2Pool running yesterday evening for the first time and didn't have port port 9333 open on my firewall so there was no incoming connections and it ran fine until I opened port 9333. Looking forward to the perm fix.

[TierNolan]

Just as I suspected. I had my P2Pool running yesterday evening for the first time and didn't have port port 9333 open on my firewall so there was no incoming connections and it ran fine until I opened port 9333. Looking forward to the perm fix.

Right, if everyone refuses incoming connections, then the p2p system cannot work.