[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[-ck]

P.P.S. A question to any developers here. May I officially state that AM blade software is a piece of crap? No stratum, no long polling and constant difficulty in the year 2013. It might be OK for their internal mining but not for a sold product!

Yes you may say that and you won't get any argument from developers (we weren't remotely involved, engaged or asked).

[baloo_kiev]

Brief report of p2pool + AM block eruptor blade investigation

1) Blade always works on difficulty 1, regardless of what server tells, so p2pool must always provide diff 1 pseudoshares to account solved work properly. This does not affect real shares difficulty, which can be tuned as usually.

2) Blade does not use long polling. Instead, it makes about 4 getwork requests per second. Looks like it makes as much requests as it can, each providing work to one chip. With 32 chips, it means about 8 seconds per full round. Maybe it doesn't send solutions before doing this full round; this must rersult in about 50% DOA. Or maybe results are put into queue and wait while software is busy requesting new work. In both cases, running it locally may help. Also, I'm not sure that real share DOA will be as high as pseudoshare DOA.

3) "p2pool + stratum proxy + AM blade" setup has some authorization-related issue. This is all I can tell by now, will try to get more details.

P.S. With the difficulty tweak, it works giving about 50% pseudoshare DOA. Real share efficiency is unknown because of little statistics. Those who have AM blades may give it a try at 78.27.191.182:8353 (with your payout address as username).

P.P.S. A question to any developers here. May I officially state that AM blade software is a piece of crap? No stratum, no long polling and constant difficulty in the year 2013. It might be OK for their internal mining but not for a sold product!

Update!
At last, the blade's mining mechanism is clear to me. Like in AM USB, found solutions are reported immediately. However, unlike AM USB and like BFL devices, work is not restarted. This means that a chip keeps mining the same work for about 12 seconds (I guess it corresponds to full nonce range of 32 bit, and is equal to average chip's time per one diff1 share). Since solutions are reported immediately, this must give slightly less than 50% DOA shares.

There's a solution that can considerably increase mining efficiency on p2pool for ASIC devices, and this solution is... share difficulty. By setting it high you not only help small miners, but also protect yourself from orphan shares! If you are mining with x10 share difficulty, your "DOA" shares will most probably get into the chain, since your share's work is bigger than even 9 min diff shares. With current mindiff of 1500 and expected time to block of 30 hours, ASIC miners (except AM USB which works fine) can set their diff to 15000 without any serious increase of variance. Shares which are first reported DOA in p2pool node log may still get into the chain!

[forrestv]

There's a solution that can considerably increase mining efficiency on p2pool for ASIC devices, and this solution is... share difficulty. Setting it high you not only help small miners, but also protect yourself from orphan shares! If you are mining with x10 share difficulty, your "DOA" shares will most probably get into the chain, since your share's work is bigger than even 9 min diff shares. With current mindiff of 1500 and expected time to block of 30 hours, ASIC miners (except AM USB which works fine) can set their diff to 15000 without any serious increase of variance!

This actually doesn't do anything (thankfully). P2Pool uses the work of a share's 5th parent when sorting sharechain tips, which prevents attacks that would allow you to effectively orphan the tip share on command.

[bitpop]

Why would they fix this only in the usb version?

[daemondazz]

The USB version doesn't run standalone.

[Krellan]

* Delay payout calculation 1 share so that payouts can be calculated ahead of time, reducing latency between a new share being received and work being distributed to miners

Good idea to precalculate the blocks that will be worked on, to reduce latency.  However, curious how this works.  If blocks are now precalculated, delayed by 1 share, then where will the most recent winning share be stored in the sharechain?  I thought it worked now by each miner creating a new block containing their payout address, and trying to solve that, until finding a winning solution, at which point the block becomes a valid share in the sharechain.  If this isn't the case, then where will the payout addresses be stored now?  I hope this doesn't open up an opportunity for a miner to cheat.  Or, do I have it all wrong here?  Thanks!

[forrestv]

* Delay payout calculation 1 share so that payouts can be calculated ahead of time, reducing latency between a new share being received and work being distributed to miners

Good idea to precalculate the blocks that will be worked on, to reduce latency.  However, curious how this works.  If blocks are now precalculated, delayed by 1 share, then where will the most recent winning share be stored in the sharechain?  I thought it worked now by each miner creating a new block containing their payout address, and trying to solve that, until finding a winning solution, at which point the block becomes a valid share in the sharechain.  If this isn't the case, then where will the payout addresses be stored now?  I hope this doesn't open up an opportunity for a miner to cheat.  Or, do I have it all wrong here?  Thanks!

The sharechain has some P2Pool-specific data (including the winner's payout address) within each share, along with everything needed to compute the current block being mined. So no, this is a very safe change.

[Krellan]

I'm not sure how Erupter blades behave with P2Pool, please give us a quick report when your reach 25 total shares then 50 (the more you have the more we will know for sure how your setup behaves).

Up to 50 shares now!

Not exactly Nasty, but still chugging along nicely, for only a single Erupter.

I increased the number of outgoing connections to the maximum allowed of 10, and that seemed to have helped a lot.  Having more network connections helps keep my node "in the loop" so that it is not the last to find out about the newest blocks.

P2Pool webpage:

Code:

Node uptime: 8.525 days Peers: 10 out, 10 in
Local rate: 344MH/s (0.0% DOA) Expected time to share: 4.97 hours
Shares: 52 total (8 orphaned, 0 dead) Efficiency: 102.5%

P2Pool console dump:

Code:

2013-06-30 23:23:42.542263 P2Pool: 17325 shares in chain (17329 verified/17329 total) Peers: 20 (10 incoming)
2013-06-30 23:23:42.542394  Local: 379MH/s in last 10.0 minutes Local dead on arrival: ~0.0% (0-7%) Expected time to share: 4.6 hours
2013-06-30 23:23:42.542474  Shares: 52 (8 orphan, 0 dead) Stale rate: ~15.4% (8-28%) Efficiency: ~102.5% (87-112%) Current payout: 0.0137 BTC
2013-06-30 23:23:42.542560  Pool: 774GH/s Stale rate: 17.4% Expected time to block: 1.4 days

BFGMiner console header text:

Code:

--------------------------------------------------------------------------------
 5s: 87.3 avg:335.9 u:326.3 Mh/s | A:55990 R:1012+0(1.8%) HW:507
 ST: 1  GF: 0  NB: 1397  AS: 0  RF: 0  E: 0.06  U:4.6/m  BS:274k
 Connected to localhost diff 1 with stratum as user P2Pool
 Block: ...4592a33c #244201  Diff:21.3M (152.7Th/s)  Started: [23:25:51]
--------------------------------------------------------------------------------
 [P]ool management [S]ettings [D]isplay options [Q]uit
 ICA 0:                | 169.5/335.9/326.3Mh/s | A:55990 R:1012+0(1.8%) HW:507
--------------------------------------------------------------------------------

BFGMiner pool information:

Code:

Queued work requests: 78282
 Share submissions: 56993
 Accepted shares: 55981
 Rejected shares: 1012 + 0 stale (0.02%)
 Accepted difficulty shares: 55981
 Rejected difficulty shares: 1012
 Efficiency (accepted * difficulty / 2 KB): 0.06
 Stale submissions discarded due to new blocks: 0
 Unable to get work from server occasions: 0
 Submitting work remotely delay occasions: 0

It's been nicely stable.  I need to upgrade Bitcoin to 0.8.3 (from 0.8.2) and BFGMiner to 3.1.1 (from 3.1.0), so will restart it soon for that.

[baloo_kiev]

Why would they fix this only in the usb version?

Usb version is controlled by third-party mining software like cgminer and others. Blades are controlled by closed-source software written (poorly!) by some AM's partner company. I guess the only thing can be done here is waiting until AM fixes its dumb software for its smart chips.

[daemondazz]

I would not be surprised if one of the biggest changes to their "next generation" blade will be the firmware. The first version of the blade's I think really were the quick and dirty to get them going. Version 2 will be the time to fix the problems.

[centove]

Is it normal for a node with 2GH/s?

No. You just step on memory leak. Git pull and restart node.

Updated and restarted, looks better now: 300 MB of memory used.
Does it increases with the hashrate or not?

It seems to be correlated to the # of peers/workers connected.

[rav3n_pl]

From my observation it is some leak in tcp/ip twisted/zope connectors.
Lak occurs when there is some switching in connected nodes and connected miners.
Probably something is "hanging" connections buffers that are spawned in next created connections and making cascade eat of ram.
Found some articles about that. Looks like connections that are not closed "directly" are moved to new instance by default and then even after connection finally drops memory is "eaten" and GC is not freeing it.

[Boing7898]

After updating to the last git, I'm starting to get a lot of "Couldn't bind: 24: too many open files"
What's the problem?
I have to restart p2pool each time it happens..

Also wtf, is this normal? That looks like a Litecoin address to me.. I'm hosting a Bitcoin pool.

2013-07-01 12:15:46.267753 Worker Lcn8gbjgv3L9V3WQMEgxxRBrbmEbCVNCF3 submitted share with hash > target:
2013-07-01 12:15:46.267822     Hash:   3509665dafb744bfcfcc932d7cf29052e51c4afe763165b4e3e1f80ec85286a1
2013-07-01 12:15:46.268389     Target: ffffffffffffffffffffffffffffffffffffffffffffffffffffffff

[forrestv]

After updating to the last git, I'm starting to get a lot of "Couldn't bind: 24: too many open files"
What's the problem?
I have to restart p2pool each time it happens..

Also wtf, is this normal? That looks like a Litecoin address to me.. I'm hosting a Bitcoin pool.

2013-07-01 12:15:46.267753 Worker Lcn8gbjgv3L9V3WQMEgxxRBrbmEbCVNCF3 submitted share with hash > target:
2013-07-01 12:15:46.267822     Hash:   3509665dafb744bfcfcc932d7cf29052e51c4afe763165b4e3e1f80ec85286a1
2013-07-01 12:15:46.268389     Target: ffffffffffffffffffffffffffffffffffffffffffffffffffffffff

Can you pastebin one of the "too many open files" tracebacks?

And that error is just someone using a Litecoin miner on your pool...

[Boing7898]

After updating to the last git, I'm starting to get a lot of "Couldn't bind: 24: too many open files"
What's the problem?
I have to restart p2pool each time it happens..

Also wtf, is this normal? That looks like a Litecoin address to me.. I'm hosting a Bitcoin pool.

2013-07-01 12:15:46.267753 Worker Lcn8gbjgv3L9V3WQMEgxxRBrbmEbCVNCF3 submitted share with hash > target:
2013-07-01 12:15:46.267822     Hash:   3509665dafb744bfcfcc932d7cf29052e51c4afe763165b4e3e1f80ec85286a1
2013-07-01 12:15:46.268389     Target: ffffffffffffffffffffffffffffffffffffffffffffffffffffffff

Can you pastebin one of the "too many open files" tracebacks?

And that error is just someone using a Litecoin miner on your pool...

Here it is: http://pastebin.com/iyPupJnp

Oh.. is there a way to block Litecoin addresses/miners? It's just spamming the console.

[freshzive]

so does p2pool work properly with bitcoind 0.8.3? wanted to double check before i upgrade

quote author=forrestv link=topic=18313.msg2562076#msg2562076 date=1372036660]
In the next couple days, I'm going to release a hard-forking change to P2Pool that will make the following changes

Bitcoin network
* Increase SHARE_PERIOD (average time between shares) from 10 seconds to 30 seconds, in order to make it more fair for high-latency miners (read: ASICs)
[/quote]

also...this hasn't been done yet, right?

[baloo_kiev]

Bitcoin network
* Increase SHARE_PERIOD (average time between shares) from 10 seconds to 30 seconds, in order to make it more fair for high-latency miners (read: ASICs)

also...this hasn't been done yet, right?

I think it has been done but switch to new share version will occur when most hashrate upgrades. P2pool graphs show only 1% new version now.

[baloo_kiev]

Another possible solution to AMblade-p2pool issue here!

Blade owners could just fork their own p2pool. The only 4 lines of code which need to be changed are share period, p2pool network identifier (prefix), bootstrap nodes, and optionally chain length. As blade's "effective latency" is about 6 seconds, reasonable values for share period are 5 or 10 minutes.

The hardest part is involving miners. You need about 100 blades in total to reach mean rate of 1 block per day. At this hashrate, 1 blade's expected time to share will be 8 and 17 hours for 5 and 10 minute share period correspondingly (or even less if multiple-blade miners adjust their difficulty properly). Maybe someone should make announcements here and in AM blade discussion threads to get feedback and estimate how many blades will support this.

[Boing7898]

After updating to the last git, I'm starting to get a lot of "Couldn't bind: 24: too many open files"
What's the problem?
I have to restart p2pool each time it happens..

Also wtf, is this normal? That looks like a Litecoin address to me.. I'm hosting a Bitcoin pool.

2013-07-01 12:15:46.267753 Worker Lcn8gbjgv3L9V3WQMEgxxRBrbmEbCVNCF3 submitted share with hash > target:
2013-07-01 12:15:46.267822     Hash:   3509665dafb744bfcfcc932d7cf29052e51c4afe763165b4e3e1f80ec85286a1
2013-07-01 12:15:46.268389     Target: ffffffffffffffffffffffffffffffffffffffffffffffffffffffff

Can you pastebin one of the "too many open files" tracebacks?

And that error is just someone using a Litecoin miner on your pool...

Here it is: http://pastebin.com/iyPupJnp

Oh.. is there a way to block Litecoin addresses/miners? It's just spamming the console.

Ugh.. I keep getting this error each 1.5 hours.. making me restart p2pool each hour or so.

[svirus]

in command line:
ulimit -n 8192
and run p2pool

and check later if errors show.

Open files in linux-like system is also opened connections.