[1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool

[Ghost of USD]

Please check the cpu load on your avalon. It is near 80% on my 3 module avalon and it can be 100% on a 4 module. If so, then thats the problem. The cpu is to tiny on the wrt board. If you realy want to use p2pool, then connect the avalon controller usb cable (remove Jumper J1 on the controller board) to a PC/Notebook and run cgminer on it. Everything is controlled by cgminer eg. fans, clocking etc.

This is the answer.

on p2pool, load average: 0.98 0.92 0.87

vs

ozco, load average: 0.19 0.16 0.20

[1713288766]

1713288766

[1713288766]

1713288766

[1713288766]

1713288766

[1713288766]

1713288766

[1713288766]

1713288766

[HellDiverUK]

Are you using the latest 13.2 version of p2pool?  Didn't 13.1 have issues with Avalons? Or was that something different?

[Ghost of USD]

Are you using the latest 13.2 version of p2pool?  Didn't 13.1 have issues with Avalons? Or was that something different?

My node is running from latest git commit:

P2Pool version: 0a3493d

Current local DOA is 5.9% and GBT Latency is < 0.1

Pretty sure the problem is cgminer requiring too many cpu cycles.

[HellDiverUK]

Tried bfgminer?  Works better for me with my little Erupters.

[xyzzy099]

Tried bfgminer?  Works better for me with my little Erupters.

What driver do you use for the erupters with BFGMiner?  I have tried WinUSB (zadig) and the Silicon Labs CP210x drivers, and BFGMiner refuses to see either.

[kano]

Latest cgminer released yesterday has libusb replaced with a version that resolves most historical AMU problems

[daemondazz]

This is the answer.

on p2pool, load average: 0.98 0.92 0.87

vs

ozco, load average: 0.19 0.16 0.20

You do understand what the load average figures are telling you, right? It is simply a count of how many processes running on the system needed CPU time per second during that interval, it does not tell you anything at all about the actual CPU time spent servicing those processes.

Eg, if I do a

Code:

while /bin/true; do echo "."; sleep 1; done

The load average will show as 2, even though you can probably see that the actual amount of CPU time spent was likely < 0.001% ?

A high load average may be an indication of the system being CPU starved, but I'd judge "high load average" to be 10-12x the number of CPUs present in the system, depending on the type of processes the machine is running.

[Ghost of USD]

This is the answer.

on p2pool, load average: 0.98 0.92 0.87

vs

ozco, load average: 0.19 0.16 0.20

You do understand what the load average figures are telling you, right? It is simply a count of how many processes running on the system needed CPU time per second during that interval, it does not tell you anything at all about the actual CPU time spent servicing those processes.

Eg, if I do a

Code:

while /bin/true; do echo "."; sleep 1; done

The load average will show as 2, even though you can probably see that the actual amount of CPU time spent was likely < 0.001% ?

A high load average may be an indication of the system being CPU starved, but I'd judge "high load average" to be 10-12x the number of CPUs present in the system, depending on the type of processes the machine is running.

Of course I do.

In this case it is the cgminer process causing that load, just as ebereon suggested.

Tried bfgminer?  Works better for me with my little Erupters.

Thanks for the suggestion.

After a bit of tinkering, and with the help of this script http://luke.dashjr.org/tmp/code/avalonhost-raminst, I managed to get the latest bfgminer working on my Avalon.

The difference is staggering. HW error rate dropped from 0.95% to 0.58%, and cpu usage while mining on p2pool went from 90% down to 8% max.

[-ck]

The difference is staggering. HW error rate dropped from 0.95% to 0.58%, and cpu usage while mining on p2pool went from 90% down to 8% max.

Are you mining directly with stratum to p2pool?

[Ghost of USD]

The difference is staggering. HW error rate dropped from 0.95% to 0.58%, and cpu usage while mining on p2pool went from 90% down to 8% max.

Are you mining directly with stratum to p2pool?

Yes.
 

Code:

root@OpenWrt:~# cgminer --lowmem --avalon-options 115200:32:10:43:300 -o stratum+tcp://192.168.1.77:9332 --api-allow W:127.0.0.1 --api-listen --avalon-cutoff 90 --avalon-temp 70
 [2013-08-10 00:44:56] Started cgminer 3.3.1 
 [2013-08-10 00:44:57] Probing for an alive pool
 [2013-08-10 00:44:57] Pool 0 difficulty changed to 82.382325                   
 [2013-08-10 00:44:57] Pool 0 difficulty changed to 128
 [2013-08-10 00:44:57] Stratum from pool 0 detected new block                   
 [2013-08-10 00:44:57] Stratum from pool 0 requested work restart

   

[gyverlb]

This is the answer.

on p2pool, load average: 0.98 0.92 0.87

vs

ozco, load average: 0.19 0.16 0.20

You do understand what the load average figures are telling you, right? It is simply a count of how many processes running on the system needed CPU time per second during that interval, it does not tell you anything at all about the actual CPU time spent servicing those processes.

Eg, if I do a

Code:

while /bin/true; do echo "."; sleep 1; done

The load average will show as 2,

No it won't. The load measures the number of processes using CPU, waiting for CPU or waiting for I/O system calls to complete (basically reads and writes on file descriptors).

even though you can probably see that the actual amount of CPU time spent was likely < 0.001% ?

A high load average may be an indication of the system being CPU starved, but I'd judge "high load average" to be 10-12x the number of CPUs present in the system, depending on the type of processes the machine is running.

Load > nb of CPU threads may not have anything to do with CPU, just fork large number of processes writing to disk, they won't use much CPU on modern hardware (where every I/O use DMA) but your load will be as high as the number of processes you forked.

In the case of cgminer on Avalon with p2pool, ckolivas already replied in an earlier conversation that p2pool makes cgminer use more CPU than traditional pools. This doesn't show on "normal" CPUs but Avalon has a very weak one.

[forrestv]

An Avalon mining on P2Pool with even 3 modules does indeed take its CPU to the limit. The only real difference between P2Pool and other pools is that P2Pool's generation transaction is much larger. I'm guessing that the bottleneck is hashing the generation transaction ~16 times per second (though that isn't much...). If that's true, perhaps cgminer could be optimized to compress everything before the Stratum nonce to a SHA-256 midstate?

[kano]

No that isn't the only difference, the other 2 major problems are:
1) it's python - so it will use more CPU
2) it only uses one thread by default and thus p2pool itself can easily get CPU starved on a machine with 16 cores ...

[forrestv]

No that isn't the only difference, the other 2 major problems are:
1) it's python - so it will use more CPU
2) it only uses one thread by default and thus p2pool itself can easily get CPU starved on a machine with 16 cores ...

I'm talking about the Avalon device's CPU.

[kano]

No that isn't the only difference, the other 2 major problems are:
1) it's python - so it will use more CPU
2) it only uses one thread by default and thus p2pool itself can easily get CPU starved on a machine with 16 cores ...

I'm talking about the Avalon device's CPU.

Ah OK, you mean profiling cgminer ... yep that does indeed need to be done.
Guessing about profiling is one of the common mistakes by most programmers ... but yes (again) it does need to be done.

[Ghost of USD]

bfgminer has now been running for the past 20 hours on my Avalon @325 mining on p2pool. So far, it is working fast and stable.

Stats:

Code:

Miner Status Uptime        MH/s        A       R HW% Utility    Invalid WU      BestShare
001 ONLINE 0d 20:12:15 103454.057 6258 601 0.74% 5.162     8.76% 1445.371 11669957

Output of top:

Code:

Mem: 21880K used, 7220K free, 0K shrd, 2164K buff, 9140K cached
CPU:   3% usr   2% sys   0% nic  93% idle   0% io   0% irq   0% sirq
Load average: 0.05 0.08 0.12 3/42 1510
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
 1274  1269 root     S    27728  95%   6% bfgminer -S avalon:/dev/ttyUSB0 -o ht
 1510  1505 root     R     1500   5%   0% top
 1505  1504 root     S     1504   5%   0% -ash
 1269  1268 root     S     1504   5%   0% -ash
  697     1 root     S     1500   5%   0% /usr/sbin/ntpd -n -p 0.openwrt.pool.n
  545     1 root     S     1472   5%   0% /sbin/netifd
    1     0 root     S     1432   5%   0% /sbin/procd
 1268   639 root     S     1244   4%   0% /usr/sbin/dropbear -F -P /var/run/dro
 1504   639 root     S     1220   4%   0% /usr/sbin/dropbear -F -P /var/run/dro
  655     1 root     S     1164   4%   0% /usr/sbin/uhttpd -f -h /www -r OpenWr
  639     1 root     S     1156   4%   0% /usr/sbin/dropbear -F -P /var/run/dro
  685     1 nobody   S      956   3%   0% /usr/sbin/dnsmasq -C /var/etc/dnsmasq
  432     1 root     S <    904   3%   0% ubusd
  434     1 root     S      768   3%   0% /sbin/askfirst ttyATH0 /bin/ash --log

[-ck]

An Avalon mining on P2Pool with even 3 modules does indeed take its CPU to the limit. The only real difference between P2Pool and other pools is that P2Pool's generation transaction is much larger. I'm guessing that the bottleneck is hashing the generation transaction ~16 times per second (though that isn't much...). If that's true, perhaps cgminer could be optimized to compress everything before the Stratum nonce to a SHA-256 midstate?

There's an obvious optimisation there that I didn't spot and it's only now that we have hardware being outstripped by the work generation requirements that I've looked into it. Thanks for the pointer, I shall see what I can do.

[-ck]

An Avalon mining on P2Pool with even 3 modules does indeed take its CPU to the limit. The only real difference between P2Pool and other pools is that P2Pool's generation transaction is much larger. I'm guessing that the bottleneck is hashing the generation transaction ~16 times per second (though that isn't much...). If that's true, perhaps cgminer could be optimized to compress everything before the Stratum nonce to a SHA-256 midstate?

There's an obvious optimisation there that I didn't spot and it's only now that we have hardware being outstripped by the work generation requirements that I've looked into it. Thanks for the pointer, I shall see what I can do.

Yes, that did it. I've committed code to the cgminer git tree which dramatically reduces the cpu usage (confirmed on my avalon mining to p2pool). When my server comes back online I'll upload new avalon firmware.

[HellDiverUK]

Tried bfgminer?  Works better for me with my little Erupters.

What driver do you use for the erupters with BFGMiner?  I have tried WinUSB (zadig) and the Silicon Labs CP210x drivers, and BFGMiner refuses to see either.

Get rid of WinUSB.  Just use the VCP drivers for bfgminer - it'll work if you see your BEs being assigned a COM port in Windows.

The WinUSB drivers are, in my opinion, a HUGE mistake that the cgminer folks have made.  It's unnecessarily difficult to get working, and gives no real benefit.  The VCP drivers are a 2 second install, and they just work (or at least have for me on a few different machines on Server 2012, Win7 and Win8.

[kano]

I guess you haven't noticed all the FTDI code in the clone that bypasses the COM ports ... of the fact that the ztex is ONLY libusb ... as is the x6500 code ... of that the COM ports are actually about 5 or 6 different drivers underneath them ... i.e. it a mix of many different drivers and code ...

cgminer is purely only direct USB, libusb for all drivers, and simple instructions (Zadig) that all but the inept can follow.