cgminer - CPU/GPU miner in C for linux/windows

[burp]

Updated tree:

Implemented never idle logic. During periods of network or server problems it takes existing work and generates more work from that till the server starts responding properly or fast enough. This means that hash rates should -never- drop now with cgminer.

This is it in action:

[Accepted] [GPU 2] [435.0 Mh/s] [Q:111  A:21  R:0  HW:0  E:19%  U:22.20/m]                 
[2011-07-09 10:26:01] Server not providing work fast enough, generating work locally
[Accepted] [GPU 3] [429.4 Mh/s] [Q:135  A:21  R:1  HW:0  E:16%  U:18.13/m]                 
[2011-07-09 10:26:53] Resumed retrieving work from server

If your network is down for extensive periods eventually this will generate more rejected blocks, but for transient blips this makes a massive difference.

Awesome feature

[-ck]

Updated source.

I cleaned up the output to make it clear what's going on instead of spitting out the meaningless network errors:

When it can not submit work:
[2011-07-09 17:05:11] Upstream communication failure, caching submissions

and it will continue to mine unhindered (either with fresh work it can retrieve or locally generated work) until:
[2011-07-09 17:05:26] Upstream communication resumed, submitting work         

and then all the work finished in that time will be pushed upstream. If the work is no longer valid because a new block has since appeared, cgminer knows this since it can detect a block change itself, and will not submit a whole lot of rejects and will instead say:

[2011-07-10 01:19:20] Stale work detected, discarding       

This may happen immediately after a longpoll as well, and is harmless, but minimises your reject count.

I've also updated the efficiency and utility values printed to be per-device when reporting on share submissions:

[Accepted] [GPU 1] [411.0 Mh/s] [Q:425  A:410  R:17  HW:0  E:96%  U:5.67/m]                 
[Rejected] [GPU 0] [411.1 Mh/s] [Q:425  A:384  R:24  HW:0  E:90%  U:5.31/m]                 
[Accepted] [GPU 1] [411.1 Mh/s] [Q:426  A:411  R:17  HW:0  E:96%  U:5.68/m]                 
[Accepted] [GPU 0] [410.9 Mh/s] [Q:426  A:385  R:24  HW:0  E:90%  U:5.32/m]                 
[(5s):1696.1  (avg):1643.1 Mh/s] [Q:1704  A:1538  R:81  HW:0  E:90%  U:21.25/m]         

The status line will continue to be a grand total with overall efficiency and utility.

[Tartarus]

With the current tree, under Linux, SDK 2.4, 6950 card, as soon as I hit 100 accepted blocks it stops getting new work.  Any ideas?

[-ck]

With the current tree, under Linux, SDK 2.4, 6950 card, as soon as I hit 100 accepted blocks it stops getting new work.  Any ideas?

That sounds very wrong. Can you do it while running with -D and store the log please? All of my miners are in the thousands of accepted shares.

[Nobu]

If you're getting this fairly often:

Code:

New block detected, possible missed longpoll, flushing work queue

Does that mean you should reduce the scan time? If it's not too often, is it okay to just leave it be?

I'm on slush's pool, which I think doesn't support longpolling, so seeing the message doesn't surprise me.

[-ck]

If you're getting this fairly often:

Code:

New block detected, possible missed longpoll, flushing work queue

Does that mean you should reduce the scan time? If it's not too often, is it okay to just leave it be?

I'm on slush's pool, which I think doesn't support longpolling, so seeing the message doesn't surprise me.

That's a harmless message that you need do nothing about. It means the automated new block detection which I implemented for cgminer is detecting the block before longpoll is telling it there's a new block. It also means that longpoll is activated, but auto detection is beating it. If longpoll was disabled it would have just said "New block detected, flushing work queue".

[kripz]

Updated tree:

Implemented never idle logic. During periods of network or server problems it takes existing work and generates more work from that till the server starts responding properly or fast enough. This means that hash rates should -never- drop now with cgminer.

This is it in action:

[Accepted] [GPU 2] [435.0 Mh/s] [Q:111  A:21  R:0  HW:0  E:19%  U:22.20/m]                
[2011-07-09 10:26:01] Server not providing work fast enough, generating work locally
[Accepted] [GPU 3] [429.4 Mh/s] [Q:135  A:21  R:1  HW:0  E:16%  U:18.13/m]                
[2011-07-09 10:26:53] Resumed retrieving work from server

If your network is down for extensive periods eventually this will generate more rejected blocks, but for transient blips this makes a massive difference.

Can you explain this further? So my card just hashes to keep its hash rate up? Why is this needed? I think i'd rater save power than generate rejected blocks.

[Tartarus]

With the current tree, under Linux, SDK 2.4, 6950 card, as soon as I hit 100 accepted blocks it stops getting new work.  Any ideas?

That sounds very wrong. Can you do it while running with -D and store the log please? All of my miners are in the thousands of accepted shares.

OK, I let it go with -D for almost 200 accepted and aside from some odd looking output:

Code:

[2011-07-09 22:58:09] [Rate (5s):35.5  (avg):267.47 Mhash/s] [Requested:196  Accepted:186  Rejected:8  HW errors:2048  Efficiency:95%  Utility:3.66/m]

I didn't see the problem.  But when I ran it again w/o -D, I saw it after 63.  Which got me thinking, when I reported this problem the first time a few days ago I know it was when I had a network hiccup.  So I forced a hiccup (ifdown, sleep 5, ifup), but that didn't do it.  So, I'm not sure what's up.

[-ck]

Updated tree:

Implemented never idle logic. During periods of network or server problems it takes existing work and generates more work from that till the server starts responding properly or fast enough. This means that hash rates should -never- drop now with cgminer.

This is it in action:

[Accepted] [GPU 2] [435.0 Mh/s] [Q:111  A:21  R:0  HW:0  E:19%  U:22.20/m]                
[2011-07-09 10:26:01] Server not providing work fast enough, generating work locally
[Accepted] [GPU 3] [429.4 Mh/s] [Q:135  A:21  R:1  HW:0  E:16%  U:18.13/m]                
[2011-07-09 10:26:53] Resumed retrieving work from server

If your network is down for extensive periods eventually this will generate more rejected blocks, but for transient blips this makes a massive difference.

Can you explain this further? So my card just hashes to keep its hash rate up? Why is this needed? I think i'd rater save power than generate rejected blocks.

Until the next block is solved, which happens every ten minutes on average, there is a valid way to take existing work from your pool, modify it and make it new work for your machine. The shares generated from solving that work are actually valid if returned to the server before the next block is solved. Thus it is not just consuming power, it is actually generating valid shares by deriving new work off the work received from the pool.

[-ck]

Important updated tree:

I didn't see the problem.  But when I ran it again w/o -D, I saw it after 63.  Which got me thinking, when I reported this problem the first time a few days ago I know it was when I had a network hiccup.  So I forced a hiccup (ifdown, sleep 5, ifup), but that didn't do it.  So, I'm not sure what's up.

Okay that got me thinking and I did some more instrumenting. I seem to have had a logic failure in the queueing of locally generated work, which may have caused this issue! It was also preventing local work generation ever occurring. Please update to the latest tree.

[-ck]

No wait, it's still not right. Everyone hang on while I fix it properly...

[-ck]

Updated tree:

Now I've fixed it for real, and incremented the version number and tag to v1.1.1 so people know which is the current good version. Everyone should upgrade.

[-ck]

New windows build of version 1.1.1

http://ck.kolivas.org/apps/cgminer-1.1.1-win32.zip

[-ck]

Updated tree:

You can now choose which device(s) to start cgminer on:

--device|-d <arg>   Select device to use, (Use repeat -d for multiple devices, default: all)

[Tartarus]

Updated tree:

Now I've fixed it for real, and incremented the version number and tag to v1.1.1 so people know which is the current good version. Everyone should upgrade.

Still having problems, seeing if they'll happen with -D now.

EDIT: -D and 411 requests, 352 accepts and the only odd thing is I start doing work before I have completed a getwork and these cause "HW" errors:

Code:

[2011-07-10 14:48:14] [thread 1: 134217728 hashes, 73762500 khash/sec]
[2011-07-10 14:48:15] [thread 0: 268435456 hashes, 244103467 khash/sec]
[2011-07-10 14:48:15] [thread 0: 134217728 hashes, 77439887 khash/sec]
[2011-07-10 14:48:15] GPU 0 found something?               
[2011-07-10 14:48:15] [thread 1: 134217728 hashes, 122021441 khash/sec]
[2011-07-10 14:48:15] No best_g found! Error in OpenCL code?

Trying to get the failure again without -D

EDIT2:
That was quick.  Any ideas on how to debug this?

Code:

[(5s):360.4  (avg):367.1 Mh/s] [Q:29  A:26  R:3  HW:3  E:90%  U:4.70/m]         [Accepted] [GPU 0] [369.5 Mh/s] [Q:29  A:28  R:3  HW:3  E:97%  U:5.06/m]                 
[(5s):360.4  (avg):367.1 Mh/s] [Q:29  A:26  R:3  HW:3  E:90%  U:4.70/m]         [(5s):248.1  (avg):363.8 Mh/s] [Q:30  A:28  R:3  HW:3  E:93%  U:4.97/m]         [(5s):124.3  (avg):357.4 Mh/s] [Q:30  A:28  R:3  HW:3  E:93%  U:4.88/m]         ^C5s):0.1  (avg):304.4 Mh/s] [Q:30  A:28  R:3  HW:3  E:93%  U:4.15/m] 

EDIT3: Ah-ha!  Finally: http://pastebin.com/vRQFkmRs

[Nobu]

Seems to be alternating between "Long poll received" and "New block detected" messages now; first the poll message, then the other, sometimes with a second or less between the messages. Here's a sample:

Code:

[2011-07-10 08:24:28] LONGPOLL received - new block detected and work flushed already
[2011-07-10 08:24:30] New block detected, possible missed longpoll, flushing work queue
[2011-07-10 08:27:23] LONGPOLL received - new block detected and work flushed already
[2011-07-10 08:27:36] New block detected, possible missed longpoll, flushing work queue
[2011-07-10 08:32:39] LONGPOLL received - new block detected and work flushed already
[2011-07-10 08:32:54] New block detected, possible missed longpoll, flushing work queue
[2011-07-10 08:38:41] LONGPOLL received - new block detected and work flushed already
[2011-07-10 08:39:16] New block detected, possible missed longpoll, flushing work queue
[2011-07-10 08:47:03] LONGPOLL received - new block detected and work flushed already
[2011-07-10 08:47:04] New block detected, possible missed longpoll, flushing work queue
[2011-07-10 08:56:18] LONGPOLL received - new block detected and work flushed already
[2011-07-10 08:56:19] New block detected, possible missed longpoll, flushing work queue
[Accepted] [CPU 0] [0.9 Mh/s] [Q:530  A:4  R:3  HW:0  E:1%  U:0.01/m]
[2011-07-10 09:36:24] LONGPOLL received - new block detected and work flushed already
[2011-07-10 09:36:51] New block detected, possible missed longpoll, flushing work queue
[2011-07-10 09:50:50] LONGPOLL received - new block detected and work flushed already
[2011-07-10 10:10:05] New block detected, possible missed longpoll, flushing work queue

I'm on commit 4bb13bda68fca91a8f96ec3c17cf6f99ecf70342

Edit: before 6:46 I was only getting this message: "LONGPOLL detected new block, flushing work queue", now I'm not getting it at all.

[-ck]

Updated tree: Much improved interface using ncurses. It now looks like this:

cgminer version 1.2.0
--------------------------------------------------------------------------------
Totals: [(5s):166.9  (avg):194.3 Mh/s] [Q:43  A:14  R:0  HW:2  E:33%  U:2.53/m]
--------------------------------------------------------------------------------
GPU 0: [183.5 Mh/s] [Q:15  A:14  R:0  HW:2  E:93%  U:2.57/m]
CPU 0: [3.2 Mh/s] [Q:1  A:0  R:0  HW:0  E:0%  U:0.00/m]
CPU 1: [3.2 Mh/s] [Q:1  A:0  R:0  HW:0  E:0%  U:0.00/m]
CPU 2: [3.2 Mh/s] [Q:1  A:0  R:0  HW:0  E:0%  U:0.00/m]
CPU 3: [3.2 Mh/s] [Q:5  A:0  R:0  HW:0  E:0%  U:0.00/m]
--------------------------------------------------------------------------------
[2011-07-11 13:35:41] Share accepted from GPU 0
[2011-07-11 13:36:00] Share accepted from GPU 0
[2011-07-11 13:36:37] Share accepted from GPU 0
[2011-07-11 13:36:57] Share accepted from GPU 0
[2011-07-11 13:37:06] Server not providing work fast enough, generating work locally
[2011-07-11 13:37:07] Resumed retrieving work from server
[2011-07-11 13:37:23] LONGPOLL detected new block, flushing work queue
[2011-07-11 13:37:41] Share accepted from GPU 0
[2011-07-11 13:37:43] LONGPOLL detected new block, flushing work queue
[2011-07-11 13:38:10] Share accepted from GPU 0
[2011-07-11 13:39:08] Share accepted from GPU 0

The text at the top stays there while the log messages below scroll. It now needs libcurses-dev to build. pdcurses on windows should achieve the same thing, and the windows version is currently being worked on.

[-ck]

@Tartarus: I can't really see anything obvious in that debug output. I'll have to code in some way of dumping all sorts of information when you tell it to, and hopefully I can figure it out from there.

@Nobu: Ddetecting new blocks faster than the server is sending out longpolls almost every single time is fine. However I'm not sure that's exactly what's going on in your case. I'll review the code further.

[iopq]

this is the same speed as poclbm for me, around 141Mhash/s
but every time a share is accepted, it overestimates the average even though every 5 seconds it's obvious the average is say 141.3Mhash/s when a share is accepted it will say something like 143.2Mhash/s which is probably not true

[-ck]

this is the same speed as poclbm for me, around 141Mhash/s
but every time a share is accepted, it overestimates the average even though every 5 seconds it's obvious the average is say 141.3Mhash/s when a share is accepted it will say something like 143.2Mhash/s which is probably not true

The 5 second average is quite a coarse measure and not meant to have great weight placed on it. I update it infrequently in order to minimise the overhead as much as possible and depending on when on the time boundary you land on it, it can be higher or lower than the real figure. The overall average is the only reliable number. As for overall speed, most mining software should give pretty much the same performance when running flat out since they all use the same opencl kernels. The difference between them is how they manage the rest (like intermittent network connectivity, ddosed servers, maximising accepted shares, minimising cpu and ram overhead and so on...).