OFFICIAL CGMINER mining software thread for linux/win/osx/mips/arm/r-pi 4.11.1

[kano]

...
Thank you. That was informative, but I'm not on windows and my hashrate estimate is clearly way off. When running with --benchmark all devices report a hashrate close to 380MH/s, so there is nothing wrong with the devices. Furthermore the utility is more or less constant at all settings (except with really low abort time, but that is expected). It is true that it takes a long time to get a good estimate of utility, but the rough neighbouhood can be achieved in an hour or so, especially when averaging over 10 devices.

So my conclusion is that the estimation has a bug and it doesn't do what it (and you) says it does. I guess I have to try to decipher the source to try to find it.

No, an hour is not long enough to get a reliable estimate for U:
An hour is probably long enough to get an estimate that is within 10%
10% of 5.3 is of course 0.53 so it could read as high 5.83 or as low as 4.77 after an hour and not be unexpected.
Even averaging over 10 devices is the equivalent of only running for 10 hours.
As I said, you'd need a few days running to START to converge on the expected value for U:

That's random probability - nothing do with code.

Easiest way to show this is to look at my 6950 GPU.
After 28hrs runtime it reads 365.96MH/s but has a U: of 5.00 - that's out by 2%
The GPU code counts all hashes exactly - no calculations involved - just simply adding them up.
Of course that 2% could be higher and I'd not consider that unexpected either.

Anyway, if you want to see exactly what's happening, then in the cgminer screen press 'd' then 'd' and look for the all "Icarus" messages.
It will tell you all the timing calculations it is doing.

However, as I said before, the timing of the termios must be correct (that includes the computer clock and of course you should be using ntp - and ntp can also tell you how good or bad your clock is)

Edit: since you mentioned p2pool before, then also taking the small effect of 10s LP's into consideration:
Each LP loses 0.1s of processing time - i.e. 1% - so you will certainly expect to lose 3 or 4 MH/s from that also.

Edit2: though I should have said this at the start - you can easily determine if it is somehow p2pool related by trying it on a normal pool

[1713982027]

1713982027

[tucenaber]

Now I'm starting to doubt your reading capabilities...

I said a "rough" estimate of U, and the utility is not the issue here. My actual hashrate is more or less what it shoud be, both according to "utility" and to p2pool stats. There.

The issue is the cgminer hashrate estimate which is less than 60% of what it should be.

I also rule out timing issues and termios because it is fully capable to get it right when benchmarking.

[kano]

... press 'd' and 'd' ...

[bitcoindaddy]

Con, have you ever considered signing your cgminer binary releases with gpg using one of your gpg keys or a new one?

http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x65483BFADA7A9915

[somenick]

cgminer has stale x2 greater than DiabloMiner
on deepbit
(0.24%)   Prop.
(0.21%)   Prop.
(0.19%)   Prop.
(0.23%)   Prop.
(0.11%)   Prop.  - DiabloMiner
(0.20%)   Prop.
(0.20%)   Prop.
(0.23%)   Prop.
(0.22%)   Prop.
(0.24%)   Prop.
(0.37%)   Prop.
(0.22%)   Prop.

[Luke-Jr]

Wrong.

...

Adjusting the abort time will not change the Hash rate unless there is something wrong either with the device or the computer's termios timing

If you are using 6.5 seconds to abandon jobs then you are aborting work too early.
However the reason you have to do that may be because you are using windows and the termios timing may not be accurate.

Well, in my setup, on Windows 7, abort time changes the hash rate.  the utility does not change much,  but hash rate is very sensitive to abort time.

Abort time lower than 8 seconds (80), increases the hash rate, higher than 8 secs (80) lowers the hash rate.

BTW, I've played with many timings, and 6.5 seconds abort time produce best utility rate.  I don't care about the hash rate reported by the cgminer. Number of submitted, valid shares per minute is what makes money.  So who cares if cgminer is reporting 410 or 210 Mh/s.  If utility is 5.35, I'm a  happy camper.

On Windows 7 (64bit), the new icarus code barely gets utility of 5. 
Kano, maybe you optimized too much for Unix and Windows performance took a hit.

It takes about 3 days to get Utility precision to even 1 decimal place on a single Icarus. I've seen both CGMiner and BFGMiner 2.4.4 keep well over 5 Utility on average (and I'm pretty sure there weren't any changes to this in 2.5.0)

[kentrolla]

Hey people before now I've been using Kiv's GUIminer but I got some brand new BFL singles that I need to use and I think cgminer is the most widely used and has bitforce enabled but I keep getting an error that I can't seem to find documented anywhere idk but here's the issue.  When I try to open this "cgminer.exe" it opens for a second or so then shuts down doesn't mine or anything just quits.  Also when I tried to build cgminer using the instructions in the "windows-build.txt" I get to the last step before running the "make" command in mingw the "CFLAGS="-02 -msse2" ./configure" command and I get and error message that I can't understand:

Code:

$ CFLAGS="-02 -msse2" ./configure
checking build system type... i686-pc-mingw32
checking host system type... i686-pc-mingw32
checking target system type... i686-pc-mingw32
checking for a BSD-compatible install... /bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... no
configure: error: in `/home/XXXXX/cgminer-2.4.2-win32':
configure: error: C compiler cannot create executables
See `config.log' for more details

and when I open the config.log it displays this:

Code:

$ config.log
./config.log: line 1: This: command not found
./config.log: line 2: running: command not found
./config.log: line 4: It: command not found
./config.log: line 5: generated: command not found
./config.log: line 7: $: command not found
hostname: extra operand `XXXXX-PC'
Try `hostname --help' for more information.
uname: extra operand `='
Try `uname --help' for more information.
./config.log: line 15: syntax error near unexpected token `('
./config.log: line 15: `uname -r = 1.0.17(0.48/3/2)'

someone else when I made my own thread in the newb boards directed me to something he's written for ozcoin which is https://ozcoin.net/content/win7-cgminer-setup-guide.  I've tried that and it works but I get the same instant shutdown thing.  If someone could help me build and run this thing it would be much appreciated or if someone knows that the fix for my problems have been mentioned earlier in this or another thread it would also be greatly appreciated if you could tell me which thread so I can go and read through it and not trouble you guys any further. 

its -O2 not -02      it's the Letter O not the number zero

[kano]

Now I'm starting to doubt your reading capabilities...

I said a "rough" estimate of U, and the utility is not the issue here. My actual hashrate is more or less what it shoud be, both according to "utility" and to p2pool stats. There.

The issue is the cgminer hashrate estimate which is less than 60% of what it should be.

I also rule out timing issues and termios because it is fully capable to get it right when benchmarking.

The benchmark only tests with a single pre-defined getwork repeatedly.
That's not a test of both cases on the Icarus.
As I said above, the Icarus has 2 cases
1) Find a share
2) Abort before idle

Anyway, what did debug tell you?
It should hopefully make it quite clear what is going on.

[tucenaber]

The benchmark only tests with a single pre-defined getwork repeatedly.
That's not a test of both cases on the Icarus.
As I said above, the Icarus has 2 cases
1) Find a share
2) Abort before idle

Anyway, what did debug tell you?
It should hopefully make it quite clear what is going on.

The reason I mentioned benchmarking was beacause you started to talk about timing issues and misbehaving hardware, so I pointed out that benchmarking should then show that. But it doesn't show that. Do you understand?

As you point out there are two cases, and only one is involved in benchmarking. From that I conclude that the problem is lurking in the second case.

I have also concluded that it is probably related to the short expiration time on p2pool, because otherwise everybody with an Icarus would experience the issue, and it doesn't seem to be that way.

The debugging info told me that the number of hashes is written in hex  Apart from that not so much (yet). Looking at the source haven't gotten me far either.

I also have another more serious problem, that may or may not be related and about which I have been posting in the p2pool thread. The problem is that I get lots (like 7%) of duplicate shares from cgminer. It turns out that some of these are rolled past the expiry (=10s) by cgminer and I don't understand that issue either

[Askit2]

Not to argue with anyone but my hashrate took 3 days to get to my expected value. It had kinda wild swings for 2 days for sure. The near instant hashrate reported what I expected. 860Mh but the average at 10 mins was 600Mh, after a day it got up to aound 800 and now it is finally nearly 844. It should hopefully land on 848 or so after long enough but above average LP could keep it down. Maybe let it run since the reported rate seems right for P2Pool and see if it evens out?

[tucenaber]

I have been running it for weeks. That is also not the issue.

[kano]

The benchmark only tests with a single pre-defined getwork repeatedly.
That's not a test of both cases on the Icarus.
As I said above, the Icarus has 2 cases
1) Find a share
2) Abort before idle

Anyway, what did debug tell you?
It should hopefully make it quite clear what is going on.

The reason I mentioned benchmarking was beacause you started to talk about timing issues and misbehaving hardware, so I pointed out that benchmarking should then show that. But it doesn't show that. Do you understand?

As you point out there are two cases, and only one is involved in benchmarking. From that I conclude that the problem is lurking in the second case.

I have also concluded that it is probably related to the short expiration time on p2pool, because otherwise everybody with an Icarus would experience the issue, and it doesn't seem to be that way.

The debugging info told me that the number of hashes is written in hex  Apart from that not so much (yet). Looking at the source haven't gotten me far either.

I also have another more serious problem, that may or may not be related and about which I have been posting in the p2pool thread. The problem is that I get lots (like 7%) of duplicate shares from cgminer. It turns out that some of these are rolled past the expiry (=10s) by cgminer and I don't understand that issue either

For the p2pool issue, my "guess" would be that p2pool might not be specifying the expiry time properly?
If you turn on protocol debug it will tell you what p2pool is telling you to do with rolled shares and expiry.

If there is an issue with the rolltime/expiry supplied by the pool, cgminer will use --expiry and --scan-time to work out what to do.

As for Icarus - in debug mode it reports the result of each piece of work it does and how long it took to run.
That should make it clear what is going on.

The 2 cases give debug:
1) Icarus %d nonce = 0x%08x = 0x%08llx hashes (%ld.%06lds)
2) Icarus %d no nonce = 0x%08llx hashes (%ld.%06lds)

And also for case 2) you should get a line before it saying:
Icarus Read: No data in %.2f seconds

Since you compiled cgminer yourself, also try using the binary release to see if that makes any difference (probably not though)

[bronan]

I have not updated cgminer since version 2.4.3 on my win7 x64 box with dual 7970 gpu's running with catalyst 12.6

So when i saw you have released 2 newer versions i firstly installed the 2.5.0 version and restarted my batch with the usual config file

Sadly i see that mining does not start on my pool
Instead i see it reporting that the pool is not providing work fast enough and switches to another pool

Another thing what instant caught my attention is the extreme low mining performance (i run my card on low clocks and temp (750 mhz core  / 800 memory speed))

So i downloaded version 2.4.4 to see if that would run normal, but here again same issue.

With the 2.4.3 i get a average mining speed of 850 mh but with the newer versions i get an enormous drop in speed and again the failure to run at the pool i wish as my primary pool

When i use the newer versions i get with both cards a mining speed between  500 and 620 mh ... my question is why or did i make a mistake in the config

here is the parameter part of my config file

"intensity" : "7",
"gpu-engine" : "750",
"gpu-memclock" : "800",
"temp-overheat" : "85",
"temp-cutoff" : "95",
"temp-target" : "75",
"failover-only" : true,
"gpu-threads" : "2",
"retry-pause" : "5",
"scan-time" : "60",
"worksize" : "256",
"expiry" : "120",
"vectors" : "2",
"algo" : "auto",
"queue" : "1",
"log" : "5"
}

[tucenaber]

For the p2pool issue, my "guess" would be that p2pool might not be specifying the expiry time properly?
If you turn on protocol debug it will tell you what p2pool is telling you to do with rolled shares and expiry.

If there is an issue with the rolltime/expiry supplied by the pool, cgminer will use --expiry and --scan-time to work out what to do.

As for Icarus - in debug mode it reports the result of each piece of work it does and how long it took to run.
That should make it clear what is going on.

The 2 cases give debug:
1) Icarus %d nonce = 0x%08x = 0x%08llx hashes (%ld.%06lds)
2) Icarus %d no nonce = 0x%08llx hashes (%ld.%06lds)

And also for case 2) you should get a line before it saying:
Icarus Read: No data in %.2f seconds

Since you compiled cgminer yourself, also try using the binary release to see if that makes any difference (probably not though)

Well, taking a look at the log I find this:

Code:

[2012-07-09 18:10:37] Icarus Read: No data in 10.00 seconds
[2012-07-09 18:10:37] Icarus 3 no nonce = 0xe280310f hashes (10.000137s)
[2012-07-09 18:10:37] [thread 7: 3800051983 hashes, 283580784 khash/sec]

0xe280310f=3800051983 and 3800051983/10.000137=379999992.3. Not 283580784! Perhaps there is a simple resoluton after all.
(BTW, it says khash/sec but clearly it should say hash/sec)
[edit]no... i was wrong again. the first calculation comes directly from Hs so it will be 380MH/s by definition. the second comes from hashmeter and is measured by real elapsed time. damn.[/edit]

Regarding roll-time, the problem is not so easy as you suggest. P2pool sets the following headers:

Code:

request.setHeader('X-Long-Polling', '/long-polling')
request.setHeader('X-Roll-NTime', 'expire=10')
request.setHeader('X-Is-P2Pool', 'true')

and the protocol debug gives:

Code:

[2012-07-10 00:54:25] HTTP hdr(X-Roll-Ntime): expire=10
[2012-07-10 00:54:25] X-Roll-Ntime expiry set to 10
[2012-07-10 00:54:25] HTTP hdr(X-Long-Polling): /long-polling

Surely, scan-time is overridden by expire?

About the duplicates. Until today I have only been able to see that a lot of duplicate hashes are submitted but now forrestv has written some code to check the timestamp (ntime) on each submitted share and write a log message if it has been increased by more than 10 seconds. On my machine it spits out error messages like this every few seconds:

Code:

2012-07-10 01:34:45.211197 > Miner digger @ 192.168.1.102 rolled timestamp improperly! This may be a bug in the miner that is causing you to lose work!

[kano]

...
Well, taking a look at the log I find this:

Code:

[2012-07-09 18:10:37] Icarus Read: No data in 10.00 seconds
[2012-07-09 18:10:37] Icarus 3 no nonce = 0xe280310f hashes (10.000137s)
[2012-07-09 18:10:37] [thread 7: 3800051983 hashes, 283580784 khash/sec]

0xe280310f=3800051983 and 3800051983/10.000137=379999992.3. Not 283580784! Perhaps there is a simple resoluton after all.
(BTW, it says khash/sec but clearly it should say hash/sec)
[edit]no... i was wrong again. the first calculation comes directly from Hs so it will be 380MH/s by definition. the second comes from hashmeter and is measured by real elapsed time. damn.[/edit]

The 10.00 seconds is based on the counter, not the elapsed time.
(this is the issue I was referring to that may need changing to deal with crappy timers on some OS's if they exist?)

The 10.000137s is elapsed time from the send work to when the abort was performed.
Thus the timer on your OS is fine for that one since they match.

You will also see a "Icarus %d sent: %s" 10seconds before at ~2012-07-09 18:10:27

The hashmeter times things differently so you cannot assume it will calculate over the same time and hash count.
Sometimes it will be lower and sometimes it will be higher - due to the different start/finish times of the calculation.
This is only with Icarus because Icarus does not complete the full nonce range in either case:
1) When it finds a share it aborts - random processing time
2) When it gets close to the end of the nonce range it aborts - in your case 10s processing time
Due to this the hash meter will of course go up and down.

... also note that work returned after an LP is not counted in the hashmeter ...

[tucenaber]

Yeah, that was basically what I wanted to say with my edit...

... also note that work returned after an LP is not counted in the hashmeter ...

Ok, since there is a long-poll before 10s 50% of the time (right?) that might explain it. Anyway it's not that important. There does not seem to be any lost work.

The roll-time issue on the other hand worries me. How can it be that expire is ignored and also what could possibly be the reason for resubmitting old hashes?

[tucenaber]

Sorry for spamming but this is what I've been talking about:

Code:

 [2012-07-09 18:10:45] Icarus 8 nonce = 0xc24d6586 = 0x849acb0e hashes (5.860506s)
 [2012-07-09 18:10:45] [thread 12: 3017971390 hashes, 379385971 khash/sec]
 [2012-07-09 18:10:45] Popping work from get queue to get work
 [2012-07-09 18:10:45] Pushing rolled converted work to stage thread
 [2012-07-09 18:10:45] Pushing work to stage thread
 [2012-07-09 18:10:45] Successfully rolled work
 [2012-07-09 18:10:45] Successfully rolled work
 [2012-07-09 18:10:45] Pushing work to stage thread
 [2012-07-09 18:10:45] Icarus 8 sent: b586fa1de5cdd76ea88f9e78c52694d0ee9b61a4f56e10e4f22c6d325e1c207d00000000000000000000000000000000000000003194091ae002fb4f58386ba3
 [2012-07-09 18:10:45] Popping work to work thread
 [2012-07-09 18:10:45] Pushing work to getwork queue
 [2012-07-09 18:10:45] Popping work to stage thread
 [2012-07-09 18:10:45] Pushing work to getwork queue
 [2012-07-09 18:10:45] Popping work to stage thread
 [2012-07-09 18:10:45] Creating extra submit work thread
 [2012-07-09 18:10:45] DBG: sending http://hack:9332 submit RPC call: {"method": "getwork", "params": [ "00000001de2f42fb1d6b7ca7a2a0b65a8d1dab19ce61c6abe2bcfd57000004540000000094e6587a379cef8109fb5cdf05540d9bf3e9803830ca439defe9a1cea36b38584ffb02bc1a09943186654dc2000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000" ], "id":1}
 [2012-07-09 18:10:45] X-Roll-Ntime expiry set to 10
 [2012-07-09 18:10:45] PROOF OF WORK RESULT: true (yay!!!)
 [2012-07-09 18:10:45] Accepted 445d545e.e713b642 ICA 8 pool 0
 [2012-07-09 18:10:45] ICA8                | (5s):312.4 (avg):227.6 Mh/s | A:649 R:9 HW:0 U:5.5/m
 [2012-07-09 18:10:45]  Proof: 00000000686e0e540ba19b828314dcf41173d03b9eda020be41ceedc093dcb25
Target: 00000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffff
TrgVal? YES (hash < target)
 [2012-07-09 18:10:45] Pushing submit work to work thread
 [2012-07-09 18:10:45] Icarus 5 nonce = 0x5c92eea3 = 0xb925dd48 hashes (8.180717s)
 [2012-07-09 18:10:45] [thread 9: 3106266440 hashes, 379680133 khash/sec]
 [2012-07-09 18:10:45] Popping work from get queue to get work
 [2012-07-09 18:10:45] Icarus 5 sent: b586fa1de5cdd76ea88f9e78c52694d0ee9b61a4f56e10e4f22c6d325e1c207d00000000000000000000000000000000000000003194091adf02fb4f58386ba3
 [2012-07-09 18:10:45] Popping work to work thread
 [2012-07-09 18:10:45] Creating extra submit work thread
 [2012-07-09 18:10:45] DBG: sending http://hack:9332 submit RPC call: {"method": "getwork", "params": [ "00000001de2f42fb1d6b7ca7a2a0b65a8d1dab19ce61c6abe2bcfd57000004540000000094e6587a379cef8109fb5cdf05540d9bf3e9803830ca439defe9a1cea36b38584ffb02981a099431a3ee925c000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000" ], "id":1}

First "Icarus 8" sends something and then "Icarus 5" sends the exact same thing, except that the nonces are different. They are very close in time so in this case it might be that no new work has been received yet, but it happens a lot. As I said earlier 7% of my shares are duplicates.

[-ck]

Expire was added in cgminer 2.4.4

Expire=10 was a bug in much pool software. It did not make sense at all to have that value for anything but p2pool so the scantime is used if it is longer than expire (expire should be more like 120 seconds). This can't be creating duplicates though.

p2pool has a recent bug where duplicate work items are sent out regardless of the mining software. Anyone on p2pool?

Lots going on folks.... There is always more than meets the eye.

[kano]

...

They are different:
[2012-07-09 18:10:45] Icarus 8 sent: b586fa1de5c...c6d325e1c207d00...003194091ae002fb4f58386ba3
[2012-07-09 18:10:45] DBG: sending http://hack:9332 submit RPC call: {...9803830ca439defe9a1cea36b38584ffb02bc1a09943186654dc20000008...
[2012-07-09 18:10:45] Icarus 5 sent: b586fa1de5c...c6d325e1c207d00...003194091adf02fb4f58386ba3
[2012-07-09 18:10:45] DBG: sending http://hack:9332 submit RPC call: {...9803830ca439defe9a1cea36b38584ffb02981a099431a3ee925c0000008...

If you look at the proof of the second one it will be different of course also
When time is rolled, the data is only very slightly different of course ...

i.e. they are not duplicate shares in the above example

[DiabloD3]

cgminer has stale x2 greater than DiabloMiner
on deepbit
(0.24%)   Prop.
(0.21%)   Prop.
(0.19%)   Prop.
(0.23%)   Prop.
(0.11%)   Prop.  - DiabloMiner
(0.20%)   Prop.
(0.20%)   Prop.
(0.23%)   Prop.
(0.22%)   Prop.
(0.24%)   Prop.
(0.37%)   Prop.
(0.22%)   Prop.

Huh, thats weird. cgminer uses pretty similar code to what I do.