I've now added testing for cl_amd_media_ops to detect which cards support BFI_INT and only patch those. This means it should work on all cards now, including nvidia and old ati that don't support it.

Thanks for starting a new topic JG. I need this project to have more exposure to attract attention to its GPU mining code so that it can be tested and I can keep improving on it. The background behind minerd is that it's built on top of Jeff Garzik's excellent cpuminer application which is programmed in (efficient, fast, lower overhead for low level stuff) c and I've added gpu mining code to it. The idea is to make an all encompassing cpu + gpu miner that performs very well, is cross platform and is being actively developed.

EDITED NOW CALLED CGMINER:
Latest source release:
http://ck.kolivas.org/apps/cgminer-1.2.3.tar.bz2

Git tree:
https://github.com/ckolivas/cgminer

Latest git source tarball:
https://github.com/ckolivas/cgminer/tarball/cgminer

Windows binary:
http://ck.kolivas.org/apps/cgminer-1.2.3-win32.zip

Summary of GUI mining features so far: Custom modified phatk and poclbm kernels, BFI_INT patching, new VECTOR code, long poll support, multi-card support while doing +/- CPU mining as well. The GPU mining code now detects the optimal parameters and capabilities for every card and sets each card separately. It has extensive failover logic to keep work going at all times and GPUs should virtually never go idle.

Initially the oclminer code was ported across to get the gpu mining working, but a lot of work has been done in gpu mining since then. I've drastically modified the way work is passed to the GPU to make it as asynchronous as possible and keep the GPU busy without making the GUI come to a standstill. Since then I've ported across the poclbm and phatk kernels, added BFI_INT patching and vectors. I've custom modified the vector code to make the most of modern pipelines and added code to detect optimal settings for each card and set them separately. Performance on my 6770 and 4x6970 is now better than any other GPU mining software.

The output looks like this:

cgminer version 1.2.0
--------------------------------------------------------------------------------
Totals: [(5s):166.9  (avg):194.3 Mh/s] [Q:43  A:14  R:0  HW:2  E:33%  U:2.53/m]
--------------------------------------------------------------------------------
GPU 0: [183.5 Mh/s] [Q:15  A:14  R:0  HW:2  E:93%  U:2.57/m]
CPU 0: [0.0 Mh/s] [Q:1  A:0  R:0  HW:0  E:0%  U:0.00/m]
CPU 1: [3.2 Mh/s] [Q:1  A:0  R:0  HW:0  E:0%  U:0.00/m]
CPU 2: [3.2 Mh/s] [Q:1  A:0  R:0  HW:0  E:0%  U:0.00/m]
CPU 3: [3.2 Mh/s] [Q:5  A:0  R:0  HW:0  E:0%  U:0.00/m]
--------------------------------------------------------------------------------
[2011-07-11 13:35:41] Share accepted from GPU 0
[2011-07-11 13:36:00] Share accepted from GPU 0
[2011-07-11 13:36:37] Share accepted from GPU 0
[2011-07-11 13:36:57] Share accepted from GPU 0
[2011-07-11 13:37:06] Server not providing work fast enough, generating work locally
[2011-07-11 13:37:07] Resumed retrieving work from server
[2011-07-11 13:37:23] LONGPOLL detected new block, flushing work queue
[2011-07-11 13:37:41] Share accepted from GPU 0
[2011-07-11 13:37:43] LONGPOLL detected new block, flushing work queue
[2011-07-11 13:38:10] Share accepted from GPU 0
[2011-07-11 13:39:08] Share accepted from GPU 0
[2011-07-11 13:39:25] Share accepted from GPU 0

Options:
--cpu-threads|-t <arg> Number of miner CPU threads (default: 0)
--debug|-D          Enable debug output
--gpu-threads|-g <arg> Number of threads per GPU (0 - 10) (default: 2)
--intensity|-I <arg> Intensity of GPU scanning (0 - 14) (default: 4)
--log|-l <arg>      Interval in seconds between log output (default: 5)
--no-longpoll       Disable X-Long-Polling support
--pass|-p <arg>     Password for bitcoin JSON-RPC server
--protocol-dump|-P  Verbose dump of protocol-level activities
--queue|-Q <arg>    Number of extra work items to queue (1 - 10) (default: 1)
--quiet|-q          Disable per-thread hashmeter output
--retries|-r <arg>  Number of times to retry before giving up, if JSON-RPC call fails (-1 means never) (default: -1)
--retry-pause|-R <arg> Number of seconds to pause, between retries (default: 5)
--scan-time|-s <arg> Upper bound on time spent scanning current work, in seconds (default: 60)
--syslog            Use system log for output messages (default: standard error)
--url|-o <arg>      URL for bitcoin JSON-RPC server (default: "http://127.0.0.1:8332/")
--user|-u <arg>     Username for bitcoin JSON-RPC server
--vectors|-v <arg>  Override detected optimal vector width (1, 2 or 4)
--verbose           Log verbose output to stderr as well as status output
--worksize|-w <arg> Override detected optimal worksize (default: 0)
--userpass|-O <arg> Username:Password pair for bitcoin JSON-RPC server
Options for command line only:
--config|-c <arg>   Load a JSON-format configuration file
See example-cfg.json for an example configuration.
--help|-h           Print this message
--ndevs|-n          Display number of detected GPUs and exit

---
Please test it and report back!

I recommend limiting intensity to no more than 8 as higher levels cause serious stalls and very little improvement (with possibly more stale shares). Some nvidia cards crash immediately on startup and that's because they report bogus values for their worksize. Try setting worksize to 128 or 256 manually if that happens.

Okay so I've spent many hours understanding GPU code and opencl code and ported stuff from all over the place and modified it to suit c and optimised it to get half decent performance out of cpuminer. So I now have a branch of cpuminer that will mine on both cpu and gpu where possible. There are no extra command line tunables at the moment, but by default it uses a modified version of the poclbm kernel, vectors, bfi_int patching and optimised worksizes for the vectors that I've tried to use the most of all pipelines. I've designed the threads that manage the GPU carefully to spend as little time wasted as possible between feeding work to the GPU, and does most of the work asynchronously. The GPU managing thread uses very little CPU of its own.

I must stress I have only just got all the features working that I wanted and have decent performance out of it now. With time I should be able to make it more configurable. For those who want to test it at this early stage, this is my development git tree and is currently usable:

https://github.com/ckolivas/cpuminer/tree/gpumine

By default it will mine on both cpu and gpu, but I've made it so you can disable cpu with --threads 0. I'm planning on hopefully merging the phatk kernel into it as well as it seems to be even better than what I currently have.

The output looks like this and is a running total of both CPU and GPU:
[2011-06-23 00:29:27] [187.15 Mhash/sec] [37 Accepted] [0 Rejected]
[2011-06-23 00:29:32] [187.05 Mhash/sec] [37 Accepted] [0 Rejected]
[2011-06-23 00:29:37] [186.94 Mhash/sec] [37 Accepted] [0 Rejected]
[2011-06-23 00:29:40] GPU 0 found something?
[2011-06-23 00:29:41] PROOF OF WORK RESULT: true (yay!!!)
[2011-06-23 00:29:42] [187.22 Mhash/sec] [38 Accepted] [0 Rejected]
[2011-06-23 00:29:47] [187.09 Mhash/sec] [38 Accepted] [0 Rejected]

It's supposed to work on all GPU cards installed at the same time, but I only have one (at the moment) to test.  Note it will ONLY work on ATI cards at the moment, and only those that support BFI_INT instructions, till I build more infrastructure into the code for the rest.

The toggle you wish to modify is this:
/sys/devices/system/cpu/cpufreq/ondemand/ignore_nice_load

Setting it to 1 will prevent CPUs from ramping up in speed when the workload is running at low priority.

Indeed the changes are unlikely to make any sort of drastic throughput improvement. The advantage of the new build is it automatically detects the number of processors and set the threads accordingly, chooses the best algorithm by default and then binds the threads to each CPU. CPU affinity does not have drastic effects on throughput unless you have a complicated cache arrangement in your hardware (such as NUMA or multiple physical CPUs) and the workload has a large cache footprint. sha256 calculations (which is all that mining is) do not have a large cache footprint. However you do realise that when it says processor 7, it means processors 0-7 which means you have 8? I didn't realise that jgarzik didn't incorporate the new total throughput counter which is in my git tree. It will allow you to really get a hold of what your throughput is rather than trying to examine each thread at a time.

Oh and the CPU affinity is disabled when the number of threads is not a multiple of the number of CPUs (8, 16, 24 etc. in your case).

I agree with ancow. The tools to do this exist already. Since cpuminer will run idle and niced priority, setting an ondemand cpu frequency governor with ignore nice load will do precisely what you want.

Yeah but you do realise that at the current level of difficulty (which will increase soon), it would take a modern cpu over 50 years to solve one block successfully? Slow it down even more and it's... never. Of course you could use them to add MH/s but even then 1GHz of modern CPU adds about 1MH/s, so a throttled 3Ghz machine would only give you just 1 MH/s per core. Which really is... nothing by today's standards and will be even less as time goes on.

Now if that doesn't deter you, then sure there are ways I can implement this crudely on linux. I haven't got a clue how to do it for other OSs, nor do I care to figure it out.

No there is not... yet(?) The threads run at ultra-low priority and should not have any impact on the machine at all except in terms of power usage and heat generation. Of course they may be issues in their own right. Is there really a demand for this?

Sweet, I appreciate the gesture and it makes me want to work on the code more, thanks!

I added some ifdefs to cope with distributions that still have older headers so if you grab my latest git tree (from the master branch) it should build for you.

Now that I've got a meaningful total throughput counter, I can confirm that running number of threads == number of logical processors on i7 is actually faster than even carefully bound number of threads == number of physical cores. This means that the default behaviour of minerd with my modifications which chooses how many threads to start up will give you the highest throughput.

My cumulative changes are here till jgarzik pulls them if anyone's interested:
https://github.com/ckolivas/cpuminer

I posted a few entries in jgarzik's cpuminer thread and even posted code that has been pulled into the cpuminer git tree. Clearly I'm no spammer so please allow me to continue posting.

That was the miner just starting up. After a while it converges more and more.

Hmm perhaps "solved" isn't quite the right word there for accepted blocks.

Hi Jeff, et al.

I've made some modifications to the output to generate a total throughput counter since there was confusion with the multiple threads issue, cleaned up the output a little, and added a solution counter. I also dropped a lot of output when only one thread is in use. Please pull the changes into your tree if you agree with the changes.

it now looks like this:

[2011-06-12 01:37:26] [Total: 8.40 Mhash/sec] [thread 3: 109989796 hashes, 3075 khash/sec] [Solved: 0]
[2011-06-12 01:37:26] PROOF OF WORK RESULT: true (yay!!!)
[2011-06-12 01:37:47] [Total: 8.45 Mhash/sec] [thread 0: 183024176 hashes, 3090 khash/sec] [Solved: 1]
[2011-06-12 01:37:48] [Total: 9.89 Mhash/sec] [thread 1: 183024176 hashes, 3085 khash/sec] [Solved: 1]
[2011-06-12 01:37:48] [Total: 11.31 Mhash/sec] [thread 2: 183024176 hashes, 3082 khash/sec] [Solved: 1]
[2011-06-12 01:38:27] [Total: 9.72 Mhash/sec] [thread 3: 183316328 hashes, 3019 khash/sec] [Solved: 1]
[2011-06-12 01:38:50] [Total: 9.54 Mhash/sec] [thread 0: 186126280 hashes, 2969 khash/sec] [Solved: 1]
[2011-06-12 01:38:50] [Total: 10.52 Mhash/sec] [thread 1: 186126280 hashes, 2989 khash/sec] [Solved: 1]
[2011-06-12 01:38:51] [Total: 11.50 Mhash/sec] [thread 2: 186126280 hashes, 3007 khash/sec] [Solved: 1]

Thanks.

Bouncing work from one CPU to another will decrease throughput a fair amount. The cost of that should not be discounted. Anyway feel free to try...

Actually that's not quite true. There was one more complex setup that produced higher throughput. If I set the number of mining threads to the total number of cores only, and then bound each worker thread to all the logical CPUs that shared cache, the throughput was a bit better again. However which cores shares threads is even harder to detect reliably without digging around in /sys and the format has changed between kernels. Furthermore, on my i7, the shared caches aren't even sequential numbers so binding threads to sequential logical CPUs was worse (i.e. CPUs 0 and 2 shared caches and 1 and 3 and so on).

I tested total threads vs total cores and got slightly more with total threads on i7. Plus there is no particularly easy and reliable way to detect cores versus threads. So total "processors" actually generated more in my testing.

Is this with the version I posted? The CPU affinity that I added can have quite strong effects on certain hardware and kernels (comparing different linux kernels and windows especially) if the kernels aren't particularly good at soft affinity effects. Having spent most of my spare time coding CPU schedulers I obsess about these things...

Yeah when I said your own version, I hadn't really envisioned that you'd programmed your own jansson library

Glad to see you got it going. Now see how you go