New version: 2.11.0, 2nd March 2013 Happy birthday to me

New unstable version preparing for ASIC hardware. This version is being released to fix any bugs found in the USB and hotplug code prior to it being needed by real ASIC hardware. I had one last go at improving the OpenCL kernels for GPU mining since this code will soon become of historical interest only except in the case of scrypt.

IF YOU ARE ON WINDOWS YOU NEED A NEW AND DIFFERENT USB DRIVER
From FPGA readme:
Human readable changelog:

- Huge updates to USB code including direct USB and hotplug changes by Kano, including updates to various FPGA driver modules to suit the new code.
- New OpenCL GPU kernels for poclbm, diablo and scrypt! First speed up in a long time (but absolutely miniscule). As always, any faster kernel tends to bring out hardware instability if you've overclocked your GPUs to within an inch of their stability.
- Resume support for stratum. Currently only one pool supports it but hopefully others will follow suit.
- Try to not lose stratum shares across pool switches.
- Fix for scrypt kernels with thread concurrency set to >9999
- Lots of minor bugfixes and improvements.
- Lots of infrastructure added to allow for queueing devices in the future (i.e. BFL asics if/when they appear) but not currently used.


Full changelog:

- Update kernel file names signifying changes.
- Update a pool's last work time when the work is popped as well as staged.
- API always report failed send() replies
- Update diff stale: total and pools when stratum throws away shares
- Keep stratum connections open for 2 minutes after the last work item was
staged to allow stray shares to be submitted on pool switching.
- Try to extract the sessionid associated with mining.notify on 3rd level array
and submit it along with the userid to support mining resume, failing gracefully
and restarting if the pool rejects it.
- Speed up watchdog interval and therefore display updates to 2 seconds.
- Update copyright dates.
- Cope with misread sessionid on stratum for now.
- Use constants from the array of __constants throughout the diablo kernel.
- Create a __constant array for use within diablo kernel.
- Fix --benchmark generating valid work for cgminer.
- Use the sessionid as passed on stratum connect to attempt to resume a
connection once and then clear it if it fails, to use a new connection.
- Move to storing the nonce1 in the work struct instead of the sessionid for the
now defunct first draft mining.resume protocol.
- Use global constant arrays for all other constants used in scrypt kernel.
- Use global __constants for sha functions in scrypt kernel.
- Use constants for endian swap macros.
- Revise scrypt kernel copyright notice.
- Separate out additions in scrypt kernel.
- Reuse some Vals[] variables that can be assigned to constants earlier in the
poclbm kernel, making for fewer ops.
- Put all constants used in poclbm kernel into __const memory array to speed up
concurrent reads on the wavefront.
- BFL stop 1st init command if no device
- Add a get_queued function for devices to use to retrieve work items from the
queued hashtable.
- Bugfix: Duplicate stratum sessionid when copying work, to avoid double-free
- Bugfix: Missing pool_no parameter to applog for no-stratum-sessionid debug
message
- Add the choice of hash loop to the device driver, defaulting to hash_sole_work
if none is specified.
- Add comments.
- Add a driver specific flush_work for queued devices that may have work items
already queued to abort working on them on the device and discard them.
- Flush queued work on a restart from the hash database and discard the work
structs.
- Create a central point for removal of work items completed by queued device
drivers.
- Create a fill_queue function that creates hashtables of as many work items as
is required by the device driver till it flags the queue full.
- Create the hash queued work variant for use with devices that are fast enough
to require a queue.
- Update copyright year.
- Fix tv_lastupdate being made into tv_end and update the hashmeter on cycle,
not opt_log_interval.
- Fix tv_lastupdate being made into tv_end and update the hashmeter on cycle,
not opt_log_interval.
- Only continue submitting shares with mining.resume support on stratum when the
session id matches.
- Provide support for mining.resume with stratum, currently re-authorising after
successful resumption pending finalising of the protocol process.
- Provide basic framework for restarting stratum depending on whether resume
support exists or not.
- Abstract out the setting up of the stratum curl socket.
- Free sessionid in clean_work and remove redundant setting of strings to NULL
since the whole work struct is zeroed.
- Only clear stratum shares mandatorily on stratum dropouts when the pool does
not support resume.
- Try resubmitting stratum shares every 5 seconds for up to 2 minutes if the
pool session id exists and matches on failure to submit.
- Do as much outside of mutex locking of sshare_lock as possible.
- Remove last reference to struct work used outside the sshare_lock in
submit_work_thread
- Unlock the sshare_lock in submit_work_thread when all references to work and
sshare are complete.
- Add timestamps to stratum_share structs as they're generated and copy the
stratum sessionid if it exists to stratum work generated.
- Store session id for stratum if the pool supports it for future mining.resume
support.
- API.java allow partial reads
- debug_cb buffer type warning
- MMQ rewrite the last of the old scanhash loop and drastically reduce CPU
- hash_sole_work can be static
- Make the numbuf larger to accept larger scrypt parameters.
- Keep the unique id of each work item across copy_work to prevent multiple work
items having the same id.
- Abstract out the main hashing loop to allow us to use a separate loop for
devices that are fast enough to require queued work.
- Provide a noop thread_enable function for drivers that don't support it.
- Provide a noop thread_shutdown function for drivers that don't support it.
- Provide a noop hw_error function for drivers that don't support it.
- Provide a noop prepare_work for drivers that don't support it.
- Provide a noop thread_init for drivers that don't support it.
- Provide a noop can_limit_work for devices that don't support it.
- Provide a noop thread_prepare function for drivers that don't use
thread_prepare.
- Use blank_get_statline_before for GPU devices that don't support adl
monitoring.
- Provide a noop get_stats function for drivers that don't support it.
- Provide a blank get_statline for drivers that don't support it.
- Provide a blank get_statline_before function for drivers that don't have one.
- Fill drivers missing reinit_device with a noop version.
- add 'count' to cumstomsummarypage 'calc'
- hotplug use get_thread() where appropriate
- convert sleep(const) to nmsleep()
- remove empty #ifdef
- call a separate get_devices() with locking, as required
- usbutils - avoid free cgusb twice
- usbutils hotplug v0.1
- Report USB nodev as ZOMBIE on the screen
- Change file modes.

That's cause it's checking all 3 protocols and has to give all of them a reasonable timeout duration... Kind of a catch 22 problem. I want to shorten it next version but some pools really need up to 60 seconds to connect.

Minirigs only, yes, and the first generation MRs didn't even support it.

On the other hand, it will be totally irrelevant with ASIC hardware since it hashes so fast there is no need to cut work up into smaller chunks.

It's not enabled...

Try reconfiguring Xorg
sudo aticonfig --adapter=all -f --initial

I knew the GPUs had the capability, it's just that Intel hadn't created an openCL SDK for them till very recently, and as you can see it's still listed as beta.

4-6 weeks

Avalon is almost trivial to set up. Initially you need a computer to set it up but after that, even the computer is not needed. But... the firmware that is currently installed on it is still not stable with the few Avalon owners out there forced to regularly reboot it when it just stops. Of course with time this will be fixed, but as trivial as it is to set up, it's not a set-up-and-forget affair just yet...

When the BFL hardware eventually comes out, we plan to work very hard to make cgminer work very well on it and become reliable software to use quickly and to run stable for very long periods without intervention. However it will require a separate PC/laptop/tablet/phone or something to actually keep running it.

I take it back, having looked at the intel websHite again, http://software.intel.com/en-us/vcsource/tools/opencl-sdk says "OpenCL 1.2
(OpenCL 1.2 on CPU mixed with OpenCL 1.1 on Intel® HD Graphics) (Version 2013 Beta)" which is new to me.

Queued work and efficiency mean pretty much nothing now, especially with stratum. It's simply a function of how often the pool is sending you a new template and there is no "standard" for that, some do 30 seconds, some 60 etc. The more often they're sending you a template, then potentially the more transactions you're supporting if the pool is adding all the latest transactions every 30 seconds instead of 60, but even that equation isn't clear with bitcoind not necessarily simply taking all transactions. The work queueing in cgminer is a fairly complex machine designed for optimally keeping devices as busy as possible, and trying to balance things between pools for whatever reason is a distant second priority. Trying to improve the latter will come at the price of the former. This will become far more acute with ASIC hashrates than it is at the moment with GPUs, and it clearly works with avalon running ~66 or 88 GH off a very low power CPU with cgminer. I doubt the equivalent of the CPU hardware in the Avalon could handle a 1.5TH minirig when they come out but a regular desktop/laptop/tablet CPU will easily do so, and cgminer is heavily multithreaded so multicore/mulithread CPUs (which are everywhere in these devices) are advantageous.

OpenCL still does not work on the integrated GPU on Intel CPUs.

Intrinsically stratum is inherently faster and lower overhead than longpoll for both the server and the client. Minerd is fast. Cgminer is fast. Stratum is fast. On the other hand I can't speak for any other implementation at the other end.

No, but it might take 4-6 weeks.

When spreading shares out to multiple pools it is guaranteed your reject rate will be higher because every pool has a slightly different time they see the block change. How to best manage that is a point of contention. It was far more useful for shares to "leak" with the old getwork protocol because it would be common one pool could not keep up the work for all miners especially across longpoll. I do recall one lucky user in this thread had shares leak to his local bitcoind and actually found a block with one of those! Since stratum (and gbt) use completely local work generation, there is not much point to this any more.

Weird. I don't change voltage on any of mine on linux and just set the engine clock to whatever I want just from cgminer. Overvolting is about the only kind of overclocking which is dangerous and hugely wasteful powerwise. Some were stable at 1145 while one was stable at 1200.

Indeed you're right, stratum alone is not enough, as higher diffs are required, but I tend to only think of 5-6 pools as being significant and perhaps I was just looking for an opportunity to point out that deepbit was not one of them.

Always a fan of a really nice looking rack.

I think submitting a new extranonce1 is all that is required. We should view the sessionid submission as a hint only and let the pool decide what to do. You will indeed need unique session IDs for this support to work. When luke-Jr tested it, he found that the old connection was not actually detached before I was requesting a new one with the sessionID. This is not surprising given how difficult it can be to tell that a raw socket is actually broken or not. I suspect that dropping the first connection there is the answer.

Pretty much every pool except for deepbit is ready for ASIC hashrates.

My 4x7970 rig had a 1250W OCZ Gold PSU. It uses ~1150 at the wall when maxed out hot fans high overclocked and no real power saving option is in use. Bear in mind it had a couple of hard drives and the case has lots of fans too.

Ok for those not watching the discussion on #stratum on IRC, I believe we have a final protocol for mining resume which should not break any clients or pools.

It was decided that the parameters could include both the user agent and the session ID as the first and second parameters. If this fails cgminer/other clients should resort to sending blank set of parameters as previously.

The session Id, it was decided, would be encoded along with the notification message "mining.notify" as a member of the 3 deep array parameters returned for "result" by the pool.

See the following for an example of an initial connect followed by a reconnect.
Note the session ID has changed between the initial subscription and the resume, BUT the extranonce1 has remained the same. This tells the miner it can (re)submit shares before the disconnection.

Hopefully we will not need to revise this any further.