(OLD) BFGMiner: modular FPGA/GPU, GBT, Stratum, RPC, Avalon/Lnx/OpnWrt/PPA/W64

[Luke-Jr]

There are a few issues with the Ztex driver, and since it seems Nelisky has disappeared, I think it would be a good idea to order one of each for myself so that I can maintain it properly.

If anyone with Ztex boards would like to donate toward this goal, I have setup an address: 1HcJMz6iTMcBS5NxzRqM5UpTAnC9fmuQqs

Regardless of whether you donate personally, please do report any issues you have with BFGMiner or any of its drivers.

Disclosure: ZTEX GmbH offered a generous discount for development use, and I have already ordered the Ztex boards on my own. Donations are to help cover the direct cost to me, since this is mainly for others' benefit.

[Luke-Jr]

NEW VERSION 2.6.5, AUGUST 20 2012

I expect to follow-up with 2.7.0 soon, but first here is a 2.6.5 with just the bugfixes and less risky changes.

Human readable changelog:

• New BitFORCE firmware flash utility for Linux (only). When compiled with BitFORCE support, run bitforce-firmware-flash to get usage.
• Fixed hanging when using "Switch User" on Windows. This feature still kills ADL, though, so use --no-restart (and don't enable fan control or overclocking) if you plan to use it.
• Fixed dynamic mode on Windows. By default, Windows timer resolution is only 15ms and we're trying to sample much smaller than that. This was leading to the time taken to do GPU work appearing as zero for many samples. Now, a Windows multimedia timing API will be used to request more precision.
• gpu-memdiff should now take effect when you change gpu engine clock from the menu as well.
• The ADL gpu-map feature should work now when you have more ADL devices than OpenCL (eg, ATI cards that don't support OpenCL).
• More tweaks to the queueing mechanism to increase efficiency and keep minirigs fully work laden.
• Failover-only can now be changed via the RPC API.
• Updated miner.php from Kano

Full changelog

• API new command 'coin' with mining information
• Add message to share if it's a resubmit.
• Add virtual adl mapping for when none is specified on the command line to not crash without a map specified.
• Fix ADL gpu-map not working when there are more ADL devices than openCL. Patch supplied and tested by Nite69.
• bitforce: Initial import of Linux-only bitforce-firmware-flash utility
• Revert stale-on-arrival failsafe, since it ends up needing exceptions for everything
• Bugfix: opencl: Declare opencl_dynamic_cleanup in header
• Even if we want to submit stale shares, give up if we have more submissions waiting on threads (even before failing)
• Even if we want to submit stale shares, give up if they've failed and we have more submissions waiting on threads
• opencl: Use timeBeginPeriod on Windows to ensure gettimeofday has sufficient precision for dynamic intensity
• Bugfix: opencl: Move ADL fanspeed warning messages to a new thread to get around summary-update deadlocking
• README: Note that user groups don't get updated until re-login
• Initialise cnt in libztex.c
• Don't try to start devices that don't support scrypt when scrypt mining.
• Repeating on timeout in ztex could make the code never return.
• Offset libusb reads/writes by length written as well in ztex.
• Cope with timeouts and partial reads in ztex code.
• If there are more devices than nDevs, don't iterate over them as they may overwrite devices mapped below that with the mapping option.
• Fix README faq on bfl auto-detect.
• Set memory clock based on memdiff if present from with engine changes, allowing it to parallel manual changes from the menu as well.
• api.c typo
• API allow display/change failover-only setting
• API-README corrections
• miner.php documentation (in API-README) v0.1
• Bugfix: opencl: Show blank device-info statline area if GPU doesn't have ADL, to fix column alignment
• README: Document usage of 0 to indicate "leave at default" for comma-delimited GPU options
• Correct API-README versions to match when BFGMiner included them
• API-README update changelog
• Minimise locking and unlocking when getting counts by reusing shared mutex lock functions.
• Avoid getting more work if by the time the getwork thread is spawned we find ourselves with enough work.
• The bitforce buffer is cleared and hw error count incremented on return from a failed send_work already so no need to do it within the send_work function.
• Don't make mandatory work and its clones last forever.
• modminer: Log debug info for nonces found

[kano]

...
README: Note that user groups don't get updated until re-login
...

(and your git is not 2.6.5)

[Luke-Jr]

NEW VERSION 2.7.0, AUGUST 21 2012

Besides the recent BFGMiner 2.6.5 improvements, this is just changes merged in from CGMiner. Unlike CGMiner 2.7.0, however, this does not include the "Work Utility" number, since it is both misleading (because it doesn't account for stales as Utility does) and redundant (BFGMiner's utility-hashrate, the 3rd hashrate column, shows a correct measurement based on accepted shares regardless of target). I contemplated adding a Weighed Utility to show a similar (but correct) number, but IMO this would just lead to uninformed attempts to compare it with CGMiner's "Work Utility" and not be of any practical use.

Human readable changelog:

• The main change in this version is a complete rewrite of the getwork requesting mechanism. Con had been slowly hacking away at the old one for some time, but finally gave up in disgust and has rewritten it to some extent. Previously, mining threads would occasionally throw out a request for more work, some arbitrary test would be done on whether more work should be requested, and it handed off the message to another thread which spawned another thread and that then sent the request and so on. Worse yet it was hard to find the right place to reuse work and so it was never reused to its utmost potential. The rewrite involves scheduling a new request based on the rate the old work items get used up, and is much better at predicting when it needs to leak work to backup pools and less likely to throw a "pool is not providing work fast enough" message. Overall you should now see much more Local Work (LW), the efficiency will be higher on pools that support rolltime, less work will be discarded, any magnitude
  rig will be kept solidly busy - note this MAY mean your overclocks will become that much more stressed if you have set clocks very aggressively. Thanks to Con and the numerous people who tested this during its development phase. For many of you, you'll be wondering what the fuss is about cause it will just appear as business as usual.
• New pool strategy: Balance.
  --balance           Change multipool strategy from failover to even share balance
  This is to differentiate itself from the existing pool strategy, Load balance:
  --load-balance      Change multipool strategy from failover to efficiency based balance
• With the change to queueing and more roll work being possible than ever before, the imbalance between pools that support rolltime and those that don't will now be extreme in load balance strategy. To offset that, and since the number of people using load balance has been increasing, the new strategy was added to try and give roughly the same number of shares to each pool.
• Other minor bugfixes.

Full changelog

• Implement a new pool strategy, BALANCE, which monitors work performed per pool as a rolling average every 10 minutes to try and distribute work evenly over all the pools. Do this by monitoring diff1 solutions to allow different difficulty target pools to be treated equally, along with solo mining. Update the documentation to describe this strategy and more accurately describe the load-balance one.
• fpga serial I/O extra debug (disabled by default)
• Getwork fail was not being detected. Remove a vast amount of unused variables and functions used in the old queue request mechanism and redefine the getfail testing.
• Consider us lagging only once our queue is almost full and no staged work.
• Simplify the enough work algorithm dramatically.
• Only queue from backup pools once we have nothing staged.
• Don't keep queueing work indefinitely if we're in opt failover mode.
• Make sure we don't opt out of queueing more work if all the queued work is from one pool.
• Set lagging flag if we're on the last of our staged items.
• Reinstate clone on grabbing work.
• Grab clones from hashlist wherever possible first.
• Cull all the early queue requests since we request every time work is popped now.
• Keep track of staged rollable work item counts to speed up clone_available.
• Make expiry on should_roll to 2/3 time instead of share duration since some hardware will have very fast share times.
• Check that we'll get 1 shares' worth of work time by rolling before saying we should roll the work.
• Simplify all those total_secs usages by initialising it to 1 second.
• Overlap queued decrementing with staged incrementing.
• Artificially set the pool lagging flag on pool switch in failover only mode as well.
• Artificially set the pool lagging flag on work restart to avoid messages about slow pools after every longpoll.
• Factor in opt_queue value into enough work queued or staged.
• Roll work whenever we can on getwork.
• Queue requests for getwork regardless and test whether we should send for a getwork from the getwork thread itself.
• Get rid of age_work().
• Don't try to get bitforce temperature if we're polling for a result to minimise the chance of interleaved responses.
• Fix harmless unused warnings in scrypt.h.
• Check we are not lagging as well as there is enough work in getwork.

[kano]

FYI - it is called Work Utility coz it is "Work Utility" i.e. work that is done - which of course is independent of if the work was accepted by the pool or not.

You are confusing that with "Share Utility" "U:" which of course is the number of shares accepted (i.e. excludes rejects etc)

[Luke-Jr]

NEW VERSION 2.7.1, AUGUST 21 2012

Just some fixes from Con and Kano that didn't make the 2.7.0 cut, plus updates to the Windows build instructions from Sharky.

Human readable changelog:

• The occasional strange behaviour where lots of work would end up on backup pools in failover mode should be fixed.
• The occasional scenario where one pool dies and the others behave like they're slow to provide work should be fixed.
• Very high hashrate (U > 100) machines should now be able to work unhindered even on pools that don't support rolltime.
• The --retry-pause/-R option no longer does anything and is deprecated in preference for faster communications and simpler code.
• -r was removed as a shortcut for --retries.
• Hopefully the dynamic mode for GPUs is fixed on windows.
• Hopefully the 7 day windows crash bug has been fixed

Full changelog

• Update windows build instructions courtesy of sharky.
• Increase max curls to number of mining threads + queue * 2, accounting for up and downstream comms.
• Queue enough requests to get started.
• There is no point trying to clone_work in get_work() any more since we clone on every get_work_thread where possible.
• There is no point subtracting 1 from maxq in get_work_thread.
• miner.php allow page title to be defined in myminer.php
• Only set lagging flag once there are no staged work items.
• select_pool does not switch back to the primary once lagging is disabled.
• Increment total work counter under mutex lock.
• Increment the queued count after the curl is popped in case there's a delay waiting on curls and we think we've queued work when in fact we're waiting on curls.
• Do the dynamic timing in opencl code over a single pass through scanhash to make sure we're only getting opencl times contributing to the measured intervals.
• Increase curl reaping time to 5 minutes since comms between  curl requests can be 2 mins apart with lots of rolltime.
• No need for extra variable in hash_push.
• Remove short options -r and -R to allow them to be reused and remove readme entries for deprecated options.
• Deprecate the opt_fail_pause parameter, leaving a null placeholder for existing configurations.
• Free work before retrying in get_work_thread.
• Don't pause after failed getwork, set lagging flag and reassess.
• We should not be pausing in trying to resubmit shares.
• Get rid of the extending fail pause on failed connects since we discard work after a period.
• get_work always returns true so turn it into a void function.
• get_work never returns false so get rid of fail pause loop.
• Get rid of pause and retry from get_upstream_work so we only do it from one place.
• Remove all cases where --retries aborts BFGMiner, making it for submission retries only, where it makes sense.

[Luke-Jr]

NEW VERSION 2.7.3, AUGUST 23 2012

Human readable changelog:

• Fix new networking code.
• OpenCL: New kernels.
• Updated miner.php

Full changelog

• Minimise the number of getwork threads we generate.
• Pick worksize 256 with Cypress if none is specified.
• Give warning with sdk2.7 and phatk as well.
• Whitelist sdk2.7 for diablo kernel as well.
• Only keep the last 6 blocks in the uthash database to keep memory usage constant. Storing more is unhelpful anyway.
• Increase kernel versions signifying changed APIs.
• BFL flash - more FPGA-README
• Check we haven't staged work while waiting for a curl entry before proceeding.
• Use atomic ops to never miss a nonce on opencl kernels, including nonce==0, also allowing us to make the output buffer smaller.
• Remove compile errors/warnings and document compile/usage in FPGA-README
• Ignore the submit_fail flag when deciding whether to recruit more curls or not since we have upper bounds on how many curls can be recruited, this test is redundant and can lead to problems.
• API-README update cgminer version number
• API-README fix groups P: example mistake
• API-README add COIN and other edits
• miner.php allow 'coin' is custom pages

[Luke-Jr]

Temporary hack to fix 2.7.3 with StreamSDK 2.1:
1) Figure out which *.cl file you use, and open it in a PLAIN text editor (eg, Notepad - NOT Wordpad!)
2) Add to the very top:

Code:

#pragma OPENCL EXTENSION cl_khr_global_int32_base_atomics : enable

3) Find and replace every instance of "atomic_add" with "atom_add"

Note that this seems to still have a 2 MH/s (on 5850) performance hit.

[Luke-Jr]

NEW VERSION 2.7.4, AUGUST 23 2012

2.6.6 is also released with more backported fixes.

Human readable changelog:

• More fixes.

Full changelog

• Perform select_pool even when not lagging to allow it to switch back if needed to the primary.
• Simplify macros in output kernels avoiding apparent loops and local variables.
• Carry the needed bool over the work command queue.
• Move the decision to queue further work upstream before threads are spawned based on fine grained per-pool stats and increment the queued count immediately.
• Track queued and staged per pool once again for future use.
• OpenCL 1.0 does not have native atomic_add and extremely slow support with atom_add so detect opencl1.0 and use a non-atomic workaround.
• Pools: add RollTime info to API 'stats' and 'Stats' button in miner.php

[mr_garrapata]

Just wanted to say that the bitforce-firmware-flash utility works like a charm under Xubuntu 12.04.1.

Thanks Luke-Jr.

[Luke-Jr]

Does the "restart BFGMiner when ADL fails" function do anything useful for anyone? I understand it was an attempt to workaround ADL/driver bugs, but I'm told that it never helped. On the other hand, it breaks when used with "Switch User" or some "hwinfo64" tester on Windows. If it doesn't do anything useful, I'll plan to remove it for 2.7.5, which I hope to release later today (so tell me ASAP if you need this!).

[Luke-Jr]

NEW VERSION 2.7.5, AUGUST 27 2012

Human readable changelog:

• ModMiner: Revamped the dynamic clocking algorithm to keep FPGAs at higher speeds with fewer hardware errors.
• OpenCL: Restored old default for Cypress (Radeon 58xx) worksize, to fix performance regression introduced in 2.7.2.
• Failover is now fixed, courtesy of Con.
• FPGAs: "-S all" option to try probing all enumerated serial ports on systems without non-miner serial devices.
• Reverted "restart on ADL failure" feature that never did anything useful (if I'm wrong, complain and I'll make a switch to add it back)

Full changelog

• Revert "Do a complete cgminer restart if the ATI Display Library fails, as it does on windows after running for some time, when fanspeed reporting fails."
• Stop special-casing worksize default to 256 for Cypress, since it incurs a 5 MH/s hit with stock config
• New "--scan-serial all" feature to probe all enumerated serial ports
• modminer: Revamp dynamic clocking algorithm per request from cablepair
• Test for lagging once more in queue_request to enable work to leak to backup pools.
• There is no need to try to switch pools in select_pool since the current pool is actually not affected by the choice of pool to get work from.
• Only clear the pool lagging flag if we're staging work faster than we're using it.
• needed flag is currently always false in queue_request. Remove it for now.
• thr is always NULL going into queue_request now.
• Fix for non-ADL OpenCL device formatting issue

[Mobius]

NEW VERSION 2.7.5, AUGUST 27 2012

Human readable changelog:

• ModMiner: Revamped the dynamic clocking algorithm to keep FPGAs at higher speeds with fewer hardware errors.
• OpenCL: Restored old default for Cypress (Radeon 58xx) worksize, to fix performance regression introduced in 2.7.2.
• Failover is now fixed, courtesy of Con.
• FPGAs: "-S all" option to try probing all enumerated serial ports on systems without non-miner serial devices.
• Reverted "restart on ADL failure" feature that never did anything useful (if I'm wrong, complain and I'll make a switch to add it back)

Full changelog

• Revert "Do a complete cgminer restart if the ATI Display Library fails, as it does on windows after running for some time, when fanspeed reporting fails."
• Stop special-casing worksize default to 256 for Cypress, since it incurs a 5 MH/s hit with stock config
• New "--scan-serial all" feature to probe all enumerated serial ports
• modminer: Revamp dynamic clocking algorithm per request from cablepair
• Test for lagging once more in queue_request to enable work to leak to backup pools.
• There is no need to try to switch pools in select_pool since the current pool is actually not affected by the choice of pool to get work from.
• Only clear the pool lagging flag if we're staging work faster than we're using it.
• needed flag is currently always false in queue_request. Remove it for now.
• thr is always NULL going into queue_request now.
• Fix for non-ADL OpenCL device formatting issue

Aren't most of the changes above taken from Conman's git comments when he committed the changes in the cgminer repository?

[cablepair]

Define Most...

Failover is now fixed, courtesy of Con.

some of the changes are in regards to the ModMiner quad which cgminer (unfortunately) does not keep up on.

[Luke-Jr]

NEW VERSION 2.7.5, AUGUST 27 2012

Aren't most of the changes above taken from Conman's git comments when he committed the changes in the cgminer repository?

Five of the ten commits between 2.7.4 and 2.7.5 are, as attributed, written by Con to fix bugs he had introduced, yes - so that's half if you're just counting commits. Line-wise, it's probably less than half. I can't even speculate on time spent. Unfortunately, I can't even speculate on "man-time spent" statistics.

[kano]

NEW VERSION 2.7.5, AUGUST 27 2012

Aren't most of the changes above taken from Conman's git comments when he committed the changes in the cgminer repository?

Five of the ten commits between 2.7.4 and 2.7.5 are, as attributed, written by Con to fix bugs he had introduced, yes - so that's half if you're just counting commits. Line-wise, it's probably less than half. I can't even speculate on time spent. Unfortunately, I can't even speculate on "man-time spent" statistics.

All 5 of them to do with fixing bugs in the new MUCH improved getwork scheduler that ckolivas/conman wrote completely himself and you had nothing to do with at all ... but it is now part of your copy miner.

... and you have the gall to make that implication ...

[loadrs2009]

i me using bfgminer 2.5.1 which work fine on my pc but when i download bfgminer 2.7.5 and run it on my pc it wont work can you help me to fix it  .
thanks

[Luke-Jr]

i me using bfgminer 2.5.1 which work fine on my pc but when i download bfgminer 2.7.5 and run it on my pc it wont work can you help me to fix it  .
thanks

Please open an Issue with the full details (eg, how doesn't it work, what devices you mine with, etc)

[loadrs2009]

it give me this error All devices disabled, cannot mine! 

[Luke-Jr]

it give me this error All devices disabled, cannot mine! 

Try running with bfgminer -D -T -d? and pastebin that.

And don't forget to mention what OS/devices/etc...