OFFICIAL CGMINER mining software thread for linux/win/osx/mips/arm/r-pi 4.11.1

[twmz]

Wrote code to append a file with this format: timestamp,disposition,target,pool,dev,thr,sharehash,sharedata

Will edit-in an example to this post when I get a chance to test (currently doing an all-day test of some other code).

If you want to test it before me, here's a paste of the code. It at least compiles for Windows. Zero checking on Linux so far.

It compiles on linux but doesn't work.  The output is happening, but not to a file (it appears on screen no matter what I have tried to pass to the command line param) for each share there is also this error:

Code:

[2012-04-24 17:59:54] sharelog fwrite error

[1715042775]

1715042775

[1715042775]

1715042775

[-ck]

I have a requested enhancement (see below for the bounty).  I have a need to log all of the shares that are found by cgminer (for statistical analysis purposes) and the existing information that is output is almost enough but In order to do my calculations, I need to know the target for the work that resulted in the share since I mine on pools that don't have a fixed share difficulty.

The current output looks something like this:

Code:

[2012-04-24 14:27:39] Accepted 00000000.131be987.c9501e82 PGA 0 thread 0 pool 1

My request is for some option to add in the target for the share.  To save output space, it would be preferable for it to be displayed as the equivalent difficulty and not the massively verbose hex target.  So as to not bother other cgminer users or those that are already parsing the current output, perhaps it should be optional (based on a new command line parameter, etc)?

I'm imagining something like:

Code:

[2012-04-24 14:27:39] Accepted 00000000.58373f06.7b5f26c7 (0.999985) PGA 1 thread 1 pool 1
[2012-04-24 14:27:44] Accepted 00000000.4534557e.a78f6b4a (2.314230) PGA 1 thread 1 pool 1
[2012-04-24 14:27:44] Accepted 00000000.db937fb0.34751330 (4.000000) PGA 1 thread 1 pool 1
[2012-04-24 14:27:44] Rejected 00000000.ca984202.8376eec0 (0.999985) PGA 1 thread 1 pool 1
[2012-04-24 14:27:47] Rejected 00000000.714a3646.4348042c (0.999985) PGA 0 thread 0 pool 1

Or some other equally parse-able format that includes the timestamp, accepted/rejected, the share hash (or at least as much as is currently included), the difficulty/target, which device it was, and which pool it was (for cases where there are multiple pools configured).  It's fine if it also includes the thread (as shown above), but I don't need that information, myself.

I have looked at the code and I don't think this is super difficult, but I am not a skilled enough C/C++ programmer to do the enhancement myself with any confidence, so I will put up a bounty for it instead.  I know ckolivas is busy and/or on a break, so I don't mind if someone else does it and provides a patch.  That said, I'd prefer to see it incorporated into the mainline source code so that I don't have to worry about it breaking in the future, so I'm going to include some incentive for it to be merged as well.  Here are the bounties I am willing to pay:

• 5 BTC to the author of a working patch against the current git cgminer source.
• 5 BTC to the author of the patch if and when it gets merged into the cgminer source. This obviously means the change has to be done well enough that ckolivas is comfortable merging it.
• 5 BTC to ckolivas if he merges the patch.

So that is 15 BTC to ckolivas if he does it all himself, or if it is done by someone else up to 10 BTC to the author and 5 BTC to ckolivas for merging it.

I can do it. Just trying to decide if it should be this way for everyone. Be aware that rejects are almost always rejected while meeting the target. They're rejected for other reasons.

[-ck]

Can multiple devices be specified in the config file?
i.e. the equivelent on -d0 -d1 -d3 -d4 -d5 on command line in the config file is ?

I might have to restructure that input as it predates my ability to parse multiple inputs with the one argument. I should change it to  -d 0,1,3,4,5 . Not even sure if you can put multiple -d on the config file, but if so, you'd just put in each -d entry on a separate line.

[twmz]

I can do it. Just trying to decide if it should be this way for everyone. Be aware that rejects are almost always rejected while meeting the target. They're rejected for other reasons.

Ok.  I'm fine with either adding the difficulty to the normal output as I originally suggested or creating a share log file as luke coded it (assuming we can get it working).

And I wouldn't expect this to change the reasons why shares appear as rejected.  I expect rejected shares are shares that cgminer thinks meet the target but that the pool rejected for whatever reason (e.g. they were stale).

[-ck]

I can do it. Just trying to decide if it should be this way for everyone. Be aware that rejects are almost always rejected while meeting the target. They're rejected for other reasons.

Ok.  I'm fine with either adding the difficulty to the normal output as I originally suggested or creating a share log file as luke coded it (assuming we can get it working).

And I wouldn't expect this to change the reasons why shares appear as rejected.  I expect rejected shares are shares that cgminer thinks meet the target but that the pool rejected for whatever reason (e.g. they were stale).

That's fine. How would you want the difficulty displayed? I'm not really sure I understand the (0.999985) type value you're displaying in your sample.

[Luke-Jr]

Wrote code to append a file with this format: timestamp,disposition,target,pool,dev,thr,sharehash,sharedata

Will edit-in an example to this post when I get a chance to test (currently doing an all-day test of some other code).

If you want to test it before me, here's a paste of the code. It at least compiles for Windows. Zero checking on Linux so far.

It compiles on linux but doesn't work.  The output is happening, but not to a file (it appears on screen no matter what I have tried to pass to the command line param) for each share there is also this error:

Code:

[2012-04-24 17:59:54] sharelog fwrite error

Try this one.

[twmz]

I can do it. Just trying to decide if it should be this way for everyone. Be aware that rejects are almost always rejected while meeting the target. They're rejected for other reasons.

Ok.  I'm fine with either adding the difficulty to the normal output as I originally suggested or creating a share log file as luke coded it (assuming we can get it working).

And I wouldn't expect this to change the reasons why shares appear as rejected.  I expect rejected shares are shares that cgminer thinks meet the target but that the pool rejected for whatever reason (e.g. they were stale).

That's fine. How would you want the difficulty displayed? I'm not really sure I understand the (0.999985) type value you're displaying in your sample.

I just used 0.999985 because that is the actual difficulty for a "diff 1" share for most pools since a diff 1 share is actually 0x00000000FFFF0000000000000000000000000000000000000000000000000000 but in practice most pool software actually uses 0x00000000FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF instead for simplicity of validation (they can just make sure that the hash starts with 8 zeros and don't have to do any actual large integer comparisons) and that target translates to slightly less than a difficulty of 1.0000.

So in other words, this is what I mean by difficulty:  https://en.bitcoin.it/wiki/Difficulty

Converting the target to difficulty is one of the things I didn't want to have to figure out myself in C/C++  I have code to do convert between difficulty and target and the 'bits' representation of target that is actually used in a block, just not in C/C++.  So I don't care which format is used since I can convert it as part of my analysis.  I just suggested difficulty because it was terse.  Target is equally fine with me, but it would then probably have to be in something like a seperate share log as luke was doing since there is no reasonable way to output the massive hex target in the normal output.

P.S.  Luke sharelog is mostly working now, so if you like it, you're welcome to avoid more work.   If you don't want to accept luke's solution, I'll probably still pay him an extra 5BTC for his effort since it works and is useful.

[twmz]

I paid luke 5BTC for the share log solution:

http://blockchain.info/tx/7732db071b843e378b458b447c4962b0ffb22d638da77096238cd266fc847eef

ckolivas, if you don't like the share log idea and want to do something else officially, I'm willing to still pay you the original 15 BTC since I should have anticipated the fact that someone else might have created a solution in the short time between when I posted the request and when you saw the bounty and decided to do something else.

Just let me know what you prefer.

[-ck]

I paid luke 5BTC for the share log solution:

http://blockchain.info/tx/7732db071b843e378b458b447c4962b0ffb22d638da77096238cd266fc847eef

ckolivas, if you don't like the share log idea and want to do something else officially, I'm willing to still pay you the original 15 BTC since I should have anticipated the fact that someone else might have created a solution in the short time between when I posted the request and when you saw the bounty and decided to do something else.

Just let me know what you prefer.

It's only money and you've already given some of it to luke who did the work. Why should I make you pay more than you originally offered? Luke's code is fine, but if you want me to include it, it'll have to be a proper pull request and fully documented as well since he'll get even more bounty according to your original offer if I include the code.

Gotta be quick to get the quick bucks it seems. I'll go back to the boring slow grind work instead

[-ck]

I paid luke 5BTC for the share log solution:

http://blockchain.info/tx/7732db071b843e378b458b447c4962b0ffb22d638da77096238cd266fc847eef

ckolivas, if you don't like the share log idea and want to do something else officially, I'm willing to still pay you the original 15 BTC since I should have anticipated the fact that someone else might have created a solution in the short time between when I posted the request and when you saw the bounty and decided to do something else.

Just let me know what you prefer.

It's only money and you've already given some of it to luke who did the work. Why should I make you pay more than you originally offered? Luke's code is fine, but if you want me to include it, it'll have to be a proper pull request and fully documented as well since he'll get even more bounty according to your original offer if I include the code.

Gotta be quick to get the quick bucks it seems. I'll go back to the boring slow grind work instead

I commented on a couple of minor issues with his code, he modified it, and I've included it in my git tree now.

[Chefnet]

that will be fine, and do you want to insert the quad support too?

"Are you talkin' to me?"

Yes, I've already asked for details on what changes in the API, and will have a look at BTCMiner sources as soon as I can get my hands on them. While I don't plan to own these quads soon, I do want cgminer to have the best possible ztex hardware support.

but it will be fine if it could be because of using cgminer with dd-wrt. Thank you for your work.

do you have an update?

[-ck]

NEW VERSION - VERSION 2.3.4, 25 APRIL 2012

The release archive for this is much bigger because of included firmware for ztex boards. The version number is only a minor increment because it is mostly bugfixes for any features already working, and extra features should not affect existing working code.

Human readable summary:
Status window fixes for >8 devices.
Ztex support
Information at startup if config file is loaded
(advanced) gpu-map option for when OpenCL and ADL disagree on device numbering
API thread will restart properly if cgminer is restarted from the menu, and this hopefully will fix the ADL deciding to stop reporting fanspeeds by successfully restarting cgminer.
More information is given off with -n on the command line now.
You can increase the queue much more than 10 now if so desired.
Updated miner.php
Better Icarus support
First ZTEX FPGA support.

Full changelog:
- Extensively document the cause of GPU device issues and the use of --gpu-map.
- Support for share logging
- Detect poorly performing combination of SDK and phatk kernel and add verbose
warning at startup.
- Icarus update to new add_cgpu()
- Icarus driver working with Linux and Windows
- api.c fix unused variable compile warning
- Display all OpenCL devices when -n is called as well to allow debugging of
differential mapping of OpenCL to ADL.
- Add a --gpu-map option which will allow arbitrarily mapping ADL devices to
OpenCL devices for instances where association by enumeration alone fails.
- Increase upper limit on number of extra items to queue as some FPGA code can't
yet reliably keep many devices busy.
- Display configuration file information when -c option is passed and only when
file exists on loading default config file.
- Display configuration file loaded, if any, and debug output if configuration
file parsing failed.
- Add missing ztex header to Makefile for distribution.
- Document long-form COM port device names on Windows, required to specify
serial ports above 9
- Include ztex bitstreams firmware in distribution and install if configured in.
- Style police on driver-ztex.c
- work_restart should only be changed by cgminer.c now
- Shut down the api cleanly when the api thread is cancelled. This should allow
the api socket to be closed successfully to next be reopened with app_restart.
- Make a union for cgpu device handles, and rename "device" to "device_ztex"
since it's Ztex-specific
- Initialise name variable.
- Remove unnecessary check for variable that always has memory allocated.
- Bugfix: Missing "break" no-op in default case
- Make the status window and log window as large as can fit on startup,
rechecking to see if it can be enlarged after the fact. This allows any number
of devices to be displayed provided the window is made long enough without
corrupting the output.
- Style police on libztex.c.
- API add removepool like the screen interface
- api.c escape required characters in return strings + pools returns the
username
- Set lp_path to NULL after free for consistency.
- Removing dmalloc import left behind by mistake
- Fixing leak in resp_hdr_cb
- miner.php warning highlight GPU stats if they are zero (e.g. ADL not enabled)
- miner.php highlight any device that isn't 'Enabled'
- miner.php highlight any Status that isn't 'Alive'
- miner.php optionally support multiple rigs
- Initial Ztex support 1.15x board.

[Chefnet]

if you write support of ztex is this only 1.15x or y too?

[-ck]

- Initial Ztex support 1.15x board.

[nelisky]

if you write support of ztex is this only 1.15x or y too?

1.15y support will come soon, but since I don't have one such board it will initially be based on reverse engineering and voluntary external testing. This approach is bound to take some time but we'll get there.

If someone wants to handle donations towards a board to send me that would likely speed things up (and Stefan will most likely give some discount based on his OS program) but it is not really needed, just helps a bit.

[-ck]

By the way, I'm having great trouble justifying the FPGA support in cgminer. It's taking me a fair amount of time vetting all the code that comes in, dealing with the drama of different coders not getting on, and that I don't have an FPGA so the code does not benefit me, nor am I earning anything off supporting FPGAs. So once again I find myself in a quandary about whether I should even be supporting them at all or not (like CPU mining, but for precisely the opposite reason!). So unless this gets easier, and it looks like it will only get harder, there is a very real possibility that someone like Luke-jr will fork the FPGA code into a separate project (he has even suggested he will) and I'll stop supporting it. I'm just giving advance warning here to people about this issue. I know it's just me whining, and harsh, but the fact is that the effort has to come from somewhere, and I do not have infinite time on my hands.

[twmz]

I paid luke 5BTC for the share log solution:

http://blockchain.info/tx/7732db071b843e378b458b447c4962b0ffb22d638da77096238cd266fc847eef

ckolivas, if you don't like the share log idea and want to do something else officially, I'm willing to still pay you the original 15 BTC since I should have anticipated the fact that someone else might have created a solution in the short time between when I posted the request and when you saw the bounty and decided to do something else.

Just let me know what you prefer.

It's only money and you've already given some of it to luke who did the work. Why should I make you pay more than you originally offered? Luke's code is fine, but if you want me to include it, it'll have to be a proper pull request and fully documented as well since he'll get even more bounty according to your original offer if I include the code.

Gotta be quick to get the quick bucks it seems. I'll go back to the boring slow grind work instead

I commented on a couple of minor issues with his code, he modified it, and I've included it in my git tree now.

Awesome, thanks.

The remaining two bounty payments have been sent:

To Luke: http://blockchain.info/tx/ba031cdf4efdaf4a7f6bce52442539c6a415b5c612f194c04a4a4971cfdebb9b
To Ckolivas: http://blockchain.info/tx/a11d526aa66121ed4b0bb5cf828f2ab223e83a34471dbf6020e664dc219b52b5

[-ck]

The remaining two bounty payments have been sent:

To Ckolivas: http://blockchain.info/tx/a11d526aa66121ed4b0bb5cf828f2ab223e83a34471dbf6020e664dc219b52b5

Much appreciated, thanks 

[tnkflx]

By the way, I'm having great trouble justifying the FPGA support in cgminer. It's taking me a fair amount of time vetting all the code that comes in, dealing with the drama of different coders not getting on, and that I don't have an FPGA so the code does not benefit me, nor am I earning anything off supporting FPGAs. So once again I find myself in a quandary about whether I should even be supporting them at all or not (like CPU mining, but for precisely the opposite reason!). So unless this gets easier, and it looks like it will only get harder, there is a very real possibility that someone like Luke-jr will fork the FPGA code into a separate project (he has even suggested he will) and I'll stop supporting it. I'm just giving advance warning here to people about this issue. I know it's just me whining, and harsh, but the fact is that the effort has to come from somewhere, and I do not have infinite time on my hands.

If we can get you FPGAs, would you be willing to support those in cgminer?

[twoBitBasher]

umm have you guys seen any issues with memory usage with cgminer under win 7 x64?
It seems like a memory leak. It doesn't go above 400mb in peak working set(taskmgr) yet slowly all of my ram is filled and then the inevitable happens. Everything goes to hell. Quitting the program doesn't free the used ram. Only reboot helps.
I have 2x 5970 2x 5870 with 11.12 catalyst + 2.5 sdk.