OFFICIAL CGMINER mining software thread for linux/win/osx/mips/arm/r-pi 4.11.1

[DavinciJ15]

(and no shares would get paid double).

Huh? What?

If a pool requires miner 1 to submit minimum difficulty 2 shares, but miner 2 can submit minimum difficulty 1 shares, then each share submitted by miner 1 will be paid twice as much as each share submitted by miner 2.  If all miners are required to submit minimum difficulty 2 shares, then no miner's shares are worth more than any other miner's shares.

I get the first part but how can a miner submit "No Shares" and get paid?

[1714919141]

1714919141

[1714919141]

1714919141

[1714919141]

1714919141

[1714919141]

1714919141

[1714919141]

1714919141

[The00Dustin]

(and no shares would get paid double).

Huh? What?

If a pool requires miner 1 to submit minimum difficulty 2 shares, but miner 2 can submit minimum difficulty 1 shares, then each share submitted by miner 1 will be paid twice as much as each share submitted by miner 2.  If all miners are required to submit minimum difficulty 2 shares, then no miner's shares are worth more than any other miner's shares.

I get the first part but how can a miner submit "No Shares" and get paid?

LOL, nice, you are joking right?  Just in case you're not, let me rephrase it this way: "none of the submitted shares get paid double."  This is because they would all get paid equally since there would be no variation from the minimum difficulty for different shares to be compared against.  Presumably you understood this from Krak's post and I am wasting my time.

[kano]

Didn't read the forum for two days (busy weekend work)
OK lets see if I can write it clearly.

Accepted 12345678 Diff X/Y GPU 1 pool 3

Y = the difficulty the pool requires and pays
X = the difficulty of the share found

Y is all that matters - X is purely for interest sake only
(and of course X will be greater than or equal to Y for all Accepted shares)

Now the easiest way to remove any confusions is to consider this:
Before vardiff, when you found a block while pool mining on a pool that doesn't pay a block bounty (most pools) you only got paid for a 1 diff share.
It didn't matter that it would have been greater than 1 million difficulty, you still only got paid for a 1 diff share.

To consider the maths is quite simple:
When you mine, on average half your shares will be above 2 diff (yeah I'm not gonna explain that one )
So if you are mining at 2 diff then on average half your shares will be submitted.
The remainder (below 2 diff) are not submitted, or if they were submitted they would be rejected.
That's why each share is worth 2 times as much as a 1 diff share.

Replace 2 with any number N above 1, and half with 1/N and it's the same.

--

Also note that X is rounded down so it depends on the pool's definition of 1 difficulty.

The value currently used is 26959946667150639794667015087019630673637144422540572481103610249216.0
Yes that's it
Or in hex:
0x00000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffff

(Edit: and ... I just realised that's wrong at the last place - it should end in 5, not 6 - so the current code is only accurate to 67 places, the 68th place is wrong )

This does mean that the value for X displayed, may on rare occasions, with a pool that uses correct difficulty values, not be correct.
It doesn't, however, mean that you will ever lose a share due to this.
The pool advertises the difficulty and the share is checked against the pools difficulty.
The problem is simply interpreting that number supplied by the pool and displaying it.

Most pools use that incorrect value for the BTC definitions of 1 difficulty above.
It means they only need to check the first 32bits instead of all 256bits and can stop at the 98.2% mark of processing the triple hash
(yeah even though it's a double hash, it's three times through the 64 steps since the first hash takes 2 goes through)
No idea why they think this is a big issue since the checking code is only a tiny fraction of the code that does the hash and the total gain in hashing is only 1.8% ... unless they trust the midstate you send back ... then it's 3% ... hmm I wonder if any pools risk that.

Anyway, the result it that they Accept an extra share every 65536 shares - so miners don't get extra Rejects, they get a rare extra Accept.

Here's an example of one from back in April:
[2012-04-20 15:50:10] Accepted 00000000.ffff7bc4.7c568d65 GPU 1 thread 3 pool 0

With correct difficulty that would have been rejected.

The correct value for 1 difficulty is:
0x00000000ffff0000000000000000000000000000000000000000000000000000

[PandaMiner]

I've been thinking why not hard code the --fix-protocol into 2.8.6+ since it seems to do the job very well. (and add a --no-fix-protocol option instead?)

Because Stratum will hopefully be the default mining protocol soon. Better to get the bugs out of the way now.

Right.

This answer doesn't make sense; in fact, counter intuitive.  Are you saying if every BTC pool switched to stratum, that NOBODY would need to use the --fix-protocol parameter anymore? Just curious.

[-ck]

I've been thinking why not hard code the --fix-protocol into 2.8.6+ since it seems to do the job very well. (and add a --no-fix-protocol option instead?)

Because Stratum will hopefully be the default mining protocol soon. Better to get the bugs out of the way now.

Right.

This answer doesn't make sense; in fact, counter intuitive.  Are you saying if every BTC pool switched to stratum, that NOBODY would need to use the --fix-protocol parameter anymore? Just curious.

So long as everything is working, yes. Long term the need for that option should even be removed, but it's still early days.

[DavinciJ15]

(and no shares would get paid double).

Huh? What?

If a pool requires miner 1 to submit minimum difficulty 2 shares, but miner 2 can submit minimum difficulty 1 shares, then each share submitted by miner 1 will be paid twice as much as each share submitted by miner 2.  If all miners are required to submit minimum difficulty 2 shares, then no miner's shares are worth more than any other miner's shares.

I get the first part but how can a miner submit "No Shares" and get paid?

LOL, nice, you are joking right?  Just in case you're not, let me rephrase it this way: "none of the submitted shares get paid double."  This is because they would all get paid equally since there would be no variation from the minimum difficulty for different shares to be compared against.  Presumably you understood this from Krak's post and I am wasting my time.

OK it appears that I understood the concept long before, but assumed it was more complicated than I thought.  However what has left me kind of confused is the fact that pool can or can't determining the difficulty of shares.  If they can then then submit the number of shares found in the db, if they can't then how the hell can they require all miners to submit a higher difficulty when the miner could just send the a diff of 1 and they would never know.

I think you guys are making it more complex than it needs to be so I will just assume that pools can detect the diff of shares sent from miners under certain circumstances and I just don't know what those circumstances are.  And leave it at that.

[The00Dustin]

I think you guys are making it more complex than it needs to be so I will just assume that pools can detect the diff of shares sent from miners under certain circumstances and I just don't know what those circumstances are.  And leave it at that.

Maybe what you need to know is that the display is just so users can tell what is going on.  The pool can tell the difficulty.  Really, the user can tell the difficulty the same way.  Here's a quick example:

[2012-11-05 09:53:34] Accepted 231bf483 Diff 7/1 GPU 0 pool 5
 [2012-11-05 09:53:38] Accepted cce04ba1 Diff 1/1 GPU 0 pool 5
 [2012-11-05 09:53:53] Accepted 7c4f2f3c Diff 2/1 GPU 0 pool 5
 [2012-11-05 09:54:04] Accepted 4ddf5594 Diff 3/1 GPU 0 pool 5
...
 [2012-11-05 09:54:36] Accepted 088ec9d0 Diff 29/1 GPU 0 pool 5
 [2012-11-05 09:54:42] Accepted 3f8bcb0c Diff 4/1 GPU 0 pool 5
 [2012-11-05 09:54:45] Accepted 140eed64 Diff 12/1 GPU 0 pool 5
...
 [2012-11-05 10:05:56] Accepted 0085e3b5 Diff 489/1 GPU 0 pool 5

Any accepted share is at least difficulty 1 (with rare exception as mentioned by Kano where pools accept shares technically lower than 1).  What I bolded is what is relevant to telling difficulty (and the difficulty it tells).  It is more complicated than just looking at the first character(s) in the displayed hash (which is actually at least the 9th character in the actual hash, with everything earlier than the displayed being 0's), but that is all that is necessary to start to see the correlation.  I believe any displayed hash that starts with 8 through f is difficulty 1; beyond that, as the hex character(s) it starts with get lower, the difficulty gets higher.

ETA:  This is with minimum difficulty 1.  If the pool had minimum difficulty higher than 1, at least of those shares wouldn't have been displayed or submitted by cgminer, and if cgminer did display and submit it(/them), it(/they) would have been rejected by the pool.

[kano]

(and no shares would get paid double).

Huh? What?

If a pool requires miner 1 to submit minimum difficulty 2 shares, but miner 2 can submit minimum difficulty 1 shares, then each share submitted by miner 1 will be paid twice as much as each share submitted by miner 2.  If all miners are required to submit minimum difficulty 2 shares, then no miner's shares are worth more than any other miner's shares.

I get the first part but how can a miner submit "No Shares" and get paid?

LOL, nice, you are joking right?  Just in case you're not, let me rephrase it this way: "none of the submitted shares get paid double."  This is because they would all get paid equally since there would be no variation from the minimum difficulty for different shares to be compared against.  Presumably you understood this from Krak's post and I am wasting my time.

OK it appears that I understood the concept long before, but assumed it was more complicated than I thought.  However what has left me kind of confused is the fact that pool can or can't determining the difficulty of shares.  If they can then then submit the number of shares found in the db, if they can't then how the hell can they require all miners to submit a higher difficulty when the miner could just send the a diff of 1 and they would never know.

I think you guys are making it more complex than it needs to be so I will just assume that pools can detect the diff of shares sent from miners under certain circumstances and I just don't know what those circumstances are.  And leave it at that.

The pool tells the miner what difficulty the shares need to be.
cgminer checks the shares difficulty and only submits shares that are above the difficulty
(which actually means below the target value )
The pool checks that the difficulty is above what it told the miner.
If it is above, it is accepted, if it is below it is rejected.

There are other complexities related to this when share difficulty is changed by the pool (that I think is bad in Stratum) but the basics are as I said above.

[DavinciJ15]

The pool tells the miner what difficulty the shares need to be.
cgminer checks the shares difficulty and only submits shares that are above the difficulty
(which actually means below the target value )
The pool checks that the difficulty is above what it told the miner.
If it is above, it is accepted, if it is below it is rejected.

There are other complexities related to this when share difficulty is changed by the pool (that I think is bad in Stratum) but the basics are as I said above.

Thanks!  Nice and simple explanation.

[slush]

There are other complexities related to this when share difficulty is changed by the pool (that I think is bad in Stratum) but the basics are as I said above.

I'm thinking about the solution and I think that I found it. When the protocol change from current "drop all shares above the target since receiving this new difficulty" to "drop all shares abouve the target for all new jobs", will it solve our problem?

a) When pool want to force user to new difficulty, it will send set_difficutly and job with clean_jobs=True like now
b) When pool want to update difficulty lazily, he will just send set_difficulty, no new job is required. In would be the most common scenario and there's also no way how to "lose" shares.

It just need some minor change in cgminer. It will basically bound difficulty with the job as you wanted and leave the difficulty as separate message as I wanted. So it looks like win-win solution.

ckolivas, Eleuthria - hm?

[lenny_]

ckolivas, currently, AVG 2012 Free Edition Antivirus saying that file:
libpdcurses.dll
contains a virus: PSW.KeyLogger.AVU http://www.avgthreatlabs.com/webthreats/info/psw.keylogger.avu/

Weird, isn't?

[crazyates]

ckolivas, currently, AVG 2012 Free Edition Antivirus saying that file:
libpdcurses.dll
contains a virus: PSW.KeyLogger.AVU http://www.avgthreatlabs.com/webthreats/info/psw.keylogger.avu/

Weird, isn't?

http://goo.gl/2wnwi

[lenny_]

ckolivas, currently, AVG 2012 Free Edition Antivirus saying that file:
libpdcurses.dll
contains a virus: PSW.KeyLogger.AVU http://www.avgthreatlabs.com/webthreats/info/psw.keylogger.avu/

Weird, isn't?

http://goo.gl/2wnwi

I've read it. It's all saying about cgminer.exe used with many malware, but not libpdcurses.dll.

[ancow]

ckolivas, currently, AVG 2012 Free Edition Antivirus saying that file:
libpdcurses.dll
contains a virus: PSW.KeyLogger.AVU http://www.avgthreatlabs.com/webthreats/info/psw.keylogger.avu/

Weird, isn't?

http://goo.gl/2wnwi

I've read it. It's all saying about cgminer.exe used with many malware, but not libpdcurses.dll.

Search the thread for libpdcurses.dll - it's been covered ad nauseam...

[Mobius]

ckolivas, currently, AVG 2012 Free Edition Antivirus saying that file:
libpdcurses.dll
contains a virus: PSW.KeyLogger.AVU http://www.avgthreatlabs.com/webthreats/info/psw.keylogger.avu/

Weird, isn't?

did you READ THE README

[sharky112065]

ckolivas, currently, AVG 2012 Free Edition Antivirus saying that file:
libpdcurses.dll
contains a virus: PSW.KeyLogger.AVU http://www.avgthreatlabs.com/webthreats/info/psw.keylogger.avu/

Weird, isn't?

did you READ THE README

This would be one of the quietest threads on the forum if more people read the included README document/file.

[-ck]

There are other complexities related to this when share difficulty is changed by the pool (that I think is bad in Stratum) but the basics are as I said above.

I'm thinking about the solution and I think that I found it. When the protocol change from current "drop all shares above the target since receiving this new difficulty" to "drop all shares abouve the target for all new jobs", will it solve our problem?

a) When pool want to force user to new difficulty, it will send set_difficutly and job with clean_jobs=True like now
b) When pool want to update difficulty lazily, he will just send set_difficulty, no new job is required. In would be the most common scenario and there's also no way how to "lose" shares.

It just need some minor change in cgminer. It will basically bound difficulty with the job as you wanted and leave the difficulty as separate message as I wanted. So it looks like win-win solution.

ckolivas, Eleuthria - hm?

Hmm... To work with the current stratum limitation, cgminer checks shares against the current stratum pool difficulty before trying to submit them. I can remove this code and then difficulty is tied to the work at the time it was generated up to how the pool does things entirely. I'm thinking I will check difficulty against the new difficulty only if difficulty has dropped and then leave it up to the pool to cope. While this may introduce more rejects with diff increases, at least there is no chance of work lost, and will work with your idea.

Tying difficulty with work entirely would still be much better.

[-ck]

New version -> 2.9.1

Main thing in this new point release is support for the pooled GBT mining protocol. This is once again a large chunk of new code and carries with it the attendant risks of new instability so I'm leaving 2.8.7 up for download. The GBT implementation is my own from scratch, not using an existing implementation. luke-jr helped debug a few things to do with how the protocol worked since no one else seems to understand it.

Note that I have serious reservations about recommending GBT as significantly advantageous for the miner (in terms of performance) unlike stratum. See my post here as to why:
https://bitcointalk.org/index.php?topic=108854.msg1317574#msg1317574



Human readable changelog

Support for GBT mining protocol.
Minor stratum tweaks, bugfixes and improvements.


Full changelog

Version 2.9.1 - November 6, 2012

- Reset work flags to prevent GBT shares from being submitted as stratum ones
after switching.


Version 2.9.0 - November 6, 2012

- Add endian swap defines for where missing.
- Only retarget stratum shares to new pool diff if diff has dropped.
- Remove resetting of probed variable when detecting GBT.
- Count lost stratum share submits and increase message priority to warning.
- Only retrieve a new block template for GBT pools that are the current pool.
- Show which pool untracked share messages have come from.
- Add management for dead GBT pools.
- Count lost shares with stratum as submit stale lost.
- Discard record of stratum shares sent and report lost shares on disconnection
since they will never be reported back.
- Swab, don't just swap the bytes in the GBT target.
- Change status window message for GBT connected pools versus LP.
- Generate a gbt work item from longpoll when required to set new block and
message appropriately.
- Use existing pool submit_old bool from gbt data.
- Retrieve a new block template if more than 30 seconds has elapsed since the
last one to keep the data current and test the pool is still alive.
- Update GBT longpollid every time we request a new longpoll.
- Manage appropriate response codes for share submission with GBT.
- Allow the longpoll thread to start with GBT and only set the longpollid once.
- Correct last few components of GBT block generation courtesy of Luke-jr.
- Use correct length for offsetting extra nonce and remaining data.
- Flip all 80 bytes in the flip function which was wrongly named flip256 for its
purpose.
- Calculate midstate for gbt work and remove now unused variable.
- Use a standard function for flipping bytes.
- Insert the extra nonce and remaining data in the correct position in the
coinbase.
- Remove txn size debugging and enlarge gbt block string to prevent overflow.
- Remove varint display debugging.
- Build varint correctly for share submission and sleep 5 seconds before
retrying submit.
- Make gbt_coinbase large enough for submissions, swap bytes correctly to make a
header from GBT and encode the number of transactions in share submission.
- Store the fixed size entries as static variables in GBT in binary form,
byteswapping as is required.
- 32 bit hex encoded variables should be in LE with GBT.
- Target and prevblockhash need to be reversed from GBT variables.
- Construct block for submission when using GBT.
- Use same string for debug as for submission and make string larger to cope
with future GBT messages.
- Skip trying to decipher LP url if we have GBT support.
- Store all the transaction hashes in pool->txn_hashes instead of separating
txn0 and correct generation of merkle root, fixing memory overwrites.
- Hook into various places to generate GBT work where appropriate.
- Create extra work fields when generating GBT work.
- Generate header from correct hashing generation of the merkle root for GBT.
- Generate the merkle root for gbt work generation.
- Create a store of the transactions with GBT in the minimum size form required
to generate work items with a varied coinbase.
- Create a function that generates a GBT coinbase from the existing pool
variables.
- Extract and store the various variables GBT uses when decoding gbt work.
- Check for invalid json result in work_decode.
- Decode work in separate functions for getwork vs gbt.
- Check for the coinbase/append mutable in GBT support to decide whether to use
it or not.
- Add a gbt mutex within the pool struct for protecting the gbt values.
- Convert work decode function to prepare for decoding block templates.
- Check for GBT support on first probing the pool and convert to using the GBT
request as the rpc request for that pool.
- Make the rpc request used with getwork a pool variable to allow it to be
converted to/from gbt requests.
- Changes to build prototypes to support building on FreeBSD 9.1-RC2 amd64
- Free old stratum_work data before replacing it
- There is no need for addrinfo any more.
- server and client sockaddr_in are no longer used in struct pool.
- Merge pull request #322 from luke-jr/bugfix_stratum_tmpwork
- Set sshare id and swork_id within the sshare mutex to avoid multiple share
submits with the same id.
- Initialize temporary stratum work

[Naelr]

OK I looked.. used the search function and even google but can't find an answer to this question...  I posted on a new thread and was directed to this one.  My question is why will cgminer running on ubuntu 12.04 64 bit will not submit shares (in a timely manner) for scrypt mining.  Hashrate in the cgminer console is ~310Kh/s but share submits are reporting 2Kh/s

Here is my post https://bitcointalk.org/index.php?topic=120773.0

When I posted it I was running 2.7.5 and now I am running 2.8.7 and having the same problem.  Any ideas?
Any help is much appreciated.

Naelr

[kano]

cut/paste ...

2.9.0 2.9.1
An Xubuntu 11.04 x86_64 executable is in my github downloads called cgminer-2.9.0a cgminer-2.9.1a
https://github.com/kanoi/cgminer/downloads
(it also works on Fedora 16 and 17)

For anyone who didn't realise, it's just the executable file to put in place of 'cgminer'
Nothing else needs changing
First get and extract the full binary release from ckolivas and then copy my file in place of 'cgminer'

No problems so far on my '2xGPU' or 'BFL+ICA' - yes I've rearranged them (40 minutes so far) GPU's on solo and BFL+ICA (1.6GH/s) on EMC with GBT (MMQ is doing other testing)

The same configure options as cvolivas' binary version
In case anyone was wondering:
CFLAGS="-O2 -W -Wall" ./autogen.sh --enable-icarus --enable-bitforce --enable-ztex --enable-modminer --enable-scrypt
make clean
make