Mining inefficiency due to discarded work

[FairUser]

I keep seeing these "invalid or stale" a couple of times a day.

30/01/2011 15:26, bea41297, accepted
30/01/2011 15:26, bea41297, invalid or stale
30/01/2011 15:27, b9c26c57, accepted
30/01/2011 15:27, fb27e19a, invalid or stale

Notice the two lines at the top in bold.  
Could there possibly be more than one answer per getwork?
As you can see, I got the answer previously, then got the same answer again.
Then on the last line, it just came back invalid...probably cause someone else got the answer to the getwork before I did.

Is every getwork unique?  If so, how did I get the same answer twice?

UPDATE:  Got two more
30/01/2011 15:30, e67d85ec, invalid or stale
30/01/2011 15:36, f1f124d9, invalid or stale

[1714033315]

1714033315

[1714033315]

1714033315

[1714033315]

1714033315

[slush]

Is every getwork unique?  If so, how did I get the same answer twice?

Hi FairUser,

'invalid or stale' for unique nonces can be explained by pool update. Pool now doesn't accept shares when new bitcoin block come (see pool thread for more). Now it works in the same way as standalone mining.

About same getworks - it looks like network troubles, which are more and more common. I hope that it will be solved by protocol change, which significantly reduce network communication. Will be introduced this week.

[FairUser]

Is every getwork unique?  If so, how did I get the same answer twice?

Hi FairUser,

'invalid or stale' for unique nonces can be explained by pool update. Pool now doesn't accept shares when new bitcoin block come (see pool thread for more). Now it works in the same way as standalone mining.

About same getworks - it looks like network troubles, which are more and more common. I hope that it will be solved by protocol change, which significantly reduce network communication. Will be introduced this week.

This is not a network communications problem.
I record my getwork request and answers.
The two getwork were ENTIRELY different getwork request, with different data and midstates, yet produced the same answers.

[bitcool]

Can anyone explain a bit on the advantages of Pool mining with Slush's pool vs Mining solo?

I'm solo mining and am still pretty new to this scene, and I've seen the details on HOW to mine in Slush's pool, but I haven't grasped the advantages if any, for my situation

I am also interested to learn what is the difference between these two. I am guessing it would be more efficient to do pool mining resource wise, but it's just a guess since I don't know much about the mining strategy algorithm in each case.

Say I have several 5970s, if I let them mine solo, there's no coordination among these miners, I'd think there would be a some kind of wasted work because of duplicate/overlapped attempts on the same hashes, while the pool mining work can be distributed in a way to avoid or minimize this waste. True or false?

[lfm]

Can anyone explain a bit on the advantages of Pool mining with Slush's pool vs Mining solo?

I'm solo mining and am still pretty new to this scene, and I've seen the details on HOW to mine in Slush's pool, but I haven't grasped the advantages if any, for my situation

I am also interested to learn what is the difference between these two. I am guessing it would be more efficient to do pool mining resource wise, but it's just a guess since I don't know much about the mining strategy algorithm in each case.

Say I have several 5970s, if I let them mine solo, there's no coordination among these miners, I'd think there would be a some kind of wasted work because of duplicate/overlapped attempts on the same hashes, while the pool mining work can be distributed in a way to avoid or minimize this waste. True or false?

False. there are zillions of possible blocks/hashes to find and no one ever duplicates any searches.

the advantage of pools is you get smaller rewards more frequently. the total rewards are on the average almost identical.

[FairUser]

False. there are zillions of possible blocks/hashes to find and no one ever duplicates any searches.

Ever?
Duplicates will never-ever-ever be found in the getwork searches?
Each getwork is unique, and no result from getwork A could possibly match getwork B?

Just trying to make sure I understand that is what you're saying.

[theymos]

Ever?
Duplicates will never-ever-ever be found in the getwork searches?
Each getwork is unique, and no result from getwork A could possibly match getwork B?

If there are no duplicate getwork responses, then the probability of re-doing work is so low that it will almost certainly never happen to anyone as long as Bitcoin exists.

Duplicate getwork responses would cause you to do work that has already been done. However, slush's pool relies on Bitcoin to handle this, so there shouldn't be any difference between mining on your own and mining in a pool. If anything, the pool is more likely to produce duplicates, since Bitcoin was not designed to produce unique work for so many miners.

[lfm]

False. there are zillions of possible blocks/hashes to find and no one ever duplicates any searches.

Ever?
Duplicates will never-ever-ever be found in the getwork searches?
Each getwork is unique, and no result from getwork A could possibly match getwork B?

Just trying to make sure I understand that is what you're saying.

Assuming it is working as designed. Bugs throw out all bets.

Hashes are 256 bits. The odds of finding two random hashes the same would be much less than the odds of getting struck by lightning on the same day you buy a single ticket and win the lottery. I call that never.

[FairUser]

False. there are zillions of possible blocks/hashes to find and no one ever duplicates any searches.

Ever?
Duplicates will never-ever-ever be found in the getwork searches?
Each getwork is unique, and no result from getwork A could possibly match getwork B?

Just trying to make sure I understand that is what you're saying.

Assuming it is working as designed. Bugs throw out all bets.

Hashes are 256 bits. The odds of finding two random hashes the same would be much less than the odds of getting struck by lightning on the same day you buy a single ticket and win the lottery. I call that never.

Assuming it is working as designed........that is indeed the question I'm raising.

So why is it when I process the ENTIRE getwork request (vs just finding the first answer and then moving on to the next getwork), I'm seeing a much different result, and quite frequently at that.

Notice that this getwork got 5 answers, and reported all of them to slush's server and were accepted.
29/01/2011 01:41:42, Getting new work.. [GW:29]
29/01/2011 01:41:57, 9d5069e9, accepted at 40.625% of getwork()
29/01/2011 01:41:58, 09fc7161, accepted at 43.75% of getwork()
29/01/2011 01:41:59, 27af8f3c, accepted at 46.875% of getwork()
29/01/2011 01:42:02, ad105798, accepted at 56.25% of getwork()
29/01/2011 01:42:12, 0e920ae8, accepted at 75.0% of getwork()

This sample shows that 3 answers were accepted, 1 invalid, then 1 more accepted.  Notice the Invalid answer is the same as the 1st accepted answer.
31/01/2011 06:46:34, Getting new work.. [GW:291]
31/01/2011 06:46:38, 41b15022, accepted at 6.25% of getwork()
31/01/2011 06:46:53, c597d2b5, accepted at 31.25% of getwork()
31/01/2011 06:46:59, 9babdadf, accepted at 50.0% of getwork()
31/01/2011 06:47:07, 41b15022, invalid or stale at 75.0% of getwork()
31/01/2011 06:47:11, 1ba08127, accepted at 87.5% of getwork()

This sample shows 4 getworks, all without a single answer, then just a single answer for 1 getwork.
31/01/2011 06:04:21, Getting new work.. [GW:212]
31/01/2011 06:04:54, Getting new work.. [GW:213]
31/01/2011 06:05:26, Getting new work.. [GW:214]
31/01/2011 06:05:59, Getting new work.. [GW:215]
31/01/2011 06:06:32, Getting new work.. [GW:216]
31/01/2011 06:06:43, d08c8f3d, accepted at 31.25% of getwork()
31/01/2011 06:07:04, Getting new work.. [GW:217]

These outputs are from a modified version of m0mchill's miner.

Changes include:
- Does not stop when first answer/hash/nonce is found.  It continues searching until all possible hashes have been searched.
- Removes the hard coded limit of 10 seconds for the askrate.  (My card does a single getwork in 30 seconds, so I just set the askrate to 3600 and forget it)

The advantages of these modifications are:
- The entire getwork request is processed, finding all possible answers to a getwork.
- Raises efficiency of getwork/answers to 100%.  Right now it's between 20-30%.
- Lowers the amount of requests and bandwidth used by pool servers.

Right now, I believe, huge portions of hashes in the getwork requests are not being processed because when just 1 answer is found, it moves on to the next getwork, or if the askrate has been reached, it then moves on to the next getwork.  Also, in comparing the results over time of the original miner and this modified version, they find the same amount of answers in the same amount of time (on average).  The original miner seems to generate about 260% more traffic for the pooled mining server and discards well over 60% of the getwork when a single answer is found, or when the askrate is reached.

SO....

If the chances are 1 in a zillion in finding the same answer for different getwork request, then how is this miner finding multiple answers for a SINGLE getwork, some are valid, some a DUPLICATES, how is this happening?

If all getwork request are unique, and possible answers are overlooked when the first answer is found, how does the pool/bitcoin handle searching the rest of that discarded hashes?  Does it?

Is this a problem with the miner, or the getwork patched version of bitcoin?

Have YOU searched the entire 2^32 possible hashes of a getwork before, or do you just trust the miner knows best when it stops short of checking all 2^32 possibilities?

If I'm correct in my findings, how many more blocks could the pool be finding by searching all of the 2^32 hashes per getwork?? 
How many blocks have we missed because we didn't search the entire getwork??

That's your food for thought.  Please think about this and provide good answers.
I'm expecting to release the modified client in the next week or so after I get the pre-compiled versions and sources wrapped up.

Looking forward to feedback on this one..

[geebus]

If I'm correct in my findings, how many more blocks could the pool be finding by searching all of the 2^32 hashes per getwork??  
How many blocks have we missed because we didn't search the entire getwork??

I've been asking m0mchill the same things in PM lately. It seems to me that multiple answers could be found for each 2^32 keyspace, and the likelihood that the first answer is the actual answer for the block is only '1 : The average number of answers found in a getwork() that results in answers', with some getwork() not resulting in answers at all.

This brings up the question; If some getwork() simply do not have answers, is this due to a 2^32 keyspace not being an actual equal portion of the block, or is this due to overlapping 2^32 getworks? ...and if it's overlapping, is that the cause of the invalid or stale?

i.e.:
I send a getwork() request and receive my 2^32 keyspace to work on.
XYZMiner belonging to another person gets another 2^32 keyspace in getwork
Do we share an overlap of 2^16 (arbitrary figure for the sake of example) in our respective keyspaces?

Meaning, am I getting invalid or stale because there are multiple people working on the same exact portions of keyspace? If so, isn't that an issue with the getwork patch?

I've also asked m0mchill about the askrate, and it seems his answer to why the client fixes the askrate is basically a "fuck it if it doesn't find it quick enough". Although, he speaks more eloquently than that.

He has also stated that, yes, we are ignoring large portions of the keyspace because we submit the first hash and ignore anything else in the keyspace, whether it's found in the first 10% of the keyspace, or the last 10% of the keyspace. He believes that this is trivial though, since you are moving on to another keyspace quick enough.

Here is a quote from my PM with him (m0mchill):

As for your example - yes, you are ignoring the potential remaining answers in this particular key space (less than 2^32). But you start to look for an answer in another fresh 2^32.

So, we're not searching the entire keyspace in a provided getwork. What if one of those possible answers you're ignoring is the answer to the block? You just fucked the pool out of a block.

I understand the premise that some getwork may not result in answers, but to the same extent, you could also get multiple valid answers (2, 5, 8, whatever) for a SINGLE getwork.

[slush]

Notice that this getwork got 5 answers, and reported all of them to slush's server and were accepted.
29/01/2011 01:41:42, Getting new work.. [GW:29]
29/01/2011 01:41:57, 9d5069e9, accepted at 40.625% of getwork()
29/01/2011 01:41:58, 09fc7161, accepted at 43.75% of getwork()
29/01/2011 01:41:59, 27af8f3c, accepted at 46.875% of getwork()
29/01/2011 01:42:02, ad105798, accepted at 56.25% of getwork()
29/01/2011 01:42:12, 0e920ae8, accepted at 75.0% of getwork()

I see no problem here.

This sample shows that 3 answers were accepted, 1 invalid, then 1 more accepted.  Notice the Invalid answer is the same as the 1st accepted answer.
31/01/2011 06:46:34, Getting new work.. [GW:291]
31/01/2011 06:46:38, 41b15022, accepted at 6.25% of getwork()
31/01/2011 06:46:53, c597d2b5, accepted at 31.25% of getwork()
31/01/2011 06:46:59, 9babdadf, accepted at 50.0% of getwork()
31/01/2011 06:47:07, 41b15022, invalid or stale at 75.0% of getwork()
31/01/2011 06:47:11, 1ba08127, accepted at 87.5% of getwork()

I don't know why miner uploaded same nonce twice, but it is fine that second attempt wasn't counted. Btw I'm expecting that nonce is rising from zero, so looks weird for me that nonces are mixed.

This sample shows 4 getworks, all without a single answer, then just a single answer for 1 getwork.
31/01/2011 06:04:21, Getting new work.. [GW:212]
31/01/2011 06:04:54, Getting new work.. [GW:213]
31/01/2011 06:05:26, Getting new work.. [GW:214]
31/01/2011 06:05:59, Getting new work.. [GW:215]
31/01/2011 06:06:32, Getting new work.. [GW:216]
31/01/2011 06:06:43, d08c8f3d, accepted at 31.25% of getwork()
31/01/2011 06:07:04, Getting new work.. [GW:217]

That's fine, isn't it?

Changes include:
- Does not stop when first answer/hash/nonce is found.  It continues searching until all possible hashes have been searched.

I thought that miner is already doing this. At least Diablo's isn't stopping when 'block' is found. Don't forget that this is tweak only for pool, because with crunching real-difficulty hashes, second nonce cannot be valid bitcoin block in any case...

- Removes the hard coded limit of 10 seconds for the askrate.  (My card does a single getwork in 30 seconds, so I just set the askrate to 3600 and forget it)

You should put it back. Otherwise you are crunching invalid work for long time (~ 30/600 seconds, so 5% of time is wasted). Plus pool now reject shares which cannot be valid bitcoin blocks, for more info read pool thread. So this isn't improvement in any case.

The advantages of these modifications are:
- Raises efficiency of getwork/answers to 100%.  Right now it's between 20-30%.
- Lowers the amount of requests and bandwidth used by pool servers.

how did you calculated 20-30%? It isn't correct.

Yes, less often getwork update saves some resources, but will make mining much less effective.

Personally I don't understand why are you posting this, because I already answered all those questions in PMs...

If I'm correct in my findings, how many more blocks could the pool be finding by searching all of the 2^32 hashes per getwork??  
How many blocks have we missed because we didn't search the entire getwork??

Nothing or not any significant amount. Learn basics of statistics and visit pool stats page...

[slush]

This brings up the question; If some getwork() simply do not have answers, is this due to a 2^32 keyspace not being an actual equal portion of the block, or is this due to overlapping 2^32 getworks?

There is no reason why there should be valid share in every getwork.

Do we share an overlap of 2^16 (arbitrary figure for the sake of example) in our respective keyspaces?

No overlapping, every getwork is unique. Read more how getwork() works. Especially the extranonce part.

Meaning, am I getting invalid or stale because there are multiple people working on the same exact portions of keyspace? If so, isn't that an issue with the getwork patch?

No. It may be because another bitcoin block was introduced in meantime between getwork() and submit. Then share from old block cannot be candidate for new bitcoin block. Read my last posts in pool thread. By the way, this is not pool/m0mchil miner related, it is how bitcoin works.

I've also asked m0mchill about the askrate, and it seems his answer to why the client fixes the askrate is basically a "fuck it if it doesn't find it quick enough". Although, he speaks more eloquently than that.

And he was true. Longer ask rate, more hashing overhead.

He has also stated that, yes, we are ignoring large portions of the keyspace because we submit the first hash and ignore anything else in the keyspace, whether it's found in the first 10% of the keyspace, or the last 10% of the keyspace. He believes that this is trivial though, since you are moving on to another keyspace quick enough.

By skipping some nonce space, you don't cut your probability to find valid share/block. There is the same probability of finding share/block by crunching any nonce.

So, we're not searching the entire keyspace in a provided getwork. What if one of those possible answers you're ignoring is the answer to the block? You just fucked the pool out of a block.

Definitely not. It is only nice optimization for pool to continue hashing of current job, it save some network roundtrips, but it basically does not affect pool success rate.

[geebus]

how did you calculated 20-30%? It isn't correct.

Yes, less often getwork update saves some resources, but will make mining much less effective.

I think he's basing the efficiency off of the number of submitted results to the number of getworks requested.

If 4 getworks are requested without having valid answers to submit back, and then on the 5th, it finds one answer and submits it back, then moves on without checking the remaining keyspace for more answers, you have a 20% efficiency.

However, if you request 4 getworks that don't result in answers, and then the 5th results in 5 answers, you have 100% efficiency.

So, over time (I'm sure it was looked at for more than just a few minutes) the averages from m0mchill's code would be ~20-30% where FairUser's fork would be closer to a 1:1 ratio, or 100% efficiency.

FairUser, feel free to correct me if my assumption is wrong here, but that seems the most logical way for me to break down what you're trying to say.

[slush]

If 4 getworks are requested without having valid answers to submit back, and then on the 5th, it finds one answer and submits it back, then moves on without checking the remaining keyspace for more answers, you have a 20% efficiency.

Maybe I'm wrong, but you are not paid for higher getwork/submit efficiency, but for finding valid blocks. So you are optimizing wrong thing . Maybe you can get 100% getwork/submit ratio, but you are crunching old jobs. But it is your choice and your hashing power.

[geebus]

No. It may be because another bitcoin block was introduced in meantime between getwork() and submit. Then share from old block cannot be candidate for new bitcoin block. Read my last posts in pool thread. By the way, this is not pool/m0mchil miner related, it is how bitcoin works.

If this is the case, and the assumption is also true that we are "statistically unlikely" to find the same hash twice inside of the same getwork, can you explain to me what FairUser is seeing here:

This sample shows that 3 answers were accepted, 1 invalid, then 1 more accepted.  Notice the Invalid answer is the same as the 1st accepted answer.
31/01/2011 06:46:34, Getting new work.. [GW:291]
31/01/2011 06:46:38, 41b15022, accepted at 6.25% of getwork()
31/01/2011 06:46:53, c597d2b5, accepted at 31.25% of getwork()
31/01/2011 06:46:59, 9babdadf, accepted at 50.0% of getwork()
31/01/2011 06:47:07, 41b15022, invalid or stale at 75.0% of getwork()
31/01/2011 06:47:11, 1ba08127, accepted at 87.5% of getwork()

Off of a SINGLE getwork, he received 3 valid answers, and then an invalid, followed by another valid. If this only happens when a new block is introduced between getwork and submission, why was the 5th hash accepted? Wouldn't it have been for the previous block? Shouldn't it have been rejected since the block hash on the answer didn't match the current block? Or is it possible that it's merely collision with SHA finding two identical (or similarly identical, taking difficulty into consideration) answers for the same getwork?

Likewise, can you explain to me exactly how ignoring a potentially large amount of hashes that could be the answer to the block doesn't effect the pool solving the block?

Your statistical analysis of the block data is accurate if you're taking into consideration that each getwork can only have one answer, and that the one answer being submitted is the only one that could be the correct answer for block. I think FairUser has shown quite plainly that multiple valid answers can be found within the same getwork.

I'll be happy to have it explained to me why I'm wrong to assume this though.

[slush]

This sample shows that 3 answers were accepted, 1 invalid, then 1 more accepted.  Notice the Invalid answer is the same as the 1st accepted answer.
31/01/2011 06:46:34, Getting new work.. [GW:291]
31/01/2011 06:46:38, 41b15022, accepted at 6.25% of getwork()
31/01/2011 06:46:53, c597d2b5, accepted at 31.25% of getwork()
31/01/2011 06:46:59, 9babdadf, accepted at 50.0% of getwork()
31/01/2011 06:47:07, 41b15022, invalid or stale at 75.0% of getwork()
31/01/2011 06:47:11, 1ba08127, accepted at 87.5% of getwork()

Did you noticed that it is the same nonce? It is absolutely fine that second attempt was rejected. As I wrote before, I don't know the reason *why* there was second attempt. Afaik, nonces should be sorted from 0 to ffffffff, but maybe there is some simple answer behind mixing of nonces (more solving thread or whatever). Maybe it was only reupload after lost connection, whatever. Maybe m0mchil will response this, I don't know miner internals.

why was the 5th hash accepted?

because - as I already wrote - share can be rejected for more reasons. The reason for this rejection is that miner uploaded same nonce twice. It is not relevant to any bug in getwork, skipping some nonce ranges or any other weird stuff you are arguing here.

Likewise, can you explain to me exactly how ignoring a potentially large amount of hashes that could be the answer to the block doesn't effect the pool solving the block?

Because with skipping some nonce range, shit happen. In fact, you are skipping zillions of existing nonces. How can you live with that? :-)

I think FairUser has shown quite plainly that multiple valid answers can be found within the same getwork.

And I said it is nice, but not necessary optimization of miner. This only optimize network latencies, because miner is asking less often, but it does not improve probability that share will be found.

[FairUser]

This brings up the question; If some getwork() simply do not have answers, is this due to a 2^32 keyspace not being an actual equal portion of the block, or is this due to overlapping 2^32 getworks?

There is no reason why there should be valid share in every getwork.

OK.

Do we share an overlap of 2^16 (arbitrary figure for the sake of example) in our respective keyspaces?

No overlapping, every getwork is unique. Read more how getwork() works. Especially the extranonce part.

That's what I thought.  Just wanted to make sure.

Meaning, am I getting invalid or stale because there are multiple people working on the same exact portions of keyspace? If so, isn't that an issue with the getwork patch?

No. It may be because another bitcoin block was introduced in meantime between getwork() and submit. Then share from old block cannot be candidate for new bitcoin block. Read my last posts in pool thread. By the way, this is not pool/m0mchil miner related, it is how bitcoin works.

Not true.  Look again.
31/01/2011 06:46:34, Getting new work.. [GW:291]
31/01/2011 06:46:38, 41b15022, accepted at 6.25% of getwork()
31/01/2011 06:46:53, c597d2b5, accepted at 31.25% of getwork()
31/01/2011 06:46:59, 9babdadf, accepted at 50.0% of getwork()
31/01/2011 06:47:07, 41b15022, invalid or stale at 75.0% of getwork()
31/01/2011 06:47:11, 1ba08127, accepted at 87.5% of getwork()

3 were accepted, then the 4th was invalid.  So if this was invalid because the block count went up by one (and work from old blocks are now discarded as invalid), how was the 5th answer from this now *old* work, why was it accepted?  Your logic doesn't work here BECAUSE the 5th answer was accepted!  And, the miner doesn't know that the block has increased until it makes the next getwork request.

I've also asked m0mchill about the askrate, and it seems his answer to why the client fixes the askrate is basically a "fuck it if it doesn't find it quick enough". Although, he speaks more eloquently than that.

And he was true. Longer ask rate, more hashing overhead.

I wouldn't call it "more" hashing overhead, since it's the same number of kHash/s regardless of *what getwork* it's on.  My kHash/s doesn't change just because I'm on a different getwork.

He has also stated that, yes, we are ignoring large portions of the keyspace because we submit the first hash and ignore anything else in the keyspace, whether it's found in the first 10% of the keyspace, or the last 10% of the keyspace. He believes that this is trivial though, since you are moving on to another keyspace quick enough.

By skipping some nonce space, you don't cut your probability to find valid share/block. There is the same probability of finding share/block by crunching any nonce.

I agree that the probability of finding the share/block is the same for every getwork....IF YOU LOOK AT THE ENTIRE GETWORK.  Skipping over half the possibilities when just the first hash is found screws with your idea for a perfect 1:1 ratio for getwork/found hashes.  You can't ever expect to see (or find) the entire puzzle (the block) when you are choosing to ignore any part (the skipped hashes in a getwork) of that puzzle.

So, we're not searching the entire keyspace in a provided getwork. What if one of those possible answers you're ignoring is the answer to the block? You just fucked the pool out of a block.

Definitely not. It is only nice optimization for pool to continue hashing of current job, it save some network roundtrips, but it basically does not affect pool success rate.

That is true! 
I'm seeing the same amount of accepted hashes in a given hour with both miners.  The only thing this does is increase efficiency by:
1) Not ignoring nonces of the getwork when a hash is found
2) Not stopping part way through the getwork because the askrate was triggered.

[geebus]

If 4 getworks are requested without having valid answers to submit back, and then on the 5th, it finds one answer and submits it back, then moves on without checking the remaining keyspace for more answers, you have a 20% efficiency.

Maybe I'm wrong, but you are not paid for higher getwork/submit efficiency, but for finding valid blocks. So you are optimizing wrong thing . Maybe you can get 100% getwork/submit ratio, but you are crunching old jobs. But it is your choice and your hashing power.

It's not crunching old jobs, it's crunching the same job all the way instead of moving on without potentially finding other valid answers.

It's more effective to crunch through the entire keyspace in (2^32)/MyHashRate seconds than it is to grab new getworks every 5 seconds and potentially not find an answer at all because I'm only checking a very small portion of the keyspace before moving on to the next getwork.

Say I get 50 million hashes a second on my GPU.
2^32 / 50,000,000 = 86 seconds to process an entire keyspace.
If my askrate is set to 5 seconds, I'm only checking 5.82% of each keyspace before moving on and assuming the getwork holds no answers.

You're potentially ignoring 94.18% of answers. Numbers obviously vary based on the speed of the GPU, but for a 5 second askrate to be effective, you would need 859 Million Hashes/s to process a keyspace of a single getwork, and even then the way m0mchills code is written, once it finds the first answer, it moves on to the next getwork anyway. This is flawed.

[slush]

I wouldn't call it "more" hashing overhead, since it's the same number of kHash/s regardless of *what getwork* it's on.  My kHash/s doesn't change just because I'm on a different getwork.

You can call it whatever, but with long getwork period, you are hashing shits for many % of time :-).

You can't ever expect to see (or find) the entire puzzle (the block) when you are choosing to ignore any part (the skipped hashes in a getwork) of that puzzle.

Well, getwork is not a puzzle. It is random walk, when you hit valid share time to time. Nonces are just numbers. It's irrelevant if you are trying to hash 0xaaaa or 0xffff. The probability that you hit valid share is still the same.

1) Not ignoring nonces of the getwork when a hash is found

Well, this is the only point which make sense. Diablo already implemented this and if it isn't in m0mchil's, it would be nice to implement it, too. But it's definitely on m0mchil decision, not on us.

Also sorry for some impatient responses, but I'm responding those questions to pool users almost every day and it become little boring . It isn't anything personal. To be honest, it isn't long time ago when I had very similar questions as you have right now. But thanks to m0mchil, Diablo and few other people on IRC, now I know how much wrong I was .

[FairUser]

If 4 getworks are requested without having valid answers to submit back, and then on the 5th, it finds one answer and submits it back, then moves on without checking the remaining keyspace for more answers, you have a 20% efficiency.

Maybe I'm wrong, but you are not paid for higher getwork/submit efficiency, but for finding valid blocks. So you are optimizing wrong thing . Maybe you can get 100% getwork/submit ratio, but you are crunching old jobs. But it is your choice and your hashing power.

Yes, I will be crunching old jobs for about 30 seconds.  Already working on a mod to check my local bitcoind between "work" (32 of them) in the python code for the current block.  You and I both know what our bitcoind's can do.   This way we can stop within 1 second on a block update and get a new getwork.  So quit thinking in terms of OLD JOBS.  Whether it's old or new, I'm talking about the ability to search the 2^32 hashes.

Your server just happens to be the only server that is running a public pool, hence, it might feel like I'm picking on it...but I'm not.  All these changes helping increase the probability of the pool as a whole to finding the block in the getwork instead of ignoring most of the getwork when just a single answer is found.  Maybe Diablo is doing it different (i hate java personally so I haven't even looked at the code), m0mchill's is ignoring part of the 2^32 POSSIBLE answers after finding just 1.

More blocks == more pay for everyone

OK Slush, do this. 

In your server stats, I want you to list:
1) The number of get requests for the CURRENT round
2) The number of submitted hashes (both ACCEPTED and INVALID/STALE listed separately) for the CURRENT round.

If you wanted to increase the accuracy of this, separate the INVALID/STALE hashes based on the reason they were rejected, ie (WRONG BLOCK) or (INVALID/ALREADY SUBMITTED).

Then take (# of getwork this round)/(# of accepted/invalid(already submitted))*100 and publish that number in real time.
That's how you check the efficiency of the pool's ability to search all hashes for each getwork sent out. 

This will show if you are really get that 1:1 ratio of getwork/solved hashes.

Can you do that?  Publish those numbers on the server stats page? 
Then we all can see what the efficiency of getwork/solved hashes is.

Also, you can't make up for the inefficiency of quiting the search after 1 submitted answer, or askrate tigger, with increasing the speed at which you get work.
That only increases the speed of your inefficiency, it doesn't solve the problem of not looking for more answers.