OFFICIAL CGMINER mining software thread for linux/win/osx/mips/arm/r-pi 4.10.0

[DrHaribo]

You clearly do NOT want to tie difficulty to the work in Stratum, as you've stated numerous times.  

Apparently he changed his mind. Look in the Stratum thread. I am implementing server-side Stratum now and I'm going to have difficulty tied to work. I'll send set_difficulty before the work, unless it is the same diff as the previous work, in which case I'll skip sending set_difficulty.

Oh sorry. Send your first set of parameters for mining.notify before sending the ack for mining.authorize.

That's not helping either. I'm wondering if it has to do with my formatting being different (from other Stratum pool implementations).

Code:

recv: {"id": 0, "method": "mining.subscribe", "params": []}
send: {"result":[["mining.notify","01"],"01020304",4],"error":null,"id":0}
send: {"method":"mining.set_difficulty","params":[1],"id":1}
send: {"method":"mining.notify","params":[["1-c9","0000000000e3441ba33cf51d11a8aef12e8ae6d01a5bd54e8ec7cc9a9c22837b","01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff56037c9600094269744d696e746572062f503253482f2cfabe6d6d8c53896d9815717a31ffc5135ccd79527cec9e360aba3738261e0a338a827eb60100000000000000136465760100000000000000c900000000000000ffffffff0100f2052a010000001976a914616c05127d60e61407f52bce21a4f894268cd89588ac00000000","",[],"00000002","1c018167","50b292ad",false]],"id":1}
recv: {"id": 1, "method": "mining.authorize", "params": ["Test_Test", "Test"]}
send: {"result":true,"error":null,"id":1}
recv: {"id": 2, "method": "mining.authorize", "params": ["Test_Test", "Test"]}
send: {"result":true,"error":null,"id":2}

The strange thing is that now it tries to authorize twice, with a 10 second pause in between, then crash.

[1495736899]

1495736899

[1495736899]

1495736899

[1495736899]

1495736899

[1495736899]

1495736899

[1495736899]

1495736899

[slush]

Fuck Off, really?  That's where we are at now?

I've been just upset by your tone. Maybe you didn't noticed the aggression in your posts.

You're absolutely right I'm not preparing a proprietary protocol!

Umm, it's a bit offtopic here, but is protocol for BFL ASICs already documented somewhere?

Why are you waiting for my proposal if you already have one that fits your needs?  That doesn't even make sense!

Because you're bashing the Stratum, without giving any solution!

I did (https://bitcointalk.org/index.php?topic=108533.msg1344256#msg1344256).  You did not respond, so what do you want me to do, pitch a fit in the thread?

Well, it was addressed to ckolivas and cgminer implementation...

don't believe you wrote such long post before you even read my post addressing this issue. Btw protocol don't define *any* window, it's up to server side implementation.

I don't know what you mean by this so it's hard to respond.

Fair enough. My english is strange, I know, but you can always ask for further explanation...

If the difficulty were tied to the work, as it should be

It already is, you just didn't notice it. I finally updated the documentation, which is backward compatible, but practically solves your issue: https://bitcointalk.org/index.php?topic=108533.msg1356993#msg1356993

From your responses in that thread, you seem to think it is ok to discard valid work simply because the difficulty rises in the middle of a nonce.

I don't think it's ok to discard valid work, I just didn't know about proper solution. But I think that I already found it, all people which I asked were fine with it, so I think we're now talking about non issue...

I did not say there were any bottlenecks in Stratum .... But this isn't really a Stratum issue

Fine to know it.

Again, see above. To overcome the flaw in the protocol, apparently the solution BTCGuild and OzCoin have come up with is to make a large hysteresis window to prevent frequent difficulty changes and to minimize the impact of lost shares.

No, at least not for BTCGuild. AFAIK "difficulty windows" were implemented by Eleuthria because he creates jobs pool-wide (exactly as I'm doing on my pool now, although my development code can generate ondemand jobs as well), so creating jobs on-demand for the miner is difficult. That's not related to difficulty manipulation at all.

Ok, fair enough.  Maybe Con can chime in then and tell me what the difference is between EMC and Slush's pool as viewed by CGMiner?  All this is ultimately due to CGMiner apparently having trouble with EMC because the difficulty changes too rapidly - but if that's the case, wouldn't the same problem be evident on Slush's pool?  If not, why not?  I'll be glad to fix it if there's a problem somewhere.

To be honest, vardiff isn't on my production servers yet. Still there's no obvious reason why miner should have problems with often changes of the difficult. Did you tried poclbm instead of cgminer to check if this behaviour is miner-dependent?

What?  Of course there is!  Tie difficulty to the work sent out.  When the work is returned, compare it to the difficulty sent out, if it's under the target you're good to go, if not, reject it.  Simple, easy, obvious and straight forward.

Umm, well, for comparing the difficulty, you firstly have to compute doublesha256 of submitted share, what I call "share validation". Is there any shortcut how to check difficulty of submitted share? Not necessary on Stratum, but maybe in GBT? I'm not aware of any easy solution.

you come back at me with Fuck Off because I answered your question.

No, that fuck off was related to bolded part of your sentence. About FUD that I created Stratum behind closed doors to dominate Bitcoin mining. I'm free person and I can develop any protocol what I want. And you're free person and you can ignore results of my work. I don't have any super power to enforce my solution to be accepted by the community. Did you noticed that Satoshi also created Bitcoin behind closed doors and then he released it as opensource?

[slush]

You clearly do NOT want to tie difficulty to the work in Stratum, as you've stated numerous times. 

Apparently he changed his mind. Look in the Stratum thread. I am implementing server-side Stratum now and I'm going to have difficulty tied to work. I'll send set_difficulty before the work, unless it is the same diff as the previous work, in which case I'll skip sending set_difficulty.

Yes, this is one of perfectly valid solutions.

Solution which I'm implementing for my pool is that code managing difficulties is completely independent from broadcasting the jobs, and it can send set_difficulty at any time. Only when the difficulty step will be big enough (typically on the beginning of the connection with high hashrate), difficulty-managing code will generate per-miner job and sends it with "clean_jobs=True", enforcing miner to change difficulty immediately and prevent pool overflooding. With this solution, there's no need for any "difficulty windows".

[MrTeal]

I did not say there were any bottlenecks in Stratum, but your response further lends credence to the fact that I don't think you understand the scale of what is about to be unleashed upon the mining world.  You are still thinking in the old mining terms were 100 TH is virtually inconceivable (Given that the whole network is a fraction of that right now).  Within less than a year, 100 TH will be well within the reach of modest professional miners and I will not be surprised to see the largest miners wielding a significant fraction of a PH.  But this isn't really a Stratum issue, beyond the fact that the discarded shares will scale in a linear fashion with the hashrate and miners will be losing a non-trivial amount of shares for no reason when mining with Stratum on current implementations.

That's interesting. At your current prices, 100TH/s would be a $2,000,000 investment and half a PH/s would be $10M. Considering that $2M invested now would make you far and away the largest miner on the network, it seems you either expect investment in mining hardware to go way up or the price per GH/s to plummet. You were asked on the BFL forums about your plans to drop prices shortly and said you had no plans for a price cut, but if you're expecting 100TH/s to be a modest pro miner in the future you must surely be expecting some large changes from the current ASIC status quo.

[Luke-Jr]

I for one would prefer to see all this back-and-forth spent improving stratum or some new protocol.
How goes preparing the first BIP draft, slush?

[-ck]

Ok, fair enough.  Maybe Con can chime in then and tell me what the difference is between EMC and Slush's pool as viewed by CGMiner?  All this is ultimately due to CGMiner apparently having trouble with EMC because the difficulty changes too rapidly - but if that's the case, wouldn't the same problem be evident on Slush's pool?  If not, why not?  I'll be glad to fix it if there's a problem somewhere.

Actually, I wasn't comparing you to slush's pool because ironically... slush's pool doesn't do vardiff yet. ozcoin and btcguild are the other pools I can compare to. Make what you want of that fact.

So the issue really comes down to the round trip of network time between pool sending out diff and miner receiving it, and the miner sending out a share. Most of us have round trips of ~500ms across real internets, though some of you are seeing ping times of <50ms but this does not translate to a processing, submission and return time of 50ms. Luckily c is fast so cgminer intrinsically doesn't add *that* much processing time, but it does add time.

There are 4 distinctly different scenarios with changing diff separate from work.
A. Diff rises before cgminer submits lower diff share and cgminer is aware of it
B. Diff falls before cgminer submits now adequate diff share and cgminer is aware of it
C. Diff rises in the round-trip window period as cgminer has already submitted a share below difficulty and cgminer is not aware of the new diff
D. Diff falls in the round-trip window period as cgminer has discarded a share that would now be adequate as cgminer is not aware of the new diff

cgminer can only do something about scenario A and B. C and D are entirely out of its control. The way I decided to manage these situations is based on maximising the possible paid share submission.

Scenario A:
cgminer can decide to retarget for the higher difficulty and not submit the share. It currently does NOT do this, instead keeping the difficulty target at the original lower value, in case pools have a grace period at the lower difficulty. It risks rejects but no shares are lost in this way. This is what causes more rejects on EMC when the difficulty is retargeted often. However no paid work is lost here.

Scenario B:
cgminer can decide to retarget for the lower difficulty and submit the share. cgminer does this since clearly this share is now work it will get paid for. No paid work is lost here, and no rejects are induced.

Scenario C:
cgminer can't do anything about it, but this registers as a reject. Any increase of diff by a pool can elicit this, and the more often the difficulty is raised, the more rejects will be seen. Once again, no paid work is lost here.

Scenario D:
cgminer can't do anything about it, and this work is lost. Any decrease of diff by a pool can elicit this, and the more often the difficulty is lowered, the more shares are lost. No rejects are induced.


So there is a round-trip window period for scenario D where shares are potentially lost. If you have a 500ms round trip window, then you lose .5 seconds worth of possible lower difficulty shares with each drop. The more decreases, the more lost.


The only changes I can make to cgminer's default behaviour to help here is scenario A, and retarget to the higher difficulty. Note that the emphasis is always on submitting shares that are possibly paid for over minimising rejects. B is fine. Nothing can be done about the rejects in C, but no paid shares are lost. D is the real issue that nothing can be done about, and unless difficulty is tied to the work, it ends up being dependent on how often the pool ends up changing difficulty that determines how much work is lost, and secondarily the visible rejects.

I agree that retargeting with vardiff is important or floods of low diff shares will be a real issue. However retargeting for small changes in difficulty based on share submission, which is subject to large amounts of variance, will see lots of changes of difficulty and the inherent rise in rejects and lost shares.

EDIT: Note that my scenario A management is in line with slush's change to the protocol stipulating changes to diff apply to next work sent by server, not existing work.

[slush]

How goes preparing the first BIP draft, slush?

Not very well. I just found https://www.btcguild.com/new_protocol.php written by Eleuthria and I think this can be quite nice start for the BIP. In contrast to Eleuthria, I'm pretty bad writer and my expression capabilities are a bit limited.

[-ck]

You clearly do NOT want to tie difficulty to the work in Stratum, as you've stated numerous times.  

Apparently he changed his mind. Look in the Stratum thread. I am implementing server-side Stratum now and I'm going to have difficulty tied to work. I'll send set_difficulty before the work, unless it is the same diff as the previous work, in which case I'll skip sending set_difficulty.

Oh sorry. Send your first set of parameters for mining.notify before sending the ack for mining.authorize.

That's not helping either. I'm wondering if it has to do with my formatting being different (from other Stratum pool implementations).

Code:

recv: {"id": 0, "method": "mining.subscribe", "params": []}
send: {"result":[["mining.notify","01"],"01020304",4],"error":null,"id":0}
send: {"method":"mining.set_difficulty","params":[1],"id":1}
send: {"method":"mining.notify","params":[["1-c9","0000000000e3441ba33cf51d11a8aef12e8ae6d01a5bd54e8ec7cc9a9c22837b","01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff56037c9600094269744d696e746572062f503253482f2cfabe6d6d8c53896d9815717a31ffc5135ccd79527cec9e360aba3738261e0a338a827eb60100000000000000136465760100000000000000c900000000000000ffffffff0100f2052a010000001976a914616c05127d60e61407f52bce21a4f894268cd89588ac00000000","",[],"00000002","1c018167","50b292ad",false]],"id":1}
recv: {"id": 1, "method": "mining.authorize", "params": ["Test_Test", "Test"]}
send: {"result":true,"error":null,"id":1}
recv: {"id": 2, "method": "mining.authorize", "params": ["Test_Test", "Test"]}
send: {"result":true,"error":null,"id":2}

The strange thing is that now it tries to authorize twice, with a 10 second pause in between, then crash.

I'm lost as to what's wrong then. It seems you're running windows which makes debugging a pain as well. Are you running the latest 2.9.5? 2.9.4 was crashing on extracting gbt information from your pool occasionally (obviously unrelated to stratum). There are debug builds here along with instructions for debugging on windows: http://ck.kolivas.org/apps/cgminer/debug/ .

[awatson824]

Ckolivas, is it my computer or graphics card that is giving me 46 stales in 8 hours?
Not being mean just asking cause I don't get it. :/

[slush]

send: {"method":"mining.notify","params":[["1-c9","0000000000e3441ba33cf51d11a8aef12e8ae6d01a5bd54e8ec7cc9a9c22837b","01000000010000000000000000000000000000000000000000000000000000000000000000fffff fff56037c9600094269744d696e746572062f503253482f2cfabe6d6d8c53896d9815717a31ffc5 135ccd79527cec9e360aba3738261e0a338a827eb60100000000000000136465760100000000000 000c900000000000000ffffffff0100f2052a010000001976a914616c05127d60e61407f52bce21 a4f894268cd89588ac00000000","",[],"00000002","1c018167","50b292ad",false]],"id":1}

! :-)

[-ck]

send: {"method":"mining.notify","params":[["1-c9","0000000000e3441ba33cf51d11a8aef12e8ae6d01a5bd54e8ec7cc9a9c22837b","01000000010000000000000000000000000000000000000000000000000000000000000000fffff fff56037c9600094269744d696e746572062f503253482f2cfabe6d6d8c53896d9815717a31ffc5 135ccd79527cec9e360aba3738261e0a338a827eb60100000000000000136465760100000000000 000c900000000000000ffffffff0100f2052a010000001976a914616c05127d60e61407f52bce21 a4f894268cd89588ac00000000","",[],"00000002","1c018167","50b292ad",false]],"id":1}

! :-)

Thanks slush.

[-ck]

How goes preparing the first BIP draft, slush?

Not very well. I just found https://www.btcguild.com/new_protocol.php written by Eleuthria and I think this can be quite nice start for the BIP. In contrast to Eleuthria, I'm pretty bad writer and my expression capabilities are a bit limited.

Whatever you do, please EXPLICITLY say what significant bit and byte order the data is in

[mdude77]

Ckolivas, is it my computer or graphics card that is giving me 46 stales in 8 hours?
Not being mean just asking cause I don't get it. :/

what pool are you pointing to?

M

[sharky112065]

Ckolivas, is it my computer or graphics card that is giving me 46 stales in 8 hours?
Not being mean just asking cause I don't get it. :/

How could he possibly answer that given that you gave so little information about your setup. Try giving him/us more details. What pool? What version of Cgminer? What is your hardware setup? ...

Note: if you are on p2pool there was a recent hard fork and issues should stabilize over time.

[slush]

Thanks slush.

Since bitcoin donations have been discovered, it is not necessary to be grateful. :-P

[awatson824]

Ckolivas, is it my computer or graphics card that is giving me 46 stales in 8 hours?
Not being mean just asking cause I don't get it. :/

How could he possibly answer that given that you gave so little information about your setup. Try giving him/us more details. What pool? What version of Cgminer? What is your hardware setup? ...

Note: if you are on p2pool there was a recent hard fork and issues should stabilize over time.

BtcGuild, latest version and I tried with my 1 HD Radeon 6670

[slush]

BtcGuild, latest version and I tried with my 1 HD Radeon 6670

There may be many factors, but one of the main reason is roundtrip to the server. Can you check ping to btcguild?

[awatson824]

BtcGuild, latest version and I tried with my 1 HD Radeon 6670

There may be many factors, but one of the main reason is roundtrip to the server. Can you check ping to btcguild?

I have 79 ms to it if thats what your asking.

[slush]

Yes, and it is pretty low latency.

[sharky112065]

Ckolivas, is it my computer or graphics card that is giving me 46 stales in 8 hours?
Not being mean just asking cause I don't get it. :/

How could he possibly answer that given that you gave so little information about your setup. Try giving him/us more details. What pool? What version of Cgminer? What is your hardware setup? ...

Note: if you are on p2pool there was a recent hard fork and issues should stabilize over time.

BtcGuild, latest version and I tried with my 1 HD Radeon 6670

And how many accepts? What percentage is your rejects?