Bitcoin Forum

Discussion goes here  :)

2013 Feb, after discussion, mining.subscribe was extended to take 2 arguments, so now it's mining.subscribe(String useragent, sessionid) where sessionid is optional, but if provided must match a previous session's mining.notify subscription id. If the server allows resuming a session, it will give you back the same subscription id and extranonce information.

BFGMiner (at least) will also send mining.suggest_target("hex target") upon connection, if the user has a preferred target.

Why would it do this?
There is already mining.suggest_difficulty(difficulty)
Someone didn't read the docs and added it to their non-standard miner clone?

No, there isn't. Nothing implements this, and it isn't documented anywhere.

Using a target fixes the inherent problems with using difficulty as a Number:

• There is no agreement over which difficulty measurement is to be used. The official spec says bdiff; BTCGuild uses pdiff; various scrypt pools use Ldiff
• Some common targets (such as pdiff 1) cannot be accurately conveyed without huge data sizes
• Implementing conversion to/from difficulty accurately requires a bignum library, so often (eg, *gminer) it is just approximated.

Let me be the first to say thank you first designing, implementing and now collecting this information for all.  Much more concise reading than https://bitcointalk.org/index.php?topic=108533.660 or what else I've found on the pools pages.

If you say something that contradicts facts in front of your face, oddly enough, that just makes you look foolish.
Try a little sense in your posts.

One of the posts above, shows where it was documented, over a year ago:
https://bitcointalk.org/index.php?topic=108533.msg1543341#msg1543341

Edit: and of course on the same page:
https://bitcointalk.org/index.php?topic=108533.msg1543185#msg1543185

Forum posts are not documentation.

You've been flipping too many burgers at McDonalds - it's softening your brain and you are forgetting things:
https://bitcointalk.org/index.php?topic=108533.msg1543185#msg1543185

But I guess that's to be expected.

Hard to see how I forget something I never read in the first place.
I don't see anything relevant there anyway. Con agreed to implement slush's spec and then never did. How is this relevant?

Seems like a valid concern to me since it was described over a year ago and yours isn't documented anywhere except as part of your code. Of course you're free to implement whatever you want on top of stratum. Cgminer happily works with any arbitrary diffs and does not have problems with the accuracy of the shares it returns in response, using true diff 1 as the base for all stratum operations. I wasn't aware there was any controversy about the move to true diff1; that old simplification only affected getwork. Since work in stratum is even defined in terms of difficulty, it makes even less sense to then request a difficulty as a target. I don't see a problem with you using it for your software and your pool but I can't see why it should be seen as part of the stratum spec.

From a protocol development perspective, stratum being defined in terms of difficulty is an embarrassing flaw of the sort that would have been avoided if it hadn't been developed behind closed doors. ... The fact that it's a floating point value which cannot precisely specify the actual values used in bitcoin is a pretty big facepalm.  All of this really should have been using bits like the bitcoin protocol does both for precision and consistency sake (or an expanded target— to accommodate legacy stupidity, but then again people will just get the byte order confused and complain it wastes bandwidth. :) ).

In any case, should have beens aside— what am I missing here?  There is no formal specification for stratum at all, slush was apparently sucked into Trezor land before publishing it.  AFAICT nothing is implementing the difficulty based message that was suggested way in the past, but at least luke's stuff has implemented the target based approach.  Whats to debate?

It wasn't developed behind closed doors, you are just regurgitating a lie spread by Luke.

+1, suggest splitting side-discussion off to leave the thread to just the documentation posts.

This post should have been on the other side of the split.

As someone who's working on a Stratum implementation right now, I must say this is a VERY narrow view of reality.

I've been using the following resources to gain my knowledge of the protocol:
https://bitcointalk.org/index.php?topic=55842.0
https://bitcointalk.org/index.php?topic=108533.0
http://mining.bitcoin.cz/stratum-mining
https://www.btcguild.com/new_protocol.php

Notice how 50% of those links were forum posts   :D

But, I'm guessing none of the discussion here (https://bitcointalk.org/?topic=108854) counts as documentation either and can be entirely ignored  ;)

Did you miss this sticky post?
https://bitcointalk.org/index.php?topic=557866.0
It's as close as it gets to the repository of the collated official documentation which was never written, only the draft proposal by slush which is quoted there.

I DID!!!!

Thank you!

Since we're talking drafts, and implementation. Both that sticky, and I think Slush's website, mention HTTP Poll as an optional transport mechanism. Do any mining apps support HTTP Poll? Standard TCP sockets seems to be the prevailing implementation.

You're free to reference whatever you want, but that doesn't change what "documentation" means.

Feel free to ignore the forum posts.
Unlike stratum, getblocktemplate is properly documented in Bitcoin standards and the wiki.

https://en.bitcoin.it/wiki/Getblocktemplate
https://en.bitcoin.it/wiki/BIP_0022
https://en.bitcoin.it/wiki/BIP_0023

The initial development was secret to the public before it was up on slush's pool along with the python proxy. I'd always understood this to be a (anti- :))competitive move.

There remains no BIP describing it, there was no design discussion prior to its release on bitcoin-development. Justifying the closed development, Slush wrote (in the second email my mailbox ever received mentioning stratum mining) "There's no requirement to have BIP for everything what people do. Stratum is NOT related to bitcoin protocol or bitcoin implementation, it is just custom, pooled-mining extension and bitcoin network doesn't need to know about Stratum existence at all."  Perhaps you were somehow an insider on it— if so, why didn't you discourage some of the poor choices in it? :)  But from my own perspective it really was developed in secret and, in my opinion, carries some of the predictable flaws of a protocol developed without broad input. It's not the end of the world, in any case. Far worse has been done elsewhere.

No doubt Luke will claim ignorance of the relevance of this link to the discussion, but it is quite obviously relevant to anyone with the intelligence level necessary to understand ...
https://bitcointalk.org/index.php?topic=108533.msg1180529#msg1180529

Yeah it was so secret that only people who visited bitcointalk and github could find out about it ....................

The basis for it (not as a mining protocol) was started in December 2011 ...
https://bitcointalk.org/index.php?topic=55842.0

https://github.com/slush0

He also made mention of the mining protocol here, but, no doubt, there are other details around:
https://bitcointalk.org/index.php?topic=61131.msg702097#msg702097

As I said, though you hid the end of it:

If only it turned out that way. Instead, it doesn't do any of that at all, and displaced the less efficient GBT stuff that did, in theory, have that capability. :(

None of this however involved any public discussion of what was actually developed for mining, alas.

Meanwhile we have the retard doing rubbish like this to try blocking the dissemination of information about Stratum:
https://en.bitcoin.it/w/index.php?title=Stratum&action=historysubmit&diff=46202&oldid=46194

I cannot rationalize those changes, short of malice.

Meanwhile, having a BIP is clearly not as important as has been suggested.

The code for BIP22 for the centralised GBT mining protocol
(yes, saying it is decentralised is also a lie being spread by Luke and gmaxwell)
was modified by Luke before submitting it to bitcoin

He implemented code that is not part of BIP22 and put it into bitcoin - for support of botnet mining.
https://github.com/bitcoin/bitcoin/issues/1985

Anyway, looks like he still can't handle having the stratum pool protocol mentioned first on the Stratum page and likes to list a bunch of meaning less complaints about it:
https://en.bitcoin.it/wiki/Stratum
The "Displacing GBT" complaint has to be the funniest - how dare someone displace GBT with something that is so much better than it.
Pity it also contains outright lies. But that is to be expected of Luke.

Is mining.get_transactions mandatory to be implemented on pool side? Is it safe to simply ignore this client request (send back nothing)?

Question for you Luke:
Does your BFGMiner correctly handle variable sizes of extranonce1 and extranonce2? Can extranonce1 be 5 bytes, extranonce2 3 bytes and shares provided valid?
As you may know already, cgminer from certain X up to certain version Y has a bug there, causing all shares to be rejected.

It can be ignored, but should send back an error if not supported.

Yes, any reasonable value (bounded by memory size I think) should work fine, but BFGMiner will only actually use at most 40 bits of extranonce2 right now (32 in most cases).
Note that the stratum proxy feature requires at least 24-bit extranonce2 (failure to meet this requirement disconnects all clients and rejects new ones).

Meanwhile, a permanent change that is part of the solution to the client.reconnect problem at the moment should be:
Require the pool domain and the reconnect domain to match.

There is no need for hard coding IP addresses, since anyone on the planet can own a domain for $10 a year :P
The reconnect has no need to use an IP address either - for the same reason - DNS records require no effort.

... and anyone wishing to use different domains for different located pools, you simply add a DNS record in the domain of the pool doing the reconnect to point to the other IP address.

DNS poisoning problems are not relevant to this since if there is a DNS problem then the miner can already be directed away from the pool when you first connect.

Sounds fine to me.

kano/ckolivas,

I'm working with the guys at NiceHash.com to try and resolve a few issues with how they are managing miners with their stratum pool.  In debugging this, I've been reading through the code that implements the stratum client - util.c - and trying to better understand the stratum protocol.

What I'm not quite clear on is the loop/state machine that you use in a mining client to "poll" a pool and attempt to connect to get work.

The NiceHash.com pool uses the password parameter in a mining.authorize to pass in a requested "threshold" for mining payouts.  (e.g. I might only want to mine if there are contracts for a minimum of 0.07BTC/TH/Day so my password becomes "p=0.07")  When the mining.authorize occurs, they will accept or error my auth based on that threshold comparison.

This is working ... mostly.

The problem we are seeing is that at some later point of time, the miner seems to "hang" or believe the pool is alive.  I'm running kano's builds on my AntMiners, and periodically during the day I'll find them attached to the pool, but not getting any work.  They think the pool is alive ... the pool is saying they sent an error on the auth.

I am debugging to see if this is pool related, miner related, or protocol related.  (Or a little of several?)

What I wanted to know is:

Within cgminer, once a pool is determined to be "dead", what is the delay before the pool is tried again, and what operations (Requests) are retried to "test" the pool for aliveness?  Where would I find the code that handles that delay and retrying?

I'm writing here, as I tried to locate and study the protocol documentation to determine what the "proper" process ought to be ... but didn't find too much on the subject.  :-)

I'm wondering if a miner like cgminer would be doing :  mining.subscribe -> mining.authorize -> error/delay/startover-at-subscribe?

If this is the case, is there any reason to believe that after looping like this - getting errors for a half day or so - the miner might hang?

I'm also glad to take this conversation elsewhere if this is not the appropriate thread.

PLEASE PLEASE PLEASE

REALLY

mining.subscribe needs extension with a third parameter being the requested URL.

WHY IS THIS NOT IMPLEMENTED?

Hello

Considering the nature of a pool that "mine" Bitcoins I think we will suffer DDOS constantly and every time more aggressive sophisticated and bigger.

In my opinion the Srtatum protocol may help with some improvements to mitigate the DDOS

May be some features like the Apache/HTTP can be implemented like the Redirect or Temporary Redirect where after connecting the node tells the miner to connect to a different IP:PORT. Before that redirect may require to execute some handshake process that needs to be a heavy process at miner side so will be complex for a DDOS complete the handshake without some significant processing power.

Is not nice, but something has to be done.

Regards

Juan
 

Stratum already supports that.

The only two things which can stop ddos are:

1. Big pipe  ;) if bad boys have 1g connection your ISP should have at least 2gig as an example
2. Expensive machines to filter out ddos sitting  in front of the target, taking the fire and capable of doing that - CPU ram and so on
3. Both 1 and 2 :D

Ports and protocols handshakes do not have anything  in common especially when it is UDP  flood. As long as UDP is stateless bad boys just fill your connection with crap

When bad  boys are clever and if they have the appropriate resources available it is very very hard to stop them

The appropriate resources do cost  money. so called "fix" happens when:
1. They are out of money and are not able to "rent" big enough resource to attack
2. They are filtered out in the first hop. Works only when they are using ip's which are not random. In most cases the attack is performed from infected PC's al over the world

Ok I will re read the documentation.

Essentially it boils down to redirecting miners to other servers using client.reconnect (https://en.bitcoin.it/wiki/Stratum_mining_protocol#client.reconnect) after they submit at least one valid share.

Luke,
Do you realy think that redirecting a miner after first share will "hide or protect" given server?
Do not get me wrong but in 99% of cases when bad boys know their shit and have the resources required you can not do much about it.
Your ISp is the one who shall take care. And depending how hard ddos is it is not an easy task at all....

It's what he was asking for. And quite often, the bad boys do not know their stuff. :)

Yeah... That is what can safe us :D

Not far above your post:
https://bitcointalk.org/index.php?topic=557991.msg6487037#msg6487037

I will also point out there is not much point looking at the link that 'other' forum member supplied.
It contain bias non-objective information and basically he is given free reign of the wiki to post as he pleases.

This thread is the most reliable source of stratum information:
https://bitcointalk.org/index.php?topic=557866.0

A certain forum member's protocol envy means taking too much notice of what he says is a waste of time and on occasion leads to misleading information designed to promote ignorance and bias.

I had found this great Stratum wiki here:
http://electrum.orain.org/wiki/Stratum_protocol_specification

Now when I go to that link it redirects me to the pornhub web site I kid you not - what gives? Sabotage?

For all the talk of documentation in posts / anywhere I'm having trouble finding out how this works and the expected syntax for commands.
Can anyone help with this problem for instance:
http://bitcoin.stackexchange.com/questions/40556/how-do-i-get-merkle-branches-using-the-stratum-protocol

https://bitcointalk.org/index.php?topic=557866.0

https://en.bitcoin.it/wiki/Stratum_mining_protocol

Unfortunately there's not much point taking notice of that.
Luke-Jr wont allow corrections to his mistakes there.

I have a question about the mining.set_difficult method.

In the example here https://mining.bitcoin.cz/help/#!/manual/stratum-protocol

the first time the parameter here is a string (I would say it's the target), but the second time its a number.
Is it supposed to be a string only if it's a result to mining.subscribe or should the client handle both?

Another question is about the result to mining.subscribe. Can the result have an arbitrary number of [method, params] pairs or am I missing it completely?

The first one is only in mining.subscribe, and is a subscription id - just ignore it.
Target is not supported by anything for mining.set_difficulty, as far as I know.

Yes, an arbitrary number is possible.

Have you read https://en.bitcoin.it/wiki/Stratum_mining_protocol ?

Yes, I had a look at the wiki.


Why do we send mining.set_difficulty in this case? Only to show the client we support this method?

I saw other examples that don't do it, only send back mining.notify. I guess the mining.notify subscription id is useful to resume the connection if it drops (I can't find now the method used by the client to do so, but I'm sure I saw it once).

And I can't think of any use case of the mining.set_difficulty subscription id. What was the original idea behind it?

In this case the default difficulty is 1 until the server sets a higher one, right?

In the end wouldn't it make much more sense to set the difficulty per worker if we already support multiple workers per connection?

Right.

Basically every subscription would get an id so you could unsubscribe from them.

Right. And the changes only take effect for notifys after it.

Yes, the stratum mining protocol is quite poorly designed.

Stratum is fine except for 2 things.

1) difficulty isn't associated with work
Solution that resolved this: difficulty affects the next work item

2) the initial connection is convoluted for the miner
Cause of this problem: Getwork and GBT
Solution: well no one has done it, but it would be best to just ignore getwork and GBT connections and throw them away
cgminer already ignores getwork connections

Edit: also read the stratum definition in this thread: https://bitcointalk.org/index.php?topic=557866.0 (also linked in my sig)
The wiki version includes the no so-nice hash hack that is a major security issue for miners

Stratum uses too much bandwidth. I would like to see a zeroconf mining protocol which could deliver new job notifications to every miner in a big farm (same network) with only one packet from the pool server.

That would be the job of a proxy, but not ideal for a pool.
So yes it could be an addition to the stratum protocol, but would not be ideal for a pool to use it.

The basis of course would be to remove the variable information from the work notification and keep that extra state information in both the work source (proxy) and the miner.

So I'd guess it would be best implemented under a new name ... whatever, whoever :)

e.g. as a suggestion :)
mining.zerostate to define the state information for all following: mining.zeronotify to send work without the state information
You'd have to watch out, again, to avoid the 'difficulty' issue that would also have the same issue with 'state'
(that is resolved by defining what 'difficulty' and 'state' applies to rather than having it part of the notify)

What are you talking about here?

Pretty sure he's talking about the mining.extranonce.subscribe extension.

Come on same network big farm even 5 usd China switch can handle that. I am  sure that you have even cheaper ones laying arround ;D You do not need Cisco for this job. Your goal is to force other people do work for you so you can save a bug from hosting expnses right ;D I like your style though ;D
Some thorny from l2 networking if that is what you REALY want ...please note you will still gona need 5 usd switches here and there in the big farm

In general what you need is multicast and some strtum miner hacks ;D
As a big pool op it will be a peace of cake for you to make first two. But for the third one you need to have the sources of your crappy CHINESE borders miners fork. And as we all know it is a top secret STOLLEN FROM COMMUNITY and it is CLOSED. And you will have to pay a lot of money to your brothers to get ti;D ;D. What a irony dude :D

I meant in terms of security issue. mining.extranonce.subscribe is not in itself one, at least in theory.

My original post was here (because I didn't want to resurrect an old thread necessarily, I see now I was wrong): https://bitcointalk.org/index.php?topic=2768236.0

My main question at the moment is around the various transports mentioned:

The 'spec' defines 4 transports: Socket transport, Websocket, HTTP Poll, and HTTP Push (which is a variation of HTTP Poll). I get the feeling that much has changed since stratum was first introduced, so I'm wondering if all 4 protocols are still needed/used - is it necessary to implement them all in a proxy? What about if one was implementing a pool? For example, if miners have moved towards supporting plain TCP Socket transport only, there's no point in implementing the other 3 transports.

It's all just Socket.

Hi, I'm new to the forum. 

It's been hard to find any complete documentation on stratum, but I've read through a few posts here, slushpool, and bitcoin.it docs.

One thing I still don't understand is how stratum can prevent a malicious mining from taking a newly found share test it for validity at the full network difficulty, and if it is valid, submit it for themselves, instead of to the pool via stratum.  If this was problem was not solved, they would submit the shares via stratum to the pool only if they were not valid, making double the amount until someone realizes they are probably cheating.

Anyone care to enlighten?

Coz you can't change the data to go somewhere else - that will produce a different hash.
When you find a block, it depends on hashing the block reward output also.
Yeah they worked that out in the original design 9 years ago fortunately :)

Ah, of course, because the pool's reward address is in included in what's being hashed.  Thanks.

Slush has posted an official page for the stratum protocol now, including new extensions.

http://stratumprotocol.org/

http://stratumprotocol.org/stratum-extensions

Of note, the most important extension added is: mining.configure which supports “version-rolling”

The executive summary is that version-rolling means poolside support for asicboost. Asicboost is now under the defensive patent license meaning anyone can add it to their hardware provided they sign the DPL.

Unfortunately this site no longer contains the documentation for stratum.

You'll need to look here:
https://bitcointalk.org/index.php?topic=557866.0