Stratum protocol discussion

[kano]

It wasn't developed behind closed doors, you are just regurgitating

The initial development was secret to the public before it was up on slush's pool along with the python proxy. I'd always understood this to be a (anti- )competitive move.

There remains no BIP describing it, there was no design discussion prior to its release on bitcoin-development. Justifying the closed development, Slush wrote (in the second email my mailbox ever received mentioning stratum mining) "There's no requirement to have BIP for everything what people do. Stratum is NOT related to bitcoin protocol or bitcoin implementation, it is just custom, pooled-mining extension and bitcoin network doesn't need to know about Stratum existence at all."  Perhaps you were somehow an insider on it— if so, why didn't you discourage some of the poor choices in it?  But from my own perspective it really was developed in secret and, in my opinion, carries some of the predictable flaws of a protocol developed without broad input. It's not the end of the world, in any case. Far worse has been done elsewhere.

Yeah it was so secret that only people who visited bitcointalk and github could find out about it ....................

The basis for it (not as a mining protocol) was started in December 2011 ...
https://bitcointalk.org/index.php?topic=55842.0

https://github.com/slush0

He also made mention of the mining protocol here, but, no doubt, there are other details around:
https://bitcointalk.org/index.php?topic=61131.msg702097#msg702097

As I said, though you hid the end of it:

From a protocol development perspective, stratum being defined in terms of difficulty is an embarrassing flaw of the sort that would have been avoided if it hadn't been developed behind closed doors.
...

It wasn't developed behind closed doors, you are just regurgitating a lie spread by Luke.

[1714174071]

1714174071

[1714174071]

1714174071

[1714174071]

1714174071

[gmaxwell]

Actually I'm on the track of protocol which will allow full democracy of the bitcoin miners like in p2pool, but still with possible zero variance (even pps with difficulty 1 payout scheme), which is something what is missing on p2pool. It should be also DDoS resilient as there won't be any real benefit in attacking the server which will just collect shares and stats...

If only it turned out that way. Instead, it doesn't do any of that at all, and displaced the less efficient GBT stuff that did, in theory, have that capability.

None of this however involved any public discussion of what was actually developed for mining, alas.

[kano]

Meanwhile we have the retard doing rubbish like this to try blocking the dissemination of information about Stratum:
https://en.bitcoin.it/w/index.php?title=Stratum&action=historysubmit&diff=46202&oldid=46194

[irrational]

Meanwhile we have the retard doing rubbish like this to try blocking the dissemination of information about Stratum:
https://en.bitcoin.it/w/index.php?title=Stratum&action=historysubmit&diff=46202&oldid=46194

I cannot rationalize those changes, short of malice.

[kano]

Meanwhile, having a BIP is clearly not as important as has been suggested.

The code for BIP22 for the centralised GBT mining protocol
(yes, saying it is decentralised is also a lie being spread by Luke and gmaxwell)
was modified by Luke before submitting it to bitcoin

He implemented code that is not part of BIP22 and put it into bitcoin - for support of botnet mining.
https://github.com/bitcoin/bitcoin/issues/1985

Anyway, looks like he still can't handle having the stratum pool protocol mentioned first on the Stratum page and likes to list a bunch of meaning less complaints about it:
https://en.bitcoin.it/wiki/Stratum
The "Displacing GBT" complaint has to be the funniest - how dare someone displace GBT with something that is so much better than it.
Pity it also contains outright lies. But that is to be expected of Luke.

[djeZo]

Is mining.get_transactions mandatory to be implemented on pool side? Is it safe to simply ignore this client request (send back nothing)?

Question for you Luke:
Does your BFGMiner correctly handle variable sizes of extranonce1 and extranonce2? Can extranonce1 be 5 bytes, extranonce2 3 bytes and shares provided valid?
As you may know already, cgminer from certain X up to certain version Y has a bug there, causing all shares to be rejected.

[Luke-Jr]

Is mining.get_transactions mandatory to be implemented on pool side? Is it safe to simply ignore this client request (send back nothing)?

It can be ignored, but should send back an error if not supported.

Does your BFGMiner correctly handle variable sizes of extranonce1 and extranonce2? Can extranonce1 be 5 bytes, extranonce2 3 bytes and shares provided valid?

Yes, any reasonable value (bounded by memory size I think) should work fine, but BFGMiner will only actually use at most 40 bits of extranonce2 right now (32 in most cases).
Note that the stratum proxy feature requires at least 24-bit extranonce2 (failure to meet this requirement disconnects all clients and rejects new ones).

[kano]

Meanwhile, a permanent change that is part of the solution to the client.reconnect problem at the moment should be:
Require the pool domain and the reconnect domain to match.

There is no need for hard coding IP addresses, since anyone on the planet can own a domain for $10 a year
The reconnect has no need to use an IP address either - for the same reason - DNS records require no effort.

... and anyone wishing to use different domains for different located pools, you simply add a DNS record in the domain of the pool doing the reconnect to point to the other IP address.

DNS poisoning problems are not relevant to this since if there is a DNS problem then the miner can already be directed away from the pool when you first connect.

[Luke-Jr]

Meanwhile, a permanent change that is part of the solution to the client.reconnect problem at the moment should be:
Require the pool domain and the reconnect domain to match.

There is no need for hard coding IP addresses, since anyone on the planet can own a domain for $10 a year
The reconnect has no need to use an IP address either - for the same reason - DNS records require no effort.

... and anyone wishing to use different domains for different located pools, you simply add a DNS record in the domain of the pool doing the reconnect to point to the other IP address.

DNS poisoning problems are not relevant to this since if there is a DNS problem then the miner can already be directed away from the pool when you first connect.

Sounds fine to me.

[Humancell]

kano/ckolivas,

I'm working with the guys at NiceHash.com to try and resolve a few issues with how they are managing miners with their stratum pool.  In debugging this, I've been reading through the code that implements the stratum client - util.c - and trying to better understand the stratum protocol.

What I'm not quite clear on is the loop/state machine that you use in a mining client to "poll" a pool and attempt to connect to get work.

The NiceHash.com pool uses the password parameter in a mining.authorize to pass in a requested "threshold" for mining payouts.  (e.g. I might only want to mine if there are contracts for a minimum of 0.07BTC/TH/Day so my password becomes "p=0.07")  When the mining.authorize occurs, they will accept or error my auth based on that threshold comparison.

This is working ... mostly.

The problem we are seeing is that at some later point of time, the miner seems to "hang" or believe the pool is alive.  I'm running kano's builds on my AntMiners, and periodically during the day I'll find them attached to the pool, but not getting any work.  They think the pool is alive ... the pool is saying they sent an error on the auth.

I am debugging to see if this is pool related, miner related, or protocol related.  (Or a little of several?)

What I wanted to know is:

Within cgminer, once a pool is determined to be "dead", what is the delay before the pool is tried again, and what operations (Requests) are retried to "test" the pool for aliveness?  Where would I find the code that handles that delay and retrying?

I'm writing here, as I tried to locate and study the protocol documentation to determine what the "proper" process ought to be ... but didn't find too much on the subject.  :-)

I'm wondering if a miner like cgminer would be doing :  mining.subscribe -> mining.authorize -> error/delay/startover-at-subscribe?

If this is the case, is there any reason to believe that after looping like this - getting errors for a half day or so - the miner might hang?

I'm also glad to take this conversation elsewhere if this is not the appropriate thread.

[rascal777]

PLEASE PLEASE PLEASE

REALLY

mining.subscribe needs extension with a third parameter being the requested URL.

WHY IS THIS NOT IMPLEMENTED?

[topminingcontracts]

Hello

Considering the nature of a pool that "mine" Bitcoins I think we will suffer DDOS constantly and every time more aggressive sophisticated and bigger.

In my opinion the Srtatum protocol may help with some improvements to mitigate the DDOS

May be some features like the Apache/HTTP can be implemented like the Redirect or Temporary Redirect where after connecting the node tells the miner to connect to a different IP:PORT. Before that redirect may require to execute some handshake process that needs to be a heavy process at miner side so will be complex for a DDOS complete the handshake without some significant processing power.

Is not nice, but something has to be done.

Regards

Juan
 

[Luke-Jr]

Hello

Considering the nature of a pool that "mine" Bitcoins I think we will suffer DDOS constantly and every time more aggressive sophisticated and bigger.

In my opinion the Srtatum protocol may help with some improvements to mitigate the DDOS

May be some features like the Apache/HTTP can be implemented like the Redirect or Temporary Redirect where after connecting the node tells the miner to connect to a different IP:PORT. Before that redirect may require to execute some handshake process that needs to be a heavy process at miner side so will be complex for a DDOS complete the handshake without some significant processing power.

Is not nice, but something has to be done.

Regards

Juan
 

Stratum already supports that.

[loshia]

The only two things which can stop ddos are:

1. Big pipe   if bad boys have 1g connection your ISP should have at least 2gig as an example
2. Expensive machines to filter out ddos sitting  in front of the target, taking the fire and capable of doing that - CPU ram and so on
3. Both 1 and 2

Ports and protocols handshakes do not have anything  in common especially when it is UDP  flood. As long as UDP is stateless bad boys just fill your connection with crap

When bad  boys are clever and if they have the appropriate resources available it is very very hard to stop them

The appropriate resources do cost  money. so called "fix" happens when:
1. They are out of money and are not able to "rent" big enough resource to attack
2. They are filtered out in the first hop. Works only when they are using ip's which are not random. In most cases the attack is performed from infected PC's al over the world

[topminingcontracts]

Hello

Considering the nature of a pool that "mine" Bitcoins I think we will suffer DDOS constantly and every time more aggressive sophisticated and bigger.

In my opinion the Srtatum protocol may help with some improvements to mitigate the DDOS

May be some features like the Apache/HTTP can be implemented like the Redirect or Temporary Redirect where after connecting the node tells the miner to connect to a different IP:PORT. Before that redirect may require to execute some handshake process that needs to be a heavy process at miner side so will be complex for a DDOS complete the handshake without some significant processing power.

Is not nice, but something has to be done.

Regards

Juan
 

Stratum already supports that.

Ok I will re read the documentation.

[Luke-Jr]

Hello

Considering the nature of a pool that "mine" Bitcoins I think we will suffer DDOS constantly and every time more aggressive sophisticated and bigger.

In my opinion the Srtatum protocol may help with some improvements to mitigate the DDOS

May be some features like the Apache/HTTP can be implemented like the Redirect or Temporary Redirect where after connecting the node tells the miner to connect to a different IP:PORT. Before that redirect may require to execute some handshake process that needs to be a heavy process at miner side so will be complex for a DDOS complete the handshake without some significant processing power.

Is not nice, but something has to be done.

Regards

Juan
 

Stratum already supports that.

Ok I will re read the documentation.

Essentially it boils down to redirecting miners to other servers using client.reconnect after they submit at least one valid share.

[loshia]

Luke,
Do you realy think that redirecting a miner after first share will "hide or protect" given server?
Do not get me wrong but in 99% of cases when bad boys know their shit and have the resources required you can not do much about it.
Your ISp is the one who shall take care. And depending how hard ddos is it is not an easy task at all....

[Luke-Jr]

Luke,
Do you realy think that redirecting a miner after first share will "hide or protect" given server?
Do not get me wrong but in 99% of cases when bad boys know their shit you can not do much about it.
Your ISp is the one who shall take care. And depending how hard ddos is it is not an easy task at all....

It's what he was asking for. And quite often, the bad boys do not know their stuff.

[loshia]

Luke,
Do you realy think that redirecting a miner after first share will "hide or protect" given server?
Do not get me wrong but in 99% of cases when bad boys know their shit you can not do much about it.
Your ISp is the one who shall take care. And depending how hard ddos is it is not an easy task at all....

It's what he was asking for. And quite often, the bad boys do not know their stuff.

Yeah... That is what can safe us

[kano]

Hello

Considering the nature of a pool that "mine" Bitcoins I think we will suffer DDOS constantly and every time more aggressive sophisticated and bigger.

In my opinion the Srtatum protocol may help with some improvements to mitigate the DDOS

May be some features like the Apache/HTTP can be implemented like the Redirect or Temporary Redirect where after connecting the node tells the miner to connect to a different IP:PORT. Before that redirect may require to execute some handshake process that needs to be a heavy process at miner side so will be complex for a DDOS complete the handshake without some significant processing power.

Is not nice, but something has to be done.

Regards

Juan
 

Not far above your post:
https://bitcointalk.org/index.php?topic=557991.msg6487037#msg6487037

I will also point out there is not much point looking at the link that 'other' forum member supplied.
It contain bias non-objective information and basically he is given free reign of the wiki to post as he pleases.

This thread is the most reliable source of stratum information:
https://bitcointalk.org/index.php?topic=557866.0

A certain forum member's protocol envy means taking too much notice of what he says is a waste of time and on occasion leads to misleading information designed to promote ignorance and bias.