Decentralized mining protocol standard: getblocktemplate (ASIC ready!)

[Luke-Jr]

One more point about the GBT protocol. If a miner chooses which transactions they include rather than accept whichever transactions the pool/bitcoind has offered in the template, the miner has to submit all the transactions with every share he submits.

No, that's (part of) what Block Proposal is for.

[firefop]

...And where do I fit in? Despite what it may look like, this is a technical discussion about the disadvantages of GBT.

imo - you're at least honest in your intent and presentation.

I'm not saying that GBT couldn't or shouldn't be replaced with something that is better that what it is now... I'm just saying given the choice between gbt and stratum --- I'll take transparent and clunkier over obfuscated and streamlined.

Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed.

[-ck]

...And where do I fit in? Despite what it may look like, this is a technical discussion about the disadvantages of GBT.

imo - you're at least honest in your intent and presentation.

I'm not saying that GBT couldn't or shouldn't be replaced with something that is better that what it is now... I'm just saying given the choice between gbt and stratum --- I'll take transparent and clunkier over obfuscated and streamlined.

Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed.

The only apparent transparency issue with stratum is that it does not transmit the transactions included in the merkle branches by default. You do know, however, that it supports a get transaction method as well? Then you can reconstruct the merkle branches for yourself and confirm it does what it says.

The reality is that 99% of miners will mine blindly trusting their pool, and they rely on the 1% of savvy users to keep the pools in check. It is a mechanism which has worked so far and will continue to do so. There is a way to confirm the pool is doing what you hope it's doing with stratum as well, and enforcing the extra information on the other 99% of users is pointless, as they're not even going to look at it.

[wizkid057]

...And where do I fit in? Despite what it may look like, this is a technical discussion about the disadvantages of GBT.

imo - you're at least honest in your intent and presentation.

I'm not saying that GBT couldn't or shouldn't be replaced with something that is better that what it is now... I'm just saying given the choice between gbt and stratum --- I'll take transparent and clunkier over obfuscated and streamlined.

Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed.

I agree with this assessment. con is at least providing constructive criticism and suggestions for improvements, which seem to make sense.   I agree that GBT needs some work to be perfect, but, it's not a terrible implementation even as it sits, despite what people may try to make it look like.

As far as I'm concerned, GBT would be damn near perfect with the following improvements:

• Reduce redundant data transfer, specifically re-transmitting transaction data when both the pool and the miner know that the other already has this data. (BIP0023 attempts to address this, but could be improved more.)
• Ditch HTTP.  While using HTTP makes deployment faster, it needs to die for mining. Really. Stratum got it correct in this regard.
• Miner-side support for all of the features of GBT, such as transaction monitoring, viewing of the coinbase txn, specify miner desired transaction processing policies, etc.

Not that GBT isn't good even as it is, but, if the above is done, then GBT will be indisputably awesome.

-wk

[DrHaribo]

It's unfortunate that we now have two competing standards. It's more unfortunate that another useless bitcointalk war is breaking out over it.

As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't.

When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that? Instead waiting until there are implementations in production and then finally spitting out nasty words and technical misunderstandings. That's not the way to go about things.

You have to ask yourself: Do I want to be someone with a positive attitude who finds ways to improve things and push Bitcoin forward? Or do I want to be some useless clockwinder who sabotages Bitcoin with FUD, lies and nonsense?

Trolling and spreading FUD may seem like fun. But remember that a lot of newbies who have just discovered Bitcoin come here and read all your nonsense. Also, some of us are trying to build something.

[wizkid057]

It's unfortunate that we now have two competing standards. It's more unfortunate that another useless bitcointalk war is breaking out over it.

As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't.

When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that? Instead waiting until there are implementations in production and then finally spitting out nasty words and technical misunderstandings. That's not the way to go about things.

You have to ask yourself: Do I want to be someone with a positive attitude who finds ways to improve things and push Bitcoin forward? Or do I want to be some useless clockwinder who sabotages Bitcoin with FUD, lies and nonsense?

Trolling and spreading FUD may seem like fun. But remember that a lot of newbies who have just discovered Bitcoin come here and read all your nonsense. Also, some of us are trying to build something.

+1 * 10^256

[-ck]

As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't.

Please read my quote. I did NOT say that.

[-ck]

And if you blame me for not paying attention at the right time earlier on when this protocol was developed well I'm terribly sorry for not keeping track of every goddamn fucking thing going on in the bitcoin world.

I will implement whatever the fuck this protocol ends up being one way or another.

Sorry for offering suggestions, I will now say no more on this.

[sharky112065]

C. Kolivas was going through a devastating family matter at the time of those discussions. We are lucky he has returned at all to continue his work on Cgminer.

People seem to blur the lines that he and Kano are the same person. They do not always agree (Just observe the cgminer IRC channel a bit and you will realize that) on everything.

If I recall, at first he did not like either solution (Stratum or GBT) and he picked the one he had the less issues with to implement first.

I'm not as into the technical end of mining and its underlying code, but from my point of view, all I'm seeing is a lot of bickering and not much getting done to improve and or fix the issues that are causing reservation on implementing GBT.

I hope everyone involved can come together and make both Stratum and GBT work out, because it looks like both will be around for a while.

[DrHaribo]

My post above was not only about one person or even one side of this. There seem to be two camps fighting rather than improving things.

When you come in late after the discussion has been going on for a long time and things have now even been implemented, you say "I'm sorry I'm so late with this. I've been busy with something else and didn't have time to look through this until now. But it seems to me that X could be done better if you do it more like Y." Notice the polite form and also not just saying that something is bad, but making suggestions on how to improve it.

You don't say "your shit sucks. Fuck this and fuck that." It's bad form and completely unnecessary.

[kano]

My post above was not only about one person or even one side of this. There seem to be two camps fighting rather than improving things.

When you come in late after the discussion has been going on for a long time and things have now even been implemented, you say "I'm sorry I'm so late with this. I've been busy with something else and didn't have time to look through this until now. But it seems to me that X could be done better if you do it more like Y." Notice the polite form and also not just saying that something is bad, but making suggestions on how to improve it.

You don't say "your shit sucks. Fuck this and fuck that." It's bad form and completely unnecessary.

Your post further above was rubbish.

The only point you made in it about comments against stratum was actually completely false
(as ckolivas pointed out to you)

What arguments are not true? Or misunderstood? - you gonna go make that statement and not even back it up?

Seriously, you standing there up on high, telling people what they should be doing, is not gonna win you anything.
All it says is that you have no input into the discussion/argument/whatever words you feel the urge to use and should, quite simply, butt out.

If you have arguments to make against the issues being brought up, go ahead make them.

Otherwise be gone - you telling people how they should do this is pointless drivel as is wizkid posting an even more pointless follow up to your post.

You don't like the way I bring up issues and and get a response from those involved? Damn shame about that.
It works and it makes people listen and either it gets resolved or it gets swept under the carpet.

If you think that BTC software should only take noticed of polite happy fairy statements from polite friendly programmers, then you are seriously deluded.

Yeah I'll say it - provide some useful input into the argument/discussion or fuck off.

[slush]

Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed.

Hm, maybe I didn't say it loudly enough, but this has been solved already: https://bitcointalk.org/index.php?topic=108533.msg1287719#msg1287719

This stratum extension give a full control online to miners about transactions included in the block template. So from side of openess, Stratum and GBT are both equal. Just Stratum is a bit more optimized ;-).

[slush]

When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that?

I did not so for three reasons:

a) I had no idea how it has been supposed to work. And I still don't understand majority of his proposal, optional parts etc. I understand GBT to point that I can load block template and put valid block back to bitcoind, but the rest seems to be a rubbish for me.
b) If I'd try to help Luke's with new mining protocol, then I'll suggest Stratum. Obviously there were different targets; Luke wanted something which can be implement in bitcoind, but I wanted something optimized for pooled mining. I'm saying it again and again, that Stratum and GBT can live together and only few people have some mental issues with it.
c) It is very hard to discuss with Luke, so creating my own solution which included all my best ideas was a bit easier ;-).

[DrHaribo]

If you think that BTC software should only take noticed of polite happy fairy statements from polite friendly programmers, then you are seriously deluded.

So you think being polite with other people is a fairy land?

What I am saying is that if we work together rather than against each other, then we will get much further.

Yeah I'll say it - provide some useful input into the argument/discussion or fuck off.

I was in this discussion long before you and made contributions before anyone had this in production. With every thing I didn't like I sent in a suggestion for how to improve it. Many, if not all, of my suggestions went in the spec. Now you are telling me to fuck off, because I don't contribute? I think we have different opinions about what a contribution is.

I think your main concern right now is that GBT doesn't provide enough transactions. Correct? Actually that's not an issue with the protocol. If bitcoind or a pool gives you 1 or 1000 transactions is up to bitcoind or the pool, not this interface.

You may have a valid concern that bitcoind is sometimes holding a lot of transactions in the memory pool without wanting to put them in a block with getwork/getmemorypool/getblocktemplate. Maybe we should take that to a different thread?

[DavinciJ15]

With that said it's easy to conclude that since cgminer and others have implemented stratum over GPT kind of gives my opinion a bit of legs to stand on. 

By that logic you should say GBT is better because its actually built directly into the official bitcoin client.  I don't see how stratum is better just because cgminer implemented it.  Just because something is implemented somewhere prominent doesn't make it superior.

Uh, do you know why VHS beat out Betamax?  Simular reason the VHS tapes where created by other businesses at no cost and less issues. Thus Stratum is implemented and used with low cost and less issues that does not make it better it just makes it what the market wants.

A quick check
./bitcoind getblocktemplate
at this very second produces a output that is 469441 bytes!

If every template request was this size and the miner used the customise coinbase option and chose transactions, they would be receiving 400kb for the block template and each share submitted would be 400kb unless they started withholding transactions.

Dam!  If this is true then I would have to ask what nut bar would use this in a pool?  Your network trafic will skyrocket.

One more point about the GBT protocol. If a miner chooses which transactions they include rather than accept whichever transactions the pool/bitcoind has offered in the template, the miner has to submit all the transactions with every share he submits.

No, that's (part of) what Block Proposal is for.

That's not implemented yet is it?

When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that? Instead waiting until there are implementations in production and then finally spitting out nasty words and technical misunderstandings. That's not the way to go about things.

Please share with us all how an open source project where anyone can create anything can be followed by anyone that maybe interested in it?  Note: I only found out that Stratum when I was told on another forum that Bitcoind will not work with my mini rig.  Then I discovered GBT.
 
There should be a bitcoin developer news web site so those that want to see technical announcements so people like me can find planned advancements that effect them.  To bad there isn't so we work with what we have.

Also I'm not saying 'kill stratum with fire' but I won't be using it until the transparency issue is addressed.

Hm, maybe I didn't say it loudly enough, but this has been solved already: https://bitcointalk.org/index.php?topic=108533.msg1287719#msg1287719

This stratum extension give a full control online to miners about transactions included in the block template. So from side of openess, Stratum and GBT are both equal. Just Stratum is a bit more optimized ;-).

Nice! But why do I feel like a I'm watching a boxing fight and you just rang the bell to start another round of the fight?  lol

but the rest seems to be a rubbish for me.

Oh never mind.  Try to remember Slush that GBT may have some of Luke-jr's best ideas and calling it "rubbish" kind of stings.

RETARDS!!!

We do what we physically can and what the market wants, if that's mentally defective then please do it better on your own and prosper.  Your prosperity will will help us all by providing proper solutions for us to follow and providing better value to the market than these guys can.

If you believe the market will not want your product or service because it's mentally challenged and must be forced to do so, then I must point you to the success of such ideas called U.S.S.R. and Obama-nation.

[flynn]

I have a question.
I must admit I didn't read everything about pools handling asics, but, ...

If solving blocks is roughly returning blocks with the first 53 bits = 0 (well, a bit more), pools request miners to return computed blocks with the first 32 bits = 0.

Why not simply make pools specialized for asics which would request returning blocks with the first (32+8) 40 bits = 0 ? that would divide the numbers of returned blocks by 256 and make everyone happy.

I probably missed something. Can someone explain the n00b I am why this would be bad .. ?

[DrHaribo]

Please share with us all how an open source project where anyone can create anything can be followed by anyone that maybe interested in it?  Note: I only found out that Stratum when I was told on another forum that Bitcoind will not work with my mini rig.  Then I discovered GBT.
 
There should be a bitcoin developer news web site so those that want to see technical announcements so people like me can find planned advancements that effect them.  To bad there isn't so we work with what we have.

That's a good point. And you basically answer it yourself. An alternative could be mailing lists. Either way it's probably important to make that a developer-only resource to get a good signal to noise ratio.

[kano]

If you think that BTC software should only take noticed of polite happy fairy statements from polite friendly programmers, then you are seriously deluded.

So you think being polite with other people is a fairy land?

What I am saying is that if we work together rather than against each other, then we will get much further.

Lulz - I can post at 2.6 and still see you've posted crap.

Post some grounds for your delusions or ... yeah I've already said it ... oh great you even quoted it below

Give some grounds for this crap

It's unfortunate that we now have two competing standards. It's more unfortunate that another useless bitcointalk war is breaking out over it.

As usual many of the arguments are just not true, or misunderstood. "You have to send in all the transactions with every share. Think of all those bytes!" No, you don't.

or be gone.

Yeah I'll say it - provide some useful input into the argument/discussion or fuck off.

I was in this discussion long before you and made contributions before anyone had this in production. With every thing I didn't like I sent in a suggestion for how to improve it. Many, if not all, of my suggestions went in the spec. Now you are telling me to fuck off, because I don't contribute? I think we have different opinions about what a contribution is.

I don't care if you designed GBT ... ... ... well actually that would be even worse coz that would make you Jesus.

Hell, my ancestor wrote the most important book in Christianity after the damn bible, but so what, who gives a crap about the past.
What matters is that part I just quoted above, not what blame you have in the past for helping creating this abortion called GBT.

I think your main concern right now is that GBT doesn't provide enough transactions. Correct? Actually that's not an issue with the protocol. If bitcoind or a pool gives you 1 or 1000 transactions is up to bitcoind or the pool, not this interface.

You may have a valid concern that bitcoind is sometimes holding a lot of transactions in the memory pool without wanting to put them in a block with getwork/getmemorypool/getblocktemplate. Maybe we should take that to a different thread?

My concerns have be stated.

Your pool ran 15 blocks that I pointed out, and of them, only 5 were large txn count/size.
5 were tiny. 5 were in the middle.

You claimed that you don't choose the transactions, GBT does ... wow

Next, GBT currently usually sends more data per minute than a GetWork pool with roll-n-time (probably even if it is ignoring transactions or there are few transactions on the network)
This is due to the fact that it sends all the details of the all transactions you are processing (no matter what, you cannot disable this and request a merkle tree)

Next, GBT uses HTTP not raw sockets, so it is slower and is also expected to have more rejected shares due to this since it isn't a constant connection.
Not using HTTP, Luke-Jr has stated as a negative in Stratum coz it takes more effort to implement a better protocol ... wow gotta love that argument.
Sounds like "Don't implement something better coz it will take more effort"
Or even "Luke-Jr sux at programming and can't do anything that isn't simple"

[Luke-Jr]

When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that?

I did not so for three reasons:

a) I had no idea how it has been supposed to work. And I still don't understand majority of his proposal, optional parts etc. I understand GBT to point that I can load block template and put valid block back to bitcoind, but the rest seems to be a rubbish for me.

Why?

b) If I'd try to help Luke's with new mining protocol, then I'll suggest Stratum.

That's fine, Stratum seems like a logical place to start for a new standard. It would have been nice if these things had taken place during the GBT standardization, but that's in the past now. How goes getting StratumMP BIP process off the ground? Are we still waiting on genjix?

Obviously there were different targets; Luke wanted something which can be implement in bitcoind, but I wanted something optimized for pooled mining. I'm saying it again and again, that Stratum and GBT can live together and only few people have some mental issues with it.

You keep saying this, and I keep having to correct you: No, the main focus of GBT was from the start was pooled mining. Citing the Motivation section of BIP 22 from all the way back in the original February draft... "There is reasonable concerns about mining currently being too centralized on pools, and the amount of control these pools hold. By exposing the details of the block proposals to the miners, they are enabled to audit and possibly modify the block before hashing it."

c) It is very hard to discuss with Luke, so creating my own solution which included all my best ideas was a bit easier ;-).

This is just trolling.

[Luke-Jr]

One more point about the GBT protocol. If a miner chooses which transactions they include rather than accept whichever transactions the pool/bitcoind has offered in the template, the miner has to submit all the transactions with every share he submits.

No, that's (part of) what Block Proposal is for.

That's not implemented yet is it?

Allowing miners to modify transactions is not implemented in miners yet at all. Block Proposal only makes sense as part of that. I do have it implemented for the Eloipool<->bitcoind link.

When Luke-Jr made a public proposal and asked for comments, I read through it and sent in my thoughts on how it could be improved. Why didn't more people do that? Instead waiting until there are implementations in production and then finally spitting out nasty words and technical misunderstandings. That's not the way to go about things.

Please share with us all how an open source project where anyone can create anything can be followed by anyone that maybe interested in it?  Note: I only found out that Stratum when I was told on another forum that Bitcoind will not work with my mini rig.  Then I discovered GBT.

There is a Bitcoin development mailing list specifically for discussing all things related to Bitcoin development. GBT had a number of threads of discussion on it since February.