Decentralized mining protocol standard: getblocktemplate (ASIC ready!)

[Luke-Jr]

If I understand correctly GBT also supports LP, doesn't it? Did you make use of LP with getwork and GBT in your test?

Yes, of course.

It would really be interesting to see the stale ratio of GBT+LP vs Stratum over one week on the same pool/same connection.

Unfortunately, I haven't optimized GBT in BFGMiner at all yet, so I doubt it would compare well vs Stratum proxy. It would be interesting for someone to make a quick port of the Stratum proxy to GBT (python-bitcoinrpc + python-blkmaker should do it very easily) to compare with. Of course, you'd also need a biprotocol pool, which AFAIK doesn't exist yet.

[jgarzik]

Longpolling is a very elegant and useful... hack.  It is a hack because HTTP clients typically initiate requests, and elicit server responses.  This HTTP client behavior does not cover server-initiated messages that the client is not expecting.

Longpolling says "wait, until the next server-initiated message to me"

Most custom designed protocols support server-to-client asynchronous messages, so there is no need for the hack.

[kuzetsa]

(GMT-0400) North America Eastern Daylight summer time, NY)
[23:39:32] <kuzetsa> is eloipool going to have stratum support?
[23:39:41] <wizkid057> ...
[23:39:50] <@Luke-Jr> kuzetsa: stratum sucks
[23:39:58] <kuzetsa> Luke-Jr: not for bandwidth it doesn't
[23:40:09] <kuzetsa> it's awesome at bandwidth


((...snip...))


[23:59:29] <@Luke-Jr> kuzetsa: you seem to be confused.
[23:59:44] <@Luke-Jr> GBT supports everyone's needs, not just me/my sw/my pool
[23:59:47] <kuzetsa> because I like stratum?
[23:59:54] <kuzetsa> liking stratum = confused?
[23:59:56] <@Luke-Jr> there is no rational reason to prefer stratum
[23:59:59] <@Luke-Jr> so sure

No real explanation given. Just that I'm irrational.

Often as I truely am irrational or even confused, this seems more like  "argumentum ad hominem" to me.




...Edited to add:

Forgot to datestamp.

Local time rolled over from wednesday. Now it is Thursday, september 27th 2012

(GMT-0400) North America Eastern Daylight summer time, NY)
[00:35:41] <@Luke-Jr> kuzetsa: this is a private channel and quoting it is against freenode policy and wiretapping laws, ESPECIALLY out of context
[00:45:52] * You were kicked from #eligius by Luke-Jr (delete the post)

Consider my official response "No"

I'll not be bullied into censoring this.

Feel free to take it up with freenode staff or law enforcement.

I'm confident that I know my local laws:

• I live in a "one party consent" jurisdiction for purposes of recording a phonecall.
• Your channel is public (not passworded, nor invite only, etc.) and freenode has no such policy as you described.

[DiabloD3]

Angry neighborhood bastard mod here.

Luke, what did I tell you about trolling the noobs? Cut it out before one of the admins decide to either scammer/troll tag you or ban you.

[jgarzik]

The pragmatic solution is to support both GBT and stratum, in mining software.

Both are already deployed, and stratum seems to currently be winning the race for deployments.

GBT is supported in bitcoind and p2pool, and has gone through the BIP process and more review, so a choice seems valid.  GBT is also more easily deployed in existing software, as it uses the more standard HTTP JSON-RPC that miners are familiar with.

[Luke-Jr]

The pragmatic solution is to support both GBT and stratum, in mining software.

Perhaps, but implementing Stratum is a lot of work (at least on the miner end). Better for pools to support both IMO; at least that gives the miners a choice of whether they want to retain control or blindly hand it over to the poolop.

Both are already deployed, and stratum seems to currently be winning the race for deployments.

I see more GBT deployment than Stratum, thankfully.

P.S. Since the mods here don't do their job, I have to point out that kuzetsa is misquoting me.

[jgarzik]

The pragmatic solution is to support both GBT and stratum, in mining software.

Perhaps, but implementing Stratum is a lot of work (at least on the miner end). Better for pools to support both IMO; at least that gives the miners a choice of whether they want to retain control or blindly hand it over to the poolop.

That is the trade-off.  GBT is far more compatible with existing software.  Stratum is a wholly new protocol requiring wholly new transport implementation, but it gains some efficiency over HTTP.

[kuzetsa]

((...snip...))
I have to point out that kuzetsa is misquoting me.

I just edited slightly, making the ((...snip...)) in the channel log thingy more obvious by inserting additional vertical whitespace (newline) above and below (3x newline padding instead of
"singlespaced" newline)

Just want to assert: There was no misquote, just an omission for the long duration of "you're wrong, no you are, no you"

--- luke. feel free to fill the ((...snip...)) with what I left out.

My main thought is that I prefer stratum because it gets fewer stale / invalid / orphan or whatever they're called. Stratum was working and had an installed userbase and is ASIC ready, predates GBT, etc.

Longpoll over HTTP-type transport is making GBT less attractive to me than stratum. Any mining protocol using a full TCP transport or even a non-stateless version of udp, etc. (basically, anything with an active connection, therefore by definition not http) will allow for incremental changes to the workload and immediate notification of new block.

There is no reason GBT's features can't be overlaid onto stratum or other non-HTTP mining protocols. (or the reverse for that matter, nothing TOO major that locks GBT's underlying structure to using HTTP)

Before all of this drama started I had planned to be in bed by 30 minutes past midnight. That was 3 hours ago. guess I'll just end with this:

Luke, I've been using IRC since the mid 90s. New to bitcoin and I don't need threats like this when I'm still a n00b because your agenda is threatend or whatever it was that upset you. please be cool.

[P_Shep]

kuzetsa, meet Luke

[slush]

Well, it is not "few kbytes saved versus save the network". I had quite long discussion with gmaxwell yesterday on -dev and I simply don't see any real attack vector which can be performed just because pool send merkle branch instead of transaction hashes (and this is REAL difference between Stratum and GBT). The only real attack vector which we found is that CIA will kill me, Tycho, Eleuthria, Graet, Inaba and some other pool operators, overtake our pools AND users won't notice anything and they'll keep on hashing invalid/broken blocks for CIA.

So from my view, GBT is wasting resources for no real gain.

[kano]

I mean - there's no reason to make mining protocol less effective just because decentralization itself.

I wonder what would you say to those who were, are and will be "lured" to Bitcoin with promise of decentralized network once
they realise that same decentralization is being sacrificed for few (k)bytes saved. Your importance list got messed up somehow.

Wait - should I be surprised with promotion of X at the cost of more centralization by pool owner? No, LOL, that's expectable!

As soon as BTCGuild releases me from "0.1 BTC prison", I'll switch to solo mining using BFGMiner with GBT. It won't really bring
a lot to Bitcoin network itself since my hashrate is just 30MH, but I'll sleep better knowing I'm using and supporting right thing.

So why were you already pool mining if this is an issue for you?

If decentralization was in ANY way relevant to your comments then you would have been solo mining or using P2Pool.

... which have been around for a long ....... time ...

[kano]

The issue is quite simple.

If you personally want 100% decentalization then you must either solo mine or use P2Pool - not GBT to a pool.
GBT to a pool is not 100% 'decentalization'

However, with 30MH/s, you will possibly never get any BTC solo mining.
Your average expected time to find a block at the current difficulty is 5062 days ...

As for pools in general ... luckily, people are allowed to choose hey?
BTC isn't a Luke-jr dictatorship (even though he wants to force that on everyone)

[mezzomix]

So from my view, GBT is wasting resources for no real gain.

That depends on your point of view. As a pool operator or miner with ultimate trust to the pool operator you are right. From the perspective of a miner that cares about what goes into a block it is important to see the complete information.

With Stratum, the pool operator is the one that creates bitcoin blocks and makes the rules. The miner is paid for doing some work for the operator. With GBT the pool operator is a work dispatcher but miners create the bitcoin blocks and make the rules. I don't regard more information and control as a waste of resources.

[kano]

So from my view, GBT is wasting resources for no real gain.

That depends on your point of view. As a pool operator or miner with ultimate trust to the pool operator you are right. From the perspective of a miner that cares about what goes into a block it is important to see the complete information.

With Stratum, the pool operator is the one that creates bitcoin blocks and makes the rules. The miner is paid for doing some work for the operator. With GBT the pool operator is a work dispatcher but miners create the bitcoin blocks and make the rules. I don't regard more information and control as a waste of resources.

... actually - no - who actually sees what goes into the blocks?
The software you mine with decides - and if you use the gbt luke-jr implemented - then luke-jr decides for you ...
Did you read the gbt code he wrote when you mined with his BarbieMiner?

[kano]

So from my view, GBT is wasting resources for no real gain.

That depends on your point of view. As a pool operator or miner with ultimate trust to the pool operator you are right. From the perspective of a miner that cares about what goes into a block it is important to see the complete information.

With Stratum, the pool operator is the one that creates bitcoin blocks and makes the rules. The miner is paid for doing some work for the operator. With GBT the pool operator is a work dispatcher but miners create the bitcoin blocks and make the rules. I don't regard more information and control as a waste of resources.

... actually - no - who actually sees what goes into the blocks?
The software you mine with decides - and if you use the gbt luke-jr implemented - then luke-jr decides for you ...
Did you read the gbt code he wrote when you mined with his BarbieMiner?

You wanna say something is wrong with GBT? Proof?

To correctly state your question:
What are the rules that BarbieMiner uses to select the transactions to put into the GBT blocks?
No doubt you don't already know what they are ... without at least first looking them up now ...
You see the point of my statement?

Edit: and to add to that ... once you finally bother to look up and learn what rules he uses, do you know how to change it to suit what you think it should be?

[kano]

I'm using simple logic here:

1. Most of people here which I think suck on so many levels, even fundamental ones, don't like Luke.
2. Luke proved he is concerned with exactly the same issues - and probably even more of them - I'm concerned with.
3. Luke is surely smart enough to know attempting to overtake / manipulate Bitcoin with open source code won't work.

Unless you can prove GBT is flawed, excluding the "it uses more bandwidth / more stales" part, you better rest your case.

Well following your link I've read a lot there of you trying to tell everyone else what they should do.

You wouldn't happen to be Luke-jr would you ...

Anyway, you've made no case either - you've simply stated you don't care what rules Luke-jr has in GBT.

I will point out the following:

1) Luke-jr's pool uses rules to ignore transactions that he will not divulge
2) Luke-jr used his pool's hashing power (without the knowledge of the miners) to effectively kill off an alt-coin
3) Luke-jr had a performance improvement for the BFL device driver for cgminer that he used himself but didn't release

The simple issue is that everyone has their own opinion of what they think is best ... yet you seem to not care what rules are in the software you use ... yet you also seem to care that you don't know the rules that pools are using.

Your argument's main point is in fact also your own ignorance of that point - that you have access to know something that you don't even care to check before you use it.

My initial statement was to simply point out that the comments about GBT being better than Pools due to 'decentralization' were only valid when solo mining and not better than P2Pool (and worse than P2Pool when not solo mining)

That fact that you blindly trust Luke-jr - well good for you.
I have learnt from dealing with him that I certainly should not do that.

Now to top that off, since you say that you can check these things, I will also point out that you can check these things with any pool.
The transactions available on the network at any particular time are visible to everyone.
The transactions placed in every block mined are also visible to everyone as soon as the block gets into the block chain
You can clearly see for every block mined, what transactions were selected and what were ignored ... if you wish to.

[Luke-Jr]

I think all pools should be shut-down immidiatelly and code put in place to prevent formation of any sort of groups.

This isn't possible even in theory.

Issues with Luke you posted are nothing but minor incidents.

They also didn't happen, or are being presented by Kano out of (important) context.

For other stuff you posted, I'll wait to see Luke's reply.

What in specific would you like a response to? Kano spouts so much lies and nonsense that I generally ignore his posts.

[-ck]

An empty (coinbase-only block) StratumMP work payload is 399 bytes.  An empty GBT work payload based on your example is 737 bytes.

StratumMP work payloads increase 67 bytes per power of 2 transactions.  GBT work payloads increase by the size of the transactions being included in a block (roughly 250 bytes for a normal tx with 1 input, 1 payout address, and 1 change address).

For StratumMP, a block containing 257-512 transactions would require an extra 603 bytes, for a total of 1002 bytes.  GBT would require approximately 128,000 bytes.
Using a more common example, lets say each block has 100 transactions on average.  StratumMP would be 868 bytes per work payload.  The GBT equivalent would be approximately 26,000 bytes.

Similarly, work submission payloads are quite different.  GBT would require 501 bytes per share submission in your example.  StratumMP requires only 109.

None of the above factors in the added overhead of wrapping GBT into HTTP Requests.

What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them.

[jgarzik]

What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them.

Unless something has changed, getblocktemplate should include a mode that skips sending the transactions themselves.  See https://en.bitcoin.it/wiki/BIP_0023

I agree it is unlikely that non-p2pool miners will use the full bore GBT + all transactions, even if it is better for network security.

But don't dismiss GBT based on faulty "full mode only" assumptions.

[kano]

What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them.

Unless something has changed, getblocktemplate should include a mode that skips sending the transactions themselves.  See https://en.bitcoin.it/wiki/BIP_0023

I agree it is unlikely that non-p2pool miners will use the full bore GBT + all transactions, even if it is better for network security.

But don't dismiss GBT based on faulty "full mode only" assumptions.

The problem with not receiving the transactions is that the miner program must have a full bitcoin connection to the network.

Suddenly mining includes bitcoin network transaction and block processing.

... and anyone with half an inkling of understanding of what that entails can see the issues of having to deal with all the complications that entails and then also the importance of the bitcoin network performance of the miner ... which before this point was zero.

The experiment of the issues with this already exists P2Pool.

The only really acceptable solution to this problem, in my opinion, would be to have bitcoind handle that fully for a GBT miner - not have yet another bitcoin network implementation added onto every miner.

But this still has the issue of the reliability of the local bitcoin network connection and the problems that can cause (that don't exist in the current implementations other than p2pool and solo mining)