[ANN] Stratum mining protocol - ASIC ready

[makomk]

There's new official release of poclbm with native Stratum support. Can anybody try it on Stratum-powered pools and confirm that it's stable? Unfortunately I don't have any GPU around .

Gah! I was using a patched version of poclbm with my toy FPGA miner, but it's down right now due to me not having the right software installed to reload the bitstream onto it. Sorry.

[eleuthria]

I've run some poclbm testing with BTC Guild and fixed a few errors in how the pool was responding (poclbm wasn't liking BTC Guild).

Everything seems to be working fine now, but I could only run limited testing since my only GPU machine is a Windows box with a 6970.  That thing turns into an electric heater very quickly and my feet are right next to the exhaust.

[jgarzik]

Any chance of doing zlib-compressed JSON on the wire, rather than uncompressed text?

[Hawkix]

Any chance of doing zlib-compressed JSON on the wire, rather than uncompressed text?

Or, even better, using something like SSL (with compression) to avoid possible man-in-the-middle attack on the Stratum connection?

[flower1024]

Yesterday I had a speech about Stratum on London conference. I think that it has been accepted by audience nicely. I was pretty surprised that nobody asked me for the difference between Stratum and getblocktemplate; I had many arguments in my sleeves :-).

Unfortunately I forgot to say many things which originally prepared for the presentation :-), especially the benefits for miners. It has been my first public talk, so hopefully I'll improve my skills for next time.

the biggest argument is: getblocktemplate gets the power of mining back in the hands of the miners.

with your proposal its still the pools who could(!!!) do malicious things.

in times of gpumax i know that i have to prefer getblocktemplate to guarantee a (long-term) healty network

[slush]

Any chance of doing zlib-compressed JSON on the wire, rather than uncompressed text?

I'm not thinking about this currently. I see 10kB/minute (at max!) as such small amount of data that it doesn't need any other optimization at this stage. Compressing one packet data to fit half of the packet don't make a sense to me.

[slush]

Or, even better, using something like SSL (with compression) to avoid possible man-in-the-middle attack on the Stratum connection?

Supporting SSL (without a compression) is a matter of enabling it in my pool config, so maybe, one time. I didn't test SSL overhead under heavy load yet, so I'll need to spend some time on it before provide SSL transport to wide audience.

[slush]

the biggest argument is: getblocktemplate gets the power of mining back in the hands of the miners.

I agree with you, except that 99% of real miners don't care. Switching over trusted pool is more optimized way for achieving diversity. Blockchain is public, so finding out malicious intent is as easy as inspect pool's blocks, so people who care can do this instead of overloading pools with more verbose protocols. I mean - hacking miners to report malicious behaviour and rejecting strange transactions is just for uber-geeks. The rest will just use miners in default configuration. With GBT, this makes major overhead for all participants - pool ops (handling higher loads on the pool), developers (supporting all features by the pools is more difficult) and miners (there's significantly higher network traffic all time).

with your proposal its still the pools who could(!!!) do malicious things. in times of gpumax i know that i have to prefer getblocktemplate to guarantee a (long-term) healty network

Yes, they could. And still tons of people trust gpumax with their hashpower. It's a nice example how people don't care. Don't be naive that GBT mining over gpumax will improve *anything*.

[flower1024]

the biggest argument is: getblocktemplate gets the power of mining back in the hands of the miners.

I agree with you, except that 99% of real miners don't care. Switching over trusted pool is more optimized way for achieving diversity. Blockchain is public, so finding out malicious intent is as easy as inspect pool's blocks, so people who care can do this instead of overloading pools with more verbose protocols. I mean - hacking miners to report malicious behaviour and rejecting strange transactions is just for uber-geeks. The rest will just use miners in default configuration. With GBT, this makes major overhead for all participants - pool ops (handling higher loads on the pool), developers (supporting all features by the pools is more difficult) and miners (there's significantly higher network traffic all time).

with your proposal its still the pools who could(!!!) do malicious things. in times of gpumax i know that i have to prefer getblocktemplate to guarantee a (long-term) healty network

Yes, they could. And still tons of people trust gpumax with their hashpower. It's a nice example how people don't care. Don't be naive that GBT mining over gpumax will improve *anything*.

hi, thanks for your good answer,
but i can't agree with you.

when the miners dont care about network health i think the devs should come up with a solution which forces the miners to take care.

i know that your protocol is superior in terms of size.

but getblocktemplate could solve the 51% problem - imagine that every miner-software ONLY supports this protocol. how could a miner say "i dont care?" - he is just forced to do the right thing.

and btw he does not have to do any hacking. afaik he needs his own bitcoind with (preferably) a good network connection - thats the only drawback i see for small miners.

[slush]

and btw he does not have to do any hacking. afaik he needs his own bitcoind with (preferably) a good network connection - thats the only drawback i see for small miners.

a) I don't think it's feasible to run bitcoind on every mining machine, in mid-term and long term.
b) Validating custom-built PoW on pool side is *insanely* difficult, from the view of CPU resources.

I see quite probable scenario for all this: Miners who care will use P2Pool. I like P2Pool and I think it's quite nice piece of software. The rest will ends up with super-scaling protocol on tens or hundreds pools around the world. Stratum is supposed to be really optimal for such situation and as far as I can tell, it's working nicely.

[flower1024]

a) I don't think it's feasible to run bitcoind on every mining machine, in mid-term and long term.
b) Validating custom-built PoW on pool side is *insanely* difficult, from the view of CPU resources.

a) atm its not a problem.
if it gets one there may be centralised bitcoind services (e.g. blockchain.info and so on) which solves that - this would be better than the current pools as it is more easy to switch your bitcoind service (as it does not affect your payouts at all)

b) yes its more load on the server. but how much more? maybe luke-jr should eloberate on this.

p2pool: i like it. but it has failed. miners are greedy - thats the reason i want a way to force them to do the right thing.
btw. i could just use getblocktemplate with eligius since a few month now...and archieve the same as i would use p2pool (except for the different reward scheme).

[Luke-Jr]

b) Validating custom-built PoW on pool side is *insanely* difficult, from the view of CPU resources.

b) yes its more load on the server. but how much more? maybe luke-jr should eloberate on this.

It's not been a problem, nor does it use any more CPU resources than StratumMP would to do the same thing (limiting changes to just a fixed-size extranonce does not improve the situation).

[slush]

p2pool: i like it. but it has failed. miners are greedy - thats the reason i want a way to force them to do the right thing.

I see this is major difference between us. I'm not here to force people to do right thing. If I want to do so, I can get quite a nice job in my government.

[flower1024]

w

p2pool: i like it. but it has failed. miners are greedy - thats the reason i want a way to force them to do the right thing.

I see this is major difference between us. I'm not here to force people to do right thing. If I want to do so, I can get quite nice job in my government.

we could argue long if thats the right thing

other way around: why is code in bitcoind to disallow double-spends? because its the right thing to do.

as i see it the same goes for "who builds the block":
a) pools build the block: huge (possible) risk that they do some nasty stuff
b) miners build the block: risk is reduced

if its possible why dont just use the way which is better for the whole network? that way anyone will profit from it in the long run

[slush]

About CPU resources - as usual, Luke is mixing all stuff together. GBT is quite complex specs, so:

a) You can achieve quite similar result as stratum with GBT, which is *almost* as effective on validating shares as stratum (i don't want to dig into differences for this moment), but it is *not* decentralized.
b) You can achieve decentralization, by leave tx selection and block creation to the miners, but then validating of shares became real hell on the pool side and bandwidth usage and I'm almost sure that no major pool is going to support *this* possibility, because it's insanely difficult to validate everything properly.

[slush]

So let's sumarize up everything:

a) We definitely need a way how to give a possibility of decentralized mining. We already have p2pool and GBT, so we can mark this as already DONE.
b) We (at least me and other pool ops and miners) want some very optimized protocol which will minimize usage of all resources to minimum. This was missing until now, but with Stratum it is DONE.

I don't understand what we're discussing here and why we're fighting each other. I'm not here to say that I want Stratum support in bitcoind. I'm not here to say that everybody HAVE TO mine on centralized pool. I just give a possibility to do centralized mining as easy as possible, replacing current non-optimized, but widely supported getwork on the pools. Why not to have such option?

[flower1024]

my (only) problem is that stratum does not support decentralized mining.

i just hope that new inventions support the core idea of bitcoin (which is IMHO power to the miners to reduce 51% risks)

as stratum is easier to implement from pool side and decentralized mining is not well understood by miners - i dont like that stratum will succeed.

this could have been a chance to reduce the risk from pools to bitcoin itself.

[slush]

This is a freedom of a choice. If you don't like stratum, don't use it. I'm pretty sure you're not using any current pools, so there's no change for you - you can already use another existing solution which fits your needs.

There's no reason to think that all services on bitcoin must be decentralized. Actually decentralization was a problem in times when only my pool and maybe deepbit were available. Now with tens of pools it ends up with *almost* decentralized network, which is still very effective (or can be, after replacing getwork to any better solution).

Let free market to decide it.

[flower1024]

This is a freedom of a choice. If you don't like stratum, don't use it. I'm pretty sure you're not using any current pools, so there's no change for you - you can already use another existing solution which fits your needs.

you're right... as soon as i get my bfl asic i'll take eligius
as i like functional programming its a natural choice anyway.

[slush]

...my bfl asic...

Looks like you care about decentralized mining, but don't care about centralized manufacturing of miners, which is much bigger risk for bitcoin network <sarcasm/>