Bitcoin Forum
November 25, 2017, 10:04:00 AM *
News: Latest stable version of Bitcoin Core: 0.15.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 [2] 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 »
  Print  
Author Topic: [ANN] Stratum mining protocol - ASIC ready  (Read 143698 times)
eleuthria
Legendary
*
Offline Offline

Activity: 1750



View Profile
September 11, 2012, 11:43:38 PM
 #21

Giving this a little boost:  This is way too important to let it slip from the top topics.
It'll take a while for those of us who've only just seen it to implement it. I'm interested in adding support for it to my pool but it'll take time to write and test.

I know it will take a while for more pools to adopt it, especially when most of the pool operators are utilizing projects like ecoinpool, poolserverj, and pushpool.  Luckily, slush has provided an open source pool example for the new protocol along with his proxy, allowing anyone to step through the protocol from both sides to gain a better understanding of it and implement their own version (either by editing the example or starting fresh).

You can contact slush or myself in #stratum on FreeNode if you're running into any problems or have questions about efficiency/scalability.


doubles, just telling anyone's support is valuable feedback at this time. Any vote counts, because there's still a long way to roll native support of this protocol into major miners. During this period we'll have painfull times while running both pools at the same time Smiley.

As slush posted, just identifying interest in implementing the new protocol on your pool will help give a push to mining software developers.  Likewise, as mining software developers express interest in native support, it will hopefully encourage more pools to begin adopting the new protocol as well.

RIP BTC Guild, April 2011 - June 2015
1511604240
Hero Member
*
Offline Offline

Posts: 1511604240

View Profile Personal Message (Offline)

Ignore
1511604240
Reply with quote  #2

1511604240
Report to moderator
Join ICO Now A blockchain platform for effective freelancing
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1511604240
Hero Member
*
Offline Offline

Posts: 1511604240

View Profile Personal Message (Offline)

Ignore
1511604240
Reply with quote  #2

1511604240
Report to moderator
doublec
Legendary
*
Offline Offline

Activity: 1078


View Profile
September 11, 2012, 11:47:59 PM
 #22

I know it will take a while for more pools to adopt it, especially when most of the pool operators are utilizing projects like ecoinpool, poolserverj, and pushpool.  Luckily, slush has provided an open source pool example for the new protocol along with his proxy, allowing anyone to step through the protocol from both sides to gain a better understanding of it and implement their own version (either by editing the example or starting fresh).
Yes, slush's example will be useful. My pool is custom software which I think will actually make it easier for me since I know it better than trying to add support to other pool software. I think it's great that you and slush sorted out the differences between the proposals and hope the new protocol gets some traction.
Mobius
Hero Member
*****
Offline Offline

Activity: 946



View Profile
September 11, 2012, 11:52:36 PM
 #23

how is the port changed on the proxy since this conflicts with p2pool proxy?
slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
September 11, 2012, 11:59:07 PM
 #24

mining_proxy.py --help will prints all options.

Specifically to your request, use "mining_proxy.py -gp 8330" (or any other port number). "gp" identifies "getwork port"

Luke-Jr
Legendary
*
Offline Offline

Activity: 2268



View Profile
September 12, 2012, 12:24:26 AM
 #25

I've written up a forum post for getblocktemplate; replies on that topic are probably off-topic on this thread.

slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
September 12, 2012, 10:15:34 PM
 #26

I just released new version of mining proxy, version 0.5.0. Older versions still works nicely, this adds some few features:

* Implemented method client.reconnect(host, port), so pool can now easily balance clients between backends or gracefully shutdown a backend without a miner outage.

* Added "--no-midstate" parameter. When used, proxy won't generate "midstate" field in the getwork response. Some old miners and Diablo still requires this field, but modern miners like poclbm/cgminer can generate midstate ourselves and in much faster way. So if you're using these miners and want to save a bit of CPU time, this argument is just for you. It speeds up getwork creation twice.

In the near future I'll publish complete API documentation of such commands like client.reconnect(), client.get_version() or mining.get_hashrate(). Stay tuned :-).

slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
September 13, 2012, 05:43:40 AM
 #27

Two hours ago my Stratum-powered pool core mined its first block: http://blockchain.info/block/0000000000000322b90111260dcbb877c62c14e672c23a99a86ca444dc5525ee?site=slush

This is a confirmation that opensource stack which I provided builds blocks which are accepted by main chain.

dooferorg
Full Member
***
Offline Offline

Activity: 156


View Profile
September 14, 2012, 06:19:15 AM
 #28

I noticed that BTCGuild, a favored pool that I use, is testing the stratum mining proxy and that's what lead me here. I've set this up and it's working well. No stales/dupes at all from a host that would occasionally show a few.

I also use a PHP based proxy for managing pools and workers: https://github.com/cdhowie/Bitcoin-mining-proxy

Of course for testing it I have a pool set to my local machine that is running the stratum proxy.

I may consider trying to integrate these (being a programmer myself) so that different pools can be set up easily and have 'stratum proxying' enabled if they support it. Seems very sensible as I look over the description of the work and protocol that you listed. If I do make a PHP version that works like your one I will of course post it here. Do you think the protocol is pretty much 'set' for now and it's just a matter of testing and getting it rolled out?


BTC: 1dooferoD3vnwgez3Jo1E4bFfgMf81LR2
ZEC: t1gnToN2HZW4GD52kofEVdijhRijWjCNfYi
slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
September 14, 2012, 09:42:51 AM
 #29

Yes, I think that protocol itself is stable. I'll just add some additional methods for managing connections, like client.reconnect() or so, but it's not directly related to protocol core and you can implement it "as is".

I'm just not sure how you can implement persistent connections with PHP. Maybe running "official" python proxy and implement your software to talk to it locally would be smarter (and easier) choice.

eleuthria
Legendary
*
Offline Offline

Activity: 1750



View Profile
September 14, 2012, 02:34:26 PM
 #30

Just to add to the good news, BTC Guild's stratum beta also found it's first block on the live Bitcoin network: http://blockchain.info/block-height/198695

Before beta ends I'll have to go back and add back our tag in the coinbase stating it came from BTC Guild.

RIP BTC Guild, April 2011 - June 2015
makomk
Hero Member
*****
Offline Offline

Activity: 686


View Profile
September 14, 2012, 03:40:37 PM
 #31

I notice that this spec doesn't seem to document what values of ntime are acceptable. That's a bit unfortunate.

Edit: Also, clean_jobs probably ought to be split up. As it stands there's no way of signalling that miners should stop working on old jobs immediately without also causing them to discard any unsubmitted shares they might have, but for some setups (e.g. ones that use merged mining or p2pool) not being able to force miners onto new work without discarding shares kills efficiency.

Quad XC6SLX150 Board: 860 MHash/s or so.
SIGS ABOUT BUTTERFLY LABS ARE PAID ADS
slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
September 14, 2012, 04:20:42 PM
 #32

I notice that this spec doesn't seem to document what values of ntime are acceptable. That's a bit unfortunate.

The best value is actual ntime. Actually rolling ntime is unnecessary with Stratum. But I understand that at least until miners will support Stratum natively, it's necessary to accept rolled ntime. My pool currently accepts ntime in interval <job_ntime, current_ntime+600>, which fits all known current configuration (it has been tested with BFL FPGA rig doing around 30GHash/s as a single device).

Quote
Edit: Also, clean_jobs probably ought to be split up. As it stands there's no way of signalling that miners should stop working on old jobs immediately without also causing them to discard any unsubmitted shares they might have, but for some setups (e.g. ones that use merged mining or p2pool)

I think it is stated clearly. If miner received "clean_job" flag, it should not submit shares from previous jobs. Nothing serious will happen if you submit stale job, they just probably won't be accepted. It's up to server if/when he decides to send such flag.

Quote
not being able to force miners onto new work without discarding shares kills efficiency.

If I understand this correctly, it's inefficiency included in merged mining concept.

jgarzik
Legendary
*
Offline Offline

Activity: 1484


View Profile
September 14, 2012, 04:38:06 PM
 #33

Custom binary protocol using Google Protocol Buffers would be much preferred.  You write

Quote
Protocol buffers by Google is interesting concept which may fit the needs, except that only C++, Python and Java are supported.

Your list of supported languages is woefully incomplete, and therefore, does not support the argument.  PB are supported for: C, C++, Python, Perl, Java, PHP, Ruby, and Haskell at a minimum.  Any language used by the bitcoin community is highly likely to be supported by PB.

Furthermore, PB encoding is strict and well debugged, unlike any newly created protocol (including my own binary protocol from years ago) and this new protocol.

Something for ASIC miners should be bandwidth and CPU efficient, and text/JSON is not that.  This has already been shown to be a problem in bitcoind, where pool server operators replaced the default JSON hex decoding routines with a faster routine, to decrease CPU usage associated with useless text encode/decode.

PB is used in thousands of apps, including most of the Google applications we all use (like search).


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
September 14, 2012, 06:00:46 PM
 #34

Jeff, it makes me smile that after all of our discussion you became a pb advocate Smiley.

Actually the amount of data is *so* tiny that even json message fits in one tcp packet in most of cases. Plus json has an advantage od human readability.

Protocol is not bounded to miner performance at all, so theres no reason why asic miners should need more compressed format. And the cpu overhead is more the issue on server side than on miners. I still picked json, because I expect that miner developers would be agains including another external dependency in their software.

Luke-Jr
Legendary
*
Offline Offline

Activity: 2268



View Profile
September 14, 2012, 06:52:11 PM
 #35

Something for ASIC miners should be bandwidth and CPU efficient, and text/JSON is not that.  This has already been shown to be a problem in bitcoind, where pool server operators replaced the default JSON hex decoding routines with a faster routine, to decrease CPU usage associated with useless text encode/decode.
I'm not sure this has been a problem for anything but bitcoind.

dooferorg
Full Member
***
Offline Offline

Activity: 156


View Profile
September 15, 2012, 06:14:08 AM
 #36

Yes, I think that protocol itself is stable. I'll just add some additional methods for managing connections, like client.reconnect() or so, but it's not directly related to protocol core and you can implement it "as is".

I'm just not sure how you can implement persistent connections with PHP. Maybe running "official" python proxy and implement your software to talk to it locally would be smarter (and easier) choice.

Yes, I had pondered this too after I wrote my previous post. It might indeed not be worth it to implement it in PHP. I have written PHP processes that sit and run outside of a webserver (daemons, if you will). Within PHP too you can do all the regular socket operations so persistent network connections would be okay, there is just the question of whether it's worth duplicating the effort - probably not Smiley

Since I run a proxy for talking with different pools and having it manage the failover to those different pools so my miners can just keep plugging away I was thinking about how best to set up multiple stratum connections, i.e. one for each of the pools that will implement it.

As you mention, it might be best to simply run N instances of the python proxy and just plug that info into the PHP web-based proxy that I use right now. Would certainly mean less effort to implement. K.I.S.S. as they say.

BTC: 1dooferoD3vnwgez3Jo1E4bFfgMf81LR2
ZEC: t1gnToN2HZW4GD52kofEVdijhRijWjCNfYi
Hawkix
Hero Member
*****
Offline Offline

Activity: 517



View Profile WWW
September 16, 2012, 10:10:22 AM
 #37

After a day of playing and running through the Stratum mining proxy I must say I am really HAPPY with it. I can now manage 5 GPUs (1700 MH/s) total over 8 kbit/sec (yes, 1000 chars per second) poor local slow-ping GPRS limited connection link (tunelled via SSH with compression) without any problems and minimal stales! Something I was not able to accomplish previously with http requests (a lot of them).

Kudos to Slush! I think you should really more advertise this method.

Donations: 1Hawkix7GHym6SM98ii5vSHHShA3FUgpV6
http://btcportal.net/ - All about Bitcoin - coming soon!
marcus_of_augustus
Legendary
*
Offline Offline

Activity: 2436



View Profile
September 16, 2012, 11:04:35 AM
 #38

Sounds interesting. Is it any way, except name, related to the Stratum (electrum) client-server protocol used for bitcoin transaction forwarding?

slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
September 16, 2012, 01:03:08 PM
 #39

The protocol has been designed for Electrum client and it is fully compatible. "Stratum mining" is just the Stratum service used for bitcoin mining instead of providing user's balance...

GernMiester
Sr. Member
****
Offline Offline

Activity: 284


View Profile
September 16, 2012, 06:37:00 PM
 #40

Tried it. Like it. Using it.
Pages: « 1 [2] 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!