Bitcoin Forum
December 11, 2016, 12:15:06 PM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: [EXPERIMENTAL] Patch to move work generation to pushpoold, faster? + merged mine  (Read 1716 times)
makomk
Hero Member
*****
Offline Offline

Activity: 686


View Profile
October 10, 2011, 04:58:08 PM
 #1

IIRC, one of the factors limiting pools has been how slow bitcoin is to answer getwork requests. It turns out that it's reasonably easy - with a few tweaks to the Bitcoin client - to move most of the work-generation out of bitcoind and into the pool server, with the pool server using one modified work item from Bitcoin to answer essentially as many getwork requests as it likes. So that's what I did:

https://github.com/makomk/bitcoin/tree/poolserv-work-gen https://github.com/makomk/pushpool/tree/local-work-gen

The changes are relatively straightforward: there's a new getworkex RPC call that provides a copy of the coinbase transaction and the merkle branch it's on, and when submitting solutions the pool server can send in a modified coinbase tx that's used instead of the original. pushpoold in turn now knows how to use getworkex, insert its own additional nonce into the coinbase, increment that nonce to generate work items,  and convert the submitted shares back into a form that bitcoind will accept. A very quick-and-dirty benchmark suggests it should be able to reach in the ballpark of around 3000 getworks/sec, which is much better than I was seeing before. (I'm planning to implement a way for bitcoin to push new work to the pool server at some point too.)

These changes are also incredibly experimental; I've basically only tested them with poclbm and cgminer on a testnet-in-a-box install so far. I hear that luke-jr has been working on something vaguely similar too, and shadders is developing another approach to poolserver work generation in PoolServerJ as well.

Edit: Exercise for the curious: the Bitcoin patch should in theory be enough to implement the parent blockchain side of merged mining, maybe even more effectively than the existing approach. You can use getworkex to add whatever you like to the coinbase transaction, and it gives you the merkle branch needed to submit the work to the auxiliary blockchains.

Quad XC6SLX150 Board: 860 MHash/s or so.
SIGS ABOUT BUTTERFLY LABS ARE PAID ADS
1481458506
Hero Member
*
Offline Offline

Posts: 1481458506

View Profile Personal Message (Offline)

Ignore
1481458506
Reply with quote  #2

1481458506
Report to moderator
1481458506
Hero Member
*
Offline Offline

Posts: 1481458506

View Profile Personal Message (Offline)

Ignore
1481458506
Reply with quote  #2

1481458506
Report to moderator
1481458506
Hero Member
*
Offline Offline

Posts: 1481458506

View Profile Personal Message (Offline)

Ignore
1481458506
Reply with quote  #2

1481458506
Report to moderator
You can see the statistics of your reports to moderators on the "Report to moderator" pages.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481458506
Hero Member
*
Offline Offline

Posts: 1481458506

View Profile Personal Message (Offline)

Ignore
1481458506
Reply with quote  #2

1481458506
Report to moderator
1481458506
Hero Member
*
Offline Offline

Posts: 1481458506

View Profile Personal Message (Offline)

Ignore
1481458506
Reply with quote  #2

1481458506
Report to moderator
1481458506
Hero Member
*
Offline Offline

Posts: 1481458506

View Profile Personal Message (Offline)

Ignore
1481458506
Reply with quote  #2

1481458506
Report to moderator
shads
Full Member
***
Offline Offline

Activity: 224


View Profile WWW
October 11, 2011, 02:09:41 AM
 #2

I hear that luke-jr has been working on something vaguely similar too, and shadders is developing another approach to poolserver work generation in PoolServerJ as well.

Actually I'm implementing a work generator module that can either run embedded inside poolserverj or as a standalone.  The reason for the latter is that psj can get pretty busy with other things as well so you may still want to spread that load across two servers... The standalone will not use rpc.  It will simply stream work to psj in a compact binary format and psj controls the flow by sending stop/start signals.

PoolServerJ Home Page - High performance java mining pool engine

1LezqRatQz7MeNoCVziYwcdwtqeEbvrdAq - http://payb.tc/shads

Quote from: Matthew N. Wright
Stop wasting the internet.
makomk
Hero Member
*****
Offline Offline

Activity: 686


View Profile
October 12, 2011, 12:23:24 AM
 #3

Actually I'm implementing a work generator module that can either run embedded inside poolserverj or as a standalone.  The reason for the latter is that psj can get pretty busy with other things as well so you may still want to spread that load across two servers... The standalone will not use rpc.  It will simply stream work to psj in a compact binary format and psj controls the flow by sending stop/start signals.
Neat! I couldn't figure out whether that made sense or not; work generation is embarassingly parallel and easy to make threaded, but some of the other bits of a pool server aren't. I wasn't sure how quickly the other bits would become a bottleneck.

Also: I appear to now have merged mining working without using merged-mining-proxy! Just for a single aux chain at the moment and it's rather untested, but still. Uses https://github.com/makomk/pushpool/tree/local-work-gen-mm and a very  slightly patched version of namecoin in
https://github.com/makomk/bitcoin/tree/namecoin-getauxblock-prevhash - each chain gets its own shares table, and in theory shares are recorded as stale for each chain independently of the others. (In practice this isn't quite working right; shares that are stale on the parent chain are treated as stale for them all, and the client is only told if they're stale on the parent.)

Code:
    "auxchains" : [
        {
            "rpc.url" : "http://127.0.0.1:9337/", "rpc.user" : "", "rpc.pass" : "foo",
            "stmt.sharelog":"INSERT INTO shares_nc (rem_host, username, our_result, upstream_result, reason, solution) VALUES (?, ?, ?, ?, ?, ?)"
        }
    ]

Quad XC6SLX150 Board: 860 MHash/s or so.
SIGS ABOUT BUTTERFLY LABS ARE PAID ADS
Balthazar
Legendary
*
Offline Offline

Activity: 2002


Post rank racist


View Profile
February 05, 2012, 12:24:22 AM
 #4

Looks very nice, and works. Good solution for systems with small amount of RAM. But I don't understand, why you aren't use getmemorypool RPC call instead of your custom RPC call. Roll Eyes

novaco.in | VMWare VMHGFS driver for 3.19.x kernels | Ukrainian translation for Novacoin [Done]
฿: 1GV8D5SRkA3cPccpYhVc2wMkjwz3UREEpy: 4RgnHWtnJWEyMhqhDdazW3Hdr7cx5ybF6i
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!