Bitcoin Forum
December 06, 2016, 08:26:52 AM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: [1]
  Print  
Author Topic: [PULL] Update RPC server to use asynchronous IO  (Read 1769 times)
jordanlewis
Newbie
*
Offline Offline

Activity: 5


View Profile
May 09, 2011, 06:59:17 PM
 #1

As per http://bitcointalk.org/index.php?topic=6599.0.

This patch set updates bitcoin's RPC server to use boost::asio's async_read, _write, etc instead of their non-asynchronous versions.

The server still exists as a single separate thread, but the async calls should greatly improve performance under high load. The next step in robustifying the RPC server is to create a pool of threads for the RPC server, each of which handle a number of asynchronously reading and writing connections.


Pull request here: https://github.com/bitcoin/bitcoin/pull/214
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481012812
Hero Member
*
Offline Offline

Posts: 1481012812

View Profile Personal Message (Offline)

Ignore
1481012812
Reply with quote  #2

1481012812
Report to moderator
1481012812
Hero Member
*
Offline Offline

Posts: 1481012812

View Profile Personal Message (Offline)

Ignore
1481012812
Reply with quote  #2

1481012812
Report to moderator
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652


Chief Scientist


View Profile WWW
May 09, 2011, 07:19:54 PM
 #2

... but the async calls should greatly improve performance under high load.

Have you done any benchmarking to see if that is true?

How often do you get the chance to work on a potentially world-changing project?
jrmithdobbs
Jr. Member
*
Offline Offline

Activity: 59


View Profile
May 27, 2011, 05:25:27 AM
 #3

... but the async calls should greatly improve performance under high load.

Have you done any benchmarking to see if that is true?


I haven't done benchmarking per-se but it's very easy to test if you have access to miners that can do ~4-5Ghash/s.

Without patch: "Problems communicating with RPC server" (or equiv depending on miner) at least once every 5 seconds.
With patch: One every now and then due to network hiccups.

Further proof: luke-jr has a similar (though VERY hacky) patch he uses or bitcoind falls over for Eligius, his is a threaded implementation that would need quite a bit of cleanup though.

I'm pretty sure, whether publicly stated or not, this is the case for all of the >10Ghash/s pools.

1B8TSDzXdyTRX5eF77gWQoXujBaDtKFE6H
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2086



View Profile
May 27, 2011, 04:26:50 PM
 #4

Further proof: luke-jr has a similar (though VERY hacky) patch he uses or bitcoind falls over for Eligius, his is a threaded implementation that would need quite a bit of cleanup though.
Sorry, but my patch probably doesn't help your case. Some corrections:
  • It's not really that hacky, I just never bothered to make it work with SSL.
  • It doesn't help failover in any way.
  • It requires each RPC command to enable threading-- currently I have only enabled it for getwork.
  • Eligius only needs/uses it for calling RPC command from inside the coinbaser (which blocks getwork).

jordanlewis
Newbie
*
Offline Offline

Activity: 5


View Profile
May 27, 2011, 08:03:09 PM
 #5

Gavin - what kind of evidence do you need in order to take this patch into consideration? I haven't had the resources to perform proper load testing on it, but jrmithdobbs provides encouraging anecdotal data that suggests that the patch does indeed improve performance significantly.

The original implementation was synchronous and blocking: it allowed only one connection to be processed at once. My patch improves this behavior by rewriting the connection handling code to use asynchronous IO - enabling many connections to be processed at once. It should be clear why this improves performance.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652


Chief Scientist


View Profile WWW
May 28, 2011, 07:29:45 AM
 #6

Sounds like this is good for pool operators that have high 'getwork' loads-- have any high-volume merchants sites or e-wallet sites tried it?

I've got a long list of things to do when I get back from vacation week after next, or I'd volunteer to run it on the Faucet to help stress test it.

How often do you get the chance to work on a potentially world-changing project?
dbitcoin
Hero Member
*****
Offline Offline

Activity: 742

BTCDig - mining pool


View Profile WWW
June 10, 2011, 04:15:24 AM
 #7

Any news here?
I already use this patch in my pool more than week and see nice performance boost with rpc.
This code should be verified(probably improved) and merged with master upstream.

BTCDig - mining pool (Stratum, VarDiff, DGM, SSL, JSON API)
xf2_org
Member
**
Offline Offline

Activity: 70


View Profile
June 10, 2011, 05:33:20 AM
 #8

Any news here?
I already use this patch in my pool more than week and see nice performance boost with rpc.
This code should be verified(probably improved) and merged with master upstream.

It's already a pull request.  Right now there is a logjam of pull requests, simply because we're trying to get out some critical bug fixes, and this is not strictly a bug fix.

Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!