Help with Bitcoin JSON RPC - more SLOOOWness

[Kazu]

This is your local wannabe-server-guy and as usual I've managed to screw things up again. After realizing that bitcoind was working very slow I found a server with a much faster disk IO I loaded bitcoind on there, and boom, instant good response time!

time bitcoind getinfo
{
    "version" : 80100,
    "protocolversion" : 70001,
    "walletversion" : 60000,
    "balance" : 0.00000000,
    "blocks" : 244761,
    "connections" : 8,
    "proxy" : "",
    "difficulty" : 21335329.11398300,
    "testnet" : false,
    "keypoololdest" : 1374707237,
    "keypoolsize" : 101,
    "paytxfee" : 0.00000000,
    "errors" : ""
}

real    0m0.006s
user    0m0.004s
sys     0m0.000s

After quickly thanking the internet for helping me with that I'd like to ask another favor. The server running bitcoind here can't be my webhost for reasons. So I have done an rpcallowip allowing connection from a remote PHP host. And, wonderfully enough, it works much of the time, with response times about 0.14-ish. Which is fine by me. However, sometimes I get:

Warning: fopen(http://...@myserver:8332/): failed to open stream: Connection timed out in /stuff/jsonRPCClient.php.

Fatal error: Uncaught exception 'Exception' with message 'Unable to connect to http://bitcoinrpc:mypassword@myserver:8332/' in myfile Stack trace: #0 /myfile: jsonRPCClient->__call('listtransaction...', Array) #1 mypath: jsonRPCClient->listtransactions('kazu') #2 mypath: include('file.php') #3 {main} thrown in /stuff/jsonRPCClient.php on line 140

Listtransactions happens to be the first call I make. This error comes after waiting a while. To test, if this was a legitimate issue, I tried telnetting. Now, I'm not 100% sure this means that it should be working working, but I did:

time telnet myserver:8332

and repeated it 100 times, and got back

telnet: could not resolve myserver:8332/telnet: System error

with an average response time of 0.016 seconds.

So is the issue with my bitcoind-machine's internet, or does the telnet experiment imply that its a problem with my client? What should I do?

EDIT: Just tested, I get "telnet: could not resolve server:8332/telnet: System error" even for servers that aren't listening. So, I tried portscanning. I get "port open" for on my server on 8332 in a number of milliseconds. What gives?

[Kazu]

A bit more information, a list of response times for the following code:

<?
$time = microtime(true);
require_once 'jsonRPCClient.php';
$bitcoin = new jsonRPCClient('http://bitcoinrpc:mypassword@myserver:8332/');
echo $bitcoin->getbalance("kazu");
echo "<br>";
echo $save = microtime(true)-$time;
echo "<br>";
file_put_contents('time.txt', $save . PHP_EOL, FILE_APPEND);
?>

0.12215495109558
0.12762403488159
0.12906503677368
0.12794995307922
0.145015001297
0.12692308425903
0.13107013702393
0.13046789169312
0.12596988677979
0.12823700904846
0.12685894966125
0.12688183784485
0.13011193275452
0.13094592094421
0.1299901008606
0.12974500656128
0.13020300865173
31.179399013519
0.12966895103455
0.13000798225403
0.1297299861908
0.13064789772034
0.14454817771912
0.12939882278442
0.12828493118286
31.177670001984
0.12911486625671
0.13026690483093
0.12602400779724
0.129154920578
0.12882518768311
0.12818312644958
0.12817215919495
15.141080856323
0.16296315193176
0.12952899932861
0.19870591163635
0.16739010810852
0.12964200973511
0.16857099533081
0.13083600997925
31.268596887589

There was 1 or 2 timeouts in that process as well.

[gweedo]

Sounds like firewall is making it timeout.

[kostagr33k]

Have you tried running your script locally on the bitcoind Node? without knowing your network setup, Id suggest try doing that a few hundred / thousand times and see if you have any problems.

If you do the test, and no problems occur, immediately run the Remote script and see if any problems come up.


If problems occur locally, then its bitcoind / your server. If problems don't occur locally but happen remotely, then there is a problem between your bitoind server and your remote host.


That could be 1 or more things that could cause that. As suggested above, Firewall would be one of them, but could also be ISP problems on either side, rate limiting, etc.

If you find the problem, I would be interested in knowing.


kosta

[Gavin Andresen]

How big is your wallet?

getbalance() is O(N) where N is the total number of transactions in the wallet. (actually, it can be even worse than O(N) if you have long chains of unconfirmed transactions in your wallet)

"Patches welcome" to fix that, I expected one of the big bitcoind-using services would submit a performance fix patch long ago.

[Kazu]

How big is your wallet?

getbalance() is O(N) where N is the total number of transactions in the wallet. (actually, it can be even worse than O(N) if you have long chains of unconfirmed transactions in your wallet)

"Patches welcome" to fix that, I expected one of the big bitcoind-using services would submit a performance fix patch long ago.

Oh great, so getbalance basically cant be scaled at all?

[Kazu]

Does BitcoinJ also have account management / JSON availability?

[Kazu]

Crap, I think I'm too lazy at this point to have to program all the JSON & Account Management myself. Maybe if my business is successful and needs the scalability I'll look into it.

[Kazu]

Crap, I think I'm too lazy at this point to have to program all the JSON & Account Management myself. Maybe if my business is successful and needs the scalability I'll look into it.

If you're successful it probably be too late. Not one bitcoin company uses a stock bitcoind anymore. The software has become too focused on the end user, which is important for bitcoin-qt but not bitcoind.

Well, how long did it take you to make the JSON/Account Management features?

Maybe I could make some sort of open source project, its truly a pain. Maybe just for me, because I'm bad, but whatever.