Title: How to improve bitcoind signing performance? Post by: dave111223 on April 28, 2015, 02:00:36 PM Currently using bitcoind 0.10 headless on Debian (although had the same issue with all previously bitcoind versions).
The wallet info: Wallet.dat size: 160MB Number of addresses: ~16,000 Unspent outputs: (varies but around: 20-40) The signing function is quite sluggish. Even for simple 1 or 2 UTXO the signing takes around 2-5 seconds. With more complex transactions the signing can take more than 30 seconds! Is there any way to speed up this process (preferably without having to do UTXO consolidation transactions, and just via a setting change) Load on the server seems fairly low, and CPU usage is very low. Memory: 4GB load average: 0.39, 0.43, 0.35 Title: Re: How to improve bitcoind signing performance? Post by: dave111223 on May 20, 2015, 07:52:23 AM Here are some specific examples of this sluggishness:
Code: $ time /path/to/my/bitcoin/bitcoin-cli getbalance "" 3 Code: $ time /path/to/my/bitcoin/bitcoin-cli listunspent The times are consistently 3-4 seconds every time But then some commands are much quicker: Code: $ time /path/to/my/bitcoin/bitcoin-cli getinfo Title: Re: How to improve bitcoind signing performance? Post by: achow101_alt on May 20, 2015, 07:13:42 PM Here are some specific examples of this sluggishness: This probably happens because you have about 16000 addresses, and listunspent and getbalance need to scan for all of the unspent outputs and calculate the balance of all 16000 addresses. This is why it takes a few seconds. getinfo on the otherhand, does not need to scan through all of your addresses, and already has the data that it needs for the output.Code: $ time /path/to/my/bitcoin/bitcoin-cli getbalance "" 3 Code: $ time /path/to/my/bitcoin/bitcoin-cli listunspent The times are consistently 3-4 seconds every time But then some commands are much quicker: Code: $ time /path/to/my/bitcoin/bitcoin-cli getinfo Title: Re: How to improve bitcoind signing performance? Post by: dave111223 on May 20, 2015, 11:55:52 PM But why is it so inefficient?
A Mysql sum on 16,000 records takes fractions of a second on very low spec hardware.... So why does bitcoind take 3-4 seconds? (on very nice hardware) Title: Re: How to improve bitcoind signing performance? Post by: achow101_alt on May 21, 2015, 12:01:30 AM But why is it so inefficient? bitcoind is slower because there are not just 16000 things to look through. It must search through all 34 gigs of blockchain and find all of one addresses transactions, add up the inputs and outputs, and find which transactions have yet to be spent. Then it must repeat this for all 16000 of your addresses in order to return the data you asked for. This is why it takes so long.A Mysql sum on 16,000 records takes fractions of a second on very low spec hardware.... So why does bitcoind take 3-4 seconds? (on very nice hardware) Title: Re: How to improve bitcoind signing performance? Post by: dave111223 on May 21, 2015, 12:26:16 AM bitcoind is slower because there are not just 16000 things to look through. It must search through all 34 gigs of blockchain and find all of one addresses transactions, add up the inputs and outputs, and find which transactions have yet to be spent. Then it must repeat this for all 16000 of your addresses in order to return the data you asked for. This is why it takes so long. Why would it need to search from all 34GB? Caches and indexes are not used? Title: Re: How to improve bitcoind signing performance? Post by: achow101_alt on May 21, 2015, 01:02:33 AM bitcoind is slower because there are not just 16000 things to look through. It must search through all 34 gigs of blockchain and find all of one addresses transactions, add up the inputs and outputs, and find which transactions have yet to be spent. Then it must repeat this for all 16000 of your addresses in order to return the data you asked for. This is why it takes so long. Why would it need to search from all 34GB? Caches and indexes are not used? Title: Re: How to improve bitcoind signing performance? Post by: dave111223 on May 21, 2015, 01:05:42 AM AFAIK Bitcoin Core keeps an index (or something like that) of all of the transactions for the addresses so it may not look through the entire blockchain. However, getbalance and listunspent must still iterate through that list for each of the 16000 addresses in order to find which ones are still unspent and what the total balance is. So what I'm saying is, with a mysql database, even of size 1,000,000 records you can SUM the balances of 16,000 records in fractions of a second. I don't see why bitcoind cannot achieve the same performance? I mean literally if I imported the data from bitcoind into mysql (a list of all my addresses and transactions), it would take me millionseconds to do these same queries? Title: Re: How to improve bitcoind signing performance? Post by: achow101_alt on May 21, 2015, 01:30:45 AM AFAIK Bitcoin Core keeps an index (or something like that) of all of the transactions for the addresses so it may not look through the entire blockchain. However, getbalance and listunspent must still iterate through that list for each of the 16000 addresses in order to find which ones are still unspent and what the total balance is. So what I'm saying is, with a mysql database, even of size 1,000,000 records you can SUM the balances of 16,000 records in fractions of a second. I don't see why bitcoind cannot achieve the same performance? I mean literally if I imported the data from bitcoind into mysql (a list of all my addresses and transactions), it would take me millionseconds to do these same queries? |