slow creation of thansaction on big wallet.dat

[kimon]

Hello community!

My problem is that creation of outgoing transactions are very slow.

My wallet have ~1500 active bitcoin-addresses with total ~20000 input transaction and ~5000 output transactions in last half of the year.
Size of file wallet.dat is 240Mb
Bitcoin Core 0.10.2

I create outgoing payment with command:
bitcoin-cli sendfrom <fromaccount> <tobitcoinaddress> <amount>

So the problem is it takes about 60 seconds every time when I execute this command (on CPU FX6300,6-Core,3.5GHz). This is too slowly.
On new wallet with minimum addresses and transactions this command executed less than 1 second.

If I have 40000 incoming transactions will it takes for 2 minutes?! ((

Why it takes so long time? How it is possible to speed it up?

Please help me to solve this problem.

[1715668901]

1715668901

[1715668901]

1715668901

[1715668901]

1715668901

[1715668901]

1715668901

[achow101]

The problem is the size. If your <fromaccount> is all of your addresses, then it must search through every single unspent output for every single address in the wallet. Since there are thousands of addresses and thousands of unspent outputs for each address, it can take a long time to search through everything in order to create your transaction.

Unfortunately, I don't know how you can speed this up, and I'm pretty sure that you can't speed it up. I would advise consolidating all of your Bitcoin into one address and create a new account in Bitcoin Core with only that address. It should be faster that way since it will only have to search through I address and a few unspent outputs.

[kimon]

it must search through every single unspent output for every single address in the wallet. Since there are thousands of addresses and thousands of unspent outputs for each address, it can take a long time to search through everything in order to create your transaction.

Does really wallet.dat has no indexes for unspent outputs?
Well known that in Relational-DB we can find 1 record from 1 million records less than for 1 second using indexes.

[achow101]

it must search through every single unspent output for every single address in the wallet. Since there are thousands of addresses and thousands of unspent outputs for each address, it can take a long time to search through everything in order to create your transaction.

Does really wallet.dat has no indexes for unspent outputs?
Well known that in Relational-DB we can find 1 record from 1 million records less than for 1 second using indexes.

I'm not sure. It isn't the wallet.dat but rather the levelDB databases that Bitcoin Core uses for everything. You should lookup some stuff about levelDB.

[kimon]

In this case I see something strange:
1. Execute command bitcoin-cli listunspent take a 5 second and return 107 unspent outputs;
2. After this command bitcoin-cli sendfrom <fromaccount> <tobitcoinaddress> <amount> nevertheless take 40 seconds.

knightdk if you are right, why so much difference in time?

[achow101]

In this case I see something strange:
1. Execute command bitcoin-cli listunspent take a 5 second and return 107 unspent outputs;
2. After this command bitcoin-cli sendfrom <fromaccount> <tobitcoinaddress> <amount> nevertheless take 40 seconds.

knightdk if you are right, why so much difference in time?

That is interesting. I thought that searching through everything and finding all of the unspent outputs would take up most of the time. The remaining 35 seconds could be from needing to sign each unspent output from a different address if they all went to different addresses and just the creation of the transaction. I'm not quite sure then.

[kimon]

May be this is some bag in Bitcoin Core.
Can some developer look at this problem?

[jonas.schnelli]

Just wanted you to know that I'm working on this problem by rewriting most of the bitcoin-cores wallet: https://github.com/jonasschnelli/bitcoin/tree/2015/05/corewallet

So help is on the way, but not for today and tomorrow.

[kimon]

In new Bitcoin Core v0.11.0 time of creation transaction (by sendfrom) reduced to about 20 second.
It's good! But is possible to faster?