EXCEPTION: St9bad_alloc std::bad_alloc bitcoin in ProcessMessages()

[buffett]

my wallet blockchain was crashed and I unable to start it back with -reindex

it already synced to block num 340870, and it will be crashed every 5-10 blocks:

************************
EXCEPTION: St9bad_alloc       
std::bad_alloc       
bitcoin in ProcessMessages()       

2016-01-17 02:12:03 ProcessMessages(block, 749055 bytes) FAILED peer=6

config:

server=1
daemon=1
rpcport=12345
rpcuser=xxx
rpcpassword=xxxx

maxconnections=8
paytxfee=0.0001

minrelaytxfee=0.0001
limitfreerelay=5

the server has 16gb of RAM and 10gb of swap.

the wallet has 100k of addresses

any help would be very appreciated.

thanks

[CIYAM]

the wallet has 100k of addresses

^ That is not a good idea.

The wallet doesn't scale well at all and despite the large amount of RAM you might be hitting a limit that is causing the bad alloc (although a bad alloc could also be caused through passing an invalid value for allocation).

To work out whether the problem is your wallet's size back it up and then delete it and re-start Bitcoin (it will create a new empty wallet). If no crashes happen with the empty wallet then I would assume your problem is the huge wallet.

[buffett]

the wallet has 100k of addresses

^ That is not a good idea.

The wallet doesn't scale well at all and despite the large amount of RAM you might be hitting a limit that is causing the bad alloc (although a bad alloc could also be caused through passing an invalid value for allocation).

To work out whether the problem is your wallet's size back it up and then delete it and re-start Bitcoin (it will create a new empty wallet). If no crashes happen with the empty wallet then I would assume your problem is the huge wallet.

i know it is not a good idea, but i need to get this wallet up. any ideas?

[CIYAM]

i know it is not a good idea, but i need to get this wallet up. any ideas?

Well first I'd suggest you work out whether the problem is your wallet - assuming that it is then I hope for your sake you have a backup that still works (if not you are going to have a very hard time especially if the wallet is encrypted).

[buffett]

what i'm really confused is the server doesn't seem to run out of memory. only 3gb being used.

[CIYAM]

what i'm really confused is the server doesn't seem to run out of memory. only 3gb being used.

Firstly, have you verified that it is the wallet that is causing your problem?

(if you haven't then there is very little point in my continuing to offer any advice)

[buffett]

i was trying to avoid resync the whole blockchain if possible (it already takes 1 day to reach 34xxxx).

but I will do it anyway. and see what happen. will update later.

thanks for your help.

[CIYAM]

i was trying to avoid resync the whole blockchain if possible (it already takes 1 day to reach 32xxxx).

I didn't suggest that you resync the whole blockchain at all - backup your wallet and then delete it is all I suggested (after first stopping bitcoind).

That shouldn't take you more than a minute to do (after you restart bitcoind it will create a new empty wallet which is done almost instantly).

[buffett]

is it possible to create a new wallet without resync?

tried to move the wallet.dat, starting bitcoind gave me

Code:

Corrupted block database detected.

Do you want to rebuild the block database now?

[buffett]

or did you mean to backup whole .bitcoin directory?

[CIYAM]

corrupted block database detected.

Then your problem is a corrupted blockchain (whether or not your wallet has anything to do with it) so unfortunately you are simply going to have to repair that before trying to use your original wallet again.

I would suggest you consider setting up a separate machine (or VM) for working on this and keep a copy of the blockchain (once you have the full uncorrupted version) somewhere handy.

The point being that if it was due to your wallet's size that somehow the blockchain got corrupted (unlikely but I guess possible) then you might find that as soon as you put back your big wallet and restart it might corrupt the blockchain again (putting you back to square one).

So once you have a clean blockchain (with an empty wallet) back that up (the whole thing) before trying to put back in you big wallet.

[buffett]

i'm installing fresh bitcoind on new server now.

[shorena]

i'm installing fresh bitcoind on new server now.

Did you use a 32-bit version by any chance? That would explain the 3GB limit.

[CIYAM]

Did you use a 32-bit version by any chance? That would explain the 3GB limit.

I was guessing the same thing but wanted to make sure exactly what was causing the crash first (if the super large wallet is causing blockchain corruption then perhaps that is a bug that should be reported).

OP should note that if they are using Windows then they are using a 32 bit version (so a possible fix might be to change to a 64 bit Linux version).

[buffett]

hahaha yes it is very silly, it turns out I was using 32bit box 

no wonder bitcoind crashes when reach 4gb and ignore the rest of the memory

i already moved it to 64bit and everything seems to work fine.

thanks for the help 

anyway, for anyone have similar issue but with limited RAM, using -dbcache=16 really do the trick. it works even for 100k+ addresses but every command become very slow.

[CIYAM]

Glad to know that you fixed your problem but I would recommend that you reconsider the idea of having such a huge wallet (especially if it is going to grow much bigger).

Many of the RPC functions will perform poorly because of that (and the problem will just get worse and worse).

[buffett]

yes i already built a load balancer for few bitcoind wallets, and of course they will be separated into fewer addresses.

and this wallet will not grow, i will leave it as it is because i don't have idea how to split the wallet.dat easily

[CIYAM]

and this wallet will not grow, i will leave it as it is because i don't have idea how to split the wallet.dat easily

You can export and then later import the private keys (am not sure if the exporting can be done in one go though - you'd need to check that).

[buffett]

as far as i know exporting privatekey can only be done one by one through API.

[CIYAM]

as far as i know exporting privatekey can only be done one by one through API.

It might not be too hard to whip up a script that iterates the addresses and then dumps the private keys for them.