Keep getting "Transaction creation failed"

[jetmine]

Hi,

This is an issue that is plaguing me since months, and I'd like to ask if there is any solution to it.

Often, when I try to send a large amount of BTC, I get the following message:
error: {"code":-4,"message":"Error: Transaction creation failed  "}

If this error happens, I try sending a smaller amount.  Eventually this succeeds.  There is always a treshold below which the transaction can be created successfully.  However the treshold is not a static value, but rather depends on some internal details of my wallet.  Sometimes the treshold is lower than 1 BTC, sometimes it is near 200 BTC.  After each successful send, the treshold jumps a different value (higher or lower).  I cannot send more than 200 BTC ever anymore since the problem started to appear, and it becomes worse and worse with time.

After googling this forum and other sites, my understanding is that the error occurs because the transaction becomes too complex.  The source of it seems to be "wallet pollution" where many small inputs exist and bitcoind wants to combine too many of them into the transaction.

My wallet file is 2.3 MB, and certainly there may be many small items in it.  However there are also plenty of large items that could be used to form the desired transaction (without reaching the excessive complexity).  Bitcoind seems to use a poor strategy in choosing the inputs from the available ones.

The problem appears accross many versions of bitcoin, including v0.3.x and v0.6.3, and on several machines.  It never appears on a fresh install, only starts to appear after the wallet has grown a lot, and then becomes worse with time.

My questions:

1. Do I have some way to influence which inputs are chosen?  For example, can I choose 10 "virgin" coins for a 500 BTC transaction, instead of thousands of 0.01xxx inputs?

2. Can I somehow view the details of my wallet?  I imagine that I could send the offending small inputs away to a different wallet, if I only knew their exact face value and the order in which they appear in the wallet.

3. Is there any other way to "clean" my wallet, or reorder the items in it so that the offending items will be selected last instead of first?

I am living with the problem, but it gets worse and worse every week.  I cannot ignore it much longer.  Right now I am thinking about this resolution strategy:  Fresh install on a new box.  Try to send 99.9 BTC to the new box.  If fails, decrease the amount by 0.1 BTC and retry (until success). Start over with 99.9.  Continue until the wallet is empty or near empty.  Delete the remaining wallet file.  Obviously this method is costly because of the TX fees involved.

[1715588625]

1715588625

[1715588625]

1715588625

[1715588625]

1715588625

[1715588625]

1715588625

[1715588625]

1715588625

[Strider Hiryu]

1. Do I have some way to influence which inputs are chosen?  For example, can I choose 10 "virgin" coins for a 500 BTC transaction, instead of thousands of 0.01xxx inputs?

2. Can I somehow view the details of my wallet?  I imagine that I could send the offending small inputs away to a different wallet, if I only knew their exact face value and the order in which they appear in the wallet.

3. Is there any other way to "clean" my wallet, or reorder the items in it so that the offending items will be selected last instead of first?

Some of the alternate clients provide 1 and 2.

3. unfortunately not exactly, the standard client has it's own way of selecting inputs.  I think there's a patch around that completely ignores small amounts, maybe there are more patches that do other things.

[BitBlitz]

I have a similar issue, and I don't think it is a transaction complexity problem.  I have an address with a single transaction that I cannot send coins from. 
In my case, I have a block reward from solo mining that caused my Bitcoin client to crash in May, when I first tried to spend it.  Shortly afterward, the client would crash or close every time it tried to open the wallet.  I used pywallet to extract the private keys for that address and other addresses with coins.  I was successful in importing the private keys and spending coins from all other addresses. 

Here's what is strange.  If I import the private key in a unencrypted wallet, the Bitcoin client crashes when it tries to open the wallet.  If I import the private key in an encrypted wallet, the Bitcoin client shows the 50.xxx coins in the wallet.  However, if I try to send them, I get "Transaction creation failed".  I have also tried several releases (from 0.3x to 0.7RC1) with no luck.

I've heard there is an issue with some addresses created on earlier clients and encrypted with newer client releases.  I didn't hear back from Gavin (he's excused if he's been busy )   I think I'll ping him again to clarify if there is a known issue.

[Stephen Gornick]

2. Can I somehow view the details of my wallet?  I imagine that I could send the offending small inputs away to a different wallet, if I only knew their exact face value and the order in which they appear in the wallet.

3. Is there any other way to "clean" my wallet, or reorder the items in it so that the offending items will be selected last instead of first?[/quote]

The bitcoin.org client doesn't provide a method to remove addresses.  Pywallet does allow wallet surgery like that (-dumpwallet).  Just make sure you make a backup prior to touching it.

You could list the transactions and for those with large enough unspent amounts you could then from a new wallet import them individually, giving you a wallet without the smaller transactions -- if those are what is really causing your problems.

I'm not sure if this does anything to help, but have you tried launching with a -rescan?

I also read that there might be a recovery mode in v0.7, I'm not sure if it made it in:
 - http://bitcointalk.org/index.php?topic=82070.msg904478#msg904478

That is still in testing though it is a RC.  I don't know that I'ld trust it with a wallet holding a larger amount of coins in it.

As long as you have the private keys, you can be assured that you can spend the funds eventually, so the problem is simply being able to use the existing wallet.dat from your client.

Right now I am thinking about this resolution strategy:  [...] Obviously this method is costly because of the TX fees involved.{/quote]

The transaction fees are based on the amount of data, and not necessarily on the number of transactions.  You might have transactions that cost way more after considering the fees than their value (e.g., the minimum commission on losing bets returned by SatoshiDICE when the wager was small).

Perhaps in v0.7 will be the fix, if you can wait.  If you can't then manual transfer of the keys (excluding the worthless ones) to a new wallet will solve this.

[Strider Hiryu]

I have a similar issue, and I don't think it is a transaction complexity problem.  I have an address with a single transaction that I cannot send coins from. 
In my case, I have a block reward from solo mining that caused my Bitcoin client to crash in May, when I first tried to spend it.  Shortly afterward, the client would crash or close every time it tried to open the wallet.  I used pywallet to extract the private keys for that address and other addresses with coins.  I was successful in importing the private keys and spending coins from all other addresses. 

Here's what is strange.  If I import the private key in a unencrypted wallet, the Bitcoin client crashes when it tries to open the wallet.  If I import the private key in an encrypted wallet, the Bitcoin client shows the 50.xxx coins in the wallet.  However, if I try to send them, I get "Transaction creation failed".  I have also tried several releases (from 0.3x to 0.7RC1) with no luck.

I've heard there is an issue with some addresses created on earlier clients and encrypted with newer client releases.  I didn't hear back from Gavin (he's excused if he's been busy )   I think I'll ping him again to clarify if there is a known issue.

Did I read somewhere that mtgox will let you import private keys?  You could try that, see if that works... or you crash them.

[BitBlitz]

Did I read somewhere that mtgox will let you import private keys?  You could try that, see if that works... or you crash them.

Heh.  I didn't crash MtGox, but I tried the privkey there, and blockchain.info.  Both showed the coins, but could not send.

Another weird data point.  When testing on an unencrypted 4.1 wallet, Jackjack's pywallet pulls a different key that reimports to a different address (with no coins).  Joric's pywallet extracts and imports the correct key without issue.  Perhaps that is just a bug or incompatibility in the older pywallet code.

[BitBlitz]

Jetmine, how old is the wallet?  Do you know the client version it was created with?  Did you encrypt it with a later client?

I think your best bet is to use pywallet to extract the keys for addresses with coins.  I'd import them into a new wallet one address at a time and send them, to make sure you don't have a bad key or address.  If you try it, keep us posted...