please advice about bitcoin core wallet issue ... "input already spent"

[aarreeaa]

The problem is that my wallet history does not include dozen or so completed outgoing transactions and I now often cant send bitcoin - it gets rejected by the network because some inputs are already spent. Thats because at some point I must have overwritten newer wallet file with older copy and continued to use that.
Wallet now is almost impossible to use - I must cancel transaction and try again and again until core wallet doesnt pick "spent" input...

Today I deleted all "blocks" and "chainstate" folders and started to sync with network from 0, but that will take weeks to finish.

Will it find and add missing transactions to my wallet?

I hope someone can help.

[hosseinimr93]

I must cancel transaction and try again and again until core wallet doesnt pick "spent" input...

Why don't you use coin control feature?
You can click on "Inputs" when making a transaction and select the UTXOs you want to spend.

If you don't see "Inputs" button, click on "Setting" at top of the window. After that, select "Wallet" and then check "Enable coin control features".

............but that will take weeks to finish.

The only solution is to use SPV wallets like electrum, unless you insist on running your own node.
In SPV wallets, you don't have to download the blockchain.

[aarreeaa]

Why don't you use coin control feature?
You can click on "Inputs" when making a transaction and select the UTXOs you want to spend.

Thanks.
It wasnt enabled, so I didnt see "Inputs" button and had no idea that I can activate it.

[kano]

...
In SPV wallets, you don't have to download the blockchain.

But you must also trust that all the blockchain information that the company that provides you with the blockchain info is all correct.

i.e. you are no longer in full control of your wallet.

[pooya87]

...
In SPV wallets, you don't have to download the blockchain.

But you must also trust that all the blockchain information that the company that provides you with the blockchain info is all correct.

i.e. you are no longer in full control of your wallet.

You are exaggerating. With an SPV wallet you are still in full control of your keys (assuming it is not some  shady software), also keep in mind that SPV is "simplified" verification not "no" verification at all.

In a correct implementation you don't rely on anything centralized like a "company" for the blockchain, you can download the block headers from any full node with a protocol version higher than 31800. These headers will be fully verified locally.
Then you'll need an indexed full node that can give you your transaction history, nodes such as the Electrum nodes (which again is not a "company"). Each transaction can be verified and a merkle proof could be requested from full nodes. At this point you put your trust in PoW and the fact that multiple nodes don't lie.

It is definitely not as secure as a full node, but it is also far from you not having any control.

[kano]

...
It is definitely not as secure as a full node, but it is also far from you not having any control.

...
i.e. you are no longer in full control of your wallet.

Thanks for agreeing with me and yet somehow trying to tell me I'm wrong

[pooya87]

Thanks for agreeing with me and yet somehow trying to tell me I'm wrong

You are mixing two entirely different arguments.
One is about the verification part and relying on a [centralized] company which is the bulk of my explanation above and the lower security comment at the end.
And the other is the the wallet part that you claim you have no control over it ("you are no longer in full control of your wallet.")which makes no sense because SPV clients aren't custodial accounts for users to not have 100% control over the wallet! Doing less verification and having a lower security has nothing to do with control over the wallet.

[nc50lc]

Today I deleted all "blocks" and "chainstate" folders and started to sync with network from 0, but that will take weeks to finish.
Will it find and add missing transactions to my wallet?

You could've just rescanned.
The latest scan might have left some invalid transactions in your wallet.

rescanblockchain (info) with default arguments will scan from height 0~tip.
That will take quite some time so be patient while it scans. After that, the correct balance should appear and the invalid transaction(s) will be identified.
It will take hours but not days and wont cost your internet bandwidth unlike redownloading the blockchain.
But it's too late since you've already deleted your data.

[kano]

..
And the other is the the wallet part that you claim you have no control over it
...

Please quote where I said this.

I cannot see it anywhere in my post, so I'm not sure why you made that up.

[LoyceV]

I must have overwritten newer wallet file with older copy and continued to use that.

After doing this, Bitcoin Core should have checked your wallet.dat from it's last known block, but clearly something went wrong. There might be a bigger concern though: if you restored an old backup, it may not have been a HD wallet, and you may have lost private keys that were created from the newer version of your wallet. If that's the case, your only chance is to recover the newer wallet from your system, and if you need to do that, you should stop writing to the same hard disk NOW, turn it off, and continue recovery from a different system.

Today I deleted all "blocks" and "chainstate" folders and started to sync with network from 0, but that will take weeks to finish.

Depending on your current bottleneck, there are ways to speed it up: more RAM, more dbcache, SSD instead of HDD, more CPU power or faster internet.

Will it find and add missing transactions to my wallet?

It should. But it doesn't hurt to make another backup of your wallet.dat first.

[NotATether]

Do what nc50lc suggested and perform a wallet rescan. A full reindex where blocks are downloaded again is not necessary (and probably won't solve your problem since the wallet still won't be updated).

You can easily perform a rescan at launch by passing the -rescan argument to bitcoin-qt when it starts. This will update your wallet to include all the newer transactions.

[pooya87]

Well that's not gonna work.

I'm not sure if you are serious or just messing with me!

An SPV wallet can accept a transaction that a full node will not accept.

There is some ambiguity in this comment. What do you mean by "accept"?
SPV clients see transactions as either unconfirmed or confirmed (with the block containing them).
All a malicious node can do is tell the SPV client that a confirmed tx is unconfirmed and the SPV client has no way of knowing this is a lie unless they connect to another honest node. Also unconfirmed transactions aren't safe to begin with whether they are valid or invalid.

Otherwise if the malicious node wants to tell the SPV client that an invalid transaction (that a full node won't accept) is confirmed the node has to also be able to provide a valid block that is also part of the chain, which they obviously can't do (it needs a massive hashrate to be able to find a block with the current difficulty target). As I said above SPV clients reply on PoW and longest chain.

[HCP]

... useful info...

All useful information... if the OP had not deleted all their block data!

Today I deleted all "blocks" and "chainstate" folders and started to sync with network from 0, but that will take weeks to finish.

Will it find and add missing transactions to my wallet?

Once this finishes... your wallet transaction history and balance will correctly reflect the details that relate to the private keys that it currently contains.

However, given that you believe you may have used an old backup of your wallet.dat, it's impossible to say for sure at this point whether or not the current wallet.dat contains all the the private keys that your old one did.


Is your current wallet.dat an "HD" wallet? Do you see this "HD" symbol in the bottom right corner:



Or do you see the HD symbol with an "X" through it?

[kano]

Well that's not gonna work.

I'm not sure if you are serious or just messing with me!

Well you changed your post after I replied to it ... that sorta makes anyone wonder what it said before vs now ...

An SPV wallet can accept a transaction that a full node will not accept.

There is some ambiguity in this comment. What do you mean by "accept"?
SPV clients see transactions as either unconfirmed or confirmed (with the block containing them).
All a malicious node can do is tell the SPV client that a confirmed tx is unconfirmed and the SPV client has no way of knowing this is a lie unless they connect to another honest node. Also unconfirmed transactions aren't safe to begin with whether they are valid or invalid.

Otherwise if the malicious node wants to tell the SPV client that an invalid transaction (that a full node won't accept) is confirmed the node has to also be able to provide a valid block that is also part of the chain, which they obviously can't do (it needs a massive hashrate to be able to find a block with the current difficulty target). As I said above SPV clients reply on PoW and longest chain.

All A malicious node can ...

Firstly, even you in your post above have made this statement incorrect, so 'All' should be removed.

It would depend on the various configuration of the SPV wallet (and those providing it) what other malicious things can be done.

e.g.
Using statistical expectation, a block of your choosing with any content you like, costs round 6.25 BTC
So not all that hard to get ...

[pooya87]

It would depend on the various configuration of the SPV wallet (and those providing it) what other malicious things can be done.

That's the difference between what the protocol is and what the implementation does, and we are talking about the protocol here not a specific implementation of it.

Using statistical expectation, a block of your choosing with any content you like, costs round 6.25 BTC
So not all that hard to get ...

That is worth $300k, so it is hard but that's not important. What matter is that the said block will be on a short chain that nobody will build upon:

A > B > C > D > E > F
          > D' 

where D' is the fake block. The SPV client is capable of noticing this and following the longest chain. If the attacker continues building on D' it still is going to be a lot shorter than the real chain and won't be followed by the SPV client.

[kano]

It would depend on the various configuration of the SPV wallet (and those providing it) what other malicious things can be done.

That's the difference between what the protocol is and what the implementation does, and we are talking about the protocol here not a specific implementation of it.

Well that's pointless, since no one will have 'the protocol', they will only have an 'implementation'.
The comment I replied to suggested that a speed up was to use an SPV wallet, and I replied with information regarding the risk if someone was to get such an 'implementation', not a discussion of the 'protocol'.

Using statistical expectation, a block of your choosing with any content you like, costs round 6.25 BTC
So not all that hard to get ...

That is worth $300k, so it is hard but that's not important. What matter is that the said block will be on a short chain that nobody will build upon:

A > B > C > D > E > F
          > D' 

where D' is the fake block. The SPV client is capable of noticing this and following the longest chain. If the attacker continues building on D' it still is going to be a lot shorter than the real chain and won't be followed by the SPV client.

The '$300k' number is effectively meaningless.
It's the 6.25BTC number that matters, and if some scoundrel company with more than 6.25 BTC (which there are plenty of companies with a lot more than that) wants to take advantage of an implemented SPV wallet with a lot more than 6.25BTC in it, where the SPV wallet only talks to them ...

I wonder why a mod deleted one of my posts and who asked for that ...