Bitcoin Forum
April 23, 2014, 01:07:55 PM *
News: Due to the OpenSSL heartbleed bug, changing your forum password is recommended.
 
   Home   Help Search Donate Login Register  
Pages: 1 [2]  All
  Print  
Author Topic: [RFC] When wallets conflict with the block chain  (Read 3418 times)
Pieter Wuille
Hero Member
*****
qt
Offline Offline

Activity: 938


View Profile WWW

Ignore
May 01, 2011, 06:39:36 PM
 #21

I finally have this implemented, see https://github.com/sipa/bitcoin/tree/rejectedtx

Initial testing shows that it can detect such conflicts, and store/restore correctly in wallet.dat.

The GUI will show rejected transactions, the RPC interface will hide them.

It still needs a way for permanently removing rejected transactions (default after X confirmations of the conflicting tx?), and a way to query them from RPC.

aka sipa, core dev team

Tips and donations: 1KwDYMJMS4xq3ZEWYfdBRwYG2fHwhZsipa
1398258475
Hero Member
*
Offline Offline

Posts: 1398258475

View Profile Personal Message (Offline)

Ignore
1398258475
Reply with quote  #2

1398258475
Report to moderator
Buy a Blade, Get a 5-Chip Free!
Start Mining with GAWMiners.com
24/7 Live Phone & Tech Support
Free Hosting & Electricity for 1 Year!

Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1398258475
Hero Member
*
Offline Offline

Posts: 1398258475

View Profile Personal Message (Offline)

Ignore
1398258475
Reply with quote  #2

1398258475
Report to moderator
Vladimir
Hero Member
*****
Offline Offline

Activity: 812


-


View Profile

Ignore
June 17, 2011, 11:29:00 PM
 #22

If anyone needs a test case for 'limbo transactions' simply send some money when your client is not up to date completely with the block chain. You will get a limbo transaction which at the moment basically the same as money lost and it is also unclear what happens with all the "change" involved which can be rather significant.

Seems like a serious bug to me.

A friend of mine has a a few such limbo transactions and no idea how to fix this. I also simply do not know how to help here.

Any suggestions?


-
ByteCoin
Sr. Member
****
expert
Offline Offline

Activity: 413


View Profile

Ignore
June 17, 2011, 11:42:53 PM
 #23

I think the transaction cancelling/superseding issue is one of the most technically complex outstanding issue needing to be addressed.

Bitcoin can cancel one input per day until one of the cancellations goes through.

Are you saying that this is something that already happens in the code or is this something you are proposing to be implemented to solve the problem in future?
If it is in the code, could you please indicate where?

Could you please also be a little bit more explicit as to how the input cancelling would solve the problem?
And why only cancel one per day if more are eligible for cancellation?

Thanks,

ByteCoin
bji
Member
**
Offline Offline

Activity: 112


View Profile

Ignore
June 18, 2011, 12:03:21 AM
 #24

I think the transaction cancelling/superseding issue is one of the most technically complex outstanding issue needing to be addressed.

Bitcoin can cancel one input per day until one of the cancellations goes through.

Are you saying that this is something that already happens in the code or is this something you are proposing to be implemented to solve the problem in future?
If it is in the code, could you please indicate where?

Could you please also be a little bit more explicit as to how the input cancelling would solve the problem?
And why only cancel one per day if more are eligible for cancellation?

Thanks,

ByteCoin

Transaction handling does seem like a weak point of the bitcoin system.  I think that the design should have included:

1) A proof of work in the transaction to limit the rate at which hosts can create transactions (and with a work function that is scaled to be of a constant difficulty similarly to how the block hash work function is scaled automatically).  Large, powerful hosts still would possibly be able to get around this but it would cost them (in electricity).

2) Transactions have a built-in expiration of 1 hour.  Blocks would be rejected by the client if they included transactions that should have expired before the block was created.  In this way, you can be sure that if you see a block which came after that hour expiration period, and your transaction still is not in the block chain, that it is gone and will never be processed.

BitterTea
Sr. Member
****
Offline Offline

Activity: 294



View Profile

Ignore
June 18, 2011, 12:26:54 AM
 #25

1) A proof of work in the transaction to limit the rate at which hosts can create transactions (and with a work function that is scaled to be of a constant difficulty similarly to how the block hash work function is scaled automatically).  Large, powerful hosts still would possibly be able to get around this but it would cost them (in electricity).

Perhaps there could be a transaction hash target, similar to that of the block. In order to be relayed, the transaction's hash must be below the target. I'm not sure how you'd get this difficulty to automatically scale with increases in processing power.
theymos
Administrator
Hero Member
*
expert
Offline Offline

Activity: 1540


View Profile
June 18, 2011, 02:29:21 AM
 #26

Are you saying that this is something that already happens in the code or is this something you are proposing to be implemented to solve the problem in future?

It's something I'm proposing.

Quote
Could you please also be a little bit more explicit as to how the input cancelling would solve the problem?

As a solution to the general problem of transactions that will never get into a block, sending a conflicting transaction (a transaction sharing one of the inputs to the "stuck" transaction) is guaranteed to cancel the old transaction if the new transaction gets into a block (and the normal SIGHASH mode is used). You can't just mark the old transaction as "canceled" without creating a conflicting transaction, since the old transaction might go through eventually, which would often be bad.

Sending just one of the stuck inputs at a time increases the chances that your new transaction will go through.

This isn't necessary if you know the transaction is already conflicting with something in the block chain, but often a transaction will not be accepted due to fee problems, and lightweight clients won't know about already-conflicting transactions.

Quote
And why only cancel one per day if more are eligible for cancellation?

There isn't a major reason why you couldn't do that, though if you sent all the transactions at once, then your peers would probably reject all but the first. Also, it seems dangerous to have many conflicting transactions floating around: if an attacker gets control of the network, they could do some extra things to transactions based on yours.

Mike Hearn
Hero Member
*****
expert
Offline Offline

Activity: 1232


View Profile

Ignore
June 19, 2011, 03:08:51 PM
 #27

Incidentally, BitCoinJ now implements something like this. Transactions that conflict with the block chain are considered "dead" and an API user can find out about them by handling an event.

There are still a few edge cases that aren't handled quite correctly, like the case of a previously dead transaction becoming live again due to a block chain re-org (it's handled but you are not notified via the event listeners). It's a very complex set of edge cases on top of edge cases.

12LMm82ZgAzf7yNDpPydEYxEr4Ap7XtSSK
TiagoTiago
Hero Member
*****
Offline Offline

Activity: 602


Firstbits.com/1fg4i                :Ƀ


View Profile

Ignore
July 05, 2011, 07:43:28 PM
 #28

What happens if you start the client with the --rescan flag or delete the blockchain and let the client redownload it?

(I dont always get new reply notifications, pls send a pm when you think it has happened)

Wanna gimme some BTC for any or no reason? 1FmvtS66LFh6ycrXDwKRQTexGJw4UWiqDX Smiley

The more you believe in Bitcoin, and the more you show you do to other people, the faster the real value will soar!

Do you like mmmBananas?!
Pieter Wuille
Hero Member
*****
qt
Offline Offline

Activity: 938


View Profile WWW

Ignore
July 05, 2011, 07:48:14 PM
 #29

What happens if you start the client with the --rescan flag or delete the blockchain and let the client redownload it?

-rescan only adds transactions to your wallet that were missing (and since 0.3.23, also updates them if they were incorrectly marked unconfirmed).

Deleting the block chain is something that should never be needed since 0.3.23.

aka sipa, core dev team

Tips and donations: 1KwDYMJMS4xq3ZEWYfdBRwYG2fHwhZsipa
netrin
Sr. Member
****
Offline Offline

Activity: 322


FirstBits: 168Bc


View Profile

Ignore
July 05, 2011, 07:51:23 PM
 #30

Sipa just pointed me to this thread after I posted http://forum.bitcoin.org/index.php?topic=26269.0

The transaction in question is now over a month old. I have two versions of the same send in my client (neither send happened). The first send has 7000 confirmations sent to no (blank) address. Another send (with the same timestamp and btc amount) has 0/confirmations to an unknown address in the block chain (presumably an address of mine somewhere).

I believe the one or two transactions are invalid due to double spending from a copied wallet but it's difficult to confirm because I can not be sure which are the sending nor receiving addresses. In any case, I would hope that before 7000 blocks the 0.3.23 client would be convinced that the transaction is never going to happen.

Greenlandic tupilak. Hand carved, traditional cursed bone figures. Sorry, polar bear, walrus and human remains not available for export.
Gavin Andresen
Hero Member
*****
qt
Offline Offline

Activity: 1330


Chief Scientist


View Profile WWW

Ignore
July 05, 2011, 08:00:35 PM
 #31

Losing double-spent transactions should probably be reported as -1/unconfirmed, -2/unconfirmed, etc -- meaning "there is a N-confirmed transaction that conflicts with this one."

-6/unconfirmed seems like the natural "this transaction ain't never gonna be confirmed, time to report it as -N/orphan."

-1/unconfirmed should be enough to stop re-transmitting it.

And -120/orphan seems like a safe time to "unspend" any inputs (let them be spent by new transactions) that weren't spent by the other, conflicting transaction.

... he says nonchalantly, knowing full well that actually making bitcoin behave that way means a lot of very careful coding and lots and lots of careful testing...

The shortcut, maybe-good-enough-for-now solution:  export all the private keys from all the messed up wallets.  Start with a clean wallet, then re-import all the private keys and let the clean-slate bitcoin figure it all out.


Will I see you in Amsterdam?
  http://bitcoin2014.com/
TiagoTiago
Hero Member
*****
Offline Offline

Activity: 602


Firstbits.com/1fg4i                :Ƀ


View Profile

Ignore
July 05, 2011, 08:05:20 PM
 #32

Would the transaction be removed from the transactions history or would it be marked as "REJECTED" or somthing like that?

(I dont always get new reply notifications, pls send a pm when you think it has happened)

Wanna gimme some BTC for any or no reason? 1FmvtS66LFh6ycrXDwKRQTexGJw4UWiqDX Smiley

The more you believe in Bitcoin, and the more you show you do to other people, the faster the real value will soar!

Do you like mmmBananas?!
Pieter Wuille
Hero Member
*****
qt
Offline Offline

Activity: 938


View Profile WWW

Ignore
July 06, 2011, 08:47:25 AM
 #33

Losing double-spent transactions should probably be reported as -1/unconfirmed, -2/unconfirmed, etc -- meaning "there is a N-confirmed transaction that conflicts with this one."

I like this idea. It would need to be extended to "there is a N-confirmed transaction that conflicts with this one, or one of its dependencies", though.

Quote
-6/unconfirmed seems like the natural "this transaction ain't never gonna be confirmed, time to report it as -N/orphan."

-1/unconfirmed should be enough to stop re-transmitting it.

And -120/orphan seems like a safe time to "unspend" any inputs (let them be spent by new transactions) that weren't spent by the other, conflicting transaction.

Not sure about this. As soon as you are at -1, the chance for the transaction to ever get into the block chain is already very small. I think you should stop counting its inputs at that point already.

When -6 is reached, you could call it "orphan" or "rejected". The latter may be a bit clearer to users.

When -120 is reached, you can indeed assume that it won't confirm anymore ever. Maybe a "Remove transaction from wallet" option can be enabled at that point?

Quote
... he says nonchalantly, knowing full well that actually making bitcoin behave that way means a lot of very careful coding and lots and lots of careful testing...
Well, i've already implemented a rejection system (see http://forum.bitcoin.org/index.php?topic=7330.0 and https://github.com/bitcoin/bitcoin/pull/195). It needs rewriting for the current codebase though, but the idea above shouldn't be too hard to incorporate.

Quote
The shortcut, maybe-good-enough-for-now solution:  export all the private keys from all the messed up wallets.  Start with a clean wallet, then re-import all the private keys and let the clean-slate bitcoin figure it all out.
That works, indeed.

EDIT: I definitely think you should not count the spending caused by rejected transactions. What if you have two transactions in your wallet that spend the same input (one confirmed, one rejected with outputs to yourself). This is a possible situation. If you still count the rejected one, your balance will be wrong.

aka sipa, core dev team

Tips and donations: 1KwDYMJMS4xq3ZEWYfdBRwYG2fHwhZsipa
gmaxwell
Moderator
Hero Member
*
qt
Offline Offline

Activity: 1078


View Profile

Ignore
July 06, 2011, 11:25:36 AM
 #34

Transaction handling does seem like a weak point of the bitcoin system.  I think that the design should have included:

1) A proof of work in the transaction to limit the rate at which hosts can create transactions (and with a work function that is scaled to be of a constant difficulty similarly to how the block hash work function is scaled automatically).  Large, powerful hosts still would possibly be able to get around this but it would cost them (in electricity).

It wouldn't have to be in the blockchain. You could hashcash complete txn to get them forwarded/mined.   But why? We have a _reusable_ proof of work system built into bitcoin:  transaction fees.

Quote
2) Transactions have a built-in expiration of 1 hour.  Blocks would be rejected by the client if they included transactions that should have expired before the block was created.  In this way, you can be sure that if you see a block which came after that hour expiration period, and your transaction still is not in the block chain, that it is gone and will never be processed.

Creates a huge to split the chain in order to kill transactions forever even after they are confirmed, even absent obviously fraudulent respend activities and just by accident:   Consider the  the possible block a txn could have gone in: it goes in,  but the chain splits and it falls onto an orphan block, now it can't ever go in, even if it wasn't respent.   It basically allows random miners to denyable create reverse&respend like theft events without even having to control the private keys for respending.

If there were something like this the horizon would need to be much longer than an hour I think.




netrin
Sr. Member
****
Offline Offline

Activity: 322


FirstBits: 168Bc


View Profile

Ignore
July 06, 2011, 05:12:10 PM
 #35

The shortcut, maybe-good-enough-for-now solution:  export all the private keys from all the messed up wallets.  Start with a clean wallet, then re-import all the private keys and let the clean-slate bitcoin figure it all out.

I would have assumed that's what --rescan would do. Throw away all block references and scan every address for it's balance record in the block chain.

Greenlandic tupilak. Hand carved, traditional cursed bone figures. Sorry, polar bear, walrus and human remains not available for export.
Eri
Sr. Member
****
Offline Offline

Activity: 252


View Profile

Ignore
July 09, 2011, 09:42:45 PM
 #36

their should be a fix for this in the client itself, relying on a response from a node is not good enough. this is my take on a few different ideas I've seen in this thread and vary glad to see someone working on it!

when a transaction is initially made it should be listed as "0/Pending", each new block the client receives that doesn't have the transaction in it should increase the number. when it hits some number, for example 10 the client can allow access to the funds again but give some sort of warning that shows the transaction may still go through(though it may be worth it to simply transfer all funds to a new private key to invalidate the other transaction?).  you can combine this with the idea of the node sending a msg back if the transaction is rejected, which would allow it to immediately set the transaction to 'failed' without having to wait which would allow respending of the funds immediately.

as someone else mentioned, rescan should delete all data except private keys and public keys, rebuilding the wallet from scratch allowing us to see it as the network sees it. a plain ole export for private keys would be awesome too :3
 
i also liked the idea of having a timeout as someone mentioned, if not *in* a block that is in the process of being solved within an hour of the transactions creation all nodes would simply ignore it. making it safe to say that if its been 2 or 3 hours and new blocks have been created and received, the transaction will /never/ go through as you would have already received it or all nodes would have safely ignored it.

i think a combination of all of these for such a serious issue, even if it is rare, would not be overkill.
TiagoTiago
Hero Member
*****
Offline Offline

Activity: 602


Firstbits.com/1fg4i                :Ƀ


View Profile

Ignore
July 09, 2011, 11:05:24 PM
 #37

Wouldn't that create the risk that if someone send a payment before downloading the whole blockchain they'll have to send it again once it's fully downloaded?

(I dont always get new reply notifications, pls send a pm when you think it has happened)

Wanna gimme some BTC for any or no reason? 1FmvtS66LFh6ycrXDwKRQTexGJw4UWiqDX Smiley

The more you believe in Bitcoin, and the more you show you do to other people, the faster the real value will soar!

Do you like mmmBananas?!
Eri
Sr. Member
****
Offline Offline

Activity: 252


View Profile

Ignore
July 10, 2011, 05:06:17 AM
 #38

Quote from: Me~
the client can allow access to the funds again but give some sort of warning that shows the transaction may still go through(though it may be worth it to simply transfer all funds to a new private key to invalidate the other transaction?).

Wouldn't that create the risk that if someone send a payment before downloading the whole blockchain they'll have to send it again once it's fully downloaded?


yes and no, that would only be an issue if you wanted to invalidate the transaction automatically(and not just have the funds back). that should likely also be left up to the user and in a case like this they would know if if they are still downloading the block chain or not. as far as being automatic goes i think you can tell when you have the latest block, something to do with requesting the next block in the blockchain. which may not give you the most recent one in cases with a slow connection.. but maybe requesting it from multiple sources? do clients keep track of when the latest block was created? if not they could include a timestamp and send that with the latest block or something. it doesn't seem like a big issue.

as far as a UI goes id have 2 categories for funds, "balance"  and "pending transfers". 'pending transfers' shows the total funds in transactions we have sent out that have not been seen in a block in the block chain yet. this would make people aware of funds that haven't been accepted into the block chain yet. at the same time we can add a new tab next to 'sent' and 'received' called 'pending transfers' and include all outgoing transactions that have yet to be seen in the block chain and include a header called 'status' with a description like 'no confirmations' and have a count for how many blocks it has yet to be seen in and possibly have a check box/button for each that would allow for 'canceling' the transaction manually.
Pages: 1 [2]  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!