Transactions ordering and confirmation problem

[rossmclochness]

I have a problem understanding how Bitcoin can ever be useful as a payment mechanism for a merchant in a retail store.

A block added to the blockchain consists of up to around 1,500 transactions which are verified (small block in kb). Apparently miners can choose larger fees over small. So you can do this: https://twitter.com/alexbosworth/status/1028031387201236992?lang=en

Essentially transaction A (at let's say at a Target retail store) has zero fees. Create a second transaction B a 30 minutes later (at Walmart) using the same unspent transaction but add a fee. Essentially B can be confirmed before a by miner 1 and be placed into a block before A is confirmed. If I am at either store, you have to wait a full hour to see which block will be the first one to be included on the blockchain. Yet the truth is that the first actual "spend" was A at the Target store. The second B at Walmart was, theoretically, the double spend. But what all these people seem to be focusing on is that Target wouldn't have lost money if they waited at least an hour (6 transactions) to determine whether they receive confirmations.

Problem 1 - this isn't true. If the transaction is stuck, you won't know a double spend occurred until finally it gets picked up by some miner and rejected and then that message relayed back to Target and the sender.

Problem 2 - the time it took to determine this is simply unacceptable, including for Walmart. You need to wait a long time AFTER your transaction is confirmed in a validly mined block. Even just 3 confirmation seems way too long - 30 minutes.

[nc50lc]

In use-case like that, either Lightning Network (no third-party) or a payment processor (w/ third-party) will be useful.
But currently, LN is still on its "testing stage" and still cannot safely utilize by huge retailers like Wallmart.

Read these articles about Lightning Network:

• What is Lightning Network (Infographic)
• What is Lightning Network (in text)

But hey, talking about "retail store" as in the person is physically there, CCTVs all around and guards on duty;
There's no reason for the person to double spend a transaction unless he wants to be locked in jail, so retailers can accept unconfirmed transactions in situations like these (with a minimum fee requirement to be confirmed within 10minutes).

Or just wait for LN-designed APIs and LN to be fully functional (with less to no bugs).

[pooya87]

if you go to Walmart and try buying something with your personal check and then go to another shop and do the same,... you are doing the same double spend with fiat (as you would do with bitcoin with 0 fee!) that doesn't mean fiat is not accepted anywhere as payment, it just means the merchants don't accept something that has a high risk of not "confirming" and they prefer payments with "less risk" like credit cards although they are still risky as it can be a stolen one.

it doesn't have to be different with bitcoin either.
merchants don't accept something that has a "high risk" which is transactions with 0 fee. instead they prefer lower risks which means doing a simple risk assessment on the transaction. this includes checking its fee and comparing it with the mempool, checking the parents and their confirmation status, the size of the payment,... and then deciding whether they want to accept that or not.

[rossmclochness]

Thanks much guys for the detailed responses, much appreciated! Please take my comments in an educational "devil's advocate" context since I am both still learning this complicated subject and my peers will be forceful with their arguments against rationales that I offer!

@nc50lc - Thanks! When you say third party payment processor, how would that work exactly, e.g. they will take a fee for payment and offer quicker processing time by taking the risk of a bad transaction by scaning mempools to identify double spends, etc.? If so, this seems very inefficient and beginning to add costs that might eventually approach costs from a traditional centralized model eventually? That's the objection I face here.

As to the LN, that appears to be a state/payment channel. This appears to make it less than ideal for individual transactions and primarily beneficial/practical for recurring transactions between merchant and customer on a channel level - but that takes place off-chain with money that is essentially required to be pre-cleared.


@pooya87 - Agreed, not a great idea with CCTV! And thanks for explanation of the logic about the size of the fee as an indicia of reliability. The situation I present isn't to be taken so literally, e.g. a $20 purchase a retail store because I can easily make the stakes higher - $2-5,000 at a furniture store, used car lot, etc. which people will try!

I'm looking at theoretically what can happen and not "probably won't but could" as the latter doesn't provide great confidence to many concerning financial transaction discussions. By comparison, in centralized systems, one can check available balance with ostensibly much greater certainty as well as providing a set expectation for transaction fees. Unless someone is really knowledgeable, there isn't any easy way to know what is an "acceptable" payment fee to bribe miners because it's dependent upon so many factors. It seems one really needs to rely upon third party processors.

[r1s2g3]

The situation I present isn't to be taken so literally, e.g. a $20 purchase a retail store because I can easily make the stakes higher - $2-5,000 at a furniture store, used car lot, etc. which people will try!

More you increase the stakes ,More the confirmation retailer will like to have before handing you the goods. 1 confirmation is good for $2 or $3 things. But for high priced item then retailer will like to wait for  at least 6 to 7 confirmations before handing you the goods.

[pooya87]

@pooya87 - Agreed, not a great idea with CCTV! And thanks for explanation of the logic about the size of the fee as an indicia of reliability. The situation I present isn't to be taken so literally, e.g. a $20 purchase a retail store because I can easily make the stakes higher - $2-5,000 at a furniture store, used car lot, etc. which people will try!

well different payment size needs different approaches. for example a hot dog stance doesn't need to worry about double spends but a real estate agent has to because of the size of the deal. but also the bigger size deal has additional legal documents (like a contract for example) which can also prevent fraud and have precautions in case of a fraud like double spend.

I'm looking at theoretically what can happen and not "probably won't but could" as the latter doesn't provide great confidence to many concerning financial transaction discussions.

well double spends can happen, that is why it has always been said that "unconfirmed" transactions are not safe. and they have happened in the past, for example there are gambling sites that accept 0-conf. deposits and allow you to play. people have abused this but they still continue to accept them because it comes down to risk factor they want to take and the way their business is designed.

Unless someone is really knowledgeable, there isn't any easy way to know what is an "acceptable" payment fee to bribe miners because it's dependent upon so many factors. It seems one really needs to rely upon third party processors.

you don't have to be knowledgeable because things like this should not happen manually. all the checks i mentioned above should happen with a program that is designed by a knowledgeable programmer and the merchant is only seeing its result on his screen.
it doesn't have to be third party processors either. it is just that so far we haven't had that much adoption so there hasn't been any applications made for merchants to accept payments on their own in that easy way they wish to. and also the price volatility creates that incentive to use them because they offer immediate conversion to fiat so the merchant doesn't lose money if price dropped.

[o_e_l_e_o]

If I make a payment with a credit card, that payment reaches my credit card company within seconds, much like a bitcoin transaction is broadcast to the network within seconds. However, it will take several days before my credit card company actually transfers any money to the payee's account, whereas a bitcoin transaction will be confirmed within 10 minutes (on average), and have 6 confirmations within an hour (again, on average). In the days before my credit card transaction is paid, and indeed, for a many months after it is paid, I can reverse that transaction by claiming my card was stolen, or lost, or cloned, or I was phished, or hacked, or defrauded, etc. And yet, fraud rates are low enough that pretty much everywhere will accept credit card payments.

The situation is not that dissimilar to accepting zero confirmation transactions, where the security of these zero confirmation transactions (and credit card payments) relies on external factors such as customer honesty and a large company being able to accept small amounts of fraud in exchange for convenience. Obviously if you are dealing with large amounts of money, or you are unwilling to accept that risk, then you stipulate that you require 1/3/6/however many you want confirmations on each transaction.

There is a good summary here: https://bitcoincore.org/en/faq/optin_rbf/#why-arent-unconfirmed-transactions-safe

[rossmclochness]

If I make a payment with a credit card, that payment reaches my credit card company within seconds, much like a bitcoin transaction is broadcast to the network within seconds. However, it will take several days before my credit card company actually transfers any money to the payee's account, whereas a bitcoin transaction will be confirmed within 10 minutes (on average), and have 6 confirmations within an hour (again, on average). In the days before my credit card transaction is paid, and indeed, for a many months after it is paid, I can reverse that transaction by claiming my card was stolen, or lost, or cloned, or I was phished, or hacked, or defrauded, etc. And yet, fraud rates are low enough that pretty much everywhere will accept credit card payments.

First - many thanks for the reply and reference. I should probably have used a debit card since that works differently but in essence it's the same - that someone has guaranteed funds are there to be transferred. In a centralized payment system, at least the card issuer can check with the issuing bank that either (a) there are sufficient funds to transfer and a hold is immediately placed on those funds, or (b) that the card holder has sufficient amount under their credit limit and there is a pending transaction placed immediately.

In this example there isn't any guarantee that payment will even be processed. Hence I agree, the 1 transaction amount is very important! And for that I understand it to mean that a consensus of nodes have agreed that Block B is valid and to be added to the blockchain's current last block, let's call it Block A. And the longer the number of blocks confirmed, e.g. A-B-C-D... a decreasing risk exists that a Block B will be thrown out (and its transactions not also confirmed in another longer chain are put back into the mempool of unconfirmed transactions.) Do I have this correct?

[rossmclochness]

More you increase the stakes ,More the confirmation retailer will like to have before handing you the goods. 1 confirmation is good for $2 or $3 things. But for high priced item then retailer will like to wait for  at least 6 to 7 confirmations before handing you the goods.

Understood - But the payee of the first transaction A (Target) will need to recheck periodically to see if the inputs in its transaction have been spent elsewhere. This doesn't happen automatically until a miner actually tries to verify the transaction, sees that the inputs have been spent already in transaction B, and broadcast to the network that transaction A is invalid. Target's software will need to periodically check both (a) the mempool for any other attempted spend of an input, and (b) the current state of the blockchain to see a confirmed spend of the same input.

-----------------------------------------------

well double spends can happen, that is why it has always been said that "unconfirmed" transactions are not safe. and they have happened in the past, for example there are gambling sites that accept 0-conf. deposits and allow you to play. people have abused this but they still continue to accept them because it comes down to risk factor they want to take and the way their business is designed.

I'm just going to stick with on-chain issues (not like legal stuff) So as per above, 1 confirm is like a magic number. Let's start with what moves the payment system forward - such as fee amount set by the user.

you don't have to be knowledgeable because things like this should not happen manually. all the checks i mentioned above should happen with a program that is designed by a knowledgeable programmer and the merchant is only seeing its result on his screen.

Well let's talk about the consumer for a moment. He does need to know what a transaction SHOULD cost. How much? I'm guessing that wallet software calculates fee averages based upon recent transactions to suggest what to use.

Back to transaction confirmations - I understand we are talking about the number of confirmations that a consensus of nodes having been polled agreed that this block (let's call it "B") is the next block on the blockchain (a long list of blocks ending with "A"). That should take at least 8-10 minutes per confirm (although I see videos of it taking much less than that such as on blockchain.info - how?)

The other concern I have is with is the potential for network latency, such as dealing in a country which doesn't have the greatest of network infrastructure (and perhaps a large number of bad acting nodes). If a consensus was reached that block B is the next block (...A-B) it's possible that nodes in a certain region may not see that another miner has presented A-C-D and my transaction, which was contained in B but not in C and D, can be tossed back to the mempool to start the cycle again and this situation can repeat itself. Thanks again very much.

[pooya87]

~1 confirm is like a magic number.

it may look like a magic number but there is a good reason for it. it is decided based on how Proof of Work works, when a new block is found and added to the blockchain, in order to "reverse" the transactions inside that block (the transactions that currently have 1 confirmation) you need to have more than 51% of the hashrate. with current total hashrate it will cost you millions of dollars worth of equipment and electricity to create a longer chain orphaning that block and building your own longer blockchain. which is why we consider a transaction with 1 confirmation to be very safe in bitcoin.
in comparison, in other altcoins the same number is a much higher one depending on how their PoW state is (difficulty, hashrate, orphan rate,...).

Well let's talk about the consumer for a moment. He does need to know what a transaction SHOULD cost. How much? I'm guessing that wallet software calculates fee averages based upon recent transactions to suggest what to use.

correct.

Back to transaction confirmations - I understand we are talking about the number of confirmations that a consensus of nodes having been polled agreed that this block (let's call it "B") is the next block on the blockchain (a long list of blocks ending with "A").

nodes don't come to an agreement (if that's what you meant) when it comes to blocks. a block is either valid or not and the blockchain that a node accepts is the longest one.

That should take at least 8-10 minutes per confirm (although I see videos of it taking much less than that such as on blockchain.info - how?)

The other concern I have is with is the potential for network latency, such as dealing in a country which doesn't have the greatest of network infrastructure (and perhaps a large number of bad acting nodes). If a consensus was reached that block B is the next block (...A-B) it's possible that nodes in a certain region may not see that another miner has presented A-C-D and my transaction, which was contained in B but not in C and D, can be tossed back to the mempool to start the cycle again and this situation can repeat itself. Thanks again very much.

let me try to explain how mining works, maybe that answers these questions:
mining is like a competition. let say person 1, 2 and 3 are all mining bitcoin. what they do is that they perform a hash (SHA256) repeated times until they come up with a correct hash that satisfies some conditions (don't get into that). that takes time. the first person to find that solution "wins", then they have to send it out to the rest of the network, which they do. so lets say person 2 found the solution and sent it out. now in matter of seconds (you are sending about 1.2 MB of data) most nodes find out about it and so do person 1 and 3. so they immediately give up the work and start working on the next block to try and "win" the next race. but if they continue working on the same solution they can come up with the answer (a different one) and create a different block which they can send out to the network. (to address your last two statements: person 1 and 3 were working on the same problem so the block they found contains nearly the same transaction, so your tx didn't get tossed back to mempool, there is a good chance they also have your tx in their block) but the problem is that now that they are sending the solution out to the network, the nodes already have a block and this new block is going to be rejected by them.
we don't need to reach any consensus to decide whether this new block should be accepted or not, this block has been too late, and if anybody continues building on it they will end up with a "shorter" blockchain because they started it late and the rest of the network that received the block found by person 2 are already ahead and have the longest chain which is the one everyone should follow.

now the thing about less than 10 minute is that as i said process of mining is to hash some data and find a solution. so this is somewhat based on chance. so sometimes you find that solution so much faster which is why there is sometimes blocks that are found sooner than 10 minutes. this is not common because the hash size is 256 bit (SHA256) and the chances of it are small.

[o_e_l_e_o]

it may look like a magic number but there is a good reason for it. it is decided based on how Proof of Work works, when a new block is found and added to the blockchain, in order to "reverse" the transactions inside that block (the transactions that currently have 1 confirmation) you need to have more than 51% of the hashrate.

That's not quite accurate. 1 confirmation can be reversed with much less than 51% of the hashrate. Section 11 of the bitcoin whitepaper explains why, and this site (https://people.xiph.org/~greg/attack_success.html) lets you play around with the numbers yourself. Indeed, with only 24% of the hashrate, an attacker has a 50% chance of being able to catch up to the main chain from a starting point of 1 block behind. Indeed, stale blocks (which are commonly but incorrectly referred to as orphaned blocks), happen not infrequently without anyone having close to 51% of the hashrate: https://www.blockchain.com/btc/orphaned-blocks

problem is that now that they are sending the solution out to the network, the nodes already have a block and this new block is going to be rejected by them.

There are times when a newer block has supplanted an older one (such as at 489644 and 488470), and the block which was found first has ended up becoming "orphaned".

[PrimeNumber7]

Essentially transaction A (at let's say at a Target retail store) has zero fees.

Target could spend the unconfirmed input they received from you, and attach a fee large enough to pay for transaction A and the transaction they are sending to themselves. In order for a miner to receive the second transaction fee, it must confirm both transactions.

This is called Child Pays for Parent, or CPFP. There is an interesting Blog Post by Coinbase on the subject.

Target could also have a special arrangement with one or more mining pools that involve Target paying the mining pool a special fee, offchain in exchange for the mining pool confirming transactions sent to Target with a low fee.

[rossmclochness]

nodes don't come to an agreement (if that's what you meant) when it comes to blocks. a block is either valid or not and the blockchain that a node accepts is the longest one.... let me try to explain how mining works, maybe that answers these questions:... we don't need to reach any consensus to decide whether this new block should be accepted or not, this block has been too late, and if anybody continues building on it they will end up with a "shorter" blockchain because they started it late and the rest of the network that received the block found by person 2 are already ahead and have the longest chain which is the one everyone should follow.

I appreciate the detailed explanation!!! So I do understand the concept and there is one part that is still somewhat unclear. Each node believes that it is working on the latest block in the chain until it receives a broadcast which it confirms and accept and realizes that it must create a new block based upon (perhaps) this latest block. At what point does forking actually begin to resolve -- a decision made by a node which is the longest chain? I'm assuming when miners begin to create and mine a new block, they perform a scan of the network to determine which chain is the longest. I'm not sure whether this is done by every block on the fly or there is some part of the protocol that would make such a request more efficient. At some point, the scanning results in the identification of the longest chain at that point in time.

That's not quite accurate. 1 confirmation can be reversed with much less than 51% of the hashrate. Section 11 of the bitcoin whitepaper explains why, and this site (https://people.xiph.org/~greg/attack_success.html) lets you play around with the numbers yourself. Indeed, with only 24% of the hashrate, an attacker has a 50% chance of being able to catch up to the main chain from a starting point of 1 block behind. Indeed, stale blocks (which are commonly but incorrectly referred to as orphaned blocks), happen not infrequently without anyone having close to 51% of the hashrate: https://www.blockchain.com/btc/orphaned-blocks

Great point and thank you. I think I understood the original context and your answer is actually correct based upon the findings of a couple of studies too, albeit many are theoretical. Still, you can impact the chain with significantly less than 51% of the hash rate. I think there were mining pools which accomplished this in the 40s such as BTC, GHash. But the first block being mined means "discovery" of a transaction unless you scan the mempool. The first block sets the events in motion towards a resolution while a transaction in the mempool can stay in limbo and even undetectable for a while if you aren't looking.

Target could spend the unconfirmed input they received from you, and attach a fee large enough to pay for transaction A and the transaction they are sending to themselves. In order for a miner to receive the second transaction fee, it must confirm both transactions.
This is called Child Pays for Parent, or CPFP. There is an interesting Blog Post by Coinbase on the subject.
Target could also have a special arrangement with one or more mining pools that involve Target paying the mining pool a special fee, offchain in exchange for the mining pool confirming transactions sent to Target with a low fee.

Great reply and thank you! This was another issue I was investigating. It's amazing how things seem to work much of the time but I have to wonder whether it really does. Essentially in this instance a CPFP can distort the transaction ordering in theory. Target (the first spend) basically protects itself by doing so. But if a less savvy merchant is first and a more savvy merchant is second and uses CPFP, essentially the second savvy merchant has protected itself and made sure that the double spend is far more likely to resolve in their favor even though technically they were the later, double spend transaction. In essence, it seems that in order for a merchant to be safe on the network, it really requires a good amount of understanding how the system works or they could be in for a very large surprise. This concept of paying a transaction processor a premium is counterintuitive to the standardization of the centralized system.

Many thanks to all of you for being generous with your replies.

[PrimeNumber7]

Target could spend the unconfirmed input they received from you, and attach a fee large enough to pay for transaction A and the transaction they are sending to themselves. In order for a miner to receive the second transaction fee, it must confirm both transactions.
This is called Child Pays for Parent, or CPFP. There is an interesting Blog Post by Coinbase on the subject.
Target could also have a special arrangement with one or more mining pools that involve Target paying the mining pool a special fee, offchain in exchange for the mining pool confirming transactions sent to Target with a low fee.

Great reply and thank you! This was another issue I was investigating. It's amazing how things seem to work much of the time but I have to wonder whether it really does. Essentially in this instance a CPFP can distort the transaction ordering in theory. Target (the first spend) basically protects itself by doing so. But if a less savvy merchant is first and a more savvy merchant is second and uses CPFP, essentially the second savvy merchant has protected itself and made sure that the double spend is far more likely to resolve in their favor even though technically they were the later, double spend transaction. In essence, it seems that in order for a merchant to be safe on the network, it really requires a good amount of understanding how the system works or they could be in for a very large surprise. This concept of paying a transaction processor a premium is counterintuitive to the standardization of the centralized system.

Yes, Target can protect itself by using a CPFP transaction. This is not unlike how merchants pay credit card processors a fee to process credit card transactions, and (somewhat/minimally) protect the merchants from fraud, only a CPFP transaction will generally cost pennies verses several percent of the value of a transaction for credit card processors.

In addition to using a CPFP transaction, a merchant can choose to only accept unconfirmed transactions that meet certain criteria, and tell their nodes to ignore unconfirmed transactions that does not meet their criteria. The merchant may choose to ignore transactions that pay a fee under a threshold that is likely to quickly confirm, and transactions whose inputs are also unconfirmed. In your example, Target may choose to ignore the zero fee transaction entirely and tell the customer he needs to wait until it is confirmed.

[squatter]

There seems to be two options for retail stores. Accept zero confirmations with a very high fraud risk -- and maybe mitigate that risk by requiring ID -- or integrate Lightning.

I guess a third option would be to have a waiting area for Bitcoin payers who are waiting for confirmations.

if you go to Walmart and try buying something with your personal check and then go to another shop and do the same,... you are doing the same double spend with fiat (as you would do with bitcoin with 0 fee!) that doesn't mean fiat is not accepted anywhere as payment

Walmart and other retailers now process personal checks like debit cards. They scan the check and immediately debit your bank account, so it's very difficult to double spend. If there's insufficient funds, the bank will just reject the debit and the store won't let you complete your purchase. This can be done in a matter of seconds, which is a lot more attractive than waiting around for Bitcoin confirmations.

[PrimeNumber7]

There seems to be two options for retail stores. Accept zero confirmations with a very high fraud risk -- and maybe mitigate that risk by requiring ID -- or integrate Lightning.

I guess a third option would be to have a waiting area for Bitcoin payers who are waiting for confirmations.

You can review my post, directly above yours for an additional option:

a merchant can choose to only accept unconfirmed transactions that meet certain criteria, and tell their nodes to ignore unconfirmed transactions that does not meet their criteria. The merchant may choose to ignore transactions that pay a fee under a threshold that is likely to quickly confirm, and transactions whose inputs are also unconfirmed. In your example, Target may choose to ignore the zero fee transaction entirely and tell the customer he needs to wait until it is confirmed.

Stores can also utilize CPFP transactions as explained in this post.