Help a newbie; why is hashing not done once but twice during Bitcoin transaction

[o_e_l_e_o]

that's not what happened. what happened is, the first coinbase transaction was logged on to the blockchain but not the 2nd one. so that's how it happened.

They are both logged quite clearly.
Block 91722: https://mempool.space/block/00000000000271a2dc26e7667f8419f2e15416dc6955e5a6c6cdf3f2574dd08e
Block 91880: https://mempool.space/block/00000000000743f190a18c5577a3c2d2a1f610ae9601ac046a38084ccb7cd721
You can see the identical transaction twice, once in each block.

what do you mean it first had to be detected? with all the intellectually powerful people in bitcoin, you would think that one would be a gimme. and they would have saw it coming a mile a way but i guess not.

You expect the bitcoin protocol and software to never have any bugs whatsoever? That's a completely unreasonable expectation.

maybe in your view that's how it worked but by looking at the two coinbase transaction time stamps they go by the first one.

Transactions don't have timestamps. Blocks do.

if you don't believe that then how do you expain the scenario of if the miner spent his 50 btc before the 2nd coinbase transaction came along?

If the miner had been quick enough in the first case, then they could have indeed spent the 50 BTC from the first transaction before the second one came along. But they didn't. And since the 50 BTC from the first transaction didn't move, when they made a second transaction with an identical identifier, then the first 50 BTC output was effectively overwritten by the second one with the same identifier.

[1715036418]

1715036418

[1715036418]

1715036418

[1715036418]

1715036418

[1715036418]

1715036418

[pooya87]

what do you mean it first had to be detected? with all the intellectually powerful people in bitcoin, you would think that one would be a gimme. and they would have saw it coming a mile a way but i guess not. an inflation bug is one thing but something like this. satoshi should have realized that could happen. suprised it didn't happen to him as many blocks as he mined!

If you some day develop a software you realize how easy it is to have all kinds of bugs, from simple silly ones to big fundamental bugs.

The blockchain is immutable and can not change. What happens is that the first transaction's output was placed in UTXO set after it was created, then the second one came along and replaced it in the UTXO set.

how exactly does that happen?

I'm not sure how to explain this, lets say the UTXO set is like this and lets assume that each subsequent block comes along after all the items in the set matured (100 block depth for coinbase transactions to become spendable):

Code:

u-01
u-02
u-03

A new block is received that spends u-02 and creates two new outputs (u-10 and u-11)
After the block was verified and added to the database the UTXO set looks like this:

Code:

u-01
u-03
u-10
u-11

A new block is received that spends u-01 and creates u-02 (we had this before in an earlier block but the utxo is already spent in a previous block). After its verification it is added to the database and the updated UTXO set looks like this:

Code:

u-02
u-03
u-10
u-11

With or without BIP30 this behavior is the same.

A new block is created (before BIP30) that doesn't spend anything and creates u-03 (this is duplicate that already exists in the set). After its verification it is added to the database and the updated UTXO set looks like this:

Code:

u-02
u-03
u-10
u-11

As you can see the UTXO set didn't change even though u-03 was created again, the behavior under the hood is that the UTXO is overwritten in the set by the new one that looks identical. But realistically nothing changed since the UTXO is the same exact thing.

Now BIP30 activates and we receive a new block that doesn't spend anything but creates u-03 (again this is duplicate). This block is rejected as invalid.

maybe in your view that's how it worked but by looking at the two coinbase transaction time stamps they go by the first one.

Transactions don't have timestamps. Blocks do.

Coinbase transactions have something that would resemble that. They need to mature before they can be spent (100 block deep).

As far as I can tell from the code, the UTXO will be replaced by the new one so the block height of the coinbase tx has to also be updated which would make it unspendable until another 100 blocks are mined.
I'm not confident in my understanding of C++ though.

[Lucius]

Very interesting topic, so even though I'm not technically savvy to discuss at this level, how much have I realized what happened in the past shouldn't happen anymore? What interests me is whether it is theoretically possible for such events to happen again, because if I understood correctly these two coinbase transactions with identical IDs actually annulled each other and we have 100 BTC less.

If it were possible to reproduce something similar, perhaps it might occur to someone to reduce the max supply a little more - or is it technically impossible to do so after a fix?

[o_e_l_e_o]

If it were possible to reproduce something similar, perhaps it might occur to someone to reduce the max supply a little more - or is it technically impossible to do so after a fix?

It is impossible to reproduce the same bug again, as this is prevented by both BIP30 and BIP34. Are there other bugs out there we are unaware of which could permanently remove some bitcoin from the supply? Most likely.

In addition to that, any miner can choose to remove coins from the supply any time they find a block by not claiming the full block reward they are permitted to claim. This has happened many times in the past, usually for small sums of a few thousand sats or a few mBTC, but occasionally for much larger sums. Block 501,726, for example, permanently destroyed 12.5 BTC by falling to claim anything in the coinbase transaction.

[pooya87]

It is not just impossible because the bug was fixed but also because an additional check was added to the consensus critical code that would reject such blocks.
In simple terms even if by a miracle that happens once every million years you produce a SHA256 collision (in any tx not just the coinbase tx), and the other output with the same hash and index (the UTXO) was not spent yet, the block will be rejected as invalid.
https://github.com/bitcoin/bitcoin/blob/747cdf1d652d8587e9f2e3d4436c3ecdbf56d0a5/src/validation.cpp#L2133-L2140

[larry_vw_1955]

that's not what happened. what happened is, the first coinbase transaction was logged on to the blockchain but not the 2nd one. so that's how it happened.

They are both logged quite clearly.
Block 91722: https://mempool.space/block/00000000000271a2dc26e7667f8419f2e15416dc6955e5a6c6cdf3f2574dd08e
Block 91880: https://mempool.space/block/00000000000743f190a18c5577a3c2d2a1f610ae9601ac046a38084ccb7cd721
You can see the identical transaction twice, once in each block.

Yeah on that particular block explorer, but not on blockchain.com. If you look at blockchain.com, you can see that block 91880 has the same timestamp as block 91722 for the coinbase transaction.

You expect the bitcoin protocol and software to never have any bugs whatsoever? That's a completely unreasonable expectation.

I think that would be the expectation of the general public who only has a passing knowledge of bitcoin from watching cnn. Their bank never has bugs that affect their account balance or if they do they get fixed so that no one loses any money...

maybe in your view that's how it worked but by looking at the two coinbase transaction time stamps they go by the first one.

Transactions don't have timestamps. Blocks do.

Blockchain.com timestamps each transaction inside each block. Not every transaction has the same timestamp.

If the miner had been quick enough in the first case, then they could have indeed spent the 50 BTC from the first transaction before the second one came along. But they didn't. And since the 50 BTC from the first transaction didn't move, when they made a second transaction with an identical identifier, then the first 50 BTC output was effectively overwritten by the second one with the same identifier.

Blockchain.com retained the timestamp of the first coinbase transaction on the 2nd coinbase transaction. Now you might wonder how can a coinbase transaction be timestamped as 2010-11-14 03:37 when the block it is located in is timestamped 2010-11-14 19:36. Normally that could never happen the only way it can happen is if a previous utxo txid happened to match it so it used that one instead. Without going into the code though we can't really know for sure what happened,

It is impossible to reproduce the same bug again, as this is prevented by both BIP30 and BIP34. Are there other bugs out there we are unaware of which could permanently remove some bitcoin from the supply? Most likely.

i would think by now with computer software and technology development being what it is, automated systems have taken over finding and fixing bugs. if they would use that on bitcoin, it might help. maybe it's more complicated than that but i think it's a reasonable goal. what would be so hard about programming a system to find bugs like this duplicate transaction id?

[o_e_l_e_o]

Yeah on that particular block explorer, but not on blockchain.com. If you look at blockchain.com, you can see that block 91880 has the same timestamp as block 91722 for the coinbase transaction.

That's a fault of blockchain.com and has nothing to do with the bitcoin protocol. Blockchain.com assigns a timestamp to each transaction based on when they first saw that transaction in their mempool, which is why the duplicate second transaction has the same timestamp as the first transaction. The internal workings of blockchain.com are irrelevant to how the network treats these transactions.

I think that would be the expectation of the general public who only has a passing knowledge of bitcoin from watching cnn. Their bank never has bugs that affect their account balance or if they do they get fixed so that no one loses any money...

I mean, people have literally gone to jail because of bugs in their bank accounts giving them money they shouldn't have received.

Blockchain.com timestamps each transaction inside each block. Not every transaction has the same timestamp.

As above, this is when blockchain.com's node first learns of that transaction. This is not related to bitcoin at a protocol level.

i would think by now with computer software and technology development being what it is, automated systems have taken over finding and fixing bugs.

You have some very unrealistic views on how software is developed.

[LoyceV]

Yeah on that particular block explorer, but not on blockchain.com. If you look at blockchain.com, you can see that block 91880 has the same timestamp as block 91722 for the coinbase transaction.

That's a good reminder you shouldn't always trust block explorers (and especially not to trust blockchain.com, they have far too many bugs).

Their bank never has bugs that affect their account balance or if they do they get fixed so that no one loses any money...

I've read countless stories about bugs in bank account balances. Say 10 billion dollar balance, or transactions that get processed twice. Usually the bank indeed fixes it, but it can still cause problems (such as spending money you don't have).

[BlackHatCoiner]

what would be so hard about programming a system to find bugs like this duplicate transaction id?

That it may function falsely. Welcome to software engineering.

You can't know you have a bug until you do something you'd never done before which appeared an unexpected result. The only way to lower your chances is to do structured programming which makes your code easier to read and develop further by third parties. It also takes less time to build things that way.

From my experience, when I start over-complicating my code for some reason, it's usually when I know I'll have trouble.

Their bank never has bugs that affect their account balance or if they do they get fixed so that no one loses any money...

Start reading: https://www.ptsecurity.com/ww-en/analytics/vulnerabilities-rbo-2019/

[larry_vw_1955]

Yeah on that particular block explorer, but not on blockchain.com. If you look at blockchain.com, you can see that block 91880 has the same timestamp as block 91722 for the coinbase transaction.

That's a fault of blockchain.com and has nothing to do with the bitcoin protocol. Blockchain.com assigns a timestamp to each transaction based on when they first saw that transaction in their mempool, which is why the duplicate second transaction has the same timestamp as the first transaction. The internal workings of blockchain.com are irrelevant to how the network treats these transactions.

ok well how am i supposed to know that? it doesn't exactly say that on their website. so i figured its part of the blockchain the timestamp data.

I mean, people have literally gone to jail because of bugs in their bank accounts giving them money they shouldn't have received.

you dont go to jail just because of a bank errorr unless you took it a step further and spent the money when it didn't belong to you. now if someone did that, they know they are doing that. and there's no excuse.

Blockchain.com timestamps each transaction inside each block. Not every transaction has the same timestamp.

As above, this is when blockchain.com's node first learns of that transaction. This is not related to bitcoin at a protocol level.

well it's a nice feature to be able to see when a transaction was done. maybe they should add that to the protocol. everything should have a timestamp i would think. or some way of deciding the ordering of objects in time. even down to the transaction level. if blockchain.com can do it why can't bitcoin?

i would think by now with computer software and technology development being what it is, automated systems have taken over finding and fixing bugs.

You have some very unrealistic views on how software is developed.

well if you think bitcoin is the best software development model ever developed then i would have to disagree. i think there are better ones out there. but as long as bitcoin keeps working its fine but hopefully no big cve comes along and makes bitcoin go to 0. that could happen. it almost did once. luckily we had satoshi to save our ass that time.
 

You should spend few minutes on website which list CVE (Common Vulnerabilities and Exposures) such as https://www.cvedetails.com/cvss-score-distribution.php (https://www.cvedetails.com/cvss-score-distribution.php). If you browse The "Top 50", you'll find out popular product/vendor have hundred to thousand CVE.

that's because they don't care about quality, they are concerned with pumping crap out the door. getting it to market to make more bucks.

[ABCbits]

Blockchain.com timestamps each transaction inside each block. Not every transaction has the same timestamp.

As above, this is when blockchain.com's node first learns of that transaction. This is not related to bitcoin at a protocol level.

well it's a nice feature to be able to see when a transaction was done. maybe they should add that to the protocol. everything should have a timestamp i would think. or some way of deciding the ordering of objects in time. even down to the transaction level. if blockchain.com can do it why can't bitcoin?

It's possible, but
1. It introduce small bloat. Could be 4 or 8 bytes depending on whether you concerned about year 2038/2106 problem.
2. Since transaction could broadcaster far before it's included on block, it's hard to verify acuuracy of the timestamp on transaction.

You should spend few minutes on website which list CVE (Common Vulnerabilities and Exposures) such as https://www.cvedetails.com/cvss-score-distribution.php (https://www.cvedetails.com/cvss-score-distribution.php). If you browse The "Top 50", you'll find out popular product/vendor have hundred to thousand CVE.

that's because they don't care about quality, they are concerned with pumping crap out the door. getting it to market to make more bucks.

But most of those vendor are big IT company with big net worth. Can you give few example of vendor or application who cares about quality?

[o_e_l_e_o]

ok well how am i supposed to know that?

Because as I said above, transactions do not have timestamps.

well it's a nice feature to be able to see when a transaction was done. maybe they should add that to the protocol. everything should have a timestamp i would think. or some way of deciding the ordering of objects in time. even down to the transaction level. if blockchain.com can do it why can't bitcoin?

Blockchain can do it because they are a single node recording the time that their node first saw a transaction, and recording it on a centralized database they control. What if my node sees a transaction now, and yours sees it 5 minutes later? Which timestamp to we go with? What if my mempool is full, but your node has different parameters meaning it accepts transaction I am ignoring, and we end up seeing the transaction days or even weeks apart? What if I sign a transaction with a timestamp of today, but choose not to broadcast it for a day, a month, a year? And what do we have to gain from it? It will make every transaction larger and therefore more expensive, and I can't see any meaningful benefit.

Note that the timestamp of blocks is irrelevant to deciding the ordering of them. Blocks are ordered based on their height, not their timestamp. There have been plenty of occasions where blocks have had earlier timestamps than blocks which came before them.

[larry_vw_1955]

Blockchain can do it because they are a single node recording the time that their node first saw a transaction, and recording it on a centralized database they control. What if my node sees a transaction now, and yours sees it 5 minutes later? Which timestamp to we go with? What if my mempool is full, but your node has different parameters meaning it accepts transaction I am ignoring, and we end up seeing the transaction days or even weeks apart? What if I sign a transaction with a timestamp of today, but choose not to broadcast it for a day, a month, a year? And what do we have to gain from it? It will make every transaction larger and therefore more expensive, and I can't see any meaningful benefit.

I don't know I thought that timestamps were supposed to be a way of putting things into order. If that's not the case then what's their purpose? Like for example if you had a way to timestamp transactions then you could create blocks based on transactions timestamps.

Note that the timestamp of blocks is irrelevant to deciding the ordering of them. Blocks are ordered based on their height, not their timestamp. There have been plenty of occasions where blocks have had earlier timestamps than blocks which came before them.

Then I'm not even sure why blocks contain a timestamp. If the timestamp isn't used.

But most of those vendor are big IT company with big net worth. Can you give few example of vendor or application who cares about quality?

Someone that is writing a program for their internal use only. Someone that develops a smart contract that has to be misssion critical and contain no errors so it has to be done not with spaghetti code but in a structured well thought out manner. now time tells how good their contract is. and it might be better than bitcoin in that sense that it has no bugs.

[pooya87]

I don't know I thought that timestamps were supposed to be a way of putting things into order. If that's not the case then what's their purpose?

We put things in order by creating a "chain" using cryptography, where block C comes after block B and before block D.
The timestamp is only used in block headers to be used for difficulty adjustment so that we can come up with a target value that lets use find 2016 blocks every 2 weeks.

[larry_vw_1955]

We put things in order by creating a "chain" using cryptography, where block C comes after block B and before block D.
The timestamp is only used in block headers to be used for difficulty adjustment so that we can come up with a target value that lets use find 2016 blocks every 2 weeks.

that's what i thought but if that's the case then blocks can't be in the wrong order based on timestamp. blocks with earlier timestamps have to come before blocks with later timestamps. otherwise, there is no well defined notation of "time between 2 blocks".

Note that the timestamp of blocks is irrelevant to deciding the ordering of them. Blocks are ordered based on their height, not their timestamp.

So a miner can just stamp a block with any old time they want? Like 50 years in the future?

There have been plenty of occasions where blocks have had earlier timestamps than blocks which came before them.

If that's the case then something isn't right. And I don't see how bitcoin could function like that. Seems like a bug.

[pooya87]

that's what i thought but if that's the case then blocks can't be in the wrong order based on timestamp. blocks with earlier timestamps have to come before blocks with later timestamps. otherwise, there is no well defined notation of "time between 2 blocks".

No, the order is, as I said, achieved using cryptography. Meaning block C references block B by including its hash in the header. That way the order can not change.
The block time can actually be different meaning block B can have a time that is bigger than block C. That is why the adjustment is not done every block but postponed to 2016 blocks/2 weeks and is a median time. There is also limitations on how wrong the time can be (up to 2 hours IIRC).

[o_e_l_e_o]

that's what i thought but if that's the case then blocks can't be in the wrong order based on timestamp.

They can, as I explained above. Here is an example:

145045 - 16:05
145046 - 16:00
145047 - 15:53
145048 - 16:04
145049 - 16:08

So a miner can just stamp a block with any old time they want? Like 50 years in the future?

The timestamp can range from the median time of the last 11 blocks (plus 1 second) to 2 hours in to the future (based on network adjusted time, not local time).

If that's the case then something isn't right. And I don't see how bitcoin could function like that. Seems like a bug.

Because the timestamp is used to determine the difficulty, not to determine the order of the blocks. It doesn't need to be accurate beyond a couple of hours.

[Cricktor]

Blockchain.com timestamps each transaction inside each block. Not every transaction has the same timestamp.

As above, this is when blockchain.com's node first learns of that transaction. This is not related to bitcoin at a protocol level.

well it's a nice feature to be able to see when a transaction was done. maybe they should add that to the protocol. everything should have a timestamp i would think. or some way of deciding the ordering of objects in time. even down to the transaction level. if blockchain.com can do it why can't bitcoin?

What makes you think what blockchain.com does is or should in any way be "authoritive" for the Bitcoin space?

A Bitcoin transaction is "done" when the block has been mined which contains this transaction and this block remains in the longest consensus chain of blocks. For the Bitcoin protocol and consensus it doesn't matter at all when exactly this transaction has been made aware into a node's mempool and how long it takes to be mined from a miner's mempool into a block. Adding timestamps and the space it needs to save such details makes only sense if you need those details. Apparantly such transaction timestamps were not considered to be needed or beneficial. (It doesn't matter if YOU find it necessary or beneficial or plain nice-to-have. Of course, you're free to try to convince a majority of Bitcoin nodes... )

[larry_vw_1955]

that's what i thought but if that's the case then blocks can't be in the wrong order based on timestamp.

They can, as I explained above. Here is an example:

145045 - 16:05
145046 - 16:00
145047 - 15:53
145048 - 16:04
145049 - 16:08

well i see what you mean but it is still counterintuitive to see it happening. is that what the timestamp was really meant for is something that miners could manipulate for their own purposes? i thought thats what the nonce was for.

What makes you think what blockchain.com does is or should in any way be "authoritive" for the Bitcoin space?

I don't know. I just thought they were.

A Bitcoin transaction is "done" when the block has been mined which contains this transaction and this block remains in the longest consensus chain of blocks. For the Bitcoin protocol and consensus it doesn't matter at all when exactly this transaction has been made aware into a node's mempool and how long it takes to be mined from a miner's mempool into a block. Adding timestamps and the space it needs to save such details makes only sense if you need those details.

Well i can think of alot of reason why a timestamp on a transaction is helpful. Every transaction in e-commerce uses timestamps. Without an accurate timestamp on your transaction how do you properly convince a tax authority exactly when a transaction occurred? You can't. All you can do is point to the block timestamp but what if that's not the same as when you actually submitted the transaction and it affects you negatively? Did you ever buy anything from amazon and not see a timestamp on your transaction?

Apparantly such transaction timestamps were not considered to be needed or beneficial.

Well whoever that was didn't have e-commerce in mind.

Adding timestamp to transaction require soft/hard fork, so you need to convince Bitcoin community instead. Although it's different case if it's opt-in where you simply utilize OP_RETURN.

But you can stuff anything you want to in OP_RETURN. Including fake timestamp.

[BlackHatCoiner]

well i see what you mean but it is still counterintuitive to see it happening. is that what the timestamp was really meant for is something that miners could manipulate for their own purposes?

It's meant to establish the difficulty. As said, there's a range they're allowed to manipulate, which doesn't affect the adjustment by a lot. You cannot avoid this limit; otherwise they'll perpetually sacrifice computational power for blocks that aren't complied with your rules.

i thought thats what the nonce was for.

The nonce is just a number used once by miners on their way to find a valid block hash. Check: Nonce.

Well i can think of alot of reason why a timestamp on a transaction is helpful. Every transaction in e-commerce uses timestamps.

Yes, but that has a point only in trust-based systems. In Bitcoin, that works asynchronously, you can't know when blockchain dot com's node received the transaction or when you did. When it gets confirmed, your running node will immediately receive it, but one who's syncing won't have it yet. If the signer can set the timestamp, which doesn't make sense, you can't know if it's faked.