Ok, so I'd say I have a very strong understanding of the fundamentals of Bitcoin, and the blockchain in general. I can explain pretty much (at a high level) the flow of almost every part of the ecosystem. << I say that not because I think I know even a fraction of what people on this forum do, but rather to give an honest assessment of where my question is coming from.
Based on what you've said in your post, I suspect that you are overestimating what you know. It seems that you've been given incorrect information, and incorporated that incorrect information into your beliefs about how bitcoin works.
I'm hoping that you at least are aware that an unconfirmed transaction is relayed from peer to peer until nearly the entire network has heard about the transaction, and that the transaction is considered to have its first "confirmation" when it is included in a block.
However, I do not understand what is happening at a technical, and I guess somewhat low-fundamental level, when a block is "added to the chain." I frequently hear people talk about, "Oh yeah, so after the miners mine a block, then they pack it with transactions, then boom, it's added to the chain. And it's secure because all the blocks require a lot of work to reverse." It seems most people are satisfied with that answer. Nothing wrong with that, it's just I'm looking for more.
That answer is incorrect. Transactions are NOT added AFTER the miner mines a block. They are added BEFORE.
I would greatly appreciate if someone (or multiple people.. the more the better) could explain what's actually going on there. What actually determines "We're done with this block, now let's add it to the chain." What is happening at a technical level when that block is added to the chain? What code is being run to facilitate this? Who has write access to the chain?
English is not my first language, so I apologize if I'm not articulating my questions accurately. There's just a knowledge void that's making me uncomfortable. I really want to understand everything about the process; every detail is meaningful to me.
Thanks in advance.
Every peer on the network makes sure that a transaction is valid before it relays it to any of the other peers that it is connected to. This makes it effectively impossible to flood the network with invalid transactions, since all peers will refuse to relay them.
Solo-miners and mining pools operate peer nodes on the bitcoin network. Those peer nodes receive and verify unconfirmed transactions from the peers they are connected to. Each solo-miner and mining pool keeps track of the valid unconfirmed transactions that it has received so far. This list is called a "mempool".
Note that mining pool participants do not typically run their own peer nodes. Instead they rely on the pool to manage that for them, and they just receive units of "Work" from the pool to complete. For the rest of this post, any time I use the word "miner", I am talking about solo-miners or the mining pool operator.
The miner builds a valid block of currently unconfirmed transactions. They then build an 80 byte header for that block of transactions. The header contains 6 pieces of information:
- A version number
- The hash of the previous block in the blockchain (Yhis is how the blocks form a "chain". Each new block has a reference to the block that proceeds it all the way back to the very first block)
- The merkle root of the transaction list (This is a hash that represents all the transactions that were chosen for the block, and the specific order that was chosen for those transactions. To change ANYTHING about the transaction list, this hash must be re-calculated and will be different. This is how the header can prove that it is a header for this specific block of transactions.)
- A timestamp
- The current mining difficulty in a format that is commonly called "nBits"
- The nonce (This value exists solely for the purpose of modifying the result of hashing the block header)
The miner then begins the process of "solving" this block. In the case of a mining pool, then pool sends the header to the participants for them to work on it.
The process of solving works like this:
- 1. Hash the 80 byte block header using the SHA256 algorithm.
- 2. Hash the result of that hash using the SHA256 algorithm again.
- 3. Check to see if the result of that second hashing is a lower value than the current difficulty target.
- 4a. If it is lower, then the block is "solved". Immediately add the block to your own copy of the blockchain, and broadcast the solved block to all connected peers.
- 4b.If it is NOT lower, then the block is NOT solved. Modify something about the block header, and go back to step 1.
That's it. That's all there is to it.
Typically, the "something" that is modified about the block header is the "nonce". However, the nonce is only a 4 byte value (32 bit value). That means there are a maximum of 4294967296 different nonce values to try. If you try all of them, and you haven't yet "solved" the block, then you need to modify something else. Often the timestamp can be modified, and they you can try another 4294967296 nonce values. In a very large mining pool, there might not be enough combinations of valid timestamp and nonce values for everyone to be getting different results (don't want participants to waste time both hashing the exact same 80 byte values). In that case, the transaction list can be slightly modified for each participant so that they are each working with a completely different merkle root. If a participant has so much hash power that they complete all available valid timestamp and nonce combinations for the merkle root they were given, then the transaction list can be slightly modified again to get them another merkle root to work with. Since there are constantly new transactions being received, the miner can easily "modify the transaction list" by adding the new transactions they have received into the list they are using to build their block (and removing cheaper transactions to keep the size under the required limit).
As each peer receives the new block from a connected peer, they check to make sure that all transactions in the block are valid. They also check to make sure that the merkle root in the block header matches the transaction list. They also check to make sure that everything else about the header is valid (timestamp, version, previous block hash, nBits). They also hcek to make sure that hashing the block header results in a low enough hash value (below the current difficulty target). Assuming that the block is found to be completely valid, the peer adds the block to their own copy of the blockchain, and then they relay it to all the peers they are connected to. Eventually nearly the entire network will have heard about the block.
As each miner receives this completed block, they do the same things that all the other peers do (validate the block, add it to their own chain, and relay it to the peers they are connected to), and then they build all new blocks to work on (using the hash of this block that they just receive as the new "previous block hash").
That describes how the blocks are built, and how everyone gets a copy for their own blockchain.
The reason it is "secure" is that the only way someone could get all the nodes of the network to replace a block that they've already accepted would be to provide that peer with a longer valid chain that contains the replacement block. This means that the attacker would need to complete the hashing "proof-of-work" for the block that they want to replace PLUS all the blocks that come after that (since their replacement block will have a different hash, and the next block includes a "previous block hash" that they will need to modify to match their replacement) PLUS one extra block (since nodes will only replace their existing chain with a LONGER chain). Since the ENTIRE world is working on continuously extending the length of the current blockchain, that means that the attacker will need more hash power than the entire rest of the world combined in order to solve blocks faster than the rest of the world and catch up to the length of their chain. If an attacker has more hash power than the rest of the world combined, then they can make a LOT more money by simply honestly hashing valid new blocks instead of trying to replace a single old block.
