Yes. Note that higher difficulty means that the target value for that block is lower than the target value for the block at the current tip, not that one block's hash is lower than the other's. Given two blocks that descend from the same ancestor, this is impossible.

When two competing blocks are received and their PoWs are equal (as two blocks descending from the same ancestor would be), the first block received is the one that remains on the tip. The second block is kept, but marked as stale. It is still tracked, so if another block is received extending the stale block, then the node will mark its original tip as stale and switch to the new most difficulty chain (a block mined on top of the stale block makes that branch the most difficulty).

Yes.

No. If you could, there would be a whole lot of forks.

Yes. Syncing is basically just this updating. The main thing is that while syncing, certain functionality is disabled to avoid operating with an old chain tip. There are a few optimizations to speed up syncing (e.g. assumevalid and minimumchainwork), but the process of receiving and validating blocks during sync is pretty much the same as receiving and validating them during normal operation.

achow, I believe this thread should be stickied, for all newbies to read before the trolls get them with the wrong information first.

Thanks for all your helpful and detailed replies! Having considered your comments and thought about it all some more, I've come up with the following updated overview. The bits in bold italics are additional assumptions I've made about the mempool, and definitely need reviewing...

Further feedback and corrections, gratefully received!

1. When a new block has been successfully mined, it is propagated throughout the network for nodes to validate.

2. When a node receives a newly mined block, it first validates the whole block (e.g. hash of block header < difficulty target; hash of previous block = hash of block which node wants to build on, etc.) If it fails, the block is immediately rejected. If it is validated, the node immediately propogates the block to other nodes in the network for them to validate, and then procedes to validate the block's individual transactions (all inputs are valid UTXOs; sum of outputs < or = sum of inputs, etc.). If it passes both of these validation steps, the block is accepted by the node, added to its version of the blockchain, and the block's transactions removed from the node's mempool.

3. If the same node subsequently receives a valid competing block (with the same parent as the one added in 2. above), the initially-added block remains at the tip of the active chain, but the valid competing block is also kept, marked stale, but still monitored. If the node then receives a valid block which links cryptographically to the stale block, the block at the tip of what was the active chain becomes stale, and the now longer branch becomes the active chain instead.

4. Stale blocks are not discarded because they are still valid blocks with an ancestor within the active chain. Their transactions, however, remain in the node's mempool, or are returned to the mempool if previously removed. If a stale block becomes part of the active chain again, its transactions will be removed from the node's mempool.

5. If a node receives a valid block which links cryptographically to a stale block, and this happens to create a chain (B) with a higher total difficulty, but which is still shorter than the active chain (A) in terms of number of blocks, then chain B will become the active chain, and all of the blocks in chain A after it splits (although not discarded) will now become stale.

Thanks for the additional insight!
Is my whole overview a better reflection now of what happens at this "block appending" stage? I've tried to take into consideration all the different feedback I received.

oh windy. [facepalm]
if anything should be learned is now that new opcodes and transaction formats  can be slid in without hardforks as they please. it should actually be very much highlighted that not all nodes on the network are the newest up-to-date full archiving nodes able to understand all the new txformats.

this means
a. not all nodes actually fully validate, take people running nodes that are not segwit compatible,, they do not validate segwit signatures/scripts(now called witnesses) instead they blindly see a tx format they do not understand and presume its safe as they recieved the block containing the tx from a peer so the peer must have validated it(trust not full validation). and the same goes for even more newer differing tx formats. again for importance without checking that the spending addresses scripts/signatures validate.

b. also apart from the criteria of validation, to be a TRUE "full node" you ned to be archival thus able to seed data for your peers to get. this also not unanimous to all peers as some run in pruned mode, etc.
just go check https://bitnodes.earn.com/nodes/?q=1037 and you will see at time of writing this. out of 9200 nodes listed only 6400 nodes are uptodate to the latest rule following standards of what would be "full node" standard.

c. some nodes use checkpoints(min chain work) and when initially syncing the blockchain. they dont validate every transactions and every block from genesis, they just download the blockchain and only validate transactions/blocks after the milestoned point indicated. again presuming if the peers have the same then it must be good.
which from a security point presuming a new funky txformat not yet understood is 'valid' or early blocks are 'valid' purely due to presumption/trust that peers done their jobs for you. is bad bad security
specially if its only one group of devs that decide what chainwork to have as the default

'trust core' [facepalm] (do you start seeing that core have more power than an open community want)

d. even without you knowing the technicals. you seem to be trying to want to make this point for a popular social debate going on about presuming who has the real power in the network. EG trying to hide the fact that devs and the major exchanges have sway
however in social attack scenarios. if you as a node runner are told merchants and miners are being made to accept one version of a block the home users react by upgrading thier node to that version out of fear that they will get thrown off the network or be unable to spend thier funds because the merchants dont recognise them if they end up on the wrong version.
yep thats right merchants and miners have more sway because people fear being left off a network that is not compatible with their favourite merchant.

take the min chainwork stuff by cor and merchants using nodes with cores prefered minchainwork. means nodes not wanting to be core sheep dont get a voice. thus core and merchants have more power than home users to decide what the rules are
(as proven by the controversy of the NYA and august 1st 2017 drama )

anyway back to the technicals and point.
if a node is not 1037 its not doing all the full node requirements but is treated blindly as being a fullnode in respect of being on the network
not all nodes validate everything.

even gmax is very wishy washy about it
for instance some nodes are not 'native' segwit ready thus although they are network nodes they are not validating segwit signatures/scripts.

If you're getting into implementation details,

It might also be worth mentioning that most of the validation is typically done *before* the block comes in. In Bitcoin a great effort has been made to design things so that most of the validation is a pure function of the transaction-- for almost all rules given transaction is either valid or invalid depending only on its own content and nothing else.  The exceptions to this are checking that the inputs exist and aren't already spent, checking that the transaction nlocktimes are valid according to the block time, and checking that various block-wide global limits are obeyed (primarily that the block weight isn't exceeded).

This means that in the common case of transactions that were received ahead of the block most of their validation (amounts, scripts, signatures, etc) is cached and doesn't need to be done again when the block shows up, greatly reducing latency.

I think it's important to be aware of this because I've seen a number of posts over the years that seem to think that every time a new block is accepted everything in it must be laboriously reprocessed or even that the whole blockchain must be reprocessed. This isn't true, and it gives the wrong impression about where bottlenecks are on average.

This is made more complicated in Bitcoin because since 2010 validity in relay uses somewhat different, more strict, rules than validity in blocks to improve forward-compatibility.

The idea consensus rules can be improved in a way which is completely compatible with old software by strictly reducing the set of things that are permitted and by purposefully setting aside extension space in the protocol to eventually get restricted (such as NOP op-codes), old nodes then know something 'future' is going on and thus won't relay or mine it which would risk including something invalid, but are happy to accept whatever new nodes mine using the extensions.   Along with that, the consensus rules for a given block might be more strict than the rules for the block before it e.g. when a new rule activates.  

So the caching has to handle the rules changing and does that by augmenting the key for the cache with the set of rules, so the cache can only have a hit when the right rules are being used... also when a node receives a transaction it validates it under both the stricter relay rules and the looser block rules (why both when the relay rules are stricter?  because then there would be a risk if there were a bug in the relay rules it could result in accepting an invalid block, which would make it much slower to develop code for relay-rules).

[Aside, I removed a thread-derailing repetitive post by franky; and I'd strongly recommend everyone follow my lead and stick him on ignore to avoid wasting time and energy on his aggressive misinformation and disruption]


Yes, in Bitcoin it is technically possible for the height to go down!  But it's not likely something we'll ever observe on mainnet, at least not near the tip of a synced node.  The reason is that the difficult of blocks only changes (retargets) once every 2016 blocks and reflects the time that the last 2015 blocks took. So for fewer blocks to have more work there has to be a fork around a retarget and the difficulty on the two forks has to be different enough that a branch with N blocks can have more work than a Block with N+1 blocks, and the fork has to be at least N blocks long.  Commonly forks are only 1 block long and for a branch with one block to have more work than a branch with two the difficulty will have had to more than double in that one fork.  It would be really hard for such a large difficulty difference because the two blocks will share most of the same history going into their difficulty calculation.  Having it happen with a longer fork would require less of a difficulty difference but longer forks are themselves more rare.

So the only time with a mainnet node that I would expect to see height go down is during initial sync, if you connected initially to a malicious clown who intentionally forked the blockchain early on and purposefully kept the difficulty down and churned out a lot of blocks at low difficulty, then height might go down temporarily as you switch from his joke chain to the real chain.

Various altcoins have various psycho over-eager difficulty update rules where its very easy for height to go down, either intentionally or by accident. In theory that's okay, but I bet a lot of software won't handle it well.

Fun fact: the Bitcoin whitepaper and the software up until IIRC early 2010 made the best valid chain decision on total blocks and ignored total work. Unfortunately, as I mentioned, it's quite possible for a fork that starts really early in the chain to win by number of blocks but not work-- even though it's something you'll never see in normal operation, so Satoshi's original way of handling it was vulnerable and easily attacked.