So I have been trying to understand the process on how transactions get verified but have a few questions that I was hoping you guys could answer. My current understanding of a transaction is like this:
I send 1 bitcoin from my public key (signed with my private key) to another public key. This is then sent to the mempool of the nearest node and then if it meets the requirements of a valid transaction this is broadcast to all other nodes on the network. If I add a high fee to my transaction it will be picked up by a miner and added to a block quickly, if I select a low fee or no fee it risks staying in the mempool. If it is not picked up by a miner after a certain amount of time it will be returned to my wallet.
While that is a reasonable high-level understanding of transactions, it gets some of the details incorrect.
Try this:
You create and sign a transaction that is funded with bitcoins that you received in earlier transactions. The public key that was indicated in those earlier transactions indicate which private key must be used to generate the signature. Your transaction indicates a public key that must be used when verifying a signature that spends those bitcoins.
This transaction is shared with ALL the nodes that your wallet is connected to.
Each node verifies for itself that the transaction is valid. If it is, then the transaction is added to the node's own mempool and is shared with all the nodes that are connected to that node. Each of those nodes verifies for itself that the transaction is valid. If it is, then the transaction is added to the node's own mempool and is shared with all the nodes that are connected to that node. And so on until nearly all nodes on the network (including mining nodes) have heard about the transaction.
If you add a high enough fee to your transaction it will be added to one of the next few blocks that are solved worldwide.
If you select a low fee or no fee it risks staying in the mempool (or perhaps even never being added to the mempool of some nodes at all). If it is not included in a block after a whilemany nodes may drop it from their mempool and forget that the transaction ever existed.
Depending on the wallet software that you are using, your wallet might choose to also forget about the transaction after a while (in whice case it will once again show those bitcoins as being available for you to try spending again) or it may re-broadcast the transaction (to remind all the nodes and miners about it), or it may choose to just continue to remember the transaction without re-broadcasting it until you take some action to drop the transaction from your wallet or re-broadcast it.
Any other node that has ever seen the transaction can re-broadcast the transaction if they want to so long as it is valid.
Question 1: How does the miner verify my transaction in the mem pool?
The solo-miner (or mining pool operator) runs node software. That node software will verify the transaction BEFORE adding it to it's own mempool.
Does the miner's software automatically reference all the existing transactions on the blockchain to ensure that my account has 1 bitcoin to send?
Node software generally keeps a list of unspent transaction outputs (often referred to as the UTXO). Your transaction is funded by listing (as inputs) unspent transaction outputs. Any transaction outputs that are listed as inputs are then considered to be "spent" and are removed from the UTXO. So, if your transaction lists any inputs that are not in the UTXO, then your transaction is invalid. If any signatures in your transaction do not validate properly, then your transaction is invalid. If any of the structure of your transaction does not line up with the required structure, then your transaction is invalid.
Questions 2: Are only valid transactions added to a block to be verified?
Correct. Nodes will not add invalid transactions to their mempool. Nodes will not relay invalid transactions to other nodes. Miners will not accept invalid transactions into the blocks they are building. Nodes will not accept into their blockchain any blocks with invalid transactions. Nodes will not relay blocks with invalid transactions to other nodes.
Question 3: It is my understanding that a block needs an even number of transactions in a block in order to hash down to a merkle root.
This would appear to be untrue. The merkle algorithm has processes for dealing with odd numbers of transactions.
If transactions are in fact verified in a block and not in the mempool,
Transactions are verified before being added to a mempool. They are again verified before adding a block to a blockchain.
is an additional transaction added to the block when one is discarded
The protocol rules allow the miner to add as many or as few transaction to their block as they like so long as they include AT LEAST the block reward transaction, AND the block does not exceed the maximum allowed block size (or block weight). The only other requirement is that the transactions in the block are valid. The miner can use any criteria they like to choose which transactions to include in the block they are working on and which to leave out. Since most miners and mining pools are profit motivated, they will generally choose the transactions that pay the highest fee per byte (or per block weight).
Question 4: Do miners create their own custom software or are all miners running the exact same software?
I'm sure there are some solo-miners and some mining pools that have created their own custom software. I'm sure there are also some solo-miners and some mining pools that are using publicly available open source software.
