Newbie Trying to understand the mining calculation process.

[skeeterskeeter]

Hello first timer here,

I was interested in figuring out exactly what happens when one mines bitcoins and I guess the over backbone of mining. Sorry for my lack of jargon and knowledge, I am learning! Also, I relate alot of the abstract ideas to that of Object Oriented Programing (OOP) So when I say "object" I mean all data that encompasses the type of object beign referred too. (Ex, transaction object hold all info about a transaction)

So let me explain what I think I know.

== Step one: Lets barter my friend. ==

• Two people want to make a transaction.
• User 1 has 10BTC to give to User 2.
• So they create a trancsaction "object" (refering to OOP). This object has X input fields and X output fields.
• The input fields refer to past transactions that basically put/hold the 10BTC in user 1's wallet. (User one never has nothing physical, just the agreement from everyone else (all other nodes) that they are allowed to I guess "work with" 10 BTC).
• The outputs of the transaction is the address(s) (bitcoin wallet(s)) that are now allowed to work with the given BTC. One of these outputs should refer to the same wallet address as "change". In effect everytime a User creates a transaction they momentairly loose all BTC they own until the output of the current transaction gives them their "change" back. (Not sure if this is right, regardless it is possible)

Questions in step one:

• Where does the "signing" come into play?
• Overall is this the jist of it? Am I missing something important?

== Step two: Lets notify er'one of our bartering ==

• User 1/User 2 submit their transaction to everyone else (all other nodes). We all collectively put this transaction in a "block" which when "mined" is added to the "block chain".

Question in step two:

• Which user notifies who about the transaction? (i.e. where is the transaction object sent to?) Is it that both user notify all users, one user notifies the "closest" node and relies on nodes close to that to relay the transaction?
• How is this transaction added to the block?

== Step three: We have a block lets get some Au from it ==

• Any user who wishes to mine gets info from the bitcoin server about the current block.
• The user tries to hash the block header to obtain a hash value smaller than the current target.
• If they succeed they then notify everyone that they found an approvable hash of the current block header, if everyone agrees, they get 25BTC.
• If not they increase the nonce and try again.

Questions in step three:

• Who/what holds information about the current block? (everyone?)
• I understand what hashing is used for (save space mostly), but why/and how is it used to verify transactions? I do not see how hashing the block header creates a "proof of work" of all transactions...
• The new 25BTC comes from where? (they are released for use by ...), along with 25BTC the winner also gets any mess ups from the previous transaction right? (say someone forgets to gives themselves their "change")?

~~Overall questions~~

• How do we get from however many possible millions of transactions to a "block"?
  :::: I figure a "block" just holds a linked list of transaction objects, whenever a new transaction is created it is just appeneded to this list. Then when someone starts to "mine" this block he/she is actually looking for the shortest spanning tree over all transactions if they were in a graph form similar to this picture (https://en.bitcoin.it/wiki/File:Transaction.png)?
• (Again from step three) How does hashing the "block header" give us the proof that we need?
• What is the proof that we need, what are we actually computing when we are hashing the block header?

I figure, as i said just above, that to solve a block means we find the minimum spanning tree over all given transactions. Given that the transaction are connected as such (https://en.bitcoin.it/wiki/File:Transaction.png)

• What happens when a fradulent transaction is encoutnered?
• How can we tell a transaction is fradulent?
• Say two users wish to exchange bitcoins, user 1 gives user 2 10BTC. User 1 tells everyone "I gave 10 BTC to user 2, allow him to use 10BTC on all future transactions". On the other hand user two might try to say "User 1 gave my 1000BTC for me to use" How do we tell who is lying (suppose user 1 has over 1000BTC)?

END NOTES:
- First thanks for reading my wall of text and any answer you may provide.
- If you think (after answering my Q's) I might have another one, try and answer that too.
- If to much to try and answer all my question specifically try to give me and overview and hit on the points you think I lack most on.

[DannyHamilton]

Questions in step one:

• Where does the "signing" come into play?

The inputs contain a signature and public key.  The public key is the public key associated with the bitcoin address that previously recevied the output that is being referenced as an input.  The signature proves that the person using the previous output has the authority to use the previous output in an input.

• Overall is this the jist of it? Am I missing something important?

This is the "jist" of it.  There are some technical details that you've glossed over, but you seem to have the basics down.

Question in step two:

• Which user notifies who about the transaction? (i.e. where is the transaction object sent to?) Is it that both user notify all users, one user notifies the "closest" node and relies on nodes close to that to relay the transaction?

Hopefully someone who understands the protoocol better than me will clarify if I'm mistaken on this point, but it is my understanding that the person sending the bitcoin notifies all the nodes that their client is connected to.  Each of those nodes then relays the transaction to every node they are connected to.  This relaying continues as the transaction works its way across the network.

• How is this transaction added to the block?

Eventually mining nodes (or mining pool nodes) receive the relayed transaction.  They put the transaction into the block they are currently working on if they wish to do so.

Questions in step three:

• Who/what holds information about the current block? (everyone?)

If you are solo mining, you hold the information about the current block that you are working on. If you are mining in a pool, then the pool operator holds all the information for the block that all the pool members are working on.

• I understand what hashing is used for (save space mostly), but why/and how is it used to verify transactions? I do not see how hashing the block header creates a "proof of work" of all transactions...

Finding a hash value smaller than the current target requires an amount of work.  Providing that hash proves that you did that work.  The target is adjusted on a regular basis to ensure that the amount of work needed to find the target hash requires the entire hashing power of all the miners on the network for approximately 10 minutes.  The hash acts as proof that the appropriate amount of work was done for the indicated block of data.  If anyone tries to modify any data in the block they will end up with a different hash.  To modify the block and have nodes running the protocol accept it, you would have to repeat the same amount of work that was required to complete the original block.  Since each new block contains the hash of the previous block in it's header, you can't change the data in any older block without having to repeat all the work that was necessary for every block since the block you are trying to replace.  Meanwhile the honest network is continuing to create new blocks. This means that any malicious attacker would have to have more hashing power than the entire combined honest network to ever catch up with the current blockchain.

• The new 25BTC comes from where? (they are released for use by ...), along with 25BTC the winner also gets any mess ups from the previous transaction right? (say someone forgets to gives themselves their "change")?

The new 25BTC comes from nowhere.  Every block contains a list of transactions.  The protocol requires exactly one transaction in every block that has no inputs. It requires this to be the first transaction in the block.  This outputs of this transaction are not allowed to exceed the current block subsidy value (25 BTC) plus the total of all transaction fees.  The transaction fees of each transaction are calculated as the difference between the total inputs and the total outputs, so yes, if you don't include your change in an output, then the change becomes a transaction fee for the miner.

~~Overall questions~~

• How do we get from however many possible millions of transactions to a "block"?
  :::: I figure a "block" just holds a linked list of transaction objects, whenever a new transaction is created it is just appeneded to this list. Then when someone starts to "mine" this block he/she is actually looking for the shortest spanning tree over all transactions if they were in a graph form similar to this picture (https://en.bitcoin.it/wiki/File:Transaction.png)?

Until a miner includes a transaction in a block it exists only in the memory pool of the various nodes on the network that are relaying it.  Any miner can choose to include any valid transactions they would like from their memory pool in the next block they choose to mine.

• What is the proof that we need, what are we actually computing when we are hashing the block header?

The proof is that you did the work necessary to find a hash value below the target.  Accomplishing this task requires a certain amount of work, and replacing or modifying an existing block in the blockchain would require an attacker to be able to supply more hashing power than the entire combined honest network.

• What happens when a fradulent transaction is encoutnered?

I suppose that depends on what you mean by "fraudulent", but in general the nodes will refuse to relay it and the miners will refuse to include it in a block.  If a miner includes it in a block, then all other honest nodes on the network will refuse to accept that block and will continue looking for a block that does not include the "fraudulent" transaction.

• How can we tell a transaction is fradulent?

It doesn't meet the requirements of being a valid transaction. Some of which are:

• All inputs must exist in a valid previous transaction
• All inputs must not be included as inputs in any other transaction
• All inputs must have a signature that matches the public key included in the input
• The public key included in the input must match the bitcoin address of the previous output being used as an input
• The total value of inputs must be greater than or equal to the total value of outputs

• Say two users wish to exchange bitcoins, user 1 gives user 2 10BTC. User 1 tells everyone "I gave 10 BTC to user 2, allow him to use 10BTC on all future transactions". On the other hand user two might try to say "User 1 gave my 1000BTC for me to use" How do we tell who is lying (suppose user 1 has over 1000BTC)?

User 2 does not have the private key that is associated with any of the bitcoin addresses that User 1 owns.  Therefore User 2 is unable to provide the necessary signature to include any of the outputs associated with the addresses owned by User 1 in any transaction that would use those outputs as inputs.

[skeeterskeeter]

Ok, after some more reading and this post some thing are clearer but I am still left with a few questions.

The validity of the BTC system rest on the idea that no one person can perform the work that we have all performed so far creating the BTC system. They would first need the computing power equal to all of us, and then the same amount of time from the transaction they wish to change to now to compute the hashes and submit the new blocks to everyone. This is semi-impossible unless someone finds a polynomial time algorithm for finding hashes, or has a crap-ola lot of money to make the computing power necessary to do such a thing.

OK.

So this means that the more work we all do the more secure the BTC system is. So we never want to do the same work someone else has done that would be asinine. This means that each hash someone performs is unique. This must mean that the nonce is unique to each hash. This must mean the nonce for the current block is global to all nodes. This must mean each time someone wants to try to "mine" the current block they have exclusive access to use the current nonce and if someone else wants to mine they have to take the next nonce value and work with it.

This is the lottery system in it all. Each nonce value is like a scratch off for 25BTC. It cost nothing to buy the lottery ticket though! But you have to scratch a crap ton of tickets to ever get any money.


OK.

So in general is it a rule in hashing that it is difficult to find a small hash value? I read 1 in 2^20 hashes will have 20 leading zero's.

Who/what states what the current target hash is?
Who/what states how hard it is to find the current proper hash?

[DannyHamilton]

. . . This means that each hash someone performs is unique. This must mean that the nonce is unique to each hash. This must mean the nonce for the current block is global to all nodes. This must mean each time someone wants to try to "mine" the current block they have exclusive access to use the current nonce and if someone else wants to mine they have to take the next nonce value and work with it . . .

This isn't quite true.  Each miner (or pool) has a different block header that they are trying to hash, since the block includes the transaction that pays out the block subsidy and each miner (or pool) has a different payment address.  This means that 2 different miners can use the same nonce on the block that each of them is working on and generate completely different hashes.  When a miner happens to come upon a hash that is of low enough value, they relay the block with the target hash that they've found, and everyone else can confirm that the block contains a qualifying hash by comparing against the target and then hashing the block that they are relaying and checking to see if the hash value matches the one stored in the new block.

So in general is it a rule in hashing that it is difficult to find a small hash value? I read 1 in 2^20 hashes will have 20 leading zero's.

Yes, in general it is understood that the results of a hash are unpredictable and therefore essentially random.  If I generate a random number between 1 and 10, there is a 90% chance that the random number will be less than 10.  There is a 50% chance that the random number will be less than 6. There is a 10% chance that the number will be less than 2. In the case of block hashes, the hashing algorithm ends up with a random number between 0 and 2^256 (between 0 and 1.158 x 10⁷⁷).

Who/what states what the current target hash is?

The protocol that every node runs has a system built in to the code that adjusts the expected target every 2,016 blocks.  If a new block received does not have a hash below this target, then none of the honest nodes on the network will accept or relay it.

Who/what states how hard it is to find the current proper hash?

The software running on every node and every miner compares the hash to the expected difficulty that they have independently calculated based on the publicly available blocks that are currently part of the blockchain.

[skeeterskeeter]

Essentially this means that no two miners could perform duplicate work then, and all work added no matter how small goes into foiling dishonest nodes.

 *** Dishonest nodes are ones with crap transactions, which are found in the ways previously listed. Thought: If one node were to not find a bad transaction though it existed in their block would the power of the other honest nodes create a "better" hash than them then? Meaning ALL bad transactions will ALWAYS be dropped from the block chain if indeed we all follow the hardest working hasher? ***

Though duplicate work could be performed by two miners working on paying out to the same wallet, such the case in pools and multi-threading. Which good, we can do it faster then!

As if each node is its own economy (Cool!) and within each node there is another almost government transposed on top of the BTC system that deduces how each miner in there gets payed out. And it all goes back to all economies following the best economy, the one with the most computing power.

-----------------------------------------------------------------------------------------------------------------------------------------------

So if I have a bitcoin wallet and I make a transaction. It gets propagated throughout each node, and once...

• a node receives enough transactions it starts to compute the next block. (Blocks created ASAP throughout entire network, no lag other than when nodes after the initial node start computing; lag is specific to your path length to the initial node)
• everyone gets enough transactions we all can start computing after some signal we all agree on. (Blocks created on average related to the average network communication time, no lag between nodes when race starts)
• someone/thing notifies nodes with some sort of information about the next block and makes it possible to start computing the block. (Blocks created in time intervals set by whoever sends info; user with most transactions get to start the computing, nodes after this experience lag the same as the first^)
  

The last means we still have to send the info to the to all nodes and whoever gets it first can peek before others get it and start first so I don't think that is the case. Now that I said that number two is out because we still have to signal all nodes and set some time to start computing that is fair. (two generals paradox?http://en.wikipedia.org/wiki/Two_Generals%27_Problem)

So either there is a set size when someone can start and thus who ever makes the most transactions ( + received possible transactions) has the edge to getting to start the next block, or I haven't thought of something.

-----------------------------------------------------------------------------------------------------------------------------------------------

Hopefully my last few questions...
- So how exactly is the new target hash generated between the nodes, is there a technical page I could read?
- And how exactly does "The software running on every node and every miner compares the hash to the expected difficulty that they have independently calculated based on the publicly available blocks that are currently part of the blockchain", again any technical pages you know?
Unless of course you know well enough to explain it, but it sounds it might be a little complicated to say eloquently on a forum post.

[DannyHamilton]

So if I have a bitcoin wallet and I make a transaction. It gets propagated throughout each node, and once...

• a node receives enough transactions it starts to compute the next block. (Blocks created ASAP throughout entire network, no lag other than when nodes after the initial node start computing; lag is specific to your path length to the initial node)

No, miners are ALWAYS working on the next block.  They don't wait for "enough transactions".  They just work with whatever transactions currently exist in the memory pool that are not yet in a block.

• everyone gets enough transactions we all can start computing after some signal we all agree on. (Blocks created on average related to the average network communication time, no lag between nodes when race starts)

The race starts as soon as the previous block is received.  This means that the person who computed the previous block actually has a very slight headstart on the new block, and the last person to hear about the recently completed block has lost some time that they could have been working on the new block.  This acts as an incentive for miners and pools to attempt to be well connected to many fast nodes, allowing them to hear about any solved block as soon as possible.  Additionally, if someone else solves and broadcasts a block before you can broadcast your own solved block, each node on the network will choose the first valid block that it hears about, adding further incentive to a miner to be well connected to propagate any block they solve as fast as possible, and eliminating the incentive to hold on to the current solved block while getting a head start on the next block.
 

• someone/thing notifies nodes with some sort of information about the next block and makes it possible to start computing the block. (Blocks created in time intervals set by whoever sends info; user with most transactions get to start the computing, nodes after this experience lag the same as the first^)
  

Not sure what you are saying here.  The receipt of a new valid block is the signal to immediately begin working on the next block.  The time interval to find a block is random, but the target hash value is such that the long term average works out to be about 10 minutes per block.

The last means we still have to send the info to the to all nodes and whoever gets it first can peek before others get it and start first so I don't think that is the case. Now that I said that number two is out because we still have to signal all nodes and set some time to start computing that is fair. (two generals paradox?http://en.wikipedia.org/wiki/Two_Generals%27_Problem)

Receipt of a completed block is a signal to start working on the next block.  If you don't receive the current completed block, then you continue to work on the wrong block.  Eventually when your communications improves, you hear about the recently completed block from a peer, and give up on the one you were working on and move to the next. Every node already has all the information it needs to work on a block. It needs to know what the hash value was of the most recently completed block, and it needs a list of transactions in its memory pool that it can put into the next block that it works on.

So either there is a set size when someone can start and thus who ever makes the most transactions ( + received possible transactions) has the edge to getting to start the next block, or I haven't thought of something.

There is no set size.  As a matter of fact, it is entirely valid to create a block that has ONLY the one transaction that pays the block subsidy to the miner and no other transactions at all.  If the node happens to have some transactions in the memory pool, then this is a foolish thing to do, since you would be passing up on the opportunity to collect the transaction fees from those transactions.

Hopefully my last few questions...
- So how exactly is the new target hash generated between the nodes, is there a technical page I could read?

Every node generates the new target independently.  Since they are all working from the same blockchain as the input to their calculations, they all compute the same target. Is the following link enough detail to answer your question, or would you prefer a link to the actual function that runs in every node?
https://en.bitcoin.it/wiki/Difficulty

The difficulty is adjusted every 2016 blocks based on the time it took to find the previous 2016 blocks. At the desired rate of one block each 10 minutes, 2016 blocks would take exactly two weeks to find. If the previous 2016 blocks took more than two weeks to find, the difficulty is reduced. If they took less than two weeks, the difficulty is increased. The change in difficulty is in proportion to the amount of time over or under two weeks the previous 2016 blocks took to find.

- And how exactly does "The software running on every node and every miner compares the hash to the expected difficulty that they have independently calculated based on the publicly available blocks that are currently part of the blockchain", again any technical pages you know?

The answer to this is the same as the previous question.

[skeeterskeeter]

Last question for the day!

So if the current pool I am in has a combined hashrate of 2000GH/s and the current difficulty is 3275464.5865656 then it will take an average of

(from the bitcoin difficulty wiki)

time = difficulty * 2^32 / hashrate
       = 3275464.5865656 * 2^32 / 2147483648000
       = 6550

This is in minutes? Average of 109 hours to find a solution at this rate?






I am successfully mining! Hopefully I will get my first bitcoins here soon!

If anyone feels generous [1Hbz4VoBfim6CK6c3KFzV397qbp2mGTmnZ] 



**Shoot me a address Danny and I'll see if I can send you some in the week to come!**

[DannyHamilton]

So if the current pool I am in has a combined hashrate of 2000GH/s and the current difficulty is 3275464.5865656 then it will take an average of

(from the bitcoin difficulty wiki)

time = difficulty * 2^32 / hashrate
       = 3275464.5865656 * 2^32 / 2147483648000
       = 6550

This is in minutes? Average of 109 hours to find a solution at this rate?

I could be mistaken, but I believe that is in seconds.
So, 6550/60 = approximately 109 minutes, and 109/60 = approximately 1.8 hours.