Does the hash power of miners who do not discover a block used for anything?

[semaforo]

 Been into bitcoin for a while, still trying to understand the fundamental mechanism of who exactly is responsible for moving transactions from the mempool to the block. Is it only the person who discovers the block header? Does the rest of the networks hash power actually go towards verifying transactions? So every block header requires every transaction ever to be the same right... but how do new transactions figure into the new block headers? Do they figure in at all?

I also read that for a miner to "win" a block, they simply have to get a value lower than the target. Does this mean there are a range of correct hashes? All below a certain value?

[1714659016]

1714659016

[1714659016]

1714659016

[1714659016]

1714659016

[dothebeats]

In generating a block, each miner competes to get the correct answer on a certain puzzle. For example, Joe, Chris and Tim each received a puzzle. In order to get their prize (or reward), they need to solve a puzzle based from the previous puzzle that was already solved. Let's say Joe can guess 10 times in a minute; Chris can do 13 and Tim can do 8. On paper, Chris will solve the puzzle faster than the two, but remember that not all guesses are always correct. Tim got the correct answer and was rewarded for it, and also made his answer the basis for the next puzzle while all of Joe and Chris' guesses were voided.

So, long story short, Chris and Joe's effort didn't amount to any reward but was able to help maintain the integrity of the guessing game by submitting all their guesses by the rules.

[semaforo]

In generating a block, each miner competes to get the correct answer on a certain puzzle. For example, Joe, Chris and Tim each received a puzzle. In order to get their prize (or reward), they need to solve a puzzle based from the previous puzzle that was already solved. Let's say Joe can guess 10 times in a minute; Chris can do 13 and Tim can do 8. On paper, Chris will solve the puzzle faster than the two, but remember that not all guesses are always correct. Tim got the correct answer and was rewarded for it, and also made his answer the basis for the next puzzle while all of Joe and Chris' guesses were voided.

So, long story short, Chris and Joe's effort didn't amount to any reward but was able to help maintain the integrity of the guessing game by submitting all their guesses by the rules.

I need more details. I need to know specifically what is the relationship between the hashes of new transactions and the puzzle. How is the hash of a new transaction determined? Does the hash of new transactions affect the block header, or is there a seperate block hash?

[codewench]

I need more details. I need to know specifically what is the relationship between the hashes of new transactions and the puzzle. How is the hash of a new transaction determined? Does the hash of new transactions affect the block header, or is there a seperate block hash?

You've probably heard that credit card numbers and other such numbers often have a check digit to detect errors. Sometimes the check value is the hash of all the other values. Sometimes the check value is chosen such that the hash of all values (including the check) equals some specified value. Bitcoin is like the latter.

For some applications, the hash function could be simple addition. In that case it is possible to work backwards from the desired hash to the needed check value. With Bitcoin, the function is too hard to run backwards. The only possibility is to try many many check values to achieve the desired hash. This is what all the miners are doing. The other difference is that Bitcoin considers many (but still a tiny percentage of) possible block hashes to be valid. This percentage is changed to adjust difficulty.

When transactions are assembled into a block, they are all ultimately hashed together (in stages), so that the block hash protects everything from being changed. The block hash itself isn't stored - other nodes calculate it to see if it is one of the valid values - but the check value (aka nonce) is stored in the block header.

[semaforo]

Okay, I see. So actually the transactions to be included in a new block do not affect the value of the new block header? But all of the previous transactions, the ones already recorded in blocks, do.

So really miners are not verifying any new transactions? Once the block header is "discovered" all the transactions included in it is just extra work that only the miner who won the race does.

But then the other miners have to verify the transaction, presumably because every miner must run a full node?

[codewench]

Okay, I see. So actually the transactions to be included in a new block do not affect the value of the new block header? But all of the previous transactions, the ones already recorded in blocks, do.

All the transactions in a block do affect their block's header. That's how the header is able to lock all the transactions from changes. (There's also a hash of the previous block keeping it from being changed.)

So really miners are not verifying any new transactions? Once the block header is "discovered" all the transactions included in it is just extra work that only the miner who won the race does.

No, all miners are filling their prospective block with transactions. They have to do this before they can start searching for the magic number that will make the header hash correctly. The block header "discovery" is the last thing that happens. At that point the winner yells BINGO! and the losers say "Awe shucks!" and everyone starts over looking for the following block. The losers will have done a bunch of work and don't get anything for it.

[semaforo]

Okay, I understand that the transactions in the block affect the header, so the value that is being searched for is not the header?

  So all of the miners are taking transactions and including them in blocks, and then forming those transactions into a value that is then calculated to see if it is the winning value?

[DARKANGEL6415]

I am not sure if you are referring to home miners or something similar where you are doing it yourself. Keep in mind any mining you are doing is adding to the global hash rate for bitcoin or anything else you may be mining.  Now as to weather or not you will find a block is a whole different story. That is why i like being in mining pools so i atleast get credit for something instead of me trying to find a block by myself despite the fact the rewards for a solo block would be great.

[semaforo]

No, I am trying to figure out exactly what the 'winning value' is made out of. Specificially because I want to know what happens if you create a consensus protocol with a "gateway" where there is a cap on hashpower and miners compete by means of another value for the "seats." Meaning there is a big pool of miners waiting in a queue for the chance to mine, the protocol selects ten based on weighted values, and those ten then compete for a low difficulty solution. This cycle would repeat with each block, maybe with two or three minute blocks.

[odolvlobo]

No, I am trying to figure out exactly what the 'winning value' is made out of.

Every 2016 blocks, the target value is changed based on how long it took to mine the last 2016 blocks. The desired time is 14 days. In order for a block to be added to the chain, all the transactions in the block must be valid and the hash for the block (got from hashing the block's header) must be less than the target value.

Specificially because I want to know what happens if you create a consensus protocol with a "gateway" where there is a cap on hashpower and miners compete by means of another value for the "seats." Meaning there is a big pool of miners waiting in a queue for the chance to mine, the protocol selects ten based on weighted values, and those ten then compete for a low difficulty solution. This cycle would repeat with each block, maybe with two or three minute blocks.

It might be possible to do, but keep in mind that Bitcoin's proof-of-work is already a way for miners to compete in a decentralized manner. There are other methods. For example, you could use proof-of-stake to limit the pool of miners that you use proof=of-work.

[semaforo]

Every 2016 blocks, the target value is changed based on how long it took to mine the last 2016 blocks. The desired time is 14 days. In order for a block to be added to the chain, all the transactions in the block must be valid and the hash for the block (got from hashing the block's header) must be less than the target value.

So all the block reward is not awarded until the block is fully formed and accepted by... a node?

It might be possible to do, but keep in mind that Bitcoin's proof-of-work is already a way for miners to compete in a decentralized manner. There are other methods. For example, you could use proof-of-stake to limit the pool of miners that you use proof=of-work.

As I understand it, difficulty scaling is a way to prevent attacks on the network. Meaning, if someone wanted to attempt a 51% attack and bought a mass of hardware, difficulty would scale up, meaning their initial hardeare outlay would no longer be enough to reach 51%.

With proof of stake the presumption is that miners will not risk losing a large amount of coins for a small profit by behaving dishonestly. Several problems with this imo. One is the economic value of destroying a chain when you have inter chain competition may be much bigger than calculations internal to a chain. PoS as far as I know also allows free access to the mining process, effectively substituting hash power for investment in the network. Both of these solutions are monetary- you gain influence in the network by means of mining rigs or by either purchasing coins or attracting users and speculators to a new network, which can also be done with money.

  I think steem is the first network that uses behavior or uses the issuance of the currency as an incentive for a certain type of behavior- in this case, generating blog content.

So my question is really is if both the number of miners and the hashpower per miner is restricted by an additional layer on the protocol but still based on a proof of work model, is this going to affect the functionality of the network?

[odolvlobo]

So all the block reward is not awarded until the block is fully formed and accepted by... a node?

Yes, the block reward is done by adding a special transaction called the coinbase transaction to the block. Your use of "fully-formed" seems to indicate that you still don't understand how it works.

Each miner gathers all the unconfirmed transactions that it knows about (the ones in its "mempool") and constructs a block. Then the miner repeatedly hashes the block's header with a different nonce each time until (A) it gets a hash whose value is less than the target, or (B) until it finds out that another miner has already added a block.

If (A), then the miner publishes the block and everyone appends the block to their copy of the chain and starts on the next block.
If (B), then the miner gives up, adds the new block to their chain, removes transactions in the block from the mempool, and starts on the next block.

As I understand it, difficulty scaling is a way to prevent attacks on the network. Meaning, if someone wanted to attempt a 51% attack and bought a mass of hardware, difficulty would scale up, meaning their initial hardware outlay would no longer be enough to reach 51%.

The sole purpose of adjusting the difficulty is to maintain an average block interval of 10 minutes. It can be adjusted either up or down. If an attacker has 51% of the hash power, then they have 51% of the hash power regardless of the difficulty.

[codewench]

So all the block reward is not awarded until the block is fully formed and accepted by... a node?

The miner preemptively includes a payment to themselves in the block, i.e. nobody gives them the payment, they take it for themselves. If they don't solve that block, or if that block isn't accepted into the main fork, then they can't spend that payment.

No, I am trying to figure out exactly what the 'winning value' is made out of.

Lets say a miner mining ExampleCoin assembled this block header (version, previous block's hash, this blocks transaction's hash, time, difficulty, nonce):

(4) (90) (42) (17) (10) (x)

In ExampleCoin, the hash is simply addition modulo 100, and the header hash must be a multiple of the difficulty. It is easy to see that a value of 7 for x produces a hash of 70, which is a multiple of 10. Thus, the miner broadcasts this header:

(4) (90) (42) (17) (10) (7)

Lets assume the difficulty adjusts up a little to a multiple of 11, so now the next block's header is:

(4) (70) (66) (18) (11) (x)

In Bitcoin the hash is much more complicated than addition - we can't work it backwards so x has to be found by trial and error.

Specificially because I want to know what happens if you create a consensus protocol with a "gateway" where there is a cap on hashpower and miners compete by means of another value for the "seats." Meaning there is a big pool of miners waiting in a queue for the chance to mine, the protocol selects ten based on weighted values, and those ten then compete for a low difficulty solution.

Who chooses the selected miners? How do you keep the system from being gamed?

[semaforo]

First off thanks for the patience and the explanations, I think there is progress being made. If I had ever actually been a miner it would probably make more sense to me.

I think a big part of the confusion for me is the existence of empty blocks containing no transactions, or consisting only of the coinbase transaction. This means that that the only incentive for including transactions in a block is the fees, which is in addition to the block reward?

Who chooses the selected miners? How do you keep the system from being gamed?

  The selected miners are chosen randomly according to their network reputation, determined by network behavior. Meaning the higher the reputation, the more likely they are to be selected.

The system is kept from being gamed by algorithms that freeze addresses engaging in suspicious behavior, for example lots of transactions to addresses outside of the main cluster of the network graph. Similar to NEM's PoI, but without the PoS element. Also transactions are weighted according to whom the transactions are made with, ie. dealing with newb accounts has lower weight.

The problem with these algorithms being that they will freeze innocent people. This could be solved by an auditing process whereby frozen addresses can appeal and be verified by members of the network with a high enough reputation. The verification itself would increase reputation for everyone involved, while fraudulent behavior in the future by the address being audited would lower the reputation of everyone involved in the previous audit.

  This is the reason I need to know if having a totally open network is somehow essential to the functionality of the network, like verification speed.

  What other purposes does having a high difficulty serve, then, other than just keeping the block time steady?