WTF is this? Someone found a trick for fast mining?

[becoin]

Either:

• You are wrong, and everyone else is right
• You are right, and everyone else is wrong
• Someone has launched a Sybil attack against you

Or...

Someone has launched a Sybil attack against gmaxell's fan club.

[altcoinex]

valiron,

Either:

• You are wrong, and everyone else is right
• You are right, and everyone else is wrong
• Someone has launched a Sybil attack against you

It takes an enlightened individual to conclude that they are likely in err despite that they may not realize why. If 10 people I respect tell me I am wrong, I have to accept that conclusion even if I do not understand why.

Of course, I would seek to understand why I am wrong, but it is not the onus of these other 10 people to be my teacher. (The fact that gmaxwell has voluntarily chosen to try to teach you, and has shown so much patience in the process, certainly says something about his character.)

I mean no disrespect. I only hope that you can be this type of enlightened individual.

As this forum doesn't run on blockchain technology, we cant be sure large corporations or nation state governments havn't affected the data that has gone into this chart.

[btchris]

• [or] Someone has launched a Sybil attack against you [valiron]

Or...

Someone has launched a Sybil attack against gmaxell's fan club.

I don't understand.... I was trying to imply "we're all alts of of gmaxwell" as being the third option (however unlikely IMO).

As this forum doesn't run on blockchain technology, we cant be sure large corporations or nation state governments havn't affected the data that has gone into this chart.

Into my "chart"? I'd rather think that if a government wanted to pollute this thread with puppets, they'd all be in support of valiron's FUD, not against it.

[becoin]

If 10 people I respect tell me I am wrong, I have to accept that conclusion even if I do not understand why.

Couple of years ago when I said BFL are crooks and low life 10 people I respected (incl. mods) told me I was wrong. They were just bribed by BFL to do so. At that time mods were not in a hurry to label BFL as scammers.

If we brush off all the pseudo scientific gibberish what gmaxell is basically saying we should treat all the irregularities as regular because of the big figures. Do we have to accept this knowledgeable explanation and move on just for the sake of protecting bitcoin from possible FUD?

[valiron]

Scanning the blockchain for nearby consecutive nonces we find 1 more ocurrence (starting at block 323246) of three consecutive closeness of less than 1.8% and 0.12% and 7.2% among the 140411 post 2013 blocks. The expected value is to not find any 4 out of 5 times since 144411*0.0012*0.018*0.072=0.218.

Now we relax the search and look for only 2 consecutive closeness. Scanning the blockchain for nearby consecutive nounces we find 15 ocurrences of two consecutive closeness of less than 1.8% and 0.12% among the 140411 post 2013 blocks. The expected value is 140411*0.0012*0.018=3, thus there are 5 times more occurrences than expected.

Some of these occurrences like to be abnormally near 2^30. For example:

Block 237144 nonce 1076288193
Block 237145 nonce  999124565
Block 237146 nonce  997353929

|nonce(237144)-nonce(2372145)|= 1.8%   of total range
|nonce(237145)-nonce(2372146)|= 0.04% of total range

and....nonce(237144) is within 0.059% of total range from 2^30=1073741824
      ...nonce(237145) is within 1.74%  of total range from 2^30
      ...nonce(237146) is within 1.78%  of total range from 2^30



Block 345506 nonce 1121422642
Block 345507 nonce 1050751563
Block 345508 nonce 1052220753

|nonce(345506)-nonce(345507)|= 1.6%   of total range
|nonce(345507)-nonce(345508)|= 0.03% of total range

and....nonce(345506) is within 1.11% of total range from 2^30=1073741824
      ...nonce(345507) is within 0.53%  of total range from 2^30
      ...nonce(345508) is within 0.50%  of total range from 2^30

[valiron]

If 10 people I respect tell me I am wrong, I have to accept that conclusion even if I do not understand why.

Couple of years ago when I said BFL are crooks and low life 10 people I respected (incl. mods) told me I was wrong. They were just bribed by BFL to do so. At that time mods were not in a hurry to label BFL as scammers.

If we brush off all the pseudo scientific gibberish what gmaxell is basically saying we should treat all the irregularities as regular because of the big figures. Do we have to accept this knowledgeable explanation and move on just for the sake of protecting bitcoin from possible FUD?

Bribed mods?  

I mean no disrespect. I only hope that you can be this type of enlightened individual.

Sorry, but I can't agree. Enlightened individual is whoever thinks by himself, and not who follows blindly a leader or guru.

I believe the proper thing to do is to have a brain and think by yourself.

"The majority is always wrong; the minority is rarely right." Henrik Ibsen.

"To doubt everything or to believe everything are two equally convenient solutions; both dispense with the necessity of reflection." Henry Poincaré

The data is there for anyone to analyze and reach conclusions if these are normal statistical anomalies or far out of normal.

Statistics are tricky and one needs to have some intuition and expertise about it to not confuse normal rare statistical deviations from irregular ones. Irregular ones are detected because of coincidence of various independent indications.

Someone can come and claim that he found a collision with a bitcoin address in the blockchain. We know that mathematically speaking this is imposible and we may call this guy a liar and a fool. But we know that defective implementations of wallets with not enough entropy can indeed duplicate addresses.

I don't want to enter into the question if gmaxwell or I are right or wrong. This always deviates into a stupid fight of egos. The data is there and everyone can reach its own conclusions about the possibilities. No hardfeelings. No stress. Don't need to get your blood boiling.And finally...if you don’t believe me or don’t get it, I don’t have time to try to convince you, sorry.  

[btchris]

If 10 people I respect tell me I am wrong, I have to accept that conclusion even if I do not understand why.

Couple of years ago when I said BFL are crooks and low life 10 people I respected (incl. mods) told me I was wrong. They were just bribed by BFL to do so.

That's a good first step, you've just admitted that you are capable, like the rest of us, of making mistakes (nobody should be respecting 10 people all of whom are easily bribed).

If we brush off all the pseudo scientific gibberish what gmaxell is basically saying we should treat all the irregularities as regular because of the big figures. Do we have to accept this knowledgeable explanation and move on

Absolutely not! However your rational are choices are limited. As with any argument based in science or logic, you can either:

• Educate/fund yourself to the point where you can make investigations and an assessment on your own, or
• Ask someone you trust who has the relevant education/funding for their opinion.

I have never, not once, been into space. Yet I still believe there is (practically) no air in space. This is mostly as a result of the second choice above.

When it comes to this particular thread, I have enough of a background in math to easily follow gmaxwell's reasoning, and to largely dismiss valiron's.

The larger problem is the hubris which valiron seems to be displaying. If valiron were a troll, this would be expected. If valiron is simply having trouble following gmaxwell's reasoning, s/he should take a moment to consider that maybe s/he wrong, given the number of opponents there are.

[altcoinex]

Is it just me, or is it silly to consider the nonce without considering the rest of the data in the block header?

Edit: Perhaps OP could provide detail on the merkleroot and transactions of the blocks in question.... I mean, if they were empty blocks it would indeed be suspect, but to produce a valid nonce and still manage to include a solid transaction set from the pool, come on now....

[valiron]

Is it just me, or is it silly to consider the nonce without considering the rest of the data in the block header?

Edit: Perhaps OP could provide detail on the merkleroot and transactions of the blocks in question.... I mean, if they were empty blocks it would indeed be suspect, but to produce a valid nonce and still manage to include a solid transaction set from the pool, come on now....

These blocks are not empty, empty blocks are another business. You don't fabricate the nonce from the rest of the block, these nonces could be not the main dynamic variable for the purpose of mining, and you may start testing nonces starting from some nonce with many zero bits for some reason.

[altcoinex]

Is it just me, or is it silly to consider the nonce without considering the rest of the data in the block header?

Edit: Perhaps OP could provide detail on the merkleroot and transactions of the blocks in question.... I mean, if they were empty blocks it would indeed be suspect, but to produce a valid nonce and still manage to include a solid transaction set from the pool, come on now....

These blocks are not empty, empty blocks are another business. You don't fabricate the nonce from the rest of the block, these nonces could be not the main dynamic variable for the purpose of mining, and you may start testing nonces starting from some nonce with many zero bits for some reason.

But you DO fabricate the block hash from the Block Header, which includes the other data in addition to the nonce. It is this hash, not a hash of the nonce itself that has to beat the Target. I am not sure if you understand how hashing works to suggest that with entirely different block header data a similar nonce could be exploiting the end hash.

[valiron]

Is it just me, or is it silly to consider the nonce without considering the rest of the data in the block header?

Edit: Perhaps OP could provide detail on the merkleroot and transactions of the blocks in question.... I mean, if they were empty blocks it would indeed be suspect, but to produce a valid nonce and still manage to include a solid transaction set from the pool, come on now....

These blocks are not empty, empty blocks are another business. You don't fabricate the nonce from the rest of the block, these nonces could be not the main dynamic variable for the purpose of mining, and you may start testing nonces starting from some nonce with many zero bits for some reason.

But you DO fabricate the block hash from the Block Header, which includes the other data in addition to the nonce. It is this hash, not a hash of the nonce itself that has to beat the Target. I am not sure if you understand how hashing works to suggest that with entirely different block header data a similar nonce could be exploiting the end hash.

I don't think you understand my previous message.
What I am saying is that a reason to have these nonces near some value is that you don't change too much the nonce and you start from this value, and for solving the block you change other fields.

[DannyHamilton]

- snip -
for solving the block you change other fields.

There aren't very many other fields to change.  That's why the block has a nonce in the first place.  There's nothing special or magical about changing the nonce, it's just 4 bytes added to the header specifically for the purpose of being very fast and easy to change.

The other fields of the block header are:

• The block version number. This 4 byte field cannot be changed without invalidating the block you are mining.
• The hash of the previous block in the chain. This 32 byte field cannot be changed without invalidating the block you are mining.
• The merkle root of the transaction list. This 32 byte field cannot be changed without modifying or rearranging the transactions in your transaction list.  This would be a MUCH slower and more complicated thing to change than the nonce. However, a pool will create MANY different merkle roots (by modifying the extranonce in the coinbase transaction) so that they can give a different block header to each miner.  This allows each individual miner to only need to deal with the nonce (and timestamp) and not need to recompute the entire merkle root.  It is also possible for the pool to provide enough information for individual miners to modify the extranonce themselves along with the nonce.  Keep in mind though that modfying the extranonce increases the time and effort involved, so it makes more sense to exhaust the nonce range available to the equipment before making any effort to move on to a new extranonce.
• The timestamp.  This is a 4 byte field that some miners change after they've run out of nonces.  It essentially becomes a secondary nonce. However, it is very limited on allowable ranges.  Since the miner would need to test to make sure a value is within the allowable range before hashing the block, it very slightly increases the effort as compared to simply using the usual nonce.  If someone chooses to modify this instead of the nonce (or while keeping the nonce in a tight range), then this field BECOMES a nonce.  There is no benefit of manipulating these 4 bytes instead of the other 4 bytes.
• The current difficulty (represented as "bits"). This 4 byte field cannot be changed without invalidating the block you are mining.
• The nonce. This 4 byte field exists solely so that is is fast and easy to modify the header before hashing it again.  It serves no other purpose, and has no restrictions on its value.  This is also the field that you are saying NOT to change when "you change other fields".

That's 4 bytes + 32 bytes + 32 bytes + 4 bytes + 4 bytes + 4 bytes = 80 bytes.

That's it.  There's the 80 bytes that are hashed during mining.  I'm very curious to know which of those fields you think might be better to change instead of the nonce?

Those in red cannot be changed without invalidating the block.  Those in green are already manipulated as part of the typical mining process, and the typical mining process already manipulates them in the most efficient order.  If there is a benefit to manipulating the timestamp or merkle root INSTEAD of the nonce, I'd be very curious to know what you think that benefit might be.

I'm not certain, but I don't think you can adjust the merkle root without needing to recompute the midstate as well?

[btchris]

I'm not certain, but I don't think you can adjust the merkle root without needing to recompute the midstate as well?

You're right, the merkle root spans both SHA-256 (512-bit) blocks, so you must recalc the entire SHA-256 from scratch.

Byte len	Byte pos	Bit pos	SHA block	Field
4	0	0	1	version
32	4	32	1	previous block header hash
32	36	288	1-2	merkle roothash
4	68	544	2	time
4	72	576	2	nBits
4	76	608	2	nonce
	80	640		(end)

[timothythomas]

i have always wondered about this thing also ... like i wanted to ask about pools how f2pool is always finding more blocks then antpool and others .. is it merely coz of it having a more hash power ? or its just being lucky ?

[smolen]

The block version number. This 4 byte field cannot be changed without invalidating the block you are mining.

Hmm, this 32 bit signed integer must be greater or equal to the current block version. Plenty of space here. Discussed on github: No forking Extra nonce added to Bitcoin header

The merkle root of the transaction list. This 32 byte field cannot be changed without modifying or rearranging the transactions in your transaction list.  This would be a MUCH slower and more complicated thing to change than the nonce.

Swapping nodes in Merkle tree seems to be cheaper than increasing extranonce2, but worse than increasing nonce or time.

[gmaxwell]

Hmm, this 32 bit signed integer must be greater or equal to the current block version. Plenty of space here. Discussed on github: No forking Extra nonce added to Bitcoin header

I believe Danny's point was that it invalidated the midstate to change it.

[smolen]

Hmm, this 32 bit signed integer must be greater or equal to the current block version. Plenty of space here. Discussed on github: No forking Extra nonce added to Bitcoin header

I believe Danny's point was that it invalidated the midstate to change it.

Oh, wait, I'm not going to prove someone wrong. This thread become quite nice mix of math and conspirology, that's very fine, let's continue
If there exist some algebraic attack that breaks two rounds of SHA256 (and OP is hunting for it with statistics), it's quite possible that the variant of such attack will bypass midstate and break 3 rounds, from 640 bit header to nonce. So the next suspicious thing are unusual block versions in the blockchain.

[2112]

quite nice mix of math and conspirology, that's very fine, let's continue

I'm not satisfied with calling the psychological aspect "conspirology".

My current thinking is that many people here can't really understand either mathematical or software engineering aspects of Bitcoin. For them it becomes a sort of gnostic theological experience, even if they claim to be atheistic.

In this thread they will search for a (bad) influence of demiurge who corrupts their ideal.

In the nearby threads by no-ice-please we observe a dissatisfaction with insufficiently immaculate conception of SHA256 algorithms.

I hate to call them all "trolls". I can settle for "crackpots", because this seems to be the prevailing practice in the scientific fields.

But theologians would simply call them "heretics", purveyors of the "bad theology".

In very broad term, what we are searching in those threads is a way to communicate with people who'll never have time, knowledge, skills and motivation to completely understand the inner details of Bitcoin. Is there way to maintain the dialogue between the two groups?

[DannyHamilton]

Hmm, this 32 bit signed integer must be greater or equal to the current block version. Plenty of space here. Discussed on github: No forking Extra nonce added to Bitcoin header

I believe Danny's point was that it invalidated the midstate to change it.

Nah, I'll own up to making a mistake there.

For some reason, I thought that there were specific Block version numbers that were acceptable under the consensus rules.  I guess I just learned something new.  I'll read up on it a bit more.  So, you guys are saying that the block version number CAN be used as a nonce?  In that case, as gmaxwell states, it still has the effect of forcing you to recompute the midstate, so there isn't a benefit to modifying this INSTEAD of the nonce.

[Sergio_Demian_Lerner]

I think gmaxwell is right, there is not enough evidence in those 4 blocks to suggest that there has been a breakthrough in mining ASICs.

On the other hand I guess valiron is also right, he knows (and probably can't disclose) that some mining companies may be testing new tricks to improve their ASIC hashing power. Maybe he hacked into one company computer or he was told by a friend who works in one of them and he promised not to say anything.

Mining companies want to maximize their profits and they are not obligated to disclose their engineering achievements. Those are trade secrets and nobody has ever complained about ASIC Mining companies having close designs. I don't even think that the Bitcoin community can even enforce ASIC companies to open their designs, because they are already using other companies IPs that they can't disclose, and because they can always go anonymous to avoid disclosing anything.

There is plenty of technical information put together in this long thread (given by gmaxwell explicitly, and DannyHamilton's analysis on which parts of the block header can be used as nonce, and my very old posts about modifications to the Bitcoin header) to ease you discover one of such tricks. Take some time to think about it, take aside all posts with personal insults, and you'll probably find the solution right in front of your eyes.

I'm not that clever so there may be more tricks to discover.

However, a trick can only give you a certain speedup, say 20%, based on a reorganization of the SHA256D operations, or the pre-computation of some operations that change less often. Other changes (such as reducing the fabrication node) can give you much higher speedups. So this isn't alarming.

A completely different thing is to find a way to invert SHA256D, which I'm absolutely sure nobody will ever be able to do without some revolutionary quantum computer that does not exists even in theory.

The only attack I was thinking of when I wrote the Bitcoin header post, was all mining companies adopting tricks that give them some little advantage, but at the same time they degrade the performance of the network as a by-product. One of such attacks is cited when I posted about using approximate adders, and the danger that a monoculture of approximate ASICs can get stuck in a header that always generates a faulty addition.
 
If such problem ever arises, the community will probably find the way out by doing the right hard fork to prevent it.

IMHO the cryptographic security of SHA256D function of Bitcoin will never be seriously compromised.
However if there were a single mining company manufacturing ASICs being 200% faster than the competition, that would clearly hurt Bitcoin in a practical econo-socio-political way. The good news is that the accumulation of tricks probably will never reach such an improvement.

Best regards,
 Sergio.