Handle much larger MH/s rigs : simply increase the nonce size

[kano]

The device are simple - hard wired into silicon - they do the hash process and cycle the nonce.

Okey doke.  I thought they had some firmware that knew how to talk over USB (or ethernet or whatever), too.

Of course - you couldn't talk to them without that - should I respond to what you are really saying - or just what you wrote
Yes I do know what you are doing

Anyway: it still is simply a ~2 stage hash of 80 bytes with 4 of the bytes cycling from 0 to 0xffffffff and then the 80 bytes being fed into the ~double sha256 - be it 1 2 4 8 or 256+ ~double sha256 streams.

Yes you could put a merkle processor into the silicon and a coinbase construction processor - and it would then have to match what the pools handle ... and with crap like GBT and Stratum spinning around and not being able to decide what to do, what will be next?
That variable target is not worth risking hundreds of thousands of dollars to implement when it could change tomorrow - backward compatibility takes a back seat in bitcoin ...

[Sergio_Demian_Lerner]

There is a possible FIX to add more nonce space while maintaining backwards compatibility:


We can take 16 bits from the block version field to be used as more nonce space:

4    version    uint32_t    Block version information, based upon the software version creating this block

split it as:

2    version    uint16_t    Block version information, based upon the software version creating this block
2    extra_nonce uint16_t    More nonce space


I think it will not break compatibility. Old nodes will only tell the users to upgrade.

What yo you think?

[Jutarul]

No need to change the protocol. Let it be an engineering problem.

AFAIK a simple solution is to implement the capability into the mining rig to do merkle tree reorganization. Then if you have 10 transactions to play with you could create 10!=3628800 permutations of the same block. If you have less than 10 transactions, the mining software could create spaceholder transactions with unique hashes.

[btharper]

No need to change the protocol. Let it be an engineering problem.

AFAIK a simple solution is to implement the capability into the mining rig to do merkle tree reorganization. Then if you have 10 transactions to play with you could create 10!=3628800 permutations of the same block. If you have less than 10 transactions, the mining software could create spaceholder transactions with unique hashes.

I've seen blocks come out after a minute with dozens of transactions. As long as the transaction rate stays relatively high I don't see this being a huge issue. Maybe the first set of blocks has fewer permutations to it, but not a huge issue.

As for having mining hardware doing the merkle tree computations. Sticking a simple microprocessor in near the upstream communication would be trivial for anyone making these things, and hopefully it's something they've already thought of (at least the ability to adapt to small changes like the version number being incremented). The old system of fetching and reporting on difficulty 1 shares is pretty dead when ASICs come out, that's no shock. If I remember correctly Eclipse was considering switching to difficulty 10 shares a few weeks ago. It may just also be the case that the old standard for getwork's just won't work anymore either and something closer to the hardware has to manage the merkle root.

[Sergio_Demian_Lerner]

No need to change the protocol. Let it be an engineering problem.

Well, it's not a change in the protocol implementation. It's a change in the protocol semantics.

Right now you can use part of the version field as nonce and nothing bad will happen AFAIK.

[btharper]

No need to change the protocol. Let it be an engineering problem.

Well, it's not a change in the protocol implementation. It's a change in the protocol semantics.

Right now you can use part of the version field as nonce and nothing bad will happen AFAIK.

Even that is still a hard fork. What you have to worry about is "How will an old client see this?" When the answer is "Like something broke" you have a hard fork.

Messing with the version field (even resizing it) means that old clients will see a bucket full of fuck and have no idea what's going on. Particularly when mixing it with a fairly random field like the nonce it starts to see random version numbers on posts. Even leaving the version number the same and shifting the field over to make room in the future (making the two bytes meaningless until the switch) still means you're left shifting the version bytes and now you're seeing version 65536. If you decide to use the first two bytes then there really isn't much change until the abrupt changeover when majority agrees to switch and start using those extra two bytes for something they weren't before.

[Sergio_Demian_Lerner]

Messing with the version field (even resizing it) means that old clients will see a bucket full of fuck ...

I can't  follow you line of reasoning.

If you force the Block.nVersion>=2 or higher, old clients will see "garbage", and will do nothing (or do the same as when a 0.6.x version sees a version 2 block, just process it).

Then you can keep the 2 less significant bytes for version, and use the 2 most significant bytes for nonce. Any miner have the power to do it, and nobody can stop them from doing it.

When Bitcoin version 7.1 is out, then the dev team will need to change the code to reflect this change, splitting the nVersion field in two.
If they try to force the 2 MSbytes of nVersion to zero, then they will force a hard fork, because there will be already a block in the chain with nVersion>65536 and because old clients will accept new blocks with nVersion>65536, so they must follow the community and accept the new semantics.

In this regard, any miner has the power to change the protocol semantics.

Obviously, having consent from all parties involved is better.

best regards,
 Sergio.

[Jutarul]

In this regard, any miner has the power to change the protocol semantics.
Obviously, having consent from all parties involved is better.

If fields are not being used right now, they can obviously be hijacked for another purpose. That shouldn't break the protocol, it's a reinterpretation of the protocol. However, it's bad practice because you never know how many independent projects use the same trick. Then you run into issues when you want to merge functionality.

[Inaba]

The BFL units have made provisions for a larger nonce range in our firmware already, so if/when this happens we should be able to accommodate it without any problems.  The rate we chew threw the nonce has already caused some concern internally, so it's been a design feature almost from the beginning.  Personally, I would like to just see a 64 bit field for the nonce, that would give us a lot of expansion room and it would take a truly monster machine to chew through that in a reasonable time frame.  I don't see monster machines like that happening anytime soon.

Maaku: The nonce range and difficulty are two entirely separate issues.  One is a problem for the hardware one is a problem for the software.  Both need to be addressed (and the software side has been addressed with variable difficulty and GBT/Stratum) - but the hardware issue still remains as Kano has outlined and I think his proposals are the best long term solution so far, everything else is just a hack that is prone to problems and also being overrun as hardware becomes faster.

[maaku]

Right now you can use part of the version field as nonce and nothing bad will happen AFAIK.

No, just.... no.

[jgarzik]

Block's nVersion policy is already specified, in the recently accepted BIP 34 - "Block v2, Height in Coinbase"

Please do not use nVersion as an additional nonce.

Stratum mining and getblocktemplate (BIP 22) already have provisions for ASIC miners, through the extraNonce field and other possibilities.  This provides unlimited nonce range.

Any change to the block header is a hard fork upgrade, and should be avoided at all costs.

[Luke-Jr]

Realistically, GBT (BIP 22/23) solved this for the pool/software end a long time ago, and stealing 8 bits from the ntime (about 3 minutes of rolling) is probably plenty for the hardware side for a while...

While using [part of] the block version as another nonce was at one time a possibility, starting with 0.7.0 any changes there will cause the client to report an upgrade required. And again, it's not necessary.

[slush]

WTF I just read. Use block version as extra-extra-nonce?

* slush is checking today's date; no today isn't 1.April

[Sergio_Demian_Lerner]

WTF I just read. Use block version as extra-extra-nonce?

Not the whole field, just take a couple of bytes from it. 2 bytes is more than enough for version information...

[btharper]

WTF I just read. Use block version as extra-extra-nonce?

* slush is checking today's date; no today isn't 1.April

Yea, that's been my reaction on a lot of these proposals. To me a lot of this thread seems like "Holy crap, ASICs mean nonce won't be enough and bitcoin will break and my 56 billion bitcoins will be worthless because the network will break and my 23 TH/s rig won't be able to request work fast enough. And then ways to radically alter the syntax of the block to allow for more play room in each block (doubling the nonce range (which is one of the more sane requests), using two bytes of the version header as extra space for nonce, making things up to add in as extra hashes).

Don't get me wrong, especially for larger rigs (BFL's 1.5TH/s for one) trying to use difficulty 1 getworks from a pool server isn't realistic at all.

What I'd like to see is a small device (think the size and power of a phone or home router) that can manage a few miners (say around 10 TH/s of miners) over a local connection (which would help keep latency low among other things). On top of that I'd like to be picky and say that everything should be connected via ethernet, possibly even using power over ethernet (up to ~25W / 25GH with BFL's current specs). Just imagine if setting up new mining gear involved plugging in 1 ethernet cable per miner (plus a little config setup, or possibly them automatically syncing to the mining sever they're connected to).

[gmaxwell]

Not the whole field, just take a couple of bytes from it. 2 bytes is more than enough for version information...

It's ridiculous, completely unnecessary, and craps on one of our very few backwards compatible upgrade mechanisms. Backwards compatibility means we may need to put some data there in the future— since even incrementing the version can't change the layout without creating a hard fork. Perhaps worst of all, it's simply tasteless. The version field? really. yuck.

Even a fairly slow computer can do extra-nonce advancing for many TH/s, esp once considering a few bits of ntime rolling— well, at least if someone doesn't insist on writing their SHA256 in javascript or other ultra slow interpenetrated language... And all these TH/s are not free. If someone can't manage a modest CPU or two per hundred thousand dollars in mining asics then they have a financial management problem that no amount of protocol fiddling can fix. Certainly it's not healthy for the network to encourage miners which are so CPU starved they can't even afford to update their coinbases enough to keep up with their hardware even given the ~2^48 work reduction that they get from nonce plus ntime rolling.

As far as communications goes, presumably people engineering products that cost as much as a nice car are smart enough to not use lawnmower wheels on it when it needs tank treads. If they aren't then I feel bad for their customers for all the other bits of the design they'll get wrong. But since its totally unprecedented that some mining hardware maker would produce a horribly mismatched design which did something daft like provide their hardware with a fraction of the required power supply we should have nothing to worry about…

[kano]

Sigh - it does become rather silly if you look at it from this perspective:

People are suggesting ways to solve the problem that the nonce size is too small (including Stratum and that other guff like that)

So ... what's the problem?

The nonce size is too small (as everyone agrees so far who has posted)

So ... make it bigger ... as I said in the first post 3 months ago ... though as I said later - 128 bits is a very long term solution.

... and as I said on the previous page ... create a version 2 block type that has the bigger nonce and set it to some time in the future
Yeah we've already lost 3 months ... though I guess in another 3, 6, or 9 months this wont be fixed and people will have the same excuse then that hard forks can't be done ...

[btharper]

Sigh - it does become rather silly if you look at it from this perspective:

People are suggesting ways to solve the problem that the nonce size is too small (including Stratum and that other guff like that)

So ... what's the problem?

The nonce size is too small (as everyone agrees so far who has posted)

So ... make it bigger ... as I said in the first post 3 months ago ... though as I said later - 128 bits is a very long term solution.

... and as I said on the previous page ... create a version 2 block type that has the bigger nonce and set it to some time in the future
Yeah we've already lost 3 months ... though I guess in another 3, 6, or 9 months this wont be fixed and people will have the same excuse then that hard forks can't be done ...

The nonce size is too small (as everyone agrees so far who has posted)

I would like to very explicitly disagree with this. I've never thought this and I joined the thread over a month ago I'd say. There is no reason to go through the trouble of a hard fork to implement a solution in search of a problem.

If this were put up for a vote by miners (similar to bip 16) where miners vote with their hashing power (though I do think this gives mining pool operators extra say since they're voting with their entire pool's hash rate), since they're the ones most affected and the ones who should care.

[kano]

...
I would like to very explicitly disagree with this. I've never thought this and I joined the thread over a month ago I'd say. There is no reason to go through the trouble of a hard fork to implement a solution in search of a problem.
...

So if nonce size isn't too small - why are we changing bitcoin at all to support ASIC?

Why is this stupid BIP in 0.7.0 that's been forced on everyone, that also removed functionality from bitcoind, necessary?
Why have people called sticking more junk in the coinbase an 'extra' nonce? What? They gave it the wrong name?
The current nonce is big enough? It's not necessary?

Why is Stratum necessary?
Coz when you get a block header to hash, you can only hash 2^32 times before needing to change it.
Why does that restriction exist?

What was the stupid (short term) hack used up until now? Roll ntime?
Why? Certainly couldn't be related to the nonce size?

It's gonna work fine without any changes right?

Oh ... no, that's not correct

Where IS the restriction at the moment?

OK back to the topic subject now

[slush]

Why is Stratum necessary?

"Solves small iteration space without need of protocol hard fork." Isn't that reason big enough?

Where IS the restriction at the moment?

The restriction is that changing nonce size don't affect only miners, but all participants in bitcoin project. Breaking compatibility is always the last possible step, because it brings unnecessary instability.

Although I may agree that bigger nonce size would be the cleanest solution, it is simply not feasible in the real world. Looks like some people are too focused in mining, but protocol change is not just about mining. There already exists solutions which solves issues for miners, so I'll definitely prefer these solutions before breaking whole network.