[ANN] Bitcoin PoW Upgrade Initiative

[muyuu]

An automatic full PoW switch trigger on extremely anomalous conditions.

[Carlton Banks]

What will prevent the SHA256 miners orphaning the new hashing algo blocks? Would it not be better to start at 51% share to the new hash algo to prevent that, or would an exponential rise in blocks built on using the the new hash algo as it approaches 51% of the hashrate be acceptable?

I doubt that any of the existing miners would agree to a proposal that immediately slashes their block reward in half, while many might agree to a 5% haircut. Especially since the miners that choose to join us will get a nice windfall at first as the difficulty adjusts downwards.

Uncooperative miners will end up on a different chain in any case—there's nothing we can do about that. We just count on the economic majority being upgraded and ignoring their chain. They can continue mining their chain at a loss or rejoin Bitcoin—the choice is theirs.

Right, well I hate to resurrect previous sore points, but.....


We should begin to build support for this. If every iteration requires a soft fork, then either an additional soft fork can let us wind back to 100% SHA-2 (in the perhaps unlikely event of the uncooperative ASIC miners changing their minds), or we just carry on until 100% CPU mining is back in charge.

If we do not build support before the threat of external hard fork attacks (note the plural "attacks"), I fear we may get outplayed with lateral attacks via the internet infrastructure, the legal system, the dev team, all 3, or something I'm not even considering. It should be an obvious fact that international collaboration between those perpetrating this BU attack (at a minimum between some English speaking + Chinese interests) should indicate quite how seriously the unknown opponent is taking Bitcoin.


If the most balanced way to do this is by gradation, and I believe a good case has been made for that, then waiting for an emergency to occur could well cause the strategy to invite the opponent to produce their most potent attacks to stop it. Switching to 100% CPU mining in an emergency makes sense, switching to 5% over weeks or months does not.

[mmgen-py]

Switching to 100% CPU mining in an emergency makes sense, switching to 5% over weeks or months does not.

Of course. We need both an emergency plan and a non-emergency plan. The emergency plan would be a 100% HF switch to the new algorithm. The non-emergency plan is the gradual one I'm proposing. And for it we of course need to work with the community and build support. If it activates, I don't envision ever going back to all-ASIC mining. On the contrary, the whole point is to build a new, more decentralized mining community around readily-available DRAM.

I proceed from the assumption that we might not get the emergency we're expecting, at least not anytime soon. It's too risky for our opponents. Instead, they'll just keep waging a war of attrition against us in the ways you've mentioned. This is why we need to be proactive in advancing the non-emergency plan while our position is still strong.

[Carlton Banks]

I proceed from the assumption that we might not get the emergency we're expecting, at least not anytime soon. It's too risky for our opponents. Instead, they'll just keep waging a war of attrition against us in the ways you've mentioned. This is why we need to be proactive in advancing the non-emergency plan while our position is still strong.

I more or less agree, but the assumption I've bolded is dangerous. There is a significant risk that a "short sharp shock" act of caprice could happen very suddenly, especially now that BU is looking increasingly unlikely to gain any foothold with users. As I mentioned in another thread, the opposition have exceeded their "crying wolf" quota. What makes anyone believe the attackers will not choose a different MO altogether?

Frankly, I would be most surprised if another hard fork campaign would ever be a genuine attempt, I would expect a sudden political act against Bitcoin (via the potential vectors I described before) to happen during the next hard fork campaign, "when it is least expected" lol. We should expect it, and prepare the user community accordingly.

[Frogolocalypse]

An extension of this post :

   https://bitcointalk.org/index.php?topic=1833391.msg18247163#msg18247163

Rather than change the core client, I have an extension of the method detailed above.  What about having a PKI (public key infrastructure) transaction pool?  PKI TxnPool.

The goal is to ensure that only miners that meet your user requirements can include your txns in a block.  The desire is to de-incentivize miners that are attacking the network, and that there is a clear path between nodes and miners for transactions that are validated.  This might not be able to stop a 51% attack, but it might be able to stop miner centralization occurring in the first place.

PKI txnpool.

Setup :
Miner creates prv/pub key pair.  Miner publishes pub key.  Could be anywhere.  Miner creates black-list of nodes (empty).  Pseudonymous.  Miners signs black-list with priv-key.

Node creates prv/pub key pair.   Pseudonymous. Node creates white-list miners using pub keys of miners.

Adding txns to PKI txnpool.

Node reads node mempool for new txn.  Node validates txn.  Node creates hash of txn.  Node encrypts txn using pub-keys of all miners in white-list.  Node signs encrypted txn.  Node creates txn-relay-file with : hash of txn, Node pubkey, Signature, list of white-listed miners, encrypted txn.  Node adds encrypted txn to PKI txnpool for relay.
   
We now have a txn that has been validated, with a node identifier, and a white-list of miners that can include their txn on the block-chain.

PKI txnpool relay.

Node peers with other node.  Nodes exchange pubkeys.  Nodes exchange miner black-lists with version numbers greater than on node.  Miner validates signature of miner black-list.  Node black-lists Node that has incorrectly signed miner black-list.  If valid continue, if not, cancel peer connection.  Nodes remove txns from PKI txnpool that are signed by black-listed nodes.  Nodes create exchange white-list, removing a miner from the white-list if it is in the miner black-list.  Nodes exchange curated miner white-lists. Nodes exchange txns that meet node white-list requirements.

PKI txnpool removing txns that have been included in blocks or are too old.

Node gets new block.  Node creates hash of each txn in block.  Node removes from PKI txnpool that have duplicate hash.  Node removes from  PKI txnpool txns that are >cfg time from adding to local pool.

Miners use PKI txnpool to get txns.

Miner queries PKI txnpool.  Miner node sets white-list to only that miner.  It doesn't matter if they have more, they just won't be able to decrypt txns that they don't have the privkey for.  Miner uses privkey to decrypt txn.  Miner node validates txn.  If txn is invalid, add node that created txn to miner black-list.  Miner updates and signs black-list.  Miner replicates new miner-black-list to peer nodes.


Through this, PKI txnpool nodes work in an independent pool from bitcoin.   Nodes must ensure they create valid transactions, or they will be black-listed by any miner that opens a txn they created, and determine it is invalid, and the txns they create won't replicated. Miners can only decrypt transactions that have been encrypted using their pubkey.  This solution should allow users to select from a group of miners, and by extension, de-select miners that don't meet their requirements.  This provides a user and miner controlled tool to route-around malicious miners and nodes.

[David Rabahy]

One word; hurry.

Oh, and also, hire a good public relations team.

[tenletters]

One word; hurry.

Oh, and also, hire a good public relations team.

For my part, I will ramp up the marketing once we have 2 or 3 working branched clients (or implementations that don't require client changes) that have been subjected to extensive peer review, and hopefully some libraries+documentation to ease the transition for 3rd party service providers like light wallets and exchanges. Until then, it might not be necessary (or wise).

Actually, the work being done by the devs in this thread has already been covered by a good number of major outlets ([1], [2], [3], [4]).

[iCEBREAKER]

Have you guys looked at Cuckoo cycle?

FWIW we evaluated Cuckoo as well for Zcash, and it was a strong second-place contender. There wasn't really anything wrong with it — it just didn't seem to have quite as much of a rigorous scientific analysis as Equihash. However, that is a very subjective thing for me to say. You could argue (and Cuckoo's author, John Tromp, does argue persuasively) that Cuckoo's history of analysis and refinement is better than Equihash's.

What about cycling through 10 unique PoWs every 10 blocks?

I'm not the best at discrete analysis and understand this multiplies attack surface 10-fold, but could we splinter miners into small, specialized, and de-fanged factions using 10 different well-chosen hash algorithms, then scatter them among CPUs/GPUs/FPGAs/ASICs?

Block 1 JH
Block 2 Skein
Block 3 Groestl
Block 4 Cuckoo
Block 5 Keccak
Block 6 Equihash
Block 7 BLAKE2
Block 8 SCrypt
Block 9 CryptoNight
Block 10 Ethash

Let's change PoW so that anyone with moderate amount of capital can attack us, sounds like a great idea!

Agreed. You can hate ASICs, but they do provide a very robust network, very expensive to attack. Change of PoW would do more harm than good.

Right now, to attack Bitcoin you only need to bamboozle Jihan.  Not very expensive.  

If Bitcoin is in danger due to such an attack, it's nice to have fully prepared contingency plans ready to deploy.

But the socioeconomic consensus will allow Roger and Jihan to initiate aggression before it reacts with overwhelming retaliatory force.

[iCEBREAKER]

A change of PoW as a quickfix (to fool currently manufactured ASICS) without too much risk of bugs can be as follows:

Instead of checking for n zero bits, implement checking for n one bits instead.

If you are bold, you can have the sequence of leading bits to check to be dependant on the trailing bits of the previous block.

Your first suggestion seems to be the simplest thing which would actually work.

But I also like the 2nd one!

An automatic full PoW switch trigger on extremely anomalous conditions.

An option worth considering on all doomsday machines....

https://en.wikipedia.org/wiki/Dead_Hand_%28nuclear_war%29

[Andre_Goldman]

My opinion from the miner's economic point of view ...

a GPU Farm get a much lower hardware depreciation than ASIC hardware economics.

A GPU farm can also be used for Deep Learning rent (for instance.:
AWS, Google, Azure GPU cluster (on average costs) $0.90 per hour to run)
also some small GPU providers https://www.leadergpu.com/

You can reseller your used hardware to gamers on ebay, etc.

[pinkflower]

Have you guys looked at Cuckoo cycle?

FWIW we evaluated Cuckoo as well for Zcash, and it was a strong second-place contender. There wasn't really anything wrong with it — it just didn't seem to have quite as much of a rigorous scientific analysis as Equihash. However, that is a very subjective thing for me to say. You could argue (and Cuckoo's author, John Tromp, does argue persuasively) that Cuckoo's history of analysis and refinement is better than Equihash's.

What about cycling through 10 unique PoWs every 10 blocks?

I'm not the best at discrete analysis and understand this multiplies attack surface 10-fold, but could we splinter miners into small, specialized, and de-fanged factions using 10 different well-chosen hash algorithms, then scatter them among CPUs/GPUs/FPGAs/ASICs?

Block 1 JH
Block 2 Skein
Block 3 Groestl
Block 4 Cuckoo
Block 5 Keccak
Block 6 Equihash
Block 7 BLAKE2
Block 8 SCrypt
Block 9 CryptoNight
Block 10 Ethash

Wont this make difficulty rise and fall like a wave? If the answer is yes, then wont this open the network to attacks if there is a fall in difficulty. Maybe it wont be the case after a year but in the beginning it could be.

If youre going that road, why not have 2 POW algorithm at first with the option of expanding it instead of starting with 10 outright.

[mmgen-py]

Wont this make difficulty rise and fall like a wave?

Each algorithm would have its own difficulty I would imagine.

[BitUsher]

There are several developers working on PoW changes already , but what we need is proper peer review testing and a big bounty for this work. I am willing to donate btc and help fund raise for this , but we need 3 trustworthy an public people to handle the funds. Who is interested or who should we ask to get this started?

[Carlton Banks]

There are several developers working on PoW changes already , but what we need is proper peer review testing and a big bounty for this work. I am willing to donate btc and help fund raise for this , but we need 3 trustworthy an public people to handle the funds. Who is interested or who should we ask to get this started?

The "public" stipulation may be difficult to satisfy. Irrespective of how much support we can build, whoever accepts an escrow role is sticking their head above the parapets rather significantly (Bitfury have already threatened legal action against PoW changes, although against who is undetermined I believe)

Can the several developers not present their designs, rates and also addresses to donate to?

[mmgen-py]

Here's how I propose to decentralize mining with a PoWA (proof-of-work additions) soft fork:


New PoW chain is shown in pink, legacy blockchain in blue.

Brief description:

• New PoW miners mine continuously, legacy miners almost continuously. Proofs for the new PoW blockchain's mini-blocks are embedded into legacy chain in a special output of the coinbase transaction.
• All assembling of TXs into blocks and reorgs happen on the new PoW chain. Legacy miners' participation is restricted to solving the SHA256 proof-of-work, adding their payout address to the coinbase TX and broadcasting the block.
• Mini-blocks are 100 Kb in size; new PoW blockchain has one-minute block discovery rate (i.e., confirmation time).
• Mini-block headers are like ordinary block headers but with an added payout address.
• Legacy miners experience an avg. of 10% downtime while waiting for new PoW miners to mine the next proto-block.
• Initially, 90% of block reward will go to legacy miners and 10% to new PoW miners. Legacy miners' share could be gradually reduced (over a period of years?) until it reaches zero.
• After legacy miner broadcasts a valid block, new PoW miner assembles a "proto-block" (a nearly complete Bitcoin block) out of TXs from all mini-blocks mined since last proto-block. Extra TXs are added from mempool to make the block full. Payout outputs for mini-blocks and proto-block are added to the coinbase TX (with 10% of block reward divided equally among them). Mini-block headers and legacy block header hash are embedded in the special output of the coinbase TX. Miner solves the proto-block and broadcasts it to legacy miners.
• Legacy miner adds his payout output and proto-block header to the coinbase TX and Merkle root to the header, making the block complete. He then solves and broadcasts the block as usual. Legacy miner is allowed to alter only four pieces of data: timestamp, nonce, coinbase nonce and his payout output.
• If new PoW miners solve nine mini-blocks faster than legacy miners solve one block, then they continue mining empty mini-blocks until legacy miners finally solve the block. Thus a proto-block may contain more than nine mini-blocks.
• In the reverse case, proto-block will contain fewer than nine mini-blocks.
• "Longest chain" according to new consensus rules is chain with the most embedded mini-block/proto-block proofs. This prevents legacy miners from initiating reorgs.
  
• System is PoW agnostic, but a memory-hard algorithm such as Cuckoo Cycle or Equihash would be preferred, as this would lead to the creation of a new  decentralized mining industry based on universally available DRAM.

This description is preliminary; certain details may be subject to change.

Benefits:

• New decentralized mining industry is created.
• Legacy miners are deprived of all decision-making power and possibility of attacking the chain.
• Gradual or partial phase-in of new PoW reduces security risk.
• Bitcoin's effective confirmation time is reduced to one minute.
• Despite the 10% "haircut" they receive, legacy miners can profit from the SF if economic majority upgrades and most value remains on upgraded chain. Thus the proposal could expect to gain broad community consensus, even among miners.

Variations:

• Legacy miners' share of block reward could be left at 90%, with no phase-out, making proposal more attractive to them.

[Carlton Banks]

Here's a schematic of my proposed PoWA (proof-of-work additions) soft-fork blockchain.




New PoW chain is shown in pink, legacy blockchain in blue.

Brief description:

• New PoW miners and legacy miners mine in parallel. Proofs for the new PoW blockchain's mini-blocks are embedded into legacy chain in a special transaction.
• All assembling of TXs into blocks and reorgs happen on the new PoW chain. Legacy miners' role is restricted to finding SHA256 hashes, final block assembly (calculating payouts, creating the coinbase TX) and broadcasting the block.
• Mini-blocks are 100 Kb in size; new PoW blockchain has one-minute block discovery rate (i.e. confirmation time).
• Mini-block headers are like ordinary block headers but with an added payout address.
• New PoW miners mine with no downtime. Legacy miners experience an avg. of 10% downtime while waiting for new PoW miners to mine one block. (It may be possible to eliminate this downtime by having new PoW miners solve only mini-blocks and not entire block.)
• Initially, 95% of block reward will go to legacy miners and 5% to new PoW miners. Legacy miners' share will be gradually reduced (over a period of years?) until it reaches zero.
• After legacy miner broadcasts a valid block, new PoW miners assemble all TXs from mini-blocks mined so far into a single Bitcoin block with no Coinbase TX, solve the block and broadcast it along with mini-block headers to legacy miners.
• Legacy miner adds mini-block proofs, TX counts and payout addresses to the special transaction, calculates payouts (initially distributing 95% of reward to himself and 5% equally among new PoW miners), adds payout outputs to Coinbase TX and then solves and broadcasts the block as usual.
• If new PoW miners solve nine mini-blocks faster than legacy miners solve one block, then they continue mining empty mini-blocks until legacy miners finally solve the block. Thus a block may contain more than 10 mini-blocks.
• In the reverse case, fewer than 10 mini-blocks will be assembled into a block, and the new PoW miner who assembles the block will add as many TXs to the final mini-block as required in order to reach the blocksize limit (currently 1MB).

All of the above is preliminary and subject to change.

What's the rationale for making the mini-blocks 10 per legacy block? I'm thinking of the orphan rate.

I'm also unconvinced about a "years" timeframe. I would propose 1 year, where the interval between the 5% steps starts at close to infinity increase for the 5-10% part, and gradually increases the interval between steps (like an exponential curve inverted about x=y, is that the cosine curve?)

Going faster to begin with should help to attract hashing power to newPoW, and in turn dissuade the BU miners from even attempting the various attacks they have no doubt developed. The "long tail" will gradually contribute to calming what would inevitably be a very febrile atmosphere surrounding the initial 5% change (the accompanying FUD would no doubt be typically disproportionate)

[tenletters]

Have you guys looked at Cuckoo cycle?

FWIW we evaluated Cuckoo as well for Zcash, and it was a strong second-place contender. There wasn't really anything wrong with it — it just didn't seem to have quite as much of a rigorous scientific analysis as Equihash. However, that is a very subjective thing for me to say. You could argue (and Cuckoo's author, John Tromp, does argue persuasively) that Cuckoo's history of analysis and refinement is better than Equihash's.

What about cycling through 10 unique PoWs every 10 blocks?

I'm not the best at discrete analysis and understand this multiplies attack surface 10-fold, but could we splinter miners into small, specialized, and de-fanged factions using 10 different well-chosen hash algorithms, then scatter them among CPUs/GPUs/FPGAs/ASICs?

Block 1 JH
Block 2 Skein
Block 3 Groestl
Block 4 Cuckoo
Block 5 Keccak
Block 6 Equihash
Block 7 BLAKE2
Block 8 SCrypt
Block 9 CryptoNight
Block 10 Ethash

DeSantis has started some work (he wants to do some testing before posting his source code for peer review though).
He's creating a Keccak fork and a Cuckoo fork, and has created a beautiful automated testing utility that I hope he gives me permission to link to you guys.

The testing utility (I've viewed the source, it's not vaporware) allows you to spin up multiple Docker containers, each containing a different Bitcoin node; some of the nodes can be Bitcoin 0.14.0, some of them can be Bitcoin Unlimited, and some of them can be Keccak, Cuckoo, etc.

With these containerized Bitcoin nodes, you can then simulate various forking scenarios, and actually observe in real-time how it plays out. With my limited bitcoin programming knowledge, I am waiting for him to document the config file that controls the node counts & types, and to create some python installation script (which are easier to debug for me at least).

tl;dr - DeSantis is testing Keccak & Cuckoo using a Bitcoin Network Simulator.

[tenletters]

There are several developers working on PoW changes already , but what we need is proper peer review testing and a big bounty for this work. I am willing to donate btc and help fund raise for this , but we need 3 trustworthy an public people to handle the funds. Who is interested or who should we ask to get this started?

The "public" stipulation may be difficult to satisfy. Irrespective of how much support we can build, whoever accepts an escrow role is sticking their head above the parapets rather significantly (Bitfury have already threatened legal action against PoW changes, although against who is undetermined I believe)

Can the several developers not present their designs, rates and also addresses to donate to?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi devs, can you all send your nominations for who the most credible individuals are for managing an m-of-n account (for holding the PoW bounty's reward funds)?

1) Please send PGP-signed emails to jecooper@alum.mit.edu (you may encrypt if you wish to remain anonymous, my PGP key ID is 331B6406 (pgp.mit.edu)).

2) Once all the nominations are received, I will make one big post containing all of the signed emails (unless the sender wishes to remain anonymous due to fear of BitfuryGeorge).

3) We reach out to the agreed upon individuals, inviting them to become custodians of the multisig address and requesting a public BTC address (for which they control the private key) from each of them.

4) Create the multisig address, notify the new custodians of the account.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJY2FrYAAoJECXDNTkzG2QGrvwH/0CjwKRgSmHmMPFXKA8F1YWa
CMcrWx0KN2ykhqxclxEAlMIs8Zb4u3KO89nejza/Guh0f2sNWSCW6NvrEhRzHodf
TSn8VpCcjpeYc1Iu5wSMBTVk6h/dqZy0eJJRukN4M8qTstnwvU2B48I7Q24x9zLe
B2lOxqhm3fIauaXCTgey4YLgvMfo058jg7+x9DrYKtmP8jht49AmvBv+XI69YfHq
XHvTpbipeNsoTR7qQUXnsnGtbMW7Sl0jywFjRUe1Gq7xGBf6ICH6WkCBLgRDCPCA
z+7gqv6zqjZaAqmbaZej/+4JShnhX2wgj1LKvtW65TdJuyxy4KG6sS231wgAp/4=
=yOSx
-----END PGP SIGNATURE-----

[mmgen-py]

What's the rationale for making the mini-blocks 10 per legacy block? I'm thinking of the orphan rate.

In order to keep the two chains in sync and ensure that the new PoW hash power is always working, the new PoW miners can assemble the next proto-block from mini-blocks and mine it only after legacy miners have mined and broadcast the current block. The period while the new PoW miners are mining the proto-block is downtime for the legacy miners; their hash power is going to waste. In order to minimize this downtime, we need a fast confirmation time for the new PoW. One minute isn't too extreme, actually, if we consider Ethereum's 20-second confirmation time.

I'm also unconvinced about a "years" timeframe. I would propose 1 year, where the interval between the 5% steps starts at close to infinity increase for the 5-10% part, and gradually increases the interval between steps (like an exponential curve inverted about x=y, is that the cosine curve?)

Going faster to begin with should help to attract hashing power to newPoW, and in turn dissuade the BU miners from even attempting the various attacks they have no doubt developed. The "long tail" will gradually contribute to calming what would inevitably be a very febrile atmosphere surrounding the initial 5% change (the accompanying FUD would no doubt be typically disproportionate)

It's a tradeoff. Yes, transitioning faster would attract more new PoW miners. So would giving them a larger share of the block reward at the beginning, say 10%.

On the other hand, since this is "non-hostile" fork proposal that seeks to gain broad community consensus, we don't want to alienate legacy miners by turning their hardware into scrap metal too quickly. This is why I would prefer to err on the side of an overly long phase-out period rather than an overly short one. A linear phase-out is preferable to an exponential one for the same reason.

As for attacks, non-upgraded miners may attempt to attack the chain to fool non-upgraded nodes, but this is a risk for any SF. We just have to rely on having most economic nodes upgraded by flag day.

[pinkflower]

Wont this make difficulty rise and fall like a wave?

Each algorithm would have its own difficulty I would imagine.

Thats what I think. My next question for that would be, wont it make the network open to attacks if the difficulty suddenly drops low? The idea might be good on paper but its really only complicating matters. Best to come with a new POW algorithm that uses less energy.