High Resolution, Dual-Difficulty Blockchain

[ben-abuya]

Imagine a Bitcoin fork with the following modification: instead of a single difficulty target, there are two. Let’s call them the major difficulty and the minor difficulty, and let’s posit that the minor difficulty is 60 times easier than the major difficulty.

Miners can hash until they find one of the two targets. If a miner finds a minor difficulty, he broadcasts the block as usual. This is expected to happen approximately every ten seconds. These blocks will be very frothy, because there will likely be more valid blocks broadcast in a cluster of a few seconds. This doesn’t worry us too much (keep reading). Rewards will somehow be proportional to difficulty.

Approximately every ten minutes, a major block is found. Being just like any block, this is the authoritative chain of all the blocks that came before it, including the minor blocks. Since the difficulty is high, this will be the authoritative branch, unless another major block appears at the same time and competes with it. Assuming there's a clear winner, all historical frothiness is resolved, and frothiness begins anew for the next ten minutes.

The advantages of this system are:

* Higher resolution chronology. Bitcoin is a distributed record of chronology, and the original blockchain has a resolution of about 10 minutes. This one will have a resolution of about 10 seconds. This may not be extremely useful for standard transactions, but may be very important for some kinds of transactions, such as betting and stock trading.

* Minor blocks make 0/confirmation type double spending more difficult. Actual 0/confirmation attacks remain the same, but a vendor need only wait ten seconds for a minor block. Gaming the minor blocks would effectively require a 51% attack.

Disadvantages:

* The behavior of the system isn’t well understood. There will likely be many branches of minor blocks between the major blocks, which leads to wasted computing power and confusion. Minor blocks are not very authoritative until they’ve been buried under a major block.

My main motivation for this is as part of an alternate chain distributed betting system, but I haven’t seen it brought up before, and I realize there are probably some good reasons this can’t work. I’m looking forward to comments!

[casascius]

This already exists today!

It's called P2Pool.

It even follows the very same ten second rule you speak of.

[BoardGameCoin]

This already exists today!

It's called P2Pool.

It even follows the very same ten second rule you speak of.

I am interest... mayhap deepbit will be losing a mini-miner. I'll have to look into this when I have a chance.

[Steve]

This is a great idea.  When I think about it, I'm reminded of approaches that mitigate the risk of a 0 confirmation transaction…they basically listen to their peers and if a high degree of peers are reporting that a transaction is valid, then you assume that the risk associated with that transaction is lower…however, that solution treats every node as equal regardless of how much or little mining power might be behind it.  As Casascius points out, p2pool works very much like this…it would be great to continue this line of thinking and get miners (and the pools) to start announcing such "minor blocks" …these don't have to be retained by anyone after a few major blocks have been found…and not every node needs to listen for them or propagate them, but it's a great solution for accepting low value transactions after a few seconds.  *If* the overwhelming majority of mining power on the network supported this, then you would have a much better indicator after just a few seconds regarding the risk of accepting a transaction.  More analysis would be needed, but I'd say it's very likely that the economics are such that for transactions of as much as $100, you could be very safe accepting a transaction after 10 seconds…and for $1000 to $10000, a minute or two would probably push the risk into the realm of insignificance (particularly if it's a transaction between parties that have just a small amount of trust or recourse).

[Sergio_Demian_Lerner]

I´ve thought about multi-level chains before, or even chains with fixed transaction/difficulty ratio for any chosen difficulty. The problem I see is that it´s difficult to predict which will be "Nash equilibrium" of the system or even if an equilibrium exists. Will all miners start making only the lowest difficulty blocks ? Or will they ignore and them always mine the ones with higher rewards?

One possible solution may be to fix the number of low difficulty blocks before a high difficulty block must appear (say in 10:1 ratio). Bit then you will have some (predictable) time intervals where there are no fast confirmations.


Best regard, Sergio.

[Steve]

I believe the only extra work required by miners is that they broadcast blocks that meet the minor block difficulty requirement.  The announced minor blocks only need to build on the last major block (they don't need to build on each other).  Not all of the network will care about minor blocks, but people that want to get a good indicator of the risk associated with accepting a block after a few seconds will care.  The key to this working will be that virtually all miners on the network support it.  If only 10% of the miners support it, then a minor block is not a very good indicator of whether the overall mining network will ultimately validate a given transaction.  This will very likely be a good revenue source for miners…they could charge a fee for notifications of these minor block when they're found.  Other services could aggregate these notifications, charging subscriber fees and passing a portion along to the miners.  The aggregators would provide a single source for these minor block notifications for people that needed a good, fast, reliable transaction risk assessment.  The aggregators would advertise that they represent 87.45% of the bitcoin mining network or some such.  Miners would certainly take advantage as it's an additional revenue opportunity for them.

[ben-abuya]

This already exists today!

It's called P2Pool.

It even follows the very same ten second rule you speak of.

Wow, it's embarrassing that I didn't know about that, thanks casascius! I'm pretty stoked that this appears to be working in real life. My main motivation for this was to get higher time resolution for stuff life stock trades and exchanges, and for that it's important that the minor blocks remain in the blockchain. So, is there any reason why this must remain a miner's hack, or could it be floated as an alternate blockchain? Clearly, another benefit that comes out of this is that the new blockchain would have a much reduced need for pools.

*If* the overwhelming majority of mining power on the network supported this, then you would have a much better indicator after just a few seconds regarding the risk of accepting a transaction.  More analysis would be needed, but I'd say it's very likely that the economics are such that for transactions of as much as $100, you could be very safe accepting a transaction after 10 seconds…and for $1000 to $10000, a minute or two would probably push the risk into the realm of insignificance (particularly if it's a transaction between parties that have just a small amount of trust or recourse).

Yeah, this is a big deal for vendors.

The problem I see is that it´s difficult to predict which will be "Nash equilibrium" of the system or even if an equilibrium exists. Will all miners start making only the lowest difficulty blocks ? Or will they ignore and them always mine the ones with higher rewards?

One possible solution may be to fix the number of low difficulty blocks before a high difficulty block must appear (say in 10:1 ratio). Bit then you will have some (predictable) time intervals where there are no fast confirmations.

A miner would be highly incentivized not to ignore the minor blocks. As part of the hashing process, they will find many minor blocks for every major block, so why not take the extra trivial step and broadcast it and have a good chance of getting the reward. This is even more true for major blocks. While hashing the minor blocks, miners will occasionally hit on a major block. It would be crazy for a miner to not broadcast that and lose out on the big reward. That said, I'm not sure the 10:1 ratio is necessary.

The announced minor blocks only need to build on the last major block (they don't need to build on each other).

In order to get the time resolution, the miners would need to build the blockchain on minor blocks. This would also get you exponential protection against fake blocks instead of linear protection. Is there an obvious reason this chaining wouldn't work? That said, a different blockchain operating as you described could solve a lot of problems, too.

[casascius]

The problem I see is that it´s difficult to predict which will be "Nash equilibrium" of the system or even if an equilibrium exists. Will all miners start making only the lowest difficulty blocks ? Or will they ignore and them always mine the ones with higher rewards?

One possible solution may be to fix the number of low difficulty blocks before a high difficulty block must appear (say in 10:1 ratio). Bit then you will have some (predictable) time intervals where there are no fast confirmations.

A miner would be highly incentivized not to ignore the minor blocks. As part of the hashing process, they will find many minor blocks for every major block, so why not take the extra trivial step and broadcast it and have a good chance of getting the reward. This is even more true for major blocks. While hashing the minor blocks, miners will occasionally hit on a major block. It would be crazy for a miner to not broadcast that and lose out on the big reward. That said, I'm not sure the 10:1 ratio is necessary.

In P2Pool, solving minor blocks results in you getting a proportional share of the payoff when somebody else solves an actual block on the Bitcoin block chain.  The minor blocks are there to prove you were working for the entire pool's benefit.

[FreeMoney]

This sounds great.

Would this be a reasonable implementation? Have a checkbox in the client for "Alert me about minor blocks" and if you have it checked and if you have a 0/conf tx it listens to miners and peers who are optionally broadcasting minor blocks and tells you if one or more minor blocks that your client is hearing contain that tx?

Am I roughly understanding?

Seems brilliant.

God I would love to have Seals accept 2 minor confirmations.

[casascius]

This sounds great.

Would this be a reasonable implementation? Have a checkbox in the client for "Alert me about minor blocks" and if you have it checked and if you have a 0/conf tx it listens to miners and peers who are optionally broadcasting minor blocks and tells you if one or more minor blocks that your client is hearing contain that tx?

If P2Pool were the biggest pool and had the majority of the hashing power, then it probably would be a better idea compared to right now where it isn't.

[FreeMoney]

This sounds great.

Would this be a reasonable implementation? Have a checkbox in the client for "Alert me about minor blocks" and if you have it checked and if you have a 0/conf tx it listens to miners and peers who are optionally broadcasting minor blocks and tells you if one or more minor blocks that your client is hearing contain that tx?

If P2Pool were the biggest pool and had the majority of the hashing power, then it probably would be a better idea compared to right now where it isn't.

Do you mean that the downside is that most miners have no incentive to broadcast minor blocks? Unless they happen to be using P2P pool in which case that's how they prove they are mining?

edit: If so, this seems like a somewhat inevitable good thing right? I don't mine though, what is the downside to P2P pool?

[casascius]

Do you mean that the downside is that most miners have no incentive to broadcast minor blocks? Unless they happen to be using P2P pool in which case that's how they prove they are mining?

edit: If so, this seems like a somewhat inevitable good thing right? I don't mine though, what is the downside to P2P pool?

If they aren't using P2Pool, they aren't generating minor blocks.  They are either generating whatever their pool op wants to generate (if in a pool), or are attempting to generate entire Bitcoin blocks worth 50 BTC a pop.

Current downside to P2Pool as far as I know, from the perspective of a miner, is that the setup is way more complex and that they need a whole local blockchain and bitcoin node, as opposed to mining from a diskless headless machine that boots from a live CD.  This is mostly an inconvenience, not a fundamental flaw: P2Pool miners help decentralize the network, traditional pool miners booting from CDs do not.

When mining P2Pool becomes as easy as booting from a CD or USB stick, I will bet more people will do it.  If this "meta tree" thing picks up steam and mining nodes can start mining after only having downloaded and verified a few weeks worth of transaction activity (rather than the whole blockchain), then perhaps P2Pool mining from boot CD's will become feasible.

[Realpra]

All these "second tree" solutions seem awfully complex and like a short term/bad solution.

When, not if, bitcoin reaches VISA volumes you will need like 3 meta trees or something.

Its the most ugly solution I have ever seen, I don't get why people here talk so much about it.


Super nodes are better or other solutions...

[jim618]

In the OP, there are 60 minor blocks in a typical 10 minute window. These will not be in a single tidy sequential chain but multiple splitting branches and twigs. Imagine the trunk of the tree as the last major block and then various branches, smaller branches and twigs sprouting from it.

Note also that this tree is 'burnt to the ground' when a next major block appears. Its structure does not get used by the major blocks.

If my transaction is on the 3rd twig, 4th branch on the left what does it mean ? It means a miner saw it, it is well formed and THAT miner has put it into the self-immolating minor block tree. There may well be a competing transaction (ie a potential double spend) on the 7th twig, 1st branch on the right. Which will get into the next major block ?  The minor block tree gives no indication.

It is an innovative idea but I am not sure it gives much more than knowing the miners saw it ie network propagation info.

[ben-abuya]

In the OP, there are 60 minor blocks in a typical 10 minute window. These will not be in a single tidy sequential chain but multiple splitting branches and twigs. Imagine the trunk of the tree as the last major block and then various branches, smaller branches and twigs sprouting from it.

Note also that this tree is 'burnt to the ground' when a next major block appears. Its structure does not get used by the major blocks.

If my transaction is on the 3rd twig, 4th branch on the left what does it mean ? It means a miner saw it, it is well formed and THAT miner has put it into the self-immolating minor block tree. There may well be a competing transaction (ie a potential double spend) on the 7th twig, 1st branch on the right. Which will get into the next major block ?  The minor block tree gives no indication.

It is an innovative idea but I am not sure it gives much more than knowing the miners saw it ie network propagation info.

That's a pretty good account of what this would look like, however you may be overestimating the "frothiness" factor. I'd be interested to see what the P2Pool structure looks like. It's certainly the case that observing a minor block is not as safe as observing a major block. It's also the case that observing a minor block is way better than looking at a 0/conf transaction floating around. Another thing you could do is to track all the minor blocks, not just the longest chain. I don't really think there will be all that many. Also, the 10 second time frame isn't set in stone, it could be made a bit longer if that makes this a lot more manageable. It probably depends on network speeds and how fast miners verify and jump to a newly released minor block.

Lastly, the main motivation behind this was to gain much greater time resolution after the fact. So we'd want to have that 10 second resolution once the major block has resolved the one true blockchain. I'd love to hear comments on that part, too.

[racerguy]

On average p2pool blocks are orphaned 7% of the time, so not too much branching.

[FreeMoney]

I was not imagining the minor blocks as forming a chain. They'd all just be orphaned/ignored eventually but in the moment you can say to yourself, "Ok, 3 out of 4 minor blocks included the tx I'm interested in and none of them included a different one using the same input, sounds good to me."

The missing piece to me is why should miners broadcast these minor blocks?

[ben-abuya]

On average p2pool blocks are orphaned 7% of the time, so not too much branching.

Awesome! That's a really useful data point.

I was not imagining the minor blocks as forming a chain. They'd all just be orphaned/ignored eventually but in the moment you can say to yourself, "Ok, 3 out of 4 minor blocks included the tx I'm interested in and none of them included a different one using the same input, sounds good to me."

The missing piece to me is why should miners broadcast these minor blocks?

If minor blocks are just regular blocks, and part of the chain, miners would publish them to get rewards, just like major blocks. As I mentioned, this would establish 10 second resolution for transactions, and it would be very useful for more advanced financial blockchains. If minor blocks are really only orphaned about 7% of the time, this could really be feasible. Frankly, with numbers like that, you might not even need to do a dual target, you might be able to get away with just 10 second blocks.

[Meni Rosenfeld]

I don't think this works.

Either the minor blocks are authoritative or they're not. If they are then it's just like having block frequency of 10 seconds with all the disadvantages (wasted hashpower, block header resource cost). If they're not then they do not add to security, a Finney attacker could just hold on to a double-spending major block, and then release it no matter how many minor blocks have been found in the mean time, likewise for >50% attacker.

By the way, p2pool is a way to reduce mining variance with fair reward distribution. It was never touted as a way to get faster security for transactions, and it probably won't work for this purpose for the reasons outlined above.

Also, having very quick confirmations isn't necessary, the blockchain is for final settlement and actual payments will more likely be in some form of Trustless, instant, off-the-chain Bitcoin payments.

Related ideas are Dynamic block frequency and Unfreezable blockchain.

[ben-abuya]

I don't think this works.

Either the minor blocks are authoritative or they're not. If they are then it's just like having block frequency of 10 seconds with all the disadvantages (wasted hashpower, block header resource cost). If they're not then they do not add to security, a Finney attacker could just hold on to a double-spending major block, and then release it no matter how many minor blocks have been found in the mean time, likewise for >50% attacker.

By the way, p2pool is a way to reduce mining variance with fair reward distribution. It was never touted as a way to get faster security for transactions, and it probably won't work for this purpose for the reasons outlined above.

Also, having very quick confirmations isn't necessary, the blockchain is for final settlement and actual payments will more likely be in some form of Trustless, instant, off-the-chain Bitcoin payments.

Related ideas are Dynamic block frequency and Unfreezable blockchain.

Thanks Meni, I hadn't seem some of those links and others merited a re-read -- fascinating stuff. A have a few comments about your reply:

Since my main motivation is time resolution, it doesn't matter if the minor blocks are a priori authoritative, only that they reliably fix the ordering of transactions in time once the major block is broadcast. Even so, I find talk about Finney attacks and 51% attacks a bit distracting. Finney attacks are almost impossible to reliably pull off, and 51% attacks are common to every Bitcoin modification I've heard of. The real issue is race attacks, which are certainly feasible. Minor blocks would make race attacks a lot more expensive than they currently are because you'd have to put in the effort to solve minor blocks (assuming the merchant is willing to wait an average of 10 seconds). That's means they can't be used to get a free cup of coffee at a cafe. If you're buying  a car, the dealer can wait for a few major blocks. On the other hand, it would be great if you didn't have to hang around the dealership for half an hour waiting for the payment to clear.

I also don't buy the suggestion that quick confirmations aren't necessary in Bitcoin. Your proposal is very interesting, but there are lots of tradeoffs involved, including trusting an established payment processor, and putting funds in escrow. These are serious considerations, and if there's a way to do this using the elegance of the blockchain idea, while retaining low trust requirements and no escrow, I think that would be a big win. Dynamic block frequency is very cool. With my arbitrary 10 second interval, I was trying to hit the minimum time that would make sense for current network and processing speeds, but a market approach would be very interesting. Perhaps there could be competing blockchains, each optimized for a different use case. Maybe that's what's beginning to happen with litecoin. Maybe we need somebody to throw a 10 second blockchain out there and see what happens?

There are advantages to the hybrid minor/major block approach (high time resolution, faster confirmations, while retaining the longer term safety of the major blocks), but I'm not at all convinced a Bitcoin fork with a 10 second interval couldn't work better. Maybe I'm missing something but the wasted hash power (assuming 7% orphan blocks) and header overhead don't seem like a huge deal to me.