New paper: Accelerating Bitcoin's Trasaction Processing

[AsymmetricInformation]

..

Would it be possible to implement this directly from the Bitcoin blockchain, creating a split? Where the current holders of bitcoins now are also the holders of Bitcoin 2.0? And then people will be vested in both with a market between the two, eventually finding out which one "wins."

This is how I would go about it when making an altcoin, too. But for most altcoiners the appeal is that they can be early adopters with their new coin. You don't have that with the bitcoin 2.0 approach.

Still, I'm not so sure how good it would be to fragment the Bitcoin economy.

Just fragment it once, on say Feb 1, 2014, to test it out and see how everything is working, with the complete understanding of everyone that, if the 'treecoin' implementation works, you will 'fragment' again on March 1st, 2014, with the intent of hard-forking THAT into new Bitcoin.

Therefore, passive users can coordinate to use Bitcoin Chain, knowing that their BTC will convert to Bitcoin Tree on March 1st if all is well, and not worry about any messy stuff in between. Those using Treecoin between Feb and March will get them for free but know that they will eventually lose them, making for a perfect test environment and no financial instability. Best of both worlds.

[1714709486]

1714709486

[1714709486]

1714709486

[1714709486]

1714709486

[1714709486]

1714709486

[nomailing]

..

Would it be possible to implement this directly from the Bitcoin blockchain, creating a split? Where the current holders of bitcoins now are also the holders of Bitcoin 2.0? And then people will be vested in both with a market between the two, eventually finding out which one "wins."

This is how I would go about it when making an altcoin, too. But for most altcoiners the appeal is that they can be early adopters with their new coin. You don't have that with the bitcoin 2.0 approach.

Still, I'm not so sure how good it would be to fragment the Bitcoin economy.

Just fragment it once, on say Feb 1, 2014, to test it out and see how everything is working, with the complete understanding of everyone that, if the 'treecoin' implementation works, you will 'fragment' again on March 1st, 2014, with the intent of hard-forking THAT into new Bitcoin.

Therefore, passive users can coordinate to use Bitcoin Chain, knowing that their BTC will convert to Bitcoin Tree on March 1st if all is well, and not worry about any messy stuff in between. Those using Treecoin between Feb and March will get them for free but know that they will eventually lose them, making for a perfect test environment and no financial instability. Best of both worlds.

This is a very good suggestion, how a hardfork should be implemented.

[Natanael]

Are anybody interested in a notary based system with temporary pay-to-script-hash wallets (P2SH) using multisig transactions?

My scheme requires no Bitcoin protocol change, and requires very limited trust in notaries which can be replaced quickly and where malice is easy to prove (just show two conflicting transactions that both got signed, these can be distributed globally in seconds to everybody that has set their clients to trust that notary). This means that notaries has practically nothing at all to gain on an attack, and still makes 0-confirmation transactions trustable.

http://roamingaroundatrandom.wordpress.com/2013/11/30/bitcoin-idea-temporary-notarized-wallets-secure-zero-confirmation-payments-using-temporary-notarized-p2sh-multisignature-wallets/

[jl2012]

I am not sure if I get it correct, but I think we can implement it like this:

1. Let say there are two miners, Alice and Bob, finding blocks of the same height at almost the same time.

2. All miners see both blocks. However, some see Alice's block first, some see Bob's block first.

3. Those seeing Alice's block first will by default mine on top of Alice's block. However, they will also include Bob's block header in the coinbase. (The hash for the previous block could be omitted to save some space)

4. Those seeing Bob's block will similarly put Alice's block header in the coinbase.

5. No matter what the next block is building on, the header of the orphaned block is permanently and irreversibly recorded in the blockchain.

6. The total PoW of the chain should also count the orphaned blocks, given that they satisfy the difficulty. The actual content of the orphaned block is irrelevant since it is only used as a confirmation to the previous block.

There is a drawback: In case there is a fork, SPV nodes have to read the coinbases in order to determine the longest chain. With 1 second blocks that would mean tons of branches and SPV nodes may have some trouble.

I think it is quite workable. We may even implement it without changing the rules first, and see how it works out. It would help strengthening the network even if we keep the 10 minutes block

[Cryddit]

How about FOUR THOUSAND (or even FORTY THOUSAND) transactions per second?

This is so cool!  I had come up with a scalability solution of my own earlier this week, and this one MULTIPLIES by it!  

A fundamental change in protocol design has been exactly what Bitcoin needs to continue to be viable as it grows.  This will do it.  Actually either of these will do it.  But combining them is AWESOME!

By combining these two ideas we could get THOUSANDS of transactions per second.  

My scalability solution was to split wallets into multiple "channels" and have one blockchain for each pair of channels.  So when you get to the point where 6 channels (15 blockchains) is having trouble coping with transaction volume, you can shift to 7 channels (21 blockchains).  Every client would be listening on one channel (5 blockchains before the additional channel is added; 6 blockchains afterward), and each channel would contain a total of 1/6 (or 1/7) the total tx volume.  By the time you get up to 20 channels (190 blockchains) you could handle a thousand tx per second even with Bitcoin's current speed.

You'd need one additional blockchain (probably with an extremely low volume of transactions) which would be in all  channels, to handle the rare tx that involves wallets in *more* than 2 different channels (all of them much more complicated than the standard pay-to-script-hash). So each single-channel client would actually be listening to 6 (or 7, or 20) blockchains - but the last one nearly negligible in bandwidth and volume.  

These blockchains would back each other up; every time a new block appears in any blockchain it would list the hashes of the most recent block so far found in all other blockchains.  The mining awards would simply be split between blockchains.  Thus all blockchains would be "merge mined" together.

One downside is that any particular transaction would only be checked by the clients on two channels rather than by all clients.   IE, all users would be relying on tx checked by 1/3 (or 2/7, or 1/10) of total users, so it's no longer a completely trustless model; you'd be trusting a few thousand other users not to collude to deceive you, rather than trusting that ALL of them would not collude to deceive you.

Upsides:

It would be easier to "throttle" bandwidth without refusing altogether to participate, simply by limiting the number of blockchains in which you advertise full node (or "download this blockchain from me") service.  Bandwidth costs could even be made much less onerous than that; see the second point below.

It would be easier to run a fully participating node for one channel.  It would use only 1/6 or 1/7 the total bandwidth and tx checking CPU load, and only 1/6 or 1/7 (or far less; see the next point) the storage, of Bitcoin's single blockchain architecture, spread across all blockchains in your channel).  

There is a huge privacy and bandwidth enhancement:  The length of a particular blockchain could be limited. You could at any time transfer all of a blockchain's UTXO to a different blockchain (and everybody could check that you did it correctly), shut down the now-empty blockchain, and start a new one to replace it.  Effectively this would amount to a very aggressive "pruning" strategy for releasing transaction history -- which would also be a huge contribution to financial privacy and a huge savings in bandwidth and storage.  At any given time, the blockchains in use might average only a week or so old.  Only one blockchain, probably the "all-channels chain" which would have a very light transaction history anyway, would need to stretch all the way back to the Genesis Block.  Of course this wouldn't help against an opponent who keeps copies of all historical blockchains, but it would preclude downloading all transactions in history from any full node, once the blockchain those tx were on has been pruned.

Any particular user who wants to would have the choice to support the protocol more fully by subscribing to multiple (or all) channels if they have enough bandwidth and storage to do so.

Now, what's awesome about this is that these two enhancements multiply by each other. If we can get *EACH BLOCKCHAIN* up to 200 tx per second, then multiplying the number of blockchains as outlined above would be many times as effective.  

With 7 channels, or 21(22) blockchains, we'd be up to over 4000 tx/second, rather than the 150 or so I had been envisioning. With 20 channels (190 blockchains) you could be approaching 40,000 tx/second rather than the 1400 or so I had been envisioning.  

[nomailing]

How about FOUR THOUSAND (or even FORTY THOUSAND) transactions per second?

This is so cool!  I had come up with a scalability solution of my own earlier this week, and this one MULTIPLIES by it!  

A fundamental change in protocol design has been exactly what Bitcoin needs to continue to be viable as it grows.  This will do it.  Actually either of these will do it.  But combining them is AWESOME!

By combining these two ideas we could get THOUSANDS of transactions per second.  

My scalability solution was to split wallets into multiple "channels" and have one blockchain for each pair of channels.  So when you get to the point where 6 channels (15 blockchains) is having trouble coping with transaction volume, you can shift to 7 channels (21 blockchains).  Every client would be listening on one channel (5 blockchains before the additional channel is added; 6 blockchains afterward), and each channel would contain a total of 1/6 (or 1/7) the total tx volume.  By the time you get up to 20 channels (190 blockchains) you could handle a thousand tx per second even with Bitcoin's current speed.

You'd need one additional blockchain (probably with an extremely low volume of transactions) which would be in all  channels, to handle the rare tx that involves wallets in *more* than 2 different channels (all of them much more complicated than the standard pay-to-script-hash). So each single-channel client would actually be listening to 6 (or 7, or 20) blockchains - but the last one nearly negligible in bandwidth and volume.  

These blockchains would back each other up; every time a new block appears in any blockchain it would list the hashes of the most recent block so far found in all other blockchains.  The mining awards would simply be split between blockchains.  Thus all blockchains would be "merge mined" together.

One downside is that any particular transaction would only be checked by the clients on two channels rather than by all clients.   IE, all users would be relying on tx checked by 1/3 (or 2/7, or 1/10) of total users, so it's no longer a completely trustless model; you'd be trusting a few thousand other users not to collude to deceive you, rather than trusting that ALL of them would not collude to deceive you.

Upsides:

It would be easier to "throttle" bandwidth without refusing altogether to participate, simply by limiting the number of blockchains in which you advertise full node (or "download this blockchain from me") service.  Bandwidth costs could even be made much less onerous than that; see the second point below.

It would be easier to run a fully participating node for one channel.  It would use only 1/6 or 1/7 the total bandwidth and tx checking CPU load, and only 1/6 or 1/7 (or far less; see the next point) the storage, of Bitcoin's single blockchain architecture, spread across all blockchains in your channel).  

There is a huge privacy and bandwidth enhancement:  The length of a particular blockchain could be limited. You could at any time transfer all of a blockchain's UTXO to a different blockchain (and everybody could check that you did it correctly), shut down the now-empty blockchain, and start a new one to replace it.  Effectively this would amount to a very aggressive "pruning" strategy for releasing transaction history -- which would also be a huge contribution to financial privacy and a huge savings in bandwidth and storage.  At any given time, the blockchains in use might average only a week or so old.  Only one blockchain, probably the "all-channels chain" which would have a very light transaction history anyway, would need to stretch all the way back to the Genesis Block.  Of course this wouldn't help against an opponent who keeps copies of all historical blockchains, but it would preclude downloading all transactions in history from any full node, once the blockchain those tx were on has been pruned.

Any particular user who wants to would have the choice to support the protocol more fully by subscribing to multiple (or all) channels if they have enough bandwidth and storage to do so.

Now, what's awesome about this is that these two enhancements multiply by each other. If we can get *EACH BLOCKCHAIN* up to 200 tx per second, then multiplying the number of blockchains as outlined above would be many times as effective.  

With 7 channels, or 21(22) blockchains, we'd be up to over 4000 tx/second, rather than the 150 or so I had been envisioning. With 20 channels (190 blockchains) you could be approaching 40,000 tx/second rather than the 1400 or so I had been envisioning.  

This is a very nice idea. How would you make sure that all these blockchains have the same monetary base?

I thought into a similar direction.
It would be a good idea to order the channels according to geographic locations or according to different industries. For example you could have bitcoin addresses like these:
/btc/1CiZPrEJdN4FJcqdLdgVLzT8tgCXxT5ion
/btc/europe/1CiZPrEJdN4FJcqdLdgVLzT8tgCXxT5ion
/btc/europe/spain/1CiZPrEJdN4FJcqdLdgVLzT8tgCXxT5ion
/btc/usa/california/1CiZPrEJdN4FJcqdLdgVLzT8tgCXxT5ion
The child blockchains are represented as unique addresses within their parent blockchain. The top blockchains will have much less TPS because most transactions can be assumed to take place within the same geographic region and therefore take place within the same sub-chains.
The child and parent blocks would always link to each other to determine the transaction processing between different chains.
The total coin-generation could then be distributed from the main /btc chain to child chains according to their stake.
If a user wants to run a full node in chain x, then he has to run a full node in all parents of x and at least SPV nodes in the direct child chains of x.

When you would create a transaction from chain /btc/usa to chain /btc/europe it would go like this:
You submit a transaction to the subchain /btc/usa with a script that says "move coins to parent and then to address xyz in subchain europe".
The transaction is first processed by a miner in the subchain /btc/usa and included in a block. The parent chain /btc sees the transaction, because all nodes in the parent have to run at least an SPV in the child. Then the parent chain includes it in a block. Internally within the parent this is just recorded by transferring some coins between the monetary base of one subchain to the other subchain. Then the childchain /btc/europe sees the transaction in the block of the parent chain and includes it so that the coins finally arrive at the destination address.

Maybe your approach is better, because it would allow direct transactions between wallets in all different channels. Maybe it is also possible to combine your approach with a tree structure?

So for example you could have a parent blockchain and channel which is called /btc. This would manage the total money supply and the distribution of this money supply between all your child-channels.
Then there are 3 child channels (/btc/usa, /btc/europe, ...) and thus 9 blockchains which handle the transactions between these channels. You could further add child channels and thereby you reduce your total number of required blockchains from 190 to something more reasonable.

[Cryddit]

Well, you could assign channels to geographic regions or industries, but each blockchain is shared by 2 channels.  So if a client in europe (channel 3 let's say) wants to transmit money to a client in America (channel 2) they'd need to use blockchain 2.3, which is distributed on both continents.  For example with 6 channels you might have:

channel 1 blockchains: 1.2 ,1.3, 1.4, 1.5, 1.6  (asia)
channel 2 blockchains: 1.2, 2.3, 2.4, 2.5, 2.6  (north america)
channel 3 blockchains: 1.3, 2.3, 3.4, 3.5, 3.6  (europe)
channel 4 blockchains: 1.4, 2.4, 3.4, 4.5, 4.6  (south america)
channel 5 blockchains: 1.5, 2.5, 3.5, 4.5, 5.6  (africa)
channel 6 blockchains: 1.6, 2.6, 3.6, 4.6, 5.6  (australia)

and all channels share blockchain zero.

(which is a total of 15 blockchains, not 36)

But no blockchain would be restricted to just one continent, because every channel has one blockchain (besides zero) in common with every other channel.  The idea is that because most transactions involve just two people, you will very rarely need a blockchain that spans more than two channels.  blockchain zero is reserved for transactions involving more than two people.

Still, your idea is valid; someone in channel 3 could give you a bitcoin.europe.Gux8854.... payment link to click on. But there's no way to enforce such a breakdown; the guy with the channel 3 address might live in Australia.

[nomailing]

Well, you could assign channels to geographic regions or industries, but each blockchain is shared by 2 channels.  So if a client in europe (channel 3 let's say) wants to transmit money to a client in America (channel 2) they'd need to use blockchain 2.3, which is distributed on both continents.  For example with 6 channels you might have:

channel 1 blockchains: 1.2 ,1.3, 1.4, 1.5, 1.6  (asia)
channel 2 blockchains: 1.2, 2.3, 2.4, 2.5, 2.6  (north america)
channel 3 blockchains: 1.3, 2.3, 3.4, 3.5, 3.6  (europe)
channel 4 blockchains: 1.4, 2.4, 3.4, 4.5, 4.6  (south america)
channel 5 blockchains: 1.5, 2.5, 3.5, 4.5, 5.6  (africa)
channel 6 blockchains: 1.6, 2.6, 3.6, 4.6, 5.6  (australia)

and all channels share blockchain zero.

(which is a total of 15 blockchains, not 36)

But no blockchain would be restricted to just one continent, because every channel has one blockchain (besides zero) in common with every other channel.  The idea is that because most transactions involve just two people, you will very rarely need a blockchain that spans more than two channels.  blockchain zero is reserved for transactions involving more than two people.

Still, your idea is valid; someone in channel 3 could give you a bitcoin.europe.Gux8854.... payment link to click on. But there's no way to enforce such a breakdown; the guy with the channel 3 address might live in Australia.

I thought it makes more sense to have the chains you described but in addition 6 chains individually for transactions that take place within the channel. So you would have blockchain 0 and 36 subchains:
channel 1 blockchains: 1, 1.2 ,1.3, 1.4, 1.5, 1.6  (asia)
channel 2 blockchains: 2, 1.2, 2.3, 2.4, 2.5, 2.6  (north america)
channel 3 blockchains: 3, 1.3, 2.3, 3.4, 3.5, 3.6  (europe)
channel 4 blockchains: 4, 1.4, 2.4, 3.4, 4.5, 4.6  (south america)
channel 5 blockchains: 5, 1.5, 2.5, 3.5, 4.5, 5.6  (africa)
channel 6 blockchains: 6, 1.6, 2.6, 3.6, 4.6, 5.6  (australia)

A transaction from channel europe to channel asia will only be recorded in blockchain 0 and blockchain 1.3.
A transaction from channel europe to channel europe will only be recorded in blockchain 0 and 3.

The header of subchain europe-asia will include a field that states the total transaction volume between channels europe to asia or vice versa.
Then channel 0 will include the header of the subchain and directly knows how many coins are within each individual channel.

In comparison to your proposal it has the advantage that someone in channel asia does not need to know a transaction which takes place from channel europe to channel europe. As I read your proposal everyone would have to record transactions which take place within another channel. Therefore I think it makes sense to have one chain per channel and in addition what you proposed a chain per pair of channels.

You can then repeat the procedure and create channels /europe/spain and /europe/italy, but now the channel /europe would act as the parent chain which just records the coin flow between the channels but not within the channels. With this tree structure it would be possible to have a lot of channels with a linear relationship between number of channels and number of chains. For example if the channels above would have each another 6 subchannels you have a total of 1+6+36 (top+second+third level) channels and in total only 1+36+216 chains.

[Cryddit]

Nah, if you have a chain that's dedicated to a single channel then transactions in that chain only get checked by half as many people.  Besides, it's better for people transacting within a single channel to "load balance" by picking the blockchain in their channel that has the smallest number of tx in the memory pool.

And transactions that are served by one of the two-channel blockchains never need to be in blockchain zero; that's just for tx that span 3 channels or more.  Which, in practice, is almost none of 'em.

Edit: you seem to be talking about a different approach: a "tree" of subchannels, each node of the tree having its own blockchain.  I considered that, but in that scheme it's really hard for two people (or more) who want to make a transaction to find a blockchain that they have in common where the transaction can be made.  You have to have both of them going up the tree to the lowest tree node that they have in common, which would concentrate fully a quarter of the traffic on the root blockchain, split another quarter between its two subchains, split an eighth of the traffic between their four second-level subnchains, etc.  That won't scale past about four times what Bitcoin can handle now, because the root blockchain of the tree scheme would become a bottleneck.

[CanaryInTheMine]

How can this modification be used to exploit Bitcoin?  Could this change become a vector for shenanigans later?

When you can't beat it, start influencing technology selection and whoila, eventually you get your backdoors...

paranoia or food for thought?

[Daily Anarchist]

How can this modification be used to exploit Bitcoin?  Could this change become a vector for shenanigans later?

When you can't beat it, start influencing technology selection and whoila, eventually you get your backdoors...

paranoia or food for thought?

It's a valid concern. I don't think anybody wants to sacrifice long-term security for quicker transactions. If it gets past its initial stage of acceptance it will be heavily debated. Worst case scenario we could just fork the chain, which isn't bad in my opinion.

[johnyj]

I read the paper, it seems not very clear to me, hope there are several real examples without using formulas, it is easy to miss some special situation if only using generalized formulas

[johnyj]

How can this modification be used to exploit Bitcoin?  Could this change become a vector for shenanigans later?

When you can't beat it, start influencing technology selection and whoila, eventually you get your backdoors...

paranoia or food for thought?

A hard fork will most possibly be rejected by miners, unless they have enough motivation and reached a consensus

[Voodah]

How can this modification be used to exploit Bitcoin?  Could this change become a vector for shenanigans later?

When you can't beat it, start influencing technology selection and whoila, eventually you get your backdoors...

paranoia or food for thought?

That's a totally sound and valid reason to worry.

I guess, if that were the main concern after the paper is assessed to be desirable, the idea of forking the Bitcoin network as it stands with its current volume, looks like a much better option than an altogether new alt-coin. We could instantly be testing on high transaction volume.

[johnyj]

From a miner's perspective, I don't see how 1 block per second is possible: It takes at least 1-2 second to re-initialize the miner and get new work from the network, if new blocks arrive every second, it means the miner would never get ready for the real calculation to take place 

[adpinbr]

great lets take something extremely complex and misunderstood, and right when people start to understand it and believe in it, lets change it. Yea thats where i want to keep my money. Sarcasim aside how will this effect the distribution of bitcoins?

[ffe]

... it's really hard for two people (or more) who want to make a transaction to find a blockchain that they have in common where the transaction can be made.  You have to have both of them going up the tree to the lowest tree node that they have in common, which would concentrate fully a quarter of the traffic on the root blockchain, split another quarter between its two subchains, split an eighth of the traffic between their four second-level subnchains, etc.  That won't scale past about four times what Bitcoin can handle now, because the root blockchain of the tree scheme would become a bottleneck.

This assumes no locality to the transactions. If transactions aggregate (by geography for example or by business type) then people will often find blockchains in common near the leaves without needing to go to the root (less often than the 25% of the time you indicate one hopes.)

Very interesting. Reminds me of banks and clearing transactions between banks within a country vs. transactions that require a transfer between banks in different countries.

[gmaxwell]

From a miner's perspective, I don't see how 1 block per second is possible: It takes at least 1-2 second to re-initialize the miner and get new work from the network, if new blocks arrive every second, it means the miner would never get ready for the real calculation to take place 

Yep most hardware in use for mining has second to multisecond latencies, not even getting into the latencies of global propagation in a network of decentralized volunteers (rather than trivially coerced corporate interests).

But I think if you fixate on one second you're not getting the value of of the paper. Just multiply their numbers by 10 if you find 1 second is a bit outrageous considering the practicalities that their analysis excluded.

[johnyj]

From a miner's perspective, I don't see how 1 block per second is possible: It takes at least 1-2 second to re-initialize the miner and get new work from the network, if new blocks arrive every second, it means the miner would never get ready for the real calculation to take place  

Yep most hardware in use for mining has second to multisecond latencies, not even getting into the latencies of global propagation in a network of decentralized volunteers (rather than trivially coerced corporate interests).

But I think if you fixate on one second you're not getting the value of of the paper. Just multiply their numbers by 10 if you find 1 second is a bit outrageous considering the practicalities that their analysis excluded.

I will read the full paper more carefully, just a glance from the paper:

"Practically, individual nodes’ limited bandwidth imposes an additional constraint on these parameters. If every node’s bandwidth is at least 0.427 MB per second, then the network is able to maintain a rate of 1 blocks per second and b = 320 KB per block, with 10000
transaction hashes per block, achieving a TPS of more than 214.0"

If drop this value from 1 block per second to 1 block per 30 second, then TPS will drop to 7, the same as current design

[gglon]

If drop this value from 1 block per second to 1 block per 30 second, then TPS will drop to 7, the same as current design

You need to multiply blocksize b as well.