Next-Gen on-chain scaling idea -- Masterblocks

[Technologov]

I think we may be able to reduce a block chain size dramatically, by re-generating automatically a new genesis block every year, (or every X blocks), and keeping the balance of the previous block chain, last block of it.
Theoretically it will solve the hard disk space issues, And will reduce network bandwidth requirements for setting a new Full node. It will not solve new block propagation delays.

Objective: My primary concern is that with on-chain scaling, our blockchain can grow into a multi-petabyte size, over a few decades, and both network capacity and hard disk space demands will out-grow our ability to support those.

Concept:
2016 [block 0]...[block n] --- the last block's balance could transfer into a new genesis block 2017.
2017 [block 0]

Balance (all utxo) of each address are copied into a new Masterblock as one transaction. Masterblocks are created in a deterministic fashion, so every node can see it and reproduce the correctness of a Masterblock.

Afterwards Mining should start build up on the new chain. Just the history of transactions will be lost.

There could be more types of nodes: Archival node, keeping all history of all the previous block chains, and the Full Node, keeping only the current block chain.

In this scenario, only block explorers and scientists will need to keep an Archival Node. It is not be needed for payments and verification at all.

This idea, if it works, will mean that Bitcoin will beat Visa x 10 transactions, with a goal to break 1 billion transactions per day, in a decentralized way.

NOTE1: Blockchain pruning, as is done in Bitcoin has very bad side effects: namely inability to start new nodes from them and doing full rescan. My approach doesn't have such downsides. https://news.bitcoin.com/pros-and-cons-on-bitcoin-block-pruning/

Clarification 01: New Term: Let's rename our so-called new Genesis Block into a Masterblock. (for clarity)

Clarification 02: New Masterblock must be generated in a deterministic way, to be reproducible by all miners and Full Nodes.

Clarification 03: tiny dust amounts  can be distributed to the last 1000 miners.
Let's define 'dust' as transaction outputs  below the median fees for the last 100 blocks.
(this removes a bunch of bloat from the blockchain)

Clarification 04: Old chain doesn't get removed immediately, but only after Y blocks (say after 1 month).
This means that new nodes connecting to the network during this period can download both new chain and old chain, can verify the new Masterblock by recalculating it from the last year's chain.

Clarification 05: Q: What do new nodes do when they come online after this one month has passed? How do they verify the correct chain from scratch?
A: Consensus must work as follow:

1. Same as nowadays (longest chain) AND
2. current timestamp (of year 2017) - proof of work must include timestamp.
(if it finds a very long chain, but with old timestamp like 2015 or 2016, it gets rejected; future timestamps also get rejected)


TODO: Research new attack vector: NTP protocol / wrong time. (For now my idea requires the administrator to manually setup clocks on critical production systems, such as payment processors. Maybe we can solve this problem later.) -- Perhaps we can solve it via an idea of cryptographic time -- a separate block-chain for time-keeping, with the same SHA256-proof-of-work and same difficulty as the main-chain, that doesn't reset on each Masterblock -- it's continuous. It should be merge-mined together with the primary block chain. Basically Segregated Time - 'SegTime'.

NOTE: New problem: My idea, as written now, will invalidate special tx-outputs, such as lock-time and multisig. But perhaps there is an elegant solution to this problem too... -- something like an extended block, that will be written into the Masterblock, and will list all previous special transactions in unspent tx outputs as valid.)

================
Okay, Let's do some maths:
4 GB block each 10 min = . This will allow us for 1 billion tx/day. In Bitcoin it would equal to 144 blocks/day x 4 GB/block = 576 GB/day.

It's 576 GB/day x 365 (year) = 210 TB-per-year.

Without my idea we will grow into a multi-petabyte territory in 5 years. Will be hard, even with good server-grade hardware.
With my idea, it would take only 18 Hard Disks (okay 20 HDDs, with RAID6) to keep an entire block chain (that's assuming big 12 TB HDDs; that both Seagate and Western Digital started producing this year).

Block propagation ? It takes only 8 seconds on a Google Fiber (1 Gbit/s Internet) -- so I believe it's very much possible and feasible to grow with on-chain transaction scalability.
================

What do you think of it?

-Technologov
05.Mar.2017
UPDATED on 02.04.2017

Link:
https://github.com/BitcoinUnlimited/BitcoinUnlimited/issues/340

[1714667596]

1714667596

[1714667596]

1714667596

[1714667596]

1714667596

[1714667596]

1714667596

[jonald_fyookball]

Have you actually calculated how big the master block would need to be currently need to be , as well as upper bounds for such a block using today's 'dust parameters' ?

[spartacusrex]

Like.

BUT. Could be easier. Don't need a Master Block.

1) Throw away EVERYTHING except the Block-header for blocks older than 1 day / (..or the longest re-org imaginable ). The block header is the ONLY bit you really need - as it has the POW. Also it links back to the previous block header, so we have our 'Proof Chain' back to the Genesis block. (Which would be tiny). But still packed with ridiculous levels of POW-er..

2) You need to store the deterministic UTXO root hash somewhere in the block header. Changes for every block obviously. But everyone would agree on  it.

3) That's it.

As with your design we lose the TXN history - but we keep the POW chain that led us to the present day UTXO.

So.. doing the Maths.. it would be about :

576GB (as you calculated for 1 billion txn a day)+ UTXO database size (orders of magnitude smaller than 0.576TB) + the proof chain size (tiny).. forever.

Whether there would be any point to keeping the TXN history - is debatable..

[jonald_fyookball]

some paper napkin math on upper bounds:

Lets say you need 100 bytes to store an address/balance pair, and let's assume currently the smallest amount non dust amount is 0.0001 BTC (about 10 cents).   If we use a round number
of 20M BTC, then the maximum number of buckets (addresses) used would  200 billion possible buckets.  Multiply by 100 bytes you get 20 trillion bytes or roughly 20 TB.

I don't know that this is currently better than pruned UTXO... and i think people would take issue with destroying dust... maybe there could be some way to combine dust if you dont
want to lose it later by specifying in a transaction. hmmmmmm

[Technologov]

Thanks a lot.

Especially "spartacusrex" for his interesting idea.

[K128kevin2]

I think that there are too many ideas and proposals for scaling. F2pool's April fools joke showed this. It fragments the community.

[Jet Cash]

What a great idea, we could ask Soros or the Rothschilds to look after the master block for us.

[ddosamerica7]

Like.

BUT. Could be easier. Don't need a Master Block.

1) Throw away EVERYTHING except the Block-header for blocks older than 1 day / (..or the longest re-org imaginable ). The block header is the ONLY bit you really need - as it has the POW. Also it links back to the previous block header, so we have our 'Proof Chain' back to the Genesis block. (Which would be tiny). But still packed with ridiculous levels of POW-er..

2) You need to store the deterministic UTXO root hash somewhere in the block header. Changes for every block obviously. But everyone would agree on  it.

3) That's it.

As with your design we lose the TXN history - but we keep the POW chain that led us to the present day UTXO.

So.. doing the Maths.. it would be about :

576GB (as you calculated for 1 billion txn a day)+ UTXO database size (orders of magnitude smaller than 0.576TB) + the proof chain size (tiny).. forever.

Whether there would be any point to keeping the TXN history - is debatable..

Your blockchain size would be incredibly smaller because the entire "4GB" block wouldnt even be completely used.

[d5000]

That proposal is a bit similar to "Ardor", an altcoin currently in the testing phase. In this solution there is a main chain with very few transactions that is preserved by all full nodes - but regular payments occur at a kind of "sidechain" (called Ignis) from which only the last blocks are fully validated by all nodes, and that is integrated in the main chain by a special kind of transaction.

So the effect is the same - to sync, you will have to download only the most recent part of the blockchain.

Maybe something similar could be implemented in Bitcoin, too.