Building the Next Generation of Crypto-Currency (developers required)

[bitfreak!]

The Mini-Blockchain Project

What is this project all about?

The goal of this project is to implement a new crypto-currency protocol designed to solve the "blockchain bloat" problem once and for all by replacing the full blockchain with a finite "mini-blockchain". Such a system would offer many benefits including consistently fast synchronization times, much more block space, faster transactions, and lower fees. Certainly not an easy problem to solve, but if it could be solved we are looking at truly new and unique improvements over the Bitcoin protocol.

We now believe that this problem finally has been cracked. The concept and technical details of the proposal can be found in the white paper (a little outdated now) and the project wiki. This new scheme has been analyzed by many intelligent people and we haven't found any major flaws. After a considerable amount of feedback, myself and several others are convinced the scheme is viable and we want to attempt an implementation.

Our objectives and approach

The main objective of this project is to bring together many new ideas and concepts while staying as true to Bitcoin as possible. This project will attempt to minimize and exclude any controversial proposals and ideas but remain open to new and experimental proposals which we believe could enhance the final system. We aim to take the safe and trusted route ever possible and avoid over-complicating the scheme by bringing together too many different ideas.

The main focus of this project is to implement the mini-blockchain protocol and show that it works before we do anything else. If or when that goal is achieved we will start looking at incorporating several other experimental concepts which have the potential to provide an array of new features and benefits. Some examples of such concepts include secure 0-confirmation transactions based on a withdrawal limit system and a dynamic max block size.

Since we are trying to avoid any controversial and unnecessary complications, that means no pre-mine and probably no PoS integration. The PoS system is controversial in a way because it appears to make the rich richer over time and the benefits of such a system really aren't observable for a long time anyway. Not to mention the mini-blockchain proposal is already complicated enough as it is, keeping things simple is the key here.

The next generation of crypto-currency

What is the next generation of crypto-currency and what separates this project from any other alt coin? Most of you will agree that 95% of the alt coins out there are essentially pump and dump scams which offer nothing new or useful. I would personally say there are less than half a dozen alt coins which shouldn't be labeled scam coins; Namecoin, Devcoin, PPCoin, Novacoin, and Litecoin, and perhaps 1 or 2 other alt coins. I'm not so sure about PPCoin and Novacoin but they do offer new PoS features.

Litecoin doesn't really offer much beyond faster block confirmation speeds and scrypt-based mining, but since it was one of the first alt coins to offer these new features it has gained a genuine foothold in the crypto-currency market. Not only does this project offer a new coin with unseen features, but those new features are not small or trivial. Imagine always being able to download the blockchain within seconds or minutes as well as significantly faster transaction speeds.

Why am I making such a big deal of this? Well consider threads like this:
WARNING! Bitcoin will soon block small transaction outputs
New video: Why the blocksize limit keeps Bitcoin free and decentralized

All this concern over max block sizes, the SD "spam dust", the transaction capacity reaching its limit, all of this is nicely solved with a mini-blockchain + account tree implementation. Now I'm not saying Bitcoin is obsolete or outdated or that this new scheme will replace it, Bitcoin still has certain advantages over the mini-blockchain scheme and vice versa. However if this scheme turns out to work it does have the potential to seriously compete with bitcoin.

It was only a matter of time before someone was able to improve on the Bitcoin protocol and offer a new coin with highly desirable advantages over the original protocol, but I don't think that will mean the original will ever die out. Just because this scheme can offer more scalability and speed doesn't make it better as a tool for storing wealth. It does sort of make it better as a currency though, which is really what we should be aiming for I believe.

So how will the project be organized?

See: [BOUNTY] $20,000 Mini-Blockchain Implementation

Project Address: 1AZjrg6h9nfvFt16kaszLTJkQi13kMwZz2

[NickCoin]

This idea sounds great, would love to be part of it as the designer!

[bytemaster]

There is potentially synergy between this blockchains features an my economic model for offering a crypto-USD, crypto-GOLD, etc that maintains near parity without having any counter party risk.   If a new chain were to be created I believe it should include as many 'good ideas' as possible because we will be stuck with it for a while if it works.

I will investigate your proposal and would appreciate if you would investigate my p2p bank/exchange system.

[bitfreak!]

There is potentially synergy between this blockchains features an my economic model for offering a crypto-USD, crypto-GOLD, etc that maintains near parity without having any counter party risk.   If a new chain were to be created I believe it should include as many 'good ideas' as possible because we will be stuck with it for a while if it works.

I will investigate your proposal and would appreciate if you would investigate my p2p bank/exchange system.

I actually took a look at your thread a few hours ago and your idea seems fairly complex. I would recommend writing a white paper or something. While I agree it may be beneficial to mix together as many good ideas as possible it's still problematic. At this point my goal is simply to just implement this mini-blockchain scheme to see if it actually works. The job is already difficult enough and I don't want to go including all these other untested ideas, it will just make the process longer and harder. As I said, the goal of this project is really just to stay as close to the bitcoin protocol as possible and just play it safe. I would recommend the same thing with your project as well, first just build it as you imagine it and see if it actually performs as you believe it will. When you try to start off with too much on your plate it can often lead you to troubles. At some later point maybe we can have a mini-blockchain + PoS + your exchange model (although I'm still skeptical of the PoS stuff and I'd need to understand your proposal better for actually backing it), but right now that isn't the best course of action to take in my opinion.

[bytemaster]

There is potentially synergy between this blockchains features an my economic model for offering a crypto-USD, crypto-GOLD, etc that maintains near parity without having any counter party risk.   If a new chain were to be created I believe it should include as many 'good ideas' as possible because we will be stuck with it for a while if it works.

I will investigate your proposal and would appreciate if you would investigate my p2p bank/exchange system.

Not to derail your design, but it is very complex (perhaps more-so than necessary).

Given the current bitcoin block chain, you can deterministically generate the min set of all 'outputs' that are still valid.  This will generate a very small list (compared to the block chain).  This list would include the hash of the transaction + the output index + script + balance + block num.   It would drop all of the inputs to the transaction and all spent outputs.  Furthermore, outputs with 'identical scripts' could in theory be compressed into a deterministic lookup table.

The hash of this list could then be included in the merkle tree of the existing block chain.  Because all nodes can deterministically calculate the same list they can all validate the hash of the list as part of the block-chain validation.

Anyone can then fetch this list and have just as much confidence in the validity of the outputs as anyone who had the full block chain.   All that would be required is to store the block-headers for the full chain + the output summary list of your 'origin' + however many blocks with full transactions you feel you need.

Thus a light-weight client could constantly prune transaction history and just keep 120 blocks, while full up banks could hold the entire chain for all of history.  

I believe this solution would be cheaper, easier, and more flexible than your solution.  It requires no significant changes to bitcoin other than generating a deterministic output summary and hashing it and then including that hash in the merkel tree.  The table could be trusted by all because otherwise the block would have been rejected by other nodes.

Such a change to bitcoin would be so 'small' that it could probably be implemented with a well-planned hard-fork.  The hard fork being validating the 'summary hash' as part of block verification.  That said, it could be a soft-fork that only miners need to be aware of because blocks could still be generated that would be accepted by old and new clients and miners supporting the new feature could only accept blocks with valid 'summary hashes' or blocks with 'no summary hashes' and then new clients could trim the block chain at any block that includes a valid summary hash.  This would only require proof that 90% of the past 10000 blocks contain valid summary hashes to be sure that no block with an invalid summary hash could possibly make it into the longest chain without sustained 51% attack.  Once 90% of miners are including valid summary hashes, they can start rejecting blocks (and miners) that do not include it.

I will probably implement this as part of BitShare currency rather than your proposed approach... that is unless you can show me something I am missing.

[bitfreak!]

Not to derail your design, but it is very complex (perhaps more-so than necessary).

Well I'm not sure what you just said is any less complex, it certainly confused me. But if what you are saying is valid, you are claiming that a small change to the existing Bitcoin protocol can basically provide something similar to a finite mini-blockchain? I'm certainly very skeptical about that, I don't think my system is overly complex, I believe it has the minimum components required to make the mini-blockchain scheme secure and robust.

[justusranvier]

There is potentially synergy between this blockchains features an my economic model for offering a crypto-USD, crypto-GOLD, etc that maintains near parity without having any counter party risk.

Based on this statement I'm not convinced that you understand counterparty risk.

[bytemaster]

Ok, I can summarize my solution as such:

Given the bitcoin blockchain C define a deterministic function F(C) that generates an output index  I

I = F(C)

Then include SHA256(I) in the merkel tree when you generate block B.

To validate block B + 1 the 'brute force' way you would calculate  SHA256(F(C+B)) and verify it exists in the the merkel tree.

To do it the 'easy way', you would remove all outputs from I that are consumed by inputs in B and then add all outputs in B to I.  This would be 'cheap' and then you can do SHA256(I+ delta from B) and then validate the result exists in the merkel tree.

Bitcoin already operates based upon an index of outputs.  The problem is that these outputs are tied to a transaction which has to be kept around to lookup the output.  The transaction (being the smallest prunable unit) includes other spent outputs + inputs and input scripts most of which is not relevant.  A transaction cannot be pruned until all of its outputs are spent.

I would design F(C) to generate the following table

TRX_HASH,  OUTPUT_NUM,  SPEND_SCRIPT, BLOCK_NUM, AMOUNT

Then sort it by TRX_HASH + OUTPUT_NUM to get a deterministic (and log(n) searchable) index.

The effort to create F(C) is probably 8 man hours (max).   The effort to create a method to implement I+B such that I+B = F(C+B) would probably be less than 4 man hours.

Including the resulting SHA256(I) in the merkel tree perhaps 4 hours.
Validating the merkel tree... 4 hours.

Thus I suspect the total implementation time required to enable pruning of all transaction history is less than 1 week for the backend / miners.   Then individual clients could pick when they want to start trusting I vs the long tail of C.  I suspect that 10,000 blocks is enough for just about anyone.  

You cannot hack nor replace outputs even with a 51% attack and thus it is entirely secure.    

The level of compression offered by your approach would be *slightly* better because it would collapse all outputs with an identical SPEND_SCRIPT into a single entry and I could not do that because transaction inputs must reference a valid output.   However, I suspect your system requires maintaining much more complex hash chains, merkel account trees, and other data structures that take up space.  Breaking backward compatibility would also be a downside.

[bytemaster]

There is potentially synergy between this blockchains features an my economic model for offering a crypto-USD, crypto-GOLD, etc that maintains near parity without having any counter party risk.

Based on this statement I'm not convinced that you understand counterparty risk.

Counter-party risk is depending upon someone else to be willing and able to honor a promise to pay.

Trading goods on the market is not counter-party-risk.  It is instead 'exchange risk' which is different.

Creating market forces that push two goods toward parity value can minimize exchange risk without introducing counter-party-risk.

[bitfreak!]

Well unfortunately math is not my strongest talent so you kind of lost me with that, but it seems like you may be onto something. What you are saying seems some what similar to the other rolling chain solution I mentioned in my paper. I hope it actually is possible to implement something like this on top of bitcoin, that will give me much more faith in the long term future of bitcoin.

[aaaxn]

Ok, I can summarize my solution as such:

Given the bitcoin blockchain C define a deterministic function F(C) that generates an output index  I

I = F(C)

Then include SHA256(I) in the merkel tree when you generate block B.

To validate block B + 1 the 'brute force' way you would calculate  SHA256(F(C+B)) and verify it exists in the the merkel tree.

To do it the 'easy way', you would remove all outputs from I that are consumed by inputs in B and then add all outputs in B to I.  This would be 'cheap' and then you can do SHA256(I+ delta from B) and then validate the result exists in the merkel tree.

Bitcoin already operates based upon an index of outputs.  The problem is that these outputs are tied to a transaction which has to be kept around to lookup the output.  The transaction (being the smallest prunable unit) includes other spent outputs + inputs and input scripts most of which is not relevant.  A transaction cannot be pruned until all of its outputs are spent.

I would design F(C) to generate the following table

TRX_HASH,  OUTPUT_NUM,  SPEND_SCRIPT, BLOCK_NUM, AMOUNT

Then sort it by TRX_HASH + OUTPUT_NUM to get a deterministic (and log(n) searchable) index.

The effort to create F(C) is probably 8 man hours (max).   The effort to create a method to implement I+B such that I+B = F(C+B) would probably be less than 4 man hours.

Including the resulting SHA256(I) in the merkel tree perhaps 4 hours.
Validating the merkel tree... 4 hours.

Thus I suspect the total implementation time required to enable pruning of all transaction history is less than 1 week for the backend / miners.   Then individual clients could pick when they want to start trusting I vs the long tail of C.  I suspect that 10,000 blocks is enough for just about anyone.  

You cannot hack nor replace outputs even with a 51% attack and thus it is entirely secure.    

The level of compression offered by your approach would be *slightly* better because it would collapse all outputs with an identical SPEND_SCRIPT into a single entry and I could not do that because transaction inputs must reference a valid output.   However, I suspect your system requires maintaining much more complex hash chainhttps://bitcointalk.org/index.php?topic=195275.msg2289285#msg2289285s, merkel account trees, and other data structures that take up space.  Breaking backward compatibility would also be a downside.

You can't expect to hash list of all outputs after every blockchain modification. It is several GB of data. UTXO would need to be stored in some kind of hash tree. It was already discussed and it would work.
https://bitcointalk.org/index.php?topic=88208.0

Problem is that it doesn't solve blockchain size problem, because mining node still needs to hold entire blockchain history. With bitfreak design no one would need to do it.
Furthermore account balances enables more nice features. I posted it in white-paper thread
here: https://bitcointalk.org/index.php?topic=195275.msg2083026#msg2083026
https://bitcointalk.org/index.php?topic=195275.msg2083400#msg2083400
https://bitcointalk.org/index.php?topic=195275.msg2289285#msg2289285

EDIT: You now what is greatest problem with improving bitcoin? Mining pools cartel run this thing now and they won't support any changes that will hurt their business in any way.

[bitfreak!]

Apparently bytemaster's idea isn't going to work out like he had hoped. He posted this in his BitShare Economic Theory thread a little while ago:

The only way to make BitShares work anything close to what I had dreamed would require users to post collateral in excess of the current exchange rate to create crypto-GLD (closer to Version 1.0)

Why didn't I see this before?  I was looking at small 'incremental' rises that would push it up and then extrapolating wrong.  I was thinking that individuals who 'wanted interest bearing USD' would pay a premium to get the return, and therefore cover losses.  It might work for small movements, but certainly not for large ones.

Why couldn't anyone convince me before?  Because no one could explain it in such simple terms.

I'm now trying to persuade him to join this project.  

[bitfreak!]

It looks like digitalindustry might have also given up on his PoS + Mini-Blockchain project because he nows appears to be managing the nibble alt-coin. It appears to be just another run of the mill coin that offers nothing new. One thing about it did catch my attention though:

Ascending reward system that proved effective and prevented insta-mining, pre-mining, unfair rewards for early adopters.

[Impaler]

Why not implement your new protocol on top of Terracoin, TRC is basically a strait BTC clone so your staying close to its code as you said you wanted to.  And it would be an excellent dry run so to speak for bringing the feature to BTC some day.  From what I've heard TRC would be friendly to trying out such a feature as it exists to do this kind of experimentation.  I always recommend people look to expand on existing coins rather then make new ones so we see less fragmentation of the ecosystem.

[bitfreak!]

Why not implement your new protocol on top of Terracoin
...
I always recommend people look to expand on existing coins rather then make new ones

Unfortunately the scheme can't be built on top of an existing blockchain scheme. The closest thing to something like that is the "ultimate blockchain compression" scheme. We were just talking about how this scheme is quite similar to the ultimate blockchain compression scheme in the white paper thread. It is very similar but because we're not trying to apply a change on top of an existing blockchain scheme we can leave out many unnecessary features and make it more efficient than the ultimate blockchain compression scheme. I did start off looking for ways to improve the Bitcoin blockchain but eventually I ended up designing this whole new scheme.

[stellarman]

There is potentially synergy between this blockchains features an my economic model for offering a crypto-USD, crypto-GOLD, etc that maintains near parity without having any counter party risk.   If a new chain were to be created I believe it should include as many 'good ideas' as possible because we will be stuck with it for a while if it works.

I will investigate your proposal and would appreciate if you would investigate my p2p bank/exchange system.

I actually took a look at your thread a few hours ago and your idea seems fairly complex. I would recommend writing a white paper or something. While I agree it may be beneficial to mix together as many good ideas as possible it's still problematic. At this point my goal is simply to just implement this mini-blockchain scheme to see if it actually works. The job is already difficult enough and I don't want to go including all these other untested ideas, it will just make the process longer and harder. As I said, the goal of this project is really just to stay as close to the bitcoin protocol as possible and just play it safe. I would recommend the same thing with your project as well, first just build it as you imagine it and see if it actually performs as you believe it will. When you try to start off with too much on your plate it can often lead you to troubles. At some later point maybe we can have a mini-blockchain + PoS + your exchange model (although I'm still skeptical of the PoS stuff and I'd need to understand your proposal better for actually backing it), but right now that isn't the best course of action to take in my opinion.

+1. I think we need a "proof-of-concept" implementation first. Then think about tying this to other ideas.

[bitfreak!]

+1. I think we need a "proof-of-concept" implementation first. Then think about tying this to other ideas.

Yes exactly, I'm fairly confident in the idea but we can't be 100% sure that the concept will work as we except it to. That's why I just want to leave out all the extra stuff for now, so we can focus on what is important and get it done as quickly as possible.

[scooter]

First altcoin proposal that truly looks interesting to me.
Looking forward to following your progress.

[tom1]

Wouldn't it be better to improve Bitcoin rather than creating a new currency?

[nomailing]

Wouldn't it be better to improve Bitcoin rather than creating a new currency?

The problem with the increasing blockchain size is within the bitcoin protocol itself. As mentioned in the white paper there is the thread about 'ultimate blockchain compression' for bitcoin. But it is by far not as good as this new proposed coin. It only tries to fix some effects without really fixing the underlying problem. If you have a proposal, which could improve the existing bitcoin protocol which is comparable to the performance of this proposal here, then everyone would definately like to hear it.

This new coin proposed by bitfreak is really designed from a new perspective to reduce the bloat which we see in BTC. So I think it really deserves to be implemented and tested!

Btw, it adds some other important features, as aaaxn said:
I think the most important one is that secure 0-confirmations would be possible. I would definately add this to the first post.