Blockchain

[israfelwater]

Basically the blockchain is getting way too long and out of control. We want to split the block chain into say block chain A and block chain B. How this would work is if you want to make a secure transaction it takes place between blocks, A->B or B->A, but it takes six days to clear. If you want to make an instant transaction it is has to be within the same block A->A or B->B.

Naturally when a block is transferred between blockchains, say A->B, it is removed from A and reborn in B without any memory, but there is a six day hold on the coin.

This would be great because it would shorten the blockchain and create some anonymity and offer a method for more secure transactions. 

[torba]

Sounds hard to implement

[Foxpup]

12 measly GB is too much to handle? 1999 called, they want their disk space back.

[jdbtracker]

What if instead we just parced it, torrent style? if people wanted to, Gavin was talking about paying the nodes... well why not just pay the nodes that have the full copy and the rest get the blockchain torrent? A fee system could be setup to pay an incentive to those who protect the network.

[PrintMule]

12 measly GB is too much to handle? 1999 called, they want their disk space back.

12 GB are not measly by any means. And with the current rate of it's growth...

Bitcoin is meant to be "portable", and blockchain feels like a ballchain around your leg.

[blueadept]

Use an SPV wallet like MultiBit. It requires very little in the way of blockchain storage and sync time.

[mustyoshi]

The unspent outputs only take up ~250MB, the entire history of the currency will always be growing faster and faster, you don't need the entire history though.

There's no need to split the chain.

[Trongersoll]

12 measly GB is too much to handle? 1999 called, they want their disk space back.

12 GB are not measly by any means. And with the current rate of it's growth...

Bitcoin is meant to be "portable", and blockchain feels like a ballchain around your leg.

In a world where multi-terabyte hard drives are available for a few $100.00, yes 12 gig is measily.

[edmundedgar]

Whether or not you agree with the OP's premise that the blockchain is too big now, it's clearly possible that it'll run into some natural limitation in the future. It would be good if someone could come up with a practical way to shard it and prove that it worked in practice with an alt-coin.

The bit about moving coins between shards may not actually be necessary at all. Assuming your client doesn't need the full blockchain for all the shards it has coins on - which it shouldn't, given SPV etc - it should be no problem having coins split among a lot of different shards. If you ask me to pay you, and I have coins on shard #123, I send you the coins on shard #123. The end user shouldn't normally even have to know about the shards, any more than they currently care about the fact that the money in their wallet is comprised of a bunch of different outputs. The only new thing from their point of view would be that sometimes a payment would come from multiple shards, so different parts of the payment would confirm at different speeds. ("40% unconfirmed, 22% 1 confirmation, 38% 3 confirmations")

The main reason to want to be able to move coins between shards would be to be absolutely 100% certain that people didn't start getting some mad idea about different shards having different values. The feature would hardly even have to be used - you'd just need to be able to reassure people that it was there if you needed it. I don't think you'd necessarily need a week - I guess you'd have a single small, expensive "transfer" shard which all the miners kept, alongside whichever normal shard they were mining. You'd need to be X certain number of blocks deep in your original shard before the coin was accepted into the transfer shard, then Y blocks deep in the transfer shard before it was accepted in the destination shard. X and Y wouldn't have to be huge numbers (6 might be OK), but even if it did take a week, it wouldn't particularly matter.

[natb]

Use an SPV wallet like MultiBit. It requires very little in the way of blockchain storage and sync time.

I agree that this is a viable solution, but then again it makes Bitcoin less distributed. The nice thing about running a full Bitcoin node is that you are one of the voices in the network contributing your say to what chain is legit and what chain is not. If everyone is running wallets across only a few 'trusted' servers we lose this awesome feature of Bitcoin.

As a practical matter, solving the blockchain length is an important one IMHO. I don't yet have any ideas to contribute to solving this.

[israfelwater]

What if instead we just parced it, torrent style? if people wanted to, Gavin was talking about paying the nodes... well why not just pay the nodes that have the full copy and the rest get the blockchain torrent? A fee system could be setup to pay an incentive to those who protect the network.

That is an awesome idea. By parce it, torrent style, I assume you mean when you make a transaction you only pull the information you need for the transaction down from the block chain? Correct? I like this over SPV, for the reason natb gave, but SPV works great for say mobile wallets.

Whether or not you agree with the OP's premise that the blockchain is too big now, it's clearly possible that it'll run into some natural limitation in the future. It would be good if someone could come up with a practical way to shard it and prove that it worked in practice with an alt-coin.

The bit about moving coins between shards may not actually be necessary at all. Assuming your client doesn't need the full blockchain for all the shards it has coins on - which it shouldn't, given SPV etc - it should be no problem having coins split among a lot of different shards. If you ask me to pay you, and I have coins on shard #123, I send you the coins on shard #123. The end user shouldn't normally even have to know about the shards, any more than they currently care about the fact that the money in their wallet is comprised of a bunch of different outputs. The only new thing from their point of view would be that sometimes a payment would come from multiple shards, so different parts of the payment would confirm at different speeds. ("40% unconfirmed, 22% 1 confirmation, 38% 3 confirmations")

The main reason to want to be able to move coins between shards would be to be absolutely 100% certain that people didn't start getting some mad idea about different shards having different values. The feature would hardly even have to be used - you'd just need to be able to reassure people that it was there if you needed it. I don't think you'd necessarily need a week - I guess you'd have a single small, expensive "transfer" shard which all the miners kept, alongside whichever normal shard they were mining. You'd need to be X certain number of blocks deep in your original shard before the coin was accepted into the transfer shard, then Y blocks deep in the transfer shard before it was accepted in the destination shard. X and Y wouldn't have to be huge numbers (6 might be OK), but even if it did take a week, it wouldn't particularly matter.

The big thing with transferring between shards is it offers both security and anonymity. If there were super nodes tracking cross shard transactions it would defeat the purpose of anonymity. I think leaving it at two shards would be the preferred approach, as it reduces complexity and is all needed.

Plus removing a block from the chain would require some torrent like behaviour, as it would create holes right?

12 measly GB is too much to handle? 1999 called, they want their disk space back.

12 GB are not measly by any means. And with the current rate of it's growth...

Bitcoin is meant to be "portable", and blockchain feels like a ballchain around your leg.

In a world where multi-terabyte hard drives are available for a few $100.00, yes 12 gig is measily.

Disk space is nominal server side, but bitcoin is the cryptocurrency of the everyday PC, and 12 gig today might as well be 12 tera tomorrow.

[Foxpup]

Disk space is nominal server side, but bitcoin is the cryptocurrency of the everyday PC, and 12 gig today might as well be 12 tera tomorrow.

Wherever did you get the idea that Bitcoin is "the cryptocurrency of the everyday PC"? Satoshi states in the whitepaper that ordinary users needn't run a full node, implying that there may come a time when most users shouldn't even try to run a full node on their PCs. The only people who really might conceivably need to run full nodes (aside from miners, who can charge fees to offset their costs so it's a non-issue) are entities like banks and payment processors, who certainly have the resources; thin clients are probably good enough for everyone else. Even so, 12 TB today is still not entirely unreasonable for a PC, and costs less than $500. Bandwidth is another issue, but since almost all of the bandwidth used by a full node is for relaying transactions, obviating the need to store the whole blockchain won't help.

[blueadept]

Use an SPV wallet like MultiBit. It requires very little in the way of blockchain storage and sync time.

I agree that this is a viable solution, but then again it makes Bitcoin less distributed. The nice thing about running a full Bitcoin node is that you are one of the voices in the network contributing your say to what chain is legit and what chain is not. If everyone is running wallets across only a few 'trusted' servers we lose this awesome feature of Bitcoin.

As a practical matter, solving the blockchain length is an important one IMHO. I don't yet have any ideas to contribute to solving this.

Most consumers, in the long run, will be fine running SPV clients. The network will still be decentralized as many businesses will want to run a full node in order to be able to directly verify transactions.

[gmaxwell]

The network will still be decentralized as many businesses will want to run a full node in order to be able to directly verify transactions.

Why?  I've found it hard to convince people that full nodes offer additional security. (Indeed, I've found it hard to convince people that SPV clients provide more security than JS web wallets).

Doubly so because the inflationary threat running full nodes is most uniquely suited to fighting is generally not that relevant to businesses. We do not see businesses standing up to fight fiat inflation. Generally businesses care more about differential advantages with their competition, can avoid fixing contract prices in inflation resistant terms, and can afford the resources to invest away (self or otherwise) inflation risks.

In some ways I'd expect businesses to prefer running SPV nodes that get blocks from a single source which cryptographically signs them. The security model is more traditional and thus easier to understand, it completely closes some kinds of risks, and gives them someone to sue if things go wrong (except when the wrongness is produced by a state level threat, but see above). Today we see businesses very frequently depending on parties like bitpay for their whole transaction processing needs, a step even further towards centralization end of the spectrum then SPV with signed blocks.

Decentralization is hard and it's costly. Anyone who believe it will be easily retained in Bitcoin if Bitcoin grows is fooling themselves.

[natb]

Most consumers, in the long run, will be fine running SPV clients. The network will still be decentralized as many businesses will want to run a full node in order to be able to directly verify transactions.

Yes I think this is a fair point. I have not studied the network side of Bitcoin much - is there any way to know how many full nodes are running at any one time?

[blueadept]

The network will still be decentralized as many businesses will want to run a full node in order to be able to directly verify transactions.

Why?  I've found it hard to convince people that full nodes offer additional security. (Indeed, I've found it hard to convince people that SPV clients provide more security than JS web wallets).

Doubly so because the inflationary threat running full nodes is most uniquely suited to fighting is generally not that relevant to businesses. We do not see businesses standing up to fight fiat inflation. Generally businesses care more about differential advantages with their competition, can avoid fixing contract prices in inflation resistant terms, and can afford the resources to invest away (self or otherwise) inflation risks.

In some ways I'd expect businesses to prefer running SPV nodes that get blocks from a single source which cryptographically signs them. The security model is more traditional and thus easier to understand, it completely closes some kinds of risks, and gives them someone to sue if things go wrong (except when the wrongness is produced by a state level threat, but see above). Today we see businesses very frequently depending on parties like bitinstant for their whole transaction processing needs, a step even further towards centralization end of the spectrum then SPV with signed blocks.

Decentralization is hard and it's costly. Anyone who believe it will be easily retained in Bitcoin if Bitcoin grows is fooling themselves.

My answers have been short as I've been writing on a phone, but I've fired up a laptop to answer this one.

I believe we'll see a huge expansion of off-chain payments as opposed to on-chain transactions.  There are two advantages:  instant payments, and scalable micropayments.  Privacy can also be an added advantage with decentralized off-chain payments as they don't hit the blockchain.  How decentralized we can make the off-chain payment solution determines how many intermediaries there will be.  I believe most intermediaries will want to have a full node for a number of reasons (verify inflation, fully verify transactions, and get routing/reputation data from the blockchain), and will be able to fund running one with fees they earn by passing payments.  My signature has a link to my own (badly-written, for now) proof-of-concept of one such scheme, and I'm (slowly) working on bringing it closer to alpha.

I think the number of full nodes that will eventually be running will be determined by necessity and funding.  The more applications we can invent which require a decentralized payment network, the more such intermediaries will both need to run a full node and be able to afford it.  These applications are precisely why I'm writing libtxchain-java.  Some examples are:

• Virtual meshnet ISPs (ISP runs intermediary, independently-owned WAPs contract with multiple ISPs to route traffic for them, client has contract with one or more ISPs, payments chain from client to ISP to WAP)
• Peer-to-peer networks which use Bitcoin micropayments for resource allocation, especially for scientific computing (first node to find a lower potential energy configuration than currently known for amino acid sequence X gets a payment, rinse, repeat, and your protein is folded without investment in infrastructure)

To answer your questions more specifically:

• Full validation will be important to off-chain payment intermediaries for self-defense against malicious input by peers in contract setup/teardown
• Access to the full blockchain will give such intermediaries the opportunity to gain differential advantage in being able to identify potentially profitable nodes with which to peer and potentially risky nodes with which to avoid peering

Yes I think this is a fair point. I have not studied the network side of Bitcoin much - is there any way to know how many full nodes are running at any one time?

There are some attempts at estimating this number, but all vary in accuracy.  Here's one:  http://bitcoinstatus.rowit.co.uk/

Luke-Jr also had a page that showed number of nodes by version, but I forgot where it is exactly.

[jdbtracker]

Most consumers, in the long run, will be fine running SPV clients. The network will still be decentralized as many businesses will want to run a full node in order to be able to directly verify transactions.

Yes I think this is a fair point. I have not studied the network side of Bitcoin much - is there any way to know how many full nodes are running at any one time?

There are currently 140k full nodes running world wide at any given time.