Bitcoin Forum

I acknowledge that eurekafag posted before me about a similar idea in http://bitcointalk.org/index.php?topic=473.0

In the near future there's a decent chance that BitCoin will be run on mobile phones and used as a substitute for cash and cards. The main barrier to adoption will be that while the block chain is not ridiculously large, it's larger than it needs to be and as the rate of transactions ramps up, it's going to get larger faster.

Also, people are already annoyed with downloading the whole block chain and hacks have sprung up to get around this. This shows that it's already a problem and it's just going to get worse.

When we're calculating a balance we're not interested in the detail of transactions that happened a year ago although the block chain will happily tell us. We just crunch all this data to work it out. Let's say that a "balance sheet" is a sorted list of all public keys with positive balances followed by the amount of BitCoins they have received. When any block is published we should all be able to calculate identical balance sheets.

I propose that the hash of the balance sheet for block n be hashed into block n+1.  When a new client starts up, unless they're interested in all the historical detail they ask for the current block and balance sheet. In my other suggestion, block numbers are tied to times so, if the client knows the time, they know which block number to ask for. They work out how paranoid p they are. They then ask for the previous p blocks. Using the current balance sheet and undoing transactions they recalculate the previous p balance sheets and they check that the hashes match the blocks. They are then up-to-date and can make transactions and participate in block generation if they wish.

If p is chosen to be at least twice as large as the number of blocks required to make a transaction to go confirmed then the scheme is at least as secure as any normal BitCoin transaction.

Similarly clients could keep a historical balance sheet and reclaim the disc space taken by all the preceeding blocks.

The size in bytes of the balance sheet has to be considerably smaller than that of the block chain. Could some kind person please calculate the current size for me?

In order to keep balance sheets small we could agree to omit from the balance sheet keys "containing" small amount of BitCoin but this sophistication introduces some complications.

The cost of my proposal is one more hash field per block. It would save a lot of network traffic from nodes getting historical blocks, a lot of storage space and a fair amount of users time.

I imagine that this scheme is superior to the Merkle hash tree, which could be dispensed with for some considerable saving in code complexity.

ByteCoin

Subscribed; definitely something to debate. Even on the pc, downloads will eventually reach an unwieldy point. It's an inconvenience to first timers even today. This could also help with privacy if it "flattens" the older transactions.

It all sounds good, but this especially.

Long time lurker first time poster:

Seriously - can someone either shoot this down or acknowledge that there is some validity to it? The block chain is a barrier to many things, new adopters, mobile applications - basically the things that will help BC grow...

The paranoia metric 'p' method as proposed gives an elegant trade off.

This goes hand in hand with the method for guaranteed time length blocks.

Would be good to see at least some debate ;)

Satoshi... any comments?

Rich

New Adopters can be solved with preloaded blocks in the install like I have for download here http://knightmb.dyndns.org/files/bitcoin/

Mobile clients would just be "terminals" to servers that are already running the full software that keeps up with the full block chain. No different than a credit card processing gateway.

Many solutions to these have already been discussed and others are working on them as I type.

Doesn't this create the role of a payments processor? Or 'terminal' - the current strength of the system is that we all participate as potential payments processors if we generate the block. Creating a market niche for a trusted terminal provider (who would guarantee they'd get your transactions into the block chain, have up-time, not be compromised etc.) breaks away from that model.

I accept your point that you can just add the entire block chain to the download, it just seems a dead-weight. It's a question of mathematical certainty versus real world pragmatism - at some point a transaction is confirmed enough for me to send you the goods you bought - 'p' blocks... if that's enough history for us to do business - for me to risk my livelihood - then why is my standard of proof any different elsewhere?

Either way - thanks very much for replying to me. Nothing's ever as black and white as it looks at first!

Rich

I like this idea, though I can't figure out how to work it in without making it a breaking change... It could be incorporated pretty easily as a breaking change, but breaking changes are difficult in a distributed system... The "new" software would look like a vagrant and be pretty much inoperable (it would see everyone else as "hackers") until over half the people updated.  Until then, everyone else would see the newer software as a "hacker", and both groups would reject eachother's blocks, resulting in a bifurcation of the block chain.  Then, old->old transactions would be lost. to the "newer" split blockchain.

A mobile device can just connect to a web interface for your wallet that sits on a home computer (or router). The block file on that server will still get huge, but you don't have to tote it around with you.

Cant mybitcoin just provide an interface for this?

If the site works on your mobile device (should for most), they already do.

If the balance sheet referenced the block, instead of the block referencing the balance sheet, then you could probably implement this system without breaking existing nodes.

Newer nodes would send the balance sheet in a separate message from the block. it would only ask for the balance sheets from nodes of the appropriate system version.

But I could be talking out of my ass too. ;-)

The idea is nice, however there may be flaws. I don't know exactly how coins are generated, are the numbers produced checked against previously generated coins? Is there a possibility to generate a coin twice if you don't have the whole chain? Anyway, something like this (snapshotting/sheets) should be implemented or we eventually get this chain grow like cancer. This will push away newcomers.

Everything in p2p is based on majority trust, so if most of clients create equal snapshots (of small size, of course), newcomers would trust it and assume the proof-of-work was done honestly. We even can delete the whole chain leaving only the snapshot after some time. If we do the sheet every 10k blocks, newcomers would download up to 9999 blocks, about 3 blocks/sec so it will take less than hour to start. Pretty acceptable I think.

Perhaps.  If it was a whole-nother "layer" on top of the existing algorithm, it'd definitely work.  And, you wouldn't have to worry as MUCH about the security of each balance sheet, because it's more of a shortcut anyway.  Anyone really worried can always fall back on the block chain.  Likewise, if someone every once in a while double checks the sheets vs the block chain, they can start "blacklisting" them, letting everyone know they're bad.  Or something.

I think this won't work because there is not a one-to-one relationship between "unspent transactions" and public keys.

Example:  I start with 0 BTC.  Two people each send me 50, to the same receiving address "GavinPubKey".

Balance Sheet:  GavinPubKey: 100
I spend the first one:
Balance Sheet:  GavinPubKey: 50

If I'm dishonest, what stops me from waiting a few months and then spending that first 50 again instead of spending that second 50?  Double-spending that first 50 will look like a perfectly valid transaction to any nodes using the balance sheet method who weren't around to see the first time I spent it.

gavin: The balance sheets are meant for more lightweight things.  If you run in "balance sheet only" mode, it allows you to get a quick glance at it, and operate with people you trust.  You trade at your own risk while in this mode.  If you're distrustful, make sure you're checking the whole block chain.  The balance sheets AUGMENT, not REPLACE the core functionality.

Aren't those first 50 coins will be tracked and bound to the reciever address hence shown in the balance sheet? Everyone could check whether this particular 50 coins were given to anyone or are still yours. If I understand it right, balance sheet is generated from the existing chain so if at the time when you want to doublespend the balance shows these coins assigned to anyone else your transaction won't be accepted.

eureka: No, the balance sheet is very lightweight and only stores balance, not actual coin ownership.  Though, that would be one way to solve that, is to store all 21,000,000 coins (identified by the hash of the block they were generated in) under the address they're "owned" by.  It'd be bigger than the chain is now, but it'd still be significantly smaller than the entirety of the block chain.

Uniquely identifying each bitcoin is a very good idea.  Think of them then as separate claims of property, like lots of land.  By doing this and keeping track of who owns which one (or fractional bit thereof) would add a tremendous amount of trust to the system. 

I'm not at all savvy vis a vis the programming logistics (eeek, run away), I'm more interested in the monetary theory, so salt all comments to taste.

*nods* It wouldn't be a reliable way to operate the underlying system, as coins are split and merged in blocks, 50 are generated at once, etc, but it'd be a nice way to provide a bit of verification and safety to the balance sheets.

Upon reflection the ByteCoin's original proposal to link to the balance sheet from the block seems better, since the blocks are the canonical source of validation.

It appears at first glance that the serialization code provides for software versioning so adding a field has hopes for not breaking previous versions.

For any given block, any client should calculate the same balance sheet and the sorted sheet would reproduce the hash listed in the authoritative block.


Gavin is right about the current need to tie and validate transactions to prior transactions rather than bitcoin addresses. However, for me that is a bit of a non-feature. Interestingly, while generating the balance sheet, the node could generate (unsigned) merging transactions and insert them into the block. Merging transactions could be deemed valid if all inputs and the output went to the same bitcoin address. That would certainly break prior clients though.

Red: It would still be a breaking change.  While it DOES provide versioning, clients only deal with others of the same version.  They don't automatically stop working or somehow learn to adapt to the new version when they see a client of another version.  Because of this, the old clients would continue on as usual, advancing the block chain.  The new clients, (0.4 for simplicity) would start rejecting the old (0.3.2).  Thus, you'd have a "0.4" blockchain, and a "0.3.2" blockchain that were exactly the same up until 0.4 was released, at which point they would diverge.  Transactions would only occur between clients of the same version, and when people finally switched over to 0.4, this new block wouldn't have the transactions from 0.3.2.  It can be done if absolutely necessary, and we'll just have to warn people that old transactions will be invalid, despite showing up as valid on the old clients.  Thus, we should avoid a breaking-change at all costs.

If, however, the sheet referenced the block chain instead, this information would be "extra", and thus safely ignored by older clients.  The block chain would still be valid when generated on older clients.

Another route to go would be to implement it this way, and then add it to the "todo list" for our first breaking change.  We then keep a list of things that would be breaking that we want to do until it gets sufficiently large/advantageous to risk the bifurcating block chain, and do it all in one version update.  This would still be a breaking change, but it would allow us to mitigate the negative effects.

As I read the code, bitcoins are not serialized or in anyway uniquely identified. There are only transactions listing the value (number of) bitcoins involved.

Someone please correct me if I'm wrong about this.

However, I do see lots of uses for a system that could track serialized digital property. It's just not a feature of bitcoin.

Red: Noone said it was.  I mentioned uniquely identifying them by the hash of the first block they were produced in (granted that would be "unique" to 50 of them, but some kind of seed, or offset, or whatever would also work) and someone said that in general, uniquely identifying them would be a good idea.

Pardon me please. Just clarifying for my own benefit. Not trying to call anyone out.

I, and from what I've read from other nooks like me in the forum, initially assumed that because of the name, they system tracked coins as individual objects.

Counter intuitively, the system tracks transactions as primary objects, accounts (addresses) as secondary objects, and coins fungibly.

I see risks to privacy from tracking coins too closely. I'm trying to decide for myself whether or not there is too much risk already.

And thank you for clarification on the breaking change.

^.^ Woops, sorry, was just trying to defend a fellow bitcoiner.  And sure thing on the clarification.  It's a huge issue faced by any distributed system like this, ESPECIALLY if you're dealing with money/something of value.

I'd just like to clarify that "balance sheets" as I outlined them were not "lightweight things" designed to augment the existing system. In order to make balance sheets reliable, the balance sheet hash needs to be included in the block chain as explained. This is a breaking change, so there's no incentive to hold back from other beneficial breaking changes.


You start off with 0BTC and one person sends you 50BTC. So there's a transaction in block x which they've signed and your public key is marked as the recipient.
Someone else sends you 50BTC.  So there's another transaction in let's say a different block y which they've signed and your public key is marked as the recipient.
Next you want to send 50BTC to someone else. Now with a bank there's no concept of where the money has come from, it loses all it's identity when it hits your bank account. Not so with BitCoin! A new transaction (apart from coin mining) has to refer to one or more transactions in the block chain which resulted in a credit to you. So when you talk about spending the first lot of 50BTC the new transaction would refer to block x and if you wanted to spend the second it would refer to block y. I presume the choice is made for you by the current software and the user does not have any input.

So to correct my balance sheet idea in one fashion, as well as the balance for each individual public key, the balance sheet would also have to keep enough information about the blocks which contain the transactions crediting the public key to enable references to those crediting transactions to be generated when spending the money. So instead of storing the public key and the balance, the balance sheet would have to store the public key, a list of crediting references and their corresponding credit amounts - the total credit being the balance. Anything more?

We'd probably have to change the name from "balance sheet" to "complete current credits list".

If we made a client that used the credits list method and we magically crammed the appropriate credits list hash into the block list somehow then the modified software would perform transactions which would reference credit transactions in blocks long since thrown away and the current software would be happy.

There doesn't seem to be much the point of storing references to the crediting transactions in the new transaction. It's easy to see that the money is there and it's not as though we can trace coins forever even if we wanted to. In the example above, if you'd made a transaction to yourself for 100BTC before spending 50BTC then the identity of the original coins is lost. If the requirement of referring to crediting transactions is lifted then the block chain would be considerably smaller, the super light weight balance sheet method is useable and the software's a little less complex.

ByteCoin

I just had a bit of a brainwave as to how we could implement "balance sheets" as a non-breaking change.
Presumably when you generate a new address you have, if you want to exercise it, complete control over what the public key part looks like.

I take the hash of the previous block's balance sheet and split it into chunks. I make new public keys that contain these chunks of hash as parts of the public key values. I make a small transaction of 0.01BTC from a fixed address to each of these new hash-chunk-encoding public keys as soon as I have calculated the hash of the balance sheet. Each of these 0.01BTC accounts has about a month to "live" before I transfer the money back to the fixed address. If I did this earlier then in theory, the information could be at risk from the Merkle tree branch pruning.

Modified, clients using the balance sheet method would of course treat the transactions as valid just like all other valid transactions but they look out for all transactions signed by the fixed address as encoding balance sheet hashes. If, as well as storing the balance sheet hash, the transactions encoded an IP address of a "balance sheet server" then new clients would know where to go to get it.

As an aside, BitCoin is a great way to record arbitrary information permanently. One problem is that it thinks that the marginal value of thousands of people's disk space is zero.


ByteCoin

I don't think that's how public keys work.  You can't arbitrarily choose a public key, it has to be generated in a very specific way, paired in a very specific way with a very carefully chosen private key.

I like the balance sheet concept because it allows the network to forget most of the historical transactions. That suits me fine.

It is worth pointing out that the balance sheet associated with a given transaction block, will very likely be much larger than the transaction block itself. It is much smaller than the entire block history, but that is a one time transmission cost. A balance sheet would be recurring.

If you generated one balance sheet a day, or week, then wiped previous blocks that would suit me fine.

Ok, I was again thinking of RSA. For this type of public key crypto it's true, you have relatively little control over the public key value. Of course if I generate a large number of public keys I could perhaps end up with about 16M =2^24 keys each with a different 3byte pattern encoded at a certain position. It would take a transaction with 11 receiving accounts to encode a 32byte hash of the balance sheet. 65536 different public keys would need 16 receiving accounts per transaction.

Of course, when you do a transaction you're just sending it to a public key. Nobody checks whether anybody has the private key so if you don't mind wasting 0.01BTC per block then you could encode two hashes in one transaction if you wanted as the public key takes 64 bytes. 50BTC would last you a bit over one month. The money would of course be completely lost. I believe you could encode arbitrary data into the block chain at a cost of 0.16BTC per 1024 bytes. Or about 312k for 50BTC. This data would live forever in servers round the world under the current scheme.

The benefit of the balance sheet is that you only have to receive it once when you need to sync back up if the size of the blocks you'd need to process to get back up to speed is larger than the balance sheet. Once you're up-to-date you keep the balance sheet updated in memory as the transactions go through and you just check that the hashes in the block chain agree with the hash of the balance sheet in memory.

So basically the balance sheet is never sent out unless you ask for it. Similarly at the moment the entire block chain is not sent out with every new block but it is sent out if you ask it to be when you start a new client.

ByteCoin

This "Balance Sheet" concept does not need to be separately implemented.
You can already forget as many transactions as you'd like.
It is covered in section 7 of the whitepaper pdf (http://www.bitcoin.org/bitcoin.pdf).

(And you actually only need to remember the 'TxOut' portion of any transaction.)

Either way you'll have to store about the same amount of data.

Let me clarify...

I'd like it if EVERYBODY forgot the old transactions. It doesn't make much sense from an anonymity perspective for just some people to forget them.

And yes, I understand Merkle trees. It's a nice feature.

That is not going to happen though.

Your anonymity is not good if it relies on other people forgetting things.

Mobile phone application won't have to be a full bitcoin client nor server.  It wouldn't make much sense.

It would be much simpler for mobile phones to use a service such as mybitcoin, or to connect via ssl to a Personnal Desktop.

I don't think the size of the block chain is an issue.  At all.  I don't care if it bothers some users.  Some people will always be unhappy about something.

Please don't touch the protocol.  It is fine as it is right now.

Thank you for rejuvenating an old thread! I'm glad you're taking an interest and you have strong opinions but there's no rationale in your post which would cause anyone to take them seriously.

Why would it not make sense for a mobile phone application to have essentially the same functionality as a full client? All the clients being the same simplifies interactions.

You seem to be making a distinction between a Bitcoin 'client' and a 'server'. What's the difference?

There are three resources which could possibly limit Bitcoin's performance: CPU, storage and bandwidth.
Let's say 1MByte per second of uncompressible incoming transaction data which needs to be recorded in the block chain.
This is a high but plausible bandwidth requirement. It might result in 10k per second ECDSA verifications which is again high but plausible in today's multi-core world. However the block chain would grow at a terabyte in under two weeks or over 30 terabytes a year which strikes me as implausibly large. This makes me think that the size of the block chain will be the first hard limit to be reached.

Do you still think it's not an issue?

ByteCoin

This is essentially already done. Bitcoin uses a hash tree structure for transactions that makes it possible for nodes to:
- Do everything except generation by downloading just the block headers and Merkle tree for each block, which is a tiny amount of data.
- Discard outputs that have been spent (referred to by a later transaction), even if you're a generator.

Neither of these are actually implemented yet, but the system of capable of doing it.

You could even generate while running in header-only mode (using getdata messages to get transactions you're missing), though the entire network could not do this.

Using the same definition, "Balance sheets" is essentially done as well!  ;)

ByteCoin


PS

The current Bitcoin client has the undeniable advantage of existing.

Neither the lightweight whitepaper client nor a "balance sheet" based client exist but both are compatible with the current protocol.
They are therefore equal on that basis.

I contend that a "balance sheet" based client is superior in every other practical way to the lightweight whitepaper client or the current client. If it were implemented correctly, it would be superior to the current client. If a new Bitcoin-like protocol were correctly  designed and implemented with "balance sheet" and other improvements in mind then it would be superior to the current protocol.

The core bitcoin network should remain how it is. The core network should not be concerned with helping ordinary end users run lightweight clients. The core network should always remain as simple and lean as possible, staying within the theory laid out in the white paper, nothing more, except for fixing real bugs and security holes if they are discovered.

As was said in an earlier post, the balance sheet functionality can be layered on top of the existing core network. Clients in balance sheet mode can connect to each other on a second network to exchange information that relates blocks to the balance sheet. Older, ordinary clients will not need to know that this other network exists.

Idea sounds interesting, but I'd like to hear satoshi opinion on this very much.
Satoshi, where R you ?

Satoshi already addressed the issue of the size of the block chain in his white paper (http://www.bitcoin.org/bitcoin.pdf).  I'm pretty sure it will never be a real problem.

We should not modify a software protocol just to allow it to fit to particular devices.  For mobile devices, a connection to a distant machine is good enough.   Isn't that what "mobility" is about ?


I don't know.  But I guess there is a difference, since the bitcoin client has a -server option.


As I said, this has already been addressed in Satoshi's white paper.  The solution is to use Hash trees, and it seems efficient enough.

The simplified payment verification system is already in place. The Merkle root required for its functioning is included in all blocks. Blocks do not include the hash of a balance sheet.

The size difference would not be significant. SPV is ~80 bytes per block plus 32 bytes per transaction, whereas balance sheets would be 20 bytes per unique address. Currently there are 132415 unique addresses in the system and 134267 transactions. SPV: 11.29 MB; balances: 2.65 MB. This is assuming that balance sheets will not have any header-like overhead, which they almost certainly will.

SPV looks through the Merkle tree to get the number of confirmations and prove that transactions and their prev_outs were not double-spent. This is the point of SPV. How would balance sheets solve this? If you're just going to download the most recent 5 blocks or whatever (an insecure method), why even have balance sheets? You can't generate with balance sheets, as you are unable to verify referenced signatures.

Using balance sheets with the current system would require receiving and processing every transaction ever made, which will become difficult as the block chain grows. SPV requires no such processing, and the amount of data stored on disk is the amount received through the network.

A balance sheet system written from scratch would not be any better than the current system. Generators need to know the contents of every non-spent transaction, so a parallel network similar to the current one would have to be kept. Clients would need to download every block header (as in the current system) because the current block with the balance hash can only be verified if you have every block in the chain.

Thank you theymos for taking the time to criticize the scheme in some detail.


After a new block is found the balance sheet is recalculated (possibly incrementally) and the hash of the balance sheet is encoded in a special transaction that serves to encode the data which is then broadcast round the network the same as a normal transaction. This is done by exploiting the broadband subchannel in ECDSA mentioned in another post of mine. There are some details to be fleshed out such as "who generates this balance sheet hash transaction" and "how do you cope with a malicious balance sheet client supplying incorrect data" but I believe there are multiple adequate ways of addressing these problems.

Thanks for supplying some real numbers. I'm afraid that "balance sheet" is not really an accurate description of what is stored. See
We need to store enough information about all the transactions which credit an address to allow appropriate references to those crediting transactions to be recorded in the transaction when you spend money from that address. It's like the stubbing-off-merkle-tree-branches idea in the white paper except that the stub hashes don't need to be stored and neither do the blocks. I doubt at the moment the "balance sheet" idea would save a significant amount of space but that's because Bitcoin is so thinly exchanged. As the fraction of spent transactions rises, the storage savings of the balance sheet method become more persuasive.

I'm not quite sure what you mean. I think this is the same objection that gavinandresen raised earlier in the thread in response to me misunderstanding exactly how transactions worked. After he put me right I changed the scheme. After reading my reply, if you're not satisfied, please explain the problem with my scheme in more detail.

Isn't it true that when you download the block chain you process it all in the current scheme? It seems to be that your criticism is more appropriately leveled at the current scheme because new "balance sheet"-using clients download the current credit list from other similar clients. The client then updates the balance sheet with all the incoming transactions to stay in sync. No processing of spent transactions ever takes place.

I'm not sure what you mean by a "parallel network" or why it would be necessary. I believe it would use the current network, as a "balance sheet"-using client looks, to the network, (mostly) like the existing client. You can't however download the older portions of block chain from it as that's data it has "forgotten". You are right in thinking that the "balance sheet" scheme becomes unmoored from the root hash. You are correct in thinking that this is a problem which needs to be addressed and I believe that my scheme can offer roughly equivalent security guarantees to the current scheme but the details are complex.

Implementing "balance sheets" without altering the current protocol is rather complex and that makes it unattractive. However I believe that Bitcoin will have little choice but to either change the protocol or to move to a client implementation in which nobody remembers all the transactions, such as "balance sheets". There's nothing stopping a small group of people spamming the network with transactions possibly encoding the latest Lady Gaga video or child pornography etc. As a method of storing data on the internet for free it has the benefit of designed-in complete permanence, distributed reliability and plausible deniability. Before becoming bandwidth or CPU limited I believe that the block chain+transaction data could grow at about 30TB a year with the rate only increasing. This is going to exclude the vast majority of people from running full clients. Either Bitcoin would have to give up the p2p label or it's going to have to start forgetting old transactions. Of all the ways of doing the latter, "balance sheets" is the best.

ByteCoin

And how can you be sure that bitcoins in wallets that you created for your kids education will be valid in the future?