Bitcoin Forum
April 25, 2024, 09:18:40 AM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1] 2 3 4 »  All
  Print  
Author Topic: Can the Block Chain get too big and make Bitcoin unworkable?  (Read 31495 times)
pyedpyper (OP)
Full Member
***
Offline Offline

Activity: 172
Merit: 100


View Profile
June 24, 2012, 10:31:25 PM
 #1

Hi,

I've just started investigating Bitcoin, and am very excited about the overall concept. I've only read a small amount of the tech, but as I understand it, the Block Chain contains a history of all Bitcoin transactions, and this Block Chain is replicated and stored on every user's computer (is that correct??). And when I installed Bitcoin for the first time, it took many hours to download the Block Chain - which seems to corroborate with how I understand it...

The immediate question which comes to mind is: As Bitcoin grows and many people start using it then could this file not become so big that the whole system becomes totally unworkable? If a billion people are using Bitcoin as their primary trading currency, with multiple billion transactions happening every single day, and every node is constantly being updated with the new information from all of these transactions then it seems this could be a very real problem. Not just for Bitcoin but in terms of overall Internet bandwidth usage!

And pity the poor soul who has to download the Block Chain for the first time! It seems this will be an absolute limit on Bitcoin growth beyond the relatively small scale it is now... Or do I have it all wrong??

Another thing - if someone wanted to fuck with Bitcoin, especially once it becomes widely used, it seems all one would need to do is set up two accounts to automatically send one Bitcoin back and forth multiple times a second (perhaps hundreds or thousands of times a second) - if the software can be so set up, and if that transaction needs to update all one billion block chains at every node every time, then it seems to me it would all just grind to a dysfunctional halt.

I'm wanting to invest heavily in developing Bitcoin and perhaps even setting up an exchange in South Africa (average bandwidth about 1mbps), but if my concerns are real then it seems that Bitcoin might eventually become impractical to actually use once it gets to a certain size and level of adoption, and that in turn would cause the value of the currency to totally devalue. Seems potentially dangerous...

I would appreciate some solid and in depth responses to this.

Thanks for your help...
1714036720
Hero Member
*
Offline Offline

Posts: 1714036720

View Profile Personal Message (Offline)

Ignore
1714036720
Reply with quote  #2

1714036720
Report to moderator
1714036720
Hero Member
*
Offline Offline

Posts: 1714036720

View Profile Personal Message (Offline)

Ignore
1714036720
Reply with quote  #2

1714036720
Report to moderator
1714036720
Hero Member
*
Offline Offline

Posts: 1714036720

View Profile Personal Message (Offline)

Ignore
1714036720
Reply with quote  #2

1714036720
Report to moderator
Each block is stacked on top of the previous one. Adding another block to the top makes all lower blocks more difficult to remove: there is more "weight" above each block. A transaction in a block 6 blocks deep (6 confirmations) will be very difficult to remove.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
Kazimir
Legendary
*
Offline Offline

Activity: 1176
Merit: 1001



View Profile
June 24, 2012, 10:42:11 PM
 #2

Not a problem. The block chain could be compressed / collapsed at dedicated reference points in history.

Actually, clients could already do so right now, as this is just a 'local' issue and would not require any change in the protocol whatsoever.

In theory, there's no difference between theory and practice. In practice, there is.
Insert coin(s): 1KazimirL9MNcnFnoosGrEkmMsbYLxPPob
Mushoz
Hero Member
*****
Offline Offline

Activity: 686
Merit: 500


Bitbuy


View Profile WWW
June 24, 2012, 10:43:35 PM
 #3

Hi,

I've just started investigating Bitcoin, and am very excited about the overall concept. I've only read a small amount of the tech, but as I understand it, the Block Chain contains a history of all Bitcoin transactions, and this Block Chain is replicated and stored on every user's computer (is that correct??). And when I installed Bitcoin for the first time, it took many hours to download the Block Chain - which seems to corroborate with how I understand it...

The immediate question which comes to mind is: As Bitcoin grows and many people start using it then could this file not become so big that the whole system becomes totally unworkable? If a billion people are using Bitcoin as their primary trading currency, with multiple billion transactions happening every single day, and every node is constantly being updated with the new information from all of these transactions then it seems this could be a very real problem. Not just for Bitcoin but in terms of overall Internet bandwidth usage!

And pity the poor soul who has to download the Block Chain for the first time! It seems this will be an absolute limit on Bitcoin growth beyond the relatively small scale it is now... Or do I have it all wrong??

Another thing - if someone wanted to fuck with Bitcoin, especially once it becomes widely used, it seems all one would need to do is set up two accounts to automatically send one Bitcoin back and forth multiple times a second (perhaps hundreds or thousands of times a second) - if the software can be so set up, and if that transaction needs to update all one billion block chains at every node every time, then it seems to me it would all just grind to a dysfunctional halt.

I'm wanting to invest heavily in developing Bitcoin and perhaps even setting up an exchange in South Africa (average bandwidth about 1mbps), but if my concerns are real then it seems that Bitcoin might eventually become impractical to actually use once it gets to a certain size and level of adoption, and that in turn would cause the value of the currency to totally devalue. Seems potentially dangerous...

I would appreciate some solid and in depth responses to this.

Thanks for your help...

Multiple solutions have been discussed to address this problem. The first solution involves pruning old transactions from the blockchain, so that the blockchain only contains coins that haven't been spend yet. A second solution involves a meta-chain that only tracks coins that haven't been spend yet, which will coexist with the full chain. This way, most people and lightweight clients can rely mostly on the meta-chain together with a little part of the full chain, namely normal blocks that were created after the latest meta-chain block. People that prefer to keep the entire chain can also do so, so this solution will give people a choice what they personally want. This meta-chain, if implemented, will be merged-mined with a higher difficulty, so that a meta-chain block won't be created every 10 minutes but every X minutes (still to be decided.). For a discussion on this solution please check this thread: https://bitcointalk.org/index.php?topic=88208.0

Also, there's a wiki article addressing some of the scaling complaints. The summary is basically that with pruning and some other compression techniques, together with Moore's Law (exponentially increasing computer power and storage), scaling won't be a problem, even with transaction volumes matching that of Visa (2000 transactions per second!). Here's the link to the wiki article: https://en.bitcoin.it/wiki/Scalability

www.bitbuy.nl - Koop eenvoudig, snel en goedkoop bitcoins bij Bitbuy!
nimda
Hero Member
*****
Offline Offline

Activity: 784
Merit: 1000


0xFB0D8D1534241423


View Profile
June 24, 2012, 11:28:40 PM
 #4

This problem is so well-known that Satoshi described a fix in his original whitepaper Roll Eyes
Now we just need it implemented...
Kluge
Donator
Legendary
*
Offline Offline

Activity: 1218
Merit: 1015



View Profile
June 24, 2012, 11:41:49 PM
Last edit: March 25, 2014, 12:33:41 PM by Kluge
 #5

this Block Chain is replicated and stored on every user's computer (is that correct??). And when I installed Bitcoin for the first time, it took many hours to download the Block Chain - which seems to corroborate with how I understand it...
There are many Bitcoin clients. The "official" Bitcoin client (Bitcoin-qt) is not really official, and there are clients out there which already provide comparable features without requiring you download the blockchain - so no, not everyone needs to download the blockchain to use Bitcoin. It's not necessary to wait hours or days to be able use Bitcoin after installing, and these thin/light clients require only an insignificant amount of computer resources. A fix may be unnecessary (though it would be ideal) while servers can host the blockchain.

Aside from e-wallets (blockchain.info is usually fairly highly-regarded), there are clients such as Electrum for your desktop computer, and clients such as BitcoinJ and Bitcoin Spinner for Android. There are more if you look, but these are the services/clients I hear the most about. With the exception of e-wallets (in all cases I'm aware of), all these lite clients do not store your private keys on a central server, so you are at no risk of losing coins if one of these services are hacked.

Cheers!
pyedpyper (OP)
Full Member
***
Offline Offline

Activity: 172
Merit: 100


View Profile
June 25, 2012, 03:11:31 PM
 #6

Thanks everyone for the solid answers and follow-up links. I see this matter has had a lot of discussion!

Seems that trimming the block chain is a viable solution if properly implemented.

Is it correct that every transaction that happens is written to the block chain of every user?

If so this would appear to present its own bandwidth problem.

In order to be viable as a long term sustainable currency, Bitcoin ultimately needs to be able to scale to billions of transactions per day (thousands per second). Is this possible if each transaction needs to update every single block chain stored on every user's computer? (Moore's Law aside...)

DannyHamilton
Legendary
*
Offline Offline

Activity: 3374
Merit: 4606



View Profile
June 25, 2012, 03:32:33 PM
 #7

. . . if someone wanted to fuck with Bitcoin . . . it seems all one would need to do is set up two accounts to automatically send one Bitcoin back and forth multiple times a second (perhaps hundreds or thousands of times a second) . . .

This is currently handled with enforced transaction fees.
https://en.bitcoin.it/wiki/Transaction_fees
wildgift
Newbie
*
Offline Offline

Activity: 16
Merit: 0


View Profile
June 25, 2012, 06:19:12 PM
 #8

I thought the solution was to take a bunch of old transactions, hash them, and replace those transactions with the hash.  Though it's a one-way process, if someone has the original transactions, it's verifiable.
Meni Rosenfeld
Donator
Legendary
*
Offline Offline

Activity: 2058
Merit: 1054



View Profile WWW
June 25, 2012, 06:37:57 PM
 #9

Thanks everyone for the solid answers and follow-up links. I see this matter has had a lot of discussion!

Seems that trimming the block chain is a viable solution if properly implemented.

Is it correct that every transaction that happens is written to the block chain of every user?

If so this would appear to present its own bandwidth problem.

In order to be viable as a long term sustainable currency, Bitcoin ultimately needs to be able to scale to billions of transactions per day (thousands per second). Is this possible if each transaction needs to update every single block chain stored on every user's computer? (Moore's Law aside...)
As mentioned, end users will not need the entire blockchain. Just the headers and transactions that involve them specifically, which is a tiny part.

1EofoZNBhWQ3kxfKnvWkhtMns4AivZArhr   |   Who am I?   |   bitcoin-otc WoT
Bitcoil - Exchange bitcoins for ILS (thread)   |   Israel Bitcoin community homepage (thread)
Analysis of Bitcoin Pooled Mining Reward Systems (thread, summary)  |   PureMining - Infinite-term, deterministic mining bond
MoonShadow
Legendary
*
Offline Offline

Activity: 1708
Merit: 1007



View Profile
June 25, 2012, 06:54:15 PM
Last edit: June 25, 2012, 07:09:21 PM by MoonShadow
 #10

Hi,

I've just started investigating Bitcoin, and am very excited about the overall concept. I've only read a small amount of the tech, but as I understand it, the Block Chain contains a history of all Bitcoin transactions, and this Block Chain is replicated and stored on every user's computer (is that correct??). And when I installed Bitcoin for the first time, it took many hours to download the Block Chain - which seems to corroborate with how I understand it...

The immediate question which comes to mind is: As Bitcoin grows and many people start using it then could this file not become so big that the whole system becomes totally unworkable? If a billion people are using Bitcoin as their primary trading currency, with multiple billion transactions happening every single day, and every node is constantly being updated with the new information from all of these transactions then it seems this could be a very real problem. Not just for Bitcoin but in terms of overall Internet bandwidth usage!


Not all clients need to download the blockchain, just full clients for users who don't want to have to trust anyone as well as miners.  Also, even full clients can prune long spent transactions from their local copy of the blockchain, this is one of the functions of the merkle tree structure within a block.  Certain types of full clients can't do this for practical reasons, but any end-user full client could do it and still be able to verify every transaction that comes it's way.

Quote
Another thing - if someone wanted to fuck with Bitcoin, especially once it becomes widely used, it seems all one would need to do is set up two accounts to automatically send one Bitcoin back and forth multiple times a second (perhaps hundreds or thousands of times a second) - if the software can be so set up, and if that transaction needs to update all one billion block chains at every node every time, then it seems to me it would all just grind to a dysfunctional halt.


Try it.  See what happens.

EDIT: To end your suspense, it won't have any noticable effect upon the network as a whole, as this has been considered and already adapted for.  You will, however, succeed in losing your entire balance due to tiny fees required for 'low priority' transactions.  This will still take some time, however, since the network will largely ignore spammy transactions, due to a built in 'point scale' system that discounts bitcoins that have recently been spent.  This doesn't prevent any valid transaction from occurring, but it does delay those that they system consiers spammy.

Quote
I'm wanting to invest heavily in developing Bitcoin and perhaps even setting up an exchange in South Africa (average bandwidth about 1mbps), but if my concerns are real then it seems that Bitcoin might eventually become impractical to actually use once it gets to a certain size and level of adoption, and that in turn would cause the value of the currency to totally devalue. Seems potentially dangerous...

I would appreciate some solid and in depth responses to this.

Thanks for your help...

there has been much discussion on the topic of end users in Africa, mostly as bitcoin as a replacement for M-Pesa.  Do more research, you will learn much of value to your position.

"The powers of financial capitalism had another far-reaching aim, nothing less than to create a world system of financial control in private hands able to dominate the political system of each country and the economy of the world as a whole. This system was to be controlled in a feudalist fashion by the central banks of the world acting in concert, by secret agreements arrived at in frequent meetings and conferences. The apex of the systems was to be the Bank for International Settlements in Basel, Switzerland, a private bank owned and controlled by the world's central banks which were themselves private corporations. Each central bank...sought to dominate its government by its ability to control Treasury loans, to manipulate foreign exchanges, to influence the level of economic activity in the country, and to influence cooperative politicians by subsequent economic rewards in the business world."

- Carroll Quigley, CFR member, mentor to Bill Clinton, from 'Tragedy And Hope'
MoonShadow
Legendary
*
Offline Offline

Activity: 1708
Merit: 1007



View Profile
June 25, 2012, 07:00:58 PM
 #11

I thought the solution was to take a bunch of old transactions, hash them, and replace those transactions with the hash.  Though it's a one-way process, if someone has the original transactions, it's verifiable.

This is functionally what the merkle tree does.  Each transaction has it's own hash, which is then included into a binary hash tree with one other transaction, and then each of those hashes are paired and hashed again to form the full merkle tree.  The last hash (merkle root) is included into the block header, which is then hashed again when the next block includes it into it's own header, thus forms the 'chain' part of the blockchain.  That merkle hash root inside the block header is intended to represent that entire block (in the sequence of the lifespan of the blockchain) once every transaction included into that block has been spent.  Presumedly in the future a great many blocks would be reduced to only the 80 byte, fixed length, headers after a few years time.  There are already several blocks that could be pruned down this way to only those headers, but the pruning functions have not yet been implimented into any client that I am aware.  Mostly because it's not consider a pressing need at the moment.

"The powers of financial capitalism had another far-reaching aim, nothing less than to create a world system of financial control in private hands able to dominate the political system of each country and the economy of the world as a whole. This system was to be controlled in a feudalist fashion by the central banks of the world acting in concert, by secret agreements arrived at in frequent meetings and conferences. The apex of the systems was to be the Bank for International Settlements in Basel, Switzerland, a private bank owned and controlled by the world's central banks which were themselves private corporations. Each central bank...sought to dominate its government by its ability to control Treasury loans, to manipulate foreign exchanges, to influence the level of economic activity in the country, and to influence cooperative politicians by subsequent economic rewards in the business world."

- Carroll Quigley, CFR member, mentor to Bill Clinton, from 'Tragedy And Hope'
wildgift
Newbie
*
Offline Offline

Activity: 16
Merit: 0


View Profile
June 26, 2012, 02:30:49 AM
 #12

I thought the solution was to take a bunch of old transactions, hash them, and replace those transactions with the hash.  Though it's a one-way process, if someone has the original transactions, it's verifiable.

This is functionally what the merkle tree does.  Each transaction has it's own hash, which is then included into a binary hash tree with one other transaction, and then each of those hashes are paired and hashed again to form the full merkle tree.  The last hash (merkle root) is included into the block header, which is then hashed again when the next block includes it into it's own header, thus forms the 'chain' part of the blockchain.  That merkle hash root inside the block header is intended to represent that entire block (in the sequence of the lifespan of the blockchain) once every transaction included into that block has been spent.  Presumedly in the future a great many blocks would be reduced to only the 80 byte, fixed length, headers after a few years time.  There are already several blocks that could be pruned down this way to only those headers, but the pruning functions have not yet been implimented into any client that I am aware.  Mostly because it's not consider a pressing need at the moment.

Thanks for the explanation
pyedpyper (OP)
Full Member
***
Offline Offline

Activity: 172
Merit: 100


View Profile
June 28, 2012, 08:27:12 PM
 #13

Thanks everyone for the solid answers and follow-up links. I see this matter has had a lot of discussion!

Seems that trimming the block chain is a viable solution if properly implemented.

Is it correct that every transaction that happens is written to the block chain of every user?

If so this would appear to present its own bandwidth problem.

In order to be viable as a long term sustainable currency, Bitcoin ultimately needs to be able to scale to billions of transactions per day (thousands per second). Is this possible if each transaction needs to update every single block chain stored on every user's computer? (Moore's Law aside...)
As mentioned, end users will not need the entire blockchain. Just the headers and transactions that involve them specifically, which is a tiny part.

OK, so it's not the case that every transaction (between anyone) needs to be written to every copy of the block chain??
nimda
Hero Member
*****
Offline Offline

Activity: 784
Merit: 1000


0xFB0D8D1534241423


View Profile
June 28, 2012, 08:51:08 PM
 #14

The transactions can be pruned from it later.
yogi
Legendary
*
Offline Offline

Activity: 947
Merit: 1042


Hamster ate my bitcoin


View Profile
June 28, 2012, 09:04:07 PM
Last edit: June 28, 2012, 11:22:09 PM by yogi
 #15

The current implementation of bitcoin does have upper limits. Primarily relating to bandwidth and processing requirements of transactions.

If the adoption of bitcoin outpaces moore's law, which I believe it will, then it will become unworkable unless a solution is found.

I think a solution will be found which will likely involve multiple chains or clients that operate in clusters.




Alwaysmining
Member
**
Offline Offline

Activity: 72
Merit: 10



View Profile
June 28, 2012, 10:15:51 PM
 #16

Blockchains can be compressed, so i dont think so
Meni Rosenfeld
Donator
Legendary
*
Offline Offline

Activity: 2058
Merit: 1054



View Profile WWW
June 29, 2012, 04:04:15 AM
 #17

Thanks everyone for the solid answers and follow-up links. I see this matter has had a lot of discussion!

Seems that trimming the block chain is a viable solution if properly implemented.

Is it correct that every transaction that happens is written to the block chain of every user?

If so this would appear to present its own bandwidth problem.

In order to be viable as a long term sustainable currency, Bitcoin ultimately needs to be able to scale to billions of transactions per day (thousands per second). Is this possible if each transaction needs to update every single block chain stored on every user's computer? (Moore's Law aside...)
As mentioned, end users will not need the entire blockchain. Just the headers and transactions that involve them specifically, which is a tiny part.

OK, so it's not the case that every transaction (between anyone) needs to be written to every copy of the block chain??
It's more accurate to say that every transaction is written to every copy of the block chain, but not everyone has a copy of the block chain.

1EofoZNBhWQ3kxfKnvWkhtMns4AivZArhr   |   Who am I?   |   bitcoin-otc WoT
Bitcoil - Exchange bitcoins for ILS (thread)   |   Israel Bitcoin community homepage (thread)
Analysis of Bitcoin Pooled Mining Reward Systems (thread, summary)  |   PureMining - Infinite-term, deterministic mining bond
pyedpyper (OP)
Full Member
***
Offline Offline

Activity: 172
Merit: 100


View Profile
July 01, 2012, 05:55:09 PM
 #18

Thanks everyone for the solid answers and follow-up links. I see this matter has had a lot of discussion!

Seems that trimming the block chain is a viable solution if properly implemented.

Is it correct that every transaction that happens is written to the block chain of every user?

If so this would appear to present its own bandwidth problem.

In order to be viable as a long term sustainable currency, Bitcoin ultimately needs to be able to scale to billions of transactions per day (thousands per second). Is this possible if each transaction needs to update every single block chain stored on every user's computer? (Moore's Law aside...)
As mentioned, end users will not need the entire blockchain. Just the headers and transactions that involve them specifically, which is a tiny part.

OK, so it's not the case that every transaction (between anyone) needs to be written to every copy of the block chain??
It's more accurate to say that every transaction is written to every copy of the block chain, but not everyone has a copy of the block chain.

OK thanks for that clarification. To help me understand better: Say there are 1,000,000 people with Bitcoin accounts then how many people would have a copy of the block chain (based on present stats)?

I'm definitely getting the sense from the various answer there actually is a problem in the Bitcoin architecture i.r.o. scalability which has not yet been resolved. And this is clearly an issue. Relying on Moore's Law or hoping that a solution will "emerge" in the future when it is needed, is a dangerous way to proceed IMHO. Has anyone articulated a detailed solution yet whereby Bitcoin could scale to say 1 billion users (each making 1 transaction per day)?

Does anyone know how many Bitcoin users there presently are? Or what the current average transactions per day are?
yogi
Legendary
*
Offline Offline

Activity: 947
Merit: 1042


Hamster ate my bitcoin


View Profile
July 01, 2012, 06:14:50 PM
 #19

IMO scalability is the most pressing issue facing bitcoin at the moment.

For stats goto http://www.blockchain.info/

Meni Rosenfeld
Donator
Legendary
*
Offline Offline

Activity: 2058
Merit: 1054



View Profile WWW
July 01, 2012, 06:18:06 PM
 #20

OK thanks for that clarification. To help me understand better: Say there are 1,000,000 people with Bitcoin accounts then how many people would have a copy of the block chain (based on present stats)?
As many as are required to keep the network decentralized. I'd say 3,000 is enough, but it's possible to make do with less.

I'm definitely getting the sense from the various answer there actually is a problem in the Bitcoin architecture i.r.o. scalability which has not yet been resolved. And this is clearly an issue. Relying on Moore's Law or hoping that a solution will "emerge" in the future when it is needed, is a dangerous way to proceed IMHO. Has anyone articulated a detailed solution yet whereby Bitcoin could scale to say 1 billion users (each making 1 transaction per day)?
Did you read the page in the wiki about scalability that was linked in a previous comment?

The solutions don't need to emerge, the solutions are known and just need to be implemented.

Does anyone know how many Bitcoin users there presently are? Or what the current average transactions per day are?
How many users is anyone's guess, depends also on whom you call a user. In the past 24 hours there were 20338 transactions.

1EofoZNBhWQ3kxfKnvWkhtMns4AivZArhr   |   Who am I?   |   bitcoin-otc WoT
Bitcoil - Exchange bitcoins for ILS (thread)   |   Israel Bitcoin community homepage (thread)
Analysis of Bitcoin Pooled Mining Reward Systems (thread, summary)  |   PureMining - Infinite-term, deterministic mining bond
Pages: [1] 2 3 4 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!