Bitcoin Forum
December 11, 2016, 10:16:18 AM *
News: Latest stable version of Bitcoin Core: 0.13.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Pages: « 1 [2]  All
  Print  
Author Topic: Will the block chain size ever become too large?  (Read 18854 times)
abstraction
Member
**
Offline Offline

Activity: 107


View Profile
March 31, 2011, 01:28:19 AM
 #21

I don't see how that's not compressing random data, which is impossible.

It's taking an idea and finding all the random letters and all the random words in the correct order to define it, which is possible. The random letters come from the blockchain. "Randomness" is a perception.
1481451378
Hero Member
*
Offline Offline

Posts: 1481451378

View Profile Personal Message (Offline)

Ignore
1481451378
Reply with quote  #2

1481451378
Report to moderator
1481451378
Hero Member
*
Offline Offline

Posts: 1481451378

View Profile Personal Message (Offline)

Ignore
1481451378
Reply with quote  #2

1481451378
Report to moderator
Bitcoin addresses contain a checksum, so it is very unlikely that mistyping an address will cause you to lose money.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481451378
Hero Member
*
Offline Offline

Posts: 1481451378

View Profile Personal Message (Offline)

Ignore
1481451378
Reply with quote  #2

1481451378
Report to moderator
1481451378
Hero Member
*
Offline Offline

Posts: 1481451378

View Profile Personal Message (Offline)

Ignore
1481451378
Reply with quote  #2

1481451378
Report to moderator
steelhouse
Hero Member
*****
Offline Offline

Activity: 700


View Profile
May 16, 2011, 08:29:47 AM
 #22

Just look a how big your blkindex.dat and blk0001.dat increase everyday.  That is 2 MB per day.  They will be 1 gb at the end of the year.  So a fresh install might take you 3 hours to download the whole index.  That is already too big imho, and will double soon after that.  Downloading 1 days blocks already takes 10 minutes.  The idea of letting the big companies run the show could work, but 1) it is not done yet 2) it centralizes bitcoin, my as well use paypal.   Then you talk about light clients, they don't exist.  Then you say disk space will increase faster than bitcoin, it does not matter.  The reality is if you take todays bitcoin economy and 10x it you are now talking 20 mb per day, making bitcoin unusable for most people and most would never download the massive "virus".

You can clean out all the history as you only really need to know the last owner.  You can require people to maintain the whole block if they are to get transaction fees. If you go to sizes less than 0.01 then you can increase the block even faster.  Most responses so far are to sweep it under the carpet.
Mike Hearn
Legendary
*
expert
Offline Offline

Activity: 1526


View Profile
May 16, 2011, 09:04:52 AM
 #23

Lightweight clients do exist. BitcoinJ is one such implementation. It's not quite ready for everyday use yet though, that is true. Gavin is talking about upgrading the C++ client to do lightweight mode as his next priority, so don't worry, it'll happen.
gigitrix
Hero Member
*****
Offline Offline

Activity: 490


Bitcoins finest!!!


View Profile
May 16, 2011, 09:31:11 AM
 #24

By the time it's a problem, Bitcoin will have thousands and thousands of hardcore enthusiasts, possibly small businesses. The network will be fine even if Joe Public doesn't need the full client.
anisoptera
Member
**
Offline Offline

Activity: 98



View Profile
May 16, 2011, 11:25:38 PM
 #25

I don't see how that's not compressing random data, which is impossible.

It's taking an idea and finding all the random letters and all the random words in the correct order to define it, which is possible. The random letters come from the blockchain. "Randomness" is a perception.

So how do you encode the locations of the data in such a way that it doesn't take up more space than just storing the data?

online poker, bitcoin style - https://betco.in/
feeling tipsy? 1Q7ktWPwu4Q8MivKdmYxnmsGaBeauMTGwU
em3rgentOrdr
Sr. Member
****
Offline Offline

Activity: 434


youtube.com/ericfontainejazz now accepts bitcoin


View Profile WWW
May 22, 2011, 06:36:55 PM
 #26

It's interesting.  I have a background in computer architecture (Multicore CPU design), and the whole concept of bitcoin as a distributed time stamp network reminds me of the problem of multi-processor memory coherency (http://en.wikipedia.org/wiki/Cache_coherence).  Basically when there are many processors sharing the same memory space, there is the same issue about how to determine the correct global ordering of memory transactions.  For instance, if processor A writes to address X about the same time that processor B writes to address X, then what should be the final value of memory address X?  The problem arises because each processor keeps frequently accessed memory addresses in their local caches (and additionally each processor is most likely speculative and executes out-of-order), so it is difficult to determine what is the "correct" memory transaction ordering without some shared arbitrator.  But for your typical shared-bus multi-core CPU that most of us are proably using right now, there is a shared bus connected all the processors' private caches together with the last level shared cache or main memory, so it is relatively easy since the the shared bus can act as global arbitrator which determines the "correct" global order of memory transactions based on the order in which memory transaction appear on the shared bus.  And the individual caches have special cache coherence protocols (e.g. MESI) that snoop the shared bus and update cached memory values efficiently.

However, shared busses are not scalable since they become very slow as the number of connected components increases, making them impracticable for more than 16 cores or so.  Larger shared-memory multiprocessor systems must use slightly more clever systems for dealing with memory coherency since the interconnect fabric is much more complex and may not have any centralized arbitrators.  There are hundreds of research papers dealing with this in the past 30 or so years.  One such solution is distributed directory-based cache coherency, whereby each processor cache controller is responsible for handling the transaction ordering for a certain memory range.  For instance, if there are 256 cores on a chip arranged in a 2-D grid of 16x16 cores, then the memory address space would be divided up into 256 different sections.  Whenever a cpu generates a memory transaction, that transaction is forwarded to the cache that is responsible for keeping track of the ordering and sharing based on some bits of that particular memory addresses.

Regarding bitcoin, it seems the current design has the same drawback of a shared bus cache coherenece since miners must process every single transaction (well technically not "every" transaction, since if a transaction is not put in the block chain then that transaction never *really* happened Smiley ).  It seems to me that as the bitcoin network reaches the scale of millions of users, then it will become impractical to broadcast every transaction to every single miner.  A better solution would be along the lines of distributed directory-based cache coherency, whereby a particular miner is only responsible for handling transactions from coins belonging to a certain address range.  Basically, the ordering of two transactions that use different coins is not important to the problem of double-spending.  Rather, it is only important to maintain the proper ordering for transactions that use the same coins.  Two transactions that use an entirely mutually-exclusive set of coins can be reordered with respect to one-another in any manner without having any issue of double spending (of course if two transactions share some coins but not all, then care will needed to be taken to ensure proper global ordering).  So I would be interested in some modification of the bitcoin protocol whereby different miners may be responsible for hashing transactions of a subset of the coin address space.  Of course selecting which address space I am responsible for mining and broadcasting this info to all bitcoin clients is not trivial.  And of course eventually these mini-block chains (which are only handling part of the coin address space) would need to be reincorporated with the rest of the global block chain whenever there are transactions involving coins from multiple addresses spaces.  But anyway, once this network bandwidth problem becomes a real issue, then I am confident that such a fork could be implemented.  (One possible "solution" that doesn't involve *any* code rewriting would be to simply allow multiple block chains to exist simultaneously, and then simply perform a standard currency exchange between the different competing bitcoin block chains whenever there is a transaction involving coins from different block chains.  This would effectively divide up the coin addresses space into different mutually non-overlapping address chunks that could be processed entirely independently.  Overtime, the exchange rate between different bitcoin block chains would stabilize, and would become effectively one currency as far as the layman is concerned.  Of course, the downside is that each individual block chain would be weaker than a single global united powerful blockchain.).

"We will not find a solution to political problems in cryptography, but we can win a major battle in the arms race and gain a new territory of freedom for several years.

Governments are good at cutting off the heads of a centrally controlled networks, but pure P2P networks are holding their own."
silversurfer1958
Full Member
***
Offline Offline

Activity: 219


View Profile WWW
May 13, 2014, 01:29:15 PM
 #27

Isn't there some way of limiting blockchain download so that each node only need download a portion. It's so obvious that it must be what you developers are working on.

If the Blockchain is to be handled by miners or other limited users, then, Bitcoin becomes centralised and open to attack as any centralised entity.  How about Clients only acting on 3 first characters worth of the Blockchain, randomly. IE, My client will only process transactions that begin with XYZ........ If this were hidden from users and done randomly, there'd be no way for anyone to know who was veryfying what portions of the Blockchain  and the blockchain would then itself effectively be decentralised.     So, the idea is, have each client only download transactions that begin  XYZ and will only verify transactions that begin XYZ.

It only means that when you send Bitcoin, the transaction sends out a verification request......ahhh, but how does it know where to send the request....
It would have to trawl the entire net to find a node that will verify it's XYZ transaction, and so would slow down verifications. 
And so Ideally, Any node should be able to verify a transaction, but of course it can't, because it doesn't have the entire blockchain.  (why do I get the feeling I'm painting myself into a corner here) 
What about then, if when you boot up the client it actively seeks out only those nodes which will deal with and verify XYZ transactions. It might mean a sync delay initially while it finds enough XYZ nodes to verify your transaction but once it's found them, anyway, My thoughts on the Blockchain size problem.   A distributed Blockchain, I'm sure that's what they are working on anyway since it's obvious so I've just wasted 5 minutes of my (and your ) life,       sorry about that  Roll Eyes     
DeathAndTaxes
Donator
Legendary
*
Offline Offline

Activity: 1218


Gerald Davis


View Profile
May 13, 2014, 01:43:07 PM
 #28

Isn't there some way of limiting blockchain download so that each node only need download a portion. It's so obvious that it must be what you developers are working on.

If the Blockchain is to be handled by miners or other limited users, then, Bitcoin becomes centralised and open to attack as any centralised entity.  How about Clients only acting on 3 first characters worth of the Blockchain, randomly. IE, My client will only process transactions that begin with XYZ........ If this were hidden from users and done randomly, there'd be no way for anyone to know who was veryfying what portions of the Blockchain  and the blockchain would then itself effectively be decentralised.     So, the idea is, have each client only download transactions that begin  XYZ and will only verify transactions that begin XYZ.

It only means that when you send Bitcoin, the transaction sends out a verification request......ahhh, but how does it know where to send the request....
It would have to trawl the entire net to find a node that will verify it's XYZ transaction, and so would slow down verifications. 
And so Ideally, Any node should be able to verify a transaction, but of course it can't, because it doesn't have the entire blockchain.  (why do I get the feeling I'm painting myself into a corner here) 
What about then, if when you boot up the client it actively seeks out only those nodes which will deal with and verify XYZ transactions. It might mean a sync delay initially while it finds enough XYZ nodes to verify your transaction but once it's found them, anyway, My thoughts on the Blockchain size problem.   A distributed Blockchain, I'm sure that's what they are working on anyway since it's obvious so I've just wasted 5 minutes of my (and your ) life,       sorry about that  Roll Eyes     

No.  What you describe is a (poorly designed) DHT but DHT in general is a poor security structure for the blockchain.   A full node is a full node.  It has to be independent.  If it need to rely on other nodes then it isn't independent, if nobody is independent then that is a huge problem.  Not everyone needs to be a full node but the network needs independent full nodes.  If you don't want to run a full node then look to an SPV client.   

The blockchain can be pruned.  The pruned size is ~1/3rd of full blockchain and will decline as a % of full size over time.   That is the solution which is being worked on.  A pruned blockchain doesn't reduce the security of a full node, they can continue to operate independently.
MoonShadow
Legendary
*
Offline Offline

Activity: 1666



View Profile
May 13, 2014, 06:17:24 PM
 #29

Isn't there some way of limiting blockchain download so that each node only need download a portion. It's so obvious that it must be what you developers are working on.

Short answer, yes.

Longer answer, it's complicated, but yes.

One method currently available is to use a 'light client' that only downloads and keeps a copy of the block headers plus the blocks that contain confirmed transactions related to it's own wallet.dat balance.  The wallet.dat keeps a copy of the particular transactions anyway, so it's relatively easy to do, but I know of no desktop clients that don't use a full-client model.  The problem here is that, while light clients can prove that they owned those coins at one time, they are dependent upon the vendor's client being a full client (or have some other method of full verification) in order to spend coins; and light clients have issues with acceptance of coins, because they can't independently confirm that a transaction received is still valid.  Only that the coins were once owned by the sender, and (usually, but not absolutely) that the transactions offered should fit in the block where the sender claims they came from.

Another method is what I've been calling "naked block transmission protocol" but I'm sure that is not what Gavin and the developers are calling it.  Wherein a new block, even for a full client, is broadcast only with the header and the full merkle tree; thus stripped of the actual transaction data.  On the assumption that all full clients would have already seen the vast majority of the transaction data when it was released loose on the network up to 10 minutes prior.  This would improve the network throughput from between 2 to 10 times, or about 6 for the average block.  This wouldn't improve a full-client bootstrapping time at all, but, a kind of vendor's client could be developed from this that has more data than a light client but not as much as a full client.

There is also the parallel network method, that Stratum & a couple of android clients are using.

And for full clients that no longer need old data locally, there is the pruning protocol that has always been in Bitcoin, but never implimented; which would allow full clients to discard long spent transactions.

"The powers of financial capitalism had another far-reaching aim, nothing less than to create a world system of financial control in private hands able to dominate the political system of each country and the economy of the world as a whole. This system was to be controlled in a feudalist fashion by the central banks of the world acting in concert, by secret agreements arrived at in frequent meetings and conferences. The apex of the systems was to be the Bank for International Settlements in Basel, Switzerland, a private bank owned and controlled by the world's central banks which were themselves private corporations. Each central bank...sought to dominate its government by its ability to control Treasury loans, to manipulate foreign exchanges, to influence the level of economic activity in the country, and to influence cooperative politicians by subsequent economic rewards in the business world."

- Carroll Quigley, CFR member, mentor to Bill Clinton, from 'Tragedy And Hope'
Pages: « 1 [2]  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!