on average, how much HD space does bitcoin-qt consume per day

[marcus_of_augustus]

No need to back up the blockchain, there are already plenty of copies around the world.

You should really make the note that only the blockchain copy on your machine has been validated by your own machine ....

... why are guys going to such lengths to obfuscate the dangers of not running your own full node I wonder?

Not everyone is quite as paranoid

Maybe, but then it wouldn't be the all bells and whistles "trust no-one currency" would it?

It must be paranoid by design ... oblivious to the human designer's subjective assessment of trust needed to function.

[1714687718]

1714687718

[1714687718]

1714687718

[1714687718]

1714687718

[Mike Christ]

Maybe, but then it wouldn't be the all bells and whistles "trust no-one currency" would it?

It must be paranoid by design ... oblivious to the human designer's subjective assessment of trust needed to function.

You're right   But that blockchain is a pain to download, and having to catch up every day takes a while.  Some find it appropriate to sacrifice that "trust no-one" stasis for convenience.

[Rassah]

Don't forget, SPV clients don't trust anyone either. They connect to different nodes and verify/transmit transactions from a few different nodes. If one is malicious, it will simply be ignored.

[caveden]

... why are guys going to such lengths to obfuscate the dangers of not running your own full node I wonder?

We're not "obfuscating" anything.
The danger you mention is comparable to the danger of running a piece of open source software without having previously read the entire source code and acquired a full understanding of how it works, before trusting to run it on your wallet.dat.

Thankfully, we can rest assured that a large enough number of people have already made such validation, as it's open to anyone to do it. Any attempt to insert bogus data, either on the source or on the blockchain, will be quickly spot. In the case of running full nodes particularly, there will always be more people capable of doing it - and actually doing it - than there are people capable of fully understanding Bitcoin source code.

I agree with that Bitcoin Magazine article that says that in 20 years, everybody will be capable of running a full node. The blockchain grows linearly once the adoption rate stabilizes. Hardware resources grow exponentially.

Don't forget, SVP clients don't trust anyone either. They connect to different nodes and verify/transmit transactions from a few different nodes. If one is malicious, it will simply be ignored.

Precisely. It's extremely difficult to lie to a SPV node. You need to be the only one it connects to, and still all you'll manage to do is hide stuff from it, not "invent" fake transactions.

[Mike Hearn]

Architecturally the system you describe is very similar to the modern banking system with high powered 'peers' upon whom the users are utterly dependent.  All that remains to be seen are what the ratio will be.  It is simply disingenuous to refer to it as a 'peer2peer' solution and retain any real meaning to the term.

If you've ever tried to set up a bank you'd know there is no comparison possible. A bank is not just a computer that tracks transactions, you know. And you can't just switch from one to another in a second or two.

SPV mode is appropriate for people with hardware constraints that let them get almost-as-good security at much less cost. If you want the full security you can of course upgrade to use a full node.

[marcus_of_augustus]

Any ETA on when these mythical SPV node solutions will be hitting github?

[caveden]

You just showed with this bad attempt of irony that you're not really aware about SPV.
BitcoinJ is a SPV node, which already uses bloom filters. MultiBit and Android Wallet are two clients which use BitcoinJ.

[tvbcof]

Architecturally the system you describe is very similar to the modern banking system with high powered 'peers' upon whom the users are utterly dependent.  All that remains to be seen are what the ratio will be.  It is simply disingenuous to refer to it as a 'peer2peer' solution and retain any real meaning to the term.

If you've ever tried to set up a bank you'd know there is no comparison possible. A bank is not just a computer that tracks transactions, you know. And you can't just switch from one to another in a second or two.

In the modern western world 'banks' (usually understood by the masses to be consumer banks) are corporations and like all corporations it is their legal obligation to maximize shareholder value.  They happen to deal with currency as an artifact of their doing business, and in doing so make use of centralized computers which interact with the computers of their 'peers' and those of central banks and regulatory bodies.  I'm just making this up, so please feel free to correct me if you feel that I've mis-spoken.

Other corporations maximize shareholder value in other ways, but visibility into the interactions inherent in a currency system can be extremely valuable to them.  Hence significant interest in 'virtual currencies'.

One thing that does not at all maximize shareholder value is crossing the government or regulatory bodies, so it would be unwise to expect corporations to do so.

SPV mode is appropriate for people with hardware constraints that let them get almost-as-good security at much less cost. If you want the full security you can of course upgrade to use a full node.

As long as it is realistic for a fair fraction of the user base to be full peers I'm not concerned or unhappy.  This is, to me, incompatible with unlimited or heavy growth.  I am not super concerned about mining because to me seems to be something which can be isolated and the system will still remain 'open'.  If latency increase dramatically (due to near-monopolization of mining power by corporations) that still will not preclude the use of Bitcoin as a reserve store of value.

My concern is that the solution will be grown enough such that full peers will require such specialized and expensive hardware and network connectivity that it will be practical for them to be isolated, controlled, and made inaccessible from large swaths of users through DPI and packet filtering.  Eventually.

I am also concerned that the business intelligence value of being able to closely monitor economic activity in a dominant currency solution will far exceed the costs of operation yielding a situation where it is provided as a 'free' service in the same vein as we see with other forms of communication over the internet these days.  This will greatly disrupt some of the economic incentives which are supposed to keep the system in balance.  Or at least 'supposed' by me to be part of the design.

[marcus_of_augustus]

You just showed with this bad attempt of irony that you're not really aware about SPV.
BitcoinJ is a SPV node, which already uses bloom filters. MultiBit and Android Wallet are two clients which use BitcoinJ.

Oh wait, you mean the java client put out by none other than Mike Hearn, the bitcoin dev. who is pushing hardest to raise the block limits, is really the only SPV implementation out there?

Gosh, wait is that a coincidence or irony, or you got me confused with someone else?

[Rassah]

You just showed with this bad attempt of irony that you're not really aware about SPV.
BitcoinJ is a SPV node, which already uses bloom filters. MultiBit and Android Wallet are two clients which use BitcoinJ.

Oh wait, you mean the java client put out by none other than Mike Hearn, the bitcoin dev. who is pushing hardest to raise the block limits, is really the only SPV implementation out there?

Gosh, wait is that a coincidence or irony, or you got me confused with someone else?

There's also a C client being developed, but, really, if there is basic code for it already, why reinvent the wheel?

Oh, and the actual clients that use BitcoinJ are quite different, with different features and usability. SPV mode is also currently being developed for the official Bitcoin-qt client. The new database format and bloom features that were released in version 0.8 were just the first steps for that process. Eventually, people will be able to install the official client, be able to use it within a few minutes as it downloads just the data relevant to its addresses, and if they have the hard drive space, be able to turn on the option to download the entire blockchain in the background. Stop complaining about features we don't have yet (especially when some of those features we do have). Bitcoin is still in beta, not even version 1.0 yet.

[marcus_of_augustus]

Stop complaining about features we don't have yet (especially when some of those features we do have). Bitcoin is still in beta, not even version 1.0 yet.

You misunderstand me (not complaining about lack of features) and agree with me at the same time.

Bitcoin needs a much more decentralised dev. structure before it comes out of beta or it will collapse in heap when the training wheels come off. At a minimum, we need a true blue team to compete, so the reds don't miss something or go off on a tangential group-think phase, just imho. Green, yellow and purples would be preferable but hey. There is also going to be too much pressure on too few brilliant, young minds, they'll blow, seen it a few times and its ugly. That is just the technological side, throw in hundreds of millions of money to the brew and it gets pretty heady ...

Read this to get a feel for some of the challenges. http://en.wikipedia.org/wiki/The_Soul_of_a_New_Machine

[Rassah]

If you think that's a problem, you are free to join github and add/critique all the suggestions on there. You are also free to come up with ideas and try to convince others that they are good, so that some developer somewhere can implement it.
I think you are under the impression that Bitcoin development consists of Gavin and maybe 3 other people. That's not how this project works. There are tons of suggested features and actual code suggested by a whole slew of people from around the world, and those features and code gets reviewed by others, and if proven to work right, get implemented in various clients. The core developers just check and fix the code and sort of herd the cats, in some attempt to keep things organized (plus write their own code, too, of course).

[arorts]

I don't think I've found the answer before but is it scalable to expect that the blockchain is downloaded to personal computers??

Today, the total volume of transactions is immaterial relative to other currencies, but say 5 years from now it represents a very large percentage. In that case, it would be ridiculous to have to download the entire history of transactions from day 0 in all computers. Unless there's a revolutionary technology that will allow the convenient storage of such info without requiring massive storage devices, it just doesn't seem practical to do so.

Perhaps I'm missing something but what did Satoshi think about this? How is this part of Bitcoin efficient? Thanks.

[marcotheminer]

Depends, each day it grows ever so quickly!

[arorts]

Depends, each day it grows ever so quickly!

"Depends"??   I don't think Satoshi left this to a random uncertainty. What's the existing architecture designed for? Each node containing all blockchain or is there an alternative option where larger miners would contain a larger section of the blockchain in the future for example?

I can't think of a different solution to deal with the infinite storage issue.  For example, does it make sense to purge the blockchain? or dynamically compress it? I'm just wondering what Satoshi actually envisioned in his paper.

[Rassah]

I don't think I've found the answer before but is it scalable to expect that the blockchain is downloaded to personal computers??

Today, the total volume of transactions is immaterial relative to other currencies, but say 5 years from now it represents a very large percentage. In that case, it would be ridiculous to have to download the entire history of transactions from day 0 in all computers. Unless there's a revolutionary technology that will allow the convenient storage of such info without requiring massive storage devices, it just doesn't seem practical to do so.

Perhaps I'm missing something but what did Satoshi think about this? How is this part of Bitcoin efficient? Thanks.

Here, read this https://en.bitcoin.it/wiki/Scalability

In short, you don't need the full copy of a blockchain to use bitcoin. Plenty of light clients just ping random nodes and request only the address balances that are relevant to them. Also, the blockchain can be pruned to only the addresses that have balances in them. If that was done today, the 12gig blockchain file would shrink to only something like 200 megs.

Satoshi envisioned that eventually only few copies of the entire blockchain will be stored for archive purposes, and large miners and pool operators (and wealthy hobbyists/supporters) will be the only ones keeping copies of the blockchain, with the general users just relying on the light clients that only request address balances, and sign/transmit transfers to the network.

[Etlase2]

Satoshi envisioned that eventually only few copies of the entire blockchain will be stored for archive purposes, and large miners and pool operators (and wealthy hobbyists/supporters) will be the only ones keeping copies of the blockchain, with the general users just relying on the light clients that only request address balances, and sign/transmit transfers to the network.

I don't think he "envisioned" that as much as "saw" that it was the only way it would work. It does not scale particularly well, and this is a weakness, not a feature. How weak a weakness is certainly debatable.

[old c coder]

Since 0.8 it became perfectly possible to run Bitcoin with its data dir in an USB disk, so I'm currently doing that to save space in my main disk (which is quite small).

Concerning your question, It's currently growing at no more than 250Kb per block, with ~144 blocks per day that should be at most ~36Mb of daily growth (it's actually less than that, not every block's hitting 250Kb). After May 15th, that growth rate might rise. It will take a hard-fork change for it to rise above 144Mb per day though, due to the 1Mb cap.

You can follow the total size here: http://blockchain.info/charts/blocks-size

144mb is way to much it already takes like 20 hours to download it. Soon we will all need 10TB hard drives for btc only <.<
And it take 4 months to download..

I beg to differ. The growth according to the graph of the link given, is more or less linear at about 10-11 GB /year (currently), so ~100GB/decade, ~1TB /century.

There will be spurts of growth in places like Cyprus, Argentina, and for the opposite reason in places such as Hungary, Iceland...Then plateaus of "tranquility"

As to how long to download, not too long. With a "good" internet connection it is easy to get 1 new block in ~<10 seconds. Now there are ~6 blocks created /hr, i.e 144/day. So to catch up for one day, you need 144 * 10 seconds = 1440 seconds = 1440/60 = 24  minutes. Notice about one minute for an hour of catch up. So a week of catch up ~ 24*7 minutes = 168 minutes  which is < 3 hrs. So a month catch up is about 3 * 4 or 12 hours, 2 months catch up will take ~ a day,  ~a year ~ 6 days,  and you can estimate further if you like. It's not that bad. It's actually better! The first blocks being small come really fast, so it takes about 3 days, more or less to get the whole block chain.

What this should tell you is that a "good" block chain is a lot of time saved if you need to start up bitcoin on a new machine! So save the files in those 3 directories:  [datadir]/blocks/*.*
[datadir]/blocks/index/*.*
[datadir]/chainstate/*.*
that comprise the levelDB database! It is time well saved. Almost worth backing up too. Since time is money for some, the block chain may be worth more to you than what's in your BTC (or is it now XBT) wallet!

Ron

[tvbcof]

Satoshi envisioned that eventually only few copies of the entire blockchain will be stored for archive purposes, and large miners and pool operators (and wealthy hobbyists/supporters) will be the only ones keeping copies of the blockchain, with the general users just relying on the light clients that only request address balances, and sign/transmit transfers to the network.

I don't think he "envisioned" that as much as "saw" that it was the only way it would work. It does not scale particularly well, and this is a weakness, not a feature. How weak a weakness is certainly debatable.

A good marketing campaign...if not a modestly technical wiki page with some appropriately designed jargon...can do wonders for turning a bug into a feature.  In a similar vein, one man's weakness is another man's strength.  There are plenty in the community who are probably completely happy to consolidate the infrastructure to a comfortable set of operators and get on with the business of running an economy with a proper level of oversight, control, and in some cases with a fair level of value extraction.

[gmaxwell]

Don't forget, SPV clients don't trust anyone either. They connect to different nodes and verify/transmit transactions from a few different nodes. If one is malicious, it will simply be ignored.

They do— they trust the peers they speak to more than a regular node does. E.g. if they show unconfirmed transactions they trust their peers to only tell them about valid unconfirmed transactions. When they receive blocks, they trust their peers to only tell them about valid blocks (and failing that, miners to only produce valid blocks. Especially when they display 1-confirmed transactions as "confirmed"). I don't believe _any_ SPV node software has a secure peer discovery implementation either, which is frustrating because they are somewhat more vulnerable to naughty peers.

IIRC electrum connects only to a single server at a time. It's SPV from there out but because it makes a single point of attachment it's pretty vulnerable. It also does no SSL certificate validation.   BitcoinJ wallets are differently bad: They connect to multiple, but they're only what get returned from DNS. Of course, the Bitcoin protocol itself is unauthenticated, so a network attacker near the node can isolate it easily.

Obviously SPV wallets will improve in the future. ... But they are no replacement for full nodes. If it wasn't important for many regular users to run full nodes then we wouldn't need full nodes at all.  When you don't run full nodes you are trusting miners to always follow the rules faithfully— to not steal from the users by inflating the coin, etc.  But if users are not rejecting unfaithful blocks, because they are SPV and can't validate them, what incentive is there for miners to behave quite so faithfully?  Will they watch themselves, mutually distrutful?  Not so clear, because of cloud mining and pooling the majority of hashpower is under the control of approximately 3 people right now. ... and increasing their income would be uniformly beneficial to them.

In classical currencies and banking the people with equivalent trust are the central banks and governments. There are many reasons the behavior of fiat currencies and banking in democratic regimes trustworthy: They are highly regulated, privileged institutions— not anonymous entities like mining— arguably controlled by elected officials.  And yet they are observably not trustworthy, and their lack of trustworthness— the lack of any institution depending on human fallibility— was a motivation for Bitcoin's creation.

But if you leave network security to a handful of self-appointed anonymous miners without the check of tens of thousands of regular users with uncorrelated interests outside of Bitcoin's integrity then the result is something even less trustworthy. I think that Bitcoin's value argument is too weak to be worth keeping it around if it doesn't offer an integrity improvement over the alternatives.

I'm just wondering what Satoshi actually envisioned in his paper.

Then perhaps you should read it? It's eight pages long and written in plain English.

At times I want to start banning people from this sub-forum who haven't read it. It is far from answering all questions, but there really is no excuse not to have read it.