Bitcoin Forum

There are some people that likes to look at the odometer of their car at the exact moment it pass from 99999 to 100000. In about three days from now people who use the official bitcoin client should see the apparition of blk0002.dat when the blockchain will be bigger than 2G.

The blockchain now weight 1950Mb. It grows at a rate of about 15Mb each day.

If it gets bigger than 4gb will windows xp support it ?

Yes no file will be larger than 2Gb.

Thanks, SpamtoshiDice!

Yes, if only people would stop using Bitcoin, THEN Bitcoin would thrive!

I don't want to highjack my own thread but there is a rumor that satoshidice was developed by a joint venture composed of Toshiba, Western digital and Seagate.  ;)

Actually maybe Bitcoin was their project!! Get hundreds of thousands of nerds to store several gigabytes of fake money on their hard drives... so stupid it might actually work :)

blk0001.dat will be at most 2130706432 bytes, or 2080768 KiB, or 2032 MiB.

There are ways of doing what you do without polluting the blockchain, you know.

If they are valid transactions then its not pollution its a problem with bitcoin itself that needs to be fixed.

To draw an analogy in real life: not everything that is legal to do, is considered well-mannered.

Thank you for saying that. 

SatoshiDice has done us a favor by identifying and demonstrating weaknesses in the algorithm that are ripe for improvement.

It's better to discover and fix vulnerabilities now, rather than have adversaries exploit them for attacks later.

/semi-threadjack

Satoshi even mentioned the issue in the original paper, so SatoshiDice most definitely wasn't first to identify this problem. You're only giving the developers less time to come up with well-written lite clients and transaction pruning, while turning more people off because of the increased size and lengthened download time of the blockchain. Nice job!

That's right, Satoshi did mention the issue(s) in his white paper.  And yes, SatoshiDice (but no, not *me* personally as I've yet to roll them) is increasing pressure on developers to deliver solutions.

Can you cite any evidence for your claim that it's "turning more people off because of the increased size and lengthened download time of the blockchain?"  Can you quantify such evidence if it exists?

Exactly (or about) how many nodes have been abandoned because their operators ragequit due to an (arguably prematurely) increased blockchain footprint?  How would you even begin an estimate?

SatoshiDice is subsidizing further progress by incentivizing (I hate that word) actual usage of a known exploit. 

While theoretical solutions may be developed based on the white paper, nothing can sufficiently approximate actual exploitation in the wild.  That's why testnets exist.

Thank you SatoshiDice, for giving us early warning of and opportunity to proactively respond to the inevitable blockchain growth problem, before any malicious adversary used the exploit to cause the type of critical, and potentially irrecoverable, damage often resulting from zero-day attacks!

Measures currently being deployed against (i.e. because of) SatoshiDice by pool operators have already hardened the network to some extent against this vector.  That's a Good Thing!

We must not blame the messenger and should instead be grateful for the valuable feedback SatoshiDice has provided, especially if blockchain bloat is as humungous an issue as many here believe.

Not to mention SatoshiDice has paid more miners' fees than all other bitcoin transactions combined in history.

That's another great point!

We want people to use Bitcoin, we *need* people to use Bitcoin.

SatoshiDice is helping popularize the use of Bitcoin for transactions. 

Blockchain growth and payment of miners' fees are direct measurements of its (and Bitcoin itself's) success. 

We're trying to present Bitcoin as ready for prime time and real world usage.  It doesn't help anybody to coddle and protect it like a delicate testnet.

This debate is simply an instance of the eternal white vs black hat approach to compsec.  It won't be resolved, but the discussion can enlighten and entertain both participants and spectators.

Does SatoshiDice have an API?  I'd like to see SatoshiDice become a back end gambling service layer for inclusion in all sorts of online/mobile gaming.  Let's accelerate Android generation penetration!

By penalizing the success of SatoshiDice we subsidize failure, and should only expect to get more of it.

More transactions please!  Grow the blockchain faster!  Peace through strength!  Onward cryptoanarchists!

/manifesto

That's right! It's also decreasing the time available for them to deliver said solutions.

These are just a few I found searching for keywords such as "slow" or "blockchain download":
https://bitcointalk.org/index.php?topic=56423.0
https://bitcointalk.org/index.php?topic=88311.0
http://www.reddit.com/r/Bitcoin/comments/o08f0/bitcoin_block_chain_server/
http://www.reddit.com/r/Bitcoin/comments/l636n/blocks_not_downloading_cant_get_my_coins_what/
http://www.reddit.com/r/Bitcoin/comments/jadmu/is_there_any_way_to_speed_up_the_initial/
http://www.reddit.com/r/Bitcoin/comments/qe90z/is_there_a_copy_of_the_bitcoin_transaction/
http://www.reddit.com/r/Bitcoin/comments/s60sj/why_is_block_downloading_so_slow/
http://www.reddit.com/r/Bitcoin/comments/vtoqg/my_bitcoin_blockchain_is_taking_days_to_download/

I don't know. It's very hard to give any real numbers. They don't have to completely quit and uninstall their client for their experience to be considered failed in my book. If the blockchain is taking a long time to download (sometimes days), they can't receive and spend bitcoins.

What?! This is not a zero-day.

The pools are only deprioritizing transations to and from the many 1dice... addresses, so this changes nothing in terms of other attackers. We would be no less or more prone to this if SatoshiDice were not around.

Same thing with the people who hacked Last.fm and LinkedIn before leaking millions of username and password combinations, right? Let's not blame the messenger, but the companies.

This is it : blk0002.dat is here.

Thanks for the on-point response mcorlett!

First, decreasing time available to deliver solutions is a good thing because that means there's also less time available for an adversary to use the vulnerability.

Second, the plural of anecdote is not data.  Some reddheads complaining about their lack of instant gratification does not indicate SatoshiDice has in any way retarded Bitcoin node growth.

That's why (third) you (and nobody else) know or have any way to give real numbers.  Extraordinary claims such as 'ZOMG SpamToshidie -> death of Bitcoin' require extraordinary evidence.  Not mere supposition and propagation of hearsay.

Fourth, while the 'transaction spam attack' and corresponding blockchain bloat are not 0-day in the narrow sense, they were in the broad sense that before SatoshiDice the network response to them was untested and no countermeasures had ever been put into place.

Fifth, pool ops are doing more than ignoring 1dice addresses.  I'm surprised that's the only response you're aware of.

No matter, the other countermeasures have already provided a number of ways to respond to intentionally malicious transaction spam.

Thus, we are in fact less prone to say, the ButtCoin clowns deciding to pay lots of transaction fees solely for the purpose of clogging up confirmations and blockchain downloads, than before SatoshiDice forced the issue.

Sixth, we are once again digressing into a generic white vs black hat debate.  Sure, I was mildly annoyed at having to change my linkedin password, but OTOH anyone registered at Gawker deserved to be doxed into a puddle of blubbering goo.  So you may scold and wag your finger at the black hats all you like, while I stand firm and enjoy epic lulz with Wikileaks and Anonymous.

This is not a security vulnerability. Please don't discuss it as if it were.

I would think it would be quite important to draw the distinction between complaining and being genuinely worried. SatoshiDice has directly lengthened the download speed when there were less resource-intensive options available.

SatoshiDice spams blockchain -> blockchain grows unnecessarily -> users complain -> users leave -> death of Bitcoin(?)

Wrong! There are attempts clearly visible in the blockchain, both on prodnet, as well as testnet. You may also take a look at the Litecoin alternative chain for some serious spamming.

Since you're so bothered with accurate sources, please list the countermeasures implemented directly because of SatoshiDice, sans "ignoring 1dice addresses" (which in itself is a flawed way to describe it).

Clogging up confirmations?! Such a claim would require extraordinary evidence.  Not mere supposition and propagation of hearsay. Please detail how exactly you go about doing such a thing in a manner more destructive than that of SatoshiDice.

I don't think I have to address this...

Well then.  That settles that.  Bitcoin is dooomed!  Because mcorlett said so.  And when pressed, endeavored to prooove (^^) it.

It's been a lot of fun and I'll miss you guys, but now there's no way that anybody will ever again bother to download such a massive blockchain.

Over 2 GIGAbytes?  How could that much data possibly be moved?  Maybe by sneakernet (forklift) on tape drives?  Lord knows our 2400 baud modems won't do it.

Not to mention that no operating system can handle such large files, what with them being limited to a 16-bit address space.

And don't forget, this is ALL the fault of SatoshiDice.  Bad, bad, terrible, evil, wicked SatoshiDice.  Why, oh why, did they have to go and ruin Bitcoin?

/so much FUD so little time

I see that my views give you cause to lash out at me. I'm not going to argue with you.

Have a nice day!

It's just a little sarcasm.  Harmless fun, yet effective at making the point that your FUD was transparently ridiculous.

That you would mischaracterize so mild a rebuke as 'lashing out at you' indicates just-a-little oversensitivity on your part.

Perhaps that's because the blockchain crossed the 2GB "milestone" and yet, almost nobody noticed, was effected, or indeed cared very much at all.

Except you, who felt the need to continue the argument while claiming 1) to be the poor victim of my 'lashing out' and 2) that you weren't going to argue with me.

Performative contradiction much?  Or only when your FUD amounts to nothing, except for inviting some gentle mockery?

Overlaying morality on top of bitcoin doesn't make it more resilient to attack.

When government-run water supplies run low, fines and invective are used to punish and embarrass those who might water their lawns or wash their cars. A free market solution is to simply raise the price of water - incentivising conservation. It sounds like that's what the miners have done (de-prioritizing SD txs). Berating SD doesn't seem to have accomplished anything.

Satoshi Dice has helped make it clear how easy it is to bloat the blockchain. If that slows adoption today, that's fine. Slow and steady growth is better than the boom and bust of publicity and exposed-flaw.

Lastly, it sounds like they're filling a market need. Apparently lots of people are "rolling the dice" because they want to. This is what P2P transactions are supposed to be about. No one should be able to stop anyone from sending bitcoin to anyone. This scenario was possible ever since the genesis block. If we've got a "tragedy of the commons" in the blockchain, then we've got to solve it in the model of P2P - not by name-calling.

Welcome to my hard drive, blk0002.dat.


We are 1/3 of the way to the blkindex.dat file also being >2GB, and there are several references to this file by name in the source, so I don't think provision has been made to split this file or use multiple databases if the index file grows beyond the filesystem file size limit (such as 4GB in FAT32).

All of those rational points don't mean anything because I can gainsay each one of them in a tit-for-tat manner.

You're just lashing out.  Ouch, stop lashing us!  I'm going to equate your rational argument to violence so I have an excuse to run away from the debate.  Just like that other guy.

Obey my boring, snotty, white hat morality or I shall tisk-tisk at you a second time, you terrible person.

Mcorlett is right, defending SatoshiDice and advocating proactive over reactive security is exactly the same as the immoral leaking of weak, innocent Yahoo passwords.  For shame, you SatoshiDice gambling addicts.  Get some help and stop destroying Bitcoin.

/SARCASM

Good question and I opened an issue for this on Github, to get some core dev attention!

Dia

You were wrong. Both of you were wrong:

07/10/2012  11:36 PM     2,097,281,755 blk0001.dat

I actually had blk0002 before you even started this thread!!!

That could aswell be the size on your hard-drive, not the real size of the file ^^. The size on your drive depends on the cluster size.

Dia

blk0002.dat  has already grown 100 MB  :(

I my opinion, the minimum 0.01 BTC transaction output rule (fees or no fees) should be reintroduced.  Yes it would be a clunky solution but it would just be temporary and it would give the bitcoin infrastructure time to develop to a point where it can scale.

Spamtoshidice and friends are getting a free ride.  Yes, they are paying transaction fees. But transaction fees go only to the miners, but not to the rest of us who are donating our HDs.  10-20 GB a year isn’t that much but it’s enough to discourage casual users from running full clients and strengthening the network.

Seriously. I know Bitcoin needs work on the scaling but seriously. A couple of gigabytes is absolutely nothing, you can get 8GB usb sticks really cheap these days. 500GB or 1 TB USB drives don't cost much. I would understand the problem if the blockchain was hundreds of gigabytes, but it is not. It's growing but there is plenty of time to work on the scaling and on top of that hard drives get cheaper all the time.

I've said this before and I'll say it again, not everyone needs to, or is supposed to, run a full node client. If you have a computer that struggles with a full node client, don't use a full node client, use a thin client! That's the number one scaling improvement we've had to this date so take advantage of it.

Pieter said "at most".  Bitcoin will not split a single block across two files, therefore blk0001.dat will almost always be smaller than precisely 2032 MiB.

The maximum block chain growth[1] is about 1GB per week.

[1] given 1MB max block size, which cannot be changed without a hard fork

This does not account for increasing hashrates or luck.

Hence "about"

Too many grammar Nazis on this forum ...

I just noticed the blk0002.dat myself and was wondering WTF is going on but it all makes sense now.

Use a good FS like ext4 and forget about Windblows trainwreck.

Even if FAT32 support would be removed, AFAIK there are used functions (was it ftell()?), which seem to have a size limitation of 4 GiB - 1 Byte (I'm sure it's in one comment in the source somwhere). So next stop 4 GB ;)?

Edit: From main.cpp (https://github.com/bitcoin/bitcoin/blob/master/src/main.cpp#L1947):

so FAT32 max file size == 4 GiB - 1 Byte
Dia

Yes, the interface is potentially limited to signed 32-bit.

We're working on replacing blkindex.dat with a LevelDB which uses many smaller files. So size limits on the bdb files should not be an issue by the time they arrive. Assuming anyone is dumb enough to run Bitcoin on a FAT filing system, of course.

As much as I hate gambling in general, SatoshiDice is providing a valuable service to Bitcoin, submitting valid transactions with fees to the network. This is exactly what we want to happen. The blockchain is growing because people are using it. 2 GB blockchain? That's nothing. I want to see a 2 TB blockchain! Bring it on!

Nooo!

I know that the general trend is that harddrive space is going up a lot, but for people like me who decided to be early adopters of SSD (<40 GB) I am slightly concerned about Bitcoin eating up my disk space.

+1 you ain't seen nothing yet.

It's going to be huge and there are going to be awesome solutions for all needs.

So you don't have equipment for a full client. In the future, you'll be in the vast majority. Just run an SPV client, nothing bad about that.

Where is this 1MB max block size coming from? And for how long it can hold?

Take a look at the latest blocks, a block with 400 transactions will reach 250KB in size, but 400 transactions in 10 minute is nothing if compared with a small stock exchange which typically can handle 1000 transactions in 1 second.

It means we are going to see at least 1000 times increase in size of each block if BTC's scale is to match one of those smaller stock exchanges

Milestone coming up today. We'll be just 1 million coins away from the drop.

OK, at least this part is scalable, I guess in the future it will more likely to be hosted on a data center like those banks do today, distributed data center maybe

All clients will reject blocks larger than 1MB as invalid.

Changing this is known as a "hard fork" change, something to be avoided because it knocks old clients off the network, limiting their access to their own bitcoins, and creates other technical problems.

I'm struggling to understand how this is not viewed as an impending catastrophe for Bitcoin. 1MB can only hold so many transactions, and there are already some blocks that are half that size. That would seem to mean that transaction volume can barely double from today's puny number before transactions are queued up for ages or even discarded unless they have juicy transaction fees. Of course a hardfork is highly undesirable, but don't you think it will have to happen, eventually?

I agree... this sounds like quite a problem.... ?

I think it will happen. In the meantime it is a guarantee that the chain can't get to some stupid size overnight.

It's no catastrophe, just means that some people will start to economize, not make 1 cent deposits and tiny bets to places or send their money back and forth for 'practice'.

http://1.bp.blogspot.com/-2ei59BEH4tU/Tf83itgomGI/AAAAAAAAAWE/3kSJxRbyL_E/s1600/Duplos.jpg (http://1.bp.blogspot.com/-2ei59BEH4tU/Tf83itgomGI/AAAAAAAAAWE/3kSJxRbyL_E/s1600/Duplos.jpg)

Once the block size limit is approached, miners will prioritize transactions based on fees.  Actually, they'll probably do that for other reasons long before it becomes necessary because of block size.

And there are several other potential reasons why we might need to do a hard fork in the future, so we can bundle them up if we need to.

Holy shit, that growth is scaring ^^.

Dia

Please, stop fooling neophytes. There is no such thing as official bitcoin client.

While some blocks may be 512KB that vast majority aren't.

If the average tx is ~512 bytes then 1 MB = 2000 tx per block.  24 * 6 * 2,000 = 288,000 tx per day.  That is roughly 7x to 8x current volume. 

The purpose of the hard fork is to ensure the network doesn't explode out of control due to a bug, unknown spam vulnerability.  Imagine if someone found a flaw in the spam rules and was able to spam 80 TB of tx.  Ooops.  Sorry new users you need to download 80 TB or you can't join Bitcoin.   Likewise if there was no max block size one could simply mine their own 1 GB blocks and even with only 1% of the network could add one or two of these every single day.  First year blockchain size = 1GB * (24*6*365*1%) = 525 GB (plus real tx volume).

The hard fork will be made in time but it will be a planned transition where miners vote by indicating their support in a flag in the block header.  Once some large (80%?) of miners support the new larger blocks and clients supporting the new blocks have been out a hardcoded date can be added to the client (i.e. 10MB blocks are supported starting in block 400,001).

Seems unlikely the 1mb limit will ever be removed.

That would be unfortunate. Bitcoin will never be anything more than a toy currency with that limit in place.

Not a toy currency, but quite the contrary. You don't trade SDR with your bartender currently, for instance, that doesn't make SDRs a "toy" currency.

Bitcoin is the currency of record, and as such it is to be used directly only by the very rich moving significant bits around. The consumer's needs are probably best handled by the currently existing infrastructure, Visa, paypal etc, except this time backed by the real deal (aka, Bitcoin) rather than by the worthless faith and missing credit of some rogue state or other.

Just like anyone wanting to open a Bitcoin exchange can buy a few S.DICE shares on MPEx and then offer a passthrough, just so any bank of the future will be buying a little Bitcoin and then offering the public various toy currencies, dollars, euros, whatever.

Interestingly enough, I've heard other people make the exact opposite argument - that we should use currencies like USD, EUR, gold, silver, etc for long-term savings because their value is far less volatile, and BTC as a way of facilitating efficient transactions between them all.

We could see other cryptocurrencies arise though. I'm a fan of Chaumian blind-coin myself, as it has far stronger anonymity properties, but it needs a centralized provider to operate, so we would need to have a few separate competing such currencies floating around to get a measure of decentralization through the free market. Once everything is done in crypto-currency it becomes easy to stick transactions between multiple crypto-currencies together, do trust-free atomic exchange and stuff like that, and use the best currency for each purpose.