Right now the capacity of Bitcoin is about half a million transactions per day. So you can participate in that level of transactions as a hobby. The value of those transactions can be as high as required. If Bitcoin does become a widespread store of value, blocks will probably be transferring hundreds of million of dollars worth of value each, tens of billions every day.

But after all, it's just information, so yes, participating will be perfectly possible as a hobby, and for a fairly affordable fee, you'll be able to even make transactions directly on the world's decentralized value transfer service, the same system big banks will use.


Absolutely. But the solution isn't to make access to the core Bitcoin network, the thing that actually keeps Bitcoin inflation free and secure, require such a huge investment in computer hardware that only big banks and other large institutions can afford access. The solution is to keep blocks small, and build payment systems that work on top of the block chain.

Remember that if the blockchain is kept small enough that validating it is affordable, you don't have to trust the payment processors very much. The protocols will be designed in ways that allow anyone to prove fraud automatically and warn the whole world. The client software people use will see these fraud proofs, and immediately stop using the payment processor, putting them out of business. Yet at the same time, using technologies like chaum tokens, those payment processors can't even know where payments are going too; you're privacy is even more protected than with on-chain transactions, because the links connecting one transaction to another are severed with unbreakable mathematics.

Do you think the banking crisis would have happened if banks were forced to have all their bank-to-bank transactions publicly recorded for the whole world to see? Keeping the blocksize limited does exactly that.


Ok, so a 10GiB block is unacceptably large. What about a 5Gib block? Or a 1GiB block? Or a 500MiB block? At some point the block will be confirmed by a large fraction of the hashing power, but not all the hashing power. The hashing power that couldn't process that gigantic block in time has effectively dropped off of the network, and is no longer contributing to the security of the network.

So repeat the process again. It's now easier to push an even bigger block through, because the remaining hashing power is now less. Maybe the hashing power has just given on on Bitcoin mining, maybe they've redirected their miners to one of the remaining pools that can process such huge blocks, either way, bit by bit the process inevitably leads to centralization.


Pool operators do own hashing power if the miners contributing the hashing power can't effectively validate the blocks they mine.

If running a validating node requires thousands, or even tens of thousands, worth of expensive equipment, how exactly do you expect to even find out that you've been mining at a dishonest pool? If >50% of the people mining and running validating pools decide to get together and create bogus transactions creating coins out of thin air, you won't even know they've been defrauding everyone.(1) If running a node requires tens of thousands of dollars worth of equipment, and it will to support Visa-scale transaction volumes, only a small handful of large banks are going to run nodes. I think you can see how collusion between half-a-dozen large banks becomes not just possible, but likely.

1) Yes, you can try to create automated fraud proof mechanisms to detect it - I wrote about the idea here - but implementing the software to process fraud proofs is extremely complex, much more complex than applying the same idea to keeping off-chain banking services honest. I also have little hope that those mechanisms will actually get written and tested before the much more simple step of just lifting the block limit is taken.


I agree %100. Increasing the block limit seems like a conservative change - it's just one little number - but the long-term implications are enormous and have the potential to drastically change what Bitcoin is. It may be a conservative change for the small number of big businesses that are heavily invested in the current system, and can afford the network power to process large blocks, but it's not a conservative change for the rest of us.

dooglus: Just sent you a 0.1BTC donation for your excellent Satoshidice analysis work so you can buy some chainsaw pants.

You have to be careful with transmitting transaction hash lists rather than the transactions themselves. While it definitely makes propagation faster in the average case, it also means that the worst-case, a block entirely composed of transactions that have not been previous broadcast on the network, is significantly worse. For the purpose of just reducing orphans, as is done by P2Pool as gmaxwell pointed out, the worst case isn't a problem, but don't make assumptions that have security implications. In particular by the mechanism I pointed out here you'll actually create a market for large hashpower miners where they can offer lower fees, paid for by the lower effective competition, provided the sender promises not to send the transaction to any other miner. I can't see such markets developing for 1MiB blocks, but they might develop for much larger block sizes.

P2Pool is interesting because miners on it have an incentive for any P2Pool block to be propagated to the network as a whole as fast as possible. In addition the perverse propagation incentives for shares within P2Pool are probably less of an issue given that the higher the hash power of P2Pool as a whole, the lower the variance for any individual miner - miners on P2Pool aren't playing a zero-sum game. P2Pool miners also have very little control over propagation because P2Pool shares are all identical.

Well, read my initial post at the top; that larger blocks don't propegate to the network as a whole is actually benefits the miner because provided the blocks propagate to more than 50% of the effective hashing power, the part that doesn't get the block is effectively wasting their mining effort and taken out of the competition.

Additionally even if miners see a rational reason to keep block-sizes low, which I already doubt, allowing them to control the size gives irrational miners who are trying to actively damage Bitcoin another way to do so. Right now all that an evil miners can do is either help double-spend attempts, easily defeated with confirmations, or launch a 51% attack, so far defeated with large amounts of hashing power. We don't want to give yet more ways for malicious people to damage Bitcoin, especially ones which are actually profitable in the short term.


I mean, I'm not totally against a one-time increase if we really need it. But what I don't want to see is an increase used as a way to avoid the harder issue of creating alternatives to on-chain transactions. For one thing, Bitcoin will never make for a good micropayments system, yet people want Bitcoin to be one. We're much better off if people work on off-chain payment systems that complement Bitcoin, and there are plenty of ways, like remote attestation capable trusted hardware and fidelity bonds, that allow such systems to be made without requiring trust in central authorities.

I would hate to see the limit raised before the most inefficient uses of blockchain space, like satoshidice and coinad, change the way they operate. In addition I would hate to see alternatives to raising the limit fail to be developed because everyone assumes the limit will be raised. I also get the sense that Gavin's mind is already made up and the question to him isn't if the limit will be raised, but when and how. That may or may not be actually true, but as long as he gives that impression, and the Bitcoin Foundation keeps promoting the idea that Bitcoin transactions are always going to be almost free, raising the block limit is inevitable.

Anyway, there are plenty of good reasons to have off-chain transactions regardless of what the block limit is. In particular they can confirm instantly, so no waiting 6 blocks, and they can use chaum tokens to truely guarantee that your transactions are private and your personal financial information stays personal.

That has got to be one of the most accurate descriptions of the security of Bitcoin I've seen in non-Bitcoin press.

For consumer products where you get a tangible object in return. Security through hashing power is nothing like kickstarter.


No, that's 1.2MiB average; you need well above that to keep your orphan rate down.

Again, you're making assumptions about the hardware available in the future, and big assumptions. And again you are making it impossible to run a Bitcoin node in huge swaths of the world, not to mention behind Tor.


I'm assuming 1% of transactions per month get added to the UTXO set. With cheap transactions increased UTXO set consumption for trivial purposes, like satoshidice's stupid failed bet messaging and timestamping, is made more likely so I suspect 1% is reasonable.

Again, other than making old UTXO's eventually become unspendable, I don't see any good solutions to UTXO growth.


I mean proof of work hashing for mining. If you don't know what transactions were spent by the previous block, you can't safely create the next block without accidentally including a transaction spent by the previous one, and thus invalidating your block.


Don't be silly. Even in 1993 people knew that you would be able to do things like have DNS servers return different IP's each time - Netscape's 1994 homepage used hard-coded client-side load-balancing implemented in the browser for instance.

DNS is another good example: the original hand-maintained hosts.txt file was unscalable, and sure enough it was replaced by a the hierarchical and scalable DNS system in the mid 80's.


...and what do you know, one of the arguments for IPv6 even back in the early days in the early 90's was the IPv4 routing space wasn't very hierarchical and would lead to scaling problems for routers down the line. The solution implemented has been to use various technological and administrative measures to keep top-level table growth in control. In 2001 there were 100,000 entries, and 12 years later in 2013 there are 400,000 - nearly linear growth. Fortunately the nature of the global routing table is that linear top-level growth can support quadratic and more growth in the number of underlying nodes; getting access to the internet does not contribute to the scaling problem of the routing table.

On the other hand, getting provider-independent address space, a resource that does increase the burden on the global routing table, gets harder and harder every year. Like Bitcoin it's an O(n^2) scaling problem, and sure enough the solution followed has been to keep n as low as possible.

The way the internet has actually scaled is more like what I'm proposing with fidelity-bonded chaum banks: some number of n banks, each using up some number of transactions per month, but in turn supporting a much larger number m of clients. The scaling problem is solved hierarchically, and thus becomes tractable.

Heck, while we're playing this game, find me a single major O(n^2) internet scaling problem that's actually been solved by "just throwing more hardware at it", because I sure can't.


Appeal to authority. Satoshi also didn't make the core and the GUI separate, among many, many other mistakes and oversights, so I'm not exactly convinced I should assume that just because he thought Bitcoin could scale it actually can.

Network assurance contracts are far from a sure thing. It's basically an attempt to solve the tragedy of the commons, and the success rate society has had there is pitiful, even with strong central authorities. Assuming they will work is a big risk.


I don't see any reason to think CPU power will be the issue. It's network capacity and disk space that is the problem. You're 100x the volume of PayPal is 4000 transactions a second, or about 1.2MiB/second, and you'll want to be able to burst quite a bit higher than that to keep your orphan rate down when new blocks come in. Like it or not that's well beyond what most internet connections in most of the world can handle, both in sustained speed and in quota. (that's 3TiB/month) Again, P2Pool will look a heck of a lot less attractive.

You also have to ask the question, what % of that 3TiB/month results in unspent txouts? Ultimately it's the UTXO set that is the hard limit on the storage requirements for full validating nodes. Even at just 1% volume growth, you're looking at 3GiB/month growth in your requirement for fast-random-access memory. That's an ugly, ugly requirement - after all if a block has n transactions, your average access time per transaction must be limited to 10minutes/n to even just keep up.

EDIT: also, it occurs me me that one of the worst things about the UTXO set is the continually increasing overhead it implies. You'll probably be lucky if cost/op/s scales by even something as good as log(n) due to physical limits, so you'll gradually be adding more and more expensive constantly on-line hardware for less and less value. All the time you're spending waiting for transactions to be retrieved from memory is time you aren't hashing. In addition your determinism goes down because inevitably the UTXO set will be striped across multiple storage devices, so at worst every tx turns out to be behind one low-bandwidth connection. God help you if an attacker figures out a way to find the worst sub-set to pick. UTXO proofs can help a bit - a transaction would include it's own proof that it is in the UTXO set for each txin - but that's a lot of big scary changes with consensus-sensitive implications.

Again, keeping blocks small means that scaling mistakes, like the stuff Sergio keeps on finding, are far less likely to turn into major problems.


You're example has nothing to do with Bitcoin. Even in the early days it would be obvious to anyone who understood comp-sci that static websites are O(1) scaling per client so there isn't any reason to think you couldn't create websites for as much load as you wanted. Meanwhile unlike Wikipedia Bitcoin requires global shared state that must be visible to, and mutable by, every client. Comparing the two ignores some really basic computer science that was very well understood even when the early internet was created in the 70's.

Well, one big objection is the code required is very similar to that required by fidelity-bonded bank/ledger implementations, but unlike the fidelity stuff, because it's consensus screwing it up creates problems that are far more difficult to fix and far more widespread in scale.



"This mining this is crazy, like all that work when you could just verify a transaction's signatures, and I dunno, ask a bunch of trusted people if the transaction existed?"

So, why do we give miners transaction fees anyway? Well, they are providing a service of "mining a block", but the real service they are providing is the service of being independent from other miners, and we value that because we don't want >50%  of the hashing power to be controlled by any one entity.

When you say these small miners are inefficient, you're completely ignoring what we actually want miners to do, and that is to provide independent hashing power. The small miners are the most efficient at providing this service, not the least.

The big issue is the cost to be a miner comes in two forms, hashing power and overhead. The former is what makes the network secure. The latter is a necessary evil, and costs the same for every independent miner. Fortunately with 1MiB blocks the overhead is low enough that individual miners can profitably mine on P2Pool, but with 1GiB blocks P2Pool mining just won't be profitable. We already have 50% of the hashing power controlled by about three or four pools - if running a pool requires thousands of dollars worth of equipment the situation will get even worse.

Of course, we've also been focusing a lot on miners, when the same issue applies to relay nodes too. Preventing DoS attacks on the flood-fill network is going to be a lot harder when when most nodes can't verify blocks fast enough to know if a transaction is valid or not, and hence the limited resource of priority or fees is being expended by broadcasting it. Yet if the "solution" is fewer relay nodes, you've broken the key security assumption that information is easy to spread and difficult to stifle.


Until Bitcoin has undergone a serious attack we just aren't going to have a firm idea of what's "too much centralization"

Well you'll have to implement the fraud proofs stuff d'aniel talked about and I later expanded on. You'll also need a DHT so you can retrieve arbitrary transactions. Both require a heck of a lot of code to be written, working UTXO for fraud proofs in particular; random transaction verification is quite useless without the ability to tell everyone else that the block is invalid.

Things get ugly though... block validation isn't deterministic anymore: I can have one tx out of a million invalid, yet it still makes the whole block invalid. You better hope someone is in fact running a full-block validator and the fraud proof mechanism is working well or it might take a whole bunch of blocks before you find out about the invalid one with random sampling. The whole fraud proofs implementation is also now part of the consensus problem; that's a lot of code to get right.

In addition partial validation still doesn't solve the problem that you don't know which tx's in your mempool are safe to include in the next block unless you know which ones were spent by the previous block. Mining becomes a game of odds, and the UTXO tree proposals don't help.  A UTXO bloom filter might, but you'll have to be very careful that it isn't subject to chosen key attacks. Transaction volume itself leads to centralization too, simply by ensuring that only a miner able to keep up with the large volume of low-fee transactions can make a profit.

I've already thought of your idea, and I'm sure gmaxwell has too... our imagination didn't "run out"

I primarily want to keep the limit fixed so we don't have a perverse incentive. Ensuring that everyone can audit the network properly is secondarily.

If there was consensus to, say, raise the limit to 100MiB that's something I could be convinced of. But only if raising the limit is not something that happens automatically under miner control, nor if the limit is going to just be raised year after year.


Yes, there will likely only be around 10 billion people on the planet, but that's a hell of a lot of transactions. At one transaction per person per day you've got 115,700 transactions per second. Sorry, but there are lots of reasons to think Moore's law is coming to an end, and in any case the issue I'm most worried about is network scaling, and network scaling doesn't even follow Moore's law.

Making design decisions assuming technology is going to keep getting exponentially better is a huge risk when transistors are already only a few orders of magnitude away from being single atoms.


The fact that miners include transactions at all is a great example of how small  the block limit is. Right now the risk of orphans due to slow propagation is low enough that the difference between a 1KiB block and a 250KiB block is so inconsequential that pools just run the reference client code and don't bother tweaking it. I wouldn't be the slightest bit surprised to be told that there aren't any pools with even a single full-time employee, so why would I expect people to really put in the effort to optimize revenue, when it'll probably lead to a bunch of angry forum posts and miners leaving because they think the pool will damage Bitcoin?


I don't have any interest in working on a system that boils down to a complicated and expensive replacement for PayPal.

Decentralization is the fundamental thing that makes Bitcoin special.

That's exactly what I mean. Unfortunately it's true, and the solution has been to use the code Satoshi wrote, the reference client, as a module by itself to talk to the network, and then write other modules, such as your wallet code and business logic, that interfaces to the Satoshi client.

If my answer results in you not being able to trust Bitcoin, then sell your coins and stop using it.


Alright, so you are talking about either one of two things:

1) The reference implementation should use a reverse-delta scheme to actually store transactions. But as I've said, the current system runs fine.

2) You're suggesting implementing the UTXO concept, probably as a hard-fork change. Funny enough though, I'm actually working on writing a prototype UTXO implementation right now based on TierNolan's suggestion to use Radix/PATRICIA trees. A UTXO set is definitely something the devs want to see in the reference client, although it's a long-term goal, and in any case no-one has even done a prototype yet. If it is adopted, yes, blocks may eventually be 100% reverse-delta, but that's a really long way off because you would need a consensus...



...or maybe you're confusing reverse-delta schemes with the blocksize limit. Reverse-delta doesn't decrease the size of blocks, only the amount of data needed to prove the existence of a given transaction. (this is why I'm doing my prototype; I'll need it for fidelity bonded trusted ledgers eventually) Again, RFCs and alternate implementations have nothing to do with this issue.

Anyway, I've wasted my time enough and have real work to do.

Hey, I want a pony too. But Bitcoin is an O(n) system, and we have no choice but to limit n.


A "highly centralized" system where anyone can get a transaction confirmed by paying the appropriate fee? A fee that would be about $20 (1) for a typical transaction even if $10 million a day, or $3.65 billion a year, goes to miners keeping the network secure for everyone?

I'd be very happy to be able to wire money anywhere in the world, completely free from central control, for only $20. Equally I'll happily accept more centralized methods to transfer money when I'm just buying a chocolate bar.


1) $10,000,000/144blocks = $69,440/block
     / 1MiB/block = $69.44/KiB

A two-in, two-out transaction with compressed keys is about 300 bytes, thus $20.35 per transaction.


That sounds like a whole lot of "maybe" I agree that we need to move cautiously, but fundamentally I've shown why a purely profit driven miner has an incentive to create blocks large enough to push other miners out of the game and gmaxwell has made the point that a purely profit driven miner has no incentive not to add an additional transaction to a block if the transaction fee is greater than the cost in terms of decreased block propagation leading to orphans. The two problems are complementary in that decreased block propagation actually increases revenues up to a point, and the effect is most significant for the largest miners. Unless someone can come up with a clear reason why gmaxwell and myself are both wrong, I think we've shown pretty clearly that floating blocksize limits will lead to centralization.

Variance already has caused the number of pools out there to be fairly limited; we really don't want more incentives for pools to get larger.


They want something impossible from an O(n) system without making it centralized. We've already got lots of centralized systems - creating another one doesn't do the world any good. We've only got one major decentralized payment system, Bitcoin, and I want to keep it that way. Users can always use centralized systems for low-value transactions, and if block sizes are limited they'll even be able to very effectively audit the on-chain transactions produced by those centralized systems. Large blocks does not let you do that.

Ultimately, the problem is the huge amount of expensive infrastructure built around the assumption that transactions are nearly free. Businesses make decisions based on what will happen at most 3-5 years in the future, so naturally the likes of Mt. Gox, BitInstance, Satoshidice and others have every reason to want the block size limit to be lifted. It'll save them money now, even if it will lead to a centralized Bitcoin five or ten years down the road.

Yes, but an empty block doesn't earn you any revenue as the block reward drops, so mining is still pointless. You still need the full block to know what transactions were mined, and thus what transactions in the mempool are safe to include in the block you want to attempt to mine.

Additionally without the full block, you don't know if the header is valid, so you are vulnerable to miners feeding you invalid blocks. Of course, someone has to create those invalid block hashes, but the large miners are the only ones who can validate them, so if smaller miners respond by taking risks like mining blocks even when they don't know what transactions have already been mined, the larger miners can run lots of nodes to find those blocks as fast as possible, and distribute them to other small miners without the equipment to validate the blocks.


Sure, but all the scenarios where extremely large blocks are allowed are also assuming that most people only run mini-SPV clients at best; if one of the smaller "full-node transaction feed" services gets put off-line, your customers, that is transaction creators, will just move to a larger service for their tx feed.


Yes, but DoSing nodes by launching DoS attacks is illegal. DoSing full-nodes by just making larger blocks isn't. For the largest miner/full-node service the cost of launching such an attack is zero, they've already paid for the extra hardware capacity, so why not use it to it's full advantage? So what if doing so causes %5 of the network to drop out.

The most dangerous part of this scenario is that you don't need miners to even act maliciously for it to happen.  The miner with the largest investment in fixed costs, network capacity and CPU power, has a profit motive to use that expensive capacity to the fullest extent possible. The fact that doing so happens to push the miner with the smallest investment in fixed costs off of the network, furthering the largest's profits due to mining, is inevitable. Furthermore the process is guaranteed to happen again, because the largest miner has no reason not to take those further mining profits and invest in yet more network capacity and CPU power.

Again, remember that those fixed costs do not make the network more secure. A 51% attacker doesn't care about valid transactions at all; they're trying to mine blocks that don't have the transactions that the main network does, so they don't need to spend any money on their network connection.

Every cent that miners spend on internet connections and fast computers because they need to process huge blocks is money that could have gone towards securing the network with hashing power, but didn't.

This.

Like it or not, we have to pay for network security somehow. People constantly complain about transactions fees but the real cost is inflation; $120k USD worth of it every day. Only because of the huge number of people adopting Bitcoin and investing in it have we been able to ignore the inflationary cost of mining.

You can troll all you want, but fundamentally the reference client runs great on fairly modest computers and because of the 1MiB block size Moore's law this will continue to be true. As I've posted elsewhere raising that limit is not an option. Right now I can run a Bitcoin node just fine on a $5/month VPN server with very little CPU power or memory. Even if your RFC specification somehow resulted in a Bitcoin node that used 10x less resources, frankly I don't care about the difference between $5/month and $0.5/month. On the other hand, I do care about network splits, and using the Satoshi codebase for full validating nodes will be the best way to prevent them for the foreseeable future.

Talking about "old money" and "new money" as somehow having anything to do with the issue of what codebase you should us to run a validating node is bizzare. The only financial interest the "old money" has is to keep Bitcoins valuable, and the best way to do that is to keep the network secure and well-used. Other than validating nodes there are lots of reasons to promote alternative implementations - I personally use Armory as a wallet and am working on a project that would create ledger services to process transactions entirely off the blockchain, while providing for a mechanism where these services could be both anonymous and trusted.

Of course, I can't think of any projects you've actually created, so I don't have any reason to think you've actually run into any the supposed serious limitations inherent in the satoshi implementation that only a complete re-write can solve.

This is a re-post of a message I sent to the bitcoin-dev mailing list. There has been a lot of talk lately about raising the block size limit, and I fear very few people understand the perverse incentives miners have with regard to blocks large enough that not all of the network can process them, in particular the way these incentives inevitably lead towards centralization. I wrote the below in terms of block size, but the idea applies equally to ideas like Gavin's maximum block validation time concept. Either way miners, especially the largest miners, make the most profit when the blocks they produce are large enough that less than 100%, but more than 50%, of the network can process them.

---

The idea that miners have a strong incentive to distribute blocks as widely and as quickly as possible is a serious misconception. The optimal situation for a miner is if they can guarantee their blocks would reach just over 50% of the overall hashing power, but no more. The reason is orphans.

Here's an example that makes this clear: suppose Alice, Bob, Charlie and David are the only Bitcoin miners, and each of them has exactly the same amount of hashing power. We will also assume that every block they mine is exactly the same size, 1MiB. However, Alice and Bob both have pretty fast internet connections, 2MiB/s and 1MiB/s respectively. Charlie isn't so lucky, he's on an average internet connection for the US, 0.25MiB/second. Finally David lives in country with a failing currency, and his local government is trying to ban Bitcoin, so he has to mine behind Tor and can only reliably transfer 50KiB/second.

Now the transactions themselves aren't a problem, 1MiB/10minutes is just 1.8KiB/second average. However, what happens when someone finds a block?

So Alice finds one, and with her 1MiB/second connection she simultaneously transfers her new found block to her three peers. She has enough bandwidth that she can do all three at once, so Bob has it in 1 second, Charlie 4 seconds, and finally David in 20 seconds. The thing is, David has effectively spent that 20 seconds doing nothing. Even if he found a new block in that time he wouldn't be able to upload it to his other peers fast enough to beat Alice's block. In addition, there was also a probabalistic time window before Alice found her block, where even if David found a block, he couldn't get it to the majority of hashing power fast enough to matter. Basically we can say David spent about 30 seconds doing nothing, and thus his effective hash power is now down by 5%


However, it gets worse. Lets say a rolling average mechanism to determine maximum block sizes has been implemented, and since demand is high enough that every block is at the maximum, the rolling average lets the blocks get bigger. Lets say we're now at 10MiB blocks. Average transaction volume is now 18KiB/second, so David just has 32KiB/second left, and a 1MiB block takes 5.3 minutes to download. Including the time window when David finds a new block but can't upload it he's down to doing useful mining a bit over 3 minutes/block on average.

Alice on the other hand now has 15% less competition, so she's actually clearly benefited from the fact that her blocks can't propegate quickly to 100% of the installed hashing power.


Now I know you are going to complain that this is BS because obviously we don't need to actually transmit the full block; everyone already has the transactions so you just need to transfer the tx hashes, roughly a 10x  reduction in bandwidth. But it doesn't change the fundamental principle: instead of David being pushed off-line at 10MiB blocks, he'll be pushed off-line at 100MiB blocks. Either way, the incentives are to create blocks so large that they only reliably propagate to a bit over 50% of the hashing power, *not* 100%

Of course, who's to say Alice and Bob are mining blocks full of transactions known to the network anyway? Right now the block reward is still high, and tx fees low. If there isn't actually 10MiB/second of transactions on the network it still makes sense for them to pad their blocks to that size anyway to force David out of the mining business. They would gain from the reduced hashing power, and get the tx fees he would have collected. Finally since there are now just three miners, for Alice and Bob whether or not their blocks ever get to Charlie is now totally irrelevant; they have every reason to make their blocks even
bigger.

Would this happen in the real world? With pools chances are people would quit mining solo or via P2Pool and switch to central pools. Then as the block sizes get large enough they would quit pools with higher stale rates in preference for pools with lower ones, and eventually the pools with lower stale rates would probably wind up clustering geographically so that the cost of the high-bandwidth internet connections between them would be cheaper. Already miners are very sensitive to orphan rates, and will switch pools because of small differences in that rate.

Ultimately the reality is miners have very, very perverse incentives when it comes to block size. If you assume malice, these perverse incentives lead to nasty outcomes, and even if you don't assume malice, for pool operators the natural effects of the cycle of slightly reduced profitability leading to less ability invest in and maintain fast network connections, leading to more orphans, less miners, and finally further reduced profitability due to higher overhead will inevitably lead to centralization of mining capacity.

Go right ahead; no-one is stopping you. Merchants especially should be using chain monitoring to determine when something may be going wrong and they may want to stop accepting orders. But don't use chain monitoring for miners, at least automatically: a false alarm of the monitoring mechanism can be used as a way to create a network split in itself.

You just don't get it do you? Bitcoin is a consensus system and we don't yet know how to create diverse implementations that meet the incredibly strict requirement of every implementation acting exactly the same way. For now, specifying the "Bitcoin standard" as source code is unfortunately the best we can do, and even that is difficult. This is why the recommended way to create a Bitcoin-using service is to run one or more full nodes using the reference (Satoshi) implementation to have some trusted nodes to connect to, and then use either the reference client or an alternative implementation as the base for your actual business logic. The reference node insulates your custom code from the network, especially malicious actors, and ensures that the rules for what are valid blocks and valid transactions are followed correctly. Your code can assume that anything the reference node communicates to it is accurate. (though you do need to handle re-orgs)

Ultimately that the GUI and the Bitcoin node were implemented in the same codebase was a big design mistake by Satoshi, but we have to live with it. Satoshi should have written a very small, very simple, validating node core and then wrote a separate client library and UI/wallet app as a separate code-base using that core. I think all the developers want to work towards that goal, but doing so is a huge project with a lot of risks. Not worth it given there aren't many disadvantages to just running bitcoind.

I think it's notably that you don't see the core devs complaining about Mike Hearns bitcoinj or jgarzik's pynode, neither of which claims to be a full validating node. The former especially has been used as the basis for a lot of services - satoshidice is built on bitcoinj IIRC. I have working on pynode on my todo list myself; we don't have enough people working on libraries to make it easy to explore new types of transactions. What we do have is people wasting a lot of time and effort attempting to make fully validating node re-implementations that are downright dangerous to their users and the network as a whole, yet don't provide any benefits.

You know gmaxwell has asked me a few times about my progress on pynode - not exactly the actions of someone trying to clamp down on alternatives. But he knows I understand the consensus problem.

In any case, for everyone who thinks an RFC or similar specification document is such a great idea, nobody is stopping you from writing one yourself. If you write a good one that manages to address the issues raised by Mike Hearn and gmaxwell, and keep it updated, it has a chance of being adopted and in any case the process will teach you a lot about how Bitcoin really works.

Proof of work via SHA256 hashing is really nice because you can validate it by machine. For instance a nifty project would be to get some remote attestation capable hardware like the IBM 4758 cryptographic coprocessor often used by banks. Basically what's special about it is the hardware itself is exceptionally difficult to tamper with, and additionally IBM includes a mechanism called remote attestation where the hardware will tell you what software is running on it. Since these co-processors are used for many, many different purposes IBM can't release hardware that lies without damaging a significant amount of trust in them.

So, what you would do is write a very small, very simple piece of code that implements the Bitcoin block hashing algorithm. What this code would do is accept encrypted messages from anyone, either the query "What's the legit block chain?" or the statement "Here is the next block in the chain" Since the messages are encrypted the operator of the service can't prevent someone from telling the hardware about the best known chain, so anyone making a query asking what the chain is can be pretty sure that the response is accurate. The existence of this service would allow others to use it to bootstrap their own clients without needing to know any honest nodes at all.(1) Smart Property is a good example where this service would be useful. Additionally it could argument or replace the checkpoint mechanism.

The problem with Ripple-style consensus is stuff like the above just can't be done because maintaining a list of public keys associated with trusted entities is fundamentally a task that only humans can do. For Ripple human consensus is probably a reasonable idea - Ripple depends on human evaluation of trust relationships anyway - but applying that concept to Bitcoin would turn it into something very different than it is now.

It also isn't a given that it would make Bitcoin any more secure either: if miners use this consensus scheme too, then by breaking the consensus you can either re-direct hashing power to your new, illegitimate chain, or failing that, turn the hashing power off to make a 51% attack easier. For non-miners consensus can help, but only in the sense that the consensus is warning you something is wrong, so you shouldn't trust transactions for now until we figure out what is wrong. Bitcoin already has a primitive version of that with the alert system anyway.


1) You do need a RNG to create nonces to prevent replay attacks. Also note that neither the client nor the server need a clock; if a 51% attacker does not exist, the length of the valid chain will always outpace any false chain. That said, at least having an uptime timer is useful so clients can determine if the server has been running long enough to have been told about the best block; similarly a message counter is also useful.

Multiple servers should exist, and clients should always query multiple servers and also automatically provide out-of-date servers with updates. You will have to be careful though to ensure queries for the chain and update queries are always exactly the same length. Additionally client behavior for either action needs to be identical to ensure updates can't be censored. (without breaking the encryption of course) The fact that the hardest PoW's in existence have about a 98% chance of being Bitcoin block hashes (2% chance of being orphans) could be useful.

Clients do need to have a way to prevent attackers with control of their upstream network connections from setting up these servers themselves. Simply having a list of trusted pubkeys of people who you trust to set such a server up with one good option; the hardware itself tends to be fairly expensive too. You can also get clever and have the server create a PoW based on their identity; a Bitcoin PoW for an invalid block is nice because determining the value expended is relatively easy. You can also have the clients pay for each message by performing the PoW on your behalf, and recording the hardest PoW found to in turn use as your proof that the server is legit.