also correct me if im wrong but individual miners wouldnt even need a 100mb connection would they? just the pools.
Correct. In this context only the entity actually building and broadcasting a block are "miners". So yes the pool server, solo miners (ASICMiner), and setups like p2pool. Basically if you are broadcasting the block yourself via bitcoind then you are a "miner". IMHO "pool workers" aren't really miners and calling them that is inaccurate. They are just computing power providers (CPPs).
I actually coined that term in a request to FinCEN for an administrative ruling to highlight the distinction between the entity creating new blocks/coins and entities merely providing the resources. We wouldn't call the power company or ISPs "miners" although electrical power and connectivity are required inputs to creating new coins/blocks.
so even with an unlimited block size there would still be a market for transaction inclusion in blocks since miners who attempted to relay a block that was too large would find it orphaned. that's important for network security.
Yes however the risk is centralization. It isn't that the network "couldn't" handle unlimited blocks it is that we might not like the consequence of unlimited blocks. As the block sizes gets larger and larger it becomes more difficult to keep orphans to a manageable level. Orphans directly affect profitability and for pools it is a double hit. High orphans mean less gross revenue per unit of hashing power but it also means the pool is less competitive so miners move to another pool. The pools revenue per unit of hashing power is reduced and the overall hashing power is reduced too. So pools have a very large incentive to manage orphans.
Now it is important to remember it doesn't matter how long it takes for ALL nodes to get your block just how long it takes for a majority of miners to get your block. The average connect between major miners is what matters.
If the average connection can handle the average block then it is a non-issue. However imagine if it can't, and orphan rates go up across the board. Pools are incentive to reduce orphans so imagine if x pools/major solo miners (enough to make up say 60% of total hashing power) moved all their servers to the same datacenter (or for redundancy the same mirrored sets of datacenters around the world). Those pools would have essentially unlimited free bandwidth at line speed (i.e. 1Gbps for cheap and 10Gbps for reaosnable cost). The communication between pools wouldn't be over the open (slow) internet but near instaneous on a private network with boatloads of excessive bandwidth. This is very simple to accomplish if the miners are in the same datacenter. The miners just share one LAN connection on a private switch to communicate directly. Now for these pools a 40MB, 400MB, or even 4000MB block is a non issue. They can relay it to each other, verify, and start on the next block in a fraction of a second. Near 0% orphan rates and reduced bandwidth costs. For other miners however the burden of these large blocks means very high oprhan rates. How long do you think it will take before CPPs abandon their pool with 5% orphan rates for ones with near zero. That isn't good for decentralization of the network.
I don't want to sound doomsday but this why BANDWIDTH (not stupid disk space) is the critical resource and one which requires some careful thought when raising the block limit. It is important that average block size not exceed what a miner on an average connection can broadcast to peers in a reasonable amount of time (3 seconds = ~0.5% orphan rate) on an average public internet connection. Granted those are pretty vague terms. Obviously 1MB block is below that critical level and 1GB block is obviously above it. Is 100MB fine today? How about in 10 years? Is 5MB fine today? How about 2MB and doubling every two years?
I am not saying I have the answers but that is the kind of thing we (community at large not just Gavin et all) need to think about critically before saying "yeah lets go to unlimited blocks and let the market figure it out". I have no doubt the market will figure it out however one might not like what end state it reaches.The good news is bandwidth is still increasing rapidly and the cost per unit of data is falling just as rapidly. This is true both at the last mile (residential connection) and in datacenters. So it is an problem which is manageable as long as average block size doesn't eclipse that growth.
100 tps is way plenty, even if we assume the load of all credit card companies combined 100 tps would be enough to allow anyone who wanted to do an on-chain transaction to be able to afford it (excluding micro transactions but who cares about that). which is all that matters, we dont need a system where every transaction avoids all counter party risk, what we need is a system where avoiding counter party risk is affordable. 100tps would provide that. this post put my mind at ease. i mean im already pretty significantly invested in bitcoin because even if there was no solution the the scalability problem bitcoin would still have great utility, its nice to know however that there are solutions.
Don't take any of this as "set in stone" it is more like when they ask you "how many windows are there in New York City?" in an interview. Nobody cares what the exact number, what the interviewer is looking for is what logic will you use to come up with an answer. If someone thinks my logic is flawed (and it certainly might be) well that is fine and I would love to hear it. If someone can convince me otherwise that is even better. However show me some contrary logic. If the counterargument is merely "unlimited or it is censorship" well that doesn't really get us anywhere.
There are four different bandwidth bottlenecks.
MinersMiners are somewhat unique in that they have to broadcast a block very quickly (say 3 second or less target) to avoid excessive orphans and the loss of revenue that comes with it. This means their bandwidth requirements are "peaky". That can be smoothed out somewhat with protocol optimizations however I would expect to see miners run into a problem first. The good news is it is not that difficult for pools or even solo-miners to setup their bitcoind node in a datacenter where bandwidth is more available and at lower cost.
Non mining full nodesFull nodes don't need to receive blocks within seconds. The positive is that as long as they receive them in a reasonable amount of time they can function. The negative is that unless we start moving to split wallets these nodes are likely on residential connections which have limited upstream bandwidth. A split wallet is where you have a private bitcoind running in a datacenter and your local wallet has no knowledge of the bitcoind network, it just communicates securely with your private bitcoind. An example of this today would be electrum client connecting to your own private secure electrum server.
Bootstrapping nodesAnother issue to consider is that if full nodes are close to peak utilization you can't bootstrap new nodes. Imagine a user has a 10 Mbps connection but the transaction volume is 9 Mbps. The blockchain is growing at 9Mbps per second so the user is only "gaining" on the end of chain at 1 Mbps. If the blockchain is say 30 GB it will take not ~7 hours (10Mbps) but 70 hours to catch up.
The good news here is there is some slack because most residential connections have more downstream bandwidth then upstream bandwidth and for synced nodes the upstream bandwidth is the critical resource.
SPV nodesThe bandwidth requirements for SPV nodes are negligible and unlikely to be an issue which is a good thing however SPV don't provide for network security. While SPV are important so casual users are not hit with the rising costs of running a full node at the same time we want to ensure that the ability to run a full node for enthusiasts remains an
realistic option. Maybe not everyone can run a full node but it shouldn't be out the reach of a the majority of potential users (i.e. requires 200Mbps symetric low latency connectivity, 20TB of storage, enterprise grade RAID controller, 64GB or RAM, and quad xeon processors). How many full nodes are needed. Well more is always better, there is no scenario where more hurts us in any way so it is more likely how few can we "get away with" and stay above that number. If 100 enough? Probably not. 1,000? 10,000? 100,000? 10% of users, 1% of users? I don't have the answer it is just something to think about. Higher requirements for full nodes means less full nodes but more on chain trust-less transaction volume. It is a tradeoff, a compromise and there is no perfect answer. 1GB blocks are non-optimal in that they favor volume over decentralization too much. Staying at 1MB blocks forever is non-optimal in that it favors decentralization over volume too much. The optimal point is probably somewhere in the middle and the middle will move with technology. We don't need to get the exact optimal point, there likely is a large range which "works" the goal should be to keep it down the middle of the lane.
