Bitcoin Forum

I know I have been out of the loop for a while, but I've started to hear about this Bitcoin versus Bitcoin XT "code fork" from semi-mainstream news sources.  I sounds like FUD, so I have checked out the Bitcoin XT website and searched a bit around this forum for details.  One thing that I don't understand is why does Bitcoin XT have the other features, such as auto-dropping of TOR nodes?  And can the Bitcoin XT node be forced into the 'quiet' node format?  From what I have found out, it looks like it leaks the node's IP address while it's supposed to be hidden.

As for the big block mod.  I guess I just don't see the value in it. jumping to 8megs on start, then auto-adjusting for transaction volumes would undermine the transaction fee market that is supposed to develop to priortize transactions.  Also, where are these overlay networks that were supposed to absorb the majority of the lower security transaction volume?  This bitcoin network proper was never supposed to be how Joe Average transacted in the currency, Joe should be using a light client on an overlay network; like Electrum was developing or M-Pesa style light client networks on smartphones.  Why are we still doing the majority of retail business using full clients anyway?

Someone with more intimate knowledge is welcome to correct me on the technical details, but please don't come to me with complaints about how the fee market would never have worked or that Bitcoin proper actually needs to scale to Visa levels; I've been here too long for those arguments to still have any merit.

Right, well you've amply described why those who consider themselves somewhat aware of the technical aspects underlying this debate dislike XT. I agree with all of that, so your post title is inaccurate, I think you do understand it. I'd just point out in addition that there are 2 other categories (both political, but relevant to development) of contention also; the necessary change in management that an XT fork would involve, and the perhaps inevitable change in development direction.


I had always assumed the "IP blacklisting" stuff was pure made up counter-offensive FUD, but what you're saying implies it has some basis in fact. The interpretation of the new behaviour in respect of Tor nodes is perhaps exaggerated; "deprioritisation" is the description given for what happens to the condition of a Tor node's connection to any given XT node, although they are awarded the harshest score within the de-priority range. How this manifests in practice, I am not yet sure.

What kind of "change in management" are you referring to here?  The different set of lead developers, perhaps? Or something else?

I'm more than a bit concerned that the big block mod, generally, will force marginal mining nodes off the network.  I know that it would prevent the broadcasting of whole blocks, and likely even naked merkle tree data, over datacasting paths such as Outernet.  If we are going to make changes for scalability, why are we not making naked blocks the standard broadcasting method first?  That alone would reduce redundancy.

That, yes.


Those concerns will end up realised no matter which way the dev team go, max blocksize is increasing above 1MB no matter what. The usage examples you gave will need to take a new approach.



Another less cited new feature of the XT client is a change to chain selection/consensus rules. XT clients don't follow the longest chain, they follow the chain with the highest XT checkpoint embedded into it.

- The code that put Tor exit ips on low priority activate it self only during DoS connections attack on the node it self
- The code is disabled if the node is set to use a proxy
- The code is disabled if the node runs with this flag -disableipprio

Miners can't make block of bigger size because they will get them orphaned.

https://tradeblock.com/blog/bitcoin-network-capacity-analysis-part-6-data-propagation
So if blocks are bigger than the the average connection of the nodes, they will be orphans.
This is the natural size limit, that if it is reached, will open the possibility for a market of fee.


An XT FAQ <- But I suggest you to read this.
https://medium.com/@octskyward/an-xt-faq-38e78aa32ff0

My tuppence!

It is a political battle wrapped in technical FUD.

What is basically being proposed is a rather gentlemanly 51% attack on the current blockchain by a splinter group of devs. Gentlemanly because they are stating when they will do it and given a date by which they will complete the takeover. They hope to persuade enough existing hashpower to force a fork onto their chain so they they can move control of the network onto their software fork. The capabilities of that client is a side-show; the issue here is that the attack is possible without any outlay of resources. If you want to see how a government would do it in the future; this is how. Not by buying a shedload of hardware to out-hash the network, but persuade or force existing miners to switch.

The blocksize is just the crowbar for leverage. The blocksize itself was a kudge to fix what they term spam - micro payments and dust transactions - because the centralisation of mining meant that miners couldn't keep up any more and transaction times were ballooning. It was taking the miners longer to get their pound of financial flesh. It was a quick-fix solution that has now become a permanent feature of the protocol and means whoever has the lowest block size loses as the higher limit is the super-set of acceptance.

 If the network could handle the transactions then the block size would be irrelevant, there would be no limit and there would not be the leverage to force the fork as any limit would be the loser.

I've been doing some research into the Lightning network proposal, and I'd say that I'd be more in favor of fixing malliability in order to permit such a network to develop, over the XT proposal of larger blocks and better support for light clients; simply because larger blocks don't really solve the scalability problem.  It has long been the idea that Bitcoin's main network would have to be a backbone mostly between major institutions that performed 99.9% of transaction volume.  Centralization really isn't a huge issue, so long as it's always possible for a new player to join the main bitcoin network as an equal peer with the rest.  XT seems to make such an idea more costly, without actually solving the scalability issue in the long run.  We will still need overlay networks with XT.  I remember the payment channels thing, and remember discussing it years ago, in the context of bitcoin banks settling up with each other by being able to identify the customers of another bank by their bitcoin addresses instead of personal data.  We still need these bitcoin banks.

I can join the current banking system as an equal peer and it already has the transaction volume, sales acceptance, speed of verification and so on. Is it probable I would succeed? Not unless I was in the club but it is possible.

If "it has long been the idea that Bitcoin's main network would have to be a backbone mostly between major institutions", then Bitcoin failed long ago when that idea was accepted and it is being falsely marketed. This sounds a lot like what a financial institution would want of Bitcoin so they could carry on their hegemony.

And no one is arguing this, we need both without exceptions or virtual limits.

You can not join the banking system as a peer, only as a customer.


Decentralization is a relative term, and Bitcoin was not developed with that as a goal.  Decentralization was a tool toward Satoshi's ends.  What makes Bitcoin work is that it is truly peer to peer, if that is what users want to be able to do.  But that's not going to work with a flexible blocksize.  Moving to 8 megs isn't a big deal, but nor does it really change the scalilibilty issue much.  I'm not even opposed to soft limits, so long as Satohsi's original idea of rising transaction fees for larger blocks is preserved; but some kind of transaction fee market needs to exist.  If we screw this up, and the hashrate collapses, it would be catastrophic.  I'm not really opposed to how the XT team is actually using a hard fork as a voting method.  It's actually a good plan, it would appear.  I came here hoping I could get forum members who were better versed in XT than myself to explain the details, but instead I get noise.

But we still need a working transaction market in the long term. How is this intention maintained in XT?

On the front page of BitcoinXT's website:


There is no such intention to maintain a fee market in XT. Hearn wants to replace miner subsidy with some sort of mining insurance contract (that I have not looked at). Maybe someone familiar with the proposal can pitch in.

What is clear is that XT supporters believe any transaction is worthy of the blockchain and that fee competition is detrimental to the network. They only see fees as serving the purpose of mining subsidy so it is expected their proposals to replace the fee market will only cover this aspect, as the filtering and competition in the transaction market are concepts they are fundamentally opposed to.

Simply put, they don't want a transaction market, they only want the validation layer.


The set of lead developers and their leadership philosophies. The idea is not to only change leaders but to change direction altogether. Hearn puts it quite clearly in his XT FAQ:

There will be a market of fee, if there is the situation where it is needed:

Nodes cannot orphan blocks, only other miners can. The average node connection is therefor irrelevant.

Large miners do not stand at a significant risk to be orphaned by smaller miners, only the other way around is true. This creates a situation where a few large pools can agree not to orphan each other (and there already are examples of large pools signing on mutual agreements) and they can easily vampirize the smaller pools market share.

The reality is that without limits on the current system, connectivity will become a barrier to entry to the mining market. By nature, barriers to entry are agents of centralization. The stance of the Core team and supporting members of the technical community is that the network needs first be more efficient before it is scaled up. Otherwise, effectively removing the block size limit is akin to amplifying an analog stream with a poor SNR. The only thing you will achieve is to drown the valid signal in noise.

This argument has been laid out several times by people much more versed on the networking layer of Bitcoin than I am. If you want to engage in a discussion with me on this matter, I would appreciate that you do not ignore this criticism to large blocks. Currently, your insistence on the idea that miners will limit themselves because of orphans instead of using it as an edge to predate on their competitors indicates otherwise.

I would also like to remind you that this isn't the topic of this thread. We are here to discuss the underlying fundamentals of XT, essentially the "why XT", not so much the "how". Gavin made it clear in XT, the end all and be all of the "how" is Moore's law.

So now, to get back on topic, your answer does not refute my presentation of XT's stance on the fee market:


Indeed, say I am wrong and miners will softcap block size to limit orphans. In your best scenario, in which your analyze is right, you perceive the fee market as nothing more than the consequence of a technical limitation, not a feature. The fact that there may or may not be a fee market in XT will not be by design but an unwanted consequence of hardware limitation. Therefor, it wouldn't surprise me to see solutions implemented in XT to entirely get rid of the fee market, like implementing LN, which you are proposing.

This is such an important point that rarely gets voiced. Free market evangelists look at Bitcoin and see it as a technology to serve the current status quo if only it had more "market forces" and allowed more "opportunities for profit". The technical trade-offs that are exploitable are seen as designs that should be expanded to make way for more of the same rather than weaknesses in the protocol that should be plugged.

Allow me to help you and others understand: that sentence you are quoting was spoken from the perspective of the XT design ideology, not as a representation of the views of the person that wrote it. The person you quoted does not believe the view you are attributing to them.

I'm sure you wouldn't want to look like you are misrespresenting the situation, so I thought I would inform the thread (careful, someone might accuse you of being yet another dishonest debater, and you don't want that!)

"Ground control to Troll Station-1. We have found your escapee."

I guess I'll repeat what Carlton said:

I was trying to present the XT point of view on the fee market, to which I deduce you adhere (if I did a good job at describing it). I was not expressing my point of view.

This can only be done by the equivalent of a 51% cartel not accepting blocks from the rest of the network, otherwise the lower their % is the higher the risk of losing blocks ... which miners don't like doing ...
Also, the "size" of the miner isn't all that relevant to % of orphan blocks - the only obvious reduction there is when they confirm their own blocks.
... and you don't "agree not to orphan" ... either your wording is incorrect or your understanding is incorrect.

Any pool can limit the size of the blocks they are generating and thus reduce their orphans - but at the current limit of 1MB that's marginally relevant at best for any well configured miner/pool.
It will of course mean they are getting less txn fees in their blocks ... so once the block reward gets low enough, that will of course be an issue.
At the moment txn fees are averaging around 1% to 0.5% extra reward.

Right, Moore's law has almost nothing to do with the profit from mining - not sure why Gavin can't see that ... other than the fact that he cares close to zero about mining ...
Electricity costs are the biggest factor for most - and finding a source of cheap power.

Miners/Pools already softcap block size limits.
They also SPV (and supposedly non-SPV on Eligius) mine empty blocks on certain pools ...

seeing as the subject of your mistake agrees with my assessment (that you made a mistake), it appears that your accusation is without merit.

Be careful throwing such accusations around: if you're wrong, it will damage your own reputation, not that of your target.

If nodes relay block solutions between miners, then yes their connection matters.  If most miners use the Corallo Relay Network, I agree that average node connection becomes less relevant.  Nonetheless, the orphan cost still exists (it is just smaller), which creates a fee market even in the absence of a block size limit (https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf).  


What I have highlighted in bold doesn't make sense to me.  How can they agree not to orphan?  What if the block contained an invalid transaction?

Please let me explain three points on the idea that orphan risk could create a marginal cost of adding transactions, which could finance mining:

• Some have argued (Gregory Maxwell¹) that if the only significant marginal costs of including a transaction is the orphan risk, then equilibrium difficulty will be too low. The reason is because average transaction fees will be equal to the marginal orphan risk increase and therefore there is no additional revenue to subsidies the cost of hashing. Therefore the equilibrium difficulty will be too low and we have a problem. I am not sure if this is correct because the marginal cost of orphan risk will be unique to each miner, therefore one could draw an industry marginal orphan risk cost curve and some miners would make more profits than others, this dynamic could be sufficient to ensure high enough equilibrium difficulty. Although it implies better connected miners are more profitable and therefore this may increase centralization. This does make the orphan risk idea idea superior to the ideas about arbitrarily fining miners a uniform amount for larger blocks, because this cost is the same for all miners, there will be no cost curve and therefore no cross subsidy for hashing, which will result in equilibrium difficulty being too low.
• Gavin and others have argued that computer science has come a long way in the last 20 years. For example bandwidth speeds and increased and storage costs have come down. This is often cited as a reason supporting BIP101, because transferring 8GB is expected to be much cheaper in 20 years time. I broadly agree with this line of thought. However, ironically this makes me more concerned about BIP101, because these technological improvements (and IBLT/relay network) may mean the marginal cost of orphan risk, when including an additional transaction into a block, will get closer to zero over time or at least become insignificant. Therefore I consider this point a large negative when evaluating the viability of a fee market without a blocksize limit.
• I am also concerned about using orphan risk as the mechanism for ensuring a fee market. Operating this way may imply the orphan risk will constantly be pushed to the limit. This is not good for the health of the network and reduces the ability of the network to effectively and decisively come to consensus about the state of the blockchain. I also consider this point a negative when evaluating the viability of a fee market without a blocksize limit.

1 - The fact that verifying and transmitting transactions has a cost isn't enough, because all the funds go to pay that cost and none to the POW "artificial" cost. - http://sourceforge.net/p/bitcoin/mailman/message/34090559/

No. Anything with "market" in it I don't adhere to. A "fee market" isn't a technical point of view, rather one for game theorists and free marketeers who I would prefer to see in prisons around the world with the misery they have caused. You did do a very good job of describing an aspect that I wanted to expand on to all of bitcoin and her offspring rather than an individual client-so I quoted you. I didn't say it was your point of view, I said it was a point that should be voiced more often.

The fear on these forums is quite a sight to see when there is any slight possibility of being misinterpreted. The frantic back peddling and heavyweight interventions when something may be construed to the detriment of a particular narrative is startling. I find it all very unhelpful but oddly amusing. This is the FUD the OP talks about, which is leaking into his thread now too. I'm just a little ashamed to be contributing to it by defending myself, but hopefully this is the end of it after one last salutation to Carlton Banks.

You need to drop it and add to the discussion not continue to try and lecture me with condescension on what I should or shouldn't do to increase my reputation. Another tart, supercilious reply from you will confirm the trolling, so move along now, there's a good chap.

bolded is hilariously hypocritical


What I am adding to this discussion, as you well know judging by your irritated tone, is weeding out the dishonestly presented arguments. I am successful at doing so, your arguments are no exception.

I think the average (and also genuine) bitcointalker can see plainly how you're behaving. Welcome to your new reputation.

As far as I'm aware, no one is proposing removing the block size limit. While one exists, be it 1MB or 8MB, there is an opportunity to financially exploit rarity in block space. Are you advocating that a block limit is a necessity to enable a fee market and that a fee market is a desirable feature of the network now it has arisen?

I never argued that.  I said that the orphan cost creates a fee market in the absence of a block size limit.  


As far as I know, no one has rigorously shown this to be true. The last paragraph of the fee market paper (https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf) speaks to this:

"We conclude by noting that the analysis presented in this paper breaks down when the block reward falls to zero. It suggests that the cost of block space is zero; however, this would suggest zero hash power, which in turn would suggest that transactions would never be mined and, paradoxically, that no block space would be produced. Happily, questions about the post-block reward future can be explored at a leisurely pace, as we have a quarter-century before it begins to become a reality. Into the distant future then, a healthy transaction fee market is expected to exist without a block size limit."

If a node is presented 2 blocks for the same height, it will always prefer the one it received first. It will not relay the second one even though it is as valid until the next block orphans one of them. Therefor nodes are poor relay points for miners and any miner not trying to connect directly to the majority of his competitors is doing it wrong. So I'll insist, nodes aren't all that relevant in block propagation between miners.


The bold part is the issue. A weak fee market is the same as no fee market. It can't sustain mining and does not filter transactions enough. The absence of limit (or an unrealistically large limit for that matter) puts it all on Moore's law.

Orphan cost can essentially be equated to friction. For that friction to be significant, the adoption rate will have to at least constantly match Moore's law. Any engineering breakthrough or any better software propagation scheme will gravely undermine the network. Also, adoption is finite. Moore's law may or may not be infinite, that's up for debate, but certainly has more growth potential than Bitcoin adoption does.

Again, the existence of a fee market is pointless if it isn't healthy.

The cost of orphan is about as bad a metric as electricity to define mining profitability. As the cost of electricity, connectivity is subject to heavy government intervention (just look at China). Let's not reinforce the reliance of the network on yet another manipulated metric.

Lastly, technological improvement happens in thresholds, it doesn't propagate evenly or at a continuous pace. One day you have 20Mb DSL, the next day you have 250Mb fiber. One day you have 3G, the next you have LTE. Adoption on the other hand is a lot more continuous. What do you expect that will do to the fee market if suddenly the cost of orphans is so low that every miner can afford to deplete the mempool on every block for even the next 6 months? Year? 2 years?

But let's admit adoption isn't continuous, let's admit it is a bumpy as technological leaps. What happens if adoption booms and technological leaps do not occur concurrently?

The cost of orphans is not a good metric to establish a fee market because it is very inconsistent. It is poorly distributed and rather unpredictable. It acts as an extra barrier to entry into the mining market, which is meant to be defined by hash rate. Why do you think Chinese miner blind mine?

Also Moore's law doesn't regress. How do accommodate the fee market in a recession or a long term stagnation on hardware friction alone?

We need another metric to underline the fee market precisely because cost of orphans is a bad one. We shouldn't look at cost of orphans as a feature, we should try to reduce it as much as possible.


And this is what we are getting to. No miner has cause to stay out of the relay network, and XT can't avoid this network either, because it reduces the cost of orphans. Any miner that doesn't use it is at a disadvantage. However it will finish off the fee market in XT before Moore's law even gets to kick in.



Imagine Corallo's relay network doesn't exist. Imagine a couple large miners set up private version of that network for their own sake. Suddenly their cost of orphan has globally reduced.

I will not explain why this is bad for the network and why it will give an edge to these collaborating miners. I invite you to reread the previous paragraph if you want more details, it is pretty much all there.

Bottom line, either XT does not implement the relay network and a couple miners will use that edge to wear and tear their competition, or XT will implement it and friction will be so insignificant compared to demand that fees won't be able to sustain decent hash power.

TL;DR what jonny1000 said.


Not really. I am not talking about rejecting blocks outright, only about how the network tells 2 blocks apart in the case of a simultaneous solution.

Assume miner A has 10% hash rate and miner B has 30% hash rate. Assume both A and B find a solution for a block at about the same time. Let's say miner C receives these 2 blocks within a few seconds of each other. Which block do you think miner C should work on? A's or B's? Obviously B's.

Bottom line is, in the case of a propagation race, you don't need 51% hash power. You only need more hash power than your competitor and the rest of the network will prefer your block to his.


How can you conclude this from the previous paragraph. If anything it means there needs to be a primary factor in determining fees that is not the cost of orphans. If you don't want a block size limit, then your prime candidate is enforcing minimum fees, and that's a whole new can of worms.

And it would not suggest zero hash power. It would suggest blocks will be mined by those who emit the transactions directly. There is always an incentive to mine blocks, with or without a coinbase reward. The reality however is that if that incentive is proportionally too low compared to the network's market capitalization, it will lack in security and make malevolent mining profitable.

It is very curious how people like Peter R and friends do this.

Peter R: "proposition A supports conclusion A, and therefore falsifies proposition B. Therefore, the correct conclusion is B!"

Non-sequiturs presented as rational argument, and the expectation is that someone thinking rationally might agree.

What I quoted above is the last paragraph from the paper titled "A Transaction Fee Market Exists Without a Block Size Limit."  The final sentence you guys are arguing about concludes a 16-page research paper--not just the quoted paragraph.  

Take a look at the full paper if you like: link (https://dl.dropboxusercontent.com/u/43331625/feemarket.pdf)

I've skimmed it already several times, but since you insist on the validity of your claims I will give it a thorough read.

Regardless, something can be said about integrating this line to the quote if it relies on a lot more context (16 pages apparently) than the conclusion you emit right before it.

As I said, you would need a 51% cartel to do that - your competitors are the rest of the entire network on the earlier block.

If current best block is height A, and some miner/pool transmits block height A+1, everyone on the network will switch to it since it makes no financial sense to stay mining on block height A without 51%+

Along comes another block height A+1 (1ms or 15minutes later)
Unless you have 51%+ switch to the replacement block height A+1, you will be mining on the least mined block height A+1 and thus average losing out because of that.
i.e. without 51%+ doing the same, you average losing out by switching blocks at the same height.

No you don't. If 2 solutions propagate simultaneously, as a miner you should choose the solution emitted by the larger miner. I'll make the example more extreme but it stands at any value really:

Miner M has 30% of the hash rate, Miner m has 1%.

Assuming the network propagation speed is equivalent at all points of the network, M will always get 51% hash rate support behind its solution faster than m will, for M only needs to propagate to an arbitrary group of miners controlling together 21%+ of the hash rate, whereas m needs to propagate to 50%+. The time to propagate to 51% of the total network hash rate is quantifiable, and in any scenario where propagation speed is non negligible, the difference in time to recruit 51% of the network between a large and a small miner will also be non negligible.

Now let's say m's average block propagation time to 51% of the network hash rate is t, and M's is T. We know t > T.

Now say miner N receives m's block first, then M's block next, for the same height. We have 3 situations (all timers starting when N receives m's block):

1) N receives M's block within T. N should switch to M's block simply because t > T. At this point there is a higher probability M will recruit 51% of the hash rate faster than m would.
2) N receives M's block after T but within t. N should switch to M's block, because there is a high probability M has already recruited 51% of the hash rate.
3) N receives M's block after t. N should stick to m, as there is a higher probability m has already recruited 51% of the hash rate.

The reality is a bit different however. Propagation speed isn't balanced across the network and while T and t are quantifiable, there is a lot of data to gather to even get valid estimates. What really matters is that for m < M you should always assume t >T, and that is enough information for any sane miner to switch to the M's block if it is received within a few seconds of m.

In a propagation race, the larger miner will always have an advantage, and it doesn't need a 51% cartel, it only needs to get a portion of the network behind its solution that is proportionally smaller to difference in hash rate with its competitor.

TL;DR: You don't need to control 51% of the hash rate, you only need to recruit 51% of the hash rate faster than your competitor. If you are larger than your competitor, you are essentially guaranteed to always recruit faster.

Obviously this all goes down the toilet once miners start using fast relay networks. But orphan cost will get flushed out along, so the assumption that a fee market can exists when propagation time is negligible, let alone that this fee market would be healthy, is completely undermined.

Ok, my point is the fee market it may create could be insufficient and inappropriate to support mining.  Actually I don't see how this kind of fee market helps with any of the potential "reasons" (https://bitcointalk.org/index.php?topic=669243.0;imode).


Yes, I agree with you here, as I said in my comment, I don't think Greg has shown enough evidence to support this point yet:

This is a great point goatpig.

The problem with relying on orphan risks are the following:

• A - The cost of orphan risks will very low, such that they are insignificant
• B - Orphan risk costs will be significant, therefore larger better connected miners have an advantage

Either A or B is true, orphan risks are significant or they are insignificant.  

There can be no balance between these two options.  We are potentially talking about 100% of mining industry revenue being financed by orphan risk, when the block reward becomes low.  It is also necessary for some miners to have different orphan risk costs.  If orphan risk is uniform across the mining industry then Greg's comment¹, becomes true.  I have come to this conclusion based on my experience as an analyst in the mining resources space.  Therefore as costs vary by miner, miners will benefit from having better propagation, which must be significant in relation to revenue, therefore the industry centralizes.

In conclusion, financing the mining industry based on orphan risk is not possible or desirable.

1 - The fact that verifying and transmitting transactions has a cost isn't enough, because all the funds go to pay that cost and none to the POW "artificial" cost. - http://sourceforge.net/p/bitcoin/mailman/message/34090559/)

You never receive 2 at the same time.
It will 100% always be at different times, one after the other.

OK a pool has X%, then they only need 51% - X% actively switching to their block instead continuing on a current block at the same height to help them win an orphan race they would expect, on average, to lose.
That's still a 51% cartel.
However you word that, you need collusion to get other pools to consider doing that.

Blocks are rarely if ever very close together.
I don't know when I last saw 2 less than 2 seconds apart in my logs ... so it's so rare that it's basically next to irrelevant.

My logs are millisecond accurate, I've changed cgminer and ckpool is the master gits to be ms accurate, all my logging scripts are ms accurate and I modify the relay to be ms accurate ... though the current commit in the relay that shows the block time with 1 second accuracy - I created the pull
The main reason for saying this is that I've done monitoring of all sorts of stuff on the network against my pool (that I know most pools haven't even looked at ...)

The reality is that if you start mining on a later, same height block, you will be mining on the losing block most of the time you do that.
Yes there may be some rare cases where that isn't the case, but a very high % of the time you will be mining on the losing fork.
BUT ...
Now does that matter which fork you mine on? No. Since all that matters is who finds the next block to decide the fork.
Bitcoin core's rule is ludicrously simple and works.
If you are building on a block at height X, and another valid block appears at height greater than X, start building on the new block. Anything else, ignore it.
It doesn't matter which (valid) fork that new block is on, just switch to it.
So yes you could choose to help some other pool confirm it's 1s, 10s, 100s late block, if you choose to, but you wont gain anything on the bitcoin network by doing that ... without a cartel setup where they say they will do that for you also.

See what matters is getting your own block confirmed, not some other block.
So if you send out a block to the network, ALL you are hoping is that the majority of the network is building on it.
Except in the rare case when some other pools sends out a block around the same time, you don't have to worry.
If on the other hand, some other pool did send out a block about the same time, you CERTAINLY do not want to switch away from building on your block.

As for block propagation and orphan rates ... well ... my little less than 1% pool has pretty good orphan rates ...
Who knows if that is due to a small sample (2 out of 317 blocks in 11 months) ... ... or what it is :)

The business intelligence value of knowing who made what transactions alone is easily enough to pay for operating the network.  Especially when one has access to 'big data' processing facilities and a big network footprint already.  Much higher value than knowing something about people's search terms or scanning their e-mail and cloud storage contents.  I would expect to see users (to use a more kind descriptor) get 'cash back' when things are ramped up.

I have little doubt that Mike is keenly aware of this principle, and I suppose that he didn't really want to use it as a sales pitch so he cobbled together some fairly questionable 'mining assurance contract' scheme or whatever it was.

Ah, I see.  Not a student of economics.  You just completely lost me with this one, and since I don't have the time to consider every point of view, you also just earned a spot in my ignore list.

This is not how Bitcoin propogation is supposed to work.  If this is part of the XT proposal, it would destroy bitcoin, by centralizing more than can be mitigated with a fee market anyway.  Such a change would render nodes no longer peers, as larger pools would be superior.  This would break the 51% rule.

That ignores co-location propagation advantages though. I would suggest that the miner should choose a solution emitted by one of the other miners in their own group to accelerate propagation even if they are the smaller miner.

So for all these years that Bitcoin's been trucking along working beautifully and gaining more and more mining power while never coming close to the cap, suddenly now when it threatens that system and prevent users from being able to make transactions, suddenly now you think it's a good idea to let that happen, even though Bitcoin's been working great without an effective cap for years?

Bitcoin's usefulness relies on people USING it.  You think clogging it up so that it becomes more expensive and less useful to users is the best long term approach to scaling and/or maintaining Bitcoin's value?

Jesus Fucking Christ, you guys are either stupid, insane, or short-term financially motivated.  There's just no nicer way to say it.

Large pools are made up of individual miners.  If a pool does something people generally don't like, miners will stop mining on that pool.

It's extremely frustrating, even if somewhat amusing, when economists continually fail to take real world data into account for their on-paper-it-sounds-good theories.

You affirm this because you are not considering the compound effect of the significant network "friction" the example assumes, as well the new incentive the orphan race creates for M and m.

Let's go back to the example in my previous post: M as the large miner, m as the small miner, N as the neutral miner, and significant propagation time. What you are stating is that in the case of a race between M and m, it doesn't matter to N which block it mines off of, it only matters that both blocks extend the last known top. After all, you are only "wasting" hash rate if you mine on top of a block below the max height.

Or are you? Clearly there is no incentive to ignore a valid solution if the only thing you are going after is the coinbase reward + average fees. But what if there was an incentive? Say someone emits a ZC paying 1000 BTC to address A with a small fee, and as soon as this address is mined, emits a transaction B that spends the same TxOuts to address B, but this time paying a 50BTC fee. Wouldn't it be in every miner's benefit (but the one which mined the block with A) to orphan that last block?

Not saying this what's happening in the orphan race per se, or that it is good practice to accept a 1000 BTC payment after a single confirmation, but what stands is that there can be incentives to actively orphan a block. And in the case of an orphan race, it is a coinbase reward + fees riding in the balance.

The proposition goes as follow: in the case of a race between M and m, M is expected to recruit over 51% of network the hash rate behind its block simply because M > m. Now we have 2 miner groups, where G mines off of M's block and g mine's of m's block. There are 2 outcomes:

1) G finds a solution first for height H+1, validating M's block at height H. At this point m has lost the race and its block reward, but this result is irrelevant to others miners in g.
2) g finds a solution first for height H+1. However, M knows it can propagate its solutions faster than m, so there is a window in which M can actively try to orphan g's block and still propagate its solution for H+1 to 51% of the network faster than the g miner can. M has an incentive to do so, which is to save its reward of H.

What does that mean for N?

a) If N is part of G and finds a solution to block H+1 first, there is virtually no chance this solution will get orphaned. After all, N is guaranteed M will back this solution, so there is at least that much extra hash rate guaranteed, which reduces the propagation time of N's solution. If m can't compete M's against propagation time, there is no reason to believe it can compete against M + N.

b) If N is part of g and finds a solution to block H+1, it knows M will attempt to orphan this block for a short period of time, and may succeed. M will try to orphan N's block simply because a window exists in which it is more profitable for M than to start work right away on top of N's block.

So N has no chance of getting orphaned if it is part of G, and a non zero chance to get actively orphaned if it's part of g. What motive does N have to be part of g? What motive does N have to not be part of G? If one solution has a quantifiable risk and the other doesn't, all of this for the same reward, why pick the risky one? Keep in mind that for N, there is no cost to switching from one solution to the other. N can start mining on top of m and switch to M later at no loss.

Point being, this situation can take place without the need of a 51% cartel.
You could argue that M is a cartel. But it doesn't need be 51% to present that active orphaning threat to smaller miners.
You could argue that over 51% of the network using the same relay network is a cartel, but then again their motivation to join the relay network is not to actively orphan smaller pools but to reduce their own orphan risk. There is no agreement that other members of the relay network will support your block in a race, but there is a strong indication that they will get your solution faster than they will get one from miners outside of the relay network.
You could argue that a mining pool is in fact a cartel in its own right, but this is only evidence that any barrier to entry will promote the formation of cartels (after all a solo miner is always guaranteed to reduce his propagation time by joining a pool, and gets other benefits on top of it).

"But I've never seen blocks emitted so close one another"

Far from me to doubt your figures, but the reality is that the 1MB block cap prevents propagation time from exploding upwards. The point isn't so much that concurrent block solutions are propagated within mere seconds of each but rather the average propagation time based on position in the network topology and hash rate. If 20MB blocks were to bump your propagation time to 30 sec, you would be clearly be a victim of this mechanic.

"But it would never reach 30 sec"

30 sec is a bloated value for the sake of this example but is it really all that unrealistic? I think that currently, ~60% of the network hash rate is in China. They naturally propagate fast to each other, and slowly to the rest of the world. How big does the average block have to be in the absence of fast relay networks for Chinese miners to receive it in an average 30 sec? Do we really want to try and find out?

"But no miners submit themselves to this logic atm"

That's mostly because the low block size hard cap prevents network propagation from getting out of hand. If the hard cap goes away with the current network topology, eventually some miners will start exploiting this aggressive orphaning scheme and the rest of the network will have to go along. And there already is a group in a prime setup for this purpose: Chinese miners.

"But cartels!"

While I am trying to demonstrate this particular scheme does not require a 51% miner/cartel, the scheme on its own is only enabled by long propagation times. This characteristic of the mining network creates imbalance and modifies the rules of the game in way that promotes the formation of cartels, the same way the heavily subsidizes cost of electricity in China is, in my opinion, the main factor for the current state of the mining market (where Chinese miners dwarf everybody else).


This ties back to the original topic, and the question that birthed this entire argument:


I am not suggesting this particular predatory mining practice is part of what XT is proposing. What I am suggesting is that this is one of the many imbalances that will emerge from removing the block cap in the current state of the mining network. My analysis clearly doesn't reach unanimity, and Kano (or someone else) may very well prove me wrong in the end (I still intent to defend my argument fiercely =P), but that doesn't nulify the other cases where large miners' profitability increases compared to small miners as blocks get bigger.

XT developers will either leave the network as is and in consequence promote a whole new class of incentives that will result in extreme mining centralization, or they will render propagation time insignificant through relay networks and have no mechanic to promote any nature of fee market whatsoever.

At this point I keep arguing the orphaning scenario with Kano for the sake of arguing (I hope he doesn't mind!), but that point is separate from the original topic, and my original contribution to the discussion, which so far still stands:

The XT developers don't want a fee market. Their statements and their decisions in code leads me to believe they do not see a fee market as useful to the network and I expect they will take the necessary steps to get rid of any remnants of such market, may it rise through unforeseen conditions (that I purport they see as limitations).

Hearn's mining insurance contract idea largely predates the XT fork debate, so at least he has been contemplating that narrative for longer than this contentious fork. I do not remember when or where I've read about it the first time nor whether it came with an actual proposal. Maybe someone has some links to contribute.

-----------------------------------------

As for Peter R's point, I don't really understand how it is relevant to the original topic. Whether a fee market can or cannot exist without a block size cap is irrelevant to both parties. XT wants no fee market to begin with, and Core wants a tight cap.

To proceed further, whether such a fee market could be healthy is an interesting discussion (although still irrelevant to the Core vs XT dilemma), but clearly Peter R's paper does not confirm or infirm that property. The main reason for that is that his definition of a healthy fee market is one in which there is constantly more demand for block space than the network topology can support, as long as there is a coinbase reward.

I don't want to sound demeaning, but to purport that network friction creates block size scarcity, sure. To then deduce a scarce resource in demand will naturally have a price... well duh. To finally conclude that this resource having a price is the necessary and sufficient condition for the market to be healthy, now that's quite the stretch. Some could even see the definition as insidious.

Moreover, you establish that in the absence of a coinbase reward, the entire model crumbles as the price of bytes in the blockchain becomes 0. This can only lead to a combination of the following conclusions:

1) Your model indicates that the cost of orphans cannot even sustain a fee market (let alone a healthy one) in the absence of inflation.
2) If indeed the cost of orphans can sustain a fee market without inflation, then your mathematical model is wrong, as it pretends otherwise. Or
3) Something is wrong with your premises. At least one of them is wrong or you are omitting a defining parameter of the network.

And before you object and link your paper again, I have read the entire thing thoroughly and maintain all the criticisms I have made so far. Among these criticisms, I will reiterate the main one:

Network propagation, as any properties of the network and the market that affects miner profitability through externalities, is a bad metric. We need a strong parameter to reign over fees specifically because among other things, it needs to dwarf the effect of externalities.


That's outright disturbing if this presentation stands true. Otherwise it's kind of incendiary. I don't want to judge before I see the actual proposal. Surely someone knows about it.

This doesn't happen.

May well in some future scenario, but certainly not now.

It's incendiary one way or another.

To get a second opinion, find someone who has worked in dynamic display ads and ask them how much they love data.  High value data like precise transaction logs probably has a broad range of opportunities for monetization which go far beyond simply spamming people with ads, but I suspect that just that alone would be enough to support a very robust monetary framework infrastructure and reason to work hard on gaining the largest footprint possible.

aaaae lookout, checkout the Street Fighter II challenge between kanoi and goatpig.
one half of the ck bitcoin mining software limited  joint venture versus Armory Technologies engineer.
Bitcoin software engineering's finest battle to the death. This will definately be in
Bitcoin the movie.  ;D

Possibly open a book on what their sfII character of preference is  :D Unless they're one of those "wildcard" assholes  >:(  ( ;D)

I am pretending it would happen with large propagation times, so we are not contradicting each other.


I am not doubting the value of data mining financial transaction logs, but you would have to prove Hearn is trying to modify the network in a way that it becomes feasible on the blockchain, and that such way can remain private to a few select people in order to turn a profit from this method first hand. After all, this dataset is public and there is no intuitive link between addresses and individuals.


I like Ken

i don't think its a matter of becoming feasible, its a matter of making it the primary data source. I.e., if blocks increase in size, bitcoin shifts from the concept of the groundlayer of cryptocurrency architecture and into the "all transactions on the primary blockchain" architecture. The more and more data leaves the primary blockchain, the less valuable it becomes.

re: second bold - once you introduce monetization into this equation, the lack of an intuitive link just transforms into another data mining problem.

hence probably why confidential transactions are in the works, which are somewhat dependent on the adoption of sidechains, which is somewhat dependent on the blocksize not changing. And if CT make their way into general adoptions (and I hope they do), you can say goodbye to data harvesting from the blockchain.

Man, dependencies are even a PITA when compiling reality!

This debate has become academic.  The network has already chosen a path, and XT ain't it. BIP 100 looks like the winner, and it also looks like a reasonable compromise; introducing a miner voting variable, and setting a 'soft' block limit based upon the lower bound of the three center quintelles of the vote distribution.  Looks like it will function similar to how the target difficulty system works, but with a 12 week period (instead of two) and a 12 week delay period.  It also has a factor of 2 max change limit, just like the target difficulty system; with a 32 meg upper hard limit.

Well written post.

The paper shows that as long as a non-zero amount of information about the transactions included in a block is communicated (on average) during block solution propagation, then the fee market will be healthy, according to the paper's definition of a healthy market. This assumption about information propagation has held over the entire history of bitcoin, will hold if the entire network uses the Corallo relay network, or adopts any implementation of IBLTs I can imagine.  The fee market exists.  

In hindsight, perhaps I should have chosen a word other than "healthy" that would have been less controversial.  Nevertheless, I define precisely what I mean by a heathy market in Section 7.  This figure illustrates it:

https://i.imgur.com/qZrh2vP.png

A healthy market is defined as one in which a rational miner will be incentivized to produce a finite block.  In other words, I show that block space behaves as a normal economic commodity (both the laws of supply and demand hold).  

You could argue that the market is "not healthy" (using a different definition) because the equilibrium block size in a free market would result in some negative externality (centralization--although I don't buy that).  And then you would propose that a quota on the production of block space needs to be put into place, for some greater good.  However, before we go down the road of production quotas, we should take a long look at other economies where they've been implemented.  The most famous is of course the USSR, but even in Canada we have quotas on the production of eggs.  These were originally implemented to ensure that farmers could earn a living wage and the public could enjoy a reliable supply of eggs; however, they now serve to keep small famers out of the commercial egg market and result in all sorts of lobbying and palm greasing (and higher prices for consumers).  He who controls the quota wields the power to pick winners and losers...



The model simple no longer applies as R/T -> 0.  It's neither right nor wrong.  It's sort of like an equation where you have 0 divided by 0.  It's undefined.  You need to evaluate the limit.  

I don't know if fees will be enough to secure Bitcoin 50 years down the road.  My work said very little about this.  That's the topic for a future paper.  

The whole point of "anti friction" measures like relay networks and IBLT is to communicate as much data as possible about a block "out of band" (while the block is still being mined). This doesn't simply mean it takes less bytes to communicate a block, it also means the bytes sent to bytes used ratio won't necessarily grow linearly anymore.

If you have to communicate as many bytes as you use in a block, then your analysis stands. If a coding gain can be achieved in the form of lossless, flat % compression, then ratio sent/used will remain roughly static, describing a linear growth. Again the model will stand.

If however the only data you have to transmit at propagation time is what you cannot communicate out of band, i.e. block header, coinbase transaction and any tx that you know aren't part of other's mempool, then the sent/used ratio will grow logarithmically, which is the same as saying that propagation speed (Note speed, not time. Higher speed means faster propagation) will increase exponentially with block size. This would put us in the case you describe as the "non-existent market".


Maybe organic or self sustained are better wordings. Semantics are relevant in a research paper.


This is from the introduction:


The full cost to post transactions is not explicitly defined so it is reasonable to expect it means "the cost to maintain proper difficulty" (which is largely accepted as the main purpose of transaction fees as inflation diminishes), and this is the assumption I was operating on until I reached page 7.


If there exists a market condition where a miner's profitability per hash increases with total hash rate, then there is a centralizing pressure on mining, as larger miners will out earn their competition. Either we're talking pools and hash rate providers will centralize around a single large pool, or we're talking actual miners and the largest of them all will win the "arms race" by default.

If you doubt propagation time can have this effect, then consider that Q* grows with a miner's hash rate, as they have to propagate their solution to an decreasing portion of the network to have it validated. After all, they only need to propagate to a cumulated 51% of the network's hash rate.


If block size limit is a production quota, then coinbase reward is minimum wage or government grants. Both are externalities and both have devastating effects on an economy. Yet both are currently in place in the Bitcoin network (and every alt coin out there has a mix of both), and the only true centralizing force at the moment is the heavily subsidized cost of electricity in certain parts of the world. Clearly the macro economics of classical commodity markets do not apply (at least as is) to crypto-currencies.


And yet that situation is defined outside of your model: some transaction emitters would mine their own transactions, and others will pay them to mine theirs. If your model cannot come to this conclusion then it is at least lacking something. Do you realize that according to your results, no one would ever mine on the testnet chain?

Correct.  Even with coding gain, the result holds.  


If the information required to be sent grows logarithmically with block size, then the market would be exactly on the border between healthy and non-healthy.  

But this is a bold claim that requires further justification.  Even doing as you suggest, I cannot see how the information won't be linear with the block size.  For example, you said you still need to communicate "any TXs that you know aren't part of other's mempool".  I agree.  But if we imagine that 95% of the transactions are common between mempools, then the number of missing TXs still grows linear with block size.  

Furthermore, you still need to communicate the order in which the transactions are sorted in a block (even if all the TXs were common between mempool).  Communicating the order of a list of transactions requires an amount of information linear in the number of transactions in that list.  

Lastly, even if you can come up with a scheme to do what you're suggesting (and I don't think you can), you still need to get 100% of the contributing hash power to go along with it.  If only 90% does, the fee market remains healthy.  

I think you are looking for ways around a Law of Physics here.  The only way the market will not be healthy (according to the definition in my paper) is if exactly zero information about the transactions included in a block is communicated with the block solution announcement and this holds for every block solution announcement.


You missed the rest of the sentence ;) .  I think I agree now that you point it out the word "full" is ambiguous and should be changed, however.

Ok, this post is long enough already so I'll leave the last half of your response for later...

Not really. It follows the same growth pattern, not necessarily the same rate. A linear growth will always surpass a logarithmic growth, regardless of the rate. Once we are using the same functions, (k log(x)), suddenly all that matters is the rate (k). There is no indication we would be on that border exactly, we could be way below or way above it. What do the law of physics have to do with block space demand again? Note that I am not arguing its effect on supply.


The only way this can happen is if a miner is purposefully creating his own tx and not emitting them publicly. Mempool hit rate is just another propagation system, with time it will always tend towards 100%. The scenario you describe where the mempool hit rate constantly hovers at 95% if only possible if a miner is purposefully creating transactions at a pace and withholding them. That makes no sense as it increases his propagation time (reduces his profit) and motivates other miners to eventually ignore him. On contrary, I would speculate all miners will make a point of sticking to well propagated transactions particularly to suppress their orphan rate.


What makes you believe that information cannot be communicated pre solution? Also look at IBLT implementation. The intent is to give other miners the rules you build your own blocks with. There is no reason a miner propagating an IBLT should stray from the pattern it describes. Again, miners have a clear incentive in fast relay networks to use as predictable a tx set as possible. That means ignore transactions that are not fully propagated and stick to your IBLT.


No I only need 51%. The rest of the network will get steamrolled out of business pronto. Part of your paper relies on how rational miners should account for propagation time in their tx set choice (the model you propose is naive but let's keep it at that). Now somehow you are arguing 2 things to me:

1) You are suggesting rational miners won't take every opportunity to reduce propagation time.

You state yourself that a high propagation time will result in lower revenue. Rational miners are motivated by profit, why would they stay away from any coding gain?

2) You are suggesting irrational miners have a stake in this game.

Clearly they don't, their only fate is to get eaten alive by rational miners, or bleed money from an external source ad nauseum (or until that source depletes I guess).


You always have to propagate the block header and your coinbase tx at solution annoucement. But there is no indication we have to stick to a network that has to propagate every other bit of information in a block along with the solution. We have solutions being developed that allow miners to only ever have to emit the header and coinbase tx. The Nash equilibrium will keep all miners doing just right that as long as propagation time inversely affects profit (read forever).

So the amount of data emitted is static and the network bandwidth will keep growing until we reach the physical limit of the carrier. Therefor the propagation time is tending towards 0 (remember math 101, tending is not reaching). So we do have a situation where blocks will propagate at insignificant speed compared to the block emission period, without the poles inverting.


No I purposefully omitted it, replacing it with ellipses to signify that it was an excerpt from a sentence or paragraph, as common citation etiquette dictates.