Bitcoin Forum

Bitcoin => Development & Technical Discussion => Topic started by: DannyHamilton on December 30, 2017, 06:04:58 AM



Title: Transaction Quantity Anomaly
Post by: DannyHamilton on December 30, 2017, 06:04:58 AM
EDIT:  Re-posted because I forgot to mark the previous post as "self-moderated".

Perhaps this has already been discussed, and I've just somehow overlooked the discussion about it in the past?  If so, let me know and I'll lock this thread.

I was looking over the transaction volume at various fee levels at the following site:
https://jochen-hoenicke.de/queue/#2h

I noticed a behavior for which I'm lacking a good explanation.

In the first 4 to 5 minutes after a block is solved, there is a sudden surge of transactions received at the 5 µBTC per byte to 7 µBTC per byte level (approximately 1200 transactions added during that 4 to 5 minute period). The rate of transactions at that fee level appears to be approximately 240 transactions to 300 transactions per minute.

Then until a block is solved, the number of transactions added at that fee level SUDDENLY drops to less than half that rate, about 100 transactions per minute.  As soon as a block is solved, there is another burst of about 1200 or so transactions at the high 240 transactions to 300 transactions per minute for 4 or 5 minutes again.

I don't see this sudden post block burst at any other fee rate right now.  The transaction count at all the rest of the fee levels seems to grow the same rate consistently.

It is difficult to see any detail on the charts that go back more than a day, so I'm not sure if the same behavior can be seen at different fee levels when the average fee level is higher or lower.

Does anyone have any good explanations of why there would be a temporary 4 to 5 minute burst of so many transactions at such a high fee level immediately after EVERY block, or why the rate of new transactions at that fee level would consistently and suddenly drop after 4 or 5 minutes?

You can see the effect in the chart below where the slope of the blue section suddenly changes:

https://i.imgur.com/Po69NsA.png


Title: Re: Transaction Quantity Anomaly
Post by: amaclin1 on December 30, 2017, 11:00:00 AM
Does anyone have any good explanations of why there would be a temporary 4 to 5 minute burst of so many transactions at such a high fee level immediately after EVERY block, or why the rate of new transactions at that fee level would consistently and suddenly drop after 4 or 5 minutes?
How do the modern Core/bc.i/etc clients calculate default fees?
Is the default fee the same immediately after the block is found and five minutes later when the mempool is filled with 1mb of transactions with this fee?

Ok, this is not an explanation. We should take "first 5 min" transactions and compare them to "next 5 min" transactions


Title: Re: Transaction Quantity Anomaly
Post by: DannyHamilton on December 30, 2017, 04:14:04 PM
How do the modern Core/bc.i/etc clients calculate default fees?
Is the default fee the same immediately after the block is found and five minutes later when the mempool is filled with 1mb of transactions with this fee?

I don't know, but if this was the cause, then I would expect the to see a change (increase) in rate of the next higher fee level at the time that the 5 µBTC - 7 µBTC level changes (decreases).  Then 5 minutes later, I'd expect that the rate would decrease at that level as the recommended fee continues to walk up.

This isn't happening.  I'm not seeing any significant shift in rates of transactions at higher or lower fee levels.

Ok, this is not an explanation.

I agree.

We should take "first 5 min" transactions and compare them to "next 5 min" transactions

At this point, I'm just relying on what is reported on that website.  It is possible that there is something wrong with their recording or reporting.  I haven't yet tried analyzing transactions that my node receives and holds in its mempool.

Let me know if you put any effort into this and what you find.


Title: Re: Transaction Quantity Anomaly
Post by: TheQuin on December 30, 2017, 04:35:01 PM
At this point, I'm just relying on what is reported on that website.  It is possible that there is something wrong with their recording or reporting.  I haven't yet tried analyzing transactions that my node receives and holds in its mempool.

Let me know if you put any effort into this and what you find.

I have noticed also on bitcoinfees.earn.com that after each block the number of transactions queued at the highest displayed price level (currently 721+ sats/byte) increases almost immediately then steadies off. This seems to correspond with your observation on jochen-hoenicke.de.

I've been scratching my head to come up with an explanation as well without any success. It does seem to have the effect of preventing the recommended fee falling even though there isn't much in the mempool at high fee levels.


Title: Re: Transaction Quantity Anomaly
Post by: amaclin1 on December 30, 2017, 06:10:04 PM
Let me know if you put any effort into this and what you find.
Certainly.
I think that I have enough knowledge ant tools to check this "anomaly". (I am not sure that it exists)
And of course I can/will publish my results.
But I am not interested in doing this job in nearest days.


Title: Re: Transaction Quantity Anomaly
Post by: jnano on December 30, 2017, 07:20:32 PM
It's not always the case. Here's Dec 26:
https://i.imgur.com/dFOGJFz.png

How do the modern Core/bc.i/etc clients calculate default fees?
Each client/wallet handles it differently, I assume.


Title: Re: Transaction Quantity Anomaly
Post by: DannyHamilton on December 30, 2017, 07:53:22 PM
It's not always the case. Here's Dec 26:

It looks like it still was happening on Dec 26.  Just at a higher fee level (7 to 10 µBTC per byte)

https://i.imgur.com/ZcqqEra.png


Title: Re: Transaction Quantity Anomaly
Post by: haltingprobability on December 30, 2017, 10:06:20 PM
Speculation: Big institutional player with HFT equipment? There has been a drastic change in the mempool since Dec 6, that's for sure.

https://i.imgur.com/Soy5INq.png


Title: Re: Transaction Quantity Anomaly
Post by: Quickseller on December 31, 2017, 02:02:18 AM
This is somewhat speculation, but I somewhat suspect this is something the fee market will cause.


Businesses that process withdrawals on a 24 hour basis on demand from their customers, need to price withdrawal transactions on a "confirm next block" basis. If they price withdrawals on a "confirm within 3 blocks" (or something less), then (a) they can only process withdrawals every 3 blocks and (b) risk havoc being wrecked if transaction fees  rise between when transactions are broadcast, and the next 3 blocks, including the possibility of having their entire hot wallet tied up in unconfirmed transactions. If the business chooses to allow a certain number of chained unconfirmed transactions, they risk a more complex problem if the mempool situation gets so bad that many nodes forget parent transactions that are unconfirmed. Hot wallet implementations at many large businesses are setup so that employees do not have direct access to the hot wallet, but rather have access to a program that will in turn have access to the hot wallet, so selecting specific inputs is more difficult.

It is possible that some businesses estimate the required fee in a way such that after ~5 minutes from the time the last block was found, they have a better idea of the required tx fee for a transaction to get confirmed in the next block, and lower fees paid accordingly.


Title: Re: Transaction Quantity Anomaly
Post by: codewench on December 31, 2017, 01:00:51 PM
Does anyone have any good explanations of why there would be a temporary 4 to 5 minute burst of so many transactions at such a high fee level immediately after EVERY block, or why the rate of new transactions at that fee level would consistently and suddenly drop after 4 or 5 minutes?

You can see the effect in the chart below where the slope of the blue section suddenly changes:

https://i.imgur.com/Po69NsA.png

Don't look at the time until the drop off. Look at the height, aka number of transactions. The height of the knee is about 3/4 of what gets mined each block.

My theory: This is an exchange (or exchanges) broadcasting send transactions. Every time a block is solved, the exchange blasts out a new set of sends with a "next block" level fee. Once there are enough transactions to fill most of the next block, there is no sense broadcasting more transactions - they wouldn't get into the next block anyway. An exchange won't want unconfirmed transactions hanging around as mentioned by Quickseller. Meanwhile, individual wallets start building transactions with a higher fee to get ahead of the exchange's transactions. This is why you see the yellow and gray lines start to grow after the knee.

The knee itself likely is much sharper, i.e. it happens much sooner after a block is solved. The jochen-hoenicke node is likely not close enough to the exchange(s) to receive all the new transactions right away. They're spreading temporally as they are relayed across the network.