Bitcoin Forum
November 19, 2018, 05:06:02 PM *
News: Latest Bitcoin Core release: 0.17.0 [Torrent].
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Why transactions inside blocks are not quite sorted by the fee?  (Read 744 times)
piotr_n
Legendary
*
Offline Offline

Activity: 1960
Merit: 1039


aka tonikt


View Profile WWW
August 01, 2017, 03:58:06 PM
 #1

It's not an issue, but I find in intriguing and would like to understand the reason.

When I draw a chart for each block that presents a fee (in SPB), in relation to how far the tx was placed inside the block, it almost always looks somehow like this:



What I mean is that the chart in general goes form the highest SPB to the lowest one. - which is understandable.
But what is strange for me is that it isn't smooth.
There are spikes - in some places inside the block you have a tx whose fee is "too high" for that place, but that is sort of compensated by a tx placed next to it whose fee is to low to end up there.

So what is the reason for placing the txs like this?

Is it because of the Child Pays for Parent feature, or is it something else?

Sorry, I know I can figure it out by reading the code, but hope that someone can just help me here.
I mean, by referring me to the specific code that is responsible for this behaviour.

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
1542647162
Hero Member
*
Offline Offline

Posts: 1542647162

View Profile Personal Message (Offline)

Ignore
1542647162
Reply with quote  #2

1542647162
Report to moderator
1542647162
Hero Member
*
Offline Offline

Posts: 1542647162

View Profile Personal Message (Offline)

Ignore
1542647162
Reply with quote  #2

1542647162
Report to moderator
1542647162
Hero Member
*
Offline Offline

Posts: 1542647162

View Profile Personal Message (Offline)

Ignore
1542647162
Reply with quote  #2

1542647162
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1542647162
Hero Member
*
Offline Offline

Posts: 1542647162

View Profile Personal Message (Offline)

Ignore
1542647162
Reply with quote  #2

1542647162
Report to moderator
1542647162
Hero Member
*
Offline Offline

Posts: 1542647162

View Profile Personal Message (Offline)

Ignore
1542647162
Reply with quote  #2

1542647162
Report to moderator
1542647162
Hero Member
*
Offline Offline

Posts: 1542647162

View Profile Personal Message (Offline)

Ignore
1542647162
Reply with quote  #2

1542647162
Report to moderator
DannyHamilton
Legendary
*
Offline Offline

Activity: 2198
Merit: 1385



View Profile
August 01, 2017, 04:22:05 PM
 #2

Many mining pools run their own private code.

There is no requirement for the transactions in the block to be in any particular order (as an exception, I think I recall that if a transaction spends the outputs of another transaction in the same block then it must be later in the block than the transaction whose outputs are being spent).

Miners/pools are allowed by the protocol to use any criteria they want for choosing which transactions to include (so they don't have to include the highest fee per byte transactions if they don't want to), and (other than the exception I listed above) they can order the transaction however they like within the block.

CPFP is a likely reason that you are seeing those spikes.  It is possible that paired up transactions (where one spends the outputs of another) are placed next to each other when building the block.  It is also possible (though probably unlikely) that the pools are randomly moving some transactions around a bit in the block as an additional method of modifying the merkle root (in addition to simply incrementing the extraNonce).

piotr_n
Legendary
*
Offline Offline

Activity: 1960
Merit: 1039


aka tonikt


View Profile WWW
August 01, 2017, 04:32:34 PM
 #3

This pattern is very common, across different pools.

I think it's something that bitcoin core must be doing.

Unless some mining software that assembles the blocks is so common as well.

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
piotr_n
Legendary
*
Offline Offline

Activity: 1960
Merit: 1039


aka tonikt


View Profile WWW
August 01, 2017, 04:47:20 PM
 #4

Here you see it even better:



It's a fresh one, mined 5 minutes ago.
And it took ~30 minutes to mine it.

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
achow101
Moderator
Legendary
*
expert
Offline Offline

Activity: 1582
Merit: 1751


bc1qshxkrpe4arppq89fpzm6c0tpdvx5cfkve2c8kl


View Profile WWW
August 01, 2017, 07:34:53 PM
 #5

Bitcoin Core 0.13.0+ (and its derivatives) use a thing called ancestor fee rate ordering. What this does is it takes transactions and their dependencies and groups them together. Then the groups are sorted by their combined fee rate. This results in a generally decreasing fee rate as you go through the transactions in the block, with occasional spikes due to CPFP where a child has a large fee rate, but combined with its parent, is a smaller fee rate so placed further down in the block.

piotr_n
Legendary
*
Offline Offline

Activity: 1960
Merit: 1039


aka tonikt


View Profile WWW
August 01, 2017, 07:39:45 PM
 #6

Thank you, I didn't know about that.
Where is it in the code? Who made it?

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
achow101
Moderator
Legendary
*
expert
Offline Offline

Activity: 1582
Merit: 1751


bc1qshxkrpe4arppq89fpzm6c0tpdvx5cfkve2c8kl


View Profile WWW
August 01, 2017, 07:49:06 PM
 #7

Thank you, I didn't know about that.
Where is it in the code? Who made it?
I believe this is the PR that made it: https://github.com/bitcoin/bitcoin/pull/7600

The code should all be in src/miner.cpp.

piotr_n
Legendary
*
Offline Offline

Activity: 1960
Merit: 1039


aka tonikt


View Profile WWW
August 01, 2017, 07:54:23 PM
 #8

great
thanks

Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2562
Merit: 1671



View Profile
August 02, 2017, 06:38:31 AM
 #9

FWIW, if you think core is doing something-- you can check the getblocktemplate output yourself and remove all doubt! Smiley

Achow101 is right about the cause, if you look in the graph you will see a dip to low feerate right before the high feerate spike: thats the parent then its high rate descendant transaction.

You can think of it as logically working like this: if grouping a transaction with its descendants gives a higher feerate for the group than the transaction had, consider including the group.  They go in that order... the whole group in its group-feerate order.


If I were making graphs like yours, I think I'd logically merge all transactions that have unconfirmed parents into their parents, and show the combined rate.  I believe that graph will be monotone.

Bitcoin will not be compromised
piotr_n
Legendary
*
Offline Offline

Activity: 1960
Merit: 1039


aka tonikt


View Profile WWW
August 02, 2017, 08:16:12 AM
 #10

If I were making graphs like yours, I think I'd logically merge all transactions that have unconfirmed parents into their parents, and show the combined rate.  I believe that graph will be monotone.

OK. That would be interesting.

But I don't quite understand how to group them.

For instance I have "parent" transactions A and B - both spend only confirmed inputs.

Then I have:
C - spends input from A
D - spends input from B
E - spends input from both, A and B

Do all five (A, b, C, D, E) end up in one group then?


Check out gocoin - my original project of full bitcoin node & cold wallet written in Go.
PGP fingerprint: AB9E A551 E262 A87A 13BB  9059 1BE7 B545 CDF3 FD0E
gmaxwell
Moderator
Legendary
*
qt
Offline Offline

Activity: 2562
Merit: 1671



View Profile
August 02, 2017, 09:09:03 AM
 #11

oh darn, yes in that case you can't tell if you should group without knowing if it used the txns natural feerate or the package feerate.  So much for my simple approximation. Smiley

Bitcoin will not be compromised
spin
Sr. Member
****
Offline Offline

Activity: 360
Merit: 250


View Profile
August 03, 2017, 11:07:56 PM
 #12

I guess it should check if any combination of transactions starting from A or B is the next highest fee rate it would include it.

I guess the possible groups could be:
A
B
A-C
B-D
or A-C B-D E

I am assuming whichever of those groups have the highest fees will be included first?

If you liked this post buy me a beer.  Beers are quite cheap where I live!
194YjsiwmGm3hcbPcJWWyzRAS9CQLX1fJL
Pages: [1]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!