Bitcoin Forum
April 23, 2024, 09:26:04 PM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
  Home Help Search Login Register More  
  Show Posts
Pages: [1]
1  Bitcoin / Development & Technical Discussion / Mempoolcp - A cmd utility to copy mempool between nodes. on: February 13, 2023, 06:46:50 PM
I've coded mempoolcp

https://github.com/dev7ba/mempoolcp

What is mempoolcp?

Mempoolpc is a command-line program to copy the mempool from one bitcoin node to another.

How does it works?

Through bitcoin nodes rpc interface, this program uses getrawmempool(verbose), getmempoolentry, getrawtransaction and sendrawtransaction rpc calls to copy the mempool between nodes.

Mempoolpc takes into account the dependencies between transactions and the fact that you can't send a child tx before a parent tx, or a parent tx before a grandparent tx... because otherwise, the sent transactions could be denied by the receiving node.

Mempoolcp is fast, as fast as rust serde is. Also, mempoolcp use multithreading when possible.

It has two modes of operation: a faster one using more memory and a normal one using less. The faster uses getrawmempool_verbose (a heavy call that uses a lot of memory if there are many txs). and then getrawtransaction + sendrawTransaction for each transaction. The normal mode uses getrawmempool (without verbose), then getmempoolentry + getrawtransaction + sendrawTransaction for each transaction.

Configuration is done via the command line or via mempoolcp.conf in a file (to avoid using passwords in the shell). It can actively ask for the user and password if needed.

It has an option to choose network (ports): mainnet, testnet, regtest...

It is compatible with any limitancestorcount value in bitcoin.conf

Currently only support user/password authorization.

Which problems solves?

At least 3 questions in bitcoin stack exchange:

https://bitcoin.stackexchange.com/questions/63675/synchronize-mempool-between-3-nodes
https://bitcoin.stackexchange.com/questions/53638/are-there-any-ways-to-sync-mempool-from-another-nodes-faster
https://bitcoin.stackexchange.com/questions/93231/how-to-achieve-fast-mempool-synchronization-at-startup

2  Bitcoin / Development & Technical Discussion / An analysis on the profit differences between mining pools due to txs fees. on: September 12, 2022, 02:26:09 PM
Hi there.

I have calculated the average profit because of txs fees (no subsidy) per block , for each miner, in the last 4 months.

You can find it here:

https://mempoolexplorer.com/miner

You can order ascending or descending by column, and change values to USD,BTC or Sats.

I was expecting for all pools having a decent % mining share (Binance, AntPool, ViaBTC, F2Pool, Slush, Foundry USA and Poolin) to have this value more or less the same. I was expecting this because mining activity is independent of mempool size, and all miners on average will have the same luck obtaining tx fees during a long period of time.

But I found to have almost a 15% variance between mining pools. (rejecting small pools as btc.com, ultimus, 1Thash... because not having enough data to be representative).

The causes of this can be:

- Optimizations in txs selection algorithm by pool operatos-> See https://gist.github.com/Xekyo/5cb413fe9f26dbce57abfd344ebbfaf2#file-candidate-set-based-block-building-md (less than 0,2% improvement).

- Better Network connectivity among miners -> See https://www.dsn.kastel.kit.edu/bitcoin/ (Txs and Block propagation time are really small to have any impact)

- Txs included by miners themselves with no broadcast-> I do not take into account this txs by comparing against my mempool when block arrives.

- Empty blocks-> I do not take into consideration for the average the empty blocks, and anyways they are only a few.

- Transaction accelerators-> This is my main point.

Could be that txs accelerators are the main factor explaining those differences? If that is the case, Poolin.com is the most used txs accelerator since they choose much worse sat/vByte ratio txs in average than the other miners (See Average lost reward column). And they are effectively way down the ranking of on-chain profit.

Have you ever seen this statistic anywhere else?
3  Bitcoin / Development & Technical Discussion / Huge transactions dependency graphs in mempool on: February 02, 2022, 10:53:43 AM
Two months ago, while developing my webpage about mempool statistics, I found that there was a huge dependency graph having more than 700 transactions in (my) mempool.
I could capture some minor examples having around 470 transactions (see below)

Of course no mempool rule is broken, i.e. max mempool ancestors for a transaction, but I'm wondering about what can be doing this kind of dependency graphs.

I posted a question about it in bitcoin.stackexchange without luck:
https://bitcoin.stackexchange.com/questions/110723/huge-dependency-graphs-for-transactions-in-mempool

The best answer given to me by https://b10c.me/projects/ was that maybe there was an agent running out of utxos in its wallet, and then started using its non-confirmed utxos in the mempool, forming the dependency graphs.
If that is the case, the graphs seemed not having a structure to minimize its depth for not exceeding the max transactions ancestors rule in the mempool (i.e. a tree), and seemed pretty unstructured?. (but investigating about the best strategy in this case and recognize its visual shape is something I haven't done. Take that with a grain of salt)
So although the former answer is the most plausible, I feel it is worthy to share this with you.

What can be creating these dependency graphs?
Are they worthy to investigate or has been investigated before?

Captured transaction graphs:
https://gist.github.com/dev7ba/c144c68127b97082652bc860cc95edf6
https://gist.github.com/dev7ba/50167d6e336100698ab2599123444efc
These .json are offered by the backend of my web page you can call it here: https://mempoolexplorer.com/txmempool/miningQueueAPI/tx/{txIdHere} to obtain a transaction statistics currently in mempool.
See txDependenciesInfo.nodes and txDependenciesInfo.edges data to obtain the dependency info (if any) for queried transaction.

You can visually see the transactions graphs currently in (my) mempool here: https://mempoolexplorer.com/txsGraphs.
These graphs can be linear: forming a long chain of dependencies, or non linear: forming a Direct Acyclic Graph.
When clicked on the number of transactions on a graph, a list of the transactions ids contained in that graph is shown. You can go to the graph by clicking any of these transactions id. The result is shown in the main page along with its position in the mining queue (from my node's perspective) and other useful data.
To avoid the annoying constant refreshing please check "lock mempool" in the upper right corner.

Be aware that I currently don't have any historic section of dependencies graphs or something like that in the webpage. The results show are in real time, so if mempool is almost empty maybe no graphs or very small ones are shown.

Note that the following graphs are minor examples compared to the huge ones I'm talking about. (sorry, I didn't take a screenshot for them, only the .json returned by my backend)



Bonus Points: This webpage has other sections discussed in https://mempoolexplorer.com/faq and also here https://bitcointalk.org/index.php?topic=5383112.0 that may be of interest to you. I'd be glad receiving feedback.

4  Bitcoin / Development & Technical Discussion / A way to know how good a miner is choosing its transanctions on: January 26, 2022, 10:30:24 AM
Hello there!

I'm building a webpage: https://mempoolexplorer.com with bitcoin mempool statistics and I have some sections there that I don't really know if they are worthy so I'm asking here for help, and also, for a constructive criticism of the webpage since I'm dealing with what to do next.

I've posted this same question in bitcoin.stackexchange without luck, And I've realized that maybe is a too general question to be asked there so I'll try here:

Is there a way to know how good a miner is choosing its transanctions to be mined from a fee maximization point of view?

Well, this is the way I've tried:

When a mined block arrives to my node, I compare it against a transaction selection algorithm using my mempool. (I know my mempool it's different than the miner's one, and that's the point so keep reading ;-) )

The transaction selection algorithm sorts transactions in the usual greedy way: using the regular Ancestor Set Based (ASB) algorithm defined [here][1], and considering transaction dependencies and [CPFP][2], so basically is the same as invoking getblockTemplate with the state of my mempool just before the block arrives. (I also compare the incoming block by using the last getBlockTemplate result of bitcoind but as it's only calculated each 30s the results are not accurate).

An example of a comparison is shown [here][3], (my webpage, in development). I divide the transactions in sets whether are ignored by the miner (from my node point of view), are in common with me, Ignored by me, Relayed to me, or not relayed to me.

And, of course, I also measure the fees I'd collected against the miner's block (having into account the incoming block coin base transaction size for better accuracy). You can check the results of the last blocks [here][4], and an aggregate for the miners [here][5].

This could give me an idea about how good a miner is selecting its transactions and/or has good network connectivity (in average, and against my mempool), but as I have no way of knowing the state of the miners mempool before they mine a block (not when I receive it), this effort seems inaccurate.

I make the assumption that a received block and my block template can be compared. But as the block propagation time is not, by any means, negligible, the comparison is skewed giving the idea that my algorithm/network situation is better. The reality is that I am receiving transactions while the block is being propagated to me, and thus, I have normally a bigger mempool than the miner when the block was mined. A bigger mempool gives my algorithm the opportunity to search for better transactions which had no opportunity to be mined by miners.

So, Is the accuracy of this methodology worthy? I've assumed its not. But as block propagation time is getting smaller (see [this][6]) And the results for some blocks/miners seems really different (see [this][7] block for example) I have my doubts.

NOTE: Be aware that I've started the webpage only 2-3 hours ago so only last blocks will be shown.

Same question in stack exchange:
https://bitcoin.stackexchange.com/questions/111940/is-there-a-way-to-know-how-good-a-miner-is-choosing-its-transanctions-to-be-mine

  [1]: https://gist.github.com/Xekyo/5cb413fe9f26dbce57abfd344ebbfaf2#file-candidate-set-based-block-building-md
  [2]: https://bitcoinops.org/en/topics/cpfp/
  [3]: https://mempoolexplorer.com/block/last/OURS
  [4]: https://mempoolexplorer.com/block/OURS
  [5]: https://mempoolexplorer.com/miner
  [6]: https://www.dsn.kastel.kit.edu/bitcoin/?ref=hackernoon.com#latency
  [7]: https://mempoolexplorer.com/block/719478/BITCOIND
5  Bitcoin / Development & Technical Discussion / Arbitrary block size increase via soft-fork on: October 11, 2016, 10:12:05 PM
Hi there, I hope not to ask a stupid question but...

Is there any way to arbitrary increase the block size via softfork? I mean, apart from Segregated Witness, Schnorr signatures or MAST which optimizes transaction size, and regardless of storage/bandwidth problems like the Great Firewall of China. Has anybody discovered/studied how to do that via softfork?

6  Local / Español (Spanish) / Las diferentes casas de cambio tienen precios muy diferentes ¿Que pasa? on: April 12, 2013, 09:29:08 AM
Despues del DDOS a mtGox, bitstamp... etc las casas de cambio tienen precios muy diferentes, ¡incluso de 5 dólares!

¿Alguien tiene idea de porqué es esto?

Muchas gracias

7  Local / Español (Spanish) / Sobre el efecto de los forks de bitcoin sobre su economía. on: February 26, 2013, 10:34:57 PM
Este sábado pasado, al salir de la conferencia de Bitcoin en el instituto Juan de Mariana tuve la oportunidad de discutir con Víctor Escudero y Luiscar sobre que pasaría si se hiciese un fork en el desarrollo de bitcoin. Me gustaría compartir con vosotros las conclusiones a las que llegamos.

Imaginemos que sobre el desarrollo de bitcoin se hace un fork llamado bitcoin2 o B2. Este fork tiene unas características diferentes al bitcoin original, que le hacen ser mas atractivo que el original para un tanto por ciento de la población minera y usuaria de bitcoin. No me interesan describir las características diferentes del bitcoin2, sino describir lo que pasaría con el fork.

Llamemos M1 a la población minera del bitcoin original y M2 a la población minera del bitcoin2. Los bloques generados por M1 no son reconocidos por M2 y viceversa, al establecer reglas diferentes en las transacciones. Sin embargo bitcoin2 es "compatible hacia atrás" del bloque 0 al X, pero a partir del bloque X establece unas reglas diferentes que como he dicho, resultan atractivas para cierta parte de la población minera/usuarios.

El resultado es que se hace un fork en la cadena de bloques teniendo B1 y B2 los bloques del 0 al X compartidos.

Una persona que tenga bitcoins en los bloques compartidos podrá gastar sus bitcoins dos veces, una vez en B1 y otra vez en B2 (gracias por caer en esto Luiscar) Sin embargo, EL VALOR de esos bitcoins no cambiará ¿Por qué?

Porque el valor de B1 en el mercado habrá caído para dárselo a los B2. A partir de que se origina un fork de este tipo, los usuarios tienen reserva de valor en ambas ramas, aunque es posible que quieran mover el valor de una rama a otra a través de sitios de intercambio entre B1, B2 que se originarían.

Así, cada una de las ramas B1 y B2 sería una moneda diferente compitiendo en el mercado por tener unas características monetarias adecuadas para los usuarios.

No he metido en esta ecuación el resto de monedas fiat estatales. Pero sería posible que ambas ramas B1 y B2 ganasen en valor frente al resto de monedas fiat estatales, al determinar el mercado que la moneda estatal es peor que B1 y que B2.

Bueno. ¿Que opináis sobre el asunto?. Soy un chico de brain-stormings :-)

Pages: [1]
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!