Metrics in Bitcoin Core with Opentelemetry

[dougEfish]

jlopp has a great fork for node metrics  https://github.com/jlopp/statoshi and I'll like to extend it.
Currently statoshi only exports metrics to statsd but I want to the ability to write (easily) to another collector.
Opentelemetry ( https://github.com/open-telemetry/opentelemetry-cpp ) is one lib I found that provides an abstract interface and already has multiple collectors (e.g. https://github.com/open-o11y/docs/tree/master/cpp-prometheus)
Thoughts on this approach? Is there another lib or methodology that makes more sense?

 

[1713527227]

1713527227

[1713527227]

1713527227

[1713527227]

1713527227

[NotATether]

Why wire Core to pass stats through a separate collector lib when you can just hardcode the information-gathering inside Core itself and have it export the data directly to a monitor like statsd or prometheus?

Any collecting will have to be done by making Core dump the stats to files since connecting Bitcoind with sockets to other auxiliary programs is not something I think the core devs are enthusiastic about doing.

[odolvlobo]

jlopp has a great fork for node metrics  https://github.com/jlopp/statoshi and I'll like to extend it.
Currently statoshi only exports metrics to statsd but I want to the ability to write (easily) to another collector.
Opentelemetry ( https://github.com/open-telemetry/opentelemetry-cpp ) is one lib I found that provides an abstract interface and already has multiple collectors (e.g. https://github.com/open-o11y/docs/tree/master/cpp-prometheus)

The actual OpenTelemetry site is this: https://opentelemetry.io/. The github project that you link to is a C++ client that is still in development.

Why wire Core to pass stats through a separate collector lib when you can just hardcode the information-gathering inside Core itself and have it export the data directly to a monitor like statsd or prometheus?

Any collecting will have to be done by making Core dump the stats to files since connecting Bitcoind with sockets to other auxiliary programs is not something I think the core devs are enthusiastic about doing.

If you are going to hard-code the information-gathering, you are going to end up creating a collection module anyway, so you might be better off using a mature well-tested library rather than rolling your own.

Bitcoin Core has performance bottlenecks. Creating a special telemetry build that would not be intended for general use is a good idea. Core devs should be enthusiastic about something like that.

[dougEfish]

My intent is not to integrate this into Core, but to fork Core so that:

1. I learn about the bitcoin software architecture through metrics
2. integrate an abstract lib that can be configured to export to statsd or prometheus (or even both and others)

 https://github.com/jlopp/statoshi hard codes statsd, I personally want prometheus.

Opentelemetry-cpp seems to be flexible in collecting metrics.   I'm wondering if opentelemetry-cpp is stable and worth it...or maybe there's another way.

[NotATether]

If you are going to hard-code the information-gathering, you are going to end up creating a collection module anyway, so you might be better off using a mature well-tested library rather than rolling your own.

My idea of performance logging is that all of those statoshi metrics don't need to be tracked, all listed as a graph in https://statoshi.info/, just the most important ones such as mempool size. This can be logged with a large interval and then written to disk while bitcoind is poll()ing for a message to come.

Opentelemetry-cpp seems to be flexible in collecting metrics.   I'm wondering if opentelemetry-cpp is stable and worth it...or maybe there's another way.

Whatever you do end up using, it had better lend itself easily to being included in automake/configure builds.