Will dynamic libraries of libconsensus and secp256k1 be distributed ?

[Nicolas Dorier]

My guess is that those libs are statically linked to bitcoin executable.
However, will you distribute dynamic version of them ? if yes, do you have any approximate timeline ?

[1714107526]

1714107526

[1714107526]

1714107526

[1714107526]

1714107526

[goregrind]

I'm not sure I understand exactly what you're looking for, but you can just compile them.

[Nicolas Dorier]

Like I said, dynamic libraries of secp256k1 and libconsensus already compiled and distributed on bitcoin.org. (.dll or .so)
I can still compile but this is not a good solution, it is not obvious to do it on windows, and also people would need to trust my dll.

I want that my library NBitcoin to give the choice to the user of using libconsensus and secp256k1, instead of relying on my .NET implemention of Bitcoin scripting evaluation & signatures implementation.
If the user choose to use libconsensus and secp, I want to download that automatically on the bitcoin.org website.

Sure, I can compile myself and host my own website, but my version would not be as trusted as one compiled by bitcoin dev team.

[CIYAM]

Sounds like a reasonable suggestion to add the .dll/.so targets to the standard make (although I guess you'd want some regression testing stuff to be sure they do actually work).

[btchris]

I'm relatively unknown in the Bitcoin world, but I do distribute bitcoin-related software via GitHub written mostly in Python (and some OpenCL). One of the nice things about scripting languages is they're easier to check for shenanigans, as opposed to binaries.

None of the bitcoin-related software I distribute via GitHub is in binary form; I wouldn't expect anyone to trust me that much. Instead I ask users to install binary components (such as Python) from sources they do trust.

As another example, for EC math that needs to be fast, I ask users to install Armory which comes with some Crypto++ libraries on the assumption that most users trust Armory.

Having "official" dynamic libraries available could be useful, especially for Windows or OS X users where the compilation process isn't particularly easy. (Of course, only when they're actually ready for distribution.)

[Nicolas Dorier]

I'm relatively unknown in the Bitcoin world, but I do distribute bitcoin-related software via GitHub written mostly in Python (and some OpenCL). One of the nice things about scripting languages is they're easier to check for shenanigans, as opposed to binaries.

None of the bitcoin-related software I distribute via GitHub is in binary form; I wouldn't expect anyone to trust me that much. Instead I ask users to install binary components (such as Python) from sources they do trust.

As another example, for EC math that needs to be fast, I ask users to install Armory which comes with some Crypto++ libraries on the assumption that most users trust Armory.

Having "official" dynamic libraries available could be useful, especially for Windows or OS X users where the compilation process isn't particularly easy. (Of course, only when they're actually ready for distribution.)

It is not a matter about scripting versus non scripting. The matter is that the only code that we can 99.9999% be sure to stay in consensus, bug free and performant is libconsensus and secp256kr1.
Other libs can't compete in term of code review, trust and brain power invested into it.

NBitcoin will always evaluate script and verify/make signature by itself by default, for ease of use of the lib, but also for portability matter.
However, I firmly believe that the most sensitive part of the infrastructure of a project should use libconsensus and secp256kr1, and not any other custom implementation, mine included, no matter how much testing and audits are done.

[UPDATE]

Having "official" dynamic libraries available could be useful, especially for Windows or OS X users where the compilation process isn't particularly easy. (Of course, only when they're actually ready for distribution.)

Even if it were easy, any custom build can't compete in terms of trust with one distributed on bitcoin.org, with checksum signed by PGP keys of the dev team.

[gmaxwell]

Eventually.

Libsecp256k1 is not used for Bitcoin consensus currently and is not yet suitable.

[Nicolas Dorier]

Eventually.

Libsecp256k1 is not used for Bitcoin consensus currently and is not yet suitable.

Ok, however, bitcoinconsensus is suitable right now if I understand right, would you distribute that one ?

[doug_armory]

Eventually.

Libsecp256k1 is not used for Bitcoin consensus currently and is not yet suitable.

Ok, however, bitcoinconsensus is suitable right now if I understand right, would you distribute that one ?

IIRC, only a very small portion of the consensus code is actually in the library right now. Quite a bit of work remains to be done to make it robust enough to deploy elsewhere.

[gmaxwell]

IIRC, only a very small portion of the consensus code is actually in the library right now. Quite a bit of work remains to be done to make it robust enough to deploy elsewhere.

Right and the API may well change, etc.   The current progress is just the first step.

[Nicolas Dorier]

IIRC, only a very small portion of the consensus code is actually in the library right now. Quite a bit of work remains to be done to make it robust enough to deploy elsewhere.

Right and the API may well change, etc.   The current progress is just the first step.

Well, whatever might be done for now, I think it is preferable to use it over any custom implementation of bitcoin for sensitive parts of applications, even if the API might change.
An API that break is less damageable than an invalid transaction being accepted.

Well, I'm glad this will eventually distributed in dynamic libs officially, looking forward for that.

[fbueller]

Looking forward to it too. I've been working on a PHP extension to wrap secp256k1 lately, and I'm hoping to work on more as they get developed further.

I too wonder about the best way to have trusted bindings to these libraries. Since bitcoin core doesn't use the latest libsecp256k1, tracking instead of every single commit is probably a good bet. But in terms of using trusted software, I don't imagine the core devs will spend time on my C code, so what's the best thing here?

I suppose porting tests and doing gitian builds are probably a good idea. Is the best bet to ask for a review, and see who will add a signature?

[Nicolas Dorier]

The best would be to get a *.dll file with a checksum published on bitcoin.org.
Frameworks would check that the checksum is right before loading the dll by querying https://bitcoin.org/.

I'll wait until one of them is publicly released before making my bindings, since I don't have time to compile that stuff. (nor would it be acceptable to use my own compiled dll for people untrusting me)

[fbueller]

I actually was surprised when you mentioned it, do you have a link for that page? I know there's one with some information on the wallet, though I can't seem to find it right now either

It'd be interesting to keep a list of verified bindings to libsecp256k1 and others. I wonder is every language relying on a decent library which can only hope to be bug-for-bug compatible, or is it better to encourage people to use bindings to the official libraries?

Diversity in the software that's out there, including nodes, is a good thing! But there's quite a burden on everyone people to keep up to date, and to be 100% compatible with everyone else. Its quite a challenge to meet

[Nicolas Dorier]

I actually was surprised when you mentioned it, do you have a link for that page? I know there's one with some information on the wallet, though I can't seem to find it right now either

It'd be interesting to keep a list of verified bindings to libsecp256k1 and others. I wonder is every language relying on a decent library which can only hope to be bug-for-bug compatible, or is it better to encourage people to use bindings to the official libraries?

Diversity in the software that's out there, including nodes, is a good thing! But there's quite a burden on everyone people to keep up to date, and to be 100% compatible with everyone else. Its quite a challenge to meet

Such page does not exist yet, the published dll of those libs are not yet released.

I wonder is every language relying on a decent library which can only hope to be bug-for-bug compatible, or is it better to encourage people to use bindings to the official libraries?

As always, the users should decide by themselves. What they should keep in mind is that if a custom implementation disagree with the official one, then the custom one will loose.
Their decision though depends on portability, ease of use, "debuggability" and risks for their own use case. So custom implementation will always beat the official one on some of these points.
Which is why, the users of any framework should have a way to decide for themselves if the official implementation is better than the custom one, and switch without rewriting code.