libBitCoin, a good idea?

[hsoft]

I find the idea of BitCoin interesting, and I looked into its source code a little bit. I find it a bit monolithic and I thought it would be a good idea to extract the non-ui (and by that, I mean every UI, including RPC and command line) code and create a new library out of it.

This way, it would be easier to create alternative clients as well as bindings to other languages (Python!!!).

As anyone thought of it yet? Is there a particular reason why this would be doomed to failure?

[1714749422]

1714749422

[RHorning]

I find the idea of BitCoin interesting, and I looked into its source code a little bit. I find it a bit monolithic and I thought it would be a good idea to extract the non-ui (and by that, I mean every UI, including RPC and command line) code and create a new library out of it.

This way, it would be easier to create alternative clients as well as bindings to other languages (Python!!!).

As anyone thought of it yet? Is there a particular reason why this would be doomed to failure?

There is the Bitcoind software that can be used as a library with an API already implemented if you choose to perform actions on the Bitcoin network but don't want to re-implement the whole protocol.  For about 80% of the applications I can think of, that would be more than sufficient.

If you are trying to tweak performance, yes a library of functions that allow you to connect to the various parts of the Bitcoin network would be nice as well.  When are you going to write it?

[hsoft]

Isn't bitcoind the exact same software, but with the "-server" flag?

What I mean to do with a libBitCoin is to make it easier to create bindings for it as well as alternate clients (instead of the wxWidgets one).

This could spur innovation within the BitCoin project (new stuff is easier to try in dynamic languages like Python). An example could be django bitcoin integration, or stuff like that. The last time I checked the RPC API, it seemed rather limited to me.

Moreover, implementing RPC stuff is much easier to do in Python than in C++. That's one of the specific things I was thinking about.

[RHorning]

Isn't bitcoind the exact same software, but with the "-server" flag?

Bitcoind is compiled from the same source code as the main Bitcoin with a user-interface, but it is two different packages created from two different makefiles.  As far as network connections are concerned, it is the same software where they both look and behave identically.

What you may want to look at is this page:

http://www.bitcoin.org/wiki/doku.php?id=api

Almost every function from examining the structure of the blocks & transactions to sending & receiving coins is there.

The primary advantages I can see to perhaps creating a new library is to permit some hooks that allow changes to block inclusion policies for miners, different fee schedules, or tweaking some of the network protocol to include new messages or "services".  Some of that may not even be supported by a library but rather simply re-implementing the protocol on a more basic level in the network itself.

It depends largely on what it is that you are trying to do, but I am suggesting that if you don't want to put the effort into writing the library yourself or to re-implement the whole protocol, that using the JSON RPC library is a good way to get much of what you may want to accomplish.  It is deliberately set up in such a way that you can access the information about Bitcoins without having to provide all of the features that a full re-implementation would force you into.

Mind you, I'm not even dissing the concept of perhaps a better connected library like libpng which allows you to render images of that graphic format.  What I am saying is that the library does exist after a fashion and if there was something specific to add to the already existing API might be a better route to take unless you are doing something on a very low level related to the network itself.  Certainly you could largely re-create a custom user interface for a specific application using the JSON API going through Bitcoind as it is currently implemented.

[brocktice]

It's very, very easy to hook up to the JSON-RPC interface of the daemon with python. Unfortunately it's lacking a few key features like a full, official listtransactions command, so it would make for a rather stunted GUI.

[RHorning]

It's very, very easy to hook up to the JSON-RPC interface of the daemon with python. Unfortunately it's lacking a few key features like a full, official listtransactions command, so it would make for a rather stunted GUI.

Let's put that into the API.  It would be by far and away easier to add a single command that merely exposes something the software is already doing than it is to create a whole new library + API from scratch.  If you have a specific request, I'm sure those involved with active development of the main Bitcoin software would at least consider the idea and might put it into a future update, particularly if you can show it is a pretty trivial request.

What other features would be needed with the JSON API that are currently missing?  Even building a "wish list" of requested features would be useful, and I don't think it would be particularly long if it involved simply exposing things already sent around on the network like transmitting data of an existing block or transactions from that block.  It is being built right now on an "as needed" basis, so what is needed?

[jgarzik]

It's very, very easy to hook up to the JSON-RPC interface of the daemon with python. Unfortunately it's lacking a few key features like a full, official listtransactions command, so it would make for a rather stunted GUI.

Let's put that into the API.

Full listtransactions has been rejected.  <shrug>  Plenty of people use it, so I'll continue publishing a working patch.

It's tempting to publish a "useful patches" patchkit, to give these useful features to a wider audience.

[hsoft]

One example I've been thinking about is that "pooled mining" thing. The proposed scheme of having a central authority push work to miners seems wrong to me. It should rather be miners that fetch work from the central authority. Doing so with the current code seems more complicated than it should.

Another one is to (I don't even know if it's possible) maybe develop a concept of "BitCoin authority", which would maintain that huge block database, and "lite clients" which would rely on central authorities. Would it be p2p anymore? no, you'd have to trust a central authority (a kind of bank), but because of the p2p protocol, you'd always have the choice to run a full client yourself if you want. But most people would run the "lite client" because the normal client is resource hungry.

Anyway, the whole thing is mainly for good software engineering practices, which increases maintainability and extensibility, which is a good idea in general. Also, the RPC API seems rather hackish to me.

It doesn't seem like that much work (ok, maybe I'm wrong there...), it's just a matter of re-organizing the existing code, which is already there.

[hsoft]

Well well, I gave it a try today (I want to learn C++, so I thought it would be a good opportunity). The (incomplete, of course) result is at:

http://bitbucket.org/hsoft/libbitcoin

I also cloned the bitcoin repository and changed it so that it uses libBitCoin:

http://bitbucket.org/hsoft/bitcoin-reorg

The more I play with that code, the more I think some restructuring would be healthy. I don't know much about C++, but is it normal that this much code end up in header files instead of cpp files?

[hsoft]

For the record: I failed.

This code is way too coupled (with the UI as well as everything else) to extract a libBitCoin from it without deep knowledge of it.

I might try again, but then I'd start by cleaning the code from within the project, and only after that, try to extract a library out of it.

Am I the only one worrying about the structure of that code?

[davout]

No, you're not the only one.

Bitcoin would greatly benefit from getting included in the official debian repos.
There is currently discussion going on, on the debian bugtracker, see http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=578157

The bitcoin code is also tightly coupled with wxwidgets too, and that's not a really good thing for inclusion in debian..

You should fork gavins git repo and build up from there, you'll get the svn commits he regularly pulls into his fork along with a couple of very useful patches.

I would love to see a headless client making it to the debian repo and I'll support it by donating (I'm a failure at C++).

[hsoft]

Well, I started working on my alternative approach which is to clean bitcoin's code first. I'm certainly learning a lot about C++ and its bad practices . The repo is at:

http://bitbucket.org/hsoft/bitcoin-clean

I wrote how I plan to proceed with the the cleaning in this file:

http://bitbucket.org/hsoft/bitcoin-clean/src/tip/cleaning_process.txt

[davout]

I just read your description of the cleaning process.

I think that instead of decoupling wxw from the code you should drop it altogether and just make it easy (JSON-RPC is easy ) to implement one as an extra layer.

[hsoft]

I just read your description of the cleaning process.

I think that instead of decoupling wxw from the code you should drop it altogether and just make it easy (JSON-RPC is easy ) to implement one as an extra layer.

Of course, that was the whole point of my "libBitCoin" initiative. But because I failed at that, what I want to do is decouple wx, one step at a time, and once this is done, extract libBitCoin out of bitcoin.

[genjix]

Just wanted to point out my project here (libbitcoin): https://bitcointalk.org/index.php?topic=30646.0

Making great progress and it's functional. There's a working application in examples/poller.cpp that simply downloads the bitcoin blocks + transactions + everything into postgresql so you can inspect it. Someone could easily throw a web gui around this and...

voila! blockexplorer clone

https://gitorious.org/libbitcoin/libbitcoin/blobs/master/examples/poller.cpp

[davout]

Just wanted to point out my project here (libbitcoin): https://bitcointalk.org/index.php?topic=30646.0

Making great progress and it's functional. There's a working application in examples/poller.cpp that simply downloads the bitcoin blocks + transactions + everything into postgresql so you can inspect it. Someone could easily throw a web gui around this and...

voila! blockexplorer clone

https://gitorious.org/libbitcoin/libbitcoin/blobs/master/examples/poller.cpp

sweet!

[Mike Hearn]

I don't think it's a good idea to encourage distributions like Debian to package Bitcoin. There have been too many cases where people need to upgrade and Linux distributions are not exactly famous for keeping up to date with the latest releases. The last thing Bitcoin needs is big chunks of the network running ancient versions because that's what apt-get gives you. Let Linux users download the binaries from the web site instead.

[zellfaze]

With apt you can add your own repos.  Perhaps rather than trying to get into the official debian repo, we could just make our own repo for Bitcoin like Tor has done (http://mirror.noreply.org/pub/tor/).

Anyhow, I think that libbitcoin is a great idea, but not exactly a project for someone new to C++.

[error]

I don't think it's a good idea to encourage distributions like Debian to package Bitcoin. There have been too many cases where people need to upgrade and Linux distributions are not exactly famous for keeping up to date with the latest releases. The last thing Bitcoin needs is big chunks of the network running ancient versions because that's what apt-get gives you. Let Linux users download the binaries from the web site instead.

This depends on two things, the distribution packaging policies and the package maintainer. Mainly the package maintainer, though if you're using Debian, good luck finding a current version of anything without permanently staying on unstable.

[ShadowOfHarbringer]

I don't think it's a good idea to encourage distributions like Debian to package Bitcoin. There have been too many cases where people need to upgrade and Linux distributions are not exactly famous for keeping up to date with the latest releases. The last thing Bitcoin needs is big chunks of the network running ancient versions because that's what apt-get gives you. Let Linux users download the binaries from the web site instead.

if you're using Debian, good luck finding a current version of anything without permanently staying on unstable.

Yep, that is it.
We should place Bitcoin in repos of distros like Gentoo, Arch, Ubuntu, or Debian Sid (unstable, "fresh" debian), which have newer versions of packets updated more frequently.

But this policy won't work with enterprise-class distros like Stable debian, CentoOS/Redhat etc.