Questions about data files/handling in bitcoind.

[BkkCoins]

First time - I'm just looking over the bitcoin source code now and trying to figure out how to add something or even if I need to add it. I'm trying to understand about the data handling as I see now there are a few different cpp files related to this.

What I see is below and my guess at what it relates to (wrt data storage)

db.cpp - appears to be older Berkeley DB code for wallet, logging, peers
leveldb.cpp - seems like interface /support for leveldb
txdb.cpp - looks like a transaction info db using leveldb (keyed by txid?)
main.cpp - appears to write accepted blocks to a simple sequential file (and also some undo data)

I don't see a data file that stores address history, or transactions indexed by address for quick retrieval by address unless I'm mistaken.

So I've been thinking of adding some code to store this data as blocks are "Accepted", enabling quick access to transaction info by address.

Am I missing something? Anyone have relevant comments. Thx.

[flatfly]

I'm not familar enough with the codebase, but in the below post, sipa mentioned he wrote a 10-line patch that could be a great starting point for you want. Not sure if he released it in the meantime or not.

https://bitcointalk.org/index.php?topic=107172.msg1333777#msg1333777

[BkkCoins]

I expect it's not going to be too hard to store the transactions indexed by address. I may be wrong there. Maybe leveldb can be used again and keyed by address?

The idea I have is to make it much easier to run an Electrum server. Right now you need to install Abe and update it fully, and have a sql server running. But with transaction info keyed by address in the bitcoind, and an rpc call to retrieve that info, a simpler Electrum server could run with just bitcoind alone and be much simpler to setup and run.

[flatfly]

I expect it's not going to be too hard to store the transactions indexed by address. I may be wrong there. Maybe leveldb can be used again and keyed by address?

The idea I have is to make it much easier to run an Electrum server. Right now you need to install Abe and update it fully, and have a sql server running. But with transaction info keyed by address in the bitcoind, and an rpc call to retrieve that info, a simpler Electrum server could run with just bitcoind alone and be much simpler to setup and run.

Hmm I'm almost sure this has been done already (very recently)! Not sure where this is documented, but I heard it on IRC - I think the 8 current pruning servers are running Abe-free already.


EDIT: I might be partly wrong - can anyone confirm?


See https://github.com/spesmilo/electrum-server/blob/master/README.leveldb

[jgarzik]

I don't see a data file that stores address history, or transactions indexed by address for quick retrieval by address unless I'm mistaken.

That is mostly correct.  If the address is in your wallet, all transactions related to that are stored in the wallet.

Otherwise, address history is not stored.

I expect it's not going to be too hard to store the transactions indexed by address. I may be wrong there. Maybe leveldb can be used again and keyed by address?

Yes.  leveldb is a generic key/value database.  You may store anything you like in there.

[BkkCoins]

Hmm I'm almost sure this has been done already (very recently)! Not sure where this is documented, but I heard it on IRC - I think the 8 current pruning servers are running Abe-free already.

EDIT: I might be partly wrong - can anyone confirm?

See https://github.com/spesmilo/electrum-server/blob/master/README.leveldb

Oh. That's very interesting and makes my idea redundant. I'll go and try the new version out. I have bitcoin 0.7.1 running on a vps. It looks like I need to update to 0.8.0 from source. I guess the next step would be a ppa that bundles it all together and makes it one-step to install. Maybe I'll look at that instead after.

Thanks for updating me on that.


@jgarzik - Thanks for confirming what I thought about the db code.

[BkkCoins]

I've been digging around in this forum looking for the info to build 0.8.0 of bitcoin. Can't seem to find it. Can someone point me at the thread?

I've git cloned bitcoin but this builds 0.7.1 according to help output. Do I need a different branch or what... or just a makefile option. Hints welcome. Thx.

Also, Electrum server info says I need to install "python-leveldb", but this package is unrecognized on Ubuntu 12.04  so I expect there must be a PPA needed. Maybe that info could be added to the doc page because it's hard to do these builds when  you keep running into problems and have to research around the forum to try to figure out how to just start out.

Thx again.

[jgarzik]

I've been digging around in this forum looking for the info to build 0.8.0 of bitcoin. Can't seem to find it. Can someone point me at the thread?

Ask in this thread: https://bitcointalk.org/index.php?topic=119525.0

I've git cloned bitcoin but this builds 0.7.1 according to help output. Do I need a different branch or what... or just a makefile option. Hints welcome. Thx.

That is normal.  There is no side branch.  The main branch is "master"

The version does not become "0.8" until release day.  My output from debug.log looks like

Code:

12/08/12 06:53:41 Bitcoin version v0.7.1-216-g8588702-dirty-beta (2012-12-04 07:18:04 -0800)

[BkkCoins]

I see. Thank you. (mine appears to be a bit newer 228 instead of 216. No problem.)

For electrum-server devs - I tried applying the patch as per instructed in HOWTO.md but it gave errors. Turns out the patch level needs to be -p1 instead of the -p2 indicated. Either that or I guess remove the "cd bitcoin" as I believe that may work as well.

Anyway I've patched and built bitcoin, set electrum.conf and about to try running.

[BkkCoins]

Info for users on Ubuntu 12.04:

python-leveldb is not available. You need to add a line to /etc/apt/sources.list as below to bring in the backports repository and then apt-get update and then apt-get install python-leveldb. Newer versions of Ubuntu appear to have this package.

This works for me. I suggest update to README.leveldb.

deb http://archive.ubuntu.com/ubuntu precise-backports main universe