NOTE: Please see: Armory Bitcoin Client which was forked from this project to be the most advanced Bitcoin client to date!
This project remains as a useful set of pure-python tools. But Armory is a full Bitcoin client!
I am releasing my Python/C++/SWIG code into the wild under the GNU Affero General Public License: PyBtcEngine on Github by etotheipi
if you are in Linux/Ubuntu, you can easily compile and execute (Windows is available too, but it's a lot of work to get it compiling): The PBE block-explorer demo.In a nutshell:
PyBtcEngine is a computational backend that could be used as a starting point for Python-based BTC tools & software. It does not include any
networking code at all, but has a fairly complete set of everything else unrelated to networking. Most library components are heavily unit-tested, so there should be a fair degree of robustness built-in.
This library enables you to read, scan and organize the entire blockchain, perform all ECDSA operations, evaluate most scripts, detect non-std scripts, collect balances and Tx lists for wallets/addresses, detect/handle blockchain reorganizations gracefully, and can even be used to test your blk0001.dat for bit-errors. And it can do all this ridiculously fast! (see timings below)
. In its current state it is perfect for an offline BlockExplorer (in progress), but could easily be expanded into other tools, or used for the backend of an alternative client. See below for a more-complete list of implemented features.
Below, I have copied the "STATUS" section of the README which shows the current capabilities in each language (which are combined in SWIG):
* STATUS: Last Updated - 28 Oct, 2011
* _ not implemented
* . implemented but not tested
* + implemented and partially tested
* X implemented and tested
* C++ Python SWIG
* (01) Ser/Unser Block Objects X X X
* (02) Hash160/Hash256 X X X
* (03) Difficulty calcs X X X
* (04) Address Generation X X
* (05) Address Verify/Manip X X
* (06) BlkHeaders read/scan/org X X X
* (07) BlkHeaders reorgs X X
* (08) Blockchain read/scan/org X X
* (09) Blockchain reorgs X X
* (10) Blockchain verify integrity X X
* (11) NonStd Tx Detection + + +
* (12) Script pprint X X X
* (13) Script OP_CHECKSIG X X
* (14*) Arbitrary script eval X X
* (15) ECDSA Sign/Verify X X X
* (16) Address/Wallet tracking X X X
* (17) Scan blkchain for Tx X X
* (18) Scan blkchain for NonStd X X
* (19) Reorg w/ double-spend X X
* (20) Add new blockdata real-time X X
* (20) SelectCoins for tx X X
* (21) Tx construct given inputs X X
* (22+) Distr Proposals for multi-sig + +
* (23) Tx broadcast
* (24) Tx fee detect/calc/handle X X
* (25) Blockchain download
+ please see https://gist.github.com/1321518
* all scripts have been implemented, most of them are tested.
OP_IF/NOTIF/ELSE/ENDIF are the only codes not implemented yet.
The current implementation holds everything in memory, and so it takes up about 1.2 GB of RAM right now. I plan to improve this in the future, but my computer has 8GB so I'm not in any hurry to make it more lightweight. On the other hand, because of this, and
my painstakingly-careful memory management, the library is extremely fast
. Here's the timings, measured on a single thread of an AMD Phenom X4 840 CPU with 8GB of 1333 MHz DDR3.
- Read entire blockchain into RAM: 5s
- Scan entire blockchain, collect headers/txs: 10s
- Organize and find longest chain: 0.5s
- Verify blkfile integrity: 2.5s
- Get balances/ledger for a set of addresses/wallets, from scratch: ~0.75s/wallet
Yes, you can load, organize and scan all 600 MB of blockchain, and find transactions for a given wallet in less than 20s
. My careful memory management guarantees that there are virtually no extraneous copy operations at any step. As such, some of the code is a bit complicated, but no one can say it isn't fast! Most of the C++ code is documented in the base directory, in the file, Using_PyBtcEngine.README
. There is also a ton of example/unit-testing code that will be critical for anyone wanting to use it. In particular, three files contains examples of nearly every available method:
Recent Updates (08 Dec, 2011)
- (C++) BlockUtilsTest.cpp
- (Python) unittest.py
- (Together) testswig.py
Development has been forked to Armory
which will be used for a client will all sorts of new, innovative features. If you are looking for pure-python tools/code for Bitcoin, keep following PyBtcEngine, but otherwise switch to Armory. I should have finished GUI development and have an alpha client released by the end of the year! (multiple encrypted wallets, address import, watching-only wallets, multi-sig tx, and even more!
)Recent Updates (16 Nov, 2011)
- Added lots of new C++ features: secure binary data handling, AES encryption, ECDSA signing, and time-and-memory-bound key-derivation function! A wallet using this library can now set the target compute time and memory for the key-derivation function to decrypt the private keys (the Satoshi client only has time-bound KDF)
- Added elaborate SelectCoins algorithm, which actually works extremely well! Created a SelectCoins-solution evaluator, and then threw in a few simple, a few elaborate coin selection algorithms. And some random ones. The SelectCoins evaluator gives each of them a sequence of scores, weights the scores by user preferences (with a default), and then chooses the best solution. The idea is to throw in a ton of solutions, each one of which may be better for an initial starting condition. Whichever one is best for the moment will be used
- Started secure wallet format and serialization. Will be using a simple binary format for the wallets, with all data encrypted via AES-256, and using a key-derivation function that nominally takes 0.5s of computation and 8 MB of RAM on the users' computer. The timing calibration makes it difficult to brute-force a solution, and the memory requirement completely disarms GPUs from being able to help with such a search.
- Created BIP 0010 to proactively figure out how clients can deal with multi-signature transactions. The core concept is "Tx Distribution Proposals." As such, I have implemented TxDPs and actually made them the basis for all transaction operations, even for single-signer transactions. If the private key is on your computer, it will be signed and broadcast immediately. If not, it will give you the TxDP which can be signed by the offline computer without needing access to the blockchain. If multiple signatures are required, the TxDPs are easily copied ASCII blocks that to be include inline in emails or as attachments, and easily combined when multiple signatures are received.
I have done my best to make this code "usable," meaning well-formatted code and lots of comments. Unfortunately, Bitcoin is complicated
, and so there's only so much one can do to make the code easier to comprehend. Feel free to offer recommendations for improving it -- but it is a lot of code, so any major refactorings will probably not happen unless you're volunteering.License:
GNU Affero General Public License v3 (AGPL) for this project. The license was picked to allow users to use it for free if they plan to create more OSS, but require a dual-licensing negotiation if someone wants to use it in closed-source software. Please contact me if you're interested.