PBE BlockExplorer: a PyBtcEngine Demo

[etotheipi]

I released the source code for PyBtcEngine the other day in this thread.  It is a highly-efficient, standalone, computational engine for Bitcoin development in Python, using C++/SWIG in the background.  Since then, I have created a powerful block-explorer to demonstrate what the library can do (compare to the same info on blockexplorer.com):


Full size img available here

This GUI is by no means complete, but it is usable and demonstrates the completeness of functionality available in Python with PBE.  You can search by block number, or any hash in any endianness - and there are no temporary/helper files to do this -- everything is loaded from blk0001.dat in less than 20 seconds!  

Recent Updates:

• Tested on Ubuntu 11.04 64-bit, and 10.04 32-bit with both Python 2.6 and 2.7.  Some strange display issues with it in 10.04 in VirtualBox, let me know if you it works for you.
  
• GUI auto-updates when your client receives new block data.
• Still haven't figured out running it on Windows, but I've done most of the hard work -- included MSVS 2010 project with the swig command as a pre-build event and compiles the wrapper.  Will be adding MSVS 2005 project soon.

There's a billion things I'm working on adding, such as address views/searching (implemented in the library, but not part of the GUI, yet), multiple search results, and more detailed information on double-click. Suggestions are welcome, but I have plenty of low-hanging fruit to grab.

Compiling and using this library in Ubuntu is very easy if you have  Python 2.6 or 2.7:

• sudo apt-get install build-essential libcrypto++-dev swig libqtcore4 libqt4-dev python-qt4 python-dev
• git clone git://github.com/etotheipi/PyBtcEngine.git
• cd PyBtcEngine/cppForSwig
• make swig
• cd ../pyqt
• python blockexplore.pyw

If the library didn't autodetect your blockfile, you can supply it as the first/only argument to the blockexplore script, usually ~/.bitcoin/blk0001.dat.  Keep in mind, this is still the full-RAM implementation, so it's going to store everything in RAM while you use it:  about 1GB.  I will be making it more lightweight in the future, but at the moment I'm focusing on implementing new features, and improving access to the existing ones.



PLEASE HELP me figure out how to get this running in Windows.  I have everything compiling and automated in MSVS.  However, I cannot seem to import the module/dll or PyQt4, so the GUI won't load.  I hope someone with more experience can help me figure this out.

To learn more about the library, read "Using_PyBtcEngine.README" in the root project directory, and see my intro thread on PyBtcEngine for more information.

[Mobius]

Slight problem:

g++  -I/usr/include/python2.7  -c -march=native -O2 -pipe  -Icryptopp -fPIC -DUSE_CRYPTOPP -D__STDC_LIMIT_MACROS -Lcryptopp -lcryptopp CppBlockUtils_wrap.cxx
CppBlockUtils_wrap.cxx:149: fatal error: Python.h: No such file or directory
compilation terminated.
make: *** [CppBlockUtils_wrap.o] Error 1

[2112]

CppBlockUtils_wrap.cxx:149: fatal error: Python.h: No such file or directory

Just install the appropriate python-dev for your system.

Edit:

To fix the next little buglet change -lcryptopp to $(LIBRARY_OPTS) in the g++ invocation under swig.

[etotheipi]

Fixed the instructions above to include python-dev.  It's tough to tell what is needed because I already have a lot of these packages installed on my system.

EDIT: I tried to get this working in Python 2.6, but found it's not a trivial change.  For now this will only be compatible with Python 2.7

[2112]

I tried to get this working in Python 2.6, but found it's not a trivial change.  For now this will only be compatible with Python 2.7

So, if I'm on Ubuntu 10.04 LTS on default repositories, is the Python 2.7 available for me without going the compile-your-own route?

I see 1227 packages with the word "python" in the name. As far as I can see there's only python 2.6 and python 3.1 to choose. Am I doing something wrong?

[Remember remember the 5th of November]

What about windows?

[etotheipi]

Sounds like I better work on getting this working in 2.6.  I didn't realize that 2.7 wasn't available in 10.04, and that's definitely a distro I want to support out-of-the-box.  I guess I'm learning about developing on a system that is too new.

As for Windows, I have a comment at the bottom of the original post.  I got swig to run and compiled the wrapper in MSVS.  But I can't seem to load PyQt4 or the SWIG'd module.  I was hoping someone might be able to help me figure out how to make it work.  There's no reason it wouldn't work, if I can just get python to load those things.

[2112]

Sounds like I better work on getting this working in 2.6.  I didn't realize that 2.7 wasn't available in 10.04, and that's definitely a distro I want to support out-of-the-box.

If you don't mind telling me: why you've choosen 2.7? What change in the Python language made your development easier?

I just looked into our build machines for the commercial software that we sell. Python versions were 2.2, 2.3 and 2.4. The software is selling rather well, and nobody asked us about "requiring support of newer version of Python". Our software is to the large extend like yours: finely tuned C++ engine plus an assortment of SWIG condoms I mean wrappers. What is that we are missing?

Do you mind telling me?

[etotheipi]

It wasn't a "choice" so to say, it's just what I have installed on my Ubuntu 11.04 system, and I started developing with it not realizing that some of the features I was using wouldn't be backwards compatible.  Similarly, I just went to the SWIG website and downloaded the latest version, and same with installing PyQt from the 11.04 repos.  I just use what's in front of me.  Perhaps I should be forcing myself to move back a few version of everything...

I'm not a software engineer, I'm a mathematician who writes algorithms for physics simulations, not in the realm of distributed end-user software.  There's a lot for me to learn about this process, but I'll get there eventually.  Until then, I'll have to backtrack a little bit to get this right. 

[2112]

It wasn't a "choice" so to say, it's just what I have installed on my Ubuntu 11.04 system, and I started developing with it not realizing that some of the features I was using wouldn't be backwards compatible.

Thank you very much. I really appreciate your answer.

[etotheipi]

Updated!  I not only added support for 2.6 (I think), but I added auto-updating when your client adds new blockdata to your blk0001.dat file.  

I tested on 10.04 in a VM, and got a strange display issue, not sure if it's real or not.  I'm sure you'll let me know  It's probably another feature I used not realizing it wasn't backwards compatible with non-bleeding-edge versions of PyQt.

Also updated the MSVS 2010 project, which successfully compiles everything and even runs SWIG as a pre-build event.  I still can't get PyQt4 or the resulting module loaded in a CLI session or script.  Perhaps the swig module has to be compiled into something other than a DLL?  I just got MSVS 2005 installed and will be setting up a project using that, soon.