Building Armory on OSX

[etotheipi]

There seems to be endless difficulty getting Armory compiled and running properly on Mac/OSX.  Some versions of OSX/Homebrew/Xcode seem to work fine.  Others keep putting up a fight.   I am starting a new thread, because the original Armory thread is getting diluted with OSX discussion.  I definitely want to see this discussed, but I want to reserve the Armory thread for OS-independent discussion and features, and let the OSX crowd discuss, here, how to get it working on all relevant versionsof relevant software.    

The current Build-OSX-from-source-code instructions (appears to work in Lion)
Red Emerald Discusses updates he made to the build instructions for OSX 10.7.3 and Xcode 4.0.3

I have listed a couple questions below to seed this discussion, mainly because I have never even touched an OSX machine, much less done any development on one.  I need some serious education on this!

• Most important question:  How many different binaries do I have to produce to support 95%+ of Mac users?  It sounds like I may not be able to compile just one binary... but maybe I can?
• What is the absolute most-recent version of OSX?  What versions correspond to different "generations" of Macs/OSX?  What is the earliest version I should be supporting?
• It sounds like compilation is going to be based on not only OSX version, but homebrew and xcode.  How much different is this software between versions of homebrew and xcode?  I am hoping it's just like gcc: the version doesn't really matter as long as you have it.
• Someone is offering me a Macbook with OSX 10.3.9 -- is that going to be sufficient for all development activities (for the near future, at least)?
• Anyone have fail-proof instructions for getting an OSX VM working?  I've tried three times and failed.  I'd much prefer to get VM running than get new hardware.

Btw, if you missed it in the main thread, I modified the repo so that Crypto++ is statically compiled into the _CppBlockUtils.so file.  This not only resolves a plethora of linking issues on OSX, but also Linux.  Crypto++ source was already included in the project anyway, so I might as well.  It increases build time significantly, but only the very first time you "make".

[1715410941]

1715410941

[1715410941]

1715410941

[1715410941]

1715410941

[1715410941]

1715410941

[torusJKL]

The current Build-OSX-from-source-code instructions (appears to work in Lion)

I use 10.6.8 which is Snow Leopard.
Red Emerald (second link) is using Lion (10.7.3).

[Red Emerald]

Figured I would respond to this here.

yes, consider it another donation.  Powerbook G4, OS 10.3.9.  other specs i don't recall offhand.  let me know.

Can you build universal binaries with it? Meaning PPC and Intel.

sorry, i wouldn't know.

I don't believe you can.  The Powerbooks are PowerPC, so I don't think they can compile for Intel. I'm also pretty sure that it is 32-bit.

The system I was talking about letting you borrow would be running Lion and could be setup with SSH and VNC.  If I can get a snow leopard VM up, I can get you remote access to that too.

I'll keep playing with py2app.  I'm reading through the docs and there is a "recipe" for sip but I have no idea to get it working.  I think that is the only problem left.  I'm using py2app with symlinks right now just fine.

The only problem is there are no images.  Its just a grey empty area where the logo should be.  I can submit a screenshot if you want.  It may just be a side effect of the dev py2app and so I wouldn't worry about it yet.

Also, I don't like having to replace the system links to python 2.7 with homebrew's.  I feel like that shouldn't be necessary.  I think DYLD_LIBRARY_PATH just needs to be set properly.  I'll experiment more after work.

[etotheipi]

The only problem is there are no images.  Its just a grey empty area where the logo should be.  I can submit a screenshot if you want.  It may just be a side effect of the dev py2app and so I wouldn't worry about it yet.

This is actually just a problem with the way I setup images in Armory.  They all look for the "./img/something.png" which doesn't exist unless you happen to have the executable in the same directory as the original build.  In Windows, I solved this by just copying the img directory into the "ArmoryStandalone" output directory and packaged it along with all the .dll's.  In the future, I think I will use qrcresources to try to embed all of them in a python file, I just haven't gotten around to figuring out how to use it yet.

[splatster]

I'm taking the plunge again.  I'm going to try Red Emerald's OS X Lion instructions and see where that gets me.

EDIT:
It looks like running make swig didn't compile CppBlockUtils.  I followed Red Emerald's instructions exactly (I think).  Maybe I missed a step?

Code:

ImportError: No module named CppBlockUtils

[etotheipi]

I'm taking the plunge again.  I'm going to try Red Emerald's OS X Lion instructions and see where that gets me.

EDIT:
It looks like running make swig didn't compile CppBlockUtils.  I followed Red Emerald's instructions exactly (I think).  Maybe I missed a step?

Code:

ImportError: No module named CppBlockUtils

What did "make swig" do?  It should've dumped a _CppBlockUtils.so/.dylib file into the base directory, along with CppBlockUtils.py.  If those are there, then it's probably an import error instead of a can't-find error.  Open a python shell and type "import CppBlockUtils".  It will tell you more-specifically what the error is. 

Also, make sure you've updated to the latest commit on master.  There cannot be any more issues with Crypto++, anymore.

[Red Emerald]

The only problem is there are no images.  Its just a grey empty area where the logo should be.  I can submit a screenshot if you want.  It may just be a side effect of the dev py2app and so I wouldn't worry about it yet.

This is fixed now! Thanks.

Also, I don't like having to replace the system links to python 2.7 with homebrew's.  I feel like that shouldn't be necessary.  I think DYLD_LIBRARY_PATH just needs to be set properly.  I'll experiment more after work.

I figured this out!

I added this to my .bashrc

Code:

export DYLD_LIBRARY_PATH=`brew --prefix`/lib:$DYLD_LIBRARY_PATH

and it means you don't have to change any system files (which is the whole point of brew and a good idea on a mac in general).

So the revised instructions are:

Code:

Prerequisites:
Mac OS 10.7.3
Xcode 4.0.3
Homebrew (http://mxcl.github.com/homebrew/)

Install homebrew packages:
brew update
brew install ccache # optional. make sure to set up your path for ccache by following homebrew's instructions
brew install python cryptopp swig sip qt pyqt berkeley-db

Setup environment so it can find the brew libraries instead of the system ones:
export DYLD_LIBRARY_PATH=`brew --prefix`/lib:$DYLD_LIBRARY_PATH
export BERKELEYDB_INCDIR=`brew --prefix berkeley-db`/include
export PYTHONPATH=`brew --prefix`/lib/python2.7/site-packages:$PYTHONPATH
export CFLAGS=”-arch x86_64″
export ARCHFLAGS=”-arch x86_64″

Install twisted:
`brew --prefix`/share/python/easy_install-2.7 twisted

Install bsddb3:
`brew --prefix`/share/python/easy_install-2.7 bsddb3

Clone Armory source code:
cd ~
git clone git://github.com/etotheipi/BitcoinArmory.git
cd ~/BitcoinArmory

Make SWIG Modules
cd ~/BitcoinArmory/cppForSwig
make swig

Run Armory:
cd ~/BitcoinArmory
`brew --prefix`/bin/python ArmoryQt.py

Also, be careful that the '--' stay '--' and don't become '–'

Now I'm running a self-compiled bitcoind and the latest git head (77bbbfb) of Bitcoin Armory. I'll let you know if I find any bugs  It does seem to lock up every now and then. I'll be in the middle of filling in a description and it just hangs for a sec.  I think it might be triggered when a block is found, but it seems to happen more often than that.  It does it more than my windows system does (Actually, I haven't noticed this happening in windows). Although that system is much more powerful.

homebrew install berkeley db 5 by default (and that's what I have done).  Version 4 is also available (I grabbed it to make bitcoind happy). Does it matter if Armory uses version 5 since it is just reading the database and not writing to it?

Soon, I'll (hopefully) have instructions for bundling Armory into a ArmoryQt.app.  For now, I'm just using the symlinked build.  It's nice to just click an icon in the dock and have it open rather than running a script.

I'm thinking about writing a script to start bitcoind if it isn't already running before opening Armory.  Although this won't be needed once Armory has the networking code built in.  That day scares me a little by the way.  So much code to re-implement that bugs are inevitable.  It's nice that you can build off bitcoind for now.


EDIT: Also, I added some stuff to the .gitignore to make git happier after 'make swig' is run.  I submitted it as a pull request.

I was thinking the 'how to build from source' might be nice to have in the actual source.  Do you think that is a good idea?

[torusJKL]

I still need to create a symlink for /usr/include/python2.7.
Because the Makefile of swig checks the path and if it does not exist it will use python 2.6.

Other then that I took some of your ideas and updated my instructions:
Building Bitcoin Armory on Mac OS X 10.6.x using Homebrew

[Red Emerald]

I still need to create a symlink for /usr/include/python2.7.
Because the Makefile of swig checks the path and if it does not exist it will use python 2.6.

Other then that I took some of your ideas and updated my instructions:
Building Bitcoin Armory on Mac OS X 10.6.x using Homebrew

Lion comes with 2.7, so that must be why I don't need to do that.  It is probably better to edit the Makefile than mess with the system python.  I know bitcoin comes with multiple makefiles.  I'm just about to submit a pull request that should make the Makefile a little smarter on mac.

Are you sure you still need to set BERKELEYDB_INCDIR even with DYLD_LIBRARY_PATH set? EDIT: looks like this is still needed.  Also, `brew doctor` gives warnings about having DYLD_LIBRARY_PATH set, but I'm not sure what the proper thing to do is then.  If its a bad idea to change system files and a bad idea to change the library path, what is left to do?!  I miss compiling in linux lol

Also, my python path is way shorter than yours.  You put way more than brew says is necessary when it installs python (at least in Lion).  For me brew, says that PYTHONPATH is only needed "For non-homebrew Python"

==> Caveats
For non-homebrew Python, you need to amend your PYTHONPATH like so:
  export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH

I also noticed that when I installed qt, it installed in the time it took download.  Now instead of compiling the whole thing, brew downloads a "bottle." It really cuts down the install time.  Looks like boost does the same thing.

[Red Emerald]

Bah. I can't get easy_install-2.7 bsddb3 to work on my second system

Not sure what is going on.

Code:

$ `brew --prefix`/share/python/easy_install-2.7 bsddb3
Searching for bsddb3
Reading http://pypi.python.org/simple/bsddb3/
Reading http://www.jcea.es/programacion/pybsddb.htm
Reading http://www.argo.es/~jcea/programacion/pybsddb.htm
Best match: bsddb3 5.3.0
Downloading http://pypi.python.org/packages/source/b/bsddb3/bsddb3-5.3.0.zip#md5=305c3d42fe598b56349161497eb9bfc4
Processing bsddb3-5.3.0.zip
Running bsddb3-5.3.0/setup.py -q bdist_egg --dist-dir /var/folders/hb/00j65cb90w7ctg_k8t0sjf040000gn/T/easy_install-WXyA5J/bsddb3-5.3.0/egg-dist-tmp-h9gkqO
warning: no files found matching 'updatedb.bat'
warning: no files found matching '*.py' under directory 'bsddb3'
warning: no files found matching '*.gif' under directory 'docs'
warning: no files found matching '*.jpg' under directory 'docs'
warning: no files found matching '*.txt' under directory 'patches'
warning: no files found matching '*.patch' under directory 'patches'
warning: no files found matching '*.py' under directory 'test'
no previously-included directories found matching 'docs/build'
no previously-included directories found matching 'docs/tools/docutils'
no previously-included directories found matching 'docs/tools/sphinx'
no previously-included directories found matching 'docs/tools/pygments'
no previously-included directories found matching 'old'
no previously-included directories found matching 'test/db_home'
no previously-included directories found matching 'test/bsddb3'
In file included from /usr/local/Cellar/python/2.7.2/include/python2.7/Python.h:126,
                 from Modules/_bsddb.c:97:
/usr/local/Cellar/python/2.7.2/include/python2.7/modsupport.h:27: warning: ‘PyArg_ParseTuple’ is an unrecognized format function type
ld: library not found for -ldb
collect2: ld returned 1 exit status
error: Setup script exited with error: command '/usr/bin/llvm-gcc' failed with exit status 1

[torusJKL]

0.70-beta builds fine with my instructions on OS X 10.6.8.
I have added checkout mmap to the instructions.

[etotheipi]

0.70-beta builds fine with my instructions on OS X 10.6.8.
I have added checkout mmap to the instructions.

I wouldn't change those directions just yet.  The intention was to get 0.70-beta-alpha out there as a testing version (hence a diff branch), and when it's stable I'll merge it back into master.  So far, it sounds like it's fairly stable...

Btw, you mention having to symlink to /usr/bin/python2.7... I'm not good at all with makefiles, so I just did whatever I could that would work.   If anyone has recommendations for improvement (other than cmake), I'm open to it.  Or maybe I should just finish the cmake stuff...

[torusJKL]

Btw, you mention having to symlink to /usr/bin/python2.7... I'm not good at all with makefiles, so I just did whatever I could that would work.   If anyone has recommendations for improvement (other than cmake), I'm open to it.  Or maybe I should just finish the cmake stuff...

I'm not a Makefile expert either.
How about doing something like this

Code:

[...]
SWIG_OPTS    += -c++ -python -classic
PYTHON_PATH := $(shell which python)

ifneq (exists, $(shell [ -d PYTHON_PATH/python2.7 ]  && echo exists ))
   SWIG_INC     = -IPYTHON_PATH/python2.6
   LIBRARY_OPTS = -lpthread -lpython2.6
   ifneq (exists, $(shell [ -d PYTHON_PATH/python2.6 ]  && echo exists ))
      SWIG_INC     = -IPYTHON_PATH/python2.5
      LIBRARY_OPTS = -lpthread -lpython2.5
   endif
endif
[...]

PYTHON_PATH will contain the first path it finds.

[Red Emerald]

0.70-beta builds fine with my instructions on OS X 10.6.8.
I have added checkout mmap to the instructions.

I wouldn't change those directions just yet.  The intention was to get 0.70-beta-alpha out there as a testing version (hence a diff branch), and when it's stable I'll merge it back into master.  So far, it sounds like it's fairly stable...

Btw, you mention having to symlink to /usr/bin/python2.7... I'm not good at all with makefiles, so I just did whatever I could that would work.   If anyone has recommendations for improvement (other than cmake), I'm open to it.  Or maybe I should just finish the cmake stuff...

Using 'which python' will not work.

I made a pull request about this a few days ago.  I think cmake would be the proper way to do it though.

https://github.com/etotheipi/BitcoinArmory/pull/3

I got armory compiled on one system, but my other system is giving me trouble and I thought they were identical.  I'll likely have some updates for the Lion build instructions soon.

[etotheipi]

There was an experimental cmake system put together by "Swift Geek" for Armory.   I never got around to finishing the implementation of it.  However, I did get it integrated into the "cmake" branch on github, so all that work is sitting there, idle.  If someone wanted to check out that branch, and try to merge it into master, I might be inspired to pull it (I've been avoiding pulling in any external changes though, to avoid copyright mixing...)

Especially, if the stuff is basically plug-n-play (with some modifications to accommodate new C++ classes since 5 months ago), then I can probably do it myself.  I just don't have time to go wandering down that path right now, at least not until I know how long that path is...

[Otoh]

following & looking forward to running Armory one day on my Mac OS X (Server) Version 10.7.3 (Lion) = latest stable release

[Red Emerald]

Well I can get Armory compiled on my Lion laptop, but no luck on my workstation.

However, it uses pretty much 100% CPU always and takes awhile to do anything.  Since this is handling my money, I'd rather run Armory somewhere I know I can compile it consistently.  I'm installing it in an xubuntu 10.04 desktop VirtualBox VM right now.  Much simpler steps.

Once the blockchain finishes downloading, I'll test it out and compare CPU usage. 

[Haplo]

Ok I just spent the last !%#@#$%@#$@ 24 hours trying to compile (technically electrum, not that there's a huge difference) on my Tiger/PPC laptop. My conclusion is that it's most likely not possible, although technically not directly the fault of Armory or Electrum in particular, just that Qt is a POS that deserves never to be used ever again .

In particular, after trying to @#%$@# compile Qt like 3 times now, it has failed Error 1 and Error 2 (after an excruciatingly long config and make process), no matter which way I configure it. At first I got some unrelated problems because it didn't like being dropped in a folder with a space in the name, but even after fixing that, using -universal and -sdk it gave me the same errors I got when it screwed up for other reasons (after it failed the first time I had to completely scrap the source folder because it decided to shuffle the makefiles all kinds of wrong and then make -confclean didn't work at all either due to the pathname issue).

I might try one more time, using the "platform agnostic" version and -arch instead of -universal and -sdk, sometime after I'm done being angry. If I can just get Qt to install (fuck you nokia) then I can probably get electrum and armory running no problem.

On an unrelated note, I think homebrew is probably responsible for a lot of the inconsistencies people are seeing. I never was fond of MacPorts, and I had nothing but trouble trying to install homebrew (gave up on that). What I did find is that the python site (and others) offer up-to-date versions that install properly (and easily) and you don't have to mess with symlinks (which occur in ~/.bash_profile, at least on my box) or anything like that. Python 2.7.x works great and SIP was no problem to install. I don't expect any problems with Cryptopp, SWIG, or BDB, as everything else compiles fine, but QT is highly dependent on OS version (on mac anyway) and PyQT won't install without it.

[Red Emerald]

Ok I just spent the last !%#@#$%@#$@ 24 hours trying to compile (technically electrum, not that there's a huge difference) on my Tiger/PPC laptop. My conclusion is that it's most likely not possible, although technically not directly the fault of Armory or Electrum in particular, just that Qt is a POS that deserves never to be used ever again .

In particular, after trying to @#%$@# compile Qt like 3 times now, it has failed Error 1 and Error 2 (after an excruciatingly long config and make process), no matter which way I configure it. At first I got some unrelated problems because it didn't like being dropped in a folder with a space in the name, but even after fixing that, using -universal and -sdk it gave me the same errors I got when it screwed up for other reasons (after it failed the first time I had to completely scrap the source folder because it decided to shuffle the makefiles all kinds of wrong and then make -confclean didn't work at all either due to the pathname issue).

You should try using precompiled binaries (if they are available for an old system).

On an unrelated note, I think homebrew is probably responsible for a lot of the inconsistencies people are seeing. I never was fond of MacPorts, and I had nothing but trouble trying to install homebrew (gave up on that).

It's a one-liner to install homebrew... What problem did you have?

[Haplo]

You should try using precompiled binaries (if they are available for an old system).

They're not. I already looked. The only Qt/PyQt binaries available on Tiger are way out of date, and I know at least Electrum requires Qt4.x/PyQt4.x.

It's a one-liner to install homebrew... What problem did you have?

The fact that the PPC fork and the Tiger fork are separate. I probably screwed it up somehow, but I couldn't get it to work at all. Although homebrew seems to have installed (or something), if I type, for example
brew install cryptopp

what I get is
/usr/local/Library/Homebrew/formula.rb:554: warning: already initialized constant CHECKSUM_TYPES
/usr/local/Library/Homebrew/global.rb:80: uninitialized constant RUBY_PATCHLEVEL (NameError)
        from /usr/local/bin/brew:10:in `require'
        from /usr/local/bin/brew:10