Building Armory on OSX

[Red Emerald]

Okay.  I've added a lot more detail to the gist.  Let me know if I should add more.

https://gist.github.com/4200620

[1714848954]

1714848954

[1714848954]

1714848954

[1714848954]

1714848954

[picobit]

It does not look like making an app is that hard - I made one by accident the other day

While looking for a solution to the printing problem in Mac OS X, I found a small test example showing a similar problem.  The test example consists of three small source files, bulding into a .app !

You can find the files print.pro, print.h and main.cpp here: https://bugreports.qt-project.org/browse/QTBUG-17913
Then you just type "qmake" at the command line, and qmake writes a Makefile.  The Makefile then builds print.app

It should be possible to work out how the Makefile does it, at least if you cheat and look in the print.app directory as well.  The "Application Bundle" (.app) is a directory that MacOS insists on showing like a file.  I don't know if it is the .app suffix or a special file attribute that does the hiding.

[picobit]

Just tried Red Emerald's brew tap.  It works as expected, thanks a lot!

Regarding the trust chain:  I think it would be a good idea if etotheipi publishes the SHA hash of Red Emerald's brew script on his web page.  Then users can verify that script, and as the script contains the SHA of the downloaded tar file, that is automatically verified as well.  The only remaining attack vector is then malicious code put into some of the dependencies, but that would then almost have to be done by somebody outside the bitcoin ecosystem.  I am not sufficiently paranoid to worry about that.

Of course etotheipi or Red Emerald could SHA sign all the dependencies as well, but that would honestly be a PITA since they are updated without warning.

[Red Emerald]

It does not look like making an app is that hard - I made one by accident the other day

While looking for a solution to the printing problem in Mac OS X, I found a small test example showing a similar problem.  The test example consists of three small source files, bulding into a .app !

You can find the files print.pro, print.h and main.cpp here: https://bugreports.qt-project.org/browse/QTBUG-17913
Then you just type "qmake" at the command line, and qmake writes a Makefile.  The Makefile then builds print.app

It should be possible to work out how the Makefile does it, at least if you cheat and look in the print.app directory as well.  The "Application Bundle" (.app) is a directory that MacOS insists on showing like a file.  I don't know if it is the .app suffix or a special file attribute that does the hiding.

Thanks.  I'll play with this later tonight.  It's easy to build a generic .app, but it becomes harder when it has to include python and pyqt.

The .app suffix is just a suffix.  You can name any folder.app and if it has the right structure, it will run.

Just tried Red Emerald's brew tap.  It works as expected, thanks a lot!

Regarding the trust chain:  I think it would be a good idea if etotheipi publishes the SHA hash of Red Emerald's brew script on his web page.  Then users can verify that script, and as the script contains the SHA of the downloaded tar file, that is automatically verified as well.  The only remaining attack vector is then malicious code put into some of the dependencies, but that would then almost have to be done by somebody outside the bitcoin ecosystem.  I am not sufficiently paranoid to worry about that.

Of course etotheipi or Red Emerald could SHA sign all the dependencies as well, but that would honestly be a PITA since they are updated without warning.

Glad it worked for you!

A hash of the formula could work.  That would be easy to verify.  I'm hesitant to write a script to do all of the validation since it seems like if someone is in place to maliciously modify a package, they might also be able to modify the check script.  Of course if it's something as simple as fetching a signed hash from etotheipi's site and making sure it matches the local hash, this should be easy to do in a single line.

Code:

$ brew fetch --deps Wysenynja/bitcoin/armory-qt
Fetching: armory-qt, cryptopp, pcre, swig, sip, libpng, qt, pyqt
==> Downloading https://github.com/etotheipi/BitcoinArmory/archive/v0.85-beta.tar.gz
Already downloaded: /Library/Caches/Homebrew/armory-qt-v0.85-beta.tar.gz
MD5:  762e5d0f5c4816cc1686aecd32c9a2a8
SHA1: 26354bdb5096ea4a9aeccbf1c80e103fb89c72a1
SHA256: 2102ad70e05567de07a985dc53a1b41d373649a8a5e7c82387d2ef4192d5c3fb
==> Downloading http://downloads.sourceforge.net/project/cryptopp/cryptopp/5.6.1/cryptopp561.zip
######################################################################## 100.0%
Downloaded to: /Library/Caches/Homebrew/cryptopp-5.6.1.zip
MD5:  96cbeba0907562b077e26bcffb483828
SHA1: 31dbb456c21f50865218c57b7eaf4c955a222ba1
SHA256: 98e74d8cb17a38033354519ac8ba9c5d98a6dc00bf5d1ec3c533c2e8ec86f268
==> Downloading ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.31.tar.bz2
Already downloaded: /Library/Caches/Homebrew/pcre-8.31.tar.bz2
MD5:  1c9a276af932b5599157f96e945391f0
SHA1: 1132276bb8bea7cb0662210fb812fbef7791621f
SHA256: 5778a02535473c7ee7838ea598c19f451e63cf5eec0bf0307a688301c9078c3c
==> Downloading http://downloads.sourceforge.net/project/swig/swig/swig-2.0.8/swig-2.0.8.tar.gz
Already downloaded: /Library/Caches/Homebrew/swig-2.0.8.tar.gz
MD5:  69f917e870efc0712c06ab53217b28d1
SHA1: 763305da320163903c69c1cdfbf9a942117d9ace
SHA256: aeeefa20bbe1c03bacf23f0af9e7d6193e8b807b7585470615b71b1ee0c8ca4f
==> Downloading http://www.riverbankcomputing.co.uk/hg/sip/archive/4.13.3.tar.gz
Already downloaded: /Library/Caches/Homebrew/sip-4.13.3.tar.gz
MD5:  b5740125c635f796a72c140f2a2e48c5
SHA1: 672f0bd9c13860979ab2a7753b2bf91475a4deeb
SHA256: baedbc8ba8657ded7e8831c35a6af118e50a9d95f9c90c0fa39cc3c392ddaf1a
==> Downloading https://downloads.sf.net/project/machomebrew/Bottles/libpng-1.5.13.mountainlion.bottle.tar.gz
Already downloaded: /Library/Caches/Homebrew/libpng-1.5.13.mountainlion.bottle.tar.gz
MD5:  18dae128d2cb1b7d23b5d08b008d1f8b
SHA1: 382eb757c38a4c326410a42deab8b4a8c22a6c13
SHA256: 88e5015ac3858cdc39431f1ce781c89bf1bffe09963573b8df83125095a65f23
==> Downloading https://downloads.sf.net/project/machomebrew/Bottles/qt-4.8.3.mountainlion.bottle.tar.gz
Already downloaded: /Library/Caches/Homebrew/qt-4.8.3.mountainlion.bottle.tar.gz
MD5:  0878f24da24c8398574a41d86fc6ef14
SHA1: ff488747325cd6771fbf0b79156fd3833ff3af63
SHA256: 47943315ea7cca39075611cdfb35fe6b1ab451436bda099c00c02ca2d291928d
==> Downloading http://downloads.sf.net/project/pyqt/PyQt4/PyQt-4.9.4/PyQt-mac-gpl-4.9.4.tar.gz
Already downloaded: /Library/Caches/Homebrew/pyqt-4.9.4.tar.gz
MD5:  e31dae8ff5294086ed5de6b9a4215b1b
SHA1: 3fe827fed91ec710746fa980f433313dfec2d5fd
SHA256: f7c21007277fb8e8e90f1d55ed96bda8d5f08f1c2b78cf39928416bcdb24b70e

Brew does make this pretty easy.  Etotheipi or I could sign that, but it would change a lot as brew updates VERY frequently.  It also wouldn't catch a malicious formula (although as I said earlier, that is extremely paranoid).

[gweedo]

Bug report:
OS: Mac OSX 10.8.2
Armory Version:       0.85
PyBtcWallet  Version: 1.35

Armory just loses connected and it seems like it is infinite loop, that just keeps printing this out over and over, but it never crashes completely and sometimes after 10mins of this it will reconnect and work with no problem.
 

Code:

ERROR) armoryengine.py:10344 - Waiting for BDM output that didn't come after 10s.
(ERROR) armoryengine.py:10345 - BDM state is currently: BlockchainReady
(ERROR) armoryengine.py:10346 - Called from: armoryengine.py:10471 (30189323)
(ERROR) armoryengine.py:10347 - BDM currently doing: ReadBlkUpdate (30189323)
(ERROR) armoryengine.py:10348 - Direct traceback
  File "/usr/local/share/armory/ArmoryQt.py", line 3629, in <module>
    os._exit(QAPP.exec_())
  File "/usr/local/Cellar/armory-qt/beta/share/armory/qt4reactor.py", line 231, in _iterate
    self.runUntilCurrent()
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/base.py", line 800, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "/usr/local/share/armory/ArmoryQt.py", line 3247, in Heartbeat
    newBlocks = TheBDM.readBlkFileUpdate(wait=True)
  File "/usr/local/Cellar/armory-qt/beta/share/armory/armoryengine.py", line 10471, in readBlkFileUpdate
    return self.waitForOutputIfNecessary(expectOutput, rndID)
  File "/usr/local/Cellar/armory-qt/beta/share/armory/armoryengine.py", line 10349, in waitForOutputIfNecessary
    traceback.print_stack()
(ERROR) armoryengine.py:10350 - Traceback:
Traceback (most recent call last):
  File "/usr/local/Cellar/armory-qt/beta/share/armory/armoryengine.py", line 10340, in waitForOutputIfNecessary
    return self.outputQueue.get(True, MT_WAIT_TIME_SEC)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/Queue.py", line 176, in get
    raise Empty
Empty

[Red Emerald]

Bug report:
OS: Mac OSX 10.8.2
Armory Version:       0.85
PyBtcWallet  Version: 1.35

Armory just loses connected and it seems like it is infinite loop, that just keeps printing this out over and over, but it never crashes completely and sometimes after 10mins of this it will reconnect and work with no problem.
 

Code:

ERROR) armoryengine.py:10344 - Waiting for BDM output that didn't come after 10s.
(ERROR) armoryengine.py:10345 - BDM state is currently: BlockchainReady
(ERROR) armoryengine.py:10346 - Called from: armoryengine.py:10471 (30189323)
(ERROR) armoryengine.py:10347 - BDM currently doing: ReadBlkUpdate (30189323)
(ERROR) armoryengine.py:10348 - Direct traceback
  File "/usr/local/share/armory/ArmoryQt.py", line 3629, in <module>
    os._exit(QAPP.exec_())
  File "/usr/local/Cellar/armory-qt/beta/share/armory/qt4reactor.py", line 231, in _iterate
    self.runUntilCurrent()
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/base.py", line 800, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "/usr/local/share/armory/ArmoryQt.py", line 3247, in Heartbeat
    newBlocks = TheBDM.readBlkFileUpdate(wait=True)
  File "/usr/local/Cellar/armory-qt/beta/share/armory/armoryengine.py", line 10471, in readBlkFileUpdate
    return self.waitForOutputIfNecessary(expectOutput, rndID)
  File "/usr/local/Cellar/armory-qt/beta/share/armory/armoryengine.py", line 10349, in waitForOutputIfNecessary
    traceback.print_stack()
(ERROR) armoryengine.py:10350 - Traceback:
Traceback (most recent call last):
  File "/usr/local/Cellar/armory-qt/beta/share/armory/armoryengine.py", line 10340, in waitForOutputIfNecessary
    return self.outputQueue.get(True, MT_WAIT_TIME_SEC)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/Queue.py", line 176, in get
    raise Empty
Empty

What version of bitcoind?  Was bitcoind doing anything at the time? Was it responsive (to a command like getinfo) while this was happening?

[gweedo]

What version of bitcoind?  Was bitcoind doing anything at the time? Was it responsive (to a command like getinfo) while this was happening?

Bitcoind is 0.7.0

nothing, it could have been getting the lastest block, since both were on 210888 and when it went to 210889 that is when the issue started cause on armory in the corner it would go from "Disconnected" to "Connected >" to "Connected (210889 blocks)" and just keep looping until I quit armory and normally, no force quit.

Yes Bitcoind was responsive and it is still running

[etotheipi]

What version of bitcoind?  Was bitcoind doing anything at the time? Was it responsive (to a command like getinfo) while this was happening?

Bitcoind is 0.7.0

nothing, it could have been getting the lastest block, since both were on 210888 and when it went to 210889 that is when the issue started cause on armory in the corner it would go from "Disconnected" to "Connected >" to "Connected (210889 blocks)" and just keep looping until I quit armory and normally, no force quit.

Yes Bitcoind was responsive and it is still running

We were just having a discussion about this on the main Armory thread.  I'm investigating right now.  Unfortunately, the new multi-threaded design doesn't handle certain errors right now, and run off into a loop.  But so far, this >10s call to readBlkFileUpdate is the only operation I've seen trigger this bad loop.  Hopefully I'll figure it out soon.

[etotheipi]

Red Emerald,

Do fresh OSX users need to install xcode?  That's not in the instructions.

And I wonder if there's a way to avoid having the user manually modify their .bashrc.  I guess a short script could just run all of those commands for them...?  Actually couldn't all of this be bundled into a script?

Sorry I'm being picky, I just see a lot of hurdles for non-techie mac users...

[Red Emerald]

Red Emerald,

Do fresh OSX users need to install xcode?  That's not in the instructions.

And I wonder if there's a way to avoid having the user manually modify their .bashrc.  I guess a short script could just run all of those commands for them...?  Actually couldn't all of this be bundled into a script?

Sorry I'm being picky, I just see a lot of hurdles for non-techie mac users...

Yes, they need to install Xcode for brew to work.  It is in the instructions.  https://gist.github.com/4200620

brew doctor will tell you to install xcode and the command line tools if you need to.

I've made the bashrc stuff easier to do for the non-technical. Someone with an existing bashrc is probably technical enough to know how to do this on their own, but this should work fine for anyone who already has a bashrc.

[Gavin Andresen]

RE: building the .app bundle:

https://github.com/bitcoin/bitcoin/blob/master/contrib/macdeploy/macdeployqtplus
... is python code with lots of useful stuff for building a nice .app (including making a pretty-looking .dmg).
Lots of Qt-specific stuff, too, but it might be a good starting point.

[etotheipi]

RE: building the .app bundle:

https://github.com/bitcoin/bitcoin/blob/master/contrib/macdeploy/macdeployqtplus
... is python code with lots of useful stuff for building a nice .app (including making a pretty-looking .dmg).
Lots of Qt-specific stuff, too, but it might be a good starting point.

Thanks Gavin, that looks great -- albeit big, and might take me a while to figure out what to do with it.  But anything helps!  Especially if it already includes Qt (but I'll need PyQt, too).


@RedEmerald

I just tried the git tag signing, and it worked flawlessly.  I don't know why I didn't do this before...

Code:

git pull
git tag -v "v0.86-beta"

That should confirm that the tag is signed by the "Armory Signing Key."  About to do the 0.86-beta release, with CoinControl and the double-tx bugfix.

[picobit]

Red Emerald:

Are you going to release 0.86 in homebrew, too?  Not that it is hard to compile by myself, but your homebrew formula is easier

[Red Emerald]

Red Emerald:

Are you going to release 0.86 in homebrew, too?  Not that it is hard to compile by myself, but your homebrew formula is easier

Updated! Sorry for the delay.

Code:

brew update
brew doctor
brew fetch --deps armory-qt
brew upgrade armory-qt

Once it's installed, you don't need to prefix armory-qt with "wysenynja/bitcoin/"

A fresh install does tho:

Code:

brew update
brew doctor
brew tap WyseNynja/bitcoin
brew fetch --deps wysenynja/bitcoin/armory-qt
brew install wysenynja/bitcoin/armory-qt

Note that case matters for `tap`, but not for the other commands.

[picobit]

Wow, you are amazingly fast!

I can confirm that it works as expects (i.e. great).

The "brew fetch" command is not really necessary, but I guess that it gives the truly paranoid a chance to check what is being installed.

Thank you very much, indeed!

[bitcoins5411]

Can't we just come up with a DMG file with an installer for all of us who don't like to deal with compilers, etc? Can't we get a mac release that doesn't take so much work to get it up and running?

Is that a crazy request?

[Red Emerald]

Wow, you are amazingly fast!

I can confirm that it works as expects (i.e. great).

Pretty much all it takes is changing the 2 lines in the formula (the download link and the file hash), so it's pretty easy. Glad it worked for you!

The "brew fetch" command is not really necessary, but I guess that it gives the truly paranoid a chance to check what is being installed.

Thank you very much, indeed!

Yeah, it isn't necessary.  I figured I might as well provide the most paranoid level of instructions as the default though.

Can't we just come up with a DMG file with an installer for all of us who don't like to deal with compilers, etc? Can't we get a mac release that doesn't take so much work to get it up and running?

Is that a crazy request?

That's the end goal.  However it is more work than providing a brew formula.  It's been a pain so far to compile custom c extensions and pyqt into a distributable app.

[Red Emerald]

I've updated the tap again!  It's now using git to checkout v0.86.3-beta and then checks to make sure the git tag is valid.

To update your existing install:

Code:

brew update
brew doctor
brew fetch --deps armory-qt
brew upgrade armory-qt

To install fresh:

Code:

brew update
brew doctor
brew tap WyseNynja/bitcoin
brew fetch --deps wysenynja/bitcoin/armory-qt
brew install wysenynja/bitcoin/armory-qt

You will need to have imported etotheipi's key for verifying the tag to work.

RSA key ID 98832223 "Alan C. Reiner (Armory Signing Key) <alan.reiner@gmail.com>"

You can skip the tag checking by adding "--skip-verify" "--without-gpg" to the install/upgrade command

[Red Emerald]

I've added support for the "--devel" flag!  You can use it to easily install the "testing" branch of armory-qt.

I've also setup a thread for my brew tap.  https://bitcointalk.org/index.php?topic=133097.0

[Rampion]

May I ask how far away in time is an Armory OSX bundle?

Thanks