I had exactly
the same problem as riz experienced (see this post: https://bitcointalk.org/index.php?topic=117909.0
When I download and install the USB using Unetbootin in windows 7, i picked 12.04 live and installed to a USB stick giving a few gig persistence thinking that can be used for the blockchain files.
I get it installed and on first boot to USB i open a terminal and
sudo add-apt-repository ppa:bitcoin/bitcoin
sudo apt-get update
sudo apt-get install bitcoin-qt
The following packages have unmet dependencies:
bitcoin-qt : Depends: libboost-filesystem1.46.1 (>= 1.46.1-1) but it is not installable
Depends: libboost-system1.46.1 (>= 1.46.1-1) but it is not installable
Depends: libboost-thread1.46.1 (>= 1.46.1-1) but it is not installable
E: Unable to correct problems, you have held broken packages.
I just have this feeling that it is not working with Ubuntu 12.04.2