I think it was suggested before, somewhere.
It seems like hardware is the main moneyburner here.
Why invent it again?
Use some small device with a bit cpu power, a good enough display, battery, usb connection and buttons.
--> MP3 Player <--
So, why are people burning truckloads of money on re-inventing the hardware? It's all in the software anyway!
This is a very good point, and can be part of a very good solution. Going with a rockbox platform, you would get a great development environment (qemu target simulation) and some reference platforms that work well with Rockbox. I can think of some downsides:
1 - Ability to scale, not tied to a manufacturer. It is very important to be free from a certain product lifecycle. If the vendor stops making your reference platform, you get kinda stuck. Designing it yourself gets you 5-10 years of useful life.
2 - Single reference platform. Rockbox looks like it is very platform-independent. It is nice to keep things simple with a single platform, though that isn't a big issue, just pick one and turn away users that have another one (or get them to port it.) Trying to debug dozens of USB HID implementations can be tedious.
3 - Crypto-specific requirements. It is pretty important to have a good hardware random number generator on board, to create a strong private key and to limit some attacks.
4 - Developing hardware is kick ass, and has a reasonably low barrier to entry. Even if you will never get to the quantities that would make your device cheaper than an MP3 device, it is still a great experience and solves all of the above problems. You also get a nice development environment, more security, and can optimize for Bitcoin-specific use cases. There is a strong open-source mentality around Bitcoin, and using open hardware fits better than an off-the-shelf retrofit.
My 2 mBTC. Having Bitcoin wallet software integrated on top of Rockbox would still be sweet.