If you compiled the current development version in git, you have a pre-release for 0.8, which uses a completely different database layout compared to older versions. As there is no auto-migration implemented yet, it is expected that it would just start downloading from scratch.
What you can do: move the old blk0001.dat and blk0002.dat to blocks/blk00000.dat and blocks/blk00001.dat, and start bitcoind with -reindex.
Also: don't expect the CPU of a Pi to be fast - the part of the block chain after 193000 (the last checkpoint, after which signature checking is enabled) is dominated by CPU power. It's bound to take many hours on a Pi, running at 100% cpu.
I suspect this is the problem. I had tried a bunch of things and no matter what it would start from scratch. Thanks for the tip.