I don't think theres a danger in having an invalid blockchain, the client would just see that the correct one is longer and download it the normal way.
That is incorrect. The blockchain files themselves are never verified by the client because it is assumed that the blocks were checked as they came in. Since the blocks aren't rechecked when you download them in this way, literally anything could be inserted in there and it could be made to appear like everything is fine until the attacker decides to use the hole they made. For most people, it is just fine if you download these files from a trusted member. If you sell much stuff or are a miner, though, it is HIGHLY RECOMMENDED that you get the blockchain using the built-in download and verification system.