This is a question I asked myself recently as well. And refering to open source code doesn't completely solve the question, because the main attack vector I see, if a hardware wallet manufacturer would want to access customers funds, would be a bad random number generator.
I don't know how trezor and ledger generate their privkeys, though. Does anybody have any insight in that?
What we really need is open source hardware. Devices that people can assemble themselves. Unfortunately the skill required to do something like this would be much higher than the average cryptocurrency enthusiast.
I was actually just yesterday thinking about a different approach: roll the dice.
Make a spreadsheet with all the words used for the seed. Align them in a way that you can divide them in a way that makes sense (see below).
Get some dice, for example two W10 dice (two w10 dice can be used to roll numbers between 00-99), then create your seed by rolling the dice until you have 24 words. Put them into your hardware wallet --> totally trustless, ultra paranoid DIY random number generator. Now, at least the seed is generated safely.
Addendum: I haven't found the word list yet, but I assume it is not divisible by 10 or 100. With that in mind: to keep the chances of hitting a word as even as possible, it may be prudent to use different dice (Your local D&D dealer can help you out).
Speaking of assembling themselves: I recently thought about getting an old smartphone, physically disconnect anything that can broadcast and flash a custom ROM on it, which functions as a wallet. Input and output will be handled via camera and QR-Codes (Seed creation, see above
).
