I have never seen a high quality PSU kill downstream components unless abused or pushed beyond specs. Honestly I haven't ever heard of any ATX PSU damaging components that wasn't related to user error or abuse.
Not all of my ATX PSUs were top-end £1,500 units, certainly not in the early days when I was a complete newbie. My single failure wasn't user error or abuse, unless you consider using a cheap brand 'user error', like some do (I don't want to start any hostility here, BTW, we're more or less along the same lines). The unit was rated at 850W, and a logic board, underclocked CPU, 1G of RAM, and three Sapphire 5770 cards burned it out. It took, interestingly, two of the GPUs but not the logic board or CPU. I suppose the big power draw was the 12V rail to the GPU connectors. No, it wasn't ideal, and no, it wasn't best practices by ANY means. I was learning and this was one of the early 'more than 2 GPU' tests.
However the consumption was below the *rated* spec - even at the wall (so even less after inefficiency losses). Poor quality PSUs bite you in the arse, it's been made clear MANY times here on the forums. I haven't (touch wood) had a PSU fail since.
But it's still a single point of failure, surely you appreciate my concern there?
In our datacenter we have $80,000 servers running 24/7/365 off power supplies (n+1 redundant). Power supplies are designed to safely power a lot more than a couple thousand in FPGAs. Your UPS idea is dubious. What if it is the UPS not the PSU which fails? Plus UPS don't really exist for DC loads at the price point you are looking at. So you are talking about some kind of modding increasing the risk of failure.
Anything can happen but internal PSU failure that also protected downstream components is very remote. If you really feel the need for supplemental protection get:
a) whole house surge arrestor (a real one) <- still won't help in a direct strike hopefully your mains are underground
b) a dedicated circuit and outlet preferably with a locking connector to prevent accidental disconnection
c) line interactive UPS
the goal being to ensure the power into the PSU is perfectly clean. Transients from upstream of the PSU are far more likely to cause damage than a failure of a PSU which results in internal unsafe voltage.
My 'UPS idea' is not 'dubious' - please don't be rude - it's not an 'idea' at all. It's a polite question to people like yourself who know whether there's established technology to 'filter' out upstream 12V DC failures (as per a catastrophic PSU failure), or whether an ATX PSU 'catastrophic failure' couldn't possibly burn out hundreds of FPGAs due to the internal build of ATX PSUs.
I don't know enough about how these PSUs work to know whether a low-probability (but still real) failure could end up putting more than 19V on the 12V rail before shutting down. My FPGAs tolerate a range of voltages, and hence a high-spec ATX power supply would give the FPGA boards the least work to do (with a constant, regular voltage, the VRMs aren't constantly working away). But if a failure resulted in the PSU dumping excess voltage across the 12V rail and ground... that'd burn out the FPGAs.
So you're saying that the chances of a high quality ATX PSU, given clean AC power, will fail safe - if they fail at all? As in any internal PSU failure resulting in a blown fuse, or brown-out before shutdown? If so, then that's very reassuring. The FPGA units can accept a wide range of voltage, so if the 12V rail on the PSU browns out from 12 down to 8 V, the FPGAs will still work. If the power then shuts off, then no damage should be done to the hardware (any more than pulling the power cable out).
My office runs off a single ring main I installed myself using slightly over-spec cable, it is 'protected' by a decent surge arrestor, and the main server and critical networking parts sit behind an APC Smart-UPS 1500VA (the server doesn't eat much power - it's a Mac Mini Server with dual SSDs for speed). And every power socket (240V, this is England) runs through a 'Gold' class surge arrestor (yes, paranoia, and whilst I don't expect the £100,000 payout due to lightning strikes claimed by the products, with two layers of surge protection I *hope* I'm safe) - I'm not running an enterprise-level datacentre here. To be frank, I've not seen inside a modern datacentre for well over a decade... So I've had a bit of 'back to my early days' fun designing, building and optimising these new BTC miner rigs. No doubt it's child's play to you and other professionals, but that's why I'm asking you the questions...
The difference this time is that I'm making a substantial, serious investment in FPGA Bitcoin mining. I believe the risk / reward ratio is favourable, so long as I can pay off the hardware costs *before* the increase in volatility due to the block reward reduction (yes, it could result in higher BTC prices, but the expected volatility and the time before the event is too high to realistically factor into business plans - yes, there's an argument that prices would rise, but I can't make capital allocations based on bets that far out).
My business plan makes sense - but hardware failures would endanger my profitability comfort zone. If I ran an entire 25 unit cluster from one PSU and that failed, destroying all 25 units, I'd be fucked. Is there any such thing as an economical 'fuse' that blows on voltage, not current? I'm sure this applies to the BFL units as well as my Ztex FPGAs, but there's a maximum input voltage that the on-board VRMs will handle before blowing up. I'd rather have a fuse blow up than an FPGA board. Whilst I can assume the current drawn by the board at maximum mining rate from the power consumption, it's not the case that the board will be able to draw more current than that - i.e. pulling more power due to input voltage being higher and the VRMs have to dissipate more heat providing the required voltages. Besides, if the PSU browns a bit and starts supplying 10V instead of 12V, the FPGA will still work just fine. I don't want fuses blowing right left and centre just because our voltage has sagged a bit (very common in England, I'm afraid)...
So I'm basically asking politely, to those who are proper qualified EEs (which I am not), whether I'm at risk using my best PSU for all my FPGAs... or using two lower-rated PSUs (e.g. Cooler Master GX600s)... or sticking to one switching-PSU brick per FPGA board and a load of custom wiring. I'm happy making custom wiring and have the facilities to test before use.