Equipment Damage Collections

[rearwheels]

Hi all,

After mining for about 6months, I've had some equipment failure and have learnt some stuff along the way. So I thought I'll start this thread to share some experience.

Damage to ATX connector (12V lines)


This is on an ASUS board. This rig was getting unstable with random reboots, and it I could not figure out what was wrong with it. The burnt marks are not visible if the PSU ATX plug is in place. I only discovered this when I took apart everything to check.

I suspect the male-female pins contacts are loose thus causing arc-ing between the pins. I ran 4 GPUs all direct from the PCIE slots and this may have been too much on the 12V lines?

I've since replaced the ATX connector on the PSU (cut off the pins + a bit of wire and re-pin it).

This board is definitely still working, and I'll try to salvage it with a new female ATX connector. But I tried de-soldering the female connector on the mobo without success. I'll try it again when I'm free.

[1713307993]

1713307993

[1713307993]

1713307993

[rearwheels]

I purchased this Cooler Master 1000W psu at about $50 pre-owned. After running it for a few days, the system became unstable. I did the same checks and took apart everything to discover this:






Incidentally, this uses the same pins as the ATX connectors. They are of the same series where the ATX is 12x2 and this is 5x1. I de-soldered the female connectors and replaced them with wires and molex. This is running stable eversince.

[rearwheels]

240v @ 13A plugs.

The black plug is an APC 5way extension with surge protector. This was pulling about 1500W-1800W in total (I can't remember, this was a while back) for 3 rigs.

In theory, 240V@13A gives over 3000W. But at 1800W, the plug was warm to the touch and over time, it melted. Luckily I discovered this as I was moving the rigs to a different location.



This COULD have caused a fire, especially you can see it was sitting on carpets.

I've moved the rigs to a different room with 3 phase power with wires rated up to 20A. I've also split up the power plugs and check them with a infra red temperature probe.

As a secondary precaution, I set the router to power down for 15mins every day at 12midnight via a timer. This, I hope gives everything a chance to cool down.

[rearwheels]

Pls feel free to share your stories. 

[CleverMiner]

Not sure about this one, 

As a secondary precaution, I set the router to power down for 15mins every day at 12midnight via a timer. This, I hope gives everything a chance to cool down.

15 min cool-down does very little in a 24 hours day

Thanks for sharing,

[Enigma81]

In the very first picture - the ASUS board with the two burnt up pins.. I suspect it's not poor contact/arcing - it would be really unusual for two pins in one connector to be loose.  These are basically molex mini-fit jr. style connectors, and, if memory serves, they're rated to 13A per pin.  Those two pins are both 12V+, and in fact, they're the only 12V+ on the 24 pin ATX connector.  It's more likely that you're actually pulling more than 26A (312 Watts) and that they burnt up due to over current.  13A is MAX contact rating, so 10A (240 Watts total) is a safer number for continuous operation.

It will probably happen again, unless you power the cards with a separate power connector.

Enigma

P.S. To de-solder the female connector on the motherboard, you'll need a pretty good soldering iron.  Most of those particular pins are (most likely) connected to a power/ground plane on an internal layer, and that will sink a LOT of heat.  Use a big tip (biggest you have), put some no-clean solder on the tip of the iron to get it wet, and then set it on the pin and wait.. It will eventually go molten.  Use a solder-sucker to remove the molten solder.. Repeat for each pin.

Replacement connector here: http://www.mouser.com/search/ProductDetail.aspx?qs=g5f4E6w8l8gzEjxCgQ0R8g%3D%3D
Solder sucker here: http://www.mouser.com/ProductDetail/Apex-Tool-Group-Formerly-Cooper-Tools/7874B/?qs=sGAEpiMZZMt%252biMJH3c40u7BCFiF%252blJXomEfWxnrcMeE%3d

[rearwheels]

Not sure about this one, 

As a secondary precaution, I set the router to power down for 15mins every day at 12midnight via a timer. This, I hope gives everything a chance to cool down.

15 min cool-down does very little in a 24 hours day

Thanks for sharing,

True...I'm being greedy. I guess I should just make sure all the power is distributed properly.

[rearwheels]

In the very first picture - the ASUS board with the two burnt up pins.. I suspect it's not poor contact/arcing - it would be really unusual for two pins in one connector to be loose.  These are basically molex mini-fit jr. style connectors, and, if memory serves, they're rated to 13A per pin.  Those two pins are both 12V+, and in fact, they're the only 12V+ on the 24 pin ATX connector.  It's more likely that you're actually pulling more than 26A (312 Watts) and that they burnt up due to over current.  13A is MAX contact rating, so 10A (240 Watts total) is a safer number for continuous operation.

It will probably happen again, unless you power the cards with a separate power connector.

Enigma

P.S. To de-solder the female connector on the motherboard, you'll need a pretty good soldering iron.  Most of those particular pins are (most likely) connected to a power/ground plane on an internal layer, and that will sink a LOT of heat.  Use a big tip (biggest you have), put some no-clean solder on the tip of the iron to get it wet, and then set it on the pin and wait.. It will eventually go molten.  Use a solder-sucker to remove the molten solder.. Repeat for each pin.

Replacement connector here: http://www.mouser.com/search/ProductDetail.aspx?qs=g5f4E6w8l8gzEjxCgQ0R8g%3D%3D
Solder sucker here: http://www.mouser.com/ProductDetail/Apex-Tool-Group-Formerly-Cooper-Tools/7874B/?qs=sGAEpiMZZMt%252biMJH3c40u7BCFiF%252blJXomEfWxnrcMeE%3d

You brought up a very good point on having both pins burnt. Though the specs for the replacement connector says 6A?

The board had 4x5850s with one of them on a 1x extension (without external power). In fact all my rigs are pulling power via the motherboard. I'll have to go and check on all of them.


Currently I'm using a solder sucker made of plastic (from goot). Does the metal type performs better?

[Enigma81]

The plastic ones are alright, I just think the metal ones are overall better-made and usually have a faster return = more suction.  Especially for pins connected to thermal planes like these, once you start moving air, you start cooling down the joint.  The faster you can evacuate the solder, the less chance there is of it solidifying...

I hate de-soldering things, so I make sure I have the best tools I can possibly have to make it a little less painful...

With a good iron (Metcal is my personal favorite), a good solder sucker and a big solder tip, I kid you not - I could de-solder that connector in under 5 minutes... All 24 pins..  Less than 2 minutes later, the new one would be soldered in..  And, really, I'm probably being generous.. 5 minutes would probably cover the whole replacement...  Good tools make all the difference.  Use a good MX Series Metcal Iron for half an hour - you'll never go back to anything less..

Don't trust mouser's data (regarding 6A).  I just confirmed what I thought I remembered - Mini Fit Jr. pins are rated for 13A (MAX); per Molex.  That means about 75% (9.75A) is a nice safe number for 24/7 continuous operation.  Anything more, and you risk exactly what you have there...

Enigma.

[ArtForz]

Enigma81: You sure?
13A is only when loaded with mini-fit HCS contacts, which is only required by the spec for PCIe 8-pin connectors (and possibly EPS 8-pin, too lazy to check), but normal atx 20/24 pin doesn't require em.
So we have normal mini-fit contacts, wire-to-board, for AWG16 or AWG18 wire, brass contacts are rated for 9A, bronze contacts 8A. derated down to 6A repective 5A depending on # of loaded circuits in housing, board layout, ... (read the fine spec sheet for the contacts, it's freely available from molex).
And that's assuming those are real mini-fit contacts, and not some cheap sub-spec lookalike... fat chance.
As for desoldering coarse stuff, *love* my Weller DS100.

Back on topic.
Burnt the 12Vs on a atx 24-pin running quad 5970s on a MSI 790FX-GD70. Replaced the connector on the board and the plug housing, replaced the 12V contacts with mini-fit HCS series. Preemptively replaced the 2 12V contacts in 5 identical other boxes (one had the plug already partially melted around one pin). No further burnt 12V lines operating 24/7 for over 6 months.
Other stuff:
Dead fans (both the well-known bearing failure) on 2 reference 5970 coolers.
4 Sapphire nonref 5770s with dead fans.
>20 Sapphire xtreme 5850s with dead fans.
Fried the VRMs of the 2ndary GPU on 2 5970s with AC Accelero coolers.
Coolermaster 650W 80+ bronze blew up running 5970+2*5770 (12V output filter coil burnt).
3 Andyson-based 1kW 80+ gold PSUs blew up, each running only 3 5970s at ~880W at the plug (secondary side synchronous rectifier FETs failed).
Box full of dead Scythe UK3K 120x120x38 mm fans, all bearing failure. Replaced with deltas... buy cheap, buy twice.

[rearwheels]

ArtForz: One of these?
http://www.newark.com/jsp/search/browse.jsp?N=422+2203+202171&Ntk=gensearch&Ntt=mini-fit+hcs&Ntx=mode+matchallpartial

Interesting information on the pins mini-fit HCS. I've replaced the PSU side ATX connector with pins I bought off a shop (unknown specs). I'll get some of these pins.

[ArtForz]

If you don't have a irrational phobia of mating Sn and Au plated surfaces, 45750-1112 should be the right one.
I used 45750-1212 - same thing, just gold plated and like 5 times more expensive.

[Enigma81]

ArtForz; You're absolutely correct.  So really, we're talking about a max of around 10A total on the 12V rail that the motherboard can provide..

ArtForz also shows an interesting part of hardware design - it's INCREDIBLY easy to miss a critical specification and design something that doesn't work because of it

Ask me how many times I've accidently laid out a part package for a PCB and not realized until after the boards were fabbed that the datasheet had in really tiny font a note that the package view was from the bottom instead of the top

Enigma

[getcopy]

For the rig to be stable, now I use one ps (300W) for one gpu. Cpu/mb/hd use a standalone ps. Before that I have melted 2 ps.

BTW: 4 pieces of 2nd hand 300W ps are cheaper than one 1200W ps.

[deepceleron]

I had the beginnings of the molex sparky discoloration in a 4 pin to PCIe power adaper, but that is understandable considering how much current would be asked to go through one +12v pin (and why PCIe has three 12v pins in it's connector, to spread the load, and for redundancy if one pin is poor). These molex connectors don't seem so reliable, especially since different manufacturers are likely using the cheapest Chinese knockoffs they can source, and the pin doesn't have anything that puts tension on or keeps pressure on the contact surface, so if the diameter is off even a bit, or the female pin connector is stretched or worn, you can get a very small contact area that would be prone to arcing.

[cicada]

For the rig to be stable, now I use one ps (300W) for one gpu. Cpu/mb/hd use a standalone ps. Before that I have melted 2 ps.

BTW: 4 pieces of 2nd hand 300W ps are cheaper than one 1200W ps.

Unless your 300W PSUs are 80+ gold (do they exist?), the 1200W 80+ gold PSU is going to save you a considerable amount of money over time, and there's no short-game in bitcoin mining if you expect any kind of ROI.

That said, an unstable rig isn't doing you any good, so if it's really the only way you can keep it running, so be it.  I have to say there are plenty of miners running multiple cards on larger single PSUs without stability problems, so perhaps this is just bad luck?

[getcopy]

For the rig to be stable, now I use one ps (300W) for one gpu. Cpu/mb/hd use a standalone ps. Before that I have melted 2 ps.

BTW: 4 pieces of 2nd hand 300W ps are cheaper than one 1200W ps.

Unless your 300W PSUs are 80+ gold (do they exist?), the 1200W 80+ gold PSU is going to save you a considerable amount of money over time, and there's no short-game in bitcoin mining if you expect any kind of ROI.

That said, an unstable rig isn't doing you any good, so if it's really the only way you can keep it running, so be it.  I have to say there are plenty of miners running multiple cards on larger single PSUs without stability problems, so perhaps this is just bad luck?

The first one is totaly bad luck. The cause of the second one is the HD4850 - at that time I am still a rookie and got no idea about how much power a HD4850 would take...

[P4man]

BTW: 4 pieces of 2nd hand 300W ps are cheaper than one 1200W ps.

Be very careful with that, to check the specs of those old 300W PSUs. Many will only offer 10A or sometimes even less on the +12V rail!