GPU mining and fire hazard

[NoOneLt]

Hi everyone,

Today I was lucky enough to be at home when heard loud noise from miner and strong smell. Luckily I was able to turn it off quickly, seems like one of Inno3d GTX 1070 went down, and it is one of newest card, ~1 month old.

I also have some remote miners, now I really wandering what could happen if I won't be near to turn miner off?? What security mechanism to avoid hard overheat is implemented in GPUs?

How are you guys monitor your remote rigs to avoid any big damage?

Thanks!

[1714644883]

1714644883

[1714644883]

1714644883

[1714644883]

1714644883

[1714644883]

1714644883

[1714644883]

1714644883

[1714644883]

1714644883

[Satlite]

I undervolt and lower the default clock to make them more stable and use less electricity, thus less heat.

[robrev]

I have had this happen on me while I wasn't home.

I realized one card was down and told someone to go check it.

Riser was completely fried and there was an absurd burning smell but that was about it, no fire.

[Ale-x1]

Thin power cables to riser or gpu could cause it. And should try to lower voltage and PL if gpus mine heavy algo.

[Max Likelihood]

Go to riserless motherboards if it was the sata adapter to the riser or riser itself. I keep mine at 70% power draw or less, and turn it off if I'm gone for more than an hour or two, though others don't like the down time and install kill switches at the outlet if a sensor exceeds some temp.

You can also set max temp on cards, but I think the major danger is from the cables--running to much power through them, compared to what they're rated for.

[DevelopmentBank]

Are you sure it was your GPU that got fried? From experience it is always either the PSU or PCIE riser that will burn first. PSUs will pop because of overload and risers will burn because of poor SATA/molex connections.

Given that your GPU is only 1 month old i also doubt it would just spontaneously give out. Unless its  a lemon. Could you provide more details to help us learn more?

[leonix007]

A regular schedule checkup is a must for the rigs, there are tools out there that are useful in doing chekups

this is a must buy

https://www.amazon.com/Seek-Thermal-Compact-Imager-iOS-Apple/dp/B00NYWABAA/ref=sr_1_1_sspa?ie=UTF8&qid=1522035577&sr=8-1-spons&keywords=seek%2B%2Bthermal%2Bimager&th=1

so any unwanted high temps could be monitored and acted upon, and so is true to your Electrical wirings and circuit breakers

 

[ivakar]

The danger is there yes, so that is why a good advice not load the cards on very high levels. I think PL75-85 levels will be good, also you have to check how good your psus, they should turn off if there is a short cut..
But it is better if someone is watching out for the rigs, imho

[Rock771]

If you're not home, I'd say to install sprinklers... but well, that's just going to fk up your whole mining operation and your life.

I think if you use a mineral oil cooling system, it's safer because well, it can't catch on fire.

[edwardceng]

Same experience. next, I have a few tips for you doing if you want:
1. don't until GPU temperature on more from 70 C.
2. On side RIG, install additional fans like PWM.
3. can be happened because of remain power PSU a bit. my every Rig has 500 W a remaining power from PSU.

for monitoring mining Rig, I use TV and lucky me has a brother which can monitor physically.

[m.vina]

A regular schedule checkup is a must for the rigs, there are tools out there that are useful in doing chekups

this is a must buy

https://www.amazon.com/Seek-Thermal-Compact-Imager-iOS-Apple/dp/B00NYWABAA/ref=sr_1_1_sspa?ie=UTF8&qid=1522035577&sr=8-1-spons&keywords=seek%2B%2Bthermal%2Bimager&th=1

so any unwanted high temps could be monitored and acted upon, and so is true to your Electrical wirings and circuit breakers

What that $200 piece of equipment is really a must buy? Are you sure?

You can easily check GPU temps on the computer / miner's interface. Don't tell me you bought that just so you can check temps of your mining rig. Poor investment if you ask me but i'm curious, can you see temps through walls? Could be a useful tool to see if someone is having sex in the next room. Lol.

[NoOneLt]

Are you sure it was your GPU that got fried? From experience it is always either the PSU or PCIE riser that will burn first. PSUs will pop because of overload and risers will burn because of poor SATA/molex connections.

Given that your GPU is only 1 month old i also doubt it would just spontaneously give out. Unless its  a lemon. Could you provide more details to help us learn more?

Hi, pretty sure it is GPU, i will investigate it deeper later but riser/cables has no sign of damage, and GPU itself has strong strong smell... And yes it is only 1 month old.

Thanks everyone for advices, i will clear thing a bit:

1. All my rigs are NVIDIA based 1060,1070/ti,1080,1080ti
2. All set up to 73C max and ~65-75 PowerLimit (and actually running 55-70C)
3. Cards has a lot of space between 10-15 cm at least (6-7GPU rig is 90cm long)
4. Almost all risers are the same manufacturer and looks like pretty quality one. Thats why they are all the same.
5. GPU OC is +50 on GPU max and 300-500 on memory depending on stock clocks, but no extremes
6. PSUs takes up to 85% of their max from the wall, so if it is 1000W it takes about 850 from the wall (and thats gives + -10-15% beacuse of effeciency so it is about 750 actual load on PSU).
7. Maybe PSU load could be a problem, but they are all EVGA/Seasonic/Corsair and has some spare power.
8. All rigs are pretty stable, some may reboots each once 7-10 days, but if it happens often i look at them closer until they are stable.

So far that much from me.

Thanks everyone involved!

[edwardceng]

A regular schedule checkup is a must for the rigs, there are tools out there that are useful in doing chekups
this is a must buy
https://www.amazon.com/Seek-Thermal-Compact-Imager-iOS-Apple/dp/B00NYWABAA/ref=sr_1_1_sspa?ie=UTF8&qid=1522035577&sr=8-1-spons&keywords=seek%2B%2Bthermal%2Bimager&th=1
so any unwanted high temps could be monitored and acted upon, and so is true to your Electrical wirings and circuit breakers

What that $200 piece of equipment is really a must buy? Are you sure?
You can easily check GPU temps on the computer / miner's interface. Don't tell me you bought that just so you can check temps of your mining rig. Poor investment if you ask me but i'm curious, can you see temps through walls? Could be a useful tool to see if someone is having sex in the next room. Lol.

lol.

the points its, I agree with you two. for me, if @OP can indeed purchasing that tool, its no problem because of investment for security indeed a bit expensive. and if @OP can't want to purchase that tool and have $200 maybe can will purchasing another part like PWM(maybe).


Edit: sorry, a problem @OP was clear.