My BFL SC Single 60 went out in a puff of smoke - what next?

[lightfoot]

So.... What to do? Note all of this is from a complete random stranger on the Internet dreaming about stuff, so disclaimer, disclaimer, don't do any of this in real life, you're responsible for your own universe, this is just me.

If *I* had a board like this....

Bit of knowledge about the BFL design: If the 60 single boards are like the little_single, then the power supplies for the FETs are connected together and connected to the 8 chips that run on the board. What I don't know is if the left side 8 chips are powered by the left two power converters (the FETs that went foom) independently of the right side 8 chips. They may all be on one bus, or two.

Update: Based on reviews of the SC30, it looks like the two sides run separate chip power converters. That is good and bad from a design stage, but it can explain how one can lose half their hashing power.

Anyway, what's the damage? Well, it could be limited to the FETs, I would say scrape off the wreckage, hope the Pads behind them are still good (test for shorts and opens to the other FET baks) and if there are no shorts, try firing it with a regulated 12 volt supply (without any mining software, just see if the chips come up with the remaining 3 supplies). If they do then either lobotomize 4 chips on that side to be safe by turning them off in software (making a 45gh unit tops) or if this thing has all the power supplied in parallel lobotomize six chips to keep the remaining three supplies from overloading (assuming they also have some damage). And make sure your power supply's 12 volt source has a *FUSE*, wire in a (12 watt*12=144 watt/12 volts=12 amp fuse *TOPS* since you blew things start with a much smaller fuse TOPS). If you blow the fuse without hashing, you have a board short, start digging.

That's the fast way. Sure it's 35-45gh instead of 60, but half a loaf is better than none, all that. Peter if you have SMD soldering skills and you were me, you might want to try taking off all six FETs, check for shorts on the 1 volt to ground and 12 volt to ground lines, put in the power supply fuse on your lines,  and see if anything works (you need to remove both sides because they might be back-feeding the oscillator)

What's the worst that could happen?

If you want to fix this whole power supply it's going to be harder. My experience is that this will blow the FETs and possibly the gate driver chips. I don't know if they have 1 gate driver powering all twelve chips on a side, or two sets of gate drivers. If the driver is blown and they have two drivers per side, the other side should still work and you can power 12 chips. If the sides are isolated and they have one set per side you can run 8. If the sides are not isolated and they use one honking driver to power everything then you're screwed till you replace the driver (a 2708 chip on the bigger controllers, look for something like that in the chip mainfest).

What's the worst that could happen?

If the FET pads are warped, you might not be able to get it going without bypassing the 1 volt lines or re-routing power from the 12 volt source supply. That's way beyond the source of my ability to help from the other side of the earth.

So anyway, a fast potential solution that I would use to get me something would be:

Remove the wreckage with a good pair of soldering irons. Don't go with insane amounts of heat; the carmelized board will conduct heat faster and melt more than normal.

Check for shorts

Try powering it up without hashing with a fuse in the supply.

Reprogram it to only run 8 chips instead of 16.

Try hashing.

Add a few more chips, do not exceed 11 of them if you want it to last.

Good luck. Time is of the essence since difficulty keeps going up and time waits for no man in Bitcoin world. Having 30gh running now is better than 60 in a month's time. Good luck and keep us posted.

And thank you for sharing this: It's given me a chance to think through a problem, which to me is always fun. If anyone else finds this useful and it makes them money do me a favor: Donate some of that to your local soup kitchen in the name of "Bob Dobbs". That way my rantings here make the world a bit better or something like that.

C

[1713859167]

1713859167

[jojo69]

hope you're right lightfoot, I'm just going on my spotty success replacing the VRM FETS on videocards and motherboards; sometimes it works, sometimes it doesn't, and sometimes the FETs smoke instantly when you apply power.

I think OP should definitely get ahold of Cranky4u if he doesn't feel comfortable soldering it himself

[lightfoot]

hope you're right lightfoot, I'm just going on my spotty success replacing the VRM FETS on videocards and motherboards; sometimes it works, sometimes it doesn't, and sometimes the FETs smoke instantly when you apply power.

Yup. Sometimes you're lucky and it just nuked the FET. However since the FET gates on low voltage stuff like this are rarely driven by optoisolators and isolated DC-DC's, it usually takes out the drive circuits as well. IGBTs in high voltage systems are typically driven that way, which firewalls the damage to the IGBT, the resistors between gate and collector, and the um... circuit that snoops collector to emitter voltage and sends the HOLY FUCK EC voltage is going through the roof! WE HAVE A SHORT CROWBAR GATE NOW! thing.

The latter is usually what saves that 50kw fire that results when one side of your IGBT shorts. The crowbar sees the voltage from C-E screaming up (which means high current draws), forces the gate down to ground and lights an optoisolator to light up the "you're fucked" light on the dash.

Then if you're a super-special person, you say "No I'm not", bypass the crowbar circuit, fire up, and send me a box that looks like someone used a flamethrower in it. Been there, dealt with that. :-)

I think OP should definitely get ahold of Cranky4u if he doesn't feel comfortable soldering it himself

Indeed. looking at the specs for these FETs I think they have a solder pad under them that is where the heat's supposed to go. Hot air is probably the best way (400c is what seems to work with my Aoyue unit) to float these off. Not hard, just need the right tools. But they're small, so a pair of soldering irons one on each end might work as well.

Never dull.

[peterepeat]

Thanks everyone - I have a solder/hot air rework station but I agree it may be better to have someone more experienced look at it before I break something else.
I have got in contact with Cranky4u who has generously offered to have a look at it for me next week. Fingers crossed.

[Gator-hex]

I think there's probably a reason BFL now sell these a 50GH/s mining machines instead of 60GH/s ones.

https://products.butterflylabs.com/homepage/50-gh-s-bitcoin-miner.html

There is a lifetime warranty, I'd send it back.

There is far too much thermal paste or pads on those ASICs it looks like cake!

[ninjaboon]

just wondering about the cooling for your 60g. was it in an aircon room ?

[jojo69]

I milled the output plate of my 60 yesterday so it is just a thin frame around the outside to hold the case together.  It cut down the wooshing air noise and where before the input fan would run all the time now it throttles in and out, so it is definitely cooling more efficiently.

The thermal pads in the pics I have seen do look very thick, I may rework those in the future, also thinking about some thicker 92mm Noctua fans for on the heatsinks themselves might really cut the noise...but I hate the downtime to implement, lol.

[lightfoot]

Just a side note, from what I am seeing the two 30gh sides of the 60gh unit may be independent. In other words, failing that burned side should not affect the right side's 1 volt supplies.

Or, remove those FETs, fire it up (with fuses in the power supply lines) and you should be able to restart hashing at 25-30gh on the remaining side. Half a loaf is better than none and all that. I'd say 75% probability of this working.

Good luck!
C

[lightfoot]

I think there's probably a reason BFL now sell these a 50GH/s mining machines instead of 60GH/s ones.

https://products.butterflylabs.com/homepage/50-gh-s-bitcoin-miner.html

There is a lifetime warranty, I'd send it back.

There is far too much thermal paste or pads on those ASICs it looks like cake!

Mmmhm. I am not going to exceed 24-27gh on my jally single based on this. Can't wait to get another chip so I can see if the temps on the FETs go up linearally or geometrically with load.

The problem with the warranty though is that they should be able to replace your unit. And if it takes 2 weeks to do that, it's two weeks of not hashing at current difficulty, the result may be a 20% hit to your profits or more based on diff changes.

That could be fixed by BFL offering some sort of conditional cloud hashing during the time your unit is being fixed. Man that would be a public relations *WIN*! Say when you get an RMA you get put into a special hashing queue where your hashes are immediately escrowed while you send the unit in.

If you didn't mod it, and they fix it, they send it back and when you take delivery they stop the mining in their cloud and give the money to you. If you did mod it, they keep the hash money.

That is a no-lose solution, someone suggest it to BFL. In fact that would be the biggest PR win in this kind of community and it would not cost BFL much of anything to do.....

[peterepeat]

The unit is on its way to cranky4u, hoping he can fix it. I decided not to return to BFL due to the time/cost for shipping and customs delays. My fingers are crossed. 

[peterepeat]

just wondering about the cooling for your 60g. was it in an aircon room ?

It was not air conditioned

[Bicknellski]

The unit is on its way to cranky4u, hoping he can fix it. I decided not to return to BFL due to the time/cost for shipping and customs delays. My fingers are crossed. 

Wish you luck for a fast fix and back to happy mining.

[Nemesis]

What was the ambient/room temp?

Do you know what was the temp reported by CGminer also?

[peterepeat]

What was the ambient/room temp?

Do you know what was the temp reported by CGminer also?

I think the ambient temperature was around 24 degrees C, temperature of the miner was 60 to 65 through the day. Cases were off, fans blowing in and down.
Cheers

[lightfoot]

What was the ambient/room temp?

Do you know what was the temp reported by CGminer also?

I think the ambient temperature was around 24 degrees C, temperature of the miner was 60 to 65 through the day. Cases were off, fans blowing in and down.
Cheers

Just to clarify, was the case open or completely off? Were the side fans on, how were they blowing?

C

[jojo69]

What was the ambient/room temp?

Do you know what was the temp reported by CGminer also?

I think the ambient temperature was around 24 degrees C, temperature of the miner was 60 to 65 through the day. Cases were off, fans blowing in and down.
Cheers

Just to clarify, was the case open or completely off? Were the side fans on, how were they blowing?

C

yeah, I think we found the problem

[lightfoot]

yeah, I think we found the problem

One of them. If there was no crossflow on the bottom heat sinks then that would be problematic. Likewise if the fans just pointed inbound then only half the sink would get cooling, the case provides the tunnel for effective crossflow top and bottom.

Someone needs to send me a blown one of these, I'll post a whole thread on how *not* to nuke your Single. Or something.

[peterepeat]

The case was off when it blew up, the fans above the ASIC clusters were blowing down through the heatsink, the end fans were both blowing inwards - i didn't consider the heatsinks underneath at the time (note, they are just stuck on to the base of the PCB with double sided tape, so fairly poor heat transfer to them anyhow), although when the case was on originally, it was hashing below 60GH and temperature in cgminer was 70/71 °C so I considered it better to run 10° lower temps in cgminer. The failure appeared to be on the upper suface though and there did not appear to be thermal damage on the underside.
As an update bfl did finally return my email with an RMA number but it was too late (and as mentioned before would take too long and was cost prohibitive)

[lightfoot]

The case was off when it blew up, the fans above the ASIC clusters were blowing down through the heatsink, the end fans were both blowing inwards - i didn't consider the heatsinks underneath at the time (note, they are just stuck on to the base of the PCB with double sided tape, so fairly poor heat transfer to them anyhow), although when the case was on originally, it was hashing below 60GH and temperature in cgminer was 70/71 °C so I considered it better to run 10° lower temps in cgminer. The failure appeared to be on the upper suface though and there did not appear to be thermal damage on the underside.
As an update bfl did finally return my email with an RMA number but it was too late (and as mentioned before would take too long and was cost prohibitive)

Thanks for reporting on this. Real data always helps, I have put three little heat sinks on the bottom of my three hot FETs as well as the top, and both are wicking away a pretty fair amount of heat.

The temp sensor for the board is in the center of the square of 8 hash engines, it does a very good job of watching the temp of those chips but does not do a good job of monitoring outliers like the FETs. Although come to think of it, the little SC box that the jally uses pulls the air from the front, right over the FETs, up through the heat sink, and out the top. Maybe that is why they did that; and putting the fan upside down will cool that center board sensor (which is really not the problem) but will not flow air over (and in the case of the SC60) or under the FETs.

Does anyone with an SC60 have access to a IR temp tool? I am wondering about the temp of those sinks, with the fans going down.

Looking forward to hearing about the repairs.

[jojo69]

yes, I have an IR therm, but I just had downtime the other day when I cut the output plate and blew the dust out...It will be a while before I want to shut it off again.