Bitcoin Forum

Bitcoin => Hardware => Topic started by: MarkAz on October 21, 2015, 08:15:16 PM



Title: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: MarkAz on October 21, 2015, 08:15:16 PM
So I just had a slightly disturbing issue with one of my S7's - I just happened to be near it when it started beeping (the same thing as if the Internet was down).  I went in to check it out, none of the other machines near it were having problems, so I power cycled it.  It didn't renew it's DHCP lease, nor did it try to get another address, but it powered up as if everything was ok... More troubling is that I was watching my power monitoring, and the power jumped as if it were hashing full speed - I went back to the unit, the fan was spinning but on a very low setting, and the air out of the back was HOT.  Having flashbacks to the S5 burnup problem, I unplugged it again, let it sit for a couple minutes, and plugged it back in.  Same issue.  This time I pressed and held in the small hidden factory reset button, after a couple seconds it gave the long solid tone that normally means it was successful, but once again - no DHCP request, and it looks like it's hashing even though it doesn't have any internet connection.

REALLY not wanting to see a repeat of the S5 issue - loosing a couple hundred bucks a pop when it happened was bad enough, but loosing thousands would be brutal.

I'm going to swap the BB with another S7 to see if that might be the issue, and I'll update as I do some more tests and see.  They've been running fine for several weeks, I power them off of a IBM 2880W PSU (one PSU for each one, since I have a bunch), so power shouldn't be an issue...


Title: Re: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: Biodom on October 21, 2015, 08:38:00 PM
So I just had a slightly disturbing issue with one of my S7's - I just happened to be near it when it started beeping (the same thing as if the Internet was down).  I went in to check it out, none of the other machines near it were having problems, so I power cycled it.  It didn't renew it's DHCP lease, nor did it try to get another address, but it powered up as if everything was ok... More troubling is that I was watching my power monitoring, and the power jumped as if it were hashing full speed - I went back to the unit, the fan was spinning but on a very low setting, and the air out of the back was HOT.  Having flashbacks to the S5 burnup problem, I unplugged it again, let it sit for a couple minutes, and plugged it back in.  Same issue.  This time I pressed and held in the small hidden factory reset button, after a couple seconds it gave the long solid tone that normally means it was successful, but once again - no DHCP request, and it looks like it's hashing even though it doesn't have any internet connection.

REALLY not wanting to see a repeat of the S5 issue - loosing a couple hundred bucks a pop when it happened was bad enough, but loosing thousands would be brutal.

I'm going to swap the BB with another S7 to see if that might be the issue, and I'll update as I do some more tests and see.  They've been running fine for several weeks, I power them off of a IBM 2880W PSU (one PSU for each one, since I have a bunch), so power shouldn't be an issue...

hashing where to?
in any case, maybe turning it off for 20 min instead of just 2 would be a better idea to make sure things equilibrate thermally.
I would put a fan (like Vornado) blowing at it as well.


Title: Re: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: MarkAz on October 21, 2015, 09:22:15 PM
Well, I left it unplugged for around half an hour - tried rebooting, still no luck.

I took apart another S7 I have and swapped the BB's, and now they're both working... The one I was having issues with had the settings reset, so the OS was still running, but for whatever reason the network portion was down.  I'm going to definitely keep an eye on things, as this doesn't leave me super excited about leaving them unattended - I might just switch back to using non-controlled fans so I don't need to worry about it.

I wish when it was happening I had put a temp probe on the exhaust-side, it was right by some other S7's and when it was in this bad state I felt the exhaust from it and the adjacent units, and it was noticeably hotter.

I also opened a ticked with Bitmain and we'll see what they say.  I really wish they had the S7 firmware available so I could have flashed one of my other BB's to test with...
 


Title: Re: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: adaseb on October 21, 2015, 11:24:35 PM
Don't you still have warranty on the S7 ?


Title: Re: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: dogie on October 22, 2015, 12:05:55 AM
I wish when it was happening I had put a temp probe on the exhaust-side, it was right by some other S7's and when it was in this bad state I felt the exhaust from it and the adjacent units, and it was noticeably hotter.

I was actually working on this 2 days ago, trying to work out why my S7 wasn't mining. Turns out my PoE had died but in the hour or so of debugging I had the "hot exhaust" issue. Exhaust temps were up a lot compared to stable mining but heatsinks weren't up anything significant.


Title: Re: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: generalt on October 22, 2015, 12:49:06 AM
At least since you have multiple S7 units you can use one BB to run at least two of them.  Even if one of your BB dies you can try to connect all 6 boards to a single BB and run it like that unit Bitmain gets you a new one.


Title: Re: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: philipma1957 on October 22, 2015, 01:56:52 AM
At least since you have multiple S7 units you can use one BB to run at least two of them.  Even if one of your BB dies you can try to connect all 6 boards to a single BB and run it like that unit Bitmain gets you a new one.

yeah i have one more on order just so I can have 2 machines and use 1 controller.


Title: Re: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: MarkAz on October 22, 2015, 03:52:27 AM
I was actually working on this 2 days ago, trying to work out why my S7 wasn't mining. Turns out my PoE had died but in the hour or so of debugging I had the "hot exhaust" issue. Exhaust temps were up a lot compared to stable mining but heatsinks weren't up anything significant.

So when you say the heatsinks weren't up - do you mean what is reported through the web interface?  I just wonder how that would be possible, since hotter air pretty much == hotter temps... I almost think it stops reading the sensors and/or controlling fan.  Either way, it was 100% using full power when it had no ethernet connection going, so something is up with it.


Title: Re: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: dogie on October 22, 2015, 06:18:09 AM
I was actually working on this 2 days ago, trying to work out why my S7 wasn't mining. Turns out my PoE had died but in the hour or so of debugging I had the "hot exhaust" issue. Exhaust temps were up a lot compared to stable mining but heatsinks weren't up anything significant.

So when you say the heatsinks weren't up - do you mean what is reported through the web interface?  I just wonder how that would be possible, since hotter air pretty much == hotter temps... I almost think it stops reading the sensors and/or controlling fan.  Either way, it was 100% using full power when it had no ethernet connection going, so something is up with it.

With my temp probe gun, was unable to connect to S7 as my network was fubar'ed. The full equation for forced convection heat transfer is as follows:

q = hc A (Ts - Ta)

where

q = heat transferred per unit time (W)
A = heat transfer area of the surface (m2)
hc = convective heat transfer coefficient of the process (W/(m2 K) or W/(m2 ° C))
Ts = Temperature of exhaust air
Ta = Temperature of intake air

In this scenario we can take a steady state snapshot at the end of the board closest to the exhaust. Comparing normal operation to weirditsstillminingbutitsreallynot operation:

q = constant
A = constant
Ts = up
Ta = up
hc = pretend its constant for now

Ie q1 = q2, A1 = A2, hc1 = hc2

In the above scenario:

q1 = hc1 A1 (Ts1 - Ta1)
q2 = hc2 A2 (Ts2 - Ta2)

So Ts1 - Ta1 = Ts2 - Ta2
or Ts2 - Ts1 = Ta2 - Ta1. Ie if the difference between exhaust and intake temperature remains constant.

Earlier I said set hc as a constant - its not. It varies on lots of properties but we can simplify it to hc = 10.45 - v + 10 v1/2 for air, where v = the relative speed of the object through the air (m/s). When you graph this its a nice easy curve (http://docs.engineeringtoolbox.com/documents/430/air_heat_transfer_coefficient.png).

If the velocity halves then you drop from ~32 to 28 (hc1, hc2) for an example, quite a small change [hc2 / hc1 = 28/32 = 87.5%]. Adding this back in above we can get:

hc1 (Ts1 - Ta1) = hc2 (Ts2 - Ta2),
hc1/hc2 (Ts1 - Ta1) = (Ts2 - Ta2),
0.875 (Ts1 - Ta1) = (Ts2 - Ta2).

Even though we halved our effective air velocity, we only have to increase our difference between exhaust and intake air temps by 14.2% (1/0.875). Putting some example numbers in:

If exhaust temps were 50C and exhaust temps 30C usually (20C diff)...
We can maintain the same heat removal with half the air speed, with the intake remaining at 30C and the exhaust.... but with an exhaust temp of 30/0.75 = 57C (27C diff)

If you want to look at the relationship between heatsink (chip) temperatures, its varies based on the average air temp. So if we raise our average air temp by 3.5C ([27 - 20] /2), we also raise our heatsink temp by 3.5C.


tldr: Decreased air velocity = smaller increased increased exhaust air temperatures = even smaller chip temperature increase.


Title: Re: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: MarkAz on October 22, 2015, 08:50:53 AM
tldr: Decreased air velocity = smaller increased increased exhaust air temperatures = even smaller chip temperature increase.

*Forehead slap*  I should have connected those dots, but thanks for spelling it out for me.  ;)

The larger issue for me is that when it's not doing work, it's annoying that it would be burning up energy for nothing - the reality is it should be basically bone cold when it's not performing any work, even if the fan is just barely moving the air through it.  As I mentioned, from the power draw it was cranking away at 100% power consumption, basically doing nothing of value.  The fact that you've stumbled across the same issue is certainly troubling to me - plus I'm not sure what the resolution was (at least in my case).


Title: Re: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: Photon939 on October 22, 2015, 05:03:41 PM
At least since you have multiple S7 units you can use one BB to run at least two of them.  Even if one of your BB dies you can try to connect all 6 boards to a single BB and run it like that unit Bitmain gets you a new one.
Can a standard BB control the antminer S 7? I thought they were a bit modded.

They are cost reduced by not populating some chips and connectors, but a standard BBB should work. (pic of a C1 - can anyone confirm if the new ones are different?)

https://dl.dropboxusercontent.com/u/1181169/20141101_105752th.jpg


Title: Re: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: Photon939 on October 22, 2015, 07:05:56 PM
Awesome that's good to know and does the controller attachment keep all the information or would a replacement need a micro sd with an image on it to run properly?

The OS and all config is on the SD card so you'd need to either keep the original one shipped with the miner and put it in the new BBB or get a new one and flash it with the S7 firmware image.


Title: Re: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: Tupsu on October 22, 2015, 07:13:25 PM
Awesome that's good to know and does the controller attachment keep all the information or would a replacement need a micro sd with an image on it to run properly?

The OS and all config is on the SD card so you'd need to either keep the original one shipped with the miner and put it in the new BBB or get a new one and flash it with the S7 firmware image.

S5/S5+ S7 they all  do not have SD card.

S5+/S7  controller.

https://i.imgur.com/4vFLHHG.jpg (https://i.imgur.com/4vFLHHG.jpg)

Click to enlarge


Title: Re: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: MarkAz on October 22, 2015, 07:17:28 PM
S5/S5+ S7 they all  do not have SD card.

S5+/S7  controller.

Plus BitMain hasn't released an S7 firmware image yet, so you're SOL on trying to use anything other than another S7 controller, for now at least.


Title: Re: S7 died, keeps hashing with fans on low, S5 burnup all over again?
Post by: Photon939 on October 22, 2015, 08:21:04 PM
Yeah looks like they spun their own version after the C1. Sounds like they went to an onboard flash chip like the retail BBB so you'd have an OS flash procedure like you'd do for installing onto internal flash on a retail version. Of course you'll still need the image - why they haven't provided an image for the S7 yet is not good.