Way overly optimistic without a schematic.  I shut it down and inspected.  Inspected and check grounds.  I'd have liked to trace the fan lines but the multilayered board precluded that and without a schematic that's that.  The only thing I found amiss is between pins 1 and 20 of the ribbon cable connection - the green covering had been lifted between the two pins as if those pins had carried quite a bit of current at one time and heated that area.  So no joy.

I would imagine that might happen if someone tried powering the S3 putting connectors to only the one side and the other side tried to hash with current across the controller board.

I've noticed that if I ssh into an S3 and command halt, unless I cold boot I can't get in.  The device will acknowledge a ping but port 80 seems closed.  And the halt command stops the cgminer-monitor so it doesn't restart at otherwise 3 minutes most.

The reset button is a bit below the hole, not aligned well.  And I found on the S3 as well as a miner from another company that the reset doesn't work all that well.  What I try is holding it in for 30 seconds, then mashing it every few seconds for a while, then turning it off, letting it sit a  minute then rebooting while mashing the reset every few seconds a couple of times then letting it boot.

Try disconnecting the power supply from the S3 while you leave it off for some period before you retry.

Yes, a cold boot restores relatively normal hashrates but there is a continual decline.  I'm hoping it will settle out at a hashrate that is a stable floor.  I had tried anticipating when it would hit 5% less than 441, knowing that end point, 418.5, if the plot tracked a capacitor charging track a single timeconstant predicted five timeconstants would have expired this afternoon.  It finished early because the cold boot was late afternoon and much of that first timeconstant the S3 was in a cooler environment so the prediction was long.

Does either of your fan speeds seem slow relative to temperature?

Rebooting will restore the hashing for a while I suppose and cost less than a return.  

Ah, my problem S3 has been running 1d 16h and has slowed to 418.15.  Three hours ago it was 418.80.

I have been in an acrimonious email exchange with the reseller.  So, for me to ask him to submit the slow hashing blade for exchange would beg an extended delay.  To request an exchange from Bitmain would probably be met by an instruction to return the slow hashing board to the reseller.

I'm not certain it would be in my interest to return the hashing board.  If Bitmain would accept the board directly what is the estimated turn around time?  What is the estimated shipping from the US?  Answers to these questions would help inform my decision how to proceed.  Thanks.

soy

The fans give a speed indication output.  I don't know what kind, digital or analog.  The fan speed is controlled by pulse width modulation.  I don't see a corresponding hashing board temperature and fan speed though one goes up so does the other.  If I saw three miners having a 40° board temperature and all three have the same speed, I'd say there was a digital feedback loop but this is not the case.  So, one might assume that there is a ramp or curve that says when the temperature is A the fan speed supply voltage will be X, temp B gets voltage Y and temp C gets voltage Z.  The speed indication just giving an indication of change versus temperature to the user.

The bad hashing board having a declining hashrate and a declining temperature might indicate the fault is elsewhere except that even after a cold start, after significant cooling time such that hashing starts over 442 and works its way down over hours, shows a lower temperature and fan speed early on.  If that slower fan speed causes errors away from the temperature sensor, those ASICs may shut down engines and slow hashrate but not quickly effect temperature indication.

So the slow fan speed relative to temperature should be addressed.  The problem may be in the pulsed width voltage supply line.  A partial open, high resistance, in a plated thru hole, e.g. dull drill bit left fiber in the hole, the hole only partly plated.  So, that's where I'll look first when I troubleshoot.  I'll follow the pulsed with fan voltage first, then the ground and perhaps the speed indication pair, looking for something amiss.  But not today.

Busy myself and only reading two Bitmain threads on bitcointalk.org.  I was informed by the post.  Thanks for posting.

soy

I appreciate the info.  Won't hijack the S3 thread to dwell on those (see a series of shift up characters from the number row here) Kansas frauds.  Suffice it to say they were not ethical in their dealings.

Having already repasted the heat sinks in the past today's effort was aimed at cleaning the ASIC pins and anything near components that might effect operation like perhaps compound on capacitors.  Since the inside heatsink paste is assumed good, I removed the cover, the fans, the outside heatsinks, the controller board.  I found the ASICs are numbered from 2 to 17 starting at the upper left then to the lower left then to the next lower ASIC then up then across to the next upper ASIC so it finishes with 17 at the upper right.  Putting the miner without fans or controller board upside down outside I used a can of brake cleaner with straw to clean the board and it's inside then used a second can to clean the other board and its inside with a little cleaner left over for the controller board.  Then firing up my air compressor I blew off the boards.  Hoping it left the controller board okay.  Experience showed me compressed air hitting an asicminer blade FPGA could cause it to lose programming.  Now to reassemble....

Note there's a void in the outside heatsink over the LM75A and a channel for the line of capacitors.

...has awakened with all ASICs o's.

Can't ask for better conditions for this to be running right.  63°F outside, 72°F in the room with miners and that's at 20%rh.

Considering right now if I were to do this again I'd put the spring screws back in place without the heatsink before cleaning the board with the brake cleaner so the cleaner doesn't get into the screwholes.  Worth mentioning clumps of thermal paste on the ASIC pins, e.g. if the heatsink had been slid sideways in removal, might be removed CAREFULLY with a wooden toothpick.  The spray brake cleaner gets used up pretty quickly.

At 30 minutes it's doing 440.82GH/s.  I suspect it will continue to be reasonable most of the day unless it warms then drop tonight again to the mid-430's.  Engines turning off in one of the ASICs for some reason and all the cleaning and reapplication of thermal paste won't fix it.  I could be wrong and maybe it will stay good but I doubt it.

...maybe

40 min 444.29(avg)
 1 hr  443.72
 2 hr  441.60
 3 hr  441.22
 4 hr  440.74
 5 hr  439.68
 6 hr  434.89
 rebooted due to low hashrate

If I were still in the game I'd look for an open.  The low fan speed is an indicator of something amiss as well.  A plated thru hole that fails would fit the bill.  Tough to find without a schematic but not impossible.

If it's a charge buildup then a complete shutdown would be more effective than a restart.

I had adjusted the cgminer-avg-monitor script to look for a hashrate of 432 rather than 439 and run the program only on the hour not every 30 minutes as there were too many restarts and changed the script to reboot instead of cgminer restart.  It triggered at 431.

I'll see how this rebooting works out.  There is a 60 second delay added before rebooting.  The loss, as someone has accurately pointed out, in not much in dollars and cents so troubleshooting it isn't high on my list of things to do.  If I do opt to troubleshoot, I have the other board with which to compare.

 18 m   438.35
 31 m  437.63
 1 hr   439.67     Temps dropping
 2 hr   440.24     Room Temp 76°
 2h59m  431.83  Room Temp 75°
 3h rebooted due to low hashrate

 30 m  437.51 75.0°
 1 hr   440.45  75.0°
 2 hr   435.66  73.7°
 3 hr   438.18  73.9°  Lower room temperature does slow the decline.  Utility down to 14.  ~Midnight.

So with the sleep I added to let it cool it takes 2 minutes before it's hashing again.  Booting at 431GH/s, when it's 10GH/s down, the 10GH/s*120s=1200GH's which is like the miner not running for 2.72 seconds.  So, triggering a reboot with delay that takes ~ 2 minutes loses 117 seconds of hashing.

The cgminer-avg-monitor restart cgminer at low hashrate is functioning but my S3 isn't cooperating.  I went out to shop and returned after a few hours.  It was restarting cgminer every half hour due to low hashrate until 5PM then passed at 5:30 and at 6PM but  then the hashrate started to drop.  It restarted again for low hashrate and woke up with two x's, one in each chain so no way it was going to pass the next restart. 

I think this might be memory that isn't cleared or chips that aren't reset rather than bad ASICs.  I've tried adding delay, e.g. sleep 60, but it still fails to restart properly.  I know a cold boot works.

What I have to do with my cable box and routers when there's an internet disconnect is use the br command & an X-10 transmitter on a linux machine to cycle cable modem and routers off and wake those up in order with the proper timeouts.  This works efficiently albeit the cycling is a bit noisy and I'd wish X-10 would move up to solid state relays.

Since a cold boot works almost every time, I'd like to use the same X-10 idea with an S3 but there is no ftp and although /etc/services lists ftp, connections are refused.  I don't know if I can just add any repository to whatever angstrom program lists repositories, this without fouling up the system with incorrect version programs.  I'm not even sure there is a br command for angstrom.

This all may be moot as I've purchased additional thermal compound and cleaner so I'm going to address the problem S3 once again tomorrow.  None the less I'd like to know how to get in and out of the S3 via ftp.  Assistance would be appreciated.

Bull.  That would be like storing rubbing alcohol in a vodka bottle.

Yep. 

Couple of ways to run the cable to outlets.  Although a 20 amp circuit with 12-2 might have 15 amp outlets, the total allowed wouldn't change.  The 15 amp outlets have a couple of ways to clamp the wire.  They could have the solid #12 looped around an outlet screw and clipped, then another length of #12 wrapped around the other screw on that side and clipped, and run to another outlet, same on the other side of the outlet.  Or they could have pre-built the circuit for the room and stripped back the #12 for 3/4" and made a loop and only attached it to one screw, not clipping the wire but tough to route if you're using holes cut in the studs before the drywall goes up.  If looped and clipped the 15 amp outlet could conceivably see more than 15 amps thru the metal on the side having the two screws.  Or, and this is more likely to cause outlet heating, the electrician ran the cables to the empty boxes, the cables cut a foot or so beyond the box.  He stripped the #12-2, both the black and white, a half inch and pressed the wire into the hole in the back of the outlet, black to the side having the gilt colored screw and white to the side having the silver colored metal screw.  Then took the cable to the next outlet and did the same to the other pair of holes on the back of the outlet.  This type of daisy-chaining, using the push in holes, likely see the solid conductor making contact with a metal tab pressing on its side and is more likely to heat.  Not sure if a standard 15 amp outlet would accept #12, hold on I'll check...nope, #12 won't fit the hole in a 15 amp outlet.

Just tried that and an x was changed to a -.  Wonder if they ever change to an o without a reboot or restart.

...the auto reboot for low hashrate by cgminer-avg-monitor at the half hour didn't clear the -.

Likely means that it's completely shut down.  Not sure what the "-" means as a comparison.  I've found that completely turning off the miner and letting it cool generally gets an x to revert to an o.

A cold reboot with the machine off a minute allowed it to wake to 441.05GH/s and the x gone.  An x in the 8 position can only be in the center of the board if upper left ASIC is 1, lower far left ASIC is 2, second upper left ASIC is 3, etc.  This would put the eighth position of the chain at the center left bottom ASIC.  Next time I take it one apart I'll look at the component numbering and finding a component common to each ASIC, see how they progress.  Who knows, the ASICs themselves might be numbered.

Up 11 minutes and there's an x in a different position, now in the #2 position of the top chain.

Miner Configuration, you'll see a gray'd out Advanced Setting to the right of General Settings.

Okay, have written/modified the scripts and changed scheduled tasks.  By the time I was done the hashrate was down to 435.09GH/s  So, in about 22 minutes the script will run and it should restart leaving a record in /usr/bin/avg.log.  Nah, will try running from the command line...failed, typing errors I presume.

Okay, the lower case L was actually a one.  Runs fine.

darn, restarted with an x at position 8 of the 16 representing the problem board.

The LM75A is in the center of the ASICs, halfway between the upper and lower rows.  I'd guess a connector fault and would try swapping ribbon cables to see if the problem changes chains.