Antminer S9 randomly stops hashing altogether?

[GipsyD]

First time posting here, and long time lurker of the forums. Be gentle with me  

Also not sure if this is the right section, but here goes nothing.

Also posted this on R/BitcoinMining

I know this topic has been discussed to death, but I am running out of options and ideas here, so hear me out.

One of my Antminer S9s recently started to act funny where it would shut down at random intervals and restart, or it would remain on and shut down all the hashing boards in the machine. It would seemingly run fine for a while before all the hashing boards begin to shutdown chips slowly, eventually leading to the machine coming to a complete standstill.

I restarted the machine manually every time I saw this issue from the web portal, and that seemed to resolve the issue momentarily. I then thought that the firmware on the machine had somehow gotten corrupt or malformed, and loaded other Bitmain firmware. The same issue occurred on every install.

I've loaded the S9 650M Firmware -

https://file12.bitmain.com/shop-product/firmware/Antminer-S9-all-201705031838-650M-user-Update2UBI-NF.tar.gz

After that, I flashed the 08-08-2019 Firmware, as I seemed to have had better luck with this firmware -

https://file12.bitmain.com/shop-product/firmware/234aa65a-3740-47ed-94dd-a1bcf4515397/2019/08/08/21/Antminer-S9-xilinx-201907302037-autofreq-user-Update2UBI-NF-sig.tar.gz

After the 2019 Firmware failed, I used the recovery method mentioned on Bitmain's website, which can be found here -

https://support.bitmain.com/hc/en-us/articles/360007577214-T9-Control-Board-Program-Recovery

I have a log from the machine, which can be found here -

https://drive.google.com/file/d/1dhcs6c2JyOHZhmhWyRvPwLeK0l_hyG8X/view?usp=sharing

On occasion, the Kernel Log will say that there is a RunTime Error with all the Hashing Boards, but this happens randomly.

Right now, the machine has got VNISH 3.8.6 loaded, and the log above is before VNISH was installed. I am starting to think that the PSU might be the culprit, but I am open to any suggestions as to how to fix this issue.

Any help at all would greatly appreciated, including SSH scripting.

Thanks for viewing!

[BitMaxz]

It seems that the overtemp protection is your issue because the miner is forcing it to run at 16THs and stop due to overheating.

Did you set up the working mode(Multi-option firmware) to the highest hashrate or it is set to "normal mode"?

Can you run it at a lower hashrate and let see if it will run stable or try to run it at "low power enhanced mode" then try to check it again if it will run stable.

Or if you already flashed it with an SD card flash it back to 2019 but this time make sure to test it first with low power mode.

[philipma1957]

At op you did all those firmware changes and all that work without down clock experiments?

Gear ages. S9's have caps (capacitors) as they age the gear becomes shittier.

A simple down clock to 12th at 1000 watts  with fans at 80% should fix all your issues.

note the six caps
other issues are chip aging
and lastly traces get old.

[GipsyD]

At op you did all those firmware changes and all that work without down clock experiments?

Gear ages. S9's have caps (capacitors) as they age the gear becomes shittier.

A simple down clock to 12th at 1000 watts  with fans at 80% should fix all your issues.

note the six caps
other issues are chip aging
and lastly traces get old.
https://www.centrix-intl.com/wp-content/uploads/2019/11/IMG_1732.jpg

The miner was underclocked to see if there would be any change in stability. The same issue occurred during underclocking. I've disconnected one of the hashing boards and loaded the 2019 firmware in place of VNISH, and have got the miner hashing in Low Power Enhanced Mode at 8TH Stable.

I will reconnect the hashing board and try the underclock to 12TH.

Which firmware is best for this? Would VNISH be fine? Or would I be better off using the Antminer 2019 Firmware I have linked above?

Apologies, mining is new to me. I've only ever traded crypto on exchanges.

[GipsyD]

It seems that the overtemp protection is your issue because the miner is forcing it to run at 16THs and stop due to overheating.

Did you set up the working mode(Multi-option firmware) to the highest hashrate or it is set to "normal mode"?

Can you run it at a lower hashrate and let see if it will run stable or try to run it at "low power enhanced mode" then try to check it again if it will run stable.

Or if you already flashed it with an SD card flash it back to 2019 but this time make sure to test it first with low power mode.

It was originally set to the Highest Hashrate, but then I ramped it down to Low Power Enhanced Mode, which it is currently hashing at right now. It's running at about 8TH stable.

I'm starting to think that the transformer inside the PSU might be failing now due to age.

I've removed VNISH on this machine, and reinstalled the 2019 firmware on the machine. It's on Low Power Enhanced Mode at 8TH stable.

I am going to try philipma's suggestion and report back

[mikeywith]

I've removed VNISH on this machine, and reinstalled the 2019 firmware on the machine. It's on Low Power Enhanced Mode at 8TH stable.

It's more often than not that if your miner does reboot at default settings and does not reboot when underclocked - it's a bad PSU, the other reason would be one or more of your hashboards just can't stand the high frequency, however, the outcome of the latter is a bit different, the miner won't usually go to a standstill mode, but it would rather keep rebooting to do a chip recount on the bad hash board/boards.

The best way to troubleshoot this is by swapping the PSU and putting the miner on the default settings.

[GipsyD]

UPDATE : It turns out that the culprit here was the PSU. It appears that the transformer inside the PSU is starting to give up the ghost. I have since replaced the PSU with another 1800W 80 plus Gold rated PSU, and the machine has recovered all it's hashrate.

The machine has returned to work with no issues.

Thanks for all the help everyone, I really appreciate it!