Bitcoin Forum
April 19, 2024, 04:09:39 PM *
News: Latest Bitcoin Core release: 26.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1]
  Print  
Author Topic: Rescuing a bricked T9 (and S9?) Control Board - HOWTO  (Read 1447 times)
cableiso (OP)
Sr. Member
****
Offline Offline

Activity: 244
Merit: 280


View Profile
November 11, 2017, 04:39:11 AM
Last edit: September 15, 2018, 09:16:02 AM by frodocooper
Merited by dbshck (8), frodocooper (7), suchmoon (4), Dr.Mann (2), OgNasty (1), BitMaxz (1), HagssFIN (1), Artemis3 (1), Steamtyme (1)
 #1

Hi Folks,

I got lucky in fixing a corrupted control board, and was able to bring it back to life using the serial console and uboot.  Wanted to mention my story to open a discussion on debricking S9/T9.  Also, if anyone wants to sell me a bricked S9 I'm interested lol Smiley

The symptoms were that upon power on, the green lights lit and the ethernet port showed connection (yellow) and data (green).  Data was occasionally blinking.  Really, the device looked like it was alive but didn't mine.  It never did its earsplitting singleboardtest, never even requested an IP from my router, so I could not SSH in or access the web server.  5 second reset, and IP reporter reset didn't help either.

I decided to check out the terminal console.  I disconnected the ribbon cables and pulled the control board out of the miner, then used a $2 USB to serial converter board to connect to the RX, TX, and GND headers on the board. [NOTE: Use 3.3v mode]  I powered the board up using the PCIE-6 plug from a spare PC power supply.  I got text!

Code:
blah blah lots of miscellaneous messages blah blah hex characters
 
Copying Linux from NAND flash to RAM...

NAND read: device 0 offset 0x1100000, size 0x800000
 8388608 bytes read: OK

NAND read: device 0 offset 0x1020000, size 0x20000
 131072 bytes read: OK
## Booting kernel from Legacy Image at 02000000 ...
   Image Name:   Linux-3.14.0-xilinx-gb190cb0-dir
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3779264 Bytes = 3.6 MiB
   Load Address: 00008000
   Entry Point:  00008000

Then a complaint about invalid kernel, and it dropped into the uboot console.  Boo!  So the thing wasn't even starting to boot, but since I had terminal access from the serial port I was hopeful.  It seems that in a borked upgrade (at least this case) the kernel image is what goes bad.  We will learn more over time, I'm sure.

Here's what I did to address it
1) Studied the shit out of all the different bootup types that can be seen using the "printenv" and "?" commands in uboot.  Bitmain must have 8-10 types of boot scripts in the environment variables, accessed by the run command.  "run qspiboot", "run nandboot", etc.  Lots of interesting reading, but they all do similar things - load stuff to ram and execute.  Hey, I'm capable of doing that!  Let me jump in the mix!

2) Formatted an SD card to FAT32, and unpacked the "xilinx" folder from a valid firmware image onto it.  This gives me BOOT.bin, devicetree.tdb, rootfs.jffs2, uImage, and upgrade-marker.bin.  Everything needed for a zero-state boot except uramdisk.image.gz.  SO IF ANYONE HAS EXTRACTED THIS, PLEASE LET ME KNOW.  I'd like to make an unbricking toolkit.
NOTE: for S9 without an SD card, you're going to have to TFTP your files.  I don't think this is a big issue though, just download a TFTP server and drop the files in it's root dir.  Should work the same.

3) Stumbled my way through the SD card commands: mmc rescan, mmc list, mmc part to find out that windows format tool had called the FAT32 partition 5, so thats where my "xilinx" files were.  Why 5?  No clue, but you need to know where they are so you can tell uboot where to load from.

4) Hacked an mmc load of the kernel from SD partition 5 into the memory location uboot was expecting:  Note that I am using 0:5 to denote mmc 0, partition 5.  ${kernel_image} is a variable defined in the environment that stores the filename uImage.  You can simply type uImage into the command below, it will work fine.

Code:
fatload mmc 0:5 0x2000000 ${kernel_image}

5) Booted the little bastard using bootp.  Here I got lucky that the ramdisk and other parts of linux were not corrupt, just the kernel image.  Worse corruptions may need more surgery, hence the need for a valid ramdisk image.  But in my case, injecting a new kernel was all it needed.

6) Once it was up, connected via the web interface and flashed a new firmware image as fast as I could.  Woo!  The miner is back!  After that, it was cured and the board went back into the miner.

All in all, very simple.  But having a $2 USB to serial adapter on hand saved me the cost of a new control board, as well as the hassle and delay of shipping, waiting, dealing with bitmain lol.  Well worth it's weight in gold this evening, for sure.  Amazon or ebay, mine was called "FT232 Mini USB to Serial".  About two inches long, mini USB on one side, 6 pins on the other side for RX, TX, GND and other signals I never use.

Anyway - if anyone else has a bricked control board you want me to hack on, just PM me.  I'd like to see some other examples of how boards brick, because developing a generic "unbricking tool" would be really nice to have.
1713542979
Hero Member
*
Offline Offline

Posts: 1713542979

View Profile Personal Message (Offline)

Ignore
1713542979
Reply with quote  #2

1713542979
Report to moderator
1713542979
Hero Member
*
Offline Offline

Posts: 1713542979

View Profile Personal Message (Offline)

Ignore
1713542979
Reply with quote  #2

1713542979
Report to moderator
1713542979
Hero Member
*
Offline Offline

Posts: 1713542979

View Profile Personal Message (Offline)

Ignore
1713542979
Reply with quote  #2

1713542979
Report to moderator
Activity + Trust + Earned Merit == The Most Recognized Users on Bitcointalk
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1713542979
Hero Member
*
Offline Offline

Posts: 1713542979

View Profile Personal Message (Offline)

Ignore
1713542979
Reply with quote  #2

1713542979
Report to moderator
1713542979
Hero Member
*
Offline Offline

Posts: 1713542979

View Profile Personal Message (Offline)

Ignore
1713542979
Reply with quote  #2

1713542979
Report to moderator
1713542979
Hero Member
*
Offline Offline

Posts: 1713542979

View Profile Personal Message (Offline)

Ignore
1713542979
Reply with quote  #2

1713542979
Report to moderator
Biffa
Legendary
*
Offline Offline

Activity: 3220
Merit: 1220



View Profile
November 11, 2017, 10:59:45 AM
 #2

That's a very informative and useful post. Thank you

Mine @ pools that pay Tx fees & don't mine empty blocks :: kanopool :: ckpool ::
Should bitmain create LPM for all models?
:: Dalcore's Crypto Mining H/W Hosting Directory & Reputation ::
NotFuzzyWarm
Legendary
*
Offline Offline

Activity: 3612
Merit: 2494


Evil beware: We have waffles!


View Profile
November 11, 2017, 08:43:14 PM
 #3

Excellent post and bookmarked!
Now if we can just find out what the s9 hash board failure modes & causes are....

- For bitcoin to succeed the community must police itself -    My info useful? Donations welcome! 1FuzzyWc2J8TMqeUQZ8yjE43Rwr7K3cxs9
 -Sole remaining active developer of cgminer, Kano's repo is here
-Support Sidehacks miner development. Donations to:   1BURGERAXHH6Yi6LRybRJK7ybEm5m5HwTr
HagssFIN
Legendary
*
Offline Offline

Activity: 2422
Merit: 1706


Electrical engineer. Mining since 2014.


View Profile WWW
November 11, 2017, 08:47:32 PM
 #4

This is very nice little guide, thank you!

Will come back to look at this, if my S9 control board bricks  Smiley

bgasolder
Copper Member
Newbie
*
Offline Offline

Activity: 24
Merit: 12


View Profile
September 29, 2018, 09:14:42 AM
Last edit: July 06, 2019, 03:49:21 AM by frodocooper
 #5

I did as shown in the video and my antminer s9 came to life Cheesy



Recovery NAND antminer s9 t9 via micro SD card
https://youtu.be/dzArOe8KuHI
technak
Newbie
*
Offline Offline

Activity: 4
Merit: 4


View Profile
July 05, 2019, 12:02:19 PM
 #6

Hi,
I am just locked my self with new secure firmware :-( and when i try boot from sd card nothing happen immediatelly as i switch last jumper to boot from sd and power on both leds green and red are constantly on :-( I am thinking about using your method to downgrade  firmware to not secure one. You mentioned tftp server could you please give us more info how to replace kernel image in nand with tftp ? I think this will be very helpfull to community as bitmain is not playing fair ... May be easiest way will be just enable ssh throught serial console but i am not sure if this is possible. Many thanks for your help. MK
Artemis3
Legendary
*
Offline Offline

Activity: 2016
Merit: 1563


CLEAN non GPL infringing code made in Rust lang


View Profile WWW
July 05, 2019, 02:05:58 PM
Merited by frodocooper (2)
 #7

Hmm this guide would be useful to recover a controller without the sd slot soldered in. I think an ftdi cable plugged to the rj45 jack would have done the same, but yeah, same thing if you plug to that 3 pin header.

Also if the nand becomes damaged, move jp4 and do everything directly from the sd card (ie load from sd into ram, then boot).

But then if you have the sd slot you might as well just use the sd card recovery method.

██████
███████
███████
████████
BRAIINS OS+|AUTOTUNING
MINING FIRMWARE
|
Increase hashrate on your Bitcoin ASICs,
improve efficiency as much as 25%, and
get 0% pool fees on Braiins Pool
technak
Newbie
*
Offline Offline

Activity: 4
Merit: 4


View Profile
July 05, 2019, 04:15:10 PM
Last edit: July 06, 2019, 03:50:49 AM by frodocooper
 #8

currently waiting for cable but I think the steps will be:
1. setup tftp server / load uImage from not secure kernel to root
2. issue tftp $0x2000000 ${kernel_image} - kernel_image = uImage
3. bootp to start kernel
4. upgrade firmware via web interface
will test it when cable arrive not sure if it will work as kernel will be from different version of firmware
MigaoMiner
Newbie
*
Offline Offline

Activity: 12
Merit: 0


View Profile
July 24, 2019, 02:57:28 AM
Last edit: July 24, 2019, 10:24:00 AM by frodocooper
 #9

Dear All,

I have connected the USB to serial converter to the S9 controller board. Connected via serial connection and got bunch of messages.
Weird is i cannot login, and the system just continue to reboot n ended at Fan detection.

What ever i type seems not received by the Controller board.

Appreciate kind advise.

regards,

MM
Mihaiinvader
Newbie
*
Offline Offline

Activity: 5
Merit: 0


View Profile
December 26, 2019, 03:48:37 PM
Last edit: December 27, 2019, 12:27:31 AM by frodocooper
 #10

Hello,

I have 2 control boards that are bricked now, my father did a firmware update from sd and during the update he reseted them and probably corrupted the boot. I want to try the steps that you presented but i dont really understand linux. All comands are done in on a linux platform ? Can i do it also on windows ? Thank you!
Matce
Jr. Member
*
Offline Offline

Activity: 38
Merit: 1


View Profile
September 03, 2021, 05:44:22 PM
 #11

Hi Folks,

I got lucky in fixing a corrupted control board, and was able to bring it back to life using the serial console and uboot.  Wanted to mention my story to open a discussion on debricking S9/T9.  Also, if anyone wants to sell me a bricked S9 I'm interested lol Smiley

The symptoms were that upon power on, the green lights lit and the ethernet port showed connection (yellow) and data (green).  Data was occasionally blinking.  Really, the device looked like it was alive but didn't mine.  It never did its earsplitting singleboardtest, never even requested an IP from my router, so I could not SSH in or access the web server.  5 second reset, and IP reporter reset didn't help either.

I decided to check out the terminal console.  I disconnected the ribbon cables and pulled the control board out of the miner, then used a $2 USB to serial converter board to connect to the RX, TX, and GND headers on the board. [NOTE: Use 3.3v mode]  I powered the board up using the PCIE-6 plug from a spare PC power supply.  I got text!

Code:
blah blah lots of miscellaneous messages blah blah hex characters
 
Copying Linux from NAND flash to RAM...

NAND read: device 0 offset 0x1100000, size 0x800000
 8388608 bytes read: OK

NAND read: device 0 offset 0x1020000, size 0x20000
 131072 bytes read: OK
## Booting kernel from Legacy Image at 02000000 ...
   Image Name:   Linux-3.14.0-xilinx-gb190cb0-dir
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3779264 Bytes = 3.6 MiB
   Load Address: 00008000
   Entry Point:  00008000

Then a complaint about invalid kernel, and it dropped into the uboot console.  Boo!  So the thing wasn't even starting to boot, but since I had terminal access from the serial port I was hopeful.  It seems that in a borked upgrade (at least this case) the kernel image is what goes bad.  We will learn more over time, I'm sure.

Here's what I did to address it
1) Studied the shit out of all the different bootup types that can be seen using the "printenv" and "?" commands in uboot.  Bitmain must have 8-10 types of boot scripts in the environment variables, accessed by the run command.  "run qspiboot", "run nandboot", etc.  Lots of interesting reading, but they all do similar things - load stuff to ram and execute.  Hey, I'm capable of doing that!  Let me jump in the mix!

2) Formatted an SD card to FAT32, and unpacked the "xilinx" folder from a valid firmware image onto it.  This gives me BOOT.bin, devicetree.tdb, rootfs.jffs2, uImage, and upgrade-marker.bin.  Everything needed for a zero-state boot except uramdisk.image.gz.  SO IF ANYONE HAS EXTRACTED THIS, PLEASE LET ME KNOW.  I'd like to make an unbricking toolkit.
NOTE: for S9 without an SD card, you're going to have to TFTP your files.  I don't think this is a big issue though, just download a TFTP server and drop the files in it's root dir.  Should work the same.

3) Stumbled my way through the SD card commands: mmc rescan, mmc list, mmc part to find out that windows format tool had called the FAT32 partition 5, so thats where my "xilinx" files were.  Why 5?  No clue, but you need to know where they are so you can tell uboot where to load from.

4) Hacked an mmc load of the kernel from SD partition 5 into the memory location uboot was expecting:  Note that I am using 0:5 to denote mmc 0, partition 5.  ${kernel_image} is a variable defined in the environment that stores the filename uImage.  You can simply type uImage into the command below, it will work fine.

Code:
fatload mmc 0:5 0x2000000 ${kernel_image}

5) Booted the little bastard using bootp.  Here I got lucky that the ramdisk and other parts of linux were not corrupt, just the kernel image.  Worse corruptions may need more surgery, hence the need for a valid ramdisk image.  But in my case, injecting a new kernel was all it needed.

6) Once it was up, connected via the web interface and flashed a new firmware image as fast as I could.  Woo!  The miner is back!  After that, it was cured and the board went back into the miner.

All in all, very simple.  But having a $2 USB to serial adapter on hand saved me the cost of a new control board, as well as the hassle and delay of shipping, waiting, dealing with bitmain lol.  Well worth it's weight in gold this evening, for sure.  Amazon or ebay, mine was called "FT232 Mini USB to Serial".  About two inches long, mini USB on one side, 6 pins on the other side for RX, TX, GND and other signals I never use.

Anyway - if anyone else has a bricked control board you want me to hack on, just PM me.  I'd like to see some other examples of how boards brick, because developing a generic "unbricking tool" would be really nice to have.

Is there any update on this?

I have a bricked S9 board, got the RX/TX/GND connected via USB to windows / putty console ... but I'm getting nothing.

thanks
BitMaxz
Legendary
*
Offline Offline

Activity: 3234
Merit: 2941


Block halving is coming.


View Profile WWW
September 03, 2021, 10:54:17 PM
 #12

Is there any update on this?

I have a bricked S9 board, got the RX/TX/GND connected via USB to windows / putty console ... but I'm getting nothing.

thanks

Take note of this

Code:
[Use 3.3v mode]  I powered the board up using the PCIE-6 plug from a spare PC power supply. 

I think he uses a Linux OS to detect the control board but try the commands above under #3 read that part.

█▀▀▀











█▄▄▄
▀▀▀▀▀▀▀▀▀▀▀
e
▄▄▄▄▄▄▄▄▄▄▄
█████████████
████████████▄███
██▐███████▄█████▀
█████████▄████▀
███▐████▄███▀
████▐██████▀
█████▀█████
███████████▄
████████████▄
██▄█████▀█████▄
▄█████████▀█████▀
███████████▀██▀
████▀█████████
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
c.h.
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
▀▀▀█











▄▄▄█
▄██████▄▄▄
█████████████▄▄
███████████████
███████████████
███████████████
███████████████
███░░█████████
███▌▐█████████
█████████████
███████████▀
██████████▀
████████▀
▀██▀▀
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!