Open Source Bitcoin ASIC miner project that uses 2x BM1387 (Antminer S9)

[n0nce]

Hey @Skot, have a look; GekkoScience just released a BM1397-based pod miner!
https://bitcointalk.org/index.php?topic=5423227.0

I'm trying to get my hands on one as soon as they're available to buy. I'd like to mine a bit on it, but would probably also be able to get some serial traces off it for our open-source project.
I'd much rather buy this than an old, used S17 which may even arrive broken.

[Skot]

Hey @Skot, have a look; GekkoScience just released a BM1397-based pod miner!
https://bitcointalk.org/index.php?topic=5423227.0

I'm trying to get my hands on one as soon as they're available to buy. I'd like to mine a bit on it, but would probably also be able to get some serial traces off it for our open-source project.
I'd much rather buy this than an old, used S17 which may even arrive broken.

Wow, that's a sweet looking machine! I'd love to try one out too. I wonder how much they'll go for.

[Skot]

Can you please push a PDF of the schematic so I can do a review.

yes, here you go; https://github.com/skot/bitaxe/blob/max_v2/bitaxeMax.pdf

About the BM1393 voltage regulation, do you plan a voltage tuning from the MCU ?

Yes, that will be new for bitaxeMax v2. I'm using the Maxim DS4432U+ current DAC with the TPS40305 -> BM1397 core voltage adjustment will be over I2C. note on the current schematic there are a couple feedback resistor values TBD.

Here is my Schematic review :
- I see DS4432U+ OUT0 being used for TPS40305 control, but does the OUT1 being used somewhere ? (if not use, better not creating a signal out of the package to give ambiguity...)
- where the 5V is generated ? I see +3V3 (U11), 1V8 (U5) and 0V8 (U6) being generated from 5V, and BM1397 VDD (alias of TPS40305 VOUT) being generated from VIN_M (the monitored VIN), but nowhere the 5V beoing generated...
- on the ESP, I think you inverted TX/RX, please double check it
- can you add pin header and TP for debug purpose on the 3V3 RX/TX/RST, (additionnaly to J3), and add also I2C SDA/SCL signal on this debug pin header ?
- can you use an optional (with a serial 0R resistor) signal from a CLK_OUT of the ESP (pin 13/14/34/33/32 up to you) going to the BM1397 CLKI instead of the U1 one. I may want to try later if MCU can generate this clock by SW so we have control on the hashrate of the BM1397...
- on the BM1397 you pulled down BI (with R4) are you sure it will not being used ? on BM1385 datasheet it is noted as a Busy Input (internal schmit trugger and pull down). Maybe you can link it to a GPIO of the MCU, we will see later if needed. (and add the 3V3 signal to the debug pin header too).
- for the voltage level translators (U2/U3/U4), I used to use TSX0104 in my previous design, it is easy to use because bidirectionnal. Did you consider them ?
- for EMC2101, the ALTER#/TACH pin can be configured as input (TACH monitor to mesure Fan RPM) or output (ALERT# signal to trigger an IRQ on MCU when special temp/tach conditons are met). I see more value having an ALERT# signal from it to the MCU to trigger an interrupt for Temperature Critical exceeded (with hysteresis to deassert the signal). The FAN RPM is not very usefull for us. Also EMC2101 porpose a Look-up Table of 8 stages so we can control by hardware FAN speed accoding to BM1397 internal temperature. Maybe what you can do, is link the EMC2101 ALERT#/TACH pin to a MCU GPIO, and the FAN TACH to another MCU GPIO (be careful of voltage level here), and SW will be able to configure EMC2101 in TACH reading for some seconds, by propagating the FAN signal, then switch back to ALERT interrupt for a longer period (1h ?). So most of the time we use Interrupt monitor, and only once in a while do a fAN RPM measurement for UI needs...
- it would be usedfull to have the BM1397 VDD on a ESP ADC input to measure the BM voltage
- it would be usefull to have the TPS40305 PGOOD linked to a ESP GPIO to know if Power is Good

I checked in an update to the schematic on the max_v2 branch with all of these changes. I'm still a little iffy on the level shifting for the interface to CLKI and the fan ALERT.

[n0nce]

Hey @Skot, have a look; GekkoScience just released a BM1397-based pod miner!
https://bitcointalk.org/index.php?topic=5423227.0

I'm trying to get my hands on one as soon as they're available to buy. I'd like to mine a bit on it, but would probably also be able to get some serial traces off it for our open-source project.
I'd much rather buy this than an old, used S17 which may even arrive broken.

Wow, that's a sweet looking machine! I'd love to try one out too. I wonder how much they'll go for.

When comparing 'pod miners', right now there are 2 options; Futurebit Apollo and R909.
The Futurebit Apollo with 2-3TH/s costs around 700 bucks incl. VAT, while the R606 from GekkoScience (R909 predecessor) cost only 400 (incl. VAT). But the 909 will also 'only' put out 1.5-2TH/s. So 400-ish would give you a similar hash/€ value, although at much higher efficiency. Roughly 100W instead of ~200W+ (should be much quieter, too, because of this). So therefore it might be sold for a bit more than the R606.
I'm wondering what number of chips it has; should be just 4-6 chips, probably on the higher side, though, as they need some pretty fine tuning to achieve 500GH/s on Compac F sticks.

I'm also excited to see these changes to cgminer by kano. We will have to look into individual chip tuning as well, once the single-chip device works and we try out the first multi-chip boards.

[Kano] has been working on adding tuning features which include setting individual chip frequencies. Core voltage is adjustable in the range of 1.4V-1.6V per chip.

[Skot]

I'm wondering what number of chips it has; should be just 4-6 chips, probably on the higher side, though, as they need some pretty fine tuning to achieve 500GH/s on Compac F sticks.

I'm also excited to see these changes to cgminer by kano. We will have to look into individual chip tuning as well, once the single-chip device works and we try out the first multi-chip boards.

I’m pretty sure the cgminer screenshot in that thread shows it’s 6 chips. The decent BM1397 (suffix AI and AG, I believe) are like $15 each new. Hopefully cheaper in quantity. I guess we’ll find out soon!

I assume individual chip tuning means changing the hash frequency separately for each chip and then seeing how it performs. AFAIK cgminer currently only lets you change the frequency of the whole chain.

[NebulaMiner]

Here is my Schematic review :
- I see DS4432U+ OUT0 being used for TPS40305 control, but does the OUT1 being used somewhere ? (if not use, better not creating a signal out of the package to give ambiguity...)
- where the 5V is generated ? I see +3V3 (U11), 1V8 (U5) and 0V8 (U6) being generated from 5V, and BM1397 VDD (alias of TPS40305 VOUT) being generated from VIN_M (the monitored VIN), but nowhere the 5V beoing generated...
- on the ESP, I think you inverted TX/RX, please double check it
- can you add pin header and TP for debug purpose on the 3V3 RX/TX/RST, (additionnaly to J3), and add also I2C SDA/SCL signal on this debug pin header ?
- can you use an optional (with a serial 0R resistor) signal from a CLK_OUT of the ESP (pin 13/14/34/33/32 up to you) going to the BM1397 CLKI instead of the U1 one. I may want to try later if MCU can generate this clock by SW so we have control on the hashrate of the BM1397...
- on the BM1397 you pulled down BI (with R4) are you sure it will not being used ? on BM1385 datasheet it is noted as a Busy Input (internal schmit trugger and pull down). Maybe you can link it to a GPIO of the MCU, we will see later if needed. (and add the 3V3 signal to the debug pin header too).
- for the voltage level translators (U2/U3/U4), I used to use TSX0104 in my previous design, it is easy to use because bidirectionnal. Did you consider them ?
- for EMC2101, the ALTER#/TACH pin can be configured as input (TACH monitor to mesure Fan RPM) or output (ALERT# signal to trigger an IRQ on MCU when special temp/tach conditons are met). I see more value having an ALERT# signal from it to the MCU to trigger an interrupt for Temperature Critical exceeded (with hysteresis to deassert the signal). The FAN RPM is not very usefull for us. Also EMC2101 porpose a Look-up Table of 8 stages so we can control by hardware FAN speed accoding to BM1397 internal temperature. Maybe what you can do, is link the EMC2101 ALERT#/TACH pin to a MCU GPIO, and the FAN TACH to another MCU GPIO (be careful of voltage level here), and SW will be able to configure EMC2101 in TACH reading for some seconds, by propagating the FAN signal, then switch back to ALERT interrupt for a longer period (1h ?). So most of the time we use Interrupt monitor, and only once in a while do a fAN RPM measurement for UI needs...
- it would be usedfull to have the BM1397 VDD on a ESP ADC input to measure the BM voltage
- it would be usefull to have the TPS40305 PGOOD linked to a ESP GPIO to know if Power is Good

I checked in an update to the schematic on the max_v2 branch with all of these changes. I'm still a little iffy on the level shifting for the interface to CLKI and the fan ALERT.

Great, did another check :
- for the CLKI signal directly on the ESP, please add a 0R (or a 33R as it is a clock) serial resistor to disconnect this signal by HW.
- on J3, maybe better to put 1V8 on this debug connector because all signals are on 1,8V level (and now the 3V3 is available on J6)
- maybe better using the 4th TSX0104 line for the CLKI signal instead of BI, and put BI on the voltage divider ? BI is low freq signal. Waht do you think about this ? I have the feeling high freq signal are better ont the TSX0104 (otherwise we should better use voltage divider for all signal from ESP to BM).
- on the Fan design, I understad you plan to use a 5V FAN directly powered by our input VIN,
    * Firstly, Fan usually generate lot of noise, we should consider filtering this noise.
    * Secondly, we will control this fan with a 3V3 level PWM, and the 5V TACH signal is directly on the EMC2101 pin. Maybe better using a TSX0102 for level converting 3V3/5V these 2 signals.
    * Finnaly, as for this dual signal we are not sure yet of the final usage, I would do a more modular (optioal) HW design using 0R resistors. For exemple:
         a/ 1 signal from the FAN TACH_3V3 (from J4 after going through TSX0102) connected to a dedicated ESP GPIO (with the 0R resistor so we can unconnect it)
         b/ 1 signal from the FAN TACH_3V3 (from J4 after going through TSX0102) connected to the EMC2101 ALERT#/TACH pin (with another 0R resistor so we can unconnect it)
         c/ 1 signal from the EMC2101 ALERT#/TACH pin connected to a dedicated ESP GPIO (diffeent from a/) (also with a 0R resistor so we can unconnect it)
      this way we can choose by HW (some 0R resistor) the logic we want to try during the development phase.

[NebulaMiner]

Hey @Skot, have a look; GekkoScience just released a BM1397-based pod miner!
https://bitcointalk.org/index.php?topic=5423227.0

I'm trying to get my hands on one as soon as they're available to buy. I'd like to mine a bit on it, but would probably also be able to get some serial traces off it for our open-source project.
I'd much rather buy this than an old, used S17 which may even arrive broken.

I would also love to have Serial trace from your pod, when possible !

The offical S17 trace is also important as there may reveal some Antminer secret on the BM1397 (Antminer FPGA control board with secret Firmware, controling Antminer ASIC using Antminer undocumented protocol). The Gekko/Kano trace will not reveal more that the Kano cgminer sourcecode is already exposing. Thats why the Braiins OS+ trace is also interesting, Braiins may use some reverted secret on the control protocol, and not published the source code (except for the Braiins OS (without +) with less feature).

I'm also excited to see these changes to cgminer by kano. We will have to look into individual chip tuning as well, once the single-chip device works and we try out the first multi-chip boards.

[Kano] has been working on adding tuning features which include setting individual chip frequencies. Core voltage is adjustable in the range of 1.4V-1.6V per chip.

This will be very instructive indeed !

[Skot]

Great, did another check :
- for the CLKI signal directly on the ESP, please add a 0R (or a 33R as it is a clock) serial resistor to disconnect this signal by HW.

there is one already, it's on the BM1397 sheet.

- on J3, maybe better to put 1V8 on this debug connector because all signals are on 1,8V level (and now the 3V3 is available on J6)

good idea. added.

- maybe better using the 4th TSX0104 line for the CLKI signal instead of BI, and put BI on the voltage divider ? BI is low freq signal. Waht do you think about this ? I have the feeling high freq signal are better ont the TSX0104 (otherwise we should better use voltage divider for all signal from ESP to BM).

I added a SN74AXC1T45 for level shifting the BM1397 clock. If we're going to try this we should prolly do it right. the SN74AXC1T45 has a max freq of 500 Mbps (and is in stock!)

- on the Fan design, I understad you plan to use a 5V FAN directly powered by our input VIN,
    * Firstly, Fan usually generate lot of noise, we should consider filtering this noise.

I looked around for some thoughts on this and didn't find more more than "it's complicated". I think we might have to try this out and see where the noise is coming from. Then we can add some bypass caps, filters, etc.

    * Secondly, we will control this fan with a 3V3 level PWM, and the 5V TACH signal is directly on the EMC2101 pin. Maybe better using a TSX0102 for level converting 3V3/5V these 2 signals.
    * Finnaly, as for this dual signal we are not sure yet of the final usage, I would do a more modular (optioal) HW design using 0R resistors. For exemple:
         a/ 1 signal from the FAN TACH_3V3 (from J4 after going through TSX0102) connected to a dedicated ESP GPIO (with the 0R resistor so we can unconnect it)
         b/ 1 signal from the FAN TACH_3V3 (from J4 after going through TSX0102) connected to the EMC2101 ALERT#/TACH pin (with another 0R resistor so we can unconnect it)
         c/ 1 signal from the EMC2101 ALERT#/TACH pin connected to a dedicated ESP GPIO (diffeent from a/) (also with a 0R resistor so we can unconnect it)
      this way we can choose by HW (some 0R resistor) the logic we want to try during the development phase.

These kinds of level conversions are exactly what the EMC2101 is supposed to solve. My hope in using it is that the ESP32 can control the fan and read the BM1397 temperature easily over I2C, without having to generate any specific timing signals.

[n0nce]

I'm wondering what number of chips it has; should be just 4-6 chips, probably on the higher side, though, as they need some pretty fine tuning to achieve 500GH/s on Compac F sticks.

I'm also excited to see these changes to cgminer by kano. We will have to look into individual chip tuning as well, once the single-chip device works and we try out the first multi-chip boards.

I’m pretty sure the cgminer screenshot in that thread shows it’s 6 chips. The decent BM1397 (suffix AI and AG, I believe) are like $15 each new. Hopefully cheaper in quantity. I guess we’ll find out soon!

I must be blind. When connecting multiple Compac F's, they show up as 1 per line.
Price-wise; sure, one ASIC is $15, but the Compac F sells for $250 with a single such chip. Yeah! Hopefully not too much.

I assume individual chip tuning means changing the hash frequency separately for each chip and then seeing how it performs. AFAIK cgminer currently only lets you change the frequency of the whole chain.

Exactly. If one chip performs bad, the other ones are usually 'restricted' by the one bad chip. With a small number of chips, it is worth tuning them individually, so it's nice that such a feature is coming to R909.

[NebulaMiner]

Great, did another check :
- for the CLKI signal directly on the ESP, please add a 0R (or a 33R as it is a clock) serial resistor to disconnect this signal by HW.

there is one already, it's on the BM1397 sheet.

- on J3, maybe better to put 1V8 on this debug connector because all signals are on 1,8V level (and now the 3V3 is available on J6)

good idea. added.

- maybe better using the 4th TSX0104 line for the CLKI signal instead of BI, and put BI on the voltage divider ? BI is low freq signal. Waht do you think about this ? I have the feeling high freq signal are better ont the TSX0104 (otherwise we should better use voltage divider for all signal from ESP to BM).

I added a SN74AXC1T45 for level shifting the BM1397 clock. If we're going to try this we should prolly do it right. the SN74AXC1T45 has a max freq of 500 Mbps (and is in stock!)

- on the Fan design, I understad you plan to use a 5V FAN directly powered by our input VIN,
    * Firstly, Fan usually generate lot of noise, we should consider filtering this noise.

I looked around for some thoughts on this and didn't find more more than "it's complicated". I think we might have to try this out and see where the noise is coming from. Then we can add some bypass caps, filters, etc.

   * Secondly, we will control this fan with a 3V3 level PWM, and the 5V TACH signal is directly on the EMC2101 pin. Maybe better using a TSX0102 for level converting 3V3/5V these 2 signals.
    * Finnaly, as for this dual signal we are not sure yet of the final usage, I would do a more modular (optioal) HW design using 0R resistors. For exemple:
         a/ 1 signal from the FAN TACH_3V3 (from J4 after going through TSX0102) connected to a dedicated ESP GPIO (with the 0R resistor so we can unconnect it)
         b/ 1 signal from the FAN TACH_3V3 (from J4 after going through TSX0102) connected to the EMC2101 ALERT#/TACH pin (with another 0R resistor so we can unconnect it)
         c/ 1 signal from the EMC2101 ALERT#/TACH pin connected to a dedicated ESP GPIO (diffeent from a/) (also with a 0R resistor so we can unconnect it)
      this way we can choose by HW (some 0R resistor) the logic we want to try during the development phase.

These kinds of level conversions are exactly what the EMC2101 is supposed to solve. My hope in using it is that the ESP32 can control the fan and read the BM1397 temperature easily over I2C, without having to generate any specific timing signals.

OK with current Schematic and your justifications.

Another idea : can you add a couple of ESP GPIO with a pullup and a dual pin header to ground, so we can use jumper to switch SW into specific mode (for exemple, mining on a pool or with a local node RPC connection,... we will discuss these mode later when designing the SW), and a couple of LED on ESP to have status from SW also.

[NebulaMiner]

OK, I just brought a T17 with only 1 hashboard fonctionnal on ebay... BM1397 serious reversing incoming...

I also received an ESP32-S3 module and the EMC2101 adafruit board, will start working on it....

By the way, Skot, can you update this thread title, it not about 2x BM1387 (Antminer S9) anymore It is misleading for new people discovering the project.

[developeralgo]

Just FYI , both ESP32-­S3-­WROOM-­1 and ESP-C3 versions with onboard antenna do tend to have disconnects WiFi issues, The range isn't that great but will work. Just watchout for that. I like ESP32-­S3-­WROOM-­1 than C3 since its slightly better , cheaper and available

[Skot]

Another idea : can you add a couple of ESP GPIO with a pullup and a dual pin header to ground, so we can use jumper to switch SW into specific mode (for exemple, mining on a pool or with a local node RPC connection,... we will discuss these mode later when designing the SW), and a couple of LED on ESP to have status from SW also.

Ooo, yes. blinkylights are always good.

I had imagined all of the configuration to happen through a webpage hosted on the ESP32, or via bluetooth. That's what the "Admin Task" is all about. But, I agree especially during this R&D phase having some buttons/jumpers UI will be useful. I'll add a couple.

[Skot]

Just FYI , both ESP32-­S3-­WROOM-­1 and ESP-C3 versions with onboard antenna do tend to have disconnects WiFi issues, The range isn't that great but will work. Just watchout for that. I like ESP32-­S3-­WROOM-­1 than C3 since its slightly better , cheaper and available

Okay, that's good to know. It will be important to have good network error handling in the firmware, in addition to a decent antenna.

[Skot]

OK, I just brought a T17 with only 1 hashboard fonctionnal on ebay... BM1397 serious reversing incoming...

I also received an ESP32-S3 module and the EMC2101 adafruit board, will start working on it....

By the way, Skot, can you update this thread title, it not about 2x BM1387 (Antminer S9) anymore It is misleading for new people discovering the project.

Niiiice!!! I'm looking forward to hearing about what you learn. This is going to be cool!

I didn't start this thread, so I don't think I can change the title.

[developeralgo]

Skot,  i got a few hours to continue working on the 4 x BM1397 chip design schematic , Do you know, if  using IO Level Shifter TXB0104RGYR and the SN74AXC1T45 (Single-Bit Dual-Supply Bus Transceiver With Configurable Voltage translation) can we just use one of each (1 x TXB0104RGYR and the 1 x SN74AXC1T45) to work with multiple 4 x BM1397 chips ? or each BM1397 chip will need a pair of this parts ?

[Skot]

Skot,  i got a few hours to continue working on the 4 x BM1397 chip design schematic , Do you know, if  using IO Level Shifter TXB0104RGYR and the SN74AXC1T45 (Single-Bit Dual-Supply Bus Transceiver With Configurable Voltage translation) can we just use one of each (1 x TXB0104RGYR and the 1 x SN74AXC1T45) to work with multiple 4 x BM1397 chips ? or each BM1397 chip will need a pair of this parts ?

just one set of level shifters for the whole chain.

The BM1397 has the ability to cascade the data from one chip to the next. ie you can connect the "CO" or "Command Output" pin of one chip to the "CI" or "Command Input" of the next. This works for all of the data pins and the clock. The BM1397 even has 2 different modes so you can change the function of the pins to make routing easier. With this in mind we don't need to level shift between BM1397s. Just when they interface with the ESP32 (which is 3.3V only)

On the Antminers they power several BM1397s in series so they don't have to deal with the low voltages as much. I'm not so sure this is the best way to go for a relatively low chip count miner. You run into problems with the exposed copper top of the BM1397 then being at different voltages (and a heatsink shorting them out). It also makes it more difficult to change the core voltage of individual chips.

In true Bitmain fashion there is almost no documentation. Just what we can figure out from the various repair guides and poking around Antminer PCBs. I've slowly been trying to gather up everything I've found and figured out here; https://github.com/skot/BM1397

[NotFuzzyWarm]

On the Antminers they power several BM1397s in series so they don't have to deal with the low voltages as much. I'm not so sure this is the best way to go for a relatively low chip count miner. You run into problems with the exposed copper top of the BM1397 then being at different voltages (and a heatsink shorting them out). It also makes it more difficult to change the core voltage of individual chips.

It was either FriedCat with his ASIC Tube miner or Bitfury that 1st used the string topology. The obvious reason is that it eliminated the need for multiple Vcore regulators giving higher efficiency but that came at the price of needing more level shifters and not being able to fine-tune the voltage for each chip in a string - the chip needing the highest Vcore is what sets the voltage for the entire string.

So ja, for just a few chips, either run them all in parallel from 1 regulator or use multiple regulators with 1 per pair of chips as done in Bitmains' S1-S4 miners.

[developeralgo]

So ja, for just a few chips, either run them all in parallel from 1 regulator or use multiple regulators with 1 per pair of chips as done in Bitmains' S1-S4 miners.

Only working with 4 x BM1397 in Series and i think 1 regulator should be able to do the job. The BM1397AG are ok depending on Binning but should be close enough in performance to use 1 regulator. If its a mix of BM1397xx Chips then use of multiple regulators would be ideal

[NotFuzzyWarm]

Only working with 4 x BM1397 in Series

This is where being clear on terminology used is crucial: When you say 'series' I assume you are talking about daisy-chaining the data chain and the Vcore is feeding power to the 4x chips in parallel right? I recall seeing that the s17 chip can pull up to 15A(?) so x4 in parallel that means the regulator will be supplying up to 60 amps. Those better be some pretty hefty power planes...