Open Source Bitcoin ASIC miner project that uses 2x BM1387 (Antminer S9)

[NebulaMiner]

I don't have BM1397 chip nor a S17 on hand, only a S9k (BM1393 chip) and some defective S9 Hashboard (BM1387), maybe you can do some Logic Analyzer (Saleae Logic 2?) log of the Serial communication from cgminer and even better from a real S17. A good way to log the Serial comm is to use female HE10 connector directly pressed on the flat cable between control board and hash board : https://i.ibb.co/yyTyRV5/PXL-20221122-173823822-Copie.jpg.

I am starting a Saleae Logic 2 High Level Analyzer https://support.saleae.com/extensions/high-level-analyzer-extensions based on the BM1385 datasheet, the BM1393 log I have, and cgminer source code for BM1397.

Probably the easiest way to get a BM1397 to start playing around with is to get one of the Gekkoscience Compac F. https://bitcoinmerch.com/products/gekkoscience-compac-f-fan-upgrade-combo-up-to-270gh-s The Compac F is super cool, and now it's available again thanks to sidehack's continued efforts.

Saleae Logic Analyzers are also amazing. I get a ton of use out of mine. From a data standpoint, the Compac F is a FTDI usbserial chip directly attached to a BM1397. You can connect with a serial terminal and prod registers to your heart's content.

I put some of my notes/logs with the BM1397 here: https://github.com/skot/ESP-Miner/blob/master/bm1397_protocol.md

I will buy a Compac F at some point, but seeing my drawers full of stuff "I will use somedays", I tend to first work on a project and then buy the hardware...
So you also have a Saleae, great ! Can you provide me .logic files (Logic 2) of some serial communication ?

Another great source of information was the Braiins OS that supported S17 miner, it was open source and written in Rust, but it look like Braiins did remove it from Github... I may have cloned it somewhere on my harddisk...

oooo that would be amazing to see.

Found it, Github never forget ! https://github.com/xertSuns1/braiins/blob/bos-devel/open/hw/zynq-io-am1-s9/design/src/ip_cores/axi_bm13xx/hdl/bm13xx_S_AXI.vhd

Also, about the I2C temperature sensor NCT218, do you think we will keep this ref, and I can start working on a driver, or will you change the source ?

the NCT218 went out of stock. I just switched to the EMC2101. It does fan control too! Hooray integrated products from the PC motherboard world. Adafruit has some driver examples to work from; https://learn.adafruit.com/emc2101-fan-controller-and-temperature-sensor

OK, let's go with EMC2101 then.

About the Firmware source, I am working in Rust because :
- the embedded abstraction is very well supported for ESP32-C3, and in the future, we can change the MCU easily
- stratum v1 and v2 are already coded in Rust by Braiins https://github.com/braiins/braiins-open/tree/master/protocols/stratum and because v2 use Noise Protocol, it is better to not have to code it ourself
- the cgminer source code is not very clean (many type mixing) and not very adapted to embedded system
- will be booting/running very fast and very reliable

From what I hear, embedded Rust is the new hotness. Are you pretty good at it? I'd love to learn.
It seems like the ESP32-C3 modules are all lacking in RAM. We're going to need a lot to deal with the big stratum ASCII JSON messages. I was thinking about switching to the ESP32-S3 for the time being. I assume that's still decently supported in Rust?

I hear you on the cgminer source. I feel like I'm going cross-eyed. In their defense, I think it's had a ton of authors over the years.

All ESP32 are pretty well supported by Rust HAL https://github.com/esp-rs/esp-hal
The good point is by using embedded-hal traits, we will be able to change HW easily. It will be possible for example to change for a Cortex-M + W5500 for Ethernet instead of Wifi (more reliable, quicker comm and easier to setup (no SSID connection, only DHCP needed)), or even better the W7500 (all in one chip for Ethernet).
I am intermediary level at Rust, but I learn it every day, it is fascinating.

My cgminer remark was not intended to be disrepectfull, but as you said, so many developer contributed over the year, it lost a bit of homogeneity...

[Skot]

I will buy a Compac F at some point, but seeing my drawers full of stuff "I will use somedays", I tend to first work on a project and then buy the hardware...
So you also have a Saleae, great ! Can you provide me .logic files (Logic 2) of some serial communication ?

I will check it out, but I'm not sure it's going to be very interesting. The serial is just 1.8V TTL. The BM1397 seems to default to 3Mbaud and then can be set to whatever you like with the baudrate register. Parsing can easily be done in C. (Or Python or Rust or whatever can connect to your serial port). There is also a reset line. that works like you might expect.

here are some of my experiments in Python with the BM1387: https://github.com/skot/bm1387_scripts

Found it, Github never forget ! https://github.com/xertSuns1/braiins/blob/bos-devel/open/hw/zynq-io-am1-s9/design/src/ip_cores/axi_bm13xx/hdl/bm13xx_S_AXI.vhd

Ooo very cool. It's been a while since I've looked at VHDL. Are you able to pick out what the BM1397 registers are?

All ESP32 are pretty well supported by Rust HAL https://github.com/esp-rs/esp-hal
The good point is by using embedded-hal traits, we will be able to change HW easily. It will be possible for example to change for a Cortex-M + W5500 for Ethernet instead of Wifi (more reliable, quicker comm and easier to setup (no SSID connection, only DHCP needed)), or even better the W7500 (all in one chip for Ethernet).
I am intermediary level at Rust, but I learn it every day, it is fascinating.

Ok, that's great! Ethernet would be cool. The ESP32-S3 doesn't have a builtin ethernet MAC, but some of the other ESP32 varieties do. I like the idea of trying to make WiFi work though. It's not too hard to setup if you have a proper provisioning tool. I also have an extended goal of mounting this thing to a solar panel and putting it on my roof for true wireless (day) mining.

[n0nce]

Hey Skot; I've had a look at the updated schematics of bitaxe max_v2. I believe some components need footprint assignments, but other than that the PSU circuit looks to be connected to the ASIC chip already, right? If I understand correctly, that's why not all components transfer over to the PCB layout when refreshing it.
There are a few small DRC violations, but that can be fixed quickly.

The max_v2 branch is still WIP. I just pushed a commit with some TODO notes included in the schematic.

I may have gotten a little carried away with changes on this rev!

I've ordered the PCBs and components for power supply and bitaxe! Excited to receive them soon and start looking at the firmware. I may experiment with the code already; the whole Stratum stuff is completely independent from the hashboard anyway. We'll use ESP32-S3 right? Not sure whether it's pin-compatible with the bitaxe I ordered, but worst-case I'll jumper it up.

Another great source of information was the Braiins OS that supported S17 miner, it was open source and written in Rust, but it look like Braiins did remove it from Github... I may have cloned it somewhere on my harddisk...

oooo that would be amazing to see.

Maybe it's all archived? https://web.archive.org/web/20210123084147/https://github.com/braiins/braiins
Or do you have something else in mind @NebulaMiner

I will buy a Compac F at some point, but seeing my drawers full of stuff "I will use somedays", I tend to first work on a project and then buy the hardware...
So you also have a Saleae, great ! Can you provide me .logic files (Logic 2) of some serial communication ?

Just some traces of the Compac F's serial comms? I can try; not so easy without broken out test pads.. ^^

Found it, Github never forget ! https://github.com/xertSuns1/braiins/blob/bos-devel/open/hw/zynq-io-am1-s9/design/src/ip_cores/axi_bm13xx/hdl/bm13xx_S_AXI.vhd

HDL sources, that's neat!

[Skot]

I've ordered the PCBs and components for power supply and bitaxe! Excited to receive them soon and start looking at the firmware. I may experiment with the code already; the whole Stratum stuff is completely independent from the hashboard anyway. We'll use ESP32-S3 right? Not sure whether it's pin-compatible with the bitaxe I ordered, but worst-case I'll jumper it up.

Yes, let's go with the ESP32-S3 for now. specifically the ESP32-S3-WROOM-1 module. I think the extra RAM and second core will be handy. Espressif's dev board with this module is called the ESP32-S3-DevKitC-1 (https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/hw-reference/esp32s3/user-guide-devkitc-1.html). It's not pin compatible with the pinout for the ESP32-C3-DevKitC on the bitaxeMax that you've ordered. Hopefully it won't be to hard to jumper up.

[n0nce]

I've ordered the PCBs and components for power supply and bitaxe! Excited to receive them soon and start looking at the firmware. I may experiment with the code already; the whole Stratum stuff is completely independent from the hashboard anyway. We'll use ESP32-S3 right? Not sure whether it's pin-compatible with the bitaxe I ordered, but worst-case I'll jumper it up.

Yes, let's go with the ESP32-S3 for now. specifically the ESP32-S3-WROOM-1 module. I think the extra RAM and second core will be handy. Espressif's dev board with this module is called the ESP32-S3-DevKitC-1 (https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/hw-reference/esp32s3/user-guide-devkitc-1.html). It's not pin compatible with the pinout for the ESP32-C3-DevKitC on the bitaxeMax that you've ordered. Hopefully it won't be to hard to jumper up.

No worries, I'll get one of those ESP32-S3-WROOM-1 dev kits and use jumpers!
The C3 is a cool module, but it's kind of an ESP8266 successor, as it has less 'horsepower' than even the old ESP32's. S3 should be the current 'flagship'.

[dj789]

Thanks to everyone who's been helping with this project - it's great to see so many people interested in this.

I just got some Bitaxe v2 (2xBM1387) boards (which took much longer than i was expecting).  They will keep me busy for a little while, but i'm looking forward to testing the next BM1397 boards.

BTW the SN74LVC1T45QDCKRQ1 is now out-of-stock on digikey.

[NebulaMiner]

EMC2101 done in Rust https://github.com/Georges760/emc2101-rs, I need to get some HW and test/debug it now...

BM13xx driver is ongoing, but I really need more log sample from Saleae Logic 2 to revert engineer the protocol (specially from a real S17).

[n0nce]

Thanks to everyone who's been helping with this project - it's great to see so many people interested in this.

I just got some Bitaxe v2 (2xBM1387) boards (which took much longer than i was expecting).  They will keep me busy for a little while, but i'm looking forward to testing the next BM1397 boards.

That's cool! I believe you should be able to test ESP32 code with the bitaxe v2, as well; a big part of the code should be going to work across chips. The whole Stratum connection and serial set-up; not sure whether the message format changed. @Skot has looked more into that so far, as far as I know.

BTW the SN74LVC1T45QDCKRQ1 is now out-of-stock on digikey.

It's on LCSC though; had to order from 2 places myself, too - that's the current supply chain struggle, I guess... :/

[Skot]

Thanks to everyone who's been helping with this project - it's great to see so many people interested in this.

I just got some Bitaxe v2 (2xBM1387) boards (which took much longer than i was expecting).  They will keep me busy for a little while, but i'm looking forward to testing the next BM1397 boards.

BTW the SN74LVC1T45QDCKRQ1 is now out-of-stock on digikey.

That’s so cool you ordered some bitaxe boards! Open source HW for the win! Let me know if you have any issues building them up.

This parts shortage thing is a pain. There are a zillion level shifters out there, but all in different packages.

[Skot]

EMC2101 done in Rust https://github.com/Georges760/emc2101-rs, I need to get some HW and test/debug it now...

BM13xx driver is ongoing, but I really need more log sample from Saleae Logic 2 to revert engineer the protocol (specially from a real S17).

Wow, Looking good! A friend was telling me that you don’t really need a RTOS with Rust because the async libraries are so good.. does that seem right to you?

I wish I had a S17 to poke around on. I’ve been watching them on eBay, but they seem so expensive still.

[NotFuzzyWarm]

Regarding that level shifter, octopart shows that 10's of thousands are available though not from the usual suspects 

[NebulaMiner]

EMC2101 done in Rust https://github.com/Georges760/emc2101-rs, I need to get some HW and test/debug it now...

BM13xx driver is ongoing, but I really need more log sample from Saleae Logic 2 to revert engineer the protocol (specially from a real S17).

Wow, Looking good! A friend was telling me that you don’t really need a RTOS with Rust because the async libraries are so good.. does that seem right to you?

I wish I had a S17 to poke around on. I’ve been watching them on eBay, but they seem so expensive still.

Exactly, I plan to play around with RTIC (Real Time Interrupt-driven Concurrency) which should fit very well with our needs here : each "tasks" can be triggered by a hardware interrupt (the Alert signal from EMC2101 to trigger the shutdown for Temperature Critial reached, the Serial RX IRQ from BM1397 for a work result, etc...)

[Skot]

Exactly, I plan to play around with RTIC (Real Time Interrupt-driven Concurrency) which should fit very well with our needs here : each "tasks" can be triggered by a hardware interrupt (the Alert signal from EMC2101 to trigger the shutdown for Temperature Critial reached, the Serial RX IRQ from BM1397 for a work result, etc...)

Wow! That sounds amazing. I’m getting really excited for this — it sounds like embedded Rust is the way to go.

Do you think you could put together a ESP32-S3 Rust project? Maybe with a little guidance for others on how to setup the toolchain, IDE, etc?

I’ve made some progress on bitaxeMax v2; https://github.com/skot/bitaxe/tree/max_v2 the schematic is pretty much done. I’ll keep working on this and hopefully we can get you some real HW soon!

[NebulaMiner]

Do you think you could put together a ESP32-S3 Rust project? Maybe with a little guidance for others on how to setup the toolchain, IDE, etc?

I’ve made some progress on bitaxeMax v2; https://github.com/skot/bitaxe/tree/max_v2 the schematic is pretty much done. I’ll keep working on this and hopefully we can get you some real HW soon!

Yes of course, I just ordered a ESP32-S3 module to try a prototype of a project on it. Setup will be very easy, coding on Rust will need a little bit of adaptation (learning curve). I propose other people in this project to start a standard C project, and I will work on a Rust one in parrallel. I will teach anyone who want to try embedded Rust with me.

Can you please push a PDF of the schematic so I can do a review.

About the BM1393 voltage regulation, do you plan a voltage tuning from the MCU ?

[Skot]

Do you think you could put together a ESP32-S3 Rust project? Maybe with a little guidance for others on how to setup the toolchain, IDE, etc?

I’ve made some progress on bitaxeMax v2; https://github.com/skot/bitaxe/tree/max_v2 the schematic is pretty much done. I’ll keep working on this and hopefully we can get you some real HW soon!

Yes of course, I just ordered a ESP32-S3 module to try a prototype of a project on it. Setup will be very easy, coding on Rust will need a little bit of adaptation (learning curve). I propose other people in this project to start a standard C project, and I will work on a Rust one in parrallel. I will teach anyone who want to try embedded Rust with me.

okay, perfect. I have started poking around with the standard C / ESP-IDF / FreeRTOS route, but I am definitely onboard for doing it in Rust!

Can you please push a PDF of the schematic so I can do a review.

yes, here you go; https://github.com/skot/bitaxe/blob/max_v2/bitaxeMax.pdf

About the BM1393 voltage regulation, do you plan a voltage tuning from the MCU ?

Yes, that will be new for bitaxeMax v2. I'm using the Maxim DS4432U+ current DAC with the TPS40305 -> BM1397 core voltage adjustment will be over I2C. note on the current schematic there are a couple feedback resistor values TBD.

[sidehack]

Dang, I might have to hire one of you guys for some embedded dev on a new project for Gekko.

[NebulaMiner]

Can you please push a PDF of the schematic so I can do a review.

yes, here you go; https://github.com/skot/bitaxe/blob/max_v2/bitaxeMax.pdf

About the BM1393 voltage regulation, do you plan a voltage tuning from the MCU ?

Yes, that will be new for bitaxeMax v2. I'm using the Maxim DS4432U+ current DAC with the TPS40305 -> BM1397 core voltage adjustment will be over I2C. note on the current schematic there are a couple feedback resistor values TBD.

Here is my Schematic review :
- I see DS4432U+ OUT0 being used for TPS40305 control, but does the OUT1 being used somewhere ? (if not use, better not creating a signal out of the package to give ambiguity...)
- where the 5V is generated ? I see +3V3 (U11), 1V8 (U5) and 0V8 (U6) being generated from 5V, and BM1397 VDD (alias of TPS40305 VOUT) being generated from VIN_M (the monitored VIN), but nowhere the 5V beoing generated...
- on the ESP, I think you inverted TX/RX, please double check it
- can you add pin header and TP for debug purpose on the 3V3 RX/TX/RST, (additionnaly to J3), and add also I2C SDA/SCL signal on this debug pin header ?
- can you use an optional (with a serial 0R resistor) signal from a CLK_OUT of the ESP (pin 13/14/34/33/32 up to you) going to the BM1397 CLKI instead of the U1 one. I may want to try later if MCU can generate this clock by SW so we have control on the hashrate of the BM1397...
- on the BM1397 you pulled down BI (with R4) are you sure it will not being used ? on BM1385 datasheet it is noted as a Busy Input (internal schmit trugger and pull down). Maybe you can link it to a GPIO of the MCU, we will see later if needed. (and add the 3V3 signal to the debug pin header too).
- for the voltage level translators (U2/U3/U4), I used to use TSX0104 in my previous design, it is easy to use because bidirectionnal. Did you consider them ?
- for EMC2101, the ALTER#/TACH pin can be configured as input (TACH monitor to mesure Fan RPM) or output (ALERT# signal to trigger an IRQ on MCU when special temp/tach conditons are met). I see more value having an ALERT# signal from it to the MCU to trigger an interrupt for Temperature Critical exceeded (with hysteresis to deassert the signal). The FAN RPM is not very usefull for us. Also EMC2101 porpose a Look-up Table of 8 stages so we can control by hardware FAN speed accoding to BM1397 internal temperature. Maybe what you can do, is link the EMC2101 ALERT#/TACH pin to a MCU GPIO, and the FAN TACH to another MCU GPIO (be careful of voltage level here), and SW will be able to configure EMC2101 in TACH reading for some seconds, by propagating the FAN signal, then switch back to ALERT interrupt for a longer period (1h ?). So most of the time we use Interrupt monitor, and only once in a while do a fAN RPM measurement for UI needs...
- it would be usedfull to have the BM1397 VDD on a ESP ADC input to measure the BM voltage
- it would be usefull to have the TPS40305 PGOOD linked to a ESP GPIO to know if Power is Good

[DaCryptoRaccoon]

Just spotted this topic fantastic project looking forward to seeing the final product on this.

Also fantastic to see open source project like this!

Keep up the good work!

[Skot]

Wow, thank you for this excellent review! this helps a lot. I've added my thoughts inline below.

Here is my Schematic review :
- I see DS4432U+ OUT0 being used for TPS40305 control, but does the OUT1 being used somewhere ? (if not use, better not creating a signal out of the package to give ambiguity...)

The DS4432 is a dual channel part. We only need one channel. I didn't see a cheaper/better stocked current DAC alternative. The DS4432 datasheet doesn't have much to say about unused outputs. my guess is that you leave the unused channel floating and make sure not to enable it in software. (They default to off on startup)

- where the 5V is generated ? I see +3V3 (U11), 1V8 (U5) and 0V8 (U6) being generated from 5V, and BM1397 VDD (alias of TPS40305 VOUT) being generated from VIN_M (the monitored VIN), but nowhere the 5V beoing generated...

The input to the bitaxeMax is 5V. that happens at J1 and J2. There isn't any protection happening yet. Maybe that's a good idea? Any suggestions on how you would do that? I created a github issue for this. https://github.com/skot/bitaxe/issues/12

- on the ESP, I think you inverted TX/RX, please double check it

wow, oops! nice catch.

- can you add pin header and TP for debug purpose on the 3V3 RX/TX/RST, (additionnaly to J3), and add also I2C SDA/SCL signal on this debug pin header ?

sure

- can you use an optional (with a serial 0R resistor) signal from a CLK_OUT of the ESP (pin 13/14/34/33/32 up to you) going to the BM1397 CLKI instead of the U1 one. I may want to try later if MCU can generate this clock by SW so we have control on the hashrate of the BM1397...

The BM1397 has a decent PLL hashrate clock that is configurable through a couple registers. I think this is the traditional way to change the hashrate (with the corresponding core voltage change). But I agree it would be interesting to play around with this.. I'll add it.

- on the BM1397 you pulled down BI (with R4) are you sure it will not being used ? on BM1385 datasheet it is noted as a Busy Input (internal schmit trugger and pull down). Maybe you can link it to a GPIO of the MCU, we will see later if needed. (and add the 3V3 signal to the debug pin header too).

I'm pretty sure it's not needed, at least on a single ASIC chain. But if we switch to the TSX0104 level shifter, there is a spare channel, so I can hook this up.

- for the voltage level translators (U2/U3/U4), I used to use TSX0104 in my previous design, it is easy to use because bidirectionnal. Did you consider them ?

I initially avoided it because I remember it being kinda slow. But now that I'm looking at the datasheet I see that's for open-drain mode (like for I2C). In push-pull mode it's 24Mbps, which should be fine for us. I see it's in stock too -- I'll switch to it. thanks for the reccomendation

- for EMC2101, the ALTER#/TACH pin can be configured as input (TACH monitor to mesure Fan RPM) or output (ALERT# signal to trigger an IRQ on MCU when special temp/tach conditons are met). I see more value having an ALERT# signal from it to the MCU to trigger an interrupt for Temperature Critical exceeded (with hysteresis to deassert the signal). The FAN RPM is not very usefull for us. Also EMC2101 porpose a Look-up Table of 8 stages so we can control by hardware FAN speed accoding to BM1397 internal temperature. Maybe what you can do, is link the EMC2101 ALERT#/TACH pin to a MCU GPIO, and the FAN TACH to another MCU GPIO (be careful of voltage level here), and SW will be able to configure EMC2101 in TACH reading for some seconds, by propagating the FAN signal, then switch back to ALERT interrupt for a longer period (1h ?). So most of the time we use Interrupt monitor, and only once in a while do a fAN RPM measurement for UI needs...

I feel like knowing the fan RPM is important, at the very least for feedback -- is the fan going the speed we set? And for doing a sanity check on the fan -- is it connected and working? I see what you are saying about the ALERT function though. I'll hook it up to a MCU GPIO with a voltage divider.

- it would be usedfull to have the BM1397 VDD on a ESP ADC input to measure the BM voltage

good idea, i'll add that

- it would be usefull to have the TPS40305 PGOOD linked to a ESP GPIO to know if Power is Good

good idea, i'll add that

[n0nce]

I wish I had a S17 to poke around on. I’ve been watching them on eBay, but they seem so expensive still.

What about getting one with a few broken hashboards (or just 1 working one)? That should suffice, right. Might be a lot cheaper.

Do you think you could put together a ESP32-S3 Rust project? Maybe with a little guidance for others on how to setup the toolchain, IDE, etc?

I’ve made some progress on bitaxeMax v2; https://github.com/skot/bitaxe/tree/max_v2 the schematic is pretty much done. I’ll keep working on this and hopefully we can get you some real HW soon!

Yes of course, I just ordered a ESP32-S3 module to try a prototype of a project on it. Setup will be very easy, coding on Rust will need a little bit of adaptation (learning curve). I propose other people in this project to start a standard C project, and I will work on a Rust one in parrallel. I will teach anyone who want to try embedded Rust with me.

I'd follow the Rust branch; my experience is limited, but existent, though just plain Rust and not embedded.

- on the ESP, I think you inverted TX/RX, please double check it

wow, oops! nice catch.

Such a classic! TX / RX are relative to the component you're looking at..