Can a newbie develop custom S19 firmware?

[tightcable]

Recently laid off from a job and now have quite a bit of free time. I am in a good spot to work on something interesting before searching for new jobs and have a goal to write a custom firmware for my S19k Pro. I have decent software engineering experience and have worked with RaspberryPis in the past, but have never worked directly with any of the Antminer hardware. Currently, I am in a position where I don't even know how much I don't know, but I am all up for banging my head against a wall for a couple of months.

I am not interested in monetizing this in any capacity and will keep it fully open-sourced (github link will be posted once that exists).

Right now I am in the very beginning stages trying to identify all of the unknowns (which is a lot), but plan to update here periodically on my progress.

I am hoping for advice and to connect with bitcoin nerds, but am also here for the discouragement

[mikeywith]

but am also here for the discouragement

Great, then allow me.

There are two paths that can be taken, but neither seems pretty.

A- Fork Cgminer, alter some or most of the code, change the UI, and you get your custom firmware like 99% of the firmware distributions out there.
B- Start from scratch like Braiins, that's a huge project that you can't do on your own, and will take many years to finish.

for A, you can't really make any money off of it unless you do it illegally, because Cgmier is protected by open-source GPL3, for B, you are going to need a lot of funding.

However, what is the end goal here? if you want to give to the community, nobody gives a fuck, people are either happy with the stock firmware, or use custom firmware and pay a small % of their rewards.

There isn't much that can be added to the mining industry, it's too saturated and all the improvement is now in the hands of those who design and manufacture the chips, if I were you I would look into developing something that I could monetize in the future, either by directly selling it or by using the skills I learn along the way to get a decent job, I don't know what that is, maybe AI related, anything that has a great potential which mining does not.

[tightcable]

However, what is the end goal here? if you want to give to the community, nobody gives a fuck, people are either happy with the stock firmware, or use custom firmware and pay a small % of their rewards.

I do not really know exactly what I am doing yet, so I can't say for sure what my end goal will be. For now, I am picturing a firmware I can put on my CVITEK or Epic board that will successfully complete a share on my s19k. I do not really have any concrete motivations for this project (helping the community, making money, etc). It is more of a hyper fixation that will most likely last 3-6 months.


What do I want to get out of it?

A big part of it is to see if I can. Another part of it is to hopefully collaborate with interesting people.

I do not see myself taking this to a marketable product like the other firmware distributions. Best case scenario, I get this to 80% of what I wanted to do, my hyper fixation other goes away or shifts to something else, then at some point down the road maybe another engineer will find some value in it.

if I were you I would look into developing something that I could monetize in the future, either by directly selling it or by using the skills I learn along the way to get a decent job

Haha I can definitely see my future self pissed that I did not take this advice (maybe more discouragement will kill the fixation). However, for now I have 9 months until I get booted off my parents health insurance, decent savings, and a few contracts that will pay the bills, so I figure why not f around with something I am interested in before going back to my bread and butter, full stack development (yuck).

[NotFuzzyWarm]

Well, you could start by using the most current (and Official) cgminer from Kano's git in my sig. Add things you'd like to see and build on it.

It already supports the s19 chips (actually all BM chips up to the ones in s19). S21 support is being added by Kano. What it does not directly support is coms between cgminer and the Bitmain PSU's to monitor & set voltage. That said, cgminer DOES support setting chip voltage(s), you just need to figure out how to point it at the PSU's that Bitmain uses. The ePIC board has the hardware needed but not sure about their firmware (AFIK is closed source) so perhaps you could do some coding to cgminer to add monitoring & setting PSU voltage?

Of course cgminer is under GPL license so the right way to build on it would be to submit changes to Kano's git and if approved it can be applied as a proper Open Source update.

[mikeywith]

would be to submit changes to Kano's git and if approved it can be applied as a proper Open Source update.

Yup, that is one thing he can do, contribute to the existing kano fork, however, it seems like for many years Kano is the sole contributor of that repo, so i am not sure if there is much that needs doing.

Also, who actually benefits from it? sidehack and maybe a few usb miner's manufacturer, those do deserve it, who else? Bitmain? they maintain their own shit, other custom firmware makers?

[NotFuzzyWarm]

Yup, that is one thing he can do, contribute to the existing kano fork, however, it seems like for many years Kano is the sole contributor of that repo, so i am not sure if there is much that needs doing.
...

One thing that does need doing is adding the combined voltage and per-chip speed tuning that makes VNISH and other 3rd party closed FW attractive. It would make many folks rather happy to see them take a hit from Open Source firmware that does that.

Overall speed tuning is already in cgminer but it is not per-chip and starting with Sidehack's A1 series voltage control has been implemented but AFAIK it is not part of the tuning process.

[tightcable]

What it does not directly support is coms between cgminer and the Bitmain PSU's to monitor & set voltage. That said, cgminer DOES support setting chip voltage(s), you just need to figure out how to point it at the PSU's that Bitmain uses.

This seems like a pretty good place to start.

It looks like coms between control board and PSU (APW12) are through i2c (https://www.zeusbtc.com/manuals/Antminer-APW12-Power-Supply-Repair-Guide.asp). My first thought is to rig two raspis in between the control board and PSU that capture/pass along messages, then try to reverse engineer with those logs.

Does that seem feasible?

Also, where does this lay in terms of what is "allowed"? I feel like if I do this with the stock firmware, which is in breach of the GPL3, there's no harm done. But what if I use epic firmware, which may or may not be in breach of GPL3?

[NotFuzzyWarm]

Also, where does this lay in terms of what is "allowed"? I feel like if I do this with the stock firmware, which is in breach of the GPL3, there's no harm done. But what if I use epic firmware, which may or may not be in breach of GPL3?

Good question. ePIC's git is here https://github.com/epicblockchain but I do not see any raw code there so I take it is closed source?

Kano blocks miners using VNISH et al from using his pool because they further violate his GPL3 license  However he DOES allow systems based on the ePIC boards so I have to assume ePIC did their own full rewrite of cgminer like Braiins (also allowed) did.

Und before you ask, yes, pretty much all miners from Bitmain, MicroBT, Innosilicon et al are using hacked closed versions of cgminer: Why does he allow them on his pool using their OEM firmware? Because he has to otherwise there would be almost no miners there... That said, the 3rd party vendors in their taking that to another level - those he can and does ban.

You can always join Kano's Discord channel (link to invite in my sig) or just PM him here to directly talk with him...

[Nexus9090]

but am also here for the discouragement

Great, then allow me.

There are two paths that can be taken, but neither seems pretty.

A- Fork Cgminer, alter some or most of the code, change the UI, and you get your custom firmware like 99% of the firmware distributions out there.
B- Start from scratch like Braiins, that's a huge project that you can't do on your own, and will take many years to finish.

for A, you can't really make any money off of it unless you do it illegally, because Cgmier is protected by open-source GPL3, for B, you are going to need a lot of funding.

However, what is the end goal here? if you want to give to the community, nobody gives a fuck, people are either happy with the stock firmware, or use custom firmware and pay a small % of their rewards.

There isn't much that can be added to the mining industry, it's too saturated and all the improvement is now in the hands of those who design and manufacture the chips, if I were you I would look into developing something that I could monetize in the future, either by directly selling it or by using the skills I learn along the way to get a decent job, I don't know what that is, maybe AI related, anything that has a great potential which mining does not.

Just out of interest, do the Antminer systems run CGMiner as part of BitMains default software stack?

Good luck to the OP., I sure wouldnt want to take on a task that big alone.

[NotFuzzyWarm]

Just out of interest, do the Antminer systems run CGMiner as part of BitMains default software stack?

Good luck to the OP., I sure wouldnt want to take on a task that big alone.

Yes, Bitmain (and MicroBT, Inno, Ebang, et al) all run hacked and closed versions of cgminer. From their s1 through s9 Bitmain at least did publish their code then closed it for all later models. Canaan also used cgminer and published their code up to their A10 series when they changed to using their FMS firmware.

[Nexus9090]

Just out of interest, do the Antminer systems run CGMiner as part of BitMains default software stack?

Good luck to the OP., I sure wouldnt want to take on a task that big alone.

Yes, Bitmain (and MicroBT, Inno. Ebang, et al) all run hacked and closed versions of cgminer. From their s1 through s9 Bitmain at least did publish their code then closed it for all later models. Canaan also used cgminer and published their code up to their A10 series when they changed to using their FMS firmware.

Thanks, I did not know that. I thought they run something proprietary. I guess now its a closed version it probably is to some extent.

[NotFuzzyWarm]

Being closed does not make it proprietary - it makes it violation of the -ck and Kano GPL3 license same as the 3rd party FW from VNISH et al is in violation of it. They just don't care.

For the miner makers, the only thing that can justifiably be called proprietary would be the ASIC chip-specific driver that is plugged into the cgminer code. If the makers wanted to they COULD have just published their tweaked cgminer code and kept the driver as a closed code module that cgminer talks to - the GPL lic allows that - but nooooo....

[mikeywith]

Thanks, I did not know that. I thought they run something proprietary. I guess now its a closed version it probably is to some extent.

It doesn't matter what they call it—the GNU General Public License (GPL) terms are very simple. The moment you distribute the binary of any GPL-protected code, you are legally required to distribute the corresponding source code. Otherwise, you can be sued for copyright violations, and there have been cases where large companies, like Cisco (IIRC), were forced by law to release the source code to the public.

The problem with Cgminer is that the original cpuminer developers don’t care about the violation of their code. CK himself doesn’t seem to care. Luke-Jr, who forked cpuminer into BFGMiner, doesn’t give a flying fuck. Every major U.S. firm that deals directly with Bitmain doesn’t seem to bother either. I think the latter has every reason to sue Bitmain for the violation—but, well, money talks.

Mind you, for years, people bashed the Chinese (Bitmain, MicroBT, etc.) and the Russians (Vnish) for using the source code without releasing it. But now, you have EU-based companies selling and distributing their own versions of Vnish. You have a Canadian distribution of Vnish. You have U.S. folks making a direct profit off Vnish. Everybody is abusing this shit to make money. 😆

[Nexus9090]

Being closed does not make it proprietary - it makes it violation of the -ck and Kano GPL3 license same as the 3rd party FW from VNISH et al is in violation of it. They just don't care.

For the miner makers, the only thing that can justifiably be called proprietary would be the ASIC chip-specific driver that is plugged into the cgminer code. If the makers wanted to they COULD have just published their tweaked cgminer code and kept the driver as a closed code module that cgminer talks to - the GPL lic allows that - but nooooo....

Very true, poor word choice on my part.

It sucks for sure, companies especially those as large as BitMain should know better than to disrespect license terms. I'm not familiar with VNISH

I'm sure the driver codes would also be classed as part of CGMiner due to the level of intigration required with the core program and thereby covered by the original licesnse, as it would be considered and adaptation of the original and not an original work in its own right.

AFAIK there's no "Driver" API for CGMiner, the code is heavily intertweaved at least it seemed to be when I last looked at it with a view to adding my own driver code to it, so much so that I gave up on the idea.

[NotFuzzyWarm]

You are right in that there is no driver API per se. However, there ARE drivers for all of the chips (and at least 1 FPGA driver) that can be used with cgminer. When you compile cgminer you tell it what chip(s) to use and only code for that, aka a driver, is compiled into it. ref https://github.com/kanoi/cgminer/blob/master/ASIC-README for details on that and switches used.
From that readme:

Unlike other software, cgminer uses direct USB communication instead of the ancient serial USB communication to be much faster, more reliable and use a lot less CPU. For this reason, setting up for mining with cgminer on these devices requires different drivers.

When Kano added support for the Bitmain s17 & s19 chips to cgminer for Sidehacks latest miners, Sidehack had to actually identify the bit streams going in & out of the chips and pass that onto Kano to translate into the needed driver. Thankfully not much has changed from the s9 chips (which BM published code for and -VH used to write drivers for Sidehack's earlier USB stick miners) so he had a good start on how the chips talk with cgminer. FYI: Working directly with the manufacturers Kano also wrote the drivers for the ancient chips from Black Arrow, BFL, and Hashfast as well as the FPGA driver.

In the US at least, case law does permit closed source drivers in otherwise open source code. Several makes of dvd/BluRay players and cable boxes are mainly open source code (Linux) with the DRM decode functions closed code. Many years ago several groups tried to fight that and all lost in the courts. The RasPi OS is another, at least with the pi5 the video driver it comes with is closed but at least you can change that.

[Nexus9090]

I use Kano's CGMiner on my rigs running on Raspberry PI's. Of all the open source miner software I've tried it was the simplest to compile and get to work the way I wanted it to. Its a complex beast for sure, at least for someone who's a novice when it comes to SW development.

My only gripe with it was it was stealing USB ports which I was using for other telemetry applications, thats when I started looking at how to make my own driver and add in the telemetry but decided it was too complicated and instead done it externally with a little bit of scripting to select only the USB ports that had miners attached, so that took care of that problem.

I'm thankful that -ck and Kano wrote it and maintained it. I think the home mining scene would probably not exist without it.

[taserz]

some people don't realize using cgminers api to work with legacy commands doesn't make it cgminer shit written in rust > shit written in C



but yes if you want to make a basic firmware sure not hard the communication between the psu is a bit tricky but you can just do what our competitors do and just reverse us same for the boot loader xD

That being said tuning features etc to be good will take a long time and you run the risk of damaging your equipment. Just be careful!

[Artemis3]

I do not really know exactly what I am doing yet, so I can't say for sure what my end goal will be. For now, I am picturing a firmware I can put on my CVITEK or Epic board that will successfully complete a share on my s19k. I do not really have any concrete motivations for this project (helping the community, making money, etc). It is more of a hyper fixation that will most likely last 3-6 months.

One thing is sure, you ain't doing anything anytime soon with the Cvitek, and i know nothing about the Epic, but oddly enough, Braiins DID release an open source control board called the BCB 100. Its funny you didn't notice and instead want to use one of the most locked down models out there.

And yes, it is a true fact that most firmware out there is infringing Copyright law by their illegal use of modified GPL software without releasing the corresponding source code changes. In particular the large asic manufacturers should be sued, but also third parties that even have the nerve to start companies or even threat with patent trolling out of selling stolen products.