[CLOSED] Bitmine CoinCraft A1 28nm chip distribution / DIY support

[hozer]

Hello Zefir,

thank you for your response and sharing what you have learned!
I was talking about a CCD200/400, yes. I'm more the software guy so I figured creating my own board, which I very wanted to do, would not make sense at all. Too much to learn from scratch and not enough time to do so. :|

I've followed your thread and posts over the last months and read that you we're trying to create such an autotune tool on your own, but more low level than I'm trying right now. As far as I remember you wanted to tune the chips individually, which would get the most out of them. I cannot do that at all, so I try to tune the individual boards inside the CCD instead, which should be easier and still would be better than nothing.
For that I though about using some existing high-level software to set a clock speed and voltage, let it hash for X minutes and evaluate the shares/stales/hw errors - and watch the temperature of course. With that, go through a list of clock speeds and voltage combinations to check where I get the best share/error/power consumption. I already did this manually to some degree, but it is very time consuming, so I wanted to automate that.

The basic idea was to use cgminer to set those settings to the CCD modules individually, in my CCD400 there are two of them.
As far as I can see, I can only set 1 clock and 1 voltage settings for the whole CCD, but I'd need to set those individually for the two modules, as they behave different - one generally needs more voltage than the other to not produce too much HW errors with the same clockspeed. Right now I have to use the conservative settings that make the "bad" module run at an acceptable performance, but it wastes potential on my second module, which could do much more.

But this is only possible if I can set the voltage and clock speed individually for each board in a CCD, using cgminer, which I think is not possible right now? Or maybe it's even not possible by design?
I tried to have a look at the A1* files in git, but I don't understand a thing of the low level stuff being done there.

I don't know for *sure* on the details of the CCD, but you should be able to modify CGminer ( driver-SPI-bitmine-A1.c ) to set voltages and clock speeds individually for each hash board. I'd be very interested in seeming some code that does this. The problem is going to be the optimal tuning point is going to change with temperature and a whole bunch of other things (including how much power each of the other boards are drawing and the corresponding voltage and/or noise on the 12v power supply lines). You're going to want some control algorithm to do the adjustment, and maybe using something from http://en.wikipedia.org/wiki/Maximum_power_point_tracking might be relevant (you want 'peak hash tracking').

The other thing to watch out for is to check the *actual accepted shares* rate, because what I've found is sometimes the reported hashrate by cgminer *looks* really good but the actual accepted shares is horrible if the chip is too hot or overclocked too much.

Now if only we could get new A1 silicon that has on-die temp sensors and maybe even a http://en.wikipedia.org/wiki/Ring_oscillator we could read from the registers to track individual chip silicon temp/performance.

[hozer]

Hello Zefir,

thank you for your response and sharing what you have learned!
I was talking about a CCD200/400, yes. I'm more the software guy so I figured creating my own board, which I very wanted to do, would not make sense at all. Too much to learn from scratch and not enough time to do so. :|

Your experience and observation is why it would benefit vendors (or maybe Innosilicon) to do a  AGPLv3 reference design board we can all experiment with. They'd sell more chips if us software/system guys had a full system we could tweak the design for.

I have been thinking about doing a single-chip A1/A2 miner design using KiCad (open source PCB tool) just for that reason.

[hozer]

Hello all,

After a few failed attempts I have decided to liquidate my remaining bitmine chips.  This endeavor ended up taking to much of our time and we made the choice to move on.  To those with working boards or under development this could be a great way to get a few extra mining chips fast.

So we have 35 untouched bitmine chips  These were purchase from Zefir a few months back.  At $3.5 per GH we are asking $3050 for them.  It would ship same day payment is received.  If you are local to the United States I would ship them overnight.  We also have 15 chips that were soldered on at one point, but have since been removed.  They didn't receive voltage spikes, it was mainly just bad board design.  Given the condition and buyer beware on these 15, if you are interested we could give them to you for $2 per GH so $750.

If anyone is interested please let me know.  If you in the United States I can give you a call for verification and peace of mind.

And here is one of our failed designs: 

http://i.imgbox.com/jCgloLVU.jpg

Thanks All.

So what failed, and would you be wiling to release the schematic and PCB under a GPL license

[zefir]

Hello Zefir,

thank you for your response and sharing what you have learned!
I was talking about a CCD200/400, yes. I'm more the software guy so I figured creating my own board, which I very wanted to do, would not make sense at all. Too much to learn from scratch and not enough time to do so. :|

I've followed your thread and posts over the last months and read that you we're trying to create such an autotune tool on your own, but more low level than I'm trying right now. As far as I remember you wanted to tune the chips individually, which would get the most out of them. I cannot do that at all, so I try to tune the individual boards inside the CCD instead, which should be easier and still would be better than nothing.
For that I though about using some existing high-level software to set a clock speed and voltage, let it hash for X minutes and evaluate the shares/stales/hw errors - and watch the temperature of course. With that, go through a list of clock speeds and voltage combinations to check where I get the best share/error/power consumption. I already did this manually to some degree, but it is very time consuming, so I wanted to automate that.

The basic idea was to use cgminer to set those settings to the CCD modules individually, in my CCD400 there are two of them.
As far as I can see, I can only set 1 clock and 1 voltage settings for the whole CCD, but I'd need to set those individually for the two modules, as they behave different - one generally needs more voltage than the other to not produce too much HW errors with the same clockspeed. Right now I have to use the conservative settings that make the "bad" module run at an acceptable performance, but it wastes potential on my second module, which could do much more.

But this is only possible if I can set the voltage and clock speed individually for each board in a CCD, using cgminer, which I think is not possible right now? Or maybe it's even not possible by design?
I tried to have a look at the A1* files in git, but I don't understand a thing of the low level stuff being done there.

The approach you are describing is similar to what I implemented as a calibration tool. It consists of a dummy cgminer that feeds the chains with reference test vectors and collects per-chip statistics over the complete parameter range. Ideally, the output of such a calibration run would provide you the optimal settings for each chip for a given power consumption / hashrate target. Unfortunately those statistics showed the unreliable characteristics I described above.

As for your high-level approach, two components to consider:
a) voltage control is completely done outside of cgminer over i2c-tools. Accessing the voltage regulator gives means to damage the chips, so for liability reasons I need to check first if and under which conditions I can publish the relevant information.
b) you are right, there is no per-board or per-chip configuration support in the A1 driver currently. This is considered useless for now, since it would lead to a configuration hell (like you don't know which slots are populated in your product and how to assign them individual configs).

If you are familiar with the cgminer source code, you can help yourself with b) easily by implementing an extension to the --bitmine-a1-options parser to consider sys_clock not as a global value but as a comma separated array and provide individual clocks for each board. For this you obviously need to be able to build your own cgminer binary, i.e. replace Bitmine's FW with a full RPi distribution including build tools. With some related howto posts in this forum this should be easy.

Good luck.

[zefir]

Hello Zefir,

thank you for your response and sharing what you have learned!
I was talking about a CCD200/400, yes. I'm more the software guy so I figured creating my own board, which I very wanted to do, would not make sense at all. Too much to learn from scratch and not enough time to do so. :|

Your experience and observation is why it would benefit vendors (or maybe Innosilicon) to do a  AGPLv3 reference design board we can all experiment with. They'd sell more chips if us software/system guys had a full system we could tweak the design for.

I have been thinking about doing a single-chip A1/A2 miner design using KiCad (open source PCB tool) just for that reason.

Well, this is a political topic meanwhile and I can not disclose as much as I would like for now. But I can tell you that this is a perfect example on how idealism was killed by bad business ethics and greed.

As this thread indicates, being open source was mine and Bitmine's goal from the beginning. As you note, the idea was to make everything public and get compensated for the engineering efforts by increased chip sales. Obviously this only worked as long as we were sure that the chip engineering contract will be respected and Bitmine remained the exclusive distributor of A1 chips. The rest of the story is well known meanwhile: chips being sold directly into the market, mass production of mining products driven by foreign open source SW, disruption of mining market.

Don't get me wrong: open source is the way to go and pays off in most cases. In my day job it took me 5 years to convince my boss to go open source and it took another 3 years to notice an economical gain for giving our work away for free. What you get back is free testing / bug fixing and maintenance, leading to a long term higher stability for long-living products. And that's the key killer argument for open source: long-term.

In our bitcoin world things are different: I am not aware of a single mining product that ever passed the prototype state - with a typical lifetime of 3-6 months there is simply not enough time to mature. Consequently, open source (at least for now) can't be economically profitable for the publisher if he has no means to prevent free-riding and plagiarism.


This went slightly off-topic, and despite all said, keep up your open source spirit - it will pay off some day.

[hozer]

Hello Zefir,

thank you for your response and sharing what you have learned!
I was talking about a CCD200/400, yes. I'm more the software guy so I figured creating my own board, which I very wanted to do, would not make sense at all. Too much to learn from scratch and not enough time to do so. :|

Your experience and observation is why it would benefit (silicon fabs) to do a  AGPLv3 reference design board we can all experiment with. They'd sell more chips if us software/system guys had a full system we could tweak the design for.

I have been thinking about doing a single-chip A1/A2 miner design using KiCad (open source PCB tool) just for that reason.

Well, this is a political topic meanwhile and I can not disclose as much as I would like for now. But I can tell you that this is a perfect example on how idealism was killed by bad business ethics and greed.

So it seems we need to work on our licensing agreements and give them more teeth. Someone will either license the http://q3u.be/patent/q3ube/ from me and I'll get paid up front, or they'll 'steal' the design, and then I will end up bankrupting or owning their company in the resulting GPL violation lawsuit a few years later.

We also have https://github.com/progranism/Open-Source-FPGA-Bitcoin-Miner and so it seems like its time to find a silicon fab that can make us a 40nm chip that fits in the A1/A2 footprint for a long-term solution.

[[gadget]]

After some distractions over the weekend, I finally have some news to report. Using the new PLL formula

Code:

sysclk = (refclk*fbdiv)/(prediv*2^(postdiv-1))

I was able to lower the sysclk to 100MHz, where the chips finally ran at a temp that did not burn my fingers when
touching them (infrared thermometer is on the way). I remember reading somewhere in this thread that chips
could be run at 200MHz without cooling, is that still true? Or does that refer to running them at idle (not under load)?

As a side note: the minimum fb_div appears to be 100. Anything lower would not PLL lock.

[Xer0]

dont want to derail you too much,
but anyone designes a simple 1-chip A1 board?

i would like to try one A2 chip on it

[intron]

dont want to derail you too much,
but anyone designes a simple 1-chip A1 board?

i would like to try one A2 chip on it

We made this single chip A1 evaluation board:



Never got it hashing, I not even have a picture
of the real board;)

[Xer0]

so you tried but it did not work?

[intron]

so you tried but it did not work?

Yes. There was some issue with initializing
the SPI chain correctly. It looked like a
chain of length '1' was not supported.

I also made a 10 chip A1 board, but never
even populated it. It all took too much time
and we moved on.

But c-cape did the coding, you should ask
him for details.

[[gadget]]

Integrated into the current cgminer version via the A1 SPI driver today. Some thoughts:

HW_RESET via Pi's GPIO needs elevated privileges and is hardware specific, so I prepared a separate binary for that. Currently HW_RESET is only done once at start, hopefully doing it this way won't be a problem.

The reference board doesn't really have any kind of detection mechanism other than probing the SPI bus, so the board selector is stubbed out.

Also, I was thinking of modifying the set_pll code so that the fb_div never drops below 100, would this be of use?

I'm not sure if these changes would be accepted into the main cgminer branch, since they are not for a commercial product, but if there is any interest, I'd be happy to share the code.

[[gadget]]

so you tried but it did not work?

Yes. There was some issue with initializing
the SPI chain correctly. It looked like a
chain of length '1' was not supported.

I also made a 10 chip A1 board, but never
even populated it. It all took too much time
and we moved on.

But c-cape did the coding, you should ask
him for details.

I can also confirm that a single chip A1 board did not function by itself. It replied to soft reset, but refused to be enumerated. It functioned fine when plugged as chip-3 into a 2 chip chain, though.

[zefir]

Integrated into the current cgminer version via the A1 SPI driver today. Some thoughts:

HW_RESET via Pi's GPIO needs elevated privileges and is hardware specific, so I prepared a separate binary for that. Currently HW_RESET is only done once at start, hopefully doing it this way won't be a problem.

The reference board doesn't really have any kind of detection mechanism other than probing the SPI bus, so the board selector is stubbed out.

Also, I was thinking of modifying the set_pll code so that the fb_div never drops below 100, would this be of use?

I'm not sure if these changes would be accepted into the main cgminer branch, since they are not for a commercial product, but if there is any interest, I'd be happy to share the code.

Commercial product or not, as soon as it is used by more than one user, it is worth being integrated. Detection would have to be fine tuned then to distinguish between them, but I can support you on that.

As for the privileged access: if I remember correctly, SPI access usually also requires root access, so it should be no problem integrating the GPIO based HW-reset functionality as part of a dedicated board selector.

Finally, as for limiting the minimum system clock to 100MHz, I have no defined response if that is really a hard limit. If you say you were not able to lock it below that, it would make sense to add a related sanity check to the code (although I doubt that this will be a use case).

I can also confirm that a single chip A1 board did not function by itself. It replied to soft reset, but refused to be enumerated. It functioned fine when plugged as chip-3 into a 2 chip chain, though.

Now this worries me somewhat, since I remember that during the very first bring-up phase we once break the chain after the first chip (shorted MISO/MOSI) and were able to work with only one chip. Are you saying that enumeration does not work at all, or the current driver failed to do so? If you had available a SPI trace of the enumeration command, that would help to ensure it is not the SW failing.

[[gadget]]

Integrated into the current cgminer version via the A1 SPI driver today. Some thoughts:

HW_RESET via Pi's GPIO needs elevated privileges and is hardware specific, so I prepared a separate binary for that. Currently HW_RESET is only done once at start, hopefully doing it this way won't be a problem.

The reference board doesn't really have any kind of detection mechanism other than probing the SPI bus, so the board selector is stubbed out.

Also, I was thinking of modifying the set_pll code so that the fb_div never drops below 100, would this be of use?

I'm not sure if these changes would be accepted into the main cgminer branch, since they are not for a commercial product, but if there is any interest, I'd be happy to share the code.

Commercial product or not, as soon as it is used by more than one user, it is worth being integrated. Detection would have to be fine tuned then to distinguish between them, but I can support you on that.

As for the privileged access: if I remember correctly, SPI access usually also requires root access, so it should be no problem integrating the GPIO based HW-reset functionality as part of a dedicated board selector.

Finally, as for limiting the minimum system clock to 100MHz, I have no defined response if that is really a hard limit. If you say you were not able to lock it below that, it would make sense to add a related sanity check to the code (although I doubt that this will be a use case).

I can also confirm that a single chip A1 board did not function by itself. It replied to soft reset, but refused to be enumerated. It functioned fine when plugged as chip-3 into a 2 chip chain, though.

Now this worries me somewhat, since I remember that during the very first bring-up phase we once break the chain after the first chip (shorted MISO/MOSI) and were able to work with only one chip. Are you saying that enumeration does not work at all, or the current driver failed to do so? If you had available a SPI trace of the enumeration command, that would help to ensure it is not the SW failing.

If any of you DIYers get to the point of starting up cgminer, give me a message and we can have a two user userbase

The devspi should be a usermode driver, so there shouldn't be a need for root. At least that's what I have observed in my adventures. GPIO (on the Pi) on the other hand is usually mmaped to lower memory, thus the best one can do is to drop privileges as soon as the mapping is established, but that would still mean starting as root. I believe this was the reason that legkodymov's bitfury driver from sometime last summer did not get incorporated into cgminer proper. But perhaps I am mistaken and there is a way to do this, in which case I'd be very happy to learn about it.

I realize that running at 100MHz is kind of silly, but one use case I can think of is if you don't have a working cooling solution but really, really want to start hashing tonight or allow self to keep working while bigger fans are on their way.

One of the boards me and mazurov built had only one of two chips on it, and jumper W1/W2 was soldered in "perpendicularly" to close the MISO and MOSI. Unfortunately, I did not save the log, and that particular board has made the ultimate sacrifice in the time since The testing was done with the simple test prog that I posted earlier, and now that I think harder about it: the chip replied to SOFT_RESET, replied to BIST_TEST (showed up as one chip chain), and I believe it replied to BIST_FIX. Now, IIRC, it did not respond to the READ_REG command that followed if it was alone, but did respond as expected if it was plugged in after (using the chaining plugs) another board as chip 3.

[hozer]

Integrated into the current cgminer version via the A1 SPI driver today. Some thoughts:

HW_RESET via Pi's GPIO needs elevated privileges and is hardware specific, so I prepared a separate binary for that. Currently HW_RESET is only done once at start, hopefully doing it this way won't be a problem.

The reference board doesn't really have any kind of detection mechanism other than probing the SPI bus, so the board selector is stubbed out.

Also, I was thinking of modifying the set_pll code so that the fb_div never drops below 100, would this be of use?

I'm not sure if these changes would be accepted into the main cgminer branch, since they are not for a commercial product, but if there is any interest, I'd be happy to share the code.

Commercial product or not, as soon as it is used by more than one user, it is worth being integrated. Detection would have to be fine tuned then to distinguish between them, but I can support you on that.

As for the privileged access: if I remember correctly, SPI access usually also requires root access, so it should be no problem integrating the GPIO based HW-reset functionality as part of a dedicated board selector.

Finally, as for limiting the minimum system clock to 100MHz, I have no defined response if that is really a hard limit. If you say you were not able to lock it below that, it would make sense to add a related sanity check to the code (although I doubt that this will be a use case).

I can also confirm that a single chip A1 board did not function by itself. It replied to soft reset, but refused to be enumerated. It functioned fine when plugged as chip-3 into a 2 chip chain, though.

Now this worries me somewhat, since I remember that during the very first bring-up phase we once break the chain after the first chip (shorted MISO/MOSI) and were able to work with only one chip. Are you saying that enumeration does not work at all, or the current driver failed to do so? If you had available a SPI trace of the enumeration command, that would help to ensure it is not the SW failing.

If any of you DIYers get to the point of starting up cgminer, give me a message and we can have a two user userbase

The devspi should be a usermode driver, so there shouldn't be a need for root. At least that's what I have observed in my adventures. GPIO (on the Pi) on the other hand is usually mmaped to lower memory, thus the best one can do is to drop privileges as soon as the mapping is established, but that would still mean starting as root. I believe this was the reason that legkodymov's bitfury driver from sometime last summer did not get incorporated into cgminer proper. But perhaps I am mistaken and there is a way to do this, in which case I'd be very happy to learn about it.

I realize that running at 100MHz is kind of silly, but one use case I can think of is if you don't have a working cooling solution but really, really want to start hashing tonight or allow self to keep working while bigger fans are on their way.

One of the boards me and mazurov built had only one of two chips on it, and jumper W1/W2 was soldered in "perpendicularly" to close the MISO and MOSI. Unfortunately, I did not save the log, and that particular board has made the ultimate sacrifice in the time since The testing was done with the simple test prog that I posted earlier, and now that I think harder about it: the chip replied to SOFT_RESET, replied to BIST_TEST (showed up as one chip chain), and I believe it replied to BIST_FIX. Now, IIRC, it did not respond to the READ_REG command that followed if it was alone, but did respond as expected if it was plugged in after (using the chaining plugs) another board as chip 3.

userbase++

If you all can confirm a single chip works, it would further motivate me to finish my schematic in KiCAD (and then work on a PCB layout) for the http://grid.coop/Minetor/ concept. Even if I have to have two chips, that would work, but I would have to run them at something like 400mhz or slower because I have a 30W power budget from power-over-ethernet. The real question about is 100mhz silly is what is the system total Gigahash/watt at that clock speed. If you can get away with some serious undervolting, it might actually make sense.

Also see http://bitspjoule.org/hg/nTekminer/rev/8128400e0aad for a patch that uses /usr/local/bin/gpio (from http://wiringpi.com/download-and-install/ ) for reset. I would rather use mercurial and bitbucket, but I will post a git version if that means someone will use it or it gets merged upstream.

[[gadget]]

userbase++

If you all can confirm a single chip works, it would further motivate me to finish my schematic in KiCAD (and then work on a PCB layout) for the http://grid.coop/Minetor/ concept. Even if I have to have two chips, that would work, but I would have to run them at something like 400mhz or slower because I have a 30W power budget from power-over-ethernet. The real question about is 100mhz silly is what is the system total Gigahash/watt at that clock speed. If you can get away with some serious undervolting, it might actually make sense.

Also see http://bitspjoule.org/hg/nTekminer/rev/8128400e0aad for a patch that uses /usr/local/bin/gpio (from http://wiringpi.com/download-and-install/ ) for reset. I would rather use mercurial and bitbucket, but I will post a git version if that means someone will use it or it gets merged upstream.

I'm aware of wiringPi, but I'm not sure this approach is that much different from what I am currently doing. It is a separate binary that runs with root privileges, albeit set via setuid rather than an explicit sudo. I do like the idea of executing it through a fork() though.

[FrictionlessCoin]

Can anyone confirm on having built a working A1 board?

[Dexter770221]

Can anyone confirm on having built a working A1 board?

http://imgur.com/fZuJCUX
No hashes yet, I'm working on software right now...

[[gadget]]

Can anyone confirm on having built a working A1 board?

my reference boards are working