Modular FPGA Miner Hardware Design Development

[O_Shovah]

I second TheSeven's statement

@ TheSeven : Do you have a more detailed idea of wich MCU's to use and wich effective setup and wiring ?

[1714155557]

1714155557

[1714155557]

1714155557

[TheSeven]

I second TheSeven's statement

@ TheSeven : Do you have a more detailed idea of wich MCU's to use and wich effective setup and wiring ?

Sorry, that's not really my area. But given the vast number of MCUs out there, there just must be some perfectly fitting one.
Anyone who knows that market better will hopefully be able to dig that one up

[fizzisist]

I agree with TheSeven, too. As for MCU, it would be wise to use one that includes USB functionality already. The MSP430F5528IRGCT suggested by Olaf.Mandel seems good, but I don't know the MSP chips well. I have more experience with Atmel AVRs and I would prefer the AT90USB82. That's only a personal preference, so don't give too much weight to that vote, especially if someone else will be writing the firmware.

[makomk]

I agree with TheSeven, too. As for MCU, it would be wise to use one that includes USB functionality already. The MSP430F5528IRGCT suggested by Olaf.Mandel seems good, but I don't know the MSP chips well. I have more experience with Atmel AVRs and I would prefer the AT90USB82. That's only a personal preference, so don't give too much weight to that vote, especially if someone else will be writing the firmware.

The only real problem I can see with the AT90USB82 is that it doesn't support 2.5V operation.

[Olaf.Mandel]

[...]
Nevertheless I would still route the SPI and JTAG interfaces to the DIMM as well. It just can't hurt. If we're going for an MCU instead of an FTDI anyway, tristating won't be an issue.

I am still not convinced that putting extra protocols on the DIMM bus cannot hurt: in the best case, we are locking future DIMMs and backplanes into a more complex interface than it needs to be. How about this: we define several interfaces, e.g. the ones we were talking about all the time (JTAG, I²C, SPI, USB) and we make all of them optional. The DIMM can inform the backplane via a couple of low impedance connections which protocols it supports and which are not connected. That makes the backplane a (very little) more complicated because you need to read out the signals from these key-pins. But it keeps the DIMM design completely open: a later revision could really employ "head-less" DIMMs that don't work stand alone. Or use an FT2232H design that is incapable of tri-stating the JTAG and SPI pins for use by the backplane.

Of course, this way all DIMMs will work on a backplane that supports all protocols. But if the backplane designer starts cutting corners, then some DIMMs won't work in that backplane. There are three solutions:

• Make all protocols mandatory (current situation). Safest option, but precludes making the boards simpler at a later stage.
• Define one of the protocols as compulsory (my preference: USB) and leave the others optional. All DIMMs and all backplanes must support the compulsory protocol and can fall back to that if no other protocol is available.
• If you have one compulsory protocol that must be supported, why do you need the others? So only support that one protocol. Very lean bus design, but forces all future DIMMs to have adapter logic to translate between the bus protocol and what is used on the DIMM.

It's basically a philosophy question: many choices but (mostly unneeded) overhead to provide them on one hand and lean design but less flexibility on the other hand ("less", not "no"! You just cannot remove the adapter logic). A bit of an IBM/PC clone vs. Mac question   

[...] And yes, we should go for an unbrickable MCU.

Haven't found one, yet. I was suggested the Analog Devices ADUC series as unbrickable, but that one does not support USB.

[...] I have more experience with Atmel AVRs and I would prefer the AT90USB82. [...]

My understanding was that it is possible to brick the Atmels. Can you confirm (or better: rebut) this?

[fizzisist]

• Make all protocols mandatory (current situation). Safest option, but precludes making the boards simpler at a later stage.
• Define one of the protocols as compulsory (my preference: USB) and leave the others optional. All DIMMs and all backplanes must support the compulsory protocol and can fall back to that if no other protocol is available.
• If you have one compulsory protocol that must be supported, why do you need the others? So only support that one protocol. Very lean bus design, but forces all future DIMMs to have adapter logic to translate between the bus protocol and what is used on the DIMM.

In my mind, if you have an MCU on the board, you could reflash the MCU to adapt to a future motherboard design. This gives maximum flexibility. We should route as many IOs to the DIMM connector as possible, with at least a basic idea of what each one will serve as, but they can be reassigned (or disabled) by downloading new firmware for the MCU on the daughterboard. If we do this, we would need to make sure that reflashing can be easily done without extra hardware by including a USB bootloader or something similar.

[...] I have more experience with Atmel AVRs and I would prefer the AT90USB82. [...]

My understanding was that it is possible to brick the Atmels. Can you confirm (or better: rebut) this?

What exactly would you define as bricked? If you had a USB bootloader and accidentally overwrote the firmware, you would need to reload the bootloader with a programmer (or have the motherboard do this). The only "bricking" I can think of with the AVRs is if you accidentally lock the flash memory or disable the RESET pin (needed to load new firmware). These can still be recovered from by using a special programmer.

The only real problem I can see with the AT90USB82 is that it doesn't support 2.5V operation.

Is 2.5V operation a requirement? Can USB run at 2.5V? I don't think so.

[Olaf.Mandel]

[...]

[...]
My understanding was that it is possible to brick the Atmels. Can you confirm (or better: rebut) this?

What exactly would you define as bricked? If you had a USB bootloader and accidentally overwrote the firmware, you would need to reload the bootloader with a programmer (or have the motherboard do this). The only "bricking" I can think of with the AVRs is if you accidentally lock the flash memory or disable the RESET pin (needed to load new firmware). These can still be recovered from by using a special programmer.

Bricked would mean either completely unrecoverable or not recoverable without desoldering from the PCB. My understanding was that, like you said, the fuses can accidentally be set to lock out the RESET pin. But I thought this is completely unrecoverable, even by a high-voltage programmer. Or isn't it? The curious thing is that the different Atmel chips have different programming strategies, and I only read about the one for the ATtiny. For a USB capable MCU: no clue.

The only real problem I can see with the AT90USB82 is that it doesn't support 2.5V operation.

Is 2.5V operation a requirement? Can USB run at 2.5V? I don't think so.

Important are the IOs that go to the FPGA. While the Spartan 6 can be operated at VCCO = 3.3V, this may not be the preferred voltage. And other FPGAs for a second generation of DIMMs may not support 3.3V at all. Basically, it's not a show stopper, not even a very huge demerit at the moment, but it is something to keep in mind when comparing otherwise nearly identical MCUs.

[makomk]

What exactly would you define as bricked? If you had a USB bootloader and accidentally overwrote the firmware, you would need to reload the bootloader with a programmer (or have the motherboard do this). The only "bricking" I can think of with the AVRs is if you accidentally lock the flash memory or disable the RESET pin (needed to load new firmware). These can still be recovered from by using a special programmer.

So long as you stick to only reprogramming these USB AVRs via the factory-preprogrammed USB bootloader, I don't think you can brick them. Which is fortunate, because recovering from a disabled RESET pin is apparently a pain; it requires a high-voltage parallel programmer that has to connect to pretty much all the pins and apply 12V to the RESET pin, which makes doing it in-circuit just a tad tricky.

The shipped bootloader is a bit annoying in that you have to manually hold a particular pin low during reset to enter it, but that's no big deal unless remote firmware updates are a requirement.

Is 2.5V operation a requirement? Can USB run at 2.5V? I don't think so.

Yeah, it just makes doing I/O interfacing a tad more tricky. As Olaf said, most of the FPGAs we're likely to use can run at 3.3V for I/O anyway, and if necessary level shifters are an option. (Looking at the Kintex-7 data sheets those support 3.3V on a subset of the I/O pins too.)

[Olaf.Mandel]

[...]
So long as you stick to only reprogramming these USB AVRs via the factory-preprogrammed USB bootloader, I don't think you can brick them. Which is fortunate, because recovering from a disabled RESET pin is apparently a pain; it requires a high-voltage parallel

In a weaker version of that sense, the MSP430 is also "unbrickable": as long as you don't touch the default BSL and as long as your code leaves the write-protect alone, everything is fine. The problem is that an inexperienced user (or someone who types command-lines on autopilot) may easily reflash the BSL.

[...]
Yeah, it just makes doing I/O interfacing a tad more tricky. As Olaf said, most of the FPGAs we're likely to use can run at 3.3V for I/O anyway, and if necessary level shifters are an option. (Looking at the Kintex-7 data sheets those support 3.3V on a subset of the I/O pins too.)

Actually, you shifted what I wanted to say where a bit: I meant that while the current FPGA can do 3.3V, I am not sure how many others can do so.

And having to use level shifters is very annoying. I just looked, though: the MSP430F5528 (and comparable chips) has several banks of GPIOs, but there is only a single supply voltage for IOs. And if you use 2.5V for the FPGA connections, you cannot use 3.3V for the connections to the DIMM bus (except for the USB, of course).

[makomk]

In a weaker version of that sense, the MSP430 is also "unbrickable": as long as you don't touch the default BSL and as long as your code leaves the write-protect alone, everything is fine. The problem is that an inexperienced user (or someone who types command-lines on autopilot) may easily reflash the BSL.

AVRs are definitely the strong version though; they have hardware write-protect for the bootloader that as I understand it can't be overridden without connecting up a programming cable, and neither can anything else that would prevent you from booting from said bootloader. At least that's the theory...

Actually, you shifted what I wanted to say where a bit: I meant that while the current FPGA can do 3.3V, I am not sure how many others can do so.

Ah, sorry. Well, Cyclone IV and earlier can, as can Spartan-6 obviously. Kintex-7 and Cyclone V will apparently also support 3.3V IO on at least some pins, though I'm not sure if they do for the configuration pins...

[fizzisist]

And having to use level shifters is very annoying. I just looked, though: the MSP430F5528 (and comparable chips) has several banks of GPIOs, but there is only a single supply voltage for IOs. And if you use 2.5V for the FPGA connections, you cannot use 3.3V for the connections to the DIMM bus (except for the USB, of course).

I see now what you mean. The MSP430F5528 has built in level shifting for the USB lines, meaning the MCU will run at 2.5V, but shift the USB I/O up to 3.3V by means of a built in LDO on the USB input voltage. That seems like the ideal behavior for us. As long as someone else is willing to work on the firmware for this, I think it will be a good MCU for this design.

[fizzisist]

Actually, you shifted what I wanted to say where a bit: I meant that while the current FPGA can do 3.3V, I am not sure how many others can do so.

Ah, sorry. Well, Cyclone IV and earlier can, as can Spartan-6 obviously. Kintex-7 and Cyclone V will apparently also support 3.3V IO on at least some pins, though I'm not sure if they do for the configuration pins...

I don't think there is much reason to discuss other FPGAs here. This board will have the Spartan-6. If someone wants to make a board with a different FPGA later, nothing is stopping them from using a different MCU or different method of interfacing with the bus entirely. The daughterboard is self contained, it just needs to conform to the specifications of the DIMM connector and bus interface, so that it's compatible with the motherboard.

[fizzisist]

I wanted to get an idea what the space would look like on this board. This shows two LX150s in the FG484 package and the MSP430 in the QFN64 package. This FPGA is available in a smaller package, CSG484, but I think this one fits nicely, and a larger pitch means easier job routing the signals to the pins. For comparison, this package is 23 mm square, the smaller one is 19 mm square.

[Olaf.Mandel]

I wanted to get an idea what the space would look like on this board. This shows two LX150s in the FG484 package and the MSP430 in the QFN64 package. This FPGA is available in a smaller package, CSG484, but I think this one fits nicely, and a larger pitch means easier job routing the signals to the pins. For comparison, this package is 23 mm square, the smaller one is 19 mm square.
[...]

I don't think linear regulators with 100mA output current are going to cut it. A reference picture using two CSG484 was posted some time ago. That still used the smaller FTDI chip, though.

[...]
That may look something like this (again: [click] for full size):

As for routing: I am currently trying to route an FGG484 package, but less because of the routing constraints but because of the availability (FGG is much easier to get than CSG).

[fizzisist]

I don't think linear regulators with 100mA output current are going to cut it. A reference picture using two CSG484 was posted some time ago. That still used the smaller FTDI chip, though.

I knew someone was going to point out those regulators... I just wanted to put something there as an idea for the space needed.

[...]
That may look something like this (again: [click] for full size):

As for routing: I am currently trying to route an FGG484 package, but less because of the routing constraints but because of the availability (FGG is much easier to get than CSG).

What are the two squares to the left of the FPGAs? Also, an important thing is missing from this sketch: mounting holes! This will be very important when not using the motherboard.

I didn't notice the difference in availability, but a larger pitch makes a lot of things easier, including assembly, so I would suggest going with the larger package.

[TheSeven]

I don't think linear regulators with 100mA output current are going to cut it. A reference picture using two CSG484 was posted some time ago. That still used the smaller FTDI chip, though.

I knew someone was going to point out those regulators... I just wanted to put something there as an idea for the space needed.

Yeah, and more powerful ones generally need more space, so you should account for that
And I'd really suggest to put the USB connector below the power connector, facing downward, to have a mechanical protection from connecting multiple hosts. In this design this even seems to be better from a space point of view.

[O_Shovah]

I wanted to get an idea what the space would look like on this board. This shows two LX150s in the FG484 package and the MSP430 in the QFN64 package. This FPGA is available in a smaller package, CSG484, but I think this one fits nicely, and a larger pitch means easier job routing the signals to the pins. For comparison, this package is 23 mm square, the smaller one is 19 mm square.

Maybe just to remember you.

We wanted to use a Molex AND a barrel connector.Of course this will result in some space issues.
The backside will be populated with condensators anyway so we might use it also for some other parts.

I guess we will have to increase heith of the DIMM anyway.

Regarding Voltage supply there were seriously considered (for the FPGA supply):

For 1.2 V http://search.digikey.com/scripts/DkSearch/dksus.dll?Detail&name=LTM4601EV-1%23PBF-ND    5A @ 1.2V

For 2.5V ( multiple of) http://search.digikey.com/scripts/DkSearch/dksus.dll?Detail&name=R-78AA2.5-1.0SMD-ND 1.9A @ 2.5V


Just out of interest. Will one Msp 430 be sufficient for both FPGA's ?  

PS.:
Today i will also start a rough layout in eagle.Maybe we could share eagle files for comparison.

[Olaf.Mandel]

[...]
What are the two squares to the left of the FPGAs? Also, an important thing is missing from this sketch: mounting holes! This will be very important when not using the motherboard.

The two squares are the voltage regulators (LTM4601EV-1#PBF) that TheSeven suggested. I initially considered the SimpleSwitcher series from National Semiconductor, which are cheaper. But they also need more external components, apparently. I am not convinced which is the better choice, at the moment.

EDIT:
And I agree on the mounting holes: I forgot them in my sketch of a board.

I didn't notice the difference in availability, but a larger pitch makes a lot of things easier, including assembly, so I would suggest going with the larger package.

Currently, Avnet has 344 of the FGG484 packages in stock and 0 of the CSG484 packages.

[Olaf.Mandel]

[...]
And I'd really suggest to put the USB connector below the power connector, facing downward, to have a mechanical protection from connecting multiple hosts. In this design this even seems to be better from a space point of view.

I still think orienting the USB socket to face the PCB makes things difficult for the user: they have to take a USB cable and cut away half the plastic on the plug. Otherwise, they will not be able to get the plug into the socket.

How about putting the USB socket on the edge of the board, but facing sideways. Then the locking handles of the DIMM socket can prevent plugging in a USB cable.

[Olaf.Mandel]

[...]
The backside will be populated with condensators anyway so we might use it also for some other parts.

I would opt against this. Having SMD components on the backside increases the manufacturing cost a lot. You need a second mask for the solder paste made and (some?) manufacturers charge extra for dual side population of the board. As for necessity: the Xilinx UG393 "Spartan-6 FPGA PCB Design and Pin Planning Guide" states under "PCB Capacitor Placement and Mounting Techniques", "0402 Ceramic Capacitor" that for thicker PCBs (>1.575mm) the best placement is the top surface. While for thinner PCBs the bottom side is suggested, I would not fret about that so much. The usual PCBs (especially when multi-layer) are right on that border of thickness, so both placement options must be sub-optimal... Anyway: only the VCCint capacitors are any important; we barely use any of the banks and VCCaux should be low current.

[...]
For 2.5V ( multiple of) http://search.digikey.com/scripts/DkSearch/dksus.dll?Detail&name=R-78AA2.5-1.0SMD-ND 1.9A @ 2.5V

We should postpone this selection until someone can give more definite numbers on the current we will need. So can someone compile a core with some interface logic and then look up the power consumption from the PowerPlanner?

Just out of interest. Will one Msp 430 be sufficient for both FPGA's ?  
[...]

One will be plenty.