Modular FPGA Miner Hardware Design Development

[fpgaminer]

We are talking about FPGA temperature at the moment. So here a question: do we add one or more temperature sensors to the DIMMs?

The other day I thought up a way to protect the FPGAs from heat-death without the use of a temperature sensor.

Designs loaded into the FPGA are typically routed to operate below 85C. That means that if the FPGA gets hotter than 85C, the FPGA will begin making random computational errors. So the controller can be programmed to shut down the FPGA if it notices a significant number of errors. According to the datasheet for Cyclone 4 FPGAs, the maximum operating Junction Temperature before damage is 125C. That's a fairly safe margin from 85C, so I imagine this technique would be effective.

This would fail if the FPGA gets hot very quickly, so it certainly isn't a perfect solution. It might be good enough though, under the assumption that most dangerous scenarios would arise from slowly rising heat levels.

[1714229589]

1714229589

[1714229589]

1714229589

[1714229589]

1714229589

[lame.duck]

I i remember right, there were made experiments by having a chain on consecutive gates with an inverting feedback that form an temperature dependant oscillator whose frequency could be compared against the normal PLL output. I don't remember the dependency found, but it could be a way to  get the junction temperature without any extra hardware.

[Olaf.Mandel]

What about something like this (klick on image to see full size):



The board is 45mm high and has 4 FPGAs (Xilinx XC6SLX150 in the smallest package). The components are all on one side, but I probably left too little room for the auxiliary components of the switchers (see my previous post on them: LM3150, 3x LM21215A-1). As you can see, I cannot make the Molex connector fit.

Comments?

[TheSeven]

LDOs for 1.2V? Seriously?
You're talking 6% efficiency and 170W of power dissipation in the regulator for just one FPGA?

You've got a point there. As i allready said i did this by mistake.
I had a look into PMIC DC DC switching regulators with 5 A 1.2A out and 18V input,but they also barely strike 70% eff
Also i asume this PWM signal may be problematik for a constant voltage supply.

I would appreciate it if you could have a look into that subject and name some parts you consider the best for this application.     

I'm thinking of something along the lines of this: http://search.digikey.com/scripts/DkSearch/dksus.dll?Detail&name=LTM4601EV-1%23PBF-ND
That would provide 0.6-5V at 12A, has a wide input voltage range (4.5-20V), gets >80% efficiency and costs $20 in quantities of 100 or $34 for a single part.

[marcus_of_augustus]

What about something like this (klick on image to see full size):



The board is 45mm high and has 4 FPGAs (Xilinx XC6SLX150 in the smallest package). The components are all on one side, but I probably left too little room for the auxiliary components of the switchers (see my previous post on them: LM3150, 3x LM21215A-1). As you can see, I cannot make the Molex connector fit.

Comments?

How much power will you be pulling through the DIMM bus with this?

And how many cards on single mobo are you thinking?

[Olaf.Mandel]

I'm thinking of something along the lines of this: http://search.digikey.com/scripts/DkSearch/dksus.dll?Detail&name=LTM4601EV-1%23PBF-ND
That would provide 0.6-5V at 12A, has a wide input voltage range (4.5-20V), gets >80% efficiency and costs $20 in quantities of 100 or $34 for a single part.

Cool part! And for this thing, the efficiency is higher by not going with a rail voltage. There are only very few external components needed. Only small issue: the package is 15x15mm^2 (compare that to the 19x19mm^2 for the CSG484 package of the FPGA).

[O_Shovah]

I think we should limit the maximum number of FPGA's per DIMM to a number of two.

Merging the layout of Olaf.Mandel and me this would provide enough space for all Components we introduced yet
 (2 FPGA's plus voltage regulators, EEPROM,Bus device,USB port,Barrel and Molex connector)

If we use the 1.2 V voltage regulators proposed by TheSeven we could supply both FPGA's with one regulator for 1.2 V and 4 voltage regulators for 2.5 V.

Further in would like to limit the number of DIMM slots to 5.This would be sufficient or a first series (1-2Ghash/s) and will limit the currents and size of the first boards.

In addition we may design the DIMM PCB to hold 2 FPGA's but leave one of the FPGA pads and the corresponding voltage supply unpopulated for a first prototyp and the budget series.  



  

[Olaf.Mandel]

[...]
How much power will you be pulling through the DIMM bus with this?

For the biggest Xilinx Spartan 6 and 80% efficiency:

(1.2V * 5A + 2.5V * (1.6A + 0.3A)) / 12V / 0.8 = 1.12A

For 4FPGAs this would be 4.5A, let's say 5A.

And how many cards on single mobo are you thinking?

I have no clue what O_Shovah thought. Maybe 8 or 16?

[Olaf.Mandel]

[...]
If we use the 1.2 V voltage regulators proposed by TheSeven we could supply both FPGA's with one regulator for 1.2 V and 4 voltage regulators for 2.5 V.
[...]

Again with the currents: for two FPGAs, you need only two regulators: 1x10A@1.2V and 1x(3.2A+0.6A)@2.5V. While they are a bit pricey (40EUR per DIMM for 25 DIMMs), they seem really simple: nearly no external components needed.

[marcus_of_augustus]

[...]
How much power will you be pulling through the DIMM bus with this?

For the biggest Xilinx Spartan 6 and 80% efficiency:

(1.2V * 5A + 2.5V * (1.6A + 0.3A)) / 12V / 0.8 = 1.12A

For 4FPGAs this would be 4.5A, let's say 5A.

And how many cards on single mobo are you thinking?

I have no clue what O_Shovah thought. Maybe 8 or 16?

So 5x16=80A on mobo DIMM bus power rails is no problem?

[Olaf.Mandel]

[...]
Merging the layout of Olaf.Mandel and me this would provide enough space for all Components we introduced yet
 (2 FPGA's plus voltage regulators, EEPROM,Bus device,USB port,Barrel and Molex connector)
[...]

That may look something like this (again: clockclick for full size):

[O_Shovah]

[...]
Merging the layout of Olaf.Mandel and me this would provide enough space for all Components we introduced yet
 (2 FPGA's plus voltage regulators, EEPROM,Bus device,USB port,Barrel and Molex connector)
[...]

That may look something like this (again: clockclick for full size):

I then would like to state that for the current design we are going to use 2 FPGA's per DIMM PCB at maximum, not more !
In addition the motherboard will hold up to 5 DIMM boards

This number may be increased with an sucessor to this project.I perfer building a small design soon instead of a full size rack solution with 16+ slots we might not pay for at the moment
 

For the biggest Xilinx Spartan 6 and 80% efficiency:
(1.2V * 5A + 2.5V * (1.6A + 0.3A)) / 12V / 0.8 = 1.12A

For 4FPGAs this would be 4.5A, let's say 5A.

So for two Fpga's this would be ~2.5 A at 12 V max per DIMM

Using all 5 DIMM slots would result in 12.5 A wich should be bearable

The molex connector will be needed if we later want to use FPGA's with higher power draw wich may not be supplied through the DIMM sockets.

[marcus_of_augustus]

How much? and how soon?

Price and availability? ... 

[Olaf.Mandel]

How much? and how soon?

Price and availability? ... 

Someone has to draw the thing first! I will put my current (1FPGA) board up soonish for someone to convert to the desired board.

[O_Shovah]

I have been emailed by people asking me if we could provide a working prototyp within the next
few weeks and they want to invest multiple thousand Euros into a working system.

How much? and how soon?
Price and availability? ... 

So how soon is determined by how long the real routing and developing of a bus system will take.

The how much question may already determined roughly.

If we follow my suggestion above a DIMM populated with 1 Altera 80K Cyclone IV FPGA would be ~155 Euro for the FPGA  ~50 euro for the power supply ~25 euro for the DIMM board
~30 for The motherboard PCB  ~40 Euro for other components (EEPROM,DIMM-sockets,Power connectors,Bus chips)
In addition you would need a ATX power supply and a Pc controlling the boards.
So we might end up somewehre around 300 Euro for a working prototype.

I want to point out that this is just a guesstimation
The demanding thing is to finish and prove a prototype.

[Olaf.Mandel]

[...]
If we follow my suggestion above a DIMM populated with 1 Altera 80K Cyclone IV FPGA would be ~155 Euro for the FPGA  ~50 euro for the power supply ~25 euro for the DIMM board
~30 for The motherboard PCB  ~40 Euro for other components (EEPROM,DIMM-sockets,Power connectors,Bus chips)
In addition you would need a ATX power supply and a Pc controlling the boards.
So we might end up somewehre around 300 Euro for a working prototype.
[...]

I am not sure if the 25EUR for the board is feasible. You cannot go with the very cheap manufacturers because of the size constrains for holes, wire thickness, clearances, ... And if you plan to have the manufacturer populate the board, I would guess a factor of 10 in price for the boards, so a total of roundabout 740EUR for the prototype. Later it gets much cheaper for quantities.

[O_Shovah]

You certainly got a point there.
I didn't account for the soldering work.
Your caculation might be more realistic for a prototype but has to be discussed at the very end of  the development process.

[O_Shovah]

Okay lets get back to the point:

I then would like to state that for the current design we are going to use 2 FPGA's per DIMM PCB at maximum, not more !
In addition the motherboard will hold up to 5 DIMM boards

No offence to anyone but i like to finalise most design features soon.This numbers may be increased with an sucessor to this project.

[TheSeven]

So are we going for Cyclone IV or Spartan 6?
And are those 5A on 1.2V realistic? I remember reading that the S6 pulls almost twice as much, and some headroom can't hurt either.

[O_Shovah]

Therefor I would choose next step to be the determination of the FPGA used on the prototype :

- It should be abled to either house the Altera FPGA miner by "fpgaminer" or an optimised Xlinix port.

- Should be in a pricerange for less than 200 Euro.

I'd aprechiate a detailed disscusion of this as it is THE KEY to a satisfying performance