stcupp (OP)
|
|
March 09, 2012, 01:05:09 PM Last edit: March 14, 2012, 01:37:31 AM by stcupp |
|
Ok to start off I am not a electrical engineer!!! If you use these designs and your fpga blows up I will not be held accountable!!! Use at your own risk!!! Having said that I do have some experience with electronics... mostly repairing and hacking old gadgets... If you are a electrical engineer and see something wrong with these schematics PLEASE let me know! I don't think I'm going to be able to test them for awhile because of the high financial cost.... If this post is useful to you please consider donating a small amount: 1BNFvb7e62j5CANx1WARq5jADaXJ6uVhZaI am still working on routing the connections for the small plugin board! When I'm finished I'll update the download link!Now for some pictures! Part list: Part | Value | Package | QTY | Price | C1 | 0.1uf | 0402-CAP | 2 | $0.04 | C3 | 4.7uf | 0805 | 1 | $0.02 | C4 | 330uf | PCK0G331MCO1GS | 5 | 7.5 | JP1 | | JP2 | 3 | PL1 | | 5566-16 | 4 | 12.80 | R1 | 4k7 | 0805 | 1 | $0.02 | R2 | 10k | 0805 | 1 | $0.02 | R3 | 619ohm | 0805 | 1 | $0.02 | R4 | 2k | 0805 | 1 | $0.02 | U$1 | USB-MINIB-5PIN | USB-MINIB | 1 | $1.00 | U$2 | D12F200A | D12F200A | 1 | $30.3 | U1 | FT232R | SSOP28DB | 1 | $4.5 | U3 | OKR-T10-W12 | OKR-T10-W12 | 1 | $10.00 | X1 | 9090-4V | 9090-4V | 1 | $1.00 | PCB | PCB | PCB | 1 | $20.00 | TOTAL: | 87.24 |
Data Sheets: Spartan 6 http://www.xilinx.com/products/silicon-devices/fpga/spartan-6/lx.htmFT232R http://www.ftdichip.com/Support/Documents/DataSheets/ICs/DS_FT232R.pdfOKR-T/10-W12 http://www.murata-ps.com/data/power/okr-t10-w12.pdfD12F200 http://www.delta.com.tw/product/ps/dcdc/std/download/data_sheet/DS_D12F200_11042008.pdfUSB Interface: This board uses the FTDI FT232R , this chip was chosen because its cheap(about $4),easy to use, and its the same chip used by the x6500 so there is already software written for it. Power supply info: This board uses the D12F200 DC to DC converter for the main core power supply. This converter was chosen because it is cheap(around $30), it can output 40 Amps, and it comes with a 2 year warranty! Why do we need a output of 40 Amps? The Spartan 6's core runs at 1.2v and each Spartan 6 consumes around 10 watts a piece. This board supports 4 Spartan 6's so its pulling about 40 watts... Lets do a simple calculation to see how many amps that would be. A=W/V A=40/1.2 A=33.33333333333333 This is why we need 40 Amps....
|
|
|
|
Glasswalker
|
|
March 09, 2012, 01:48:24 PM |
|
First let me say, I'm glad you released this It's nice to see others working on designs and sharing them (even if they may not be perfect lol). I also like the super "spartan" approach (minimal board, modular via backplane and so on). It's similar to an approach I'm taking in my own design. I haven't looked at the PCB/Schematic in detail, though at first glance I suspect the following will be big issues: - Your FPGAs are pretty close together. Judging from the cooling needed on other designs (on both the back of the PCB and the chip via heatsink) you're going to need more space between them. But that's just first impression. - It looks like you're only using 0.1 DIL headers for the backplane? and if so you're only using 2 pins per power line? I'm a bit concerned about the current you're jamming through those pins. (and relatively small traces). Remember that a mining FPGA can draw upwards of 10W per chip (I realize that's divided across the power lines, but VCCINT will take the brunt of it) - Can your voltage regulators handle the draw from all 4 FPGAs? (it does look like your VCCINT reg is rated for 40A which should handle it, but to be considered. Also are you isolating powerline noise well enough between FPGAs? I personally hope this design works out well, just throwing some feedback out there
|
|
|
|
stcupp (OP)
|
|
March 09, 2012, 02:13:14 PM Last edit: March 09, 2012, 02:35:09 PM by stcupp |
|
First let me say, I'm glad you released this It's nice to see others working on designs and sharing them (even if they may not be perfect lol). I also like the super "spartan" approach (minimal board, modular via backplane and so on). It's similar to an approach I'm taking in my own design. I haven't looked at the PCB/Schematic in detail, though at first glance I suspect the following will be big issues: - Your FPGAs are pretty close together. Judging from the cooling needed on other designs (on both the back of the PCB and the chip via heatsink) you're going to need more space between them. But that's just first impression. - It looks like you're only using 0.1 DIL headers for the backplane? and if so you're only using 2 pins per power line? I'm a bit concerned about the current you're jamming through those pins. (and relatively small traces). Remember that a mining FPGA can draw upwards of 10W per chip (I realize that's divided across the power lines, but VCCINT will take the brunt of it) - Can your voltage regulators handle the draw from all 4 FPGAs? (it does look like your VCCINT reg is rated for 40A which should handle it, but to be considered. Also are you isolating powerline noise well enough between FPGAs? I personally hope this design works out well, just throwing some feedback out there The connectors in the 3d renders arnt the actual ones in using. I'm using these: http://search.digikey.com/us/en/products/15-24-7160/WM17742-ND/1633805these connecters can handle alot more than im putting through it and VCCINT gets 4 pins VCCIO gets 4 pins Ground gets 4 pins and the last 4 are for jtag also look at my update in the post about the power supply 40 watts at 1.2v is around 33 amps and regarding noise I'm following all the guidelines in the Spartan 6 PCB and pin planning guide on using low esr capacitors I do think the cooling may be a problem though it might be good to space them out a bit more they are spaced a inch apart EDIT: Btw my power traces are huge lol I dont think you noticed but check out this pic of the bottom of the board http://img268.imageshack.us/img268/6672/eagleuptestboardbottom.png
|
|
|
|
DeathAndTaxes
Donator
Legendary
Offline
Activity: 1218
Merit: 1079
Gerald Davis
|
|
March 09, 2012, 02:35:41 PM |
|
I love it but honestly I would like to see a "bigger one".
PCIe 6/8 pin connector is good for 150W. I would love to see a backplane which gets power from a PCIe 8 pin "socket" (and can work with either 6 pin or 8 pin connector, usb controller for connection to a host, and a 120W 12VDC to 1.2VDC supply and 12 (yes 12) connectors.
Each board would have only a heatsink no fan and you could put 2x 140mm fans along the "long side of the board to blow cool air across all the heatsinsk. The board could have L brackets (or just screw holes) to mount the fans and 2x 3pin fan headers to power the fans.
210 MH/s * 12 = ~2.5 GH/s fully loaded and user could add daughter cards incrementally. An 900 to 1200W PSU w/ 6 PCIe power connectors could power 6 boards or 12.5 GH/s. It could be a "poor mans" Rig Box. Start small w/ one backplane and 4 or 6 FPGA and build as you go.
|
|
|
|
stcupp (OP)
|
|
March 09, 2012, 03:00:39 PM |
|
I love it but honestly I would like to see a "bigger one".
PCIe 6/8 pin connector is good for 150W. I would love to see a backplane which gets power from a PCIe 8 pin "socket" (and can work with either 6 pin or 8 pin connector, usb controller for connection to a host, and a 120W 12VDC to 1.2VDC supply and 12 (yes 12) connectors.
Each board would have only a heatsink no fan and you could put 2x 140mm fans along the "long side of the board to blow cool air across all the heatsinsk. The board could have L brackets (or just screw holes) to mount the fans and 2x 3pin fan headers to power the fans.
210 MH/s * 12 = ~2.5 GH/s fully loaded and user could add daughter cards incrementally. An 900 to 1200W PSU w/ 6 PCIe power connectors could power 6 boards or 12.5 GH/s. It could be a "poor mans" Rig Box. Start small w/ one backplane and 4 or 6 FPGA and build as you go.
Well actually I don't think a whole bunch of pcie connectors are needed...... my model here should only be pulling about 3.5 amps at 40 watts on the 12v line normal molex connectors like im using should be able to go up to around 35-40 amps max on the 12v line as long as the power supply can handle it. although I wouldn't recommend that as you would be pulling like 400 watts off one wire EDIT: errr..... well it depends on what gauge the wiring is im not sure what is normally used on molex
|
|
|
|
nelisky
Legendary
Offline
Activity: 1540
Merit: 1002
|
|
March 09, 2012, 03:03:28 PM |
|
Watching closely...
|
|
|
|
DeathAndTaxes
Donator
Legendary
Offline
Activity: 1218
Merit: 1079
Gerald Davis
|
|
March 09, 2012, 03:06:07 PM |
|
Well actually I don't think a whole bunch of pcie connectors are needed...... my model here should only be pulling about 3.5 amps at 40 watts on the 12v line normal molex connectors like im using should be able to go up to around 35-40 amps max on the 12v line as long as the power supply can handle it. although I wouldn't recommend that as you would be pulling like 400 watts off one wire
Well it isn't a "whole bunch" just 1 PCIe connector per backplane (12 boards). The reason to use PCIe connector instead of Molex is simply a practical matter instead of a technical limit. Most PSU are designed to give full amperage of a rail to a pair of PCIe connectors. On the other hand all the Molex connectors and likely either ATX24 connector and/or 8pin MB supplemental connector are usually all crammed on the same rail. Even in single rail designs "virtual rails" are often used to limit current on one "set of wires". So as a practical matter it is pretty easy to push 900W to 1200W over 6 PCIe connectors (1 per backplane). If you had the same 1200W PSU and tried to pull 900W from the Molex connectors you would find the PSU would trip off. Maybe not all PSU would but a lot would. Trying to find the "right" PSU then becomes difficulty. Using PCIe connector is simply a "practical solution". If a PSU had 4 PCIe connectors it likely can power 4 boards, 6 connectors = 6 boards. Doesn't take a lot of trying (and failure). Just plug and go. I understand Molex connector can handle a lot of amperage but most PSU aren't setup to deliver max amperage on Molex connectors. In modern computer almost nothing uses them anymore except fans.
|
|
|
|
stcupp (OP)
|
|
March 09, 2012, 03:21:28 PM |
|
Well actually I don't think a whole bunch of pcie connectors are needed...... my model here should only be pulling about 3.5 amps at 40 watts on the 12v line normal molex connectors like im using should be able to go up to around 35-40 amps max on the 12v line as long as the power supply can handle it. although I wouldn't recommend that as you would be pulling like 400 watts off one wire
Well it isn't a "whole bunch" just 1 PCIe connector per backplane (12 boards). The reason to use PCIe connector instead of Molex is simply a practical matter instead of a technical limit. Most PSU are designed to give full amperage of a rail to a pair of PCIe connectors. On the other hand all the Molex connectors and likely either ATX24 connector and/or 8pin MB supplemental connector are usually all crammed on the same rail. Even in single rail designs "virtual rails" are often used to limit current on one "set of wires". So as a practical matter it is pretty easy to push 900W to 1200W over 6 PCIe connectors (1 per backplane). If you had the same 1200W PSU and tried to pull 900W from the Molex connectors you would find the PSU would trip off. Maybe not all PSU would but a lot would. Trying to find the "right" PSU then becomes difficulty. Using PCIe connector is simply a "practical solution". If a PSU had 4 PCIe connectors it likely can power 4 boards, 6 connectors = 6 boards. Doesn't take a lot of trying (and failure). Just plug and go. I understand Molex connector can handle a lot of amperage but most PSU aren't setup to deliver max amperage on Molex connectors. In modern computer almost nothing uses them anymore except fans.I see lol I thought you meant like a couple per board.... when I started designing this I was thinking of using a cheap $25 300 watt single rail PSU
|
|
|
|
DeathAndTaxes
Donator
Legendary
Offline
Activity: 1218
Merit: 1079
Gerald Davis
|
|
March 09, 2012, 03:25:57 PM |
|
I see lol I thought you meant like a couple per board.... when I started designing this I was thinking of using a cheap $25 300 watt single rail PSU Yeah maybe Molex would be fine. I guess one could use a custom PCIe 6pin to Molex adapter. For a large backplane though might need to do some testing to see how much amperage most PSU can push on a single wire. Sure the wire and connector can handle it but most PSU may not be designed for such a high load on a single wire. Maybe they are and the whole thing is just academic.
|
|
|
|
stcupp (OP)
|
|
March 09, 2012, 03:33:11 PM |
|
I see lol I thought you meant like a couple per board.... when I started designing this I was thinking of using a cheap $25 300 watt single rail PSU Yeah maybe Molex would be fine. I guess one could use a custom PCIe 6pin to Molex adapter. For a large backplane though might need to do some testing to see how much amperage most PSU can push on a single wire. Sure the wire and connector can handle it but most PSU may not be designed for such a high load on a single wire. Maybe they are and the whole thing is just academic. Check this out http://search.digikey.com/us/en/products/D12S300-1%20C/941-1046-ND/2501318you could throw 2 of those on a board and have enough power for 14 spartan 6's and its only about $10 more than the regulator i got on this small board
|
|
|
|
DeathAndTaxes
Donator
Legendary
Offline
Activity: 1218
Merit: 1079
Gerald Davis
|
|
March 09, 2012, 03:42:02 PM |
|
Nice. Yeah 14 daughter boards. Drool.
Got a guesstimate on what just the backplane would cost w/ 14 connectors. $80? $200?
|
|
|
|
stcupp (OP)
|
|
March 09, 2012, 04:21:19 PM |
|
Nice. Yeah 14 daughter boards. Drool.
Got a guesstimate on what just the backplane would cost w/ 14 connectors. $80? $200?
It all depends on how many youd want to make...... stuff is ALOT cheaper in bulk..... you would also prob need around a 12 x 5 PCB which could be pretty expensive in small qty even these connectors are over $1 cheaper a peice if you buy like 50 I'll do a rough estimate for 3 boards for you
|
|
|
|
nebulus
|
|
March 09, 2012, 04:31:11 PM |
|
Watchin
|
|
|
|
stcupp (OP)
|
|
March 09, 2012, 04:45:59 PM |
|
so heres a rough estimate:
50 female connectors 137.75 50 male connectors 114.57 6 psu 250.56 3 FT232R 12 50 330uf 22.33 3 5x12 pcb 60 3 pcie connectors 3 rand caps & res 5 fuses 3 anything im forgetting??? 10
618.1033 618.1033/3 = 206.0344333333333
However I didn't factor in the power supplies for VCCIO bc im not sure what exactly youd need for 14 chips =p but id expect it to be a extra $20-$30 per board
|
|
|
|
dropt
Legendary
Offline
Activity: 1512
Merit: 1000
|
|
March 09, 2012, 05:17:22 PM |
|
errr..... well it depends on what gauge the wiring is im not sure what is normally used on molex
18 awg for a standard good quality molex.
|
|
|
|
rjk
Sr. Member
Offline
Activity: 448
Merit: 250
1ngldh
|
|
March 09, 2012, 05:21:21 PM |
|
errr..... well it depends on what gauge the wiring is im not sure what is normally used on molex
18 awg for a standard good quality molex. Are you talking about 4-pin HDD molexes, or 6/8 pin PCIe connectors? My PCIe 6-pins are all 16AWG from a PSU I have here.
|
|
|
|
dropt
Legendary
Offline
Activity: 1512
Merit: 1000
|
|
March 09, 2012, 05:41:40 PM Last edit: March 09, 2012, 05:58:48 PM by dropt |
|
errr..... well it depends on what gauge the wiring is im not sure what is normally used on molex
18 awg for a standard good quality molex. Are you talking about 4-pin HDD molexes, or 6/8 pin PCIe connectors? My PCIe 6-pins are all 16AWG from a PSU I have here. The 4 pin molex, since that was what the OP has on his render. 16 awg is used as well for high-end PSU's (Primarily used for PCI-E and the 24-pin ATX connector.) That said, IIRC 18 is standard fair, but 16 allows greater wire length without a noticeable increase in resistance. I've seen "shitty" adapters use fine strand 20awg, and if you see threads where adapters have burned out, this is usually the culprit from what I've seen.
|
|
|
|
dropt
Legendary
Offline
Activity: 1512
Merit: 1000
|
|
March 09, 2012, 05:57:33 PM Last edit: March 09, 2012, 08:09:07 PM by dropt |
|
If I find some time in the next couple of days I'll take a gander at the data sheets and your design.
Just curious, have you done any circuit analysis on this yet?
I think glasswalker covered the off-the-top of my head concerns. It'd be nice if you could isolate the VCC input for each core.
I might also think about using two 30 amp DC-DC converters instead of the single. Doing so would give you some head room to add a header at each core location for a fan if ever there was a desire for it. At 33.3/40 amps you're not leaving yourself a lot of room for other losses in the circuit(s) as well as a the converter dies you don't lose the whole board.
Non-EE note: Holes for heat sinks on each core daughter card.
|
|
|
|
stcupp (OP)
|
|
March 09, 2012, 07:03:43 PM |
|
If I find some time in the next couple of days I'll take a gander at the data sheets and your design.
Just curious, have you done any circuit analysis on this yet?
I think glasswalker covered the off-the-top of my head concerns. It'd be nice if you could isolate the VCC input for each core.
I might also think about using two 30 amp DC-DC converters instead of the single. Doing so would give you some head room to add a header at each core location for a fan if ever there was a desire for it. At 33.3/40 amps you're not leaving yourself a lot of room for other losses in the circuit(s) as well as a the converter dies you don't lose the whole board.
Non-EE note: Holes for heat sinks on each core daughter card.
No I haven't done any circuit analysis on it... can you recommend a program? Just wondering..... Why would I want to hook up fan headers to my converter thats running at 1.2v and put more load oon it when there is perfectly good 12v and 5v lines coming from a PC power supply?
|
|
|
|
DeathAndTaxes
Donator
Legendary
Offline
Activity: 1218
Merit: 1079
Gerald Davis
|
|
March 09, 2012, 07:06:40 PM |
|
Just wondering..... Why would I want to hook up fan headers to my converter thats running at 1.2v and put more load oon it when there is perfectly good 12v and 5v lines coming from a PC power supply?
This. All that is needed is a 2pin male fan connector and a trace to the 12V input. Nothing more complicated than that is necessary.
|
|
|
|
|