Modular FPGA Miner Hardware Design Development

[WesleyK]

I had a qoute from pcbcart for a 4 layer board of our size with gold finish and 240 connection pinns for the DIMM ind 1.2 mm thikness resulting in 18 euro per board  for a quatity of 10 boards.

I would like to avoid manufayturing the boards in germany. The liddle to no increase in quality often doesn't justify the price over the asian manufacturers in my experience.

@Olaf.Mandel:

Wich size of the board did you set for your design and your quote ?

I tought of increasing the heith  of the daughterboard above standart DIMM size to 50 maybe 60mm to get more space.

I also think we will have to populate the backside with capacitors anyway.The Xilinx docu tells to put the smallest capacitors right behind the Vint and Vcc vias.
So having to additional layers for ground and signal would be a  plus.

Edit: just to have a frame of capabilitys we are talking about here. The restictions published by pcbcart:

Layers                         1-10 layers
Material                         FR4
Copper Thickness         1/2 to 2oz  (18um-70um)
Board Thickness         .016-.126"  (0.4mm-3.2mm)
Surface finish                 HASL,Ni/Au,OSP
Soldermask                 LPI, different colors
Board Dimension         600x700mm
Min Hole Diameter         8mil (0.2mm)
Min line width                 8mil (0.2mm)
Min line spacing          8mil (0.2mm)
Min SMT pitch             16mil (0.4mm)
Min. Annular Ring         .008" (0.2mm)
Aspect Ratio                 5:1
Surface/hole plating         ave. 25um min. 20um
Tolerance:    
Hole Tolerance (PTH)      .002" (0.05mm)
Hole Tolerance (NPTH)    .003" (0.075mm)
SM Tolerance (LPI)        .003" (0.075mm)
Dimension                .004" (0.1mm)
Electrically test                10V-250V, flying probe or testing fixture
   
   
   
 

I've had very good experiences with PCBCart, excellent choice .

[1714254058]

1714254058

[1714254058]

1714254058

[1714254058]

1714254058

[1714254058]

1714254058

[1714254058]

1714254058

[li_gangyi]

The solder pad itself can be .4mm, but the solder mask clearance around the pad should be .5mm. We should be able to easily fit the design into 4 layers, 2 for power and ground planes.

Again I can assemble the prototype boards for close to nothing. We can decide on this later when the design is finalised.


What kinda voltages are we planning to use for the various rails? Perhaps I can start on the power supply design.

[Olaf.Mandel]

I registered with pcbcart now, and my concerns about min wire thickness and distance are put to rest: you can specify many different combinations. Also: their 4-layer is nearly as cheap as the 2-layer board, so there is not much point going two-layer (I was used to having to pay much more for 4-layer boards because of the extra manufacturing steps).

But can someone answer me these questions:

• What is the pcbcart default layer stack? Alternatively: what is a good layer stack we should specify?
• What does pcbcart mean when they ask for the "Min. Annular Ring"? The width of the ring on one side ((D_via-D_hole)/2) or the sum of the width on both sides of the hole (D_via-D_hole)?
• What does "Impedance Control" mean in this context? A better selection of the board stock?
• I went with the "Lead Free HASL - RoHS" surface finish. Other suggestions?
• Do you want to order the prototype from there, too? The minimum lead time of 12 working days plus shipping time translates to waiting three weeks for the board.

All in all: we should probably finalize PCB specs (see below), so people can do the FPGA routing. This is nearly independent of the interface logic and power supply, but changing the specifications means having to redo a lot of the work. I would like to know:

• # of layers
• thickness of copper for each layer
• min trace width
• min space between things
• min drill size
• min annulus

[O_Shovah]

I've had very good experiences with PCBCart, excellent choice .

Yes, i did so two:) they also have a very good and fast support.

The copper thickness can be selected: very good, we may need more than the 35µm default. The 8mil minimal via diameter is even better (though you get less current through a smaller via)! The 8mil line width and line spacing are a problem, though: I cannot get an 8mil wire between two pads of the FPGAs: the clearance is too small. This problem can be solved by using more vias and routing everything on the other layers, but is is less than ideal.

There are "special PCBs" on the pcbcart webpage, though: 4mil for both specs. And they offer much more copper! I wonder how much that costs?

The point I made about the clearance depends on which pad diameter you use. My library used 0.5mm pads, but I read somewhere that 0.4mm may be sufficient. Any insight?

I have contacted them already they would also give us a price quote if we send some gerber files of a nearly final design and a BOM so they would also do the assembley.

Currently i am modifying your layout and routing to fit a 4 layer PCB.  Acording to my knowledge alls unused IO pins are bound to ground by default so i changed this in your schematic.(please someone verify this).
I added the MSP430 and will custom build the voltage regulators and add the power connectors.

Saddly i asume there is no way to merge the egale layout files automatic.(this used to be a lot easier with Altium.... I am totaly new to Eagle) And the library of eagle is confusing me over and over again.

We should define some fix dimensions and global rail namings to be abled to on this side by side.

Board : 133mmx60mmx1.25mm (DIMM standart with increase heigh)
4 layers
 
Vccint: 1.2V
Vccaux: 2.5V
Vccio:2.5V

[Olaf.Mandel]

The solder pad itself can be .4mm, but the solder mask clearance around the pad should be .5mm. We should be able to easily fit the design into 4 layers, 2 for power and ground planes.

Ok, so I can reduce the size of the pads in my library. I wanted to say that then the 8mil trace will "just" fit between two pads. Unfortunately, I didn't think straight: (pad diameter) + (trace width) + 2*(min clearance) = 0.4mm + 8mil + 2*8mil = 1.0096mm > 1mm = (pad spacing). So it "just" does not fit.

Which is the better of the two options?

• pad=0.5mm, trace=6mil, clearance=6mil
• pad=0.4mm, trace=8mil, clearance=6mil

[...]
What kinda voltages are we planning to use for the various rails? Perhaps I can start on the power supply design.

For the routing, it makes no difference, but I planned on either having VCCAUX=VCCIO=2.5V or 3.3V. I strongly prefer 2.5V, as that reduces the power consumption of the FPGA. In all cases, VCCINT=1.2V.

When you design the power supplies, I found the National Instruments webbench very helpful: it spits out schematics, full BOMs, the works. It even claims to know the maximum required current for the FPGA. Just disable the other three banks, as they should a´only draw minimal current (so only one VCCIO load, not four). But the Linear Technologies switcher we discussed recently is also nice (if pricey).

[O_Shovah]

Seems thats what i get for writing to long posts. 

I registered with pcbcart now, and my concerns about min wire thickness and distance are put to rest: you can specify many different combinations. Also: their 4-layer is nearly as cheap as the 2-layer board, so there is not much point going two-layer (I was used to having to pay much more for 4-layer boards because of the extra manufacturing steps).

But can someone answer me these questions:

• What is the pcbcart default layer stack? Alternatively: what is a good layer stack we should specify?
• What does pcbcart mean when they ask for the "Min. Annular Ring"? The width of the ring on one side ((D_via-D_hole)/2) or the sum of the width on both sides of the hole (D_via-D_hole)? -> annular ring specifys a radius measure so annular ring(0,2mm)+drill radius(0.1mm) results in a minimum total diameter of 0.6mm,  objections ?
• What does "Impedance Control" mean in this context? A better selection of the board stock?
• I went with the "Lead Free HASL - RoHS" surface finish. Other suggestions? -> I was told Gold finish is suited better for BGA it would increase the total price by 15$
• Do you want to order the prototype from there, too? The minimum lead time of 12 working days plus shipping time translates to waiting three weeks for the board.-> you have got a point there, but i prefere the unbeatable price and industry quality over a quick delivery

All in all: we should probably finalize PCB specs (see below), so people can do the FPGA routing. This is nearly independent of the interface logic and power supply, but changing the specifications means having to redo a lot of the work. I would like to know:

• # of layers -> 4 to minimize board size
• thickness of copper for each layer -> im not an expert on this but ground and 1.2V will need 70um or more to allow narrow lines
• min trace width -> in acordance with the pcbcart requirements 0,2mm
• min space between things -> in acordance with the pcbcart requirements 0,2mm, might be reduced if needed for wiring between the FPGA pins
• min drill size -> in acordance with the pcbcart requirements 0,2mm
• min annulus -> in acordance with the pcbcart requirements 0,2mm

[Olaf.Mandel]

[...]
I have contacted them already they would also give us a price quote if we send some gerber files of a nearly final design and a BOM so they would also do the assembley.

They also accept Eagle files, so no need for exporting anything.

Currently i am modifying your layout and routing to fit a 4 layer PCB.  Acording to my knowledge alls unused IO pins are bound to ground by default so i changed this in your schematic.(please someone verify this).

I just checked: you are right, the default is pull down, pull up or floating is an option. The reason why I thought the default was pull up is the HSWAPEN pin: it enables pull ups during configuration. So during configuration, the choice is between pull up and floating. So there are several options:

• All pins connected to VCCIO:
  
  • Program pull ups in the bitstream: should work nicely.
  • Leave bitsream at default pull-down: quite a power consumption during operation (bad choice).
• All pins connected to GND:
  
  • Program pull ups in the bitstream: quite a power consumption during operation (bad choice).
  • Leave bitsream at default pull-down: quite a power consumption during configuration.

[...]
Saddly i asume there is no way to merge the egale layout files automatic.(this used to be a lot easier with Altium.... I am totaly new to Eagle) And the library of eagle is confusing me over and over again.
[...]

As long as the signal names are correct, you can merge files, it is just very cumbersome:

• Open the first file, on the first sheet
• GROUP all
• CUT (0 0)
• Open the second file, create a new sheet or go to the sheet where you want to place stuff.
• PASTE, find a spot to place stuff (e.g. (0 0))
• Save file.
• Repeat for all sheets in the first file.

The above procedure has a minor fault if ICs are spread over more than one sheet (like in my design): once an IC is already present in the design, pasting in more gates of that IC will instead cause a new IC to be instantiated. You need to repair this manually, e.g.:

• For each gate that has the wring IC name:
• DELETE the gate
• INVOKE the gate from the original IC
• Place that gate at the position where the old one was: the connections are made automatically.

[O_Shovah]

[...]
What kinda voltages are we planning to use for the various rails? Perhaps I can start on the power supply design.

For the routing, it makes no difference, but I planned on either having VCCAUX=VCCIO=2.5V or 3.3V. I strongly prefer 2.5V, as that reduces the power consumption of the FPGA. In all cases, VCCINT=1.2V.

When you design the power supplies, I found the National Instruments webbench very helpful: it spits out schematics, full BOMs, the works. It even claims to know the maximum required current for the FPGA. Just disable the other three banks, as they should a´only draw minimal current (so only one VCCIO load, not four). But the Linear Technologies switcher we discussed recently is also nice (if pricey).

I discovered this one here http://search.digikey.com/scripts/DkSearch/dksus.dll?Detail&name=LTM4627EV%23PBF-ND

I could provide  the 1.2 V rail for both FPGA's with a safe overhead.
A second one with less power could serve the 2.5 V rail for both FPGA's.
So those are my favorite ones.

I would appreciate it if you could do the power supply design.
remember :

Input 11-20V
Molex 8981
Barell 2.5mm
110 pins ground plus 110 pins 12V on the DIMM pins

~6,5 A @ 1.2 V for one FPGA

~? A @ 2,5V

So you may do the "Power supply" sheet.
Is there any way to piece different sheets together ?

[Olaf.Mandel]

[...]
Is there any way to piece different sheets together ?

In Eagle, the different sheets have no connectors like in Altium. You cannot get a symbol that stands for a full sheet like in Altium to do overview diagrams with detailed schematics in different sheets. Also, I used connection symbols in my design, but be aware: they are only decorative and do absolutely nothing!!! The connection between different sheets is done only through the name of the nets. That is why I put labels on the different nets, to keep track of what goes where. So all nets that have a dedicated name (changed from N$?? to something else) will be connected when you copy the sheets. I think (please verify this!) that the different unnamed nets (N$??) get renumbered if you paste new stuff. But if they are not, then this can get really ugly...

[O_Shovah]

Just to get an update.

Since dicussion almost cheased on this.  Is it decied to use SPI  I2C and USB on the bus or have there been any further ideas or objections.?


Maybe you could give some feedback so we may create a standardfor our PCB layouts.

size:133mmx60mmx1,25mm DIMM standart 240 pins DDR3

4 layers

cooper thikness: 35um min , may be increased for high current rails (1,2V) 

Where are we going to share future files ?

Do we use my dropbox or someones github? 

[Olaf.Mandel]

Just to get an update.

Since dicussion almost cheased on this.  Is it decied to use SPI  I2C and USB on the bus or have there been any further ideas or objections.?

I think we agreed to not use I²C for the connection to the FPGA because it eats too many resources compared to SPI. For the DIMM connector: we haven't finished discussing that, yet. The use case was reading out an EEPROM, but if we place an MCU on the DIMM, that is no longer an issue: the MCU can identify itself via any of the other busses. And you need dedicated busses or extra switchers on the backplane for I²C, so my feeling is to get rid of I²C altogether.

[...]
Where are we going to share future files ?

Do we use my dropbox or someones github? 

Sorry I didn't use your dropbox before. I didn't want to download an extra program for that, but I had git installed... My feeling is that for all programmers, using a version control system should be easy. The problems arise when you have binary files (like Eagle), as there it is not possible to easily detect what was changed. Many of the cool features of any version control software fail here. But you still have branches, webpages for direct file access (no software to download) and to forth.

For Dropbox: Should be easy enough so even Windows users can use it (*ducks*). And there is a basic versioning control. I just cannot get "warm" with it.

Currently, there are several independent designs. Once these start merging, the service to use will emerge by itself.

[O_Shovah]

Ok so we will have i2c out of this  i m fine wit that.

concerning dropbox : you dont need to download and install anything, the web surface works just fine(never used anything else)


One question in addition: is there any way to playe a connected via directly under a FPGA pin without violating any of eagles design rules.
i allways get  "can't  set via to layer Vcc on xx xx xx " when i try to rout a pin to the Vcc layer ? what do i do wrong

[fizzisist]

One question in addition: is there any way to playe a connected via directly under a FPGA pin without violating any of eagles design rules.
i allways get  "can't  set via to layer Vcc on xx xx xx " when i try to rout a pin to the Vcc layer ? what do i do wrong

Can you describe more what you are doing and what is happening? I'm guessing you have two problems? The first is that Eagle gives you a DRC "overlap" error, and the second is this other error? Or are they the same thing?

For the Dropbox folder, I suggest this organization scheme:

• Each person keeps a folder for their own "branch" of development. A person can also have multiple branches if they are working are different things.
• Later, we can create a "trunk" that is closer to the final design. We will have to be careful about checking this out so as not to overwrite someone's work right after they finished it.
• Create a parts library file for any new parts we create for this project. This can be accessed in Eagle using Library->Use, or with a ln -s to Eagle's lbr directory, or simply copied.

I don't have access yet, so if an organization scheme is already in place, feel free to ignore my suggestion.

[O_Shovah]

I thougt switching from Altium to eagle wouldn't be a big deal but i'm feeling like a school boy

I try to connect my GND pins of the FPGA to my GND layer ( LAYER 3) but it seems eagle doesn't allow me to place vias (between surface and Layer 3) directly under one GND pin. 
It displays this overlap error everytime i try.

But i think its inevitable to place vias directly under FPGA supply pins. There's just no space to do it otherwise.(i never had problems doing so in Altium, but hey its 4000 bucks a year so maybe thats a feature worth the buggy thing )

I hope that describes my problem better.

[Olaf.Mandel]

[...]
But i think its inevitable to place vias directly under FPGA supply pins. There's just no space to do it otherwise.(i never had problems doing so in Altium, but hey its 4000 bucks a year so maybe thats a feature worth the buggy thing )
[...]

I am not su sure that you should be placing vias directly at the pad. The Xilinx UG393 certainly suggests puting the vias in the centre between four neighbouring pins. If that doesn't fit, your drill or annulus is too large. I think Xilinx actually argues against placing the via in the middle of your pad (they call that a "land"):

Due to manufacturing constraints of PTH technology, it is rarely
possible to place a via inside the area of the land. Instead, this technology uses a short
section of trace connecting to a surface pad.

[fizzisist]

A DRC error in Eagle doesn't mean it won't let you do it. It's more of a warning. You can clear the errors by clicking "Approve." Still, the warning may have some merit as I've read that it can be very bad for reflow soldering to have a via under a pad. We should ask our reflow expert, li_gangyi, if he has an opinion on this.

[O_Shovah]

Thanks a lot people.

I am currently trying to rout it without vias under teh FPGA pins. Its tricky but it works. On the other hand i try using the minimum settings allowed by PCBcart wich are 4 mil for spacing and annular ring,
but that would increase the board costs  from 39 $ to 63$ in my calculation.

Considering me as a newbe in eagle i'm afraid i will not be abled to contribute a complete layout very soon.I hope everyone else is bit more sucessfull.

One last thing : how do i tell eagle that certain parts (eg capacitors) are located on the backside of the board so it doesn't require me to rout those lane back to the top?


@li_gangyi: Please confirm if you take over the power supply part so we know who is doing what.   

[li_gangyi]

Doing the power supply, do I just draw up the schematic so we can route later into the main board ? I'm having some fun creating the part library though LOL.

Vias directly under pads, I wouldn't recommend it, that hole, even if solder doesn't suck down into it, can create voids due to air migrating up.

Best to connect up the rest of the supply pins together in a criss cross net and via that centre point.

[fizzisist]

O_Shovah, you want to use the mirror command to put a component on the backside.

[Olaf.Mandel]

[...]
One last thing : how do i tell eagle that certain parts (eg capacitors) are located on the backside of the board so it doesn't require me to rout those lane back to the top?
[...]

MIRROR the part. You will notice the color for the pads changing. Make sure the bPlace, bOrigins, bNames, bValues layers are active.