Modular FPGA Miner Hardware Design Development

[newMeat1]

the Xilinx PCB design guide calls for 0402 imperial size caps. Does anyone have a good feel what happens if that size is changed?

Yes... the most important effect of bigger physical size is higher ESR. That will change the specific frequency that the capacitor filters.

Obviously Xilinx went through very carefully and chose those capacitors to filter specific frequencies. I would follow their recommendations.

[fizzisist]

the Xilinx PCB design guide calls for 0402 imperial size caps. Does anyone have a good feel what happens if that size is changed?

Yes... the most important effect of bigger physical size is higher ESR. That will change the specific frequency that the capacitor filters.

Obviously Xilinx went through very carefully and chose those capacitors to filter specific frequencies. I would follow their recommendations.

Actually, I think your confusing ESR with ESL. Generally, larger capacitors have lower ESR. The ESL will go up, though, because of the increased size of the current loop, because the cap will be further away from the pin. At least that's my understanding.

[newMeat1]

Generally, larger capacitors have lower ESR.

Yeah I guess that makes sense. Either way, if L or R change, it's going to throw off their careful system.

[Olaf.Mandel]

I uploaded new versions of the PSU design and a BOM script to github and dropbox. Commit logs:

• Fixed LMZ22010 <-> LMZ12010 mixup:
  
  • Renamed GND back to AGND and PGND in LMZ22010
  • Changed description and default device name for LMZ22010
  • Added an explicit LMZ12010 device and symbol to the library (the symbols have the same pin lications for easier exhanges)
  • Used the new LMZ12010 in the PSU design
• Added script to update BOM database: The script updateavail.sh (POSIX systems only) queries the distributors webpage for each part in the *.tsv file and adds columns for number of on-stock parts and prices for different price-breaks. The project.tsv file has been reformatted to match the expected input of this script. TODO: Currently supports only DigiKey as a supplier. Need to add Avnet support.

I haven't used fizzisist newest database as the basis for my reformated one. Will do so by the next upload.

[Olaf.Mandel]

Actually, concerning the update-script: this does not (easily) work for Windows-users. This should be convertible to Eagle ULP if you try. Does anyone think that effort is justified?

[fizzisist]

Wow, that is an awesome script! I ran it (on Ubuntu) and it worked perfectly. I don't think there's reason to convert this to a ULP, because this script won't need to be run very often. As long as there's someone around with a POSIX system, we can update the prices. Great work!

[fizzisist]

Thanks for merging my old database into the new project.tsv. I added a few more items to it, including the MCU and 25MHz oscillator.

I made some minor changes to the MCU schematic and saved it into a new folder under my name, to make sure I didn't overwrite your work. If you're ok with it, feel free to copy it back into your folder. Changes:

• Changed USB connector to one that's available on Digikey.
• Added LEDs. These aren't connected to anything yet, but they are placed in the schematic and the BOM database. I added options for red, yellow, and green, all in 0603 and 0805 packages. Later, we can delete the ones we don't want. We should use the same LEDs for all other parts of the design.

li_gangyi, are you ok with 0603 LEDs or should we use the 0805s?

[Olaf.Mandel]

I uploaded a new version of the BOM to github and dropbox. Commit log:

Finished updateavail script:

• updateavail.sh now works correctly with avnet
• Merged in new version of the database from fizzisist
• Changed the package for the FPGAs to what is being used
• Added second supplier for the FPGAs and big switchers
• Updated extended database: runtime = 45s

Back to the boards. I will now look at fizzisist's new MCU.

[Olaf.Mandel]

• Changed USB connector to one that's available on Digikey.
• Added LEDs. These aren't connected to anything yet, but they are placed in the schematic and the BOM database. I added options for red, yellow, and green, all in 0603 and 0805 packages. Later, we can delete the ones we don't want. We should use the same LEDs for all other parts of the design.

Looks good. I didn't pay attention when looking at / changing the USB connector. How many LEDs do people envision we need? We have 11 free pins on the MCU, but that seems excessive. Maybe 3? How many each for the FPGAs: five each?

[Olaf.Mandel]

[...]
I currently placed two MCU supply voltages on the DIMM connector: +5V which is connected to the USB 5V, and +3V3, which is connected to the output of the LDO inside the MSP430 (this LDO is fed by the +5V). I wanted to have the backplane supply 5V only if it used the USB connection and supply 3.3V otherwise. But there should be a different way to detect the presence of the USB host. So any concerns about removing the +3V3 signal from the DIMM?

Can any of the MSP430 guys give their opinion here?

[Olaf.Mandel]

[...]
4) Speaking of the LMZx2010, the datasheet is calling for slightly different resistor values on R_FBT and _FBB. Am I just reading this wrong or was this a mistake?

I just looked through the LMZ12010 datasheet. According to eq. (4), li_gangyi's resistor choice should get us VCCIO=2.484V and VCCINT[01]=1.193V. The recommended resistor values in the table on page 2 would give us  VCCIO=2.474V and VCCINT[01]=1.210V, instead. So li_gangyi's seems better to me, as it is closer to correct in both cases.

[MonkeyThink]

Hi,

Not sure if this should go in a separate thread or if it's OK here, but while the guys have been doing such fine work on the mining hardware design I've been wondering about the 'front end' computer that will be feeding the miner hardware and submitting proof of work back to the network.

What other people's idea's on this.  Is everyone thinking that they will just hook up a hardware miner to their desktop machines or laptops and leave them running 24x7?  Because I was thinking it would be nice to have a totally stand alone solution.

I remember that someone did post about having an FPGA development board mining totally stand alone, but I can't seem to find that thread at the moment.

If people are interested then perhaps we could start a discussion along these lines with some suggested solutions.  Obviously it would be possible to build the solution into the FPGA but it seems like it would be a waste of valuable space and effort and might be better (and more cheaply) implemented in other ways.

I was thinking along the lines of something like one of this tiny Gumstix boards: http://www.gumstix.com/ running Ubuntu or perhaps a more mainstream, small motherboard. Mini-itx perhaps.

Thoughts?

[fizzisist]

[...]
4) Speaking of the LMZx2010, the datasheet is calling for slightly different resistor values on R_FBT and _FBB. Am I just reading this wrong or was this a mistake?

I just looked through the LMZ12010 datasheet. According to eq. (4), li_gangyi's resistor choice should get us VCCIO=2.484V and VCCINT[01]=1.193V. The recommended resistor values in the table on page 2 would give us  VCCIO=2.474V and VCCINT[01]=1.210V, instead. So li_gangyi's seems better to me, as it is closer to correct in both cases.

Good eye, I didn't see that equation before, only the table. So, li_gangyi's choices would work because the ratio is right. On the other hand, the table values are the recommended values. Also, there's this note in the datasheet:

These resistors should generally be chosen from values in the range of 1.0 kΩ to 10.0 kΩ.

R_FBT is only 620 ohm on the 1.2V supplies right now, so we should increase those. I'd suggest just going with the recommended values as long as we're changing things...

[Olaf.Mandel]

[...]
What other people's idea's on this.  Is everyone thinking that they will just hook up a hardware miner to their desktop machines or laptops and leave them running 24x7?  Because I was thinking it would be nice to have a totally stand alone solution.
[...]
If people are interested then perhaps we could start a discussion along these lines with some suggested solutions.  Obviously it would be possible to build the solution into the FPGA but it seems like it would be a waste of valuable space and effort and might be better (and more cheaply) implemented in other ways.

I was thinking along the lines of something like one of this tiny Gumstix boards: http://www.gumstix.com/ running Ubuntu or perhaps a more mainstream, small motherboard. Mini-itx perhaps.

Thoughts?

This has already been discussed and is on the roadmap. Basically, the plan is as follows: First we walk, then we run.  Meaning that first we get something working that requires a computer and then we change the backplane to include an embedded, ethernet capable computer. This will leave the FPGA design completely untouched (it being on a different PCB and all). We haven't discussed which CPU to use later, because we first have to get this one working...

[TheSeven]

[...]
4) Speaking of the LMZx2010, the datasheet is calling for slightly different resistor values on R_FBT and _FBB. Am I just reading this wrong or was this a mistake?

I just looked through the LMZ12010 datasheet. According to eq. (4), li_gangyi's resistor choice should get us VCCIO=2.484V and VCCINT[01]=1.193V. The recommended resistor values in the table on page 2 would give us  VCCIO=2.474V and VCCINT[01]=1.210V, instead. So li_gangyi's seems better to me, as it is closer to correct in both cases.

Especially for the VCCINT regulators I'd tend to stay on the positive side of the tolerance, as there will be non-neglegible voltage drops across the traces and FPGA pads. Remember that VCCINT directly affects achievable hashrate. Up to 1.26V are allowed here (1.32V absolute maximum), so I'd probably go for 1.25V for this rail if the regulator feedback comes directly from the FPGA's pads.
To take advantage of this you'll need to define the guaranteed minimum voltage in the UCF file.

[Olaf.Mandel]

I went ahead and merged the different parts temporarily. I still feel that at least the MCU is unfinished enough to warrant not doing the final merge, but this temporary merge lets us see where we are going:



The BOM total for one board is about 340EUR (a few components are still missing, but those should fit in that price limit). The board price at pcbcart is 27EUR per board plus 135EUR one-time costs with the following settings:

• Material: FR4
• Layers: 4 layer
• Material Details: Standard Tg 140C
• Board type: single unit
• Board Size (width): 135mm
• Board Size (height): 95mm
• Quantity: 5pcs
• Thickness (Finished Board): 1.2mm
• Layer Stack: As pcbcart default
• Layer Stack Details: -
• Impedance Control: No
• Surface Finish: Lead Free HASL - RoHS
• Outer Layer Copper Weight (Finished): 35um
• Inner Layer Copper Weight: 35um
• Min. Tracing/Spacing: 0.15mm
• Min. Annular Ring: 0.10mm
• Smallest Holes: 0.30mm
• Holes Number: Over 600
• Buried/Blind Vias: No
• Times of Buried/Blind Via: --
• Surface Mount: 2 sides
• Soldermask: Both Sides
• Peelable Soldermask: None
• Soldermask Color: Green
• Matt Color (only add to Green or Black): None
• Silkscreen Legend: 2 sides
• Silkscreen Legend Color: White
• Gold Fingers: Yes
• Gold Fingers Number: 240
• Gold Fingers Chamfer: 60°
• Slots in Board: No Slot in Board
• Slots quantity in board: -
• Testing: Yes
• UL Marking: Yes - as pcbcart default
• Date Code Marking: Yes - as pcbcart default
• Lead Time: in 18 days
• Special Requirement Note: -

So a full set of 5 prototypes is about 5x400EUR = 2000EUR.

PS: When doing the merge, I also found two small bugs in the PSU (not replaced the resistors, yet), so reuploaded that, too.

[Olaf.Mandel]

[...]
Especially for the VCCINT regulators I'd tend to stay on the positive side of the tolerance, as there will be non-neglegible voltage drops across the traces and FPGA pads. Remember that VCCINT directly affects achievable hashrate. Up to 1.26V are allowed here (1.32V absolute maximum), so I'd probably go for 1.25V for this rail if the regulator feedback comes directly from the FPGA's pads.
To take advantage of this you'll need to define the guaranteed minimum voltage in the UCF file.

I think moving the resistors below the FPGAs is a given by now. But if we increase the core voltage to achieve higher clock rates, then I would ask how close we want to cut it: is there a risk of having any EMF mess with the trace from the resistors to the switcher, so that the switcher runs at a too high voltage? What about required or suggested resistor tolerance: is 1% sufficient?

And our of curiosity: what is the gain in terms of clock rate for a voltage increase of 0.05V? I am willing to take ISE estimates after whichever stage you feel is remotely reliable.

[TheSeven]

[...]
Especially for the VCCINT regulators I'd tend to stay on the positive side of the tolerance, as there will be non-neglegible voltage drops across the traces and FPGA pads. Remember that VCCINT directly affects achievable hashrate. Up to 1.26V are allowed here (1.32V absolute maximum), so I'd probably go for 1.25V for this rail if the regulator feedback comes directly from the FPGA's pads.
To take advantage of this you'll need to define the guaranteed minimum voltage in the UCF file.

I think moving the resistors below the FPGAs is a given by now. But if we increase the core voltage to achieve higher clock rates, then I would ask how close we want to cut it: is there a risk of having any EMF mess with the trace from the resistors to the switcher, so that the switcher runs at a too high voltage? What about required or suggested resistor tolerance: is 1% sufficient?

And our of curiosity: what is the gain in terms of clock rate for a voltage increase of 0.05V? I am willing to take ISE estimates after whichever stage you feel is remotely reliable.

IIUC the resistors shouldn't be below the FPGA, but instead as close to the switcher as possible, but fed from a trace that's coming back from one of the FPGA VCCINT pins. Also remember that this will only compensate for the losses on the VCCINT path, not the GND return, so the effective voltage will still be a bit below the programmed value. Ask one of our more experienced electrical engineers regarding how close to cut it, I'm just a computer engineering student. I just wanted to suggest to stay on the positive side of tolerance, as the suggested values were on the negative side, which doesn't seem like a good idea.

I don't think ISE even takes these values into account before the PAR stage, so you'll probably have to do a full synthesis run without timing constraints enabled or even step constraints to check the maximum achievable clock for both voltages. I've done a quick search on the web and didn't find anything even remotely related, so we'll probably have to analyze this ourselves.

Are there any components on the back of the board as well? If yes, would you mind posting a picture of the back side of the board as well? I'm a bit worried by the small number of capacitors that I'm seeing on the front of the board...

[Olaf.Mandel]

[...]
Are there any components on the back of the board as well? If yes, would you mind posting a picture of the back side of the board as well? I'm a bit worried by the small number of capacitors that I'm seeing on the front of the board...

There are caps on the back (the smallest ones, only). The board is only 1.2mm thick, in which case the Xilinx PCB design guide assigns them to the back. Instead of an image, how about a PDF with the schematics, views of the board with all layers, from the front and back and a BOM? This PDF is available on GIThub if you don't have Dropbox access. The only problem: it's too large for direct download from the webpage, you need to clone the repository.

[old_engineer]

In general, a smaller capacitor will be able to get closer to the pin and therefore do a better job decoupling. I think it would make sense to just use as few 0402s as possible: the probability of making a mistake soldering is just going to be increased for each additional capacitor. I don't think we have to think about it like reducing trace width or such (therefore raising the cost of the whole board). When these boards are eventually loaded by a professional shop, they won't even blink when we ask them to load 0402s.

Seconded that pro shops won't blink at 0402s.  If there's going to be a hand-soldered prototype, avoid them unless you absolutely need them.  Later on, switch to 0402s to reduce board space and cost (assuming large quantities of boards, of course).

I rework 0402s all the time.  My suggestion: when laying out a prototype that has to include 0402s, like in the bypass capacitor example above where using a 0805 might cause a difference in behavior, make sure to leave room around the 0402 so you have room to get a soldering iron in there.  Just because it's a small component doesn't mean that the board needs to be packed densely, and densely packed 0402s make it tough to only remove the desired resistor without accidentally removing nearby resistor(s).  Those little suckers come on in like two seconds if your soldering iron strays.  A quality pair of tweezers and a magnifying glass for inspection are also must-haves.

FWIW, I use one of these cheapo $90 USB "microscopes", which is a handy gadget for documenting issues:
http://www.amazon.com/ViTiny-Handheld-Microscope-measurement-functions/dp/tags-on-product/B003NGIBQI

and a pic of a 0402 cap that is accessible enough, with enough space to the left & right for soldering iron: