Modular FPGA Miner Hardware Design Development

[fizzisist]

I really know almost nothing about this stuff, but can OSHW help us? http://freedomdefined.org/OSHW

Olaf.Mandel, I'm logged in to Dropbox but I don't see your picture posted in the forum. I think you need to place the image in your Public folder, then copy that link. Also, as far as I know, there are only two options for shared folders with Dropbox: shared or not shared. There's no way to grant read access to everyone, etc. I think that the Dropbox is the most convenient way to easily exchange these files at this stage. Once we get some files closer to finalization, we might want to create a wiki.

About the FPGA design, why tie those pins to anything? I'll admit I'm a little confused about the HSWAPEN pin and all of this. The documentation doesn't seem to provide a clear answer, at least in my reading. In case of a mistake, bahnfire's suggestion to leave the option to change it after the fact is best. I think his idea is to connect all of the unused pins together, then leave that net unconnected to anything. If we want to connect it to GND, we can jumper it. This is best done with a small "solder jumper" part in Eagle, or a resistor part where we can load 0 ohm.

[1714274492]

1714274492

[1714274492]

1714274492

[1714274492]

1714274492

[bahnfire]

What I was saying RE: the extra pins was to take as many as you can and connect to individual test points not to just one. If you are working on a prototype this is the way to do it since you have easy access to pins if you need extras and will not need to do a whole new board spin (especially since this is a BGA). For production you can remove as many of these points as you can without sacrificing your future debug capability.

My opinion (and based on many years of experience): The goal should NOT be to design the first prototype(s) with the final dimensions (PCB size) as this will bit you later. You should always build a larger PCB that has all components and maybe a few extra items for testing to allow for easier debugging, dead bugging, and testing. Once the prototype is proven you can move to a smaller iteration of the PCB. Designs similar to this module could take several PCB runs from prototype to production.

If we do not do this we will run the risk of this adventure costing a lot of money and time.

[O_Shovah]

What I was saying RE: the extra pins was to take as many as you can and connect to individual test points not to just one. If you are working on a prototype this is the way to do it since you have easy access to pins if you need extras and will not need to do a whole new board spin (especially since this is a BGA). For production you can remove as many of these points as you can without sacrificing your future debug capability.

My opinion (and based on many years of experience): The goal should NOT be to design the first prototype(s) with the final dimensions (PCB size) as this will bit you later. You should always build a larger PCB that has all components and maybe a few extra items for testing to allow for easier debugging, dead bugging, and testing. Once the prototype is proven you can move to a smaller iteration of the PCB. Designs similar to this module could take several PCB runs from prototype to production.

If we do not do this we will run the risk of this adventure costing a lot of money and time.

I see your point.
Im willing to invest a certain amount of money but as a student i will be limited to a few hundred Euro.
Creating a testable board sure is a good way i second that.

Does somebody disagree or as some founded other ideas? 

[bahnfire]

I am willing to invest some of my resources as needed - I have access to many supply chains and resources (my company has a factory in China and I can help purchase items from there). I also have a full test lab (oscilloscopes, testers, etc) and have contacts with many US and China PCB/SMT houses for prototype and production.

[fizzisist]

I agree with bahnfire that the prototyping will be much easier if we don't try to fit it to the final dimensions, and we should include as much flexibility and debug-ability into it as possible. On the other hand, though, we should do our best to make this board useful as a mining board if we're lucky and it happens to work. For that reason, it should include all of the features of the real board, including USB and DIMM connector.

On the same line of thought, I'd be willing to put up part of the investment for a 3 or 4 board prototype run, if I could get one of the boards. I'm sure others feel the same way. In this way, we will have a sort of private beta test, where the people willing to put up the money can participate.

I also have oscilloscopes, power supplies, JTAG programmers, Xilinx license, etc., at my disposal so can help with the testing and debugging.

[li_gangyi]

What I was saying RE: the extra pins was to take as many as you can and connect to individual test points not to just one. If you are working on a prototype this is the way to do it since you have easy access to pins if you need extras and will not need to do a whole new board spin (especially since this is a BGA). For production you can remove as many of these points as you can without sacrificing your future debug capability.

My opinion (and based on many years of experience): The goal should NOT be to design the first prototype(s) with the final dimensions (PCB size) as this will bit you later. You should always build a larger PCB that has all components and maybe a few extra items for testing to allow for easier debugging, dead bugging, and testing. Once the prototype is proven you can move to a smaller iteration of the PCB. Designs similar to this module could take several PCB runs from prototype to production.

If we do not do this we will run the risk of this adventure costing a lot of money and time.

This would be why we need those FPGA guys right now, what kinda IOs are gonna be required? Is jtag and what not we have now sufficient? If we need more IOs that what we have planned now, I don't think a 4 layer board is gonna to suffice, if we want to route out ALL the IOs, that's gonna take at least another 2 layers IMO.

Do we need some sorta external memory for 'bitcoining'? If we later decide that we need it, it's going to require a new run of boards, there's no easy way to add the functionality later. I.E even with all the IOs routed out it's not much help. I'm willing to fork out money to produce the 1st batch of boards, but without these critical bits of info, we're shooting in the dark. We're gonna need some guy or girl for that matter who has dabbled with this, and knows for sure what we need, otherwise we'll just end up producing a dev board in the end to get to the results.

We're working on a smallish budget, let's put in solutions that are known to work, instead of stuff that 'should work', if it's new ground, we should trash it out and discuss it before putting it into the design.

[O_Shovah]

So ok lets agree on a test board.

Just to throw some numbers in here:

Maybe 150mm x 100 mm maximum size so there's no need for space optimisation in the first place.

Barrel connector , Molex connector for power . Power supply as planned by li_gangyi

Msp 430 and mini usb port

FPGA routing as Olaf.Mandels setup but with the options to rout unused IO pins either to ground , Vaux or left floating by Jumper settings.
(other modifications essentially nessesary ?)

I would leave away the 240 Pins of the DIMM card just to keep it simple for the first time.

Pin header for the Spi BUS system normaly going to the motherboard.


Another idea would be to produce each of the subparts ( power supply , FPGa setup, Msp430 and USB) on an individual board (maybe two of the MSP430and USB boards to simulate motherboard) to make it possible to test them in their one all being connected by pinheaders and jumpercables.Aso this would only need one 4 layer and 2 two layer boards.
(this last one is just a option i dont know how much the boards signals ar affected by long connection lines)


@li_gangyi:
it seemed Bahnfire had some experience on this matter. I personally remember the fpga's internal memory to be sufficient. Maybe our FPGA programmers could also give a hint on that subject.

[phillipsjk]

Edit: It has been decided I was going off on an unnecessary tangent here.

I created a first try at a very high level block diagram. Mainly, my idea was to have a "straw man" that we can all have in mind when we discuss. Hopefully, it will get better as the ideas get more refined.

This is uploaded to the dropbox at Documentation/Block_Diagram, in PPT format. Here's a link to a PNG of the diagram, so that those without access can see it, too:

http://dl.dropbox.com/u/13472215/block_diagram_daughter.png

Please point out any mistakes I made or edit the file yourself.

If we want the MCU to be able to turn the power supplies for the FPGAs on and off, it should be bus powered. If USB is being used, we are talking  5 volts, 500mA (100mA without permission).

If the FPGAs have a small enough 2.5V current draws, no extra part is needed. According to page 7 of the datasheet "...Spartan-6 devices do not have a required power-on sequence." (Table 6 note 2).

I couldn't really find anything saying you can power up V_ccaux and V_cco with V_ccint unpowered more explicitly.

[Olaf.Mandel]

Olaf.Mandel, I'm logged in to Dropbox but I don't see your picture posted in the forum. I think you need to place the image in your Public folder, then copy that link. [...]

I assume it is because you have not been given access to the directory. I will post the stuff somewhere else in addition to dropbox.

[Olaf.Mandel]

[...]
This would be why we need those FPGA guys right now, what kinda IOs are gonna be required? Is jtag and what not we have now sufficient? If we need more IOs that what we have planned now, I don't think a 4 layer board is gonna to suffice, if we want to route out ALL the IOs, that's gonna take at least another 2 layers IMO.

For mining you need three things: you need to be able to load a bitstream into the FPGA, you need a clock and you need to communicate with the firmware inside the FPGA afterwards. A slow communication link (>1kbit/s) would even do, but more is (slightly) better. That is all: no extra pins needed. For debugging, an extra LED or a few pins may be useful, but as you have to have a working connection to talk to the thing, it is not critical in my point of view (I will bow to the greater experience of others, though).

Routing all IOs out is a complete waste of money if you are building a mining board. If you do something else like building a video-transcoder, a DNA folding computer or whatnot, it may help. But for mining? If they had the FPGA in a 50-pin housing, it would be plenty!

Do we need some sorta external memory for 'bitcoining'? If we later decide that we need it, it's going to require a new run of boards, there's no easy way to add the functionality later. I.E even with all the IOs routed out it's not much help. I'm willing to fork out money to produce the 1st batch of boards, but without these critical bits of info, we're shooting in the dark. We're gonna need some guy or girl for that matter who has dabbled with this, and knows for sure what we need, otherwise we'll just end up producing a dev board in the end to get to the results.

We're working on a smallish budget, let's put in solutions that are known to work, instead of stuff that 'should work', if it's new ground, we should trash it out and discuss it before putting it into the design.

No memory needed. The basic algorithm has been designed to not need a lot of memory. Look at the code others have announced in this forum thread. It already works well, even if there is no version directly for our FPGA. If you read the HDL code, you see that calculating SHA-256 amounts to a lot of register pipelines, but no memory needed.

[li_gangyi]

Alright, so we're pretty much basically set in terms of IOs required. Do we still want to route IOs out? IMO it's a waste of time, but if it helps debugging I'm all for it.

[Olaf.Mandel]

I uploaded a new version of the board to both dropbox and github:



The new version fixes a bug where M1 of both FPGAs had the wrong connection (GND instead of VCC_O). I only uploaded the schematic and board, not a new PDF or PNG. You cannot even see it in the bitmap and in the PDF of the schematic: just imagine that M1 is connected differently or open the actual Eagle files   

[Olaf.Mandel]

I was trying to route the FPGAs with all unused pins on a new net (for later connection to whatnot). This pretty much destroys my GND layout. GND would have to go to a different layer and the top polygon would have to become this NCPINS signal. How important is that, actually? I would hate to do that if it turns out that this was just a nice to have feature that noone really needs, even for debugging.

[li_gangyi]

Yeah, that's what I was afraid of, let's not do that, in fact I think we can leave Vccaux and Vccio seperate still and join them upstream later in case we need to put in some filters to make it work. We still need to split the Vccint. I'm trying to route in the PSU as part of the board.

[fizzisist]

Olaf.Mandel, I'm logged in to Dropbox but I don't see your picture posted in the forum. I think you need to place the image in your Public folder, then copy that link. [...]

I assume it is because you have not been given access to the directory. I will post the stuff somewhere else in addition to dropbox.

I do have access to the Dropbox. Your new method works perfectly though, so stick with that.

I created a first try at a very high level block diagram. Mainly, my idea was to have a "straw man" that we can all have in mind when we discuss. Hopefully, it will get better as the ideas get more refined.

This is uploaded to the dropbox at Documentation/Block_Diagram, in PPT format. Here's a link to a PNG of the diagram, so that those without access can see it, too:

http://dl.dropbox.com/u/13472215/block_diagram_daughter.png

Please point out any mistakes I made or edit the file yourself.

If we want the MCU to be able to turn the power supplies for the FPGAs on and off, it should be bus powered. If USB is being used, we are talking  5 volts, 500mA (100mA without permission).

If the FPGAs have a small enough 2.5V current draws, no extra part is needed. According to page 7 of the datasheet "...Spartan-6 devices do not have a required power-on sequence." (Table 6 note 2).

I couldn't really find anything saying you can power up V_ccaux and V_cco with V_ccint unpowered more explicitly.

This MCU, MSP430F55xx, has the ability to be partially bus powered with a built in LDO. The main purpose of this is to set the level of the USB I/Os independently of the other I/Os. We can't power the entire chip with USB, or we will have to use level shifters to interface with the FPGAs. Furthermore, when the board is used in the motherboard, this USB connection will not be present, so it needs to be powered off the ATX supply.

I was trying to route the FPGAs with all unused pins on a new net (for later connection to whatnot). This pretty much destroys my GND layout. GND would have to go to a different layer and the top polygon would have to become this NCPINS signal. How important is that, actually? I would hate to do that if it turns out that this was just a nice to have feature that noone really needs, even for debugging.

I have a feeling it's mostly unnecessary and definitely complicates things. Like I said earlier, though, I can't seem to find a good answer in the documentation about how best to handle those unused pins. One possibility seems to be to connect them all to GND, then set them as outputs driven low. We need someone with experience to make a decision about this. You could post a question on the Xilinx forums...

[Olaf.Mandel]

Actually, making the voltage on the NC pins selectable pretty much requires you to merge VCCAUX + VCCIO to free up a layer.

Correct me if I am wrong: putting all unneeded pins onto an extra net allows to test things like power consumption during configuration and operation for different connected voltages, maybe check if some weird effect makes one voltage connection more stable than the other. But except for some esoteric problem, there is not much difference between VCCIO and GND on the pins (if the FPGA bitpattern matches that).

The merging of the VCCAUX and VCCIO is "bigger", though: it frees up a layer, but we actually have to test if there is really no filter needed (National Webbench recommends one).

[O_Shovah]

I created a first try at a very high level block diagram. Mainly, my idea was to have a "straw man" that we can all have in mind when we discuss. Hopefully, it will get better as the ideas get more refined.

This is uploaded to the dropbox at Documentation/Block_Diagram, in PPT format. Here's a link to a PNG of the diagram, so that those without access can see it, too:

http://dl.dropbox.com/u/13472215/block_diagram_daughter.png

Please point out any mistakes I made or edit the file yourself.

If we want the MCU to be able to turn the power supplies for the FPGAs on and off, it should be bus powered. If USB is being used, we are talking  5 volts, 500mA (100mA without permission).

If the FPGAs have a small enough 2.5V current draws, no extra part is needed. According to page 7 of the datasheet "...Spartan-6 devices do not have a required power-on sequence." (Table 6 note 2).

I couldn't really find anything saying you can power up V_ccaux and V_cco with V_ccint unpowered more explicitly.


This MCU, MSP430F55xx, has the ability to be partially bus powered with a built in LDO. The main purpose of this is to set the level of the USB I/Os independently of the other I/Os. We can't power the entire chip with USB, or we will have to use level shifters to interface with the FPGAs. Furthermore, when the board is used in the motherboard, this USB connection will not be present, so it needs to be powered off the ATX supply.

I was trying to route the FPGAs with all unused pins on a new net (for later connection to whatnot). This pretty much destroys my GND layout. GND would have to go to a different layer and the top polygon would have to become this NCPINS signal. How important is that, actually? I would hate to do that if it turns out that this was just a nice to have feature that noone really needs, even for debugging.

I have a feeling it's mostly unnecessary and definitely complicates things. Like I said earlier, though, I can't seem to find a good answer in the documentation about how best to handle those unused pins. One possibility seems to be to connect them all to GND, then set them as outputs driven low. We need someone with experience to make a decision about this. You could post a question on the Xilinx forums...

I also agree the MSP should be powered by the power solution of the fpga.The level shifters would add adtitional parts and complicate things so we my use this in another version.

Regarding the IO pins of the FPGA's: If there is no really pressing need for unused IO pins to be accesible for debugging i vote against routing them out as it would produce a lot more work and complicate design.
Altough it might be an option to route a few pins in case they are really nessecary.

Please someone state if we really need those routing out as otherwise we would just skip it.   

[li_gangyi]

Uploaded new sch and brd files to the main folder, which combines both the FPGA routed so far and the PSU, I haven't layed out everything yet, about to hit the sack, it's now 4AM here.

What I've done:
1. Put layer names for better clarity
2. Put in the PSU in a new sheet
3. Layout most of the components on the board


Need to do:
1. Split up the VccInt for both FPGAs (I'm planning to power them seperatly)
2. Complete routing the PSU
3. Maybe provide a little bit of seperation for the Jtag traces, I don't like how they are all in the same plane right now.
4. Put in suitable clock(s).
5. Put in the MSP430

[TheSeven]

Right now I'm not too worried about the hardware design, I can see active development, but no software dev has stepped up to offer help just yet (or maybe I'm just missing their posts). I'm no good with MSP430s just yet, so I can't help here.

There were several guys who promised to help with the software side of things.
I'm generally willing to help, and have limited experience with Xilinx, but could do µC stuff as well if need be.
However there were people who claimed to know the MSPs by heart, so that's probably their job.

Pusle was the one telling me of his 150mhash/s in ISE. He said the design wasn't complete and that it should be possible to pump out much more than that. He (or someone) said that FPGAMiner was able to achieve 190mhash/s on the LX150. But I haven't really been able to find any posts from FPGAMiner stating this.

The 190MH/s claim originally came from ArtForz as well. He also claimed to build a 6GH/s FPGA rig (~350W total) for ~$6-8k.

[pusle]

Please guys, use a full unbroken GND plane for the entire board ( preferrably on layer 2 right below the FPGA )
The core voltage should also be a power plane under the fpga (layer 3 is best), extending out to the decoupling caps (as unbroken as possible).  Feedback to the core regulator should come from the power plane so it "sees" what the fpga "feels" ^_^

Just route out the IO pads you get "for free" down the the connector. 4 Layers should give you plenty without having to compromise on power integrity.
Merging VCCAUX  and VCCIO is smart as there is very little IO toggling to generate noise into the PLL's anyway.
Good decoupling is always a must.
Add decoupling footprints at the back of the board directly under the FPGA just in case. (not mounted for now)

Have a look here:  http://www.xilinx.com/support/documentation/user_guides/ug393.pdf   
The decoupling section is kinda overkill but lots of nice tips on several topics.
My recommendation is to add a few 4.7 or 10uf caps and several 100nF as near the power pins as possible for each  rail.
VccInt is the one to give priority if there is a conflict.

I'm sorry if this stuff is well know to you already,  but better safe than sorry