Bitcoin Forum
November 12, 2024, 02:07:44 AM *
News: Latest Bitcoin Core release: 28.0 [Torrent]
 
   Home   Help Search Login Register More  
Poll
Question: Wich FPGA shall be used on our prototype ?
Xilinx Spartan 6 LX 150 - 17 (70.8%)
Altera Cyclone IV 75k - 7 (29.2%)
Total Voters: 24

Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 [25] 26 27 28 29 30 31 32 33 »
  Print  
Author Topic: Modular FPGA Miner Hardware Design Development  (Read 119298 times)
bahnfire
Jr. Member
*
Offline Offline

Activity: 139
Merit: 1

The World’s First Blockchain Core


View Profile
July 27, 2011, 03:55:37 AM
 #481

For the MSP430 code - what functions will need to be written (have not looked at updated schematic yet). What clock rates will the communication bus need to run at? Do I need to come up with a protocol, or will this be a community effort? How do the community envision the USB connection (CDC, HID, etc)? For the connection to the mainboard - what will the packet look like?

▄▄▄▄▄▄▄▄▄▄▄ ▄ ■       SKYNET.co       ■ ▄ ▄▄▄▄▄▄▄▄▄▄▄
▐▬▬▬▬▬▬▬▬▬     PRIVATE SALE is LIVE     ▬▬▬▬▬▬▬▬▬▌
li_gangyi
Full Member
***
Offline Offline

Activity: 157
Merit: 100



View Profile
July 27, 2011, 04:44:28 AM
 #482

[...] For the FPGA, I suggest the ASEM1-100.000MHZ-LC-T: it's small (3.2x2.5mm2) and should be sufficiently stable. [...]

The oscillator you've suggested runs @ 3.3V, we'd need a 2.5V unit.

Quote
The design as it is on dropbox and github calls for the clock to be connected to IO_L30N_GCLK0_USERCCLK_2, pin AB13. The clock is shared between the two FPGAs (no need to have a dedicated clock for each). I will put the oscillator into the design, so it is no longer routed to the MCU.

I'm not sure how you plan to do this, to do it properly you'd need to buffer the oscillator and/or series termination at each end to reduce reflections.

FPGAminer: certainly good news, when this board get's made, I'm sure one of them will go to you.
newMeat1
Full Member
***
Offline Offline

Activity: 210
Merit: 100



View Profile
July 27, 2011, 05:41:09 AM
 #483

I'll be the voice of reason here... are you sure you guys want to populate 5 boards on the first batch? I think it's wiser to make 1 or 2. You know, get your feet wet before jumping in

fpgaminer
Hero Member
*****
Offline Offline

Activity: 560
Merit: 517



View Profile WWW
July 27, 2011, 06:00:45 AM
 #484

Quote
I'll be the voice of reason here... are you sure you guys want to populate 5 boards on the first batch? I think it's wiser to make 1 or 2. You know, get your feet wet before jumping in
Perhaps spin 5+ PCBs, and populate them one by one while testing each time? Assuming who ever is doing the soldering can do them one at a time. You could still order the parts up-front. If the boards turn out to have a mistake, then it's only one set of parts wasted (unless you can reflow the FPGA back off) and the others will just sit in their anti-static bags waiting for the revised PCBs.

Also, is the FPGA landing currently designed to be compatible with all Spartan-6's in the same package? The Spartan-6 chips are designed to allow for that, so the first prototypes could be tested with a far cheaper FPGA before trying out the expensive LX150.

Olaf.Mandel
Member
**
Offline Offline

Activity: 70
Merit: 10


View Profile
July 27, 2011, 06:07:44 AM
 #485

[...] For the FPGA, I suggest the ASEM1-100.000MHZ-LC-T: it's small (3.2x2.5mm2) and should be sufficiently stable. [...]

The oscillator you've suggested runs @ 3.3V, we'd need a 2.5V unit.

You are right, didn't pay attention to that. But the ASEMB-100.000MHZ-LY-T is a 2.5V capable oscillator that has 10ppm accuracy (enough?). The matching 25MHz oscillator is ASEMB-25.000MHZ-XY-T (higher temperature range, but the -LX-T version is non-stock at Diikey).

Quote
The design as it is on dropbox and github calls for the clock to be connected to IO_L30N_GCLK0_USERCCLK_2, pin AB13. The clock is shared between the two FPGAs (no need to have a dedicated clock for each). I will put the oscillator into the design, so it is no longer routed to the MCU.

I'm not sure how you plan to do this, to do it properly you'd need to buffer the oscillator and/or series termination at each end to reduce reflections.
[...]

I planned to just connect the oscillator to the pins and have a Thevenin termination on the other end of the line. pusle suggested to remove the termination and instead put a resistor (which value?) between the oscillator and the FPGA pins. What are your suggestions?
Olaf.Mandel
Member
**
Offline Offline

Activity: 70
Merit: 10


View Profile
July 27, 2011, 06:13:34 AM
 #486

[...]
Also, is the FPGA landing currently designed to be compatible with all Spartan-6's in the same package? The Spartan-6 chips are designed to allow for that, so the first prototypes could be tested with a far cheaper FPGA before trying out the expensive LX150.

I only looked at the XC6SLX75 to XC6SLX150, but those both fit the board. I guess the XC6SLX100 will also fit, but I didn't check. The XC6SLX??-?N will also fit, as there is just some missing functionality (not needed). The XC6SLX??T should not fit, as they put various supply voltages on pins that I considered on the NC net. They may work, but I honestly didn't look too closely at the datasheet there.
O_Shovah (OP)
Sr. Member
****
Offline Offline

Activity: 410
Merit: 252


Watercooling the world of mining


View Profile
July 27, 2011, 06:23:44 AM
 #487

@ fpgaminer:
Great surely gets us closer to a working prototype.Should give us nice first results Cheesy

@ bahnfire:
I certainly no expert on the software side,but i would prefere the HID version.
The frequency is not critical in my eyes as we dont need high bandwith. So rather slow for not getting into HF issues.
Please somebody more expertised in USB and software join up with you.
I dont know how easy it is to rout out all the debugging functions?  

@ li_gangyi:
I would also prefere a single frequency source for each FPGA to prevent ozcillation between them.

Of course we will build some boards in minor quantity first.
But as li_gangyi is doing the assembley himself he certainly may check if the design works at all when he finished the first one.  

Im also not suree about the interchangeablility of the Spartan series.
But i agree using the LX75 instead of the LX150 would cut costs an minimise risks regarding investments.

Comments?

fpgaminer
Hero Member
*****
Offline Offline

Activity: 560
Merit: 517



View Profile WWW
July 27, 2011, 07:01:00 AM
 #488

Quote
I only looked at the XC6SLX75 to XC6SLX150, but those both fit the board. I guess the XC6SLX100 will also fit, but I didn't check. The XC6SLX??-?N will also fit, as there is just some missing functionality (not needed).
How about, for example, the XC6SLX25-N3FGG484C. It's ~$50USD, one third the price of an LX150. If you can verify that both chips will happily use the same landing, then the first prototype can have that LX25 soldered on. Full testing of the board, FPGA, power supply, and FPGA with firmware can be performed. If everything looks good the remaining boards can be soldered with LX150s.

It may also be good for those who wish to sample the product before buying the more expensive LX150 version, or just generally play around with the board without feeling like they are risking destroying an expensive device.

Quote
For the MSP430 code - what functions will need to be written (have not looked at updated schematic yet). What clock rates will the communication bus need to run at? Do I need to come up with a protocol, or will this be a community effort? How do the community envision the USB connection (CDC, HID, etc)? For the connection to the mainboard - what will the packet look like?
I'm not involved in this design enough to give a perfectly accurate answer, but since no one else has chimed in yet I'll make some quick comments. I've written controllers for my FPGA firmware before, so I have a bit of experience with it already.

Communications between uC and FPGA:
The uC needs to deliver Work to the FPGA at least every 2^32/(MH/s) seconds. Supposing FPGA performance of 200MH/s that is every 21 seconds. However, Long Polling will result in massive spikes of BUS bandwidth, requiring ASAP deliver of Work to every FPGA.

Work going to the FPGA is a data packet with a raw size of 96+256 (352) bits of data, but could go up to 608 bits in highly optimized FPGA firmware.

With clever optimization it would be possible to eliminate the peak bandwidth issues caused by a Long Polling event. Hence I would suggest that the delivery of ~608bits in a 1/4th of a second should be sufficient at worst case. That's about 2.5Kbits/s of bandwidth. Any better than that increases the MH/s performance of the system as a whole during Long Poll events, which are typically devastating regardless.

The delivery of 608-bit Work units to at most 64 FPGAs over a 20 second period would also give under 2.5Kbit/s. So 2.5Kbit/s seems like a good target. Double or triple that for communication fluff.

The uC needs to pull successful calculations from the FPGA on average every 21 seconds. These Results will be buffered in a FIFO on the FPGA, so you do not need to worry about peak bandwidth here. As long as you can pull the data out a little bit faster than one per 21 seconds per FPGA then that is sufficient.

Results coming from the FPGA will have a raw size of 96+32 (128) bits. This is unlikely to be larger, but could potentially be 1024bits.

The uC may want to send various commands to the FPGA and pull back some information. For example:

Commands: Change Clock Frequency, Self Test
Queries: Version, Self Test Result, Current Nonce, Estimated Temperature, Current Clock Frequency, Fault Error


Communications between uC and PC:
The uC needs to talk to the PC to give information about the current status of the FPGAs, update stored firmware, report current hashing rate, enable/disable FPGAs, etc. The PC needs to give the uC new work, and indicate Long Polling events. The uC needs to give the PC results from the FPGAs.


If the uC is responsible for programming the FPGAs, it must do that. Hopefully the FPGAs will have fault detection and downclocking built-in. If that is the case the uC needs to report faults, and possibly re-enable FPGAs at lower clock speeds. This is under the assumption that the FPGAs might have the ability to adjust their internal clock speeds. If they don't then they'll only have two clock speeds: MAX and OFF, possibly more with the aid of firmwares.

... That's all I can think of off the top of my head. I hope it is useful.

Olaf.Mandel
Member
**
Offline Offline

Activity: 70
Merit: 10


View Profile
July 27, 2011, 08:14:17 AM
 #489

[...]
Quote
For the MSP430 code - what functions will need to be written (have not looked at updated schematic yet). What clock rates will the communication bus need to run at? Do I need to come up with a protocol, or will this be a community effort? How do the community envision the USB connection (CDC, HID, etc)? For the connection to the mainboard - what will the packet look like?
I'm not involved in this design enough to give a perfectly accurate answer, but since no one else has chimed in yet I'll make some quick comments. I've written controllers for my FPGA firmware before, so I have a bit of experience with it already.
[...]

Sorry for not saying anything earlier. The MCU has two main communication channels to the FPGAs, at least one of which needs to be implemented. One channel is the JTAG interface, where the FPGAs are connected in a standard JTAG ring. I unfortunately have no feeling how "normal" software on a PC interacts with a JTAG programmer, but the MCU should probably expose a similar interface to the PC via USB. Comments?

The second interface is SPI: the two FPGAs are connected in parallel, having different SSEL lines for addressing. Speaking in a layer model, the first layer is a serial interface engine for SPI, with the signals SCLK, MOSI, MISO, SSEL0 and SSEL1. This bus can also be used to program the FPGAs, for which a few extra signal lines are used: !PROGRAM0, !PROGRAM1 reset the respective FPGA, DONE and !IRQ inform the MCU about the status of the programming. !IRQ can also be an output from the MCU, delaying the configuration of the FPGAs.

The second layer should contain low-level functions: "send a binary buffer to FPGAs specified by a SSEL mask", "read a given bit number into a binary buffer from FPGAs specified by a SSEL mask", "reset FPGAs specified by PROGRAM mask", "Check status of DONE", "check status of IRQ", "install interrupt for IRQ"...

The third layer should contain functions like "reset and bootload an FPGA", "send new workload", "read golden nonce"... This layer depends on how the firmware inside the FPGAs works.

Additionally, the MCU must expose a selection of these functions (or all) to the PC in some format. I am always in favour of a serial connection with a text protocol, but that may be unnecessarily difficult.
Olaf.Mandel
Member
**
Offline Offline

Activity: 70
Merit: 10


View Profile
July 27, 2011, 08:21:04 AM
 #490

Quote
I only looked at the XC6SLX75 to XC6SLX150, but those both fit the board. I guess the XC6SLX100 will also fit, but I didn't check. The XC6SLX??-?N will also fit, as there is just some missing functionality (not needed).
How about, for example, the XC6SLX25-N3FGG484C. It's ~$50USD, one third the price of an LX150. If you can verify that both chips will happily use the same landing, then the first prototype can have that LX25 soldered on. Full testing of the board, FPGA, power supply, and FPGA with firmware can be performed. If everything looks good the remaining boards can be soldered with LX150s.
[...]

The XC6SLX25 seems to be a bit very small to me. The price difference to a XC6SLX75 is 38.02EUR (67.33EUR-29.31EUR). The smallest stock chip with avnet is the XC6SLX25-3FGG484C for 32.24EUR, anyway. I am basically objecting to not being able to use the prototypes for serious mining in case they work. It's a gamble of course: with the small FPGA you either loose less money if there is a bug or you throw away money if it works.
li_gangyi
Full Member
***
Offline Offline

Activity: 157
Merit: 100



View Profile
July 27, 2011, 09:02:59 AM
 #491

That's what I planned on doing, populate 1 board, test it out and if it works, populate the rest. I have no idea how to test the functionality yet except for the PSUs and stuff, unless some1 can come up with some smallish test firmware to be loaded on the FPGA + MCU.

We could just get 1 cheaper FPGA to populate on the 1st board, if that works (meaning the PCB is adequate), I'll then populate the rest of the boards with LX150s.

@Olaf: I'd think a 50Ohm resistor should suffice, however you'd need to route the oscillator traces to only have stray capacitances between it and ground (so it doesn't pickup stuff from power or other IOs). The oscillator you've in mind is pretty good (10ppm), however you'd be pissing away that performance if we do this, I'd prefer to have the oscillator dedicated to each FPGA and keep everything nice and tight, it doesn't take up much board space anyways.
Olaf.Mandel
Member
**
Offline Offline

Activity: 70
Merit: 10


View Profile
July 27, 2011, 09:35:30 AM
 #492

@Olaf: I'd think a 50Ohm resistor should suffice, however you'd need to route the oscillator traces to only have stray capacitances between it and ground (so it doesn't pickup stuff from power or other IOs). The oscillator you've in mind is pretty good (10ppm), however you'd be pissing away that performance if we do this, I'd prefer to have the oscillator dedicated to each FPGA and keep everything nice and tight, it doesn't take up much board space anyways.

So you think we need more oscillators. Would a better termination also suffice (reintroduction of the Thevenin termination) or is that still much worse than two separate oscillators?
Olaf.Mandel
Member
**
Offline Offline

Activity: 70
Merit: 10


View Profile
July 27, 2011, 12:20:58 PM
 #493

I uploaded a new version of the FPGA and PSU designs to github and dropbox. Commit logs:

Added 100MHz oscillator to FPGAs and added vias below switchers in PSU section.
li_gangyi
Full Member
***
Offline Offline

Activity: 157
Merit: 100



View Profile
July 27, 2011, 02:49:37 PM
 #494

You'd need seperate resistors to each FPGA and yes, 2 seperate oscillators is better then anything.  Grin
Olaf.Mandel
Member
**
Offline Offline

Activity: 70
Merit: 10


View Profile
July 27, 2011, 04:24:44 PM
 #495

You'd need seperate resistors to each FPGA and yes, 2 seperate oscillators is better then anything.  Grin

The reason why I am harping so much on keeping it down to one oscillator and using a suitable termination: we have the same problem with the SPI bus (clockrate there is 25MHz instead of 100MHz) and to a lesser extend also with JTAG. So I want to learn how to make the bus the correct way, and if at all possible I want to not use a star-shaped bus with the MCU at the centre.
pusle
Member
**
Offline Offline

Activity: 89
Merit: 10


View Profile
July 27, 2011, 05:24:57 PM
 #496


Just use one osc but make sure it has proper cmos drive output rail to rail (not clipped sine).
Then put two resistors close to the osc with individual trace from each resistor to each FPGA and with a continous GND plane under the traces from end to end.

infested999
Hero Member
*****
Offline Offline

Activity: 854
Merit: 500



View Profile
July 27, 2011, 06:29:30 PM
 #497

I am not sure if this was made clear in this thread or not, but I will bring the news regardless:

Over in the Open Source FPGA Miner thread we have succeeded in getting a working, and verified design running on my LX150T dev kit. I did verification against a live pool at 50MH/s. The design can currently be clocked up to 100MH/s. I thought you folks would enjoy the good news if you hadn't heard it yet Cheesy

Most of us believe the LX150 can push beyond 100MH/s, so we are working towards that end through either a dual-core design or higher clocking. I am hoping for anywhere between 150MH/s and 200MH/s.

I should also note that I use a DCM to adjust the incoming 100MHz clock to the desired frequency (I saw some discussion on that in this thread). I use the clkfx output, because it gives the range I need for testing weird frequencies. There should be no problems using a different frequency external clock, but 100MHz is what I am getting from the Avnet board.

Can you give us some info on exactly how many Watts of power this device uses?

              ▄███▄   ▄███▄
              █████   █████
      ▄███▄    ▀▀▀     ▀▀▀    ▄███▄
      █████     ▄██▄ ▄██▄     █████
       ▀▀▀ ▄██▄ ▀██▀ ▀██▀ ▄██▄ ▀▀▀
 ▄███▄     ▀██▀           ▀██▀     ▄███▄
 █████ ▄██▄                   ▄██▄ █████
  ▀▀▀  ▀██▀                   ▀██▀  ▀▀▀
                       ▄█
▄███▄ ▄██▄            ███ ███  ▄██▄ ▄███▄
█████ ▀██▀  ████      █████    ▀██▀ █████
 ▀▀▀         ▀███▄    ████           ▀▀▀
       ▄██▄    ████   ███     ▄██▄
 ▄███▄ ▀██▀     ▀███  ███     ▀██▀ ▄███▄
 █████            ███▄██           █████
  ▀▀▀              ▀████            ▀▀▀
                     ███
                     ███
                     ██
                   ███

████    ██
  ████    ██
    ████    ██
      ████    ██
        ████    ██
          ████    ██
          ████    ██
        ████    ██
      ████    ██
    ████    ██
  ████    ██
████    ██










White Paper
Yellow Paper
Pitch Deck
Telegram
LinkedIn
Twitter
Olaf.Mandel
Member
**
Offline Offline

Activity: 70
Merit: 10


View Profile
July 27, 2011, 07:54:06 PM
 #498

Just use one osc but make sure it has proper cmos drive output rail to rail (not clipped sine).
Then put two resistors close to the osc with individual trace from each resistor to each FPGA and with a continous GND plane under the traces from end to end.

The drive is no problem according to the data sheet (hopefully the load capacitance is close to 15pF). The problem is the star configuration: doing this for one signal is not so much of problem. But having to do that three or even five times makes the bus considerably wider. I know we first want to build "just" a two-FPGA board, but I still want to know how to build a scalable bus for later boards with more FPGAs. An option that uses a star configuration just seems wasteful to me.
pusle
Member
**
Offline Offline

Activity: 89
Merit: 10


View Profile
July 28, 2011, 02:46:15 AM
 #499

Just use one osc but make sure it has proper cmos drive output rail to rail (not clipped sine).
Then put two resistors close to the osc with individual trace from each resistor to each FPGA and with a continous GND plane under the traces from end to end.

The drive is no problem according to the data sheet (hopefully the load capacitance is close to 15pF). The problem is the star configuration: doing this for one signal is not so much of problem. But having to do that three or even five times makes the bus considerably wider. I know we first want to build "just" a two-FPGA board, but I still want to know how to build a scalable bus for later boards with more FPGAs. An option that uses a star configuration just seems wasteful to me.

When you're at 100MHz and especially for a clock signal you have to do this coz you want the steeper edges to have lowest jitter.
JTAG and the SPI are less critical and you can slow those edges down.  At say 25MHz you have one driver resistor and just use 10R resistors at the junctions splitoff ( and/or connector crossover points) and omit them for short T split lengths. At say 2MHz you just need one "slow down the edge rise/fall" resistor at the driver end.

For a bigger board you can/must use several  buffers for the clock and then you can make other (distributed) topologies rather than the usual star if you will.
Even with a star topology you don't have to route in a star, nor do they have to be the same length as each trace is individual after the resistor at the driver.

15pF is just a measurement point. It will be more "happy" with lower pF but still work with higher within reason.


magik
Newbie
*
Offline Offline

Activity: 44
Merit: 0


View Profile
July 28, 2011, 04:00:08 AM
Last edit: July 28, 2011, 04:20:26 AM by magik
 #500

li_gangyi: do you think there will be an inrush current issue with the voltage regulators you suggested?

The board I work on at work used to have issues with the regulators locking up if they could not grab a lot of amperage on power up.  At running mode, the system only sucks out like under 1 A @ 24 V, but when you power it on, if you limit the current at the power supply to somewhere around 2-3 A, it will lock up the regulators, and they never get to their full voltage.  Our system has a bit more stuff on it that's sucking out power on boot, like a DSP, FPGA, a bunch of 24-bit A2Ds, and probably a handful of other stuff I'm forgetting at the moment - and mebe that's part of the problem, all the chips turning on at the same time.  But I'm unsure if this is typical for voltage regulators, and I want to be sure that once this board gets fabbed/assembled, that it will actually turn on without some deadbugging.

for the clocks - I don't think phase synchronization between the two FPGAs in an issue - they shouldn't be communicating with each other, and thus it doesn't matter if the phase relationship between the two FPGAs is synchronous - everything will be talking to the MCU.  Personally I would say go with 2 clock crystals as you'll get a better clock signal this way - and we don't gain much from having the 2 clocks in phase - other than PCB area, routing, and crystal cost.  Having a pure clock signal with low jitter and an even duty cycle will prevent the FPGA from fucking up because to get a fast hash engine out of the FPGA we are going to have to push it to it's limit.  Just for this reason alone I would say go for 2 clock crystals.  But then again, if you guys are sure you won't get a polluted clock signal this way, then it is viable.

Also, if you guys add any more IO connections to the FPGA - keep them all in the same bank.  I believe the Spartan6 has restrictions on which signals can be used in which clock domains based on quadrant.  You can get by the ISE by forcing it, but it's typically not good practice because the tools will have to route that one signal a long distance, which will impact max core speed.  But the FPGA should have more than enough IO on that one Bank 2 you guys are already using.
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 [25] 26 27 28 29 30 31 32 33 »
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!