FPGA development board "Icarus" - DisContinued/ important announcement

[kano]

...

i will add a temperature sensor on the next design, but IMHO, it's useless.
because it can't be installed inside the FPGA package, and the temperature read out is just a board temp. the response time is not quick enough for protection purpose.

board interconnect can be done now, only need a bitsteam modify. this feature is still under development.

The temperature sensor would not be for the sake of a microsecond shut down - but more for noting a trend to reduce the work to help reduce the temperature.
I don't run my GPU cards near the limits - I set them below those limits to promote a longer life

There is, however, a rather simple need for it: if the fan should fail, you would see a temperature rise and possibly have the software notice this and reduce the amount of work or even stop it from processing.
It may not be fast enough - but I guess only experimentation would tell?

Different subject ...
Actually one thing I've not asked or found written anywhere yet: is it possible to upload a new bitsteam using USB?
Or do you need the extra developer hardware to do that?

[1714227363]

1714227363

[1714227363]

1714227363

[ngzhang]

...

i will add a temperature sensor on the next design, but IMHO, it's useless.
because it can't be installed inside the FPGA package, and the temperature read out is just a board temp. the response time is not quick enough for protection purpose.

board interconnect can be done now, only need a bitsteam modify. this feature is still under development.

The temperature sensor would not be for the sake of a microsecond shut down - but more for noting a trend to reduce the work to help reduce the temperature.
I don't run my GPU cards near the limits - I set them below those limits to promote a longer life

There is, however, a rather simple need for it: if the fan should fail, you would see a temperature rise and possibly have the software notice this and reduce the amount of work or even stop it from processing.
It may not be fast enough - but I guess only experimentation would tell?

Different subject ...
Actually one thing I've not asked or found written anywhere yet: is it possible to upload a new bitsteam using USB?
Or do you need the extra developer hardware to do that?

the speed adjust  feature should build in the bitsteam it self. the FPGA will automatically sample its temperature (with out a independent sensor device, just "it self" ) and rise and down the operation speed continuously.
why need to read the temperature, calculate, then adjust by sending a command from PC?  

about the bitsteam update: 

icarus need a platform cable to update the bitsteam. next product will have some changes.

[kano]

...

i will add a temperature sensor on the next design, but IMHO, it's useless.
because it can't be installed inside the FPGA package, and the temperature read out is just a board temp. the response time is not quick enough for protection purpose.

board interconnect can be done now, only need a bitsteam modify. this feature is still under development.

The temperature sensor would not be for the sake of a microsecond shut down - but more for noting a trend to reduce the work to help reduce the temperature.
I don't run my GPU cards near the limits - I set them below those limits to promote a longer life

There is, however, a rather simple need for it: if the fan should fail, you would see a temperature rise and possibly have the software notice this and reduce the amount of work or even stop it from processing.
It may not be fast enough - but I guess only experimentation would tell?

Different subject ...
Actually one thing I've not asked or found written anywhere yet: is it possible to upload a new bitsteam using USB?
Or do you need the extra developer hardware to do that?

the speed adjust  feature should build in the bitsteam it self. the FPGA will automatically sample its temperature (with out a independent sensor device, just "it self" ) and rise and down the operation speed continuously.
why need to read the temperature, calculate, then adjust by sending a command from PC?  
...

As I said, I prefer to have my hardware run a little bit below suggested - and the software would simply reduce the amount of work or stop sending work.
Just like cgminer now does with GPU cards - you can decide on the temperature limit - the GPU cards themselves also have an internal shutdown - but few if anyone lets the cards go that high.

If there is only hardware control and no software control, then ALL boards would have to work exactly the same and have exactly the same limitations - and the hardware control would have to always be correct.

Just like my comments about being able to adjust the MHz of the board, in that case I'd have it err on the side of safety - others may prefer to risk and gain a little more performance.
I want my hardware to last a long time - so I'll prefer to have it run a little lower than some who may prefer to gain that extra performance.

[needbmw]

the speed adjust  feature should build in the bitsteam it self. the FPGA will automatically sample its temperature (with out a independent sensor device, just "it self" ) and rise and down the operation speed continuously.
why need to read the temperature, calculate, then adjust by sending a command from PC?  

For large rig with 12-16 boards packaged in the suitable case, it will be better to use two big fans instead of dozen smaller ones. Their rotation speed (and thus consumed power and noise level) may be adjusted by cgminer depending on highest temperature readout of all FPGAs (as cgminer does for GPU).
So please consider FPGA temperature readout in your future design if possible, regardless of sensor type you will use.

Will be very nice if you support SPI in the future bitstream.
I'm going to start developing such carrier board, but we should agree with daisy-chaining protocol details and pinout first.

[allinvain]

Hi ngzhang,

I've heard you are developing next hardware revision of Icarus board.
Below is my 0.02$, comments are welcome.

One necessary thing is FPGA temperature sensors. For example TI digital sensors with serial output will be OK but I have no idea how to mechanically put them to the chips (why FPGA manufacturer didn't place one directly to the die?)

Second idea (which can be easily implemented with current revision boards btw) is simple but fast SPI interface using some DIMM connector pins. SPI address and SPI/USB interface selection can be done with existing DIP switches. The goal is to develop neat carrier board with 12-16 Icarus slots, ARM/MIPS/ATOM/etc CPU board slot and good 250-350W PSU behind. We can install several big 200mm fans for the whole rack and adjust their rotation speed according to temperature sensors output.

i will add a temperature sensor on the next design, but IMHO, it's useless.
because it can't be installed inside the FPGA package, and the temperature read out is just a board temp. the response time is not quick enough for protection purpose.

board interconnect can be done now, only need a bitsteam modify. this feature is still under development.

BTW,

the most challenge for the design is to reduce the core temperature.

I think the ideal solution to reduce core temps would be to look for a heatpipe cooling solution. I'm not sure if there is a small enough heatpipe solution out there, but maybe you can get one custom built? Also some sort of heatsinking on the underside of the board at the fpga core would be ideal. But it's a shame that the lx150 doesn't have a metal heat spreader on it as the way I see it the major limitation to better cooling is the plastic top.

[ngzhang]

the speed adjust  feature should build in the bitsteam it self. the FPGA will automatically sample its temperature (with out a independent sensor device, just "it self" ) and rise and down the operation speed continuously.
why need to read the temperature, calculate, then adjust by sending a command from PC?  

For large rig with 12-16 boards packaged in the suitable case, it will be better to use two big fans instead of dozen smaller ones. Their rotation speed (and thus consumed power and noise level) may be adjusted by cgminer depending on highest temperature readout of all FPGAs (as cgminer does for GPU).
So please consider FPGA temperature readout in your future design if possible, regardless of sensor type you will use.

Will be very nice if you support SPI in the future bitstream.
I'm going to start developing such carrier board, but we should agree with daisy-chaining protocol details and pinout first.

for a large chain, i support CAN or RS485 for the linkage specification.

As I said, I prefer to have my hardware run a little bit below suggested - and the software would simply reduce the amount of work or stop sending work.
Just like cgminer now does with GPU cards - you can decide on the temperature limit - the GPU cards themselves also have an internal shutdown - but few if anyone lets the cards go that high.

If there is only hardware control and no software control, then ALL boards would have to work exactly the same and have exactly the same limitations - and the hardware control would have to always be correct.

Just like my comments about being able to adjust the MHz of the board, in that case I'd have it err on the side of safety - others may prefer to risk and gain a little more performance.
I want my hardware to last a long time - so I'll prefer to have it run a little lower than some who may prefer to gain that extra performance.

i understand your meaning.

next bitsteam we will set the frequency to let the core temperature below 85C. this can let the device work for ever. this means the frequency will adjust by core temperature but error rate.
the miner may report the frequency and core temperature, but not software adjustable.
because it's hard to realize when neither DCM or PLL enabled. they consumption too much power. we will use another way to adjust frequency.

and next bitsteam will really take some time to release...

I think the ideal solution to reduce core temps would be to look for a heatpipe cooling solution. I'm not sure if there is a small enough heatpipe solution out there, but maybe you can get one custom built? Also some sort of heatsinking on the underside of the board at the fpga core would be ideal. But it's a shame that the lx150 doesn't have a metal heat spreader on it as the way I see it the major limitation to better cooling is the plastic top.

yes, the reality is negative.
"any" attempt to reduce the core temperature by enhance cooling the chip's top is absolute useless. the LX150 FGG484 use a high - thermal resistance plastic package. the core temperature is much higher than the package surface. a heatpipe will not help at all.

there are some solutions now, but we still need to do some measurement. 

[nedbert9]

I think the ideal solution to reduce core temps would be to look for a heatpipe cooling solution. I'm not sure if there is a small enough heatpipe solution out there, but maybe you can get one custom built? Also some sort of heatsinking on the underside of the board at the fpga core would be ideal. But it's a shame that the lx150 doesn't have a metal heat spreader on it as the way I see it the major limitation to better cooling is the plastic top.

yes, the reality is negative.
"any" attempt to reduce the core temperature by enhance cooling the chip's top is absolute useless. the LX150 FGG484 use a high - thermal resistance plastic package. the core temperature is much higher than the package surface. a heatpipe will not help at all.

there are some solutions now, but we still need to do some measurement. 

I'm thinking harder about immersion cooling with dielectric fluid due to FPGA package complications and desire to protect investment.

[ngzhang]

I think the ideal solution to reduce core temps would be to look for a heatpipe cooling solution. I'm not sure if there is a small enough heatpipe solution out there, but maybe you can get one custom built? Also some sort of heatsinking on the underside of the board at the fpga core would be ideal. But it's a shame that the lx150 doesn't have a metal heat spreader on it as the way I see it the major limitation to better cooling is the plastic top.

yes, the reality is negative.
"any" attempt to reduce the core temperature by enhance cooling the chip's top is absolute useless. the LX150 FGG484 use a high - thermal resistance plastic package. the core temperature is much higher than the package surface. a heatpipe will not help at all.

there are some solutions now, but we still need to do some measurement. 

I'm thinking harder about immersion cooling with dielectric fluid due to FPGA package complications and desire to protect investment.

some complex ways are:

1, use semiconductor refrigeration chip.
2, use Compressor refrigeration.
3, use liquid nitrogen or dry ice.


but certainly, we will use a more simple way.... 

[antirack]

I just saw this today. Liquid submerged servers.

http://www.hardcorecomputer.com/liquid-blade-video/index.html

[ngzhang]

I just saw this today. Liquid submerged servers.

http://www.hardcorecomputer.com/liquid-blade-video/index.html

why not put this money to buy some more cards? 

[allinvain]

but certainly, we will use a more simple way.... 

Care to give us some clues  ?

[needbmw]

for a large chain, i support CAN or RS485 for the linkage specification.

Not a good idea in my opinion.

RS-485 is the same old-school as RS-232, it is slow (115200), unreliable, etc, but it may be kilometers long (at 2400 baud, this option definitely not for Icarus
CAN is definitely faster and reliable but is very complicated, I've worked with CAN node programming about a decade ago and it was a hell.

And both are not supported by Raspberry Pi.

All we need - fast synchronous addressed shift register (SPI). No overhead to FPGA design, easy to implement and debug, also we will reduce overall rig stale rate due to fast syncronous data exchange between master CPU and FPGAs.

And Raspberry PI has 3 SPI ports (I hope it is true).

[DeathAndTaxes]

I think the ideal solution to reduce core temps would be to look for a heatpipe cooling solution. I'm not sure if there is a small enough heatpipe solution out there, but maybe you can get one custom built? Also some sort of heatsinking on the underside of the board at the fpga core would be ideal. But it's a shame that the lx150 doesn't have a metal heat spreader on it as the way I see it the major limitation to better cooling is the plastic top.

yes, the reality is negative.
"any" attempt to reduce the core temperature by enhance cooling the chip's top is absolute useless. the LX150 FGG484 use a high - thermal resistance plastic package. the core temperature is much higher than the package surface. a heatpipe will not help at all.

there are some solutions now, but we still need to do some measurement. 

I'm thinking harder about immersion cooling with dielectric fluid due to FPGA package complications and desire to protect investment.

With a water chiller to keep fluid at <25C.

[rjk]

for a large chain, i support CAN or RS485 for the linkage specification.

Not a good idea in my opinion.

RS-485 is the same old-school as RS-232, it is slow (115200), unreliable, etc, but it may be kilometers long (at 2400 baud, this option definitely not for Icarus
CAN is definitely faster and reliable but is very complicated, I've worked with CAN node programming about a decade ago and it was a hell.

And both are not supported by Raspberry Pi.

All we need - fast synchronous addressed shift register (SPI). No overhead to FPGA design, easy to implement and debug, also we will reduce overall rig stale rate due to fast syncronous data exchange between master CPU and FPGAs.

And Raspberry PI has 3 SPI ports (I hope it is true).

RS-485 has more than enough speed to support bitcoin mining, and you can daisy-chain it fine. CAN bus sounds interesting, but I dunno how well it would work with all the devices chattering at once.

[DeepBit]

RS-485 is the same old-school as RS-232, it is slow (115200), unreliable, etc, but it may be kilometers long (at 2400 baud, this option definitely not for Icarus
CAN is definitely faster and reliable but is very complicated, I've worked with CAN node programming about a decade ago and it was a hell.

And both are not supported by Raspberry Pi.

All we need - fast synchronous addressed shift register (SPI). No overhead to FPGA design, easy to implement and debug, also we will reduce overall rig stale rate due to fast syncronous data exchange between master CPU and FPGAs.

And Raspberry PI has 3 SPI ports (I hope it is true).

RS-485 is reliable and fast enough for mining.
But why would we even care about Raspberry Pi ?

[nedbert9]

for a large chain, i support CAN or RS485 for the linkage specification.

Not a good idea in my opinion.

RS-485 is the same old-school as RS-232, it is slow (115200), unreliable, etc, but it may be kilometers long (at 2400 baud, this option definitely not for Icarus
CAN is definitely faster and reliable but is very complicated, I've worked with CAN node programming about a decade ago and it was a hell.

And both are not supported by Raspberry Pi.

All we need - fast synchronous addressed shift register (SPI). No overhead to FPGA design, easy to implement and debug, also we will reduce overall rig stale rate due to fast syncronous data exchange between master CPU and FPGAs.

And Raspberry PI has 3 SPI ports (I hope it is true).

@ngzhang    If it is simple and does a good job at investment protection, then great.

PHY spec for RS-485 is 32 Mbps at 10m and full-duplex using 4 wire.  

[needbmw]

RS-485 is the same old-school as RS-232, it is slow (115200), unreliable, etc, but it may be kilometers long (at 2400 baud, this option definitely not for Icarus
CAN is definitely faster and reliable but is very complicated, I've worked with CAN node programming about a decade ago and it was a hell.

And both are not supported by Raspberry Pi.

All we need - fast synchronous addressed shift register (SPI). No overhead to FPGA design, easy to implement and debug, also we will reduce overall rig stale rate due to fast syncronous data exchange between master CPU and FPGAs.

And Raspberry PI has 3 SPI ports (I hope it is true).

RS-485 is reliable and fast enough for mining.
But why would we even care about Raspberry Pi ?

Because it is 35$ ARM platform, possible working (who knows . Please advise me another cost-effective solution for command&control of FPGA boards.
I have several linux-based dev.boards on my desktop now (TI OMAP, Atmel's AT91RM9200, Atheros MIPS) but they all are over-sized and cost more than $300 each. 

[needbmw]

ngzhang, below is my proposal to interconnect Icarus boards inside the rig.

Initial conditions:

boards in the rig: up to 16 (4-bits address)
physical interface: SPI (four 3.3V or 5V TTL signals - MOSI, MISO, SCK, /SS, all parallel between all boards and the CPU)
protocol: TDM over SPI

Communication flow:

Master CPU asserts /SS signal each macrocycle to reset all internal FPGA counters to zero state.
On the falling edge of /SS the macrocycle has started.
Each macrocycle cosist of 16 TDM frames and has a length of 16*64*8=8192 SCK clock cycles.
For first TDM frame (512 clock cycles) master CPU transmits via MOSI 64 bytes work buffer for board #0, for second TMD frame - for board #1 and so on.
Board can return found 8 byte nonce to master via MISO at any 64 bit boundary of its TDM frame (at clock cycles 0, 64, 128, 192, 256, 320, 384, 448 for board #0), OR only at beginning of each TDM frame, if we do not care about small delays in results reporting. If it is nothing to return it returns all zeroes.

With SCK frequency 10MHz full communication macrocycle for all 16 boards will take about 0.82 ms.
Of course you can add additional registers (FPGA status register, FPGA temperature, command register, etc) to a TDM frame if needed. Also we can add a CRC word to each TDM frame to check workbuffer contents and so on.

Note: FPGA can drive MISO line only inside it's assigned TDM frame, otherwise it should be HiZ.
MISO line will be pulled up on CPU side, so if CPU reads all 1s from the MISO for full TDM frame therefore corresponding board is not connected to the slot.

[DeepBit]

RS-485 is reliable and fast enough for mining.
But why would we even care about Raspberry Pi ?

Because it is 35$ ARM platform, possible working (who knows . Please advise me another cost-effective solution for command&control of FPGA boards.
I have several linux-based dev.boards on my desktop now (TI OMAP, Atmel's AT91RM9200, Atheros MIPS) but they all are over-sized and cost more than $300 each. 

I really doubt that many users will want to set this platform just for mining. And it's obviously much more difficult to get.
Everyone already has at least one PC that can be used for this without any significant CPU load, also any old PC can be bought almost for free.

There are also many mini-ATX solutions based on Atom, VIA or even AMD. Just plug it in and use any OS, including Windows.

[needbmw]

I really doubt that many users will want to set this platform just for mining. And it's obviously much more difficult to get.
Everyone already has at least one PC that can be used for this without any significant CPU load, also any old PC can be bought almost for free.

My goal is to develop complete mining system for up to 16 Icarus boards, including carrier motherboard, CPU board, 350W PSU, fans, and fit it to rack mount case. There is nothing impossible, but using USB or RS-485 to communicate inside the rig is impractical at my point of view.
The CPU board will work under embedded linux and will be provided with precompiled cgminer and all necessary tools to compile your own application if needed. This is why I want to use something like 35$ Rasbperry board.