Cairnsmore1 - Quad XC6SLX150 Board

[Bitbird]

@Luke-Jr @Lethos

Finally, after install some dependencies libraries I make bfgminer working on one of my rig (although still need to figure out how to enable GPU mining with bfgminer).

Code:

ECM 0:                | 379.8/379.8/397.8Mh/s | A:124 R:0 HW:0 U: 5.56/m
 ECM 1:                | 380.5/378.0/349.7Mh/s | A:109 R:0 HW:0 U: 4.88/m
 BFL 0:  51.2C         | 862.1/856.5/930.3Mh/s | A:290 R:0 HW:0 U:13.00/m

However, on the other rig with bfgminer I got:

Code:

All devices disabled, cannot mine!

@kano
Thanks! Will try that.

[1713487633]

1713487633

[makomk]

Yeah, that didn't add up to me either. I was always under the impression bitstreams can't effect it's "power" or "voltage", nor would it change automatically even if you pushed it hard via overclocking. The worst that happens is invalids or errors in the output.

Well, the FGG484 package that Enterpoint are using is apparently much worse at dissapating heat compared to the CSG484 package that Ztex's boards use, but...

[Lethos]

@Luke-Jr @Lethos

Finally, after install some dependencies libraries I make bfgminer working on one of my rig (although still need to figure out how to enable GPU mining with bfgminer).

Code:

ECM 0:                | 379.8/379.8/397.8Mh/s | A:124 R:0 HW:0 U: 5.56/m
 ECM 1:                | 380.5/378.0/349.7Mh/s | A:109 R:0 HW:0 U: 4.88/m
 BFL 0:  51.2C         | 862.1/856.5/930.3Mh/s | A:290 R:0 HW:0 U:13.00/m

However, on the other rig with bfgminer I got:

Code:

All devices disabled, cannot mine!

@kano
Thanks! Will try that.

Assuming you are pointing at the right usbports...

First do a power cycle, and then reconnect stuff and use the modprobe command, try again, did that help?
If not then try reflashing the board. Try again after doing the above.

[Lethos]

Yeah, that didn't add up to me either. I was always under the impression bitstreams can't effect it's "power" or "voltage", nor would it change automatically even if you pushed it hard via overclocking. The worst that happens is invalids or errors in the output.

Well, the FGG484 package that Enterpoint are using is apparently much worse at dissapating heat compared to the CSG484 package that Ztex's boards use, but...

Pretty sure the CM1 uses a bigger fan and better heatsinks though...

[yohan]

Interesting:

Some bitstreams such as some current “220” bitstreams can use excessive power and hence generate heat that is impossible to extract from the FPGA packaging. This may damage FPGAs and our warranties will not support boards damaged by running “220” bitstreams or similar. The maximum recommended bitstream is currently “200”. If in doubt email bitcoin.support@enterpoint.co.uk for a list of approved bitstreams.

Yohan, can you comment on this?

We have seen what appear to be one or two total internal FPGA failures in the field which we are still gathering data and information on. It may be coincidence but 220 bitstreams were running on these boards. From some of the work we have done here we can see there is usual a big increase in power used when 210 and 220 bitstreams are in used and that is what we are worried about. Tied to that there is very little real performance to be gained particluarly on a 220 bitstream on most boards and often boards will actually give better return on a 200 bitstream. So for now our official recommendation is 200 or less bitstreams. 210 is probably ok but we need more data to confirm that.

behind all of this there is an effect called thermal runaway that is likely to be happening where the presence of heat allows higher currents to flow and hence more heat generated again i.e. runaway. We think all of the higher end bitstreams above 180 have an noticable element of this it is really down to how bad. Plotting performance against current isn't linear as it should be if clock rate was the only effect so we are pretty sure that is what is happening. There will be variations of this effect with silicon and to some small degree an enviromental effect. The cooling solution we have on CM1 is good but fundamentially the FPGA package is the main limitation to extracting heat given the performance of what we have in cooling. If room temperature is cold that will help a little and conversely if the room is hot.

We think the real problem comes when the devices go unstable often manifesting as high invalids and it is possible in this senario the currents go even higher than when we have measured on "stable" boards. Eberon has made some comments on this previously in one of the threads somewhere. The "damaging effect" may be one of a couple of things. First is the die just getting to hot. It's also possible that internal bond wires fail with too high a current and something basically melts. It is going to take us a while to get anything more solid than we know now so that is about as much as I can tell you all now.  

[kakobrekla]

Bull, 220 is certainly not any real issue for spartan (mine is runing runing 250-270mhash). And I know for a fact it can go to 300 and above.

[tnkflx]

Interesting:

Some bitstreams such as some current “220” bitstreams can use excessive power and hence generate heat that is impossible to extract from the FPGA packaging. This may damage FPGAs and our warranties will not support boards damaged by running “220” bitstreams or similar. The maximum recommended bitstream is currently “200”. If in doubt email bitcoin.support@enterpoint.co.uk for a list of approved bitstreams.

Yohan, can you comment on this?

We have seen what appear to be one or two total internal FPGA failures in the field which we are still gathering data and information on. It may be coincidence but 220 bitstreams were running on these boards. From some of the work we have done here we can see there is usual a big increase in power used when 210 and 220 bitstreams are in used and that is what we are worried about. Tied to that there is very little real performance to be gained particluarly on a 220 bitstream on most boards and often boards will actually give better return on a 200 bitstream. So for now our official recommendation is 200 or less bitstreams. 210 is probably ok but we need more data to confirm that.

behind all of this there is an effect called thermal runaway that is likely to be happening where the presence of heat allows higher currents to flow and hence more heat generated again i.e. runaway. We think all of the higher end bitstreams above 180 have an noticable element of this it is really down to how bad. Plotting performance against current isn't linear as it should be if clock rate was the only effect so we are pretty sure that is what is happening. There will be variations of this effect with silicon and to some small degree an enviromental effect. The cooling solution we have on CM1 is good but fundamentially the FPGA package is the main limitation to extracting heat given the performance of what we have in cooling. If room temperature is cold that will help a little and conversely if the room is hot.

We think the real problem comes when the devices go unstable often manifesting as high invalids and it is possible in this senario the currents go even higher than when we have measured on "stable" boards. Eberon has made some comments on this previously in one of the threads somewhere. The "damaging effect" may be one of a couple of things. First is the die just getting to hot. It's also possible that internal bond wires fail with too high a current and something basically melts. It is going to take us a while to get anything more solid than we know now so that is about as much as I can tell you all now.  

Are there any signs of hardware problems in cgminer or bfgminer before everything goes bad? (Invalids, HW errors, ...?)

[ShadesOfMarble]

yohan, unless invalid (i.e. difficulty < 1) shares are below a certain level (5%?) using high-performance bitstreams should not be an issue. Do you disagree on that?

[yohan]

yohan, unless invalid (i.e. difficulty < 1) shares are below a certain level (5%?) using high-performance bitstreams should not be an issue. Do you disagree on that?

The number that can be used as good or bad can be debated. It is only an indicator and I doubt totally accurate. I would suggest a slow bitstream with less invalids might be a better earner anyway if you were at 5% invalids. The invalid number also has it's own debate here about what it actually means.

In CGminer I would think the U value will also give an indication of this.

We are doing more work on this and should know more as we gather more data about what causes an issue.

[gyverlb]

@yohan : when the Cairnsmore1 won't be profitable anymore (probably during 2013 depending on actual ASIC deliveries), lots of us will either try to use them for something else or sell them.

Is Enterpoint interested in setting up a buy-back program? Even if the board itself doesn't interest you, I suppose cheap Spartans would? Any thought on this?

[ebereon]

@yohan : when the Cairnsmore1 won't be profitable anymore (probably during 2013 depending on actual ASIC deliveries), lots of us will either try to use them for something else or sell them.

Is Enterpoint interested in setting up a buy-back program? Even if the board itself doesn't interest you, I suppose cheap Spartans would? Any thought on this?

+1

Interessting question.

[ShadesOfMarble]

The number that can be used as good or bad can be debated. It is only an indicator and I doubt totally accurate.

ztex boards don't have temperatur sensors, so the invalid rate is used as overheat protection (invalids rising too fast => board shut down). Furthermore, dynamic clocking is also attached too the invalid rate, i.e. the clock is rising unless as certain level of invalids is reached. Many ztex boards run at 220 Mhz or even faster and you don't hear any problems. So my guess would be that the invalid rate *is* a good indicator whether you are pushing the fpga too much, IOW 220 Mhz and low invalids => no problem. (?)

[yohan]

@yohan : when the Cairnsmore1 won't be profitable anymore (probably during 2013 depending on actual ASIC deliveries), lots of us will either try to use them for something else or sell them.

Is Enterpoint interested in setting up a buy-back program? Even if the board itself doesn't interest you, I suppose cheap Spartans would? Any thought on this?

To do anything we would need a market to sell them into or their processing power and whilst we have heard of a couple of small customers that might take a few CM1s at the right price there is nothing currently that we know of that would take that many boards.

The chips themselves don't really have a resale market unless they become obsolete and hard to get. That's about a 15-20 years wait. The costs in time and money to recover FPGAs from boards is very large and they would be hard to sell because of "quality" issues related to recovery process. The recovery costs would almost certainly exceed whatever resale value there might be in the chips.

[salty]

The cooling solution we have on CM1 is good but fundamentially the FPGA package is the main limitation to extracting heat given the performance of what we have in cooling. If room temperature is cold that will help a little and conversely if the room is hot.

Does the rev 1.5 controller firmware supply temperature information in some way?

[LazyOtto]

OK, with this new announcement about warrentee voiding, I have now dropped both my boards to makomk's 200mhz bitstream.

Previously, with both of them running at 210mhz, one of them was generating about 0.15% / 0.25% invalids and the other around 1.1% / 2.65%.

With the two boards in a master/slave setup and the USB cable connected to the 'better' one, I still had a 'USB fail' event after about seven days of running. (Had two failures within 24hrs with a USB cable connected to both / the 'worse' one.)

Hoping for even more stability/duration with the reduced MH/s flash.

I want to be able to go on a vacation without worries that they dropped offline on the first day I wasn't watching.

-- edit --

Oh, and possibly relevant, ambient temperature currently, over the last two weeks, hits a high of 80f where they are sitting.

[ShadesOfMarble]

@LazyOtto are you using an USB hub?

[LazyOtto]

@LazyOtto are you using an USB hub?

Yes.

One which has operated reliably for years with a variety of devices.

[yohan]

The cooling solution we have on CM1 is good but fundamentially the FPGA package is the main limitation to extracting heat given the performance of what we have in cooling. If room temperature is cold that will help a little and conversely if the room is hot.

Does the rev 1.5 controller firmware supply temperature information in some way?

To use the temperature sensors it will need bitstream support as these each temperature sensor hang off their respective adjacent FPGA. Even with such support the accuracy of these won't be great or even very responsive as they will effectively be measuring the PCB temperature and or the air temperature under the heatsink. That's a lot of delay/variation between actual die temperature and what is measured. It is one weakness in using Spartan-6 for Bitcoin mining that they don't have internal diodes for temperature measurement that more expensive parts do have. Otherwise we would be using that feature and have a much better measurement.

However they are not totally useless in thermal management and it is our intention to add some Controller support once we have the support in bitstreams. Until the bitstreams have the support and we know how to access them in there isn't much we can do.

When we designed CM1 we did obviously knew about these temperature measurement limitations so that is why we put so much work into the heatsink and fan solution. You might ask why we didn't use the CSG package for the Spartan-6. It is a bit better than the FGG that we use in transferring heat from the silicon die to the case surface. The biggest reason we didn't use the CSG was that we couldn't offer the price we did. The second big reason is that the project would have been 2 months later than it was. The FGG also wins over the CSG in some other respects in the PCB design. We would have had to use a lighter copper weight on the power planes with a CSG package and most likely the PCB would have been more expensive again affecting the product cost. The copper weight does have some bearing on the thermal solution as well but probably more importantly the electrical performance on the power supply side. So in summary it's a toss up designers choice of which is actually best. For a first attempt, in a new market, I don't think we did too bad as a product.

If we had the luxary of maybe 2-4 more months development time we might have done 2 or 3 prototype levels and established the better of the 2 package options but CM1 probably wouldn't have happened at all then.

[Bitbird]

@Lethos  Thanks for the advice. After modified some parameter it working fine now!

Based on the default setting of CM1 it gave 760 to 763Mh/s(avg) with bfgminer. Should it be enhanced after tweak the DIP switches or updating the Controller FPGA firmware and array FPGAs firmware? How could I know the version of my CM1 board's currently firmware for determine the necessity of updating?

[yohan]

@Lethos  Thanks for the advice. After modified some parameter it working fine now!

Based on the default setting of CM1 it gave 760 to 763Mh/s(avg) with bfgminer. Should it be enhanced after tweak the DIP switches or updating the Controller FPGA firmware and array FPGAs firmware? How could I know the version of my CM1 board's currently firmware for determine the necessity of updating?

Generally it's not easy to tell but anything recently sent out will most likely have Controller 1.5 and a 190 bitstream. Controller 1.6 when it releases will have a flash code to indicate what it is.