X6500 Custom FPGA Miner

[fizzisist]

would this fit on an 6500 rev 3? http://compare.eu/608683

I think it would, since it appears to have a "universal" mount. Without dimensional drawings, though, it's impossible to tell. It could be that those arms would run into some other components on the board or something. Have you seen a drawing of it with dimensions anywhere? Or do you have one that you could measure on hand? If it's helpful, we do have some drawings of the X6500, including a Google Sketchup file, at http://fpgamining.com/documentation/hardware/drawings.

[antirack]

http://www.alphacool.com/images/product_images/popup_images/840_5.jpg



Also here (on page 2), but no dimensions etc:
http://www.alphacool.com/download/Anleitung_HF14_SmartMotion_Universal_de_en_fr.pdf

[fizzisist]

http://www.alphacool.com/images/product_images/popup_images/840_5.jpg

Also here (on page 2), but no dimensions etc:
http://www.alphacool.com/download/Anleitung_HF14_SmartMotion_Universal_de_en_fr.pdf

Thanks, antirack, nice find. Still not easy to be 100% sure that it will fit properly, but it does look like it will. If you are thinking about getting a bunch of these, please order one or two to test it out first, and return them if they don't fit. Since those drawings don't give all of the relevent dimensions (only a few useful ones that can be used to make an educated guess about the rest), I wouldn't be confident investing a lot of money in these without first trying it out.

Another one worth looking at is probably this one: http://www.quietpcusa.com/Zalman-ZM-NWB1-Northbridge-Water-Block--P147.aspx

[antirack]

I have never worked with water cooling, so I may be not the best person to comment on this. But I have been looking into water blocks and water cooling for an FPGA cluster, but somehow gave up on the idea. Mostly because we don't have the parts available where I live (we have a quadrillion of real life shops selling air cooling stuff though). But also because it seems not to make sense to cool a chip with around 10 Watt heat dissipation with expensive water cooling gear. I may still build a prototype one day if I find these north bridge waterblocks by coincidence.
 
DeathAndTaxes, who seems to have a lot of knowledge in this area, also concluded the following in one of my other threads:

In theory yes.  In practice it likely is not worth it unless the power draw is very high and the output PER CHIP is very high.  The waterblock (part that attaches to the heat source) is a large cost of any system.  And 50 GH/s made up of 500 MH/s chips means 100 waterblocks.  50 GH/s made up of 200 MH/s chips means 250 waterblocks.  

One should have a LOT of experience water cooling before looking into custom setups.

He also said it would get interesting with a higher heat dissipation per chip (ie. 80 Watt) though.

[coretechs]

So does program.py not work to program the new rev of boards? I get this error on both

Code:

2012-04-16 21:45:07 | Device 3 opened (A4014PHJ)
Traceback (most recent call last):
  File "program.py", line 91, in <module>
    fpga.detect()
  File "/Users/christian/x6500-6/fpga.py", line 82, in detect
    with self.ft232r.lock: self.jtag.detect()
  File "/Users/christian/x6500-6/jtag.py", line 81, in detect
    raise NoDevicesDetected

Shows up in listDevices.py fine though?

Did you ever get this working?  I'm trying to run some rev 3 boards and I'm having the exact same problem, but in Linux.  The rev 2 boards work fine but the rev 3s report the NoDevicesDetected error when attempting to load the bitstream even though the hotplug manager worker is able to detect them.  I'm running the latest MPBM testing branch and have checked all the cables.  I'm manually disabling ftdi_sio prior to running MPBM but I'm going to try blacklisting it again next, however that will cause problems with other boards.

[TheSeven]

So does program.py not work to program the new rev of boards? I get this error on both

Code:

2012-04-16 21:45:07 | Device 3 opened (A4014PHJ)
Traceback (most recent call last):
  File "program.py", line 91, in <module>
    fpga.detect()
  File "/Users/christian/x6500-6/fpga.py", line 82, in detect
    with self.ft232r.lock: self.jtag.detect()
  File "/Users/christian/x6500-6/jtag.py", line 81, in detect
    raise NoDevicesDetected

Shows up in listDevices.py fine though?

Did you ever get this working?  I'm trying to run some rev 3 boards and I'm having the exact same problem, but in Linux.  The rev 2 boards work fine but the rev 3s report the NoDevicesDetected error when attempting to load the bitstream even though the hotplug manager worker is able to detect them.  I'm running the latest MPBM testing branch and have checked all the cables.  I'm manually disabling ftdi_sio prior to running MPBM but I'm going to try blacklisting it again next, however that will cause problems with other boards.

This actually looks like the board aren't powered properly. (They USB interface logic is powered by the USB bus, however the FPGAs aren't.)

[coretechs]

This actually looks like the board aren't powered properly. (They USB interface logic is powered by the USB bus, however the FPGAs aren't.)

Ah, I think you just helped me figure it out.  I was trying to use the modified molex connectors with the 12V/5V swapped that worked on the rev 2 boards but it looks like the rev 3 boards are only connected on the 12v so it was not probably not getting any power!

[TheSeven]

This actually looks like the board aren't powered properly. (They USB interface logic is powered by the USB bus, however the FPGAs aren't.)

Ah, I think you just helped me figure it out.  I was trying to use the modified molex connectors with the 12V/5V swapped that worked on the rev 2 boards but it looks like the rev 3 boards are only connected on the 12v so it was not probably not getting any power!

Yes, the Rev3 boards use the 12V rail of the molex connector.

[BR0KK]

http://www.caseking.de/shop/catalog/Watercooling/Chipset-Coolers/Watercool/Watercool-Heatkiller-NB-MICRO::4374.html

This watercooler could fit too and do the job (Its a block of Chopper and no fancy cooling technicis inside)

I have one here and it looks like it fits onto Ztex boards so it could fit onto x6500 aswell?. One odd thing is the tubing (6/8) and you need T-Adapters for normal Watercooling integration.

[freshzive]

This actually looks like the board aren't powered properly. (They USB interface logic is powered by the USB bus, however the FPGAs aren't.)

Ah, I think you just helped me figure it out.  I was trying to use the modified molex connectors with the 12V/5V swapped that worked on the rev 2 boards but it looks like the rev 3 boards are only connected on the 12v so it was not probably not getting any power!

Yes, the Rev3 boards use the 12V rail of the molex connector.

power is not my issue as they work properly using the old x6500 miner software. not really sure what is up, but in the process of building a dedicated ubuntu machine to control all of mine instead of my shitty mac.

[jav]

Hey there, quick update from the mineral oil cooling front. This is what my setup looks like now:



That board on the left is a Beagle Bone which I use to run MPBM (Beagle Bone + X6500 use 20 Watt in total). I wanted to get rid of the fan, but still keep the temperature down, so I added this water tank, which helps to dissipate the heat from the mineral oil container. This way I can even add a lid to the container and still have the heat dissipate quickly enough to have the FPGA stay at around 43 to 45 °C. The miner has been running without any problems so far. There have also not been any problems with oil running along cables or anything like that.

Now, in my chemical newbness I haven't accounted for the following effect though, that I see in the water tank:





I'm not exactly sure what that stuff is, that is floating around the water. Is some of the mineral oil evaporating and then condensing again in the colder water? But it seems a little bit much for that, as I thought that mineral oil evaporation would be pretty minimal, from what I read. Or is the mineral oil attacking some of the container material? But why would it be on the outside then, where the water is, instead of on the inside of the mineral oil container?

Does anyone have an idea what is going on here?

[fizzisist]

Hey there, quick update from the mineral oil cooling front. This is what my setup looks like now:



That board on the left is a Beagle Bone which I use to run MPBM (Beagle Bone + X6500 use 20 Watt in total). I wanted to get rid of the fan, but still keep the temperature down, so I added this water tank, which helps to dissipate the heat from the mineral oil container. This way I can even add a lid to the container and still have the heat dissipate quickly enough to have the FPGA stay at around 43 to 45 °C. The miner has been running without any problems so far. There have also not been any problems with oil running along cables or anything like that.

Very cool to see this! I'm wondering, do you have some type of metal container you could use instead of the plastic one? That would do much more to dissipate the heat from the oil to the environment. Also helpful would be a larger mass of oil.

[TheSeven]

How long has that water been in there?

I'm not an expert on that area, but I could think of three possibilities:

• Perfect temperature for some micro-organisms to populate the outside of the oil tank
• Some dirt that was in the water (or water tank) from the beginning relocating itself to the outside of the oil tank due to thermal effects
• The plastic tank "sweating" something wax-ish that is frozen by the cold water on the outside

The first option seems most likely to me, but the consistency of that ooze seems weird for that.

[simon66]

When will you have more of the x6500? Or will there be a new board 

[matthewh3]

Sorry for not reading the thread through but how well do these boards work on P2Pool compared to a normal PPS pool.  Thanks.

[Garr255]

Does anyone have an idea what is going on here?

Definitely use a metal container with a larger volume. But no, unless the mineral oil is reacting with that plastic in a very odd way...

Sorry for not reading the thread through but how well do these boards work on P2Pool compared to a normal PPS pool.  Thanks.

x6500s work the same assuming adequate available resources on the host machine.

[jav]

Very cool to see this! I'm wondering, do you have some type of metal container you could use instead of the plastic one? That would do much more to dissipate the heat from the oil to the environment. Also helpful would be a larger mass of oil.

I don't have a metal one handy at the moment but yes, I might try that in the future. It might also help with the ooze stuff.

Although:

Definitely use a metal container with a larger volume.

I don't see why I should "definitely" use a larger volume. The setup produces zero invalids as it is already, so I don't really see the need for temperature related improvements. Or why did you suggest this?

How long has that water been in there?

A couple of weeks now. But I did change the water once before, because I was also thinking that it might just be some oil that I spilled from when I was filling the inner container. I will change the water again a couple of times and see if the effect goes away over time, but at the moment it seems very consistent. Although I don't think it is anything biological. As you said, it doesn't really fit the consistency of the stuff.

[TheSeven]

When will you have more of the x6500? Or will there be a new board 

There should be a small amount of boards in stock next week, and this should start improving in about 2-3 weeks when the next production run finishes.

[matthewh3]

Sorry for not reading the thread through but how well do these boards work on P2Pool compared to a normal PPS pool.  Thanks.

x6500s work the same assuming adequate available resources on the host machine.

Would a RaspberryPi be enough computing resource's for an X6500 on P2Pool?

[shackleford]

Hey there, quick update from the mineral oil cooling front. This is what my setup looks like now:

That board on the left is a Beagle Bone which I use to run MPBM (Beagle Bone + X6500 use 20 Watt in total). I wanted to get rid of the fan, but still keep the temperature down, so I added this water tank, which helps to dissipate the heat from the mineral oil container. This way I can even add a lid to the container and still have the heat dissipate quickly enough to have the FPGA stay at around 43 to 45 °C. The miner has been running without any problems so far. There have also not been any problems with oil running along cables or anything like that.

Now, in my chemical newbness I haven't accounted for the following effect though, that I see in the water tank:


I'm not exactly sure what that stuff is, that is floating around the water. Is some of the mineral oil evaporating and then condensing again in the colder water? But it seems a little bit much for that, as I thought that mineral oil evaporation would be pretty minimal, from what I read. Or is the mineral oil attacking some of the container material? But why would it be on the outside then, where the water is, instead of on the inside of the mineral oil container?

Does anyone have an idea what is going on here?

I am not done with my oil setup but right now with 2 boards I am at 39/40c with active convection (the oil seems to thick for good natural convection). Just a bit ago I took two heatsinks and mashed them together with epoxy and submerged 1/4 of the combined heatsink (not half due to space). So far in the past few hours it lowered the temp 1c (with the same ambient temp). I have since in my inpatience put a fan on it and now it is down 1.5c. I am working with about 4 gallons so it takes some time for results. 2 days and no invalids with the 200mhz firmware.

Maybe you don't care about temps but perhaps with two good big passive heatsinks you could ditch that outer tub.

In my mind mineral oil is a good setup for expandibility. 1 fan internaly for forced convection and when you have your setup done you can just add cards to it with no need for more fans or waterblocks (heatsinks?). I don't know if a heatsink on the chips is needed but I am going to try it and see how it works. that right there would have saved me about 40$ per card.  

Update: No heatsink doesn't seem like an option. The temps were going up extremly fast so I pulled the plug after about 30 seconds @ 42c (highist I have ever been).