Bitcoin Forum
December 07, 2016, 10:45:05 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 [39] 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 »
  Print  
Author Topic: X6500 Custom FPGA Miner  (Read 208082 times)
jav
Sr. Member
****
Offline Offline

Activity: 249


View Profile
March 28, 2012, 06:19:42 PM
 #761

I also wanted to point out a few things I ran into when setting up the software side of things, as it might be helpful to others as well.

This is on Debian Linux: When I first connected the board using USB, I used 'lsusb' to see all connected devices and did a 'lsusb -v -d 0403:6001' to show more details about the board I just connected. That gave me some infos, but also a "Operation not permitted". To be able to access the board using my user account, I added this udev rule: /etc/udev/rules.d/90-fpgamining-x6500.rules with this contents:

Code:
SUBSYSTEM=="usb", ATTR{idVendor}=="0403", ATTR{idProduct}="6001", MODE="0660", GROUP="fpgamining"

and created the group 'fpgamining' and added myself to it (you have to logout and login again for the change to show up):

Code:
# addgroup --system fpgamining
# adduser jan fpgamining

MPBM still remained silent though and after I added some debug output about what exceptions are thrown, I got "could not set config 1: Device or resource busy". This, as I was quickly informed in the IRC channel, is because of the module "ftdi_sio" which is loaded automatically and grabs access to the device. So I did a 'rmmod ftdi_sio' and added "blacklist ftdi_sio" to /etc/modprobe.d/blacklist.conf to prevent the module from loading again in the future.

After that it was smooth sailing with MPBM and this cool piece of hardware. =)

Hive, a beautiful wallet with an app platform for Mac OS X, Android and Mobile Web. Translators wanted! iOS and OS X devs see BitcoinKit. Tweets @hivewallet. Donations appreciated at 1HLRg9C1GsfEVH555hgcjzDeas14jen2Cn.
1481107505
Hero Member
*
Offline Offline

Posts: 1481107505

View Profile Personal Message (Offline)

Ignore
1481107505
Reply with quote  #2

1481107505
Report to moderator
1481107505
Hero Member
*
Offline Offline

Posts: 1481107505

View Profile Personal Message (Offline)

Ignore
1481107505
Reply with quote  #2

1481107505
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481107505
Hero Member
*
Offline Offline

Posts: 1481107505

View Profile Personal Message (Offline)

Ignore
1481107505
Reply with quote  #2

1481107505
Report to moderator
1481107505
Hero Member
*
Offline Offline

Posts: 1481107505

View Profile Personal Message (Offline)

Ignore
1481107505
Reply with quote  #2

1481107505
Report to moderator
TheSeven
Hero Member
*****
Offline Offline

Activity: 504


FPGA Mining LLC


View Profile WWW
March 28, 2012, 06:59:58 PM
 #762

Some mining rig photos: I went through with my plans for a mineral oil cooled setup. :-)

What you can't see here: The USB and power cable is rising up to the top, as I placed a nail there above the whole setup to run the cables along that. This is an attempt to prevent oil from trickling through the cables. So far the oil stays where it should, hopefully that will last.

All in all it seems to work fairly well. I would like your input though on what is an acceptable temperature to run the FPGA at. First I tried running without a fan and the oil would slowly start heating up and after about 30 minutes the thermometer showed 46 deg C (you can see in the picture that the thermometer is measuring in the middle of the heat sink). At that point I stopped and added a fan. It's not really helping all that much, but it is keeping the setup at about 43 deg C right now.

On the other hand I have yet to see a single invalid share. The system has been mining for over a day at 200 MHz (per FPGA) with 0 invalids. So would that indicate that I can risk a slightly higher temperature? I would really like to run this completely passive.

I was also thinking that this might be a good excuse to play around with 3d printing a little bit and maybe build something that would increase the surface area for the oil to cool down and maybe even have some kind of circular setup driven by convection. Although if I do that, which seems like a fun project, it would probably be just me guessing how to do that rather than being based on actual physics. =)

Looking at the thermal data of the FPGA, I'd say that a heatsink temperature of 50°C and a board temperature of 60°C should be perfectly fine. 10°C more might cause invalids but shouldn't really hurt it. So as long as the oil stays around 50°C it should be fine.

I'm not sure if the heatsink (and thermal adhesive) improves or worsens the situation here though... If you manage to establish oil convection across the FPGA's package, you might be better off without a heatsink at all, if the oil at the FPGA's surface stays at 50°C that would be ideal.

My tip jar: 13kwqR7B4WcSAJCYJH1eXQcxG5vVUwKAqY
shackleford
Full Member
***
Offline Offline

Activity: 191


View Profile
March 28, 2012, 08:51:54 PM
 #763

Some mining rig photos: I went through with my plans for a mineral oil cooled setup. :-)







What you can't see here: The USB and power cable is rising up to the top, as I placed a nail there above the whole setup to run the cables along that. This is an attempt to prevent oil from trickling through the cables. So far the oil stays where it should, hopefully that will last.

All in all it seems to work fairly well. I would like your input though on what is an acceptable temperature to run the FPGA at. First I tried running without a fan and the oil would slowly start heating up and after about 30 minutes the thermometer showed 46 deg C (you can see in the picture that the thermometer is measuring in the middle of the heat sink). At that point I stopped and added a fan. It's not really helping all that much, but it is keeping the setup at about 43 deg C right now.

On the other hand I have yet to see a single invalid share. The system has been mining for over a day at 200 MHz (per FPGA) with 0 invalids. So would that indicate that I can risk a slightly higher temperature? I would really like to run this completely passive.

I was also thinking that this might be a good excuse to play around with 3d printing a little bit and maybe build something that would increase the surface area for the oil to cool down and maybe even have some kind of circular setup driven by convection. Although if I do that, which seems like a fun project, it would probably be just me guessing how to do that rather than being based on actual physics. =)

You beat me to it, I just received my tech grade mineral oil yesterday and the rest of my parts should arrive by the end of the week. I think I have a pretty good design, building it completely out of acrylic glued like a fish tank. I plan to include the cooling into the lid. One lid will be two heat sinks connected through the lid. The other plan is the nuclear option with a peltier cooler sandwiched between them.  The enclosure itself is designed for convention so the cooling in the lid may be completely unneeded.. we will see.
BR0KK
Hero Member
*****
Offline Offline

Activity: 742



View Profile
March 28, 2012, 09:39:29 PM
 #764

Awww you know that you will end up with oil on you desks? And u lost your warranty btw. Tongue

It's called cappilar effect .... i did that once with my computer and the oil crawled up the mouse cable and after a short while my mouse began to bleed oil ...

For the Cooling you could try some parts from Watercooling projects. A pump that moves the oil through a radiator and back into the tank; or stick a small Fan in there that moves the oil thru the x6500 cooler.

O_Shovah
Sr. Member
****
Offline Offline

Activity: 410


Watercooling the world of mining


View Profile
March 28, 2012, 10:00:44 PM
 #765

Hello everybody,

As promised i release some images and data of my FPGA family at its daily work.

[...]

[From left to right: ARM board "Gumstix" (the first i developed on), Icarus(rev0),120mmfan ,x6500(rev2),120mmfan,Pandaboard ES(revB2),Bitforce single(1.0)]

[...]


Do you use the Pandaboard to control the FPGAs?
How does the Pandaboard cope with the workload? What do you think, how many FPGAs could you manage with it?

The panda board acts a a full size desktop with a full size ubuntu distribution.
It is running the mpbm miner and cotrols the boards via one or multiple USB hubs.
( and is using 8,5 W with panda board plus hub mouse etc)

It is also capaple of fully running a bitcoin-qt client( i recompiled one for ARM boards) and also act as a p2pool server at marginal cpu usage levels.
But p2pool is not recommended for the current mpbm version as this will cause high stale levels due to timing constraints.
While it is used as a board running the miner connected to a pool its mostly ideling. The cpu is really bored.

The board most on the left is my first ARM board  the "Gumstix"  it also ran the miner without major computation problems whe connected to a pool.(It consumes 4 W on the wall for a full ubuntu)
But it encounters performance limitations when you try to run a p2pool server on it. 


I am currently working to provide a full working image  for the panda board and the gumstix as well.
I also started creating a server to distribute .deb packages with fully ported ARM aplications.

Furthermore i want to publish similar things for the rasberry pi ( wich would be perfect as a host pc for mining in an pool)
as soon as i get my hands on one.

Time flys you know, so i cant give a certain date for publishing now.


I also took thermal imaging of the bitforce box today an redid the power measurement.
But more of that tomorrow.

TheSeven
Hero Member
*****
Offline Offline

Activity: 504


FPGA Mining LLC


View Profile WWW
March 28, 2012, 10:08:06 PM
 #766

It is also capaple of fully running a bitcoin-qt client( i recompiled one for ARM boards) and also act as a p2pool server at marginal cpu usage levels.
But p2pool is not recommended for the current mpbm version as this will cause high stale levels due to timing constraints.
While it is used as a board running the miner connected to a pool its mostly ideling. The cpu is really bored.

P2Pool should work fine with current master / testing tree. However it looks like p2pool is running into performance issues on the pandaboard as well, causing ~6% DOA for me. If I host it somewhere else it's running fine.

My tip jar: 13kwqR7B4WcSAJCYJH1eXQcxG5vVUwKAqY
O_Shovah
Sr. Member
****
Offline Offline

Activity: 410


Watercooling the world of mining


View Profile
March 28, 2012, 10:27:01 PM
 #767

It is also capaple of fully running a bitcoin-qt client( i recompiled one for ARM boards) and also act as a p2pool server at marginal cpu usage levels.
But p2pool is not recommended for the current mpbm version as this will cause high stale levels due to timing constraints.
While it is used as a board running the miner connected to a pool its mostly ideling. The cpu is really bored.

P2Pool should work fine with current master / testing tree. However it looks like p2pool is running into performance issues on the pandaboard as well, causing ~6% DOA for me. If I host it somewhere else it's running fine.

Might be somehow related to the I/O perfromance of your SD card. Mine is very fast and it just unns fine. ( well i also did some modifications to ubuntu to optimise).
As far as TheSeven told me mpbm has some latency issues wich are the price of the modular approach.

TheSeven
Hero Member
*****
Offline Offline

Activity: 504


FPGA Mining LLC


View Profile WWW
March 28, 2012, 11:09:54 PM
 #768

It is also capaple of fully running a bitcoin-qt client( i recompiled one for ARM boards) and also act as a p2pool server at marginal cpu usage levels.
But p2pool is not recommended for the current mpbm version as this will cause high stale levels due to timing constraints.
While it is used as a board running the miner connected to a pool its mostly ideling. The cpu is really bored.

P2Pool should work fine with current master / testing tree. However it looks like p2pool is running into performance issues on the pandaboard as well, causing ~6% DOA for me. If I host it somewhere else it's running fine.

Might be somehow related to the I/O perfromance of your SD card. Mine is very fast and it just unns fine. ( well i also did some modifications to ubuntu to optimise).
As far as TheSeven told me mpbm has some latency issues wich are the price of the modular approach.

Yes, it might well be related to SD card performance, that's what I was suspecting as well. (Getting a very high response latency variance close to found shares.)
MPBM surely isn't the miner with the lowest latencies, but since I've fixed a new block detection bug and decoupled the cancellation of jobs into another thread, it seems to work well enough. I'm getting around 1.7% DOA stales with a remote P2Pool instance, running MPBM on a pandaboard.

My tip jar: 13kwqR7B4WcSAJCYJH1eXQcxG5vVUwKAqY
twmz
Hero Member
*****
Offline Offline

Activity: 737



View Profile
March 28, 2012, 11:40:45 PM
 #769

Some mining rig photos: I went through with my plans for a mineral oil cooled setup. :-)

What you can't see here: The USB and power cable is rising up to the top, as I placed a nail there above the whole setup to run the cables along that. This is an attempt to prevent oil from trickling through the cables. So far the oil stays where it should, hopefully that will last.

All in all it seems to work fairly well. I would like your input though on what is an acceptable temperature to run the FPGA at. First I tried running without a fan and the oil would slowly start heating up and after about 30 minutes the thermometer showed 46 deg C (you can see in the picture that the thermometer is measuring in the middle of the heat sink). At that point I stopped and added a fan. It's not really helping all that much, but it is keeping the setup at about 43 deg C right now.

On the other hand I have yet to see a single invalid share. The system has been mining for over a day at 200 MHz (per FPGA) with 0 invalids. So would that indicate that I can risk a slightly higher temperature? I would really like to run this completely passive.

I was also thinking that this might be a good excuse to play around with 3d printing a little bit and maybe build something that would increase the surface area for the oil to cool down and maybe even have some kind of circular setup driven by convection. Although if I do that, which seems like a fun project, it would probably be just me guessing how to do that rather than being based on actual physics. =)

You beat me to it, I just received my tech grade mineral oil yesterday and the rest of my parts should arrive by the end of the week. I think I have a pretty good design, building it completely out of acrylic glued like a fish tank. I plan to include the cooling into the lid. One lid will be two heat sinks connected through the lid. The other plan is the nuclear option with a peltier cooler sandwiched between them.  The enclosure itself is designed for convention so the cooling in the lid may be completely unneeded.. we will see.

These guys did a lot of experimenting across several revisions to a fully submerged aquarium gaming PC.  Might be some tips in their articles about how they did cooling, etc:

http://www.pugetsystems.com/aquarium-computer.php

http://www.pugetsystems.com/mineral-oil-pc.php

http://www.pugetsystems.com/submerged.php

Was I helpful?  1TwmzX1wBxNF2qtAJRhdKmi2WyLZ5VHRs
WoT, GPG

Bitrated user: ewal.
shackleford
Full Member
***
Offline Offline

Activity: 191


View Profile
March 29, 2012, 12:02:06 AM
 #770



These guys did a lot of experimenting across several revisions to a fully submerged aquarium gaming PC.  Might be some tips in their articles about how they did cooling, etc:

http://www.pugetsystems.com/aquarium-computer.php

http://www.pugetsystems.com/mineral-oil-pc.php

http://www.pugetsystems.com/submerged.php

Yup, I got the oil from their partnered vendor. Mineral oil has the general opinion that it is not worth it in the PC arena and water cooling is better. I think mainly because people would like to use it to cool high end gaming rigs. For FPGA's we have significantly less wattage to worry about.
antirack
Hero Member
*****
Offline Offline

Activity: 491


Immersionist


View Profile
March 29, 2012, 12:36:14 AM
 #771

I am a bit surprised that there are no finished solutions for this problem. I can't imagine that there are no companies (scientific research?) that wouldn't need to submerge some equipment in mineral oil for cooling once in a while.

A (small) tank with some sort of rail/frame to attach equipment/PCBs and then established oil flow with external heat exchanger would be a neat product. But my five minutes of research didn't end up on any web site/manufacturer with a product like this. The closest thing I found was cryogenic equipment...

I can imagine it would only be interesting if it would work for a very high density, say minimum of 125 X6500 or Icarus boards (since this is the hashing power that the rig "box" is supposed to deliver, if it ever shows up).
kakobrekla
Hero Member
*****
Offline Offline

Activity: 714


Psi laju, karavani prolaze.


View Profile
March 29, 2012, 12:41:18 AM
 #772

I am a bit surprised that there are no finished solutions for this problem. I can't imagine that there are no companies (scientific research?) that wouldn't need to submerge some equipment in mineral oil for cooling once in a while.

A (small) tank with some sort of rail/frame to attach equipment/PCBs and then established oil flow with external heat exchanger would be a neat product. But my five minutes of research didn't end up on any web site/manufacturer with a product like this. The closest thing I found was cryogenic equipment...

I can imagine it would only be interesting if it would work for a very high density, say minimum of 125 X6500 or Icarus boards (since this is the hashing power that the rig "box" is supposed to deliver, if it ever shows up).


Inert fluids are better than mineral oil. Like 3M Novec...

Iceotope Liquid Cooled Cabinets (with Novec)
http://www.youtube.com/watch?v=d2TS_qQ3UNQ

3M™ Novec™ 1230 Fire Protection Fluid Electronics
http://www.youtube.com/watch?v=1E7Nr72rxB8

You can put a piece of paper in this and it will be dry when you pull it out.

antirack
Hero Member
*****
Offline Offline

Activity: 491


Immersionist


View Profile
March 29, 2012, 02:05:23 AM
 #773

Thanks for the links kakobrekla, I was more talking about a solution that a mere mortal like me or a small business can buy off the shelf (or through some specialized company). A case and some equipment just for cooling small stuff or small computers like we use them for mining... so far all I could find was home-made stuff.

Basically something like this, but built by a company so people/organizations in need could just place an order:
http://www.g4tv.com/articles/37474/build-a-submersion-cooling-case/

lame.duck
Legendary
*
Offline Offline

Activity: 1242


View Profile
March 29, 2012, 09:11:41 AM
 #774

I am a bit surprised that there are no finished solutions for this problem. I can't imagine that there are no companies (scientific research?) that wouldn't need to submerge some equipment in mineral oil for cooling once in a while.

A (small) tank with some sort of rail/frame to attach equipment/PCBs and then established oil flow with external heat exchanger would be a neat product. But my five minutes of research didn't end up on any web site/manufacturer with a product like this. The closest thing I found was cryogenic equipment...

I can imagine it would only be interesting if it would work for a very high density, say minimum of 125 X6500 or Icarus boards (since this is the hashing power that the rig "box" is supposed to deliver, if it ever shows up).

Why somebody should invent a suboptimal solution while there is already the better solution watercooling? Water can carry a lot of energy per volume and the temperature range  is perfect. Oil cooling would only make sense if the temperature would be  higher or there were high voltages involved.

And there are companies that produce  Water cooling equipment, and yes, you have to adapt it to your special needs, its like the water pipe system in a house that has to be assembled by a plumber as avery house is more or less different.
DeepBit
Donator
Hero Member
*
Offline Offline

Activity: 532


We have cookies


View Profile WWW
March 29, 2012, 09:41:16 AM
 #775

I can imagine it would only be interesting if it would work for a very high density, say minimum of 125 X6500 or Icarus boards (since this is the hashing power that the rig "box" is supposed to deliver, if it ever shows up).
Why somebody should invent a suboptimal solution while there is already the better solution watercooling? Water can carry a lot of energy per volume and the temperature range  is perfect. Oil cooling would only make sense if the temperature would be  higher or there were high voltages involved.
Water cooling is WAY too expensive.

Welcome to my bitcoin mining pool: https://deepbit.net ~ 3600 GH/s, Both payment schemes, instant payout, no invalid blocks !
Coming soon: ICBIT Trading platform
lame.duck
Legendary
*
Offline Offline

Activity: 1242


View Profile
March 29, 2012, 10:42:41 AM
 #776

I can imagine it would only be interesting if it would work for a very high density, say minimum of 125 X6500 or Icarus boards (since this is the hashing power that the rig "box" is supposed to deliver, if it ever shows up).
Why somebody should invent a suboptimal solution while there is already the better solution watercooling? Water can carry a lot of energy per volume and the temperature range  is perfect. Oil cooling would only make sense if the temperature would be  higher or there were high voltages involved.
Water cooling is WAY too expensive.

I doubt if cryogenic or oil would be less expensive, since you need customized heat exchanger and  pipes to.
I bet with proper heat sinks, dense packing and a clean air flow through the heatsinks air cooling should be  sufficient.

 
DeepBit
Donator
Hero Member
*
Offline Offline

Activity: 532


We have cookies


View Profile WWW
March 29, 2012, 11:16:07 AM
 #777

Why somebody should invent a suboptimal solution while there is already the better solution watercooling? Water can carry a lot of energy per volume and the temperature range  is perfect. Oil cooling would only make sense if the temperature would be  higher or there were high voltages involved.
Water cooling is WAY too expensive.
I doubt if cryogenic or oil would be less expensive, since you need customized heat exchanger and  pipes to.
I bet with proper heat sinks, dense packing and a clean air flow through the heatsinks air cooling should be  sufficient.
For water-cooling you'll need to machine special heat exchangers for EACH chip (or each two in some cases). May be some chipset waterblocks are compatible, but those aren't really cheap too.
For simple oil immersion you just need a bucket and the oil :)

Chipset waterblock is about $40 to $50 for low quantity orders, so it's at least $80 for each board. Notice that I'm not talking about pipes, pumps, water-to-air exchangers, fans and other accessories. FPGA miner costs about $420 to $570, so just two waterblocks will cost you around 16% of your miner and that's too much. Mining gear should be as cheap as possible.

2. Spartan package is not so good for heat transfer, so you can't cool it efficiently from just the top. PCB backside will definitely need some airflow.

Welcome to my bitcoin mining pool: https://deepbit.net ~ 3600 GH/s, Both payment schemes, instant payout, no invalid blocks !
Coming soon: ICBIT Trading platform
Icoin
Hero Member
*****
Offline Offline

Activity: 585



View Profile WWW
March 29, 2012, 11:58:12 AM
 #778

Hi everyone

I realized that there is a cooling problem for the fpga, thats why i developed a watercooling element specialy for the 6500 boards. The element is developed in a way that it should fit to all generations developed by now. Till today i could only use the measurements of the 1. gen but a 3. gen Board is on its way to me.


This element should be avalable i guess some weeks after the 3. gen arrives




This is easy integratable in a existing CPU/GPU watercooling solution

li_gangyi
Full Member
***
Offline Offline

Activity: 127



View Profile
March 29, 2012, 03:37:43 PM
 #779


This element should be avalable i guess some weeks after the 3. gen arrives


Hi Icoin, wow that looks fantastic. I suppose it'll come with spring loaded screws or something to mount onto the board. I just wished the Spartan-6 had a metal lid instead of it being encapsulated in epoxy like at the moment, that seriously hampers our cooling efforts. Ah well...

Currently active on Custom FPGA board https://bitcointalk.org/index.php?topic=37904.0
And on X6*** FPGA board https://bitcointalk.org/index.php?topic=40058.0
1PvzLC3FKEjBiWejWe4BXbnXTJe4pKCpU5
hashking
Sr. Member
****
Offline Offline

Activity: 308


View Profile
March 29, 2012, 06:32:48 PM
 #780

Does the x6500 work on gpumax?
Pages: « 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 [39] 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 »
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!