My Build Info , 3 psu / 3 mobo / 16 cards combo for ETH 480MH/s

[xxcsu]

So there is some information about my newest build  

Updated with pictures and some additional info 23/sept/2016

My project is completed today all , 16 cards installed with custom ROM's, Take me forever to mod all those BIOSes . YEP , i have 12 XFX rx 480 , same model number , but i had to create 7 different rom for those cards to work stable. I was not able to use 1 modded bios for all 12 cards . Higher ASIC quality cards have more overclocking potential for sure , and much easier to mod those rom files for those cards:) Also i got 4 Nitro 8GB OC , those card have 3 different BIOSes . ( 13-2E3470U.X4L / 13-2E3470U.X51 / 113-2E3472U.053 ) Rom files are not interchangeable . You can flash them to another nitro , but then your hashrate will drop to lower 20's or under or just going up and down. Some of my card running over 30MH/s some is between 29.2-30MH/s STABLE !
My first priority was , creating a custom rom , with lower power draw and higher hash rate than stock .

My Build Total Hashrate is : 480 MH/s
Total Power draw at the wall : 2439Watts / 120volts
Still working on my setup for lower power draw

Parts :
 
1.00” X 1.00” Square aluminum tubing
Local hardware store

Fiber cement siding board as a motherboard tray ( for safety reason i would not recomending any kind of wood as a mobo tray . There is a high amps running all over if something going wrong , It can be a fire hazard !!!
Local hardware store , Lowe , Home Depot

3 Motherboards
with cpu / 8GB mem
 
2x ASRock H97 anniversary edition
http://www.asrock.com/mb/intel/h97%20anniversary/
1x GigaByte GA-X99M-Gaming 5 (rev. 1.0)
http://www.gigabyte.us/products/product-page.aspx?pid=5221&dl=1#ov

3 PSU
 
1x EVGA SUPERNOVA 1600
http://www.evga.com/articles/00840/
1x Corsair AXi Series, AX1500i
http://www.corsair.com/en/company/press-release/corsair-ax1500i-the-worlds-first-1500-watt-80-plus-titanium-psu-now-available
1x Corsair AX860i
http://www.corsair.com/en-us/ax860i-digital-atx-power-supply-860-watt-80-plus-platinum-certified-fully-modular-psu

16x PCI-E 16x to 1x Powered Riser Adapter Card w/ 50cm USB 3.0 Extension Cable & MOLEX to SATA Power Cable
easy to find on amazon , ebay

3x WeMo Insight Switch Smart Plug
with Energy Monitoring, Wi-Fi
http://www.belkin.com/us/p/P-F7C029/
Best way to monitor energy usage and remotley restart  , turn on , off your builds

And the most important
16 x RX 480 card .
13 arrived , 3 its on the way
I had to shop around a little bit , bc this card had a very limited availability when i placed my orders , so i got few from amazon , ebay , newegg All cards have 8GB samsung memory , 4 x shappire nitro , 12 xfx , 1 MSI ( for my PC )

3x 120GB SSD hard drive

After placed everything together , installed win 10 on the SSDs , before connected to internet , blocked all automatic updates , downloads , reportings , sharings ect ...used Win 10 built in group policy editor , to prevent the system crashing from some  stupid forced driver update by Microsoft . Then installed Intel inf utility , then the RX 480 driver with one RX 480 connected to the mobo ! if all 6 card was connected , AMD driver installation hang during install ...
After installation completed with 1 CARD connected ,set the virtual memory to 16GB min (, turned off the pc , connected another 5 card , WIN 10 booted up , recognized all the cards ...

Downloaded Claymore's Dual Ethereum+Decred AMD+NVIDIA GPU Miner v7.0 , created a bat file
https://bitcointalk.org/index.php?topic=1433925.0

TIMEOUT 90
setx GPU_FORCE_64BIT_PTR 0
setx GPU_MAX_HEAP_SIZE 100
setx GPU_USE_SYNC_OBJECTS 1
setx GPU_MAX_ALLOC_PERCENT 100
setx GPU_SINGLE_ALLOC_PERCENT 100
EthDcrMiner64.exe -epool eth-us-west1.nanopool.org:9999 -ewal 0xmyethwallletaddress/workername

Fired up the bat file , the miner started hashing with 148-149MH/s , at this point it was pulling around 1100 Watts at the wall from a 120 Volts outlet . It was running stable for a few hours , then it was time to cut back that energy usage

After a little playing with wattman settings i was able to manage the energy usage down to 725-750Watts so it saved me 350 watts / mobo or 700 watts from the entire setup with 2 mobo/12cards

I let the 2 mobo with the same settings running for a few hours , it was not any stability issue , so cranked up the memory a little bit one by one under wattman settings to 2050 (8200) hash rate went up to 151-152MH/s then it pulled around 770Watts at the wall ...., both mobos running with this setting more than 24 hours without any problem .

For underclocking i reduced the gpu speed , voltage , memory voltage , target temp , power limit .
Dont want to post my settings yet , still testing the stability and will increase the mem speeds.

I was able to run my setup for 3 days with 162MH/s each mobo  , with some high mem speed settings , but there was a lot of reboots , wattman reset the tune settings to factory default ... so i had to tune again one by one each 12 cards ...
was not able to sucessful underclock the miner with any other tune program , used ASUS GPU tweak , MSI afterburner..etc
wattman the only one worked for me the way i want

And there is some picrures :

Pictures added 2017

[1713409079]

1713409079

[1713409079]

1713409079

[1713409079]

1713409079

[1713409079]

1713409079

[1713409079]

1713409079

[1713409079]

1713409079

[ps_jb]

Super cool!

Great job

[rockyforever]

Nice. Love the aluminum build, just did an aluminum angle build today. I have always wanted to try the square ones but haven't been able to find the various connectors at lowes or home depot.

[BTCBusinessConsult]

Now just flash them all with a 29 MHs low power mod 

[ps_jb]

Now just flash them all with a 29 MHs low power mod 

And got performance drop in an hour to normal value (like you have). Why?

[xxcsu]

Now just flash them all with a 29 MHs low power mod 

Tried to flash with that modded bios , but was not able to complete the flashing process under win 10 . That mod is for reference card only  , maybe im wrong :-)

[BTCBusinessConsult]

Yeah based on the pictures they are not reference cards.

I would check out the ethereum forums - mining category. Pretty sure I saw some "non reference" mods that will help.

Nice set up man nice setup

[ioglnx]

Lovely build rig ..can you make me one :-D
Build me one too^^ the aluframe rig

[Ambros]

Now just flash them all with a 29 MHs low power mod 

Tried to flash with that modded bios , but was not able to complete the flashing process under win 10 . That mod is for reference card only  , maybe im wrong :-)

You are right, mods are only for reference cards.

[ioglnx]

How you manage that the cards are working without monitor plugged in?
I have a XFX like you but if I don't have monitor connected it gets disabled by the driver.

[xxcsu]

make sure under control panel / power settings
Put computer to sleep : never
Turn off display : never

Let me know if it's help .

[CoinBreader]

awesome build! it reminds me the glory days back in 2012 ! where i had a 62 cards ,
anyhow good job loved that wemo plugs!

[ioglnx]

make sure under control panel / power settings
Put computer to sleep : never
Turn off display : never

Let me know if it's help .

standard settings for me but it still requires that I have attached a DVI, HDMI or DP to the card otherwise the Radeon control is going nuts and also monitoring of the card is gone.
Like this :-(

[xxcsu]

are u using win 10 or win 7 ? What AMD  driver u have installed ?

[philipma1957]

nice but i do not see a shot with 16 cards and 3 mobos.

[xxcsu]

3 cards on the way it was pre ordered :-) you can see the motherboard , psu in the corner on this pictures ;-)
It's there ;-) but not in the open case yet 

[philipma1957]

One more question.

Do you think two threaded rods could be used to support the long bars or are they solid and won't sag?

[ps_jb]

Another question:

Could you please put the dimensions on the frame's rods? Did you cut them at Home Depot?

I will try to make similar - but shorter, for 6 cards only

[Amph]

to be honest i prefer one rig setup than 3-4 way, with the latter would be a pain in the ass to move it, too heavy and cumbersome

[ioglnx]

I use Windows 10 x64 and the latest nonWHQL Radeon Driver Hotfix and you?