Building Cheap Miners : My "Secret"

[sundownz]

I have one of the DELL Servers rolling... getting ~2100 H/S out of the box first try.

Using ~675 watts or so.

What config are you using? I finally got setup today and am only getting 1500 h/s from the Dell server on Ubuntu 16 and XMR-STAK-CPU

This is my config :

"cpu_threads_conf" :
[
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 0 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 1 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 2 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 3 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 4 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 5 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 6 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 8 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 9 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 10 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 11 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 12 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 13 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 14 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 16 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 17 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 18 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 19 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 20 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 21 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 22 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 24 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 25 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 26 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 27 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 28 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 29 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 30 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 32 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 33 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 34 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 35 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 36 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 37 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 38 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 40 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 41 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 42 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 43 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 44 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 45 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 46 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 48 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 49 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 50 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 51 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 52 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 53 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 54 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 56 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 57 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 58 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 59 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 60 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 61 },
    { "low_power_mode" : false, "no_prefetch" : true, "affine_to_cpu" : 62 },

],

In your initial post #475 you stated "servers are loaded with 4x 12-Core AMD Opteron 6300 series CPUs each".
https://bitcointalk.org/index.php?topic=1955358.msg29265925#msg29265925

That would imply that a maximum of 48 cores. Since the above is showing 63 cores being used that would mean that the CPUs must be 16-Core Opterons.
http://www.cpu-world.com/CPUs/Bulldozer/TYPE-Opteron%206300%20series.html

I am confused in the above XMR-STAK settings. XMR-STAK tuning states that for optimal settings that each thread uses 2MB of cache.

The 16-Core Opterons have 16MB L2 Cache and 16MB L3 Cache.

So 8 threads fit into the 16MB L2 and/or 16MB L3 Cache.

I have a dual Opteron 6348 system that I have tuned for best power/performance using XMR-STAK.

http://www.cpu-world.com/CPUs/Bulldozer/AMD-Opteron%206348%20-%20OS6348WKTCGHK.html

These have 12-Core, 12MB L2, 16MB L3.

My optimal settings are for each processor:

"cpu_threads_conf" :
[
    { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 0 },
    { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 2 },
    { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 4 },
    { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 6 },
    { "low_power_mode" : 2, "no_prefetch" : true, "affine_to_cpu" : 8 },
    { "low_power_mode" : 2, "no_prefetch" : true, "affine_to_cpu" : 10 },

    { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 12 },
    { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 14 },
    { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 16 },
    { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 18 },
    { "low_power_mode" : 2, "no_prefetch" : true, "affine_to_cpu" : 20 },
    { "low_power_mode" : 2, "no_prefetch" : true, "affine_to_cpu" : 22 },
],

The Opteron 6348's are running at 3.1 GHz using this setting and the 1x threads produce 66 H/s and the 2x threads produce 76 H/s. If I try to change one more of the 1x threads to 2x the resultant hash rate drops overall. Note the above settings fit exactly into the 16MB L3 cache so that additional thread (1x to 2x) overflowed the L3 cache and caused the drop in hash rate.

I did try the settings you use (all 12 cores on each Opteron at 1x thread) and the resultant hash rate for each Opteron is slightly higher (456 vs 416) but the power consumption shoots much much higher causing much more heat which then causes the fans to spin faster and louder.

For the sake of longevity and power usage I am continuing to use my settings above.

The system I have is a Supermicro H8DGI-F Server that has 6 PCIe slots. I currently have 3 of them filled with Nvidia GTX 750 GPUs which results in an overall hash rate of 1547: CPUs (832) and GPUs (715).

Yes, you are correct, I messed up on my description earlier -- they are 6380 model CPUs with 16 cores.

I really haven't dived in heavily trying to tune them for efficiency as I have been concentrating on getting all my hardware running first, haha.

Thanks for sharing your info =)

[1714822384]

1714822384

[Seniorius]

is it worth??
 how about hashrate??

[MinersRus]

is it worth??
 how about hashrate??

Not sure who this is directed to?

But for me the Supermicro Server with dual Opteron 6348's and three Nvidia GTX 750's produces 1545 H/s and uses 426 watts. My electric cost is 7.4 cents per KWh.

The Monero profitability chart show a yearly profit of $465.

https://www.cryptocompare.com/mining/calculator/xmr?HashingPower=1545&HashingUnit=H%2Fs&PowerConsumption=426&CostPerkWh=0.074&MiningPoolFee=1

The Supermicro Server only cost me $252 and the three GTX 750's $120 for a total of $372. Payback is about 10 months. Realize though this hardware has value even if not for mining and that if I were to resell the server I would get well over $600 for it alone.

[PharmEcis]

: CPUs (832)

This would correlate to the 6344's I tested.  A pair does 800h/s.  Xmr-stak default config is 9T per CPU with 3T at low power for double mem and consumption of all L2/L3.  In dual configuration this is drawing 250w from the wall.  I tried setting 24T and 1x on memory usage but that was 40H/s slower and same power draw.

Sundownz don't like me.  I just wanna be fwiends and swap knowledge. 

[smutexar]

Hey. I'll tell you my secret. I sold all my mining equipment long ago, even though I didn't have to pay for electricity. Mining has long been a profitable business. Those who had at one time to build a huge farm, and they will continue to earn,Yu long since paid off the cost of the equipment.

[MinersRus]

: CPUs (832)

This would correlate to the 6344's I tested.  A pair does 800h/s.  Xmr-stak default config is 9T per CPU with 3T at low power for double mem and consumption of all L2/L3.  In dual configuration this is drawing 250w from the wall.  I tried setting 24T and 1x on memory usage but that was 40H/s slower and same power draw.

Sundownz don't like me.  I just wanna be fwiends and swap knowledge.  

Upon further review it appears that Sundownz is on the right track with 1x threads on every core. These Opterons behave differently than the Xeons that I have optimized before.

The 6300 series Opterons have a CC (Core Complex) that consists of 2 integer cores stuck together.

When I used:

1x thread on one of the two cores in the CC results in 65 H/s.
2x thread on one of the two cores in the CC results in 75 H/s.
1x threads on both of the two cores in the CC results in 82 H/s.

Power does go up with more cores running but not as much as I expected.

For some reason the hash rate does not drop much when the thread count exceeds the L2/L3 cache. When running all 12 cores the hash rate goes to 39/40 from 41.

I am now running this configuration on my dual 6348 Opteron system:

"cpu_threads_conf" :
   [
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 0  },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 2  },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 3  },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 4  },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 5  },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 6  },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 7  },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 8  },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 9  },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 10 },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 11 },

        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 12 },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 13 },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 14 },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 15 },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 16 },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 17 },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 18 },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 19 },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 20 },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 21 },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 22 },
        { "low_power_mode" : 1, "no_prefetch" : true, "affine_to_cpu" : 23 },
   ],

The reason I am not using CPU #0 core 1 is because I have that core affined to the Nvidia GPU's.

My overall hash rate for the CPU's only has jumped from 827 to 947. Power only increased 32 watts so it is profitable to use those extra 120 H/s.
https://www.cryptocompare.com/mining/calculator/xmr?HashingPower=120&HashingUnit=H%2Fs&PowerConsumption=32&CostPerkWh=0.074&MiningPoolFee=1

My system now produces 1640 H/s while using 486 watts.

Profit per year now has increased to $510.
https://www.cryptocompare.com/mining/calculator/xmr?HashingPower=1640&HashingUnit=H%2Fs&PowerConsumption=486&CostPerkWh=0.074&MiningPoolFee=1

[phobik]

What's the advantage of building a $500 system doing 1600hs and then building 60x of them? That's still $30k. When you can get the same hash with just 2x 4xx/5xx cards or 1x vega and still have more slots to lower the system power overhead? Just spend your $30k on modern, faster, more efficient hardware...

[PharmEcis]

What's the advantage of building a $500 system doing 1600hs and then building 60x of them? That's still $30k. When you can get the same hash with just 2x 4xx/5xx cards or 1x vega and still have more slots to lower the system power overhead? Just spend your $30k on modern, faster, more efficient hardware...

Please show me where you can purchase these cards in bulk at a price that is not astronomical as I would love to fill my empty PCIe slots with them.  Some of us are on the path of putting whatever iron on the fire that is making money, even if it isn't exactly the most efficient or latest wiz bang contraption.  Even better, when / if () GPU prices get back to normal, we have tons of open slots ready to accept them with minimal config and downtime.  Until then, they at the minimum, more than cover the expense for electricity of the combined GPU/CPU mining activity and earn back their own cost, slowly but surely, even in the depressed market we are in compared to 3 months ago.

-=Slow Growth=-

Someone might see what I did there...

Until then, I snipe cards when and where I can if they are relatively close to what msrp was.

[ronnylov]

What's the advantage of building a $500 system doing 1600hs and then building 60x of them? That's still $30k. When you can get the same hash with just 2x 4xx/5xx cards or 1x vega and still have more slots to lower the system power overhead? Just spend your $30k on modern, faster, more efficient hardware...

Please show me where you can purchase these cards in bulk at a price that is not astronomical as I would love to fill my empty PCIe slots with them.  Some of us are on the path of putting whatever iron on the fire that is making money, even if it isn't exactly the most efficient or latest wiz bang contraption.  Even better, when / if () GPU prices get back to normal, we have tons of open slots ready to accept them with minimal config and downtime.  Until then, they at the minimum, more than cover the expense for electricity of the combined GPU/CPU mining activity and earn back their own cost, slowly but surely, even in the depressed market we are in compared to 3 months ago.

-=Slow Growth=-

Someone might see what I did there...

Until then, I snipe cards when and where I can if they are relatively close to what msrp was.

Aha, you are mining IntenceCoin :-)

Regarding filling empty PCIe slots I think these server power supplies does not have PCIe power connectors? So we are limited to cards with power from PCIe slot ony or using external power supplies? I have a Supermicro quad socket Opteron 6276 machine but CPU mining only on that one. I figured out that by using external power supply and powered risers it may be possible using GPUs too but right now I have a separate GPU mining rig.

[MinersRus]

What's the advantage of building a $500 system doing 1600hs and then building 60x of them? That's still $30k. When you can get the same hash with just 2x 4xx/5xx cards or 1x vega and still have more slots to lower the system power overhead? Just spend your $30k on modern, faster, more efficient hardware...

Please show me where you can purchase these cards in bulk at a price that is not astronomical as I would love to fill my empty PCIe slots with them.  Some of us are on the path of putting whatever iron on the fire that is making money, even if it isn't exactly the most efficient or latest wiz bang contraption.  Even better, when / if () GPU prices get back to normal, we have tons of open slots ready to accept them with minimal config and downtime.  Until then, they at the minimum, more than cover the expense for electricity of the combined GPU/CPU mining activity and earn back their own cost, slowly but surely, even in the depressed market we are in compared to 3 months ago.

-=Slow Growth=-

Someone might see what I did there...

Until then, I snipe cards when and where I can if they are relatively close to what msrp was.

Aha, you are mining IntenceCoin :-)

Regarding filling empty PCIe slots I think these server power supplies does not have PCIe power connectors? So we are limited to cards with power from PCIe slot ony or using external power supplies? I have a Supermicro quad socket Opteron 6276 machine but CPU mining only on that one. I figured out that by using external power supply and powered risers it may be possible using GPUs too but right now I have a separate GPU mining rig.

Some servers do have PCIe slots and VGA power cables. My Supermicro server is a dual 6348 Opteron that has dual 1200 watt power supplies and has 6 PCIe slots and four 6-pin VGA power cables.

Expansion Slots
• Three (3) PCI-Express x16 Gen. 2
• One (1) PCI-Express x8 Gen. 2
• Two (2) PCI-Express x4 (in x8 slot) Gen. 2

I currently have three Nvidia GTX 750 in the three PCI-Express x16 Gen. 2 slots.

[PharmEcis]

Regarding filling empty PCIe slots I think these server power supplies does not have PCIe power connectors? So we are limited to cards with power from PCIe slot ony or using external power supplies? I have a Supermicro quad socket Opteron 6276 machine but CPU mining only on that one. I figured out that by using external power supply and powered risers it may be possible using GPUs too but right now I have a separate GPU mining rig.

Breakout boards...  All my HP server iron have GPUs running along with the CPUs.  Boxes like the r815 that don't have the onboard connectors to power them but could easily run GPUs with the addition of external PSU and risers.

[ronnylov]

Regarding filling empty PCIe slots I think these server power supplies does not have PCIe power connectors? So we are limited to cards with power from PCIe slot ony or using external power supplies? I have a Supermicro quad socket Opteron 6276 machine but CPU mining only on that one. I figured out that by using external power supply and powered risers it may be possible using GPUs too but right now I have a separate GPU mining rig.

Breakout boards...  All my HP server iron have GPUs running along with the CPUs.  Boxes like the r815 that don't have the onboard connectors to power them but could easily run GPUs with the addition of external PSU and risers.

Its kind of annoying having dual 1400W Gold class PSU in the server but no PCIe power connectors from the PSUs.
So there is breakout-boards that can solve this so I can use existing PSUs?

My server is Supermicro 2042-6RF.

[itmanvn]

any update man?  

Been going slower than I had hoped... been very busy with my main business.

But the 5th rack of GPU rigs is 100% done and the 2nd rack of Dell servers is 50% done.

I am starting on the 6th rack of GPU rigs tomorrow.

Im building rigs using HP Z400, in my country it's about 230$/pcs, xeon w3565, 12G ram, 160G HDD 10.000rpm. I using HP server PSU, so cheap on ebay: https://www.ebay.com/i/302651472969?rt=nc its about 70$ for me, riser 5$/pcs. 4x1070 for mining ETH at 130Mh/s.

Just want to say thank you  

https://preview.ibb.co/d0Dusn/IMG_6426.jpg
https://preview.ibb.co/k2HyJS/IMG_6416.jpg
https://preview.ibb.co/nfHb57/IMG_6414.jpg
https://preview.ibb.co/mt8usn/IMG_6421.jpg

[PharmEcis]

Its kind of annoying having dual 1400W Gold class PSU in the server but no PCIe power connectors from the PSUs.
So there is breakout-boards that can solve this so I can use existing PSUs?

My server is Supermicro 2042-6RF.

From the quick look I did, I don't think you have PSU's that can use breakout boards.  The Liteon's in the Dells and the HP ones are readily available on Ebay.  From what I saw of the connectors, technically I guess you could MacGyver something by soldering leads to the pins of the PSU but you would still have to buy cabling.

Picking up an HP1200w PSU and breakout board with cables is normally ~100 or so on ebay.

[MinersRus]

A quick question. How are posters like sundownz able to get picture links to show the actual picture in the post and not just a link to the picture?


I have used the "Insert Image" and put the link between the [i m g]link[/i m g]

like this:



but all I get when this is posted is the link to the picture and not the picture directly.

[greyday]

That's a link to a url that shows the image. You need to use an image link (ends in .jpg, .gif, .png, etc).

[MinersRus]

That's a link to a url that shows the image. You need to use an image link (ends in .jpg, .gif, .png, etc).

sundownz posts have pictures showing and they do not have any extension.

See here: https://bitcointalk.org/index.php?topic=1955358.msg31588057#msg31588057

The link to one of his pictures is: https://ibb.co/jsWe57

You can put that link in a browser and see the picture and you can do the same with my picture link.

So the answer is not the extension but must be something else.

[spinx]

That's a link to a url that shows the image. You need to use an image link (ends in .jpg, .gif, .png, etc).

sundownz posts have pictures showing and they do not have any extension.

See here: https://bitcointalk.org/index.php?topic=1955358.msg31588057#msg31588057

The link to one of his pictures is: https://ibb.co/jsWe57

You can put that link in a browser and see the picture and you can do the same with my picture link.

So the answer is not the extension but must be something else.

He used this code, note the [img] tags and the url pointing directly to the .jpg -file. Remove the asterix (*) from the code below to use.

Code:

[*url=https://ibb.co/jsWe57][*img]https://preview.ibb.co/bYhmk7/0303181208_resized.jpg[/img*][/url*]"

[greyday]

That's a link to a url that shows the image. You need to use an image link (ends in .jpg, .gif, .png, etc).

sundownz posts have pictures showing and they do not have any extension.

See here: https://bitcointalk.org/index.php?topic=1955358.msg31588057#msg31588057

The link to one of his pictures is: https://ibb.co/jsWe57

You can put that link in a browser and see the picture and you can do the same with my picture link.

So the answer is not the extension but must be something else.

I don't know about sundownz, but that is how you use the img tags.

Example:

Code:

 

[img]http://www.greydayrecords.com/bands/photos/heligoats.jpg[/img]

Results in this:

[Michael20]

On the picture only 2 1070?! You also mining with cpu, you know how much wattage with cpu only mining? Or the complete system?

any update man?  

Been going slower than I had hoped... been very busy with my main business.

But the 5th rack of GPU rigs is 100% done and the 2nd rack of Dell servers is 50% done.

I am starting on the 6th rack of GPU rigs tomorrow.

Im building rigs using HP Z400, in my country it's about 230$/pcs, xeon w3565, 12G ram, 160G HDD 10.000rpm. I using HP server PSU, so cheap on ebay: https://www.ebay.com/i/302651472969?rt=nc its about 70$ for me, riser 5$/pcs. 4x1070 for mining ETH at 130Mh/s.

Just want to say thank you