Bitcoin Forum

Bitcoin => Mining => Topic started by: kefir on May 15, 2011, 07:35:49 PM



Title: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: kefir on May 15, 2011, 07:35:49 PM
I'll post this howto here, because it could be useful for documentation:

1. It could help people configure other (similar) systems, and
2. It's not worth it. The Amazon servers cost more than the bitcoin earnings.

Anyway, here's what to do to get maximum juice out of one of these servers. They have 2 Tesla C2050 GPU cards, and 16x CPU cores. With these instructions, you'll run poclbm (http://bitcointalk.org/index.php?topic=1334.0) on each GPU, and jgarzik's cpuminer (https://github.com/jgarzik/cpuminer) on all processors, all mining for deepbit.net.

First, start up an Amazon EC2 GPU (cg1.4xlarge) instance with Amazon's own cluster Linux distribution (I used ami-321eed5b). When it's up and running, log on and run the following commands. Cut'n'paste is fine but remember to fix username/password/pooled mining server.
Code:
# set up path stuff
echo >> $HOME/.bash_profile
echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/tools/lib' >> $HOME/.bash_profile
echo 'export PATH=$PATH:$HOME/tools/bin' >> $HOME/.bash_profile
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/tools/lib
export PATH=$PATH:$HOME/tools/bin
cpus=$(cat /proc/cpuinfo | grep ^processor | wc -l)

# initial package config
sudo yum -y groupinstall "Development Tools"
sudo yum -y install git libcurl-devel python-devel screen rsync

# install yasm
git clone git://github.com/yasm/yasm.git
cd yasm
./autogen.sh
./configure --prefix=$HOME/tools
make -j $cpus
make install
cd -

# install and start cpuminer
git clone https://github.com/jgarzik/cpuminer.git
cd cpuminer
./autogen.sh
./configure
make -j $cpus
screen -d -m ./minerd --threads $cpus --algo sse2_64 --url http://deepbit.net:8332/ --userpass YOUR_EMAIL:YOUR_PASSWORD
cd -

# install numpy
git clone git://github.com/numpy/numpy.git numpy
cd numpy
git checkout remotes/origin/maintenance/1.6.x
sudo python setup.py install
cd -

# set up newer nvidia library
wget http://developer.download.nvidia.com/compute/cuda/3_2_prod/drivers/devdriver_3.2_linux_64_260.19.26.run
wget http://developer.download.nvidia.com/compute/cuda/3_2_prod/toolkit/cudatoolkit_3.2.16_linux_64_fedora13.run
sudo mv -v /lib/modules/$(uname -r)/kernel/drivers/video/nvidia.ko /root/

At this point, you need to reboot your server. The easiest way to do that is to run sudo reboot. Log back in to the server after reboot, and continue. You'll need to interact with the NVIDIA installers, so you can't cut'n'paste everything here. You'll also need to edit  the siteconf.py file in the command vi siteconf.py and make sure it says CL_ENABLE_DEVICE_FISSION = False
Code:
cpus=$(cat /proc/cpuinfo | grep ^processor | wc -l)

# restart cpuminer
cd cpuminer
screen -d -m ./minerd --threads $cpus --algo sse2_64 --url http://deepbit.net:8332/ --userpass YOUR_EMAIL:YOUR_PASSWORD
cd -
sudo bash devdriver_3.2_linux_64_260.19.26.run
sudo bash cudatoolkit_3.2.16_linux_64_fedora13.run

# install pyopencl
git clone http://git.tiker.net/trees/pyopencl.git
cd pyopencl
sudo easy_install Mako
git submodule init
git submodule update
python configure.py --cl-inc-dir=/usr/local/cuda/include --cl-lib-dir=/usr/local/cuda/lib64
vi siteconf.py # set CL_ENABLE_DEVICE_FISSION = False
sudo make install
cd -

# get poclbm and start for each device
git clone https://github.com/m0mchil/poclbm.git
cd poclbm/
screen -d -m python poclbm.py -o deepbit.net -p 8332 -u YOUR_EMAIL --pass=YOUR_PASSWORD -v -w 256 --device 0
screen -d -m python poclbm.py -o deepbit.net -p 8332 -u YOUR_EMAIL --pass=YOUR_PASSWORD -v -w 256 --device 1

If you're not familiar with the screen program, it runs your programs on a virtual console. You can access them with screen -r <id>, and you release them again with the key-combo Ctrl-a Ctrl-d.

Now you should be running 2 poclbm.py instances and one instance of cpuminer with 16 threads. In my test, each GPU calculated ~75000khash/s, while each CPU core did approx 1400khash/s, for a grand total of approximately 170mhash/s.

...and that's not worth it given an instance price of $2.10 per hour.


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: ErMurazor on May 17, 2011, 04:40:44 AM
Thank you working well now I have a use of my 20$ Amazon voucher. Just need stop it when the Amazon credit is spent :)

I can also confirm the flowing hash stats on Amazon EC2 GPUs.
70Mhash/s per GPU with a total 140Mhash/s


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: 1bitc0inplz on May 17, 2011, 04:44:50 AM
Thank you working well now I have a use of my 20$ Amazon voucher. Just need stop it when the Amazon credit is spent :)

I can also confirm the flowing hash stats on Amazon EC2 GPUs.
70Mhash/s per GPU with a total 140Mhash/s

70 MHash? wow....

How long did your $20 credit last you? I picked up my Radeon 5670 for $59 and it's giving me 85Mhash.... I guess this Amazon GPU service is am crummy as people say.


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: kefir on May 17, 2011, 06:56:34 AM
It's not so bad, but that depends on what you want to use it for. The Tesla GPU cards support double precision floating point operations, which can be useful for scientific stuff, but it doesn't help when mining for bitcoins. The Tesla doesn't have a much higher rating when it comes to raw GFLOPS.


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: Anonymous on May 17, 2011, 08:55:48 AM
Couldnt the bitcoin cloud itself compete with amazon ec2 ?



Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: Sukrim on May 17, 2011, 11:00:31 AM
Couldnt the bitcoin cloud itself compete with amazon ec2?
Currently not, as long as you only run bitcoin mining programs, you can only mine bitcoins (duh!).

If you write a client that allows for more arbitrary data, you'll re-invent something like BOINC probably.


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: kefir on May 17, 2011, 08:35:23 PM
Bitcoin and EC2 aren't comparable. With EC2 you can rent a virtual Windows or Linux servers for whatever use you like (with root access).

Perhaps it could be possible to change the type of work done by bitcoin miners to actually be something useful, rather than just solving a bunch of hashes. I don't know anything about how easy/hard that would be to implement, or what that could mean for security or anything, so I don't know if it's a good idea.


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: ErMurazor on May 17, 2011, 08:59:06 PM
How long did your $20 credit last you? I picked up my Radeon 5670 for $59 and it's giving me 85Mhash.... I guess this Amazon GPU service is am crummy as people say.

It lasted 9 hours so not a good deal for the 0.38BTC I got :) But is was a free 20$ voucher.


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: rezin777 on May 17, 2011, 09:08:14 PM
It will be interesting if the exchange rate goes high enough to make this profitable.   :D


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: kefir on May 17, 2011, 10:23:22 PM
Congratulations! You've just discovered a ceiling above which price of bitcoin will never rise (at current difficulty).  20/0.38 = 52.63 USD/BTC.

Never at current difficulty, so never this week? :)


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: kefir on May 17, 2011, 11:14:41 PM
OK, it's approximately valid for more than a week, I guess. ;) Sorry, I'm just a pedant sometimes. :D

But watching the FPGA thread something might happen to difficulty soon, and on a much longer perspective I guess there'll be an overnight doubling in 2013 when bitcoins-per-block falls from 50 to 25, and again in 2017, etc.

As an indicator of how much you definitely should NOT be paying for a bitcoin, it could be a good number. :) But it's actually very inflated, because while the Amazon EC2 Cluster GPU instance is (probably) good for scientific stuff, it has a bad price/performance ratio for generating bitcoins, because the hardware is very suboptimal for this task. If Amazon's GPU instances had two ATI Radeon 5970 mining at maybe 700Mhash/s each (instead of 70Mhash/s for each Tesla card), then ErMurazor should have had approx 3.8BTC instead of 0.38BTC for his $20. Unless I made a mistake already, this would make the calculation looks like:

20/3.8 = 5.26 USD/BTC

Of course, my assumption does assume that someone working at the same efficiency as Amazon EC2 actually starts setting up servers suitable for mining at similar prices. That's not happening for a while... And it probably means that people selling bitcoins on Mt Gox even as high as $8.50 aren't making a fortune, unless they've generated those bitcoins when it was a lot easier.


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: lagmo on May 20, 2011, 12:22:09 AM
Indeed the EC2 compute boxes aren't that powerful for Bitcoins.
But the figures quoted are a little lowballed, i got 70-80Mhashes/s on each Tesla GPU and about 1.6Mhashes/s per 'CPU' core ~170-190Mhashes/s total.

Though it was still a fun idea you had there Kefir  8)

I used Amazons 'spot pricing' to bring the cost down to around $0.70-0.80/hour, though that still doesn't seem to offset the cost enough, even when i had the miners autostart with the instance to run automatically during the off-peak hours  :'(

Esp. when my HD5850 easily does >370Mhashes/sec at a cost of about $0.10/hour.


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: saqwe on June 19, 2011, 12:36:11 AM
It will be interesting if the exchange rate goes high enough to make this profitable.   :D

yes sure will  ;D


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: NeonFlash on September 29, 2011, 04:33:28 PM
I was interested to see how well the Amazon EC2 GPU Clusters perform at GPGPU tasks such as Bitcoin Minings and I came across this thread. Very well defined procedure for how to go about using Amazon EC2 for Bitcoin Mining.

This is ofcourse for fun and to learn how to use Amazon EC2 GPU Clusters. Probably, if GPU Clusters with a huge stack of ATI HD 6990's were online as a Virtual Server (with root access), would be interesting to see how much it would cost per hour :D


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: Pipesnake on September 29, 2011, 08:36:05 PM
Watching


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: bulanula on October 02, 2011, 05:43:25 PM
What about mining tenebrix with the cpu in this baby ?

Maybe that makes it profitable ?


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: Adamlm on November 02, 2011, 07:59:02 PM
What about mining tenebrix with the cpu in this baby ?

Maybe that makes it profitable ?

Not really now, but you can mine CPU coins (Litecoin, Tenebrix) for free:

https://bitcointalk.org/index.php?topic=50708.0


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: btc_artist on November 08, 2011, 06:29:43 PM
watching


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: romanticon on December 19, 2011, 03:43:57 PM
Hi,

I am curious: How can I launch that miner to mine on the test network ?

Thank in advance.


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: btc_artist on December 19, 2011, 03:50:03 PM
Set testnet=1 in the bitcoin.conf file.


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: romanticon on December 19, 2011, 03:58:32 PM
Thanks


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: btc_artist on December 19, 2011, 03:59:05 PM
No problem.


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: romanticon on December 19, 2011, 04:06:03 PM
btw , is that address  http://deepbit.net:8332/  stays the same ?


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: btc_artist on December 19, 2011, 04:26:12 PM
btw , is that address  http://deepbit.net:8332/  stays the same ?
I imagine so.


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: romanticon on December 19, 2011, 04:28:17 PM
I hope you are right , cool I'll check it.


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: romanticon on December 20, 2011, 09:18:54 AM
I have started the miner and it is working fine with the production,
now I am looking how to switch it to the test network and can't
found the bitcoin.conf file. Will appreciate any help on that ?

Thanks in advance.


Title: Re: Howto: Mining on Amazon EC2 Cluster GPU instance
Post by: btc_artist on December 22, 2011, 08:55:22 AM
You need to create it in your .bitcoin directory.