How to start your own mining pool using bitcoind + eloipool.

[whitehathacker]

This was a struggle for me. So my struggle bears fruit for you!

Eloipool is a Python3 based pool server with stratum support. You can read more about it
on Luke-Jr's post here: https://bitcointalk.org/index.php?topic=61731.0

I would like to thank Luke-Jr, jgarzik, and the folks at the
following threads for helping me get this sucker rolling.

https://bitcointalk.org/index.php?topic=61731.0;all
https://bitcointalk.org/index.php?topic=158105.0;all


This is example is being installed on Ubuntu (Precise). You may have to adjust the downloading of packages to fit
your system.

1. Add the bitcoin repository to your sources and update (ubuntu)

Code:

add-apt-repository ppa:bitcoin/bitcoin

Code:

apt-get update

2. Install bitcoind

Code:

apt-get install bitcoind

3. Run bitcoind

Code:

bitcoind -daemon

You should receive an error that says something like this:

Code:

 lsError: To use the "-daemon" option, you must set a rpcpassword in the configuration file:
/home/user/.bitcoin/bitcoin.conf
It is recommended you use the following random password:
rpcuser=bitcoinrpc
rpcpassword=*password redacted*          <----This password is randomly generated by bitcoind
(you do not need to remember this password)
The username and password MUST NOT be the same.
If the file does not exist, create it with owner-readable-only file permissions.
It is also recommended to set alertnotify so you are notified of problems;
for example: alertnotify=echo %s | mail -s "Bitcoin Alert" admin@foo.com

4. Edit your bitcoin.conf file (at ~/.bitcoin/bitcoin.conf) to read

Code:

server=1
rpcuser=bitcoinrpc
rpcpassword=*the password in the error above*
rpcallowip=127.0.0.1
logtimestamps=1

5. Run bitcoind again

Code:

bitcoind -daemon

If you check the log file ~/.bitcoin/debug.log you should see it start the block download.
This syncs up with the network. This will take a long time so run this now and you can finish
set up. If you want a better visual representation and your system has a desktop installed,
install bitcoin-qt (apt-get intall bitcoin.qt). By leaving it up you can watch the progress
bar at the bottom.

6. Make eloipool directory

Pick a directory where you want to store eloipool.

I used "/bitcoin". This is in the root directory and different than "~/.bitcoin"

Code:

mkdir /bitcoin

7. Install python 3.2

Code:

apt-get install python3.2 python3.2-dev

8. Download python-bitcoinrpc

Code:

cd /bitcoin
git clone https://github.com/jgarzik/python-bitcoinrpc

Make sure you check out an older commit due to bugs with the current commit:

Code:

cd python-bitcoinrpc
git checkout -b 770881c8bd9b1f92427290270b37a28751cf9df0

Then go back to the main folder

Code:

cd ../../

9. Download python-base58

Code:

git clone git://gitorious.org/bitcoin/python-base58

10. Download and compile midstate

This is the only one you will have to compile yourself.

Code:

git clone http://gitorious.org/midstate/midstate
cd midstate

We have to edit midstate's Makefile to adhere to our system. Verify your python
flags are correct first by running these two commands:

Code:

python3.2-config --ldflags
python3.2-config --cflags

The output of these should match the last entry in the CFLAGS line and LDFLAGS line.

I edited mine from this:

Code:

CFLAGS = -march=native -Wall -funroll-all-loops -O3 -fstrict-aliasing -Wall -std=c99 -I/usr/include/python3.2
LDFLAGS = -Wl,-O1 -Wl,--as-needed -lpython3.2

to this:

Code:

CFLAGS = -march=native -Wall -funroll-all-loops -O3 -fstrict-aliasing -Wall -std=c99 -I/usr/include/python3.2mu
LDFLAGS = -Wl,-O1 -Wl,--as-needed -lpython3.2mu

Then make

Code:

sudo make

Then go back

Code:

cd ../

11. Download eloipool

Code:

sudo git clone git://gitorious.org/bitcoin/eloipool.git
cd eloipool

12. Edit config.py

Copy the file "config.py.example" to "config.py" and edit it.

Code:

cp config.py.example config.py

Change "TrackerAddr" to reflect the address of the local machine's bitcoin wallet.
MAKE SURE YOU CHANGE THIS TO YOUR ADDRESS!

Code:

TrackerAddr = '14tpWuy3pMf8A1zMxysg3rrEhD3bpveXGC'

If you are solo mining with a private pool, comment out the following line:

Code:

CoinbaserCmd = 'echo -e "1\\n$((%d / 100))\\n1579aXhdwvKZEMrAKoCZhzGuqMa8EonuXU"'

Update Template Sources primary and comment out the secondary entry:

Use the rpcuser and rpcpassword you listed in your ~/.bitcoin/bitcoin.conf file

Code:

TemplateSources = (
	{
		'name': 'primary',
		'uri': 'http://rpcuser:rpcpass@localhost:8332'
		'priority': 0,
		'weight': 1,
	}
)

Comment out or delete all entrties in TemplateChecks

Code:

TemplateChecks = (
)

Change UpstreamBitcoindNode to:

Code:

UpstreamBitcoindNode = ('127.0.0.1', 8333)

Change UpstreamNetworkId from testnet to main

Code:

UpstreamNetworkId = b'\xF9\xBE\xB4\xD9'

Disable (comment) transaction-related but workaround:

Code:

#POT = 2

Unless you want to use SQL for sharelogging, comment out all but the entry
with type 'logfile'. In order to use MySQL, you must install PyMySql.

Code:

ShareLogging = (
        {
                'type': 'logfile',
                'filename': 'share-logfile',
                'format': "{time} {Q(remoteHost)} {username} {YN(not(rejectReason))} {dash(YN(upstreamResult))} {dash(rejectReason)} {solution}\n",
        },
)

For your LogFile setting, I recommend changing the backup count depending on your preferences. It backs up every midnight,
and the number reflects how many backups to keep (the oldest being deleted).


13. Create scripts

Go to your "/bitcoin" directory and create a file called "run-bitcoind.sh" and enter the following. NOTE: your paths may
be different depending on where you chose your directory. For this example I used my /bitcoin one.

Code:

#!/bin/sh
/usr/bin/bitcoind -daemon -blocknotify=/bitcoin/newblock.sh

Now create the file "newblock.sh" and enter this VERBATIM:

Code:

#!/bin/sh

killall -USR1 eloipool.py

Now change directory into your "eloipool" folder (for me it was /bitcoin/eloipool) and create this
script called run-eloipool.sh. Again, your directories may be different, but leave "./eloipool" as is:

Code:

#!/bin/bash

PYTHONPATH=/bitcoin/python-bitcoinrpc:/bitcoin/python-base58:/bitcoin/midstate \
     nohup ./eloipool.py 2>&1 >/dev/null &

Make sure all the scripts are executable by issuing these commands:

Code:

chmod 0755 /bitcoin/run-bitcoind.sh
chmod 0755 /bitcoin/newblock.sh
chmod 0755 /bitcoin/eloipool/run-eloipool.sh

14. Run!

Bitcoind/bitcoin-qt MUST BE FULLY SYNC'd TO THE NETWORK.

Make sure you're firewalls and port forwarding allows port 3334 on TCP. For a better connection
to the bitcoin network, allow 8333 in as well (such as you notice only 8 connections max when synchronizing)

Make sure all related processes are not running.

I added the scripts and eloipool in case you tried a different setup first:

Code:

killall bitcoind
killall run-bitcoind.sh
killall eloipool.py

Run bitcoind from the script.

Code:

./run-bitcoind.sh

Wait a little bit for bitcoind to start up. Then run eloipool from the script.

Code:

./run-eloipool.sh

To verify that both processes are running, run the following commands. They should return
with a PID and the name of the process. Such as:

Code:

ps -e | grep bitcoin

should return something like:

Code:

26762 ?        03:27:37 bitcoind

Code:

ps -e | grep eloipool.py

should return something like:

Code:

26788 ?	       03:30:00 eloipool.py

15. Check connection

Now test your connecton by pointing your miner at your server with the settings:

user: YourBitcoinAddress
pass: x (this doesnt matter, it is ignored with the allow all setting in config.py)
address: stratum+tcp://IpAddressOrDomainName:3334

An example for bfgminer:

Code:

bfgminer.exe --userpass 134dV6U7gQ6wCFbfHUz2CMh6Dth72oGpgH:snoogins --url stratum+tcp://192.168.0.22:3334

Port 3334 is the port used for the Stratum Protocol.


Known Errors:

-----
If you have an error such as:

Code:

2013-05-12 23:08:19,014 merkleMaker     CRITICAL        Traceback (most recent call last):
  File "/usr/local/lib/python3.2/json/decoder.py", line 361, in raw_decode
    obj, end = self.scan_once(s, idx)
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/eloipool/merklemaker.py", line 692, in run
    self.merkleMaker_I()
  File "/home/eloipool/merklemaker.py", line 682, in merkleMaker_I
    self.merkleMaker_II()
  File "/home/eloipool/merklemaker.py", line 648, in merkleMaker_II
    return self._updateMerkleTree()
  File "/home/eloipool/merklemaker.py", line 548, in _updateMerkleTree
    self._updateMerkleTree_I()
  File "/home/eloipool/merklemaker.py", line 512, in _updateMerkleTree_I
    r = self._updateMerkleTree_fromTS(TS)
  File "/home/eloipool/merklemaker.py", line 477, in _updateMerkleTree_fromTS
    MP = self._CallGBT(TS)
  File "/home/eloipool/merklemaker.py", line 327, in _CallGBT
    MP = access.getblocktemplate(self.GBTReq)
  File "/usr/local/lib/python3.2/site-packages/bitcoinrpc/authproxy.py", line 102, in __call__
    response = self._get_response()
  File "/usr/local/lib/python3.2/site-packages/bitcoinrpc/authproxy.py", line 128, in _get_response
    parse_float=decimal.Decimal)
  File "/usr/local/lib/python3.2/json/__init__.py", line 320, in loads
    return cls(**kw).decode(s)
  File "/usr/local/lib/python3.2/json/decoder.py", line 345, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.2/json/decoder.py", line 363, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

A known fix is to change the line in python-bitcoinrpc/bitcoinrpc/authproxy.py
somewhere around line 72

Code:

self.__auth_header = "Basic %s" % base64.b64encode(authpair)

to read:

Code:

self.__auth_header = "Basic %s" % base64.b64encode(authpair).decode()

----------

An error reading:

Code:

TypeError: _stratum_mining_subscribe() takes exactly 1 positional argument (

Can be resolved by editing eloipool/stratumserver.py around lines 106-107 from

Code:

if not hasattr(e, 'StratumQuiet'):
		self.logger.debug(fexc)

to read:

Code:

if not hasattr(e, 'StratumQuiet'):
	if fexc.find('takes exactly 1 positional argument') == -1:
		self.logger.debug(fexc)

Using MySql to log from Eloipool

This one took me a little bit.

Change directory to this location:

Code:

cd /usr/local/lib/python3.2/dist-packages/PyMySQL3-0.5-py3.2.egg/pymysql/

Locate the lines that look like this: (they  should be together)

Code:

def unpack_int24(n):
    return struct.unpack('B',n[0])[0] + (struct.unpack('B', n[1])[0] << 8) +\
        (struct.unpack('B',n[2])[0] << 16)

def unpack_int32(n):
    return struct.unpack('B',n[0])[0] + (struct.unpack('B', n[1])[0] << 8) +\
        (struct.unpack('B',n[2])[0] << 16) + (struct.unpack('B', n[3])[0] << 24)

def unpack_int64(n):
    return struct.unpack('B',n[0])[0] + (struct.unpack('B', n[1])[0]<<8) +\
    (struct.unpack('B',n[2])[0] << 16) + (struct.unpack('B',n[3])[0]<<24)+\
    (struct.unpack('B',n[4])[0] << 32) + (struct.unpack('B',n[5])[0]<<40)+\
    (struct.unpack('B',n[6])[0] << 48) + (struct.unpack('B',n[7])[0]<<56)

Delete them all and in their place put this:

Code:

def unpack_int24(n):
    try:
        return struct.unpack('B',n[0])[0] + (struct.unpack('B', n[1])[0] << 8) +\
            (struct.unpack('B',n[2])[0] << 16)
    except TypeError:
        return n[0]+(n[1]<<8)+(n[2]<<16)

def unpack_int32(n):
    try:
        return struct.unpack('B',n[0])[0] + (struct.unpack('B', n[1])[0] << 8) +\
            (struct.unpack('B',n[2])[0] << 16) + (struct.unpack('B', n[3])[0] << 24)
    except TypeError:
        return n[0]+(n[1]<<8)+(n[2]<<16)+(n[3]<<24)

def unpack_int64(n):
    try:
        return struct.unpack('B',n[0])[0] + (struct.unpack('B', n[1])[0]<<8) +\
        (struct.unpack('B',n[2])[0] << 16) + (struct.unpack('B',n[3])[0]<<24)+\
        (struct.unpack('B',n[4])[0] << 32) + (struct.unpack('B',n[5])[0]<<40)+\
        (struct.unpack('B',n[6])[0] << 48) + (struct.unpack('B',n[7])[0]<<56)
    except TypeError:
        return n[0]+(n[1]<<8)+(n[2]<<16)+(n[3]<<24) \
              +(n[4]<<32)+(n[5]<<40)+(n[6]<<48)+(n[7]<<56)

Save and quit.

Make sure your database configurations are correct in your correct in your config.py and now you are logging to MySQL.

----------------------------------------------------------------
If you have any questions please post them below. If I don't have the answer hopefully somebody will.
If you see any errors in my guide that I dont see please let me know and I will fix it.



If you liked this guide, feel free to tip/donate me for my time.
bitcoin:1Nnb5S3z8kPMiiej9fUnvRqHorwb5C5mzq

[1715546758]

1715546758

[1715546758]

1715546758

[vmfx]

Awesome guide, will be trying this out now!!! What version of ubuntu are you running here ?

Sorry - just caught it - Precise 12

[softron]

Great work, + bookmark for later use.
Thanks

[Elerntta]

Thanks a lot for this guide! I am going to try to create my own pool and wish it would be succesful a least now I know main points of creating it

[7queue]

At one point I had this working...

With a fresh install of Ubuntu 12.04.4 LTS (GNU/Linux 3.8.0-39-generic x86_64)
Python 3.2
bitcoin-0.9.1 on testnet

Code:

git clone https://github.com/jgarzik/python-bitcoinrpc python-bitcoinrpc

Code:

git clone https://github.com/jgarzik/python-bitcoinrpc python-bitcoinrpc-solo

cd ./python-bitcoinrpc-solo

git checkout -b 770881c8bd9b1f92427290270b37a28751cf9df0

Code:

diff ./python-bitcoinrpc/bitcoinrpc/authproxy.py ./python-bitcoinrpc-solo/bitcoinrpc/authproxy.py

I see no difference ?? is that branch still available?


Get this in the log file

Code:

../eloipool/stratumserver.py", line 63, in found_terminator
    inbuf = b"".join(self.incoming).decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf2 in position 14: ordinal not in range(128)

With bfgminer-master it starts at diff 15 then goes to diff 0 with nothing but stale shares.


Ubuntu 14.04 LTS & Python 3.4 is a no go.

8 )

[BitCoinDream]

As it is python based, can it be implemented on Google app engine ?

[bgibso01]

This guide has helped me the most at getting my solo pool up and running.  My only problem is not being about to get the pool to use var diff.  All of my miners return diff 1 shares.

In my config.py I've got

ShareTarget = 0x000000000ff.........
DynamicTargetGoal = 6
DynamicTargetWindow = 120


Am I missing another setting somewhere?

[Luke-Jr]

This guide has helped me the most at getting my solo pool up and running.  My only problem is not being about to get the pool to use var diff.  All of my miners return diff 1 shares.

In my config.py I've got

ShareTarget = 0x000000000ff.........
DynamicTargetGoal = 6
DynamicTargetWindow = 120


Am I missing another setting somewhere?

I don't think you enabled it?

[bgibso01]

This guide has helped me the most at getting my solo pool up and running.  My only problem is not being about to get the pool to use var diff.  All of my miners return diff 1 shares.

In my config.py I've got

ShareTarget = 0x000000000ff.........
DynamicTargetGoal = 6
DynamicTargetWindow = 120


Am I missing another setting somewhere?

I don't think you enabled it?

Thanks Luke-Jr.  I have tried DynamicTargetting=1 and 2.  Neither seem to make a difference.  Is it a typo for "Targetting"?

DynamicTargetting=2

Never mind. Found the problem.  Was using master that didn't have the var diff files.

[bgibso01]

Has anyone got this working with Ubuntu 14.04?  I made the mistake of doing a fresh install with updates instead of 12.10.  I figure either 14 doesn't work with it or I've got something screwed up again.  At least I'm getting some practice.  It's been a long time since I used a terminal for very long.

[pjcltd]

Has anyone got this working with Ubuntu 14.04?  I made the mistake of doing a fresh install with updates instead of 12.10.  I figure either 14 doesn't work with it or I've got something screwed up again.  At least I'm getting some practice.  It's been a long time since I used a terminal for very long.

Hi i use nomp on 14.04
work fine no problems at all

[cheeha]

I get the error "jsonrpc_getwork WARNING Error importing 'midstate' module; work will not provide midstates".

I found someone else came across the same error on https://bitcointa.lk/threads/avalon-users-bitcoind-eloipool-configuration.93173/page-3 but they don't say they fixed it.

Any ideas?
Posted from Bitcointa.lk - #aX5f29tbI3UolmyL

[7queue]

Has anyone got this working with Ubuntu 14.04?  I made the mistake of doing a fresh install with updates instead of 12.10.  I figure either 14 doesn't work with it or I've got something screwed up again.  At least I'm getting some practice.  It's been a long time since I used a terminal for very long.

What version of Python are you using? I ran into this when I went to 14.04

https://bitcointalk.org/index.php?topic=61731.msg6735551#msg6735551

8 )

[NginUS]

My bitcoind & eloipool are running, but when I do this:

<code>C:/Progra~1/bfgminer-5.3.0-win64/bfgminer.exe --userpass 1DPLoA65tQmkA9RBqgYXuwbFvfbhF24Sjr:password --url stratum+tcp://173.246.106.141:3334,</code>

I get this error:
<code>No servers could be used! Exiting.</code>

What do I have to do to fix it?

[jmintuck02]

My bitcoind & eloipool are running, but when I do this:

<code>C:/Progra~1/bfgminer-5.3.0-win64/bfgminer.exe --userpass 1DPLoA65tQmkA9RBqgYXuwbFvfbhF24Sjr:password --url stratum+tcp://173.246.106.141:3334,</code>

I get this error:
<code>No servers could be used! Exiting.</code>

What do I have to do to fix it?

What worked for me was

C:/Progra~1/bfgminer-5.3.0-win64/bfgminer.exe --url stratum+tcp://173.246.106.141:3334 --userpass 1DPLoA65tQmkA9RBqgYXuwbFvfbhF24Sjr:password ?

That is how I configure my miner bat file. I put the Stratum FIRST, you had it the wrong way, methinks.

[whitehathacker]

So... funny thing...

I start looking into starting up a pool again. So I come across this thread thinking "oh good, step by step". Then I realize, oh $#!t ... I WROTE THIS.

I'm happy that this is helping people still, I'm working on getting mine back up and running on 16.04.1 LTS and have found a couple of things in the guide that could be improved on. Currently
re-syncing my blockchain.