[ANN] Eloipool - FAST Python3 pool server software - GBT/stratum/dyntarget/proxy

[Inaba]

What's this got to do with Eloipool?

Mining is already centralized, there's only about 12 miners on the network at the moment.  Cloud hashing farms will help to decentralize the currently ultra-centralized mining.  If you are mining into a pool, you aren't mining.  You are contributing work to the pool, who is mining; The pool is the only miner of the thousands of people using the pool.  If you aren't creating blocks, you aren't mining, either.  So "contributing" your CPU hashing power to "help support the network" isn't doing anything for the network, you're just consuming power. 

All of this should be split into a different thread, though.  Maybe a mod can split it off, I'll be happy to continue the discussion in a thread that's not the Eloipool thread.

[vmfx]

Got eloipool up and running perfectly, wicked cool. mad respect to Luke

just checking now to see if there are any web based front ends for it ? i can only find a old thread about wizstats...

[Polyatomic]

Running on Ubuntu Saucy , cloned from gittorious.

Code:

2014-04-27 19:03:26,247	JSONRPCServer	INFO	Nobody to longpoll
2014-04-27 19:03:26,893	JSONRPCServer	INFO	Nobody to longpoll
2014-04-27 19:04:53,349	newBlockNotification	INFO	Received new block notification
2014-04-27 19:04:53,488	merkleMaker	INFO	New block: 000000000000000064a6ac54154247706dca44c4e697e164d87fdc47fe1bc570 (height: 297943; bits: 19009d8c)
2014-04-27 19:04:53,489	JSONRPCServer	INFO	Nobody to longpoll
2014-04-27 19:04:54,111	JSONRPCServer	INFO	Nobody to longpoll
2014-04-27 19:06:47,871	newBlockNotification	INFO	Received new block notification
2014-04-27 19:06:48,038	merkleMaker	INFO	New block: 00000000000000002bc6d22a6b1ab8a40f7067d3fe78f01c2741ec42ca6074c4 (height: 297944; bits: 19009d8c)
2014-04-27 19:06:48,039	JSONRPCServer	INFO	Nobody to longpoll
2014-04-27 19:06:48,611	JSONRPCServer	INFO	Nobody to longpoll

AsicA , same host as Eloipool, is getting alot of these.. #using bfgminer fixed this for now until I find out why this happens with cgminer.

Code:

 Accepted untracked stratum share from pool 0
 [2014-04-27 19:23:30] Accepted untracked stratum share from pool 0
 [2014-04-27 19:24:31] Accepted untracked stratum share from pool 0
 [2014-04-27 19:24:47] Accepted untracked stratum share from pool 0
]

AsicB is on another host pointed at Eloipool and is running good.

Code:

 Accepted 12482944 Diff 3.58K/256
 [2014-04-27 09:55:18] Accepted 569fc31b Diff 757/256
 [2014-04-27 09:55:58] Accepted ed8fa478 Diff 276/256
 [2014-04-27 09:56:01] Accepted dc4a95b5 Diff 297/256
 [2014-04-27 09:56:06] Accepted a09251c8 Diff 408/256 
 [2014-04-27 09:56:06] Accepted 1b0e730e Diff 2.42K/256
 [2014-04-27 09:56:07] Accepted 4535354a Diff 947/256
 [2014-04-27 09:56:08] Accepted b5311db9 Diff 362/256
 [2014-04-27 09:56:36] Accepted d8740099 Diff 303/256 
 [2014-04-27 09:56:37] Accepted 437c30d3 Diff 971/256 
 [2014-04-27 09:56:53] Stratum from pool 0 detected new block
 [2014-04-27 09:57:06] Accepted ea4e54fe Diff 280/256 
 [2014-04-27 09:57:44] Accepted 5e8c0424 Diff 693/256 
 [2014-04-27 09:57:46] Accepted 1d9e5e55 Diff 2.21K/256 

Exellent software man. I've set this up somewhere out of sight out of mind, maybe i'll get lucky.

[tom19990101]

Hi Luke, everyone,

Eloipool have what kind of attack, how to prevent?


The following information properly?
2014-05-10 19:48:50,568 StratumServer   ERROR   Traceback (most recent call last):
  File "/home/tools/eloipool/networkserver.py", line 449, in serve_forever
    o.handle_read()
  File "/home/tools/eloipool/networkserver.py", line 62, in handle_read
    self.handle_readbuf()
  File "/home/tools/eloipool/networkserver.py", line 114, in handle_readbuf
    self.found_terminator()
  File "/home/tools/eloipool/stratumserver.py", line 60, in found_terminator
    inbuf = b"".join(self.incoming).decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0x87 in position 8: ordinal not in range(128)

2014-05-10 19:48:50,569 StratumHandler  DEBUG   Traceback (most recent call last):
  File "/home/tools/eloipool/networkserver.py", line 449, in serve_forever
    o.handle_read()
  File "/home/tools/eloipool/networkserver.py", line 62, in handle_read
    self.handle_readbuf()
  File "/home/tools/eloipool/networkserver.py", line 114, in handle_readbuf
    self.found_terminator()
  File "/home/tools/eloipool/stratumserver.py", line 60, in found_terminator
    inbuf = b"".join(self.incoming).decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0x87 in position 8: ordinal not in range(128)




thks.

[Luke-Jr]

Hi Luke, everyone,

Eloipool have what kind of attack, how to prevent?


The following information properly?
2014-05-10 19:48:50,568 StratumServer   ERROR   Traceback (most recent call last):
  File "/home/tools/eloipool/networkserver.py", line 449, in serve_forever
    o.handle_read()
  File "/home/tools/eloipool/networkserver.py", line 62, in handle_read
    self.handle_readbuf()
  File "/home/tools/eloipool/networkserver.py", line 114, in handle_readbuf
    self.found_terminator()
  File "/home/tools/eloipool/stratumserver.py", line 60, in found_terminator
    inbuf = b"".join(self.incoming).decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0x87 in position 8: ordinal not in range(128)

2014-05-10 19:48:50,569 StratumHandler  DEBUG   Traceback (most recent call last):
  File "/home/tools/eloipool/networkserver.py", line 449, in serve_forever
    o.handle_read()
  File "/home/tools/eloipool/networkserver.py", line 62, in handle_read
    self.handle_readbuf()
  File "/home/tools/eloipool/networkserver.py", line 114, in handle_readbuf
    self.found_terminator()
  File "/home/tools/eloipool/stratumserver.py", line 60, in found_terminator
    inbuf = b"".join(self.incoming).decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0x87 in position 8: ordinal not in range(128)

Shouldn't affect anything.

[7queue]

Noticed this for Python 3.4,

    class http.client.HTTPConnection(host, port=None, [timeout, ]source_address=None)

Changed in version 3.4: The strict parameter was removed.

    https://docs.python.org/3.4/library/http.client.html


python-bitcoinrpc/bitcoinrpc/authproxy.py needs adjusting.

8 )

[tom19990101]

Hi Luke,

Under the different hashrate, how to optimize config.Py.

For example: 40T, 200T, 500T, 1000T, how should adjust.





thks.

[DISSIDIA]

Hi,Luke. are there any api docs about Eloipool? 

[Luke-Jr]

Hi,Luke. are there any api docs about Eloipool? 

There's no API, so no.

[bgibso01]

Hi,Luke. are there any api docs about Eloipool?  

There's no API, so no.

Luke-Jr,

By chance would you know off the top of your head what could be causing my target issue posted a couple above?

Update: Nevermind, found out the var diff part was not in the master that I pulled.  Works like a charm now.  Big thanks to Polyatomic for pointing me in the right direction!

[guruvan]

Luke-Jr, we've been running eloiopol to connect a pool to eligius, quite successfully (Thanks!) and have a few questions:

1) To connect to an upstream pool (ie eligius) the loop branch is required
    - If I understand everything, I need coinbase/append and submit/coinbase to do this, and those don't appear to be in the main branch, correct?
2) I also need the latest version of your bitcoind, eligius branch? (it's what I am running now)
3) when using this branch, so I need or want "BlockSubsissions" section, or just "UpstreamURI"
    - and if I need/want it, mining on upstream pool like eliguis, whose servers should be here?

- FWIW, we're using your suggested commit for bitcoinrpc, and mostly everything else you suggest, except gentoo (I feel like I might have shot myself in the foot here!) eloipool is running in a fairly stock ubuntu14.04, with python3.2 and setup in virtualenv.

The issues I've seen so far seem common, and mostly above you suggest ignoring most, but

- when stratum workers become disconnected, they don't want to reconnect without the human restarting miner software
- they seem to disconnect when we see a socket timeout connecting to (either eligius or our own bitcoind, this is not clear)

Finally, is there any guidance on the amount of RAM required to run eloipool as a downstream pool? Is this more based on the number of workers connected, the amount of hashrate, or?? I definitely ran a 4GB box out of RAM with just 2THs on it. Bitcoind is running on it's own machine, and we're just logging shares to text for now.

Well. Maybe one more. I'm sure you've seen nomp's mining keys? What do you think of this idea and code? And, if you think that's a good idea, would that be something you'd have an interest in supporting in eloipool? (maybe a module that figures a pubkey from a miningkey and drops it into config.py?)

I know you're not the biggest altcoin fan, but I can't see any reason you shouldn't earn something from pools using your software to mine them. (forrestv has something similar for --give-author in p2pool I've not seen how this works though)

Thanks for your time! (we know where to put some BTC )

[Luke-Jr]

Luke-Jr, we've been running eloiopol to connect a pool to eligius, quite successfully (Thanks!) and have a few questions:

1) To connect to an upstream pool (ie eligius) the loop branch is required

The loop branch was never finished and does not work completely.
Use it at your own risk...

- when stratum workers become disconnected, they don't want to reconnect without the human restarting miner software

Be sure miners are using the latest version of BFGMiner.
Unmaintained versions and forks thereof, my have bugs - those are however unrelated to Eloipool.

Well. Maybe one more. I'm sure you've seen nomp's mining keys? What do you think of this idea and code? And, if you think that's a good idea, would that be something you'd have an interest in supporting in eloipool? (maybe a module that figures a pubkey from a miningkey and drops it into config.py?)

Nope, no idea what you're talking about.

I know you're not the biggest altcoin fan, but I can't see any reason you shouldn't earn something from pools using your software to mine them. (forrestv has something similar for --give-author in p2pool I've not seen how this works though)

The sample config file has a Coinbaser example that donates.

[guruvan]

Thanks for the info Luke.
NOMP mining key creates a pubkey/privkey pair that can be used for any coin - the coin's pubkey can be derived from the mining-key pubkey, and likewise for the privkey.

Allows zone117x to collect from coinbase on pretty much every coin folks mine with his solution. And also, this is part of the magic in nomp that allows for multipools - coinswitching ports require mining-keys for usernames, and payment processor pays the coin to the derived coin pubkey

I don't have one running ATM, but raskul has his here: http://eu.centralcavern.uk:8080 the mining key creation web script is here http://eu.centralcavern.uk:8080/key.html - his setup looks pretty stock.

[Luke-Jr]

Sounds pointless, since addresses should only ever be used once.

[heromember]

bump

[peter_liang]

found error in script authproxy.py

Code: (Python)

       authpair = "%s:%s" % (self.__url.username, self.__url.password)
        authpair = authpair.encode('utf8')
        # <icreator 
        authpair = base64.b64encode(authpair) # here return as bytes
        authpair = str(authpair, encoding='utf-8') # convert bytes to str
        # >
        self.__auth_header = "Basic " + authpair
 

and I load directly authproxy:

Code: (Python3.3)

'''import jsonrpc
try:
	import jsonrpc.authproxy
	jsonrpc.authproxy.USER_AGENT = 'Eloipool/0.1'
except:
	from jsonrpclib import jsonrpc
	pass
'''
from authproxy import AuthServiceProxy as ServiceProxy, JSONRPCException

and copy client.py from /usr/lib/python3.3/http into my folder with eloipool

BUT!
when call getblocktemplate  from merklemaker.py

Code:

for TC in TCList:
			caccess = TC['access'] # =ServiceProxy(uri)
			try:
				propose = caccess.getblocktemplate(ProposeReq)

rise error in responce:

2013-08-28 15:11:39,777   makeCoinbaseTxn_responce   ERROR   {'error': {'message': 'Invalid mode', 'code': -8}, 'result': None, 'id': 1}
2013-08-28 15:11:39,780   merkleMaker   CRITICAL   Traceback (most recent call last):
  File "/home/icreator/pools/eloipool-3/merklemaker.py", line 732, in run
    self.merkleMaker_I()
  File "/home/icreator/pools/eloipool-3/merklemaker.py", line 722, in merkleMaker_I
    self.merkleMaker_II()
  File "/home/icreator/pools/eloipool-3/merklemaker.py", line 700, in merkleMaker_II
    return self._updateMerkleTree()
  File "/home/icreator/pools/eloipool-3/merklemaker.py", line 588, in _updateMerkleTree
    self._updateMerkleTree_I()
  File "/home/icreator/pools/eloipool-3/merklemaker.py", line 552, in _updateMerkleTree_I
    r = self._updateMerkleTree_fromTS(TS)
  File "/home/icreator/pools/eloipool-3/merklemaker.py", line 523, in _updateMerkleTree_fromTS
    (AcceptedScore, TotalScore) = self._CheckTemplate(newMerkleTree, TS)
  File "/home/icreator/pools/eloipool-3/merklemaker.py", line 456, in _CheckTemplate
    propose = caccess.getblocktemplate(ProposeReq)
  File "/home/icreator/pools/eloipool-3/authproxy.py", line 135, in __call__
    raise JSONRPCException(response['error'])
authproxy.JSONRPCException

"mode" set here:

Code:

		ProposeReq = {
			 "mode": "proposal",
			 "data": b2a_hex(data).decode('utf8'),
                        }

BUT if I do:

Code:

		ProposeReq = {
			 "data": b2a_hex(data).decode('utf8'),
                        }

new error:

Code:

2013-08-28 15:35:29,709	merkleMaker	ERROR	Upstream 'primary' rejected proposed block from 'primary': 0000000000000007d8be81c938a2519ba3b2eb8fe1720ad82210d4f82716a6f7
2013-08-28 15:35:29,710	merkleMaker	CRITICAL	Traceback (most recent call last):
  File "/home/icreator/pools/eloipool-3/merklemaker.py", line 732, in run
    self.merkleMaker_I()
  File "/home/icreator/pools/eloipool-3/merklemaker.py", line 722, in merkleMaker_I
    self.merkleMaker_II()
  File "/home/icreator/pools/eloipool-3/merklemaker.py", line 688, in merkleMaker_II
    return self._updateMerkleTree()
  File "/home/icreator/pools/eloipool-3/merklemaker.py", line 588, in _updateMerkleTree
    self._updateMerkleTree_I()
  File "/home/icreator/pools/eloipool-3/merklemaker.py", line 574, in _updateMerkleTree_I
    raise RuntimeError('Failed to create usable template')
RuntimeError: Failed to create usable template

and how use  jsonrpc_getblocktemplate.py ?? here may  be right code?

I stuck in the same problem for days. I'm using ubuntu 14.04, I tried both bitcoind v0.9.2.1 and v0.9.3rc1 and nothing is different. Can anybody tell me what should I do. Thank you.

[bitzcointalk]

    

I am hopefully on my last error but, since I have not seen any of the errors above this line, I am skeptical this will be the end-all.

Hello, I am seeing a bug, tried changing my hostname already, that was not the cause.

2014-09-21 19:13:26,818 SocketListener  ERROR   ('127.0.0.1', 8338)
Traceback (most recent call last):
  File "/home/pi/eloipool/util.py", line 144, in tryErr
    return func(*a, **kw)
  File "/home/pi/eloipool/networkserver.py", line 262, in setup_socket
    sock = self._makebind(server_address)
  File "/home/pi/eloipool/networkserver.py", line 253, in _makebind
    return self._makebind_py(*a, **ka)
  File "/home/pi/eloipool/networkserver.py", line 229, in _makebind_py
    sock = socket.socket(self.address_family, socket.SOCK_STREAM)
  File "/usr/lib/python3.2/socket.py", line 94, in __init__
    _socket.socket.__init__(self, family, type, proto, fileno)
socket.error: [Errno 97] Address family not supported by protocol

    

[bitzcointalk]

I changed everything in networkserver.py from AF_INET6 to AF_INET.
That error seems to be gone, but my miners cannot connect?!
(yes I also changed the config file removing the ::ffff:)

If there another log I should look at?
The process is indeed running, the miners are saying connection refused.

[bitzcointalk]

Owh, I also had to fix /home/pi/eloipool/bitcoin/node.py for the NET6 issue.

Okay, almost there. I have another error, I think it has to do with configuration (trying to get this to work for LiteCoin).

2014-09-21 20:23:58,032 merkleMaker     CRITICAL        Traceback (most recent call last):
  File "/home/pi/eloipool/merklemaker.py", line 694, in run
    self.merkleMaker_I()
  File "/home/pi/eloipool/merklemaker.py", line 684, in merkleMaker_I
    self.merkleMaker_II()
  File "/home/pi/eloipool/merklemaker.py", line 650, in merkleMaker_II
    return self._updateMerkleTree()
  File "/home/pi/eloipool/merklemaker.py", line 550, in _updateMerkleTree
    self._updateMerkleTree_I()
  File "/home/pi/eloipool/merklemaker.py", line 513, in _updateMerkleTree_I
    r = self._updateMerkleTree_fromTS(TS)
  File "/home/pi/eloipool/merklemaker.py", line 479, in _updateMerkleTree_fromTS
    newMerkleTree = self._ProcessGBT(MP, TS)
  File "/home/pi/eloipool/merklemaker.py", line 379, in _ProcessGBT
    cbtxn = self.makeCoinbaseTxn(MP['coinbasevalue'], prevBlockHex = MP['previousblockhash'])
  File "eloipool.py", line 131, in makeCoinbaseTxn
    pkScript = BitcoinScript.toAddress(config.TrackerAddr)
  File "/home/pi/eloipool/bitcoin/script.py", line 45, in toAddress
    raise ValueError('invalid address version')
ValueError: invalid address version

[bitzcointalk]

Owh, okay I found another version of the pool with slight modifications containing the scripts I need for scrypt.
(pun intended)