New pure-python CPU miner, for fun and testing

[kseistrup]

Oh, and one more thing:

In the work loop, the line “hashes_done += 1” is effectively useless since nonce is already running from 0 to MAX_NONCE.  So at the end of the loop — either because a winning solution has been found, or because MAX_NONCE has been reached — hashes_done equals nonce+1.  Dropping the “hashes_done += 1” line gives a few extra khash/sec since this line was executed so many times.

Cheers,

[1713938741]

1713938741

[1713938741]

1713938741

[1713938741]

1713938741

[kseistrup]

I wonder if this is a bug:

Code:

        # hash final 4b, the nonce value
        hash1_o = hash1_precalc_o
        hash1_o.update(nonce_bin)
        hash1 = hash1_o.digest()

Do I interpret your meaning right when I say that the code — for each value of nonce — intends to calculate the SHA256 digest of the precalculated value (intended to stay constant throughout the loop) updated with the nonce?  If so, that is not what the code does.  Please see this simplified example:

Code:

>>> from hashlib import sha256
>>> pre = sha256('abc')
>>> post = pre
>>> post.update('xyz')
>>> pre.digest() == post.digest()
True
>>> 

So pre and post (like hash1_precalc_o and hash1_o) both points to the same object, and so I guess we should really write:

Code:

        # hash final 4b, the nonce value
        hash1_o = hashlib.sha256(blk_hdr + nonce_bin)
        hash1 = hash1_o.digest()

or even

Code:

        # hash final 4b, the nonce value
        hash1 = hashlib.sha256(blk_hdr + nonce_bin).digest()

Or did I misunderstand the whole concept?

Cheers,

[jgarzik]

Do I interpret your meaning right when I say that the code — for each value of nonce — intends to calculate the SHA256 digest of the precalculated value (intended to stay constant throughout the loop) updated with the nonce?  If so, that is not what the code does.  Please see this simplified example:

Well, that is disappointing.  If true, yes, that is a bug.  Updated.

[kseistrup]

Updated.

Do you happen to have a git repository of the code?

Cheers,

[jgarzik]

Updated.

Do you happen to have a git repository of the code?

No, just the URL.

[kseistrup]

No, just the URL.

Okay, no pull-request for you, then. 

Please consider the following patch for inclusion:

Code:

--- pyminer.py	2011-02-19 19:05:29.794843964 +0100
+++ pyminer-opt.py	2011-02-19 19:20:06.136093826 +0100
@@ -113,28 +113,22 @@
 		targetbin_str = targetbin.encode('hex')
 		target = long(targetbin_str, 16)
 
-		hashes_done = 1
-		for nonce in range(MAX_NONCE):
+		for nonce in xrange(MAX_NONCE):
 
 			# encode 32-bit nonce value
 			nonce_bin = struct.pack("<I", nonce)
 
 			# hash final 4b, the nonce value
-			hash1_o = hashlib.sha256()
-			hash1_o.update(blk_hdr)
+			hash1_o = hashlib.sha256(blk_hdr)
 			hash1_o.update(nonce_bin)
 			hash1 = hash1_o.digest()
 
 			# sha256 hash of sha256 hash
-			hash_o = hashlib.sha256()
-			hash_o.update(hash1)
+			hash_o = hashlib.sha256(hash1)
 			hash = hash_o.digest()
 
-			hashes_done += 1
-
 			# quick test for winning solution: high 32 bits zero?
-			H = struct.unpack('<I', hash[28:32])
-			if H:
+			if hash[-4:] != '\0\0\0\0':
 				continue
 
 			# convert binary hash to 256-bit Python long
@@ -147,12 +141,12 @@
 			# proof-of-work test:  hash < target
 			if l < target:
 				print time.asctime(), "PROOF-OF-WORK found: %064x" % (l,)
-				return (hashes_done, 
+				return (nonce+1, 
 					static_data[:76] + nonce_bin)
 			else:
 				print time.asctime(), "PROOF-OF-WORK false positive %064x" % (l,)
 
-		return (hashes_done, None)
+		return (nonce+1, None)
 
 	def iterate(self, rpc):
 		work = rpc.getwork()

The patch eliminates 2 calls to hashlib.sha256(), 1 call to struct.unpack() and 1 one variable increment per work loop.

Cheers,

[mail2345]

Psyco should speed things up.

[kseistrup]

Psyco should speed things up.

Psyco only runs on “obsolete” versions of Python (and it doesn't work on any 64-bit systems at all).

Cheers,

[bitjet]

Are you guys getting near triple efficiency with this vs over poclbm?

[Kiv]

I'm trying to run this and getting an exception:

Code:

Process Process-1:
Traceback (most recent call last):
  File "C:\Python26\lib\multiprocessing\process.py", line 232, in _bootstrap
    self.run()
  File "C:\Python26\lib\multiprocessing\process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\Chris\workspace\pyminer\pyminer.py", line 197, in miner_thread
    miner.loop()
  File "C:\Users\Chris\workspace\pyminer\pyminer.py", line 187, in loop
    rpc = BitcoinRPC(settings['host'], settings['port'],
KeyError: 'host'
1 mining threads started
Sun Feb 27 15:51:18 2011 Miner Starts - 127.0.0.1:8332
Sun Feb 27 15:51:18 2011 Miner Stops - 127.0.0.1:8332

It seems that the settings dictionary is empty, because when we're using separate processes the children don't have access to the parent's variables. Probably the settings should be passed in as constructor arguments. I might put this up on GitHub and hack on it a bit if you don't mind

[Kiv]

Are you guys getting near triple efficiency with this vs over poclbm?

Definitely not, I get about 900 KHash per core on poclbm and only 182 KHash per core on pure-python.

[kseistrup]

Code:

  File "C:\Users\Chris\workspace\pyminer\pyminer.py", line 187, in loop
    rpc = BitcoinRPC(settings['host'], settings['port'],
KeyError: 'host'

Pyminer takes as its single argument the name of the config file.  Please see message #10 in this thread.

Cheers,

[Kiv]

Thank you, but I did supply the argument. In fact the program will not start without the config file. My config file is the right format as well and has "host=127.0.0.1" in it.

I suspect that the program worked fine at one point and I have a more recent version with a bug in it.

Code:

  File "C:\Users\Chris\workspace\pyminer\pyminer.py", line 187, in loop
    rpc = BitcoinRPC(settings['host'], settings['port'],
KeyError: 'host'

Pyminer takes as its single argument the name of the config file.  Please see message #10 in this thread.

Cheers,

[kseistrup]

I suspect that the program worked fine at one point and I have a more recent version with a bug in it.

The current version runs fine with this config file (username and password edited for obvious reasons):

Code:

host = mining.bitcoin.cz
port = 8332
rpcuser = myRPCUser
rpcpass = myRPCPass
logdir = /tmp
threads = 1
hashmeter = 1

Cheers,

[Kiv]

This is probably a Windows-only bug. From the multiprocessing docs:


Since Windows lacks os.fork() it has a few extra restrictions:
Global variables

Bear in mind that if code run in a child process tries to access a global variable, then the value it sees (if any) may not be the same as the value in the parent process at the time that Process.start() was called.

If everyone else is running on Linux but I am running on Windows, this explains why it would work for others but not for me.

[kseistrup]

If everyone else is running on Linux but I am running on Windows, this explains why it would work for others but not for me.

It sure does.

Cheers,

[rasputin]

I wonder if this is a bug:

Code:

        # hash final 4b, the nonce value
        hash1_o = hash1_precalc_o
        hash1_o.update(nonce_bin)
        hash1 = hash1_o.digest()

Do I interpret your meaning right when I say that the code — for each value of nonce — intends to calculate the SHA256 digest of the precalculated value (intended to stay constant throughout the loop) updated with the nonce?  If so, that is not what the code does.  Please see this simplified example:

Code:

>>> from hashlib import sha256
>>> pre = sha256('abc')
>>> post = pre
>>> post.update('xyz')
>>> pre.digest() == post.digest()
True
>>> 

So pre and post (like hash1_precalc_o and hash1_o) both points to the same object

Instead of assignment, you'd use hashlib's copy() method to duplicate the hash state.

This code should work as intended:

Code:

# hash final 4b, the nonce value
hash1_o = hash1_precalc_o.copy()
hash1_o.update(nonce_bin)
hash1 = hash1_o.digest()

[rasputin]

I'm not sure you caught my point.

You're right, I didn't catch your point.  I have to chew on that, though.  What we need is a binary buffer with the right endianness, not a python long integer, right?

No, we really do need a 256-integer, because that is the fundamental proof-of-work test in the bitcoin system, comparing two 256-bit integers:

     hash < target

You may just as well compare the two arrays of binary data:

Code:

>>> a = "aaaa".decode('hex')
>>> b = "bbbb".decode('hex')
>>> a > b
False
>>> b > a
True

Got to love high level languages.

Considered putting this on github as well, btw? Being smaller and higher level, I find it a lot easier to grasp than the C miner. Too bad about all the byte order manipulation, but I guess that can't be helped.

[TehRainbowGuy]

I am loving the idea of this miner, much simpler than all of the others.

Just one question, any chance of a git repo?

[jgarzik]

Sure, here's a git repo:  https://github.com/jgarzik/pyminer