Modular Python Bitcoin Miner - Official Thread

[nelisky]

https://github.com/nelisky/Modular-Python-Bitcoin-Miner/tree/ztex

This is a very crude first attempt, many things are still missing, the devices need to be preflashed and the bitstreams must match the firmware. Only one device is supported (it does filter by serial, but then uses only the first device found) and the freq change algorithm from the error rate is much, much simpler than ztex's, and I only implemented the low speed FPGA config so far.

Next step would be multiple device suport but as I only have one currently I'm not going to blindly implement that. I might add to the code base to support more of ztex's original java source goodies ported to python, like the high speed config, but I'll be happy to pass that anyone else wanting to lend a hand.

Have fun (I'm mining on p2pool with my single ztex using this code right now).

[1713279298]

1713279298

[1713279298]

1713279298

[1713279298]

1713279298

[1713279298]

1713279298

[allinvain]

Awesome nelisky! Thanks for your contribution. Even though I do not have a ztex board I can definitely appreciate your working on support for it.

[coretechs]

I just set up MPBM yesterday and have to say I was impressed.  I will definitely be using this software in the future.  Donation sent, thanks!

[nelisky]

I have updated https://github.com/nelisky/Modular-Python-Bitcoin-Miner/tree/ztex with high speed FPGA configuration (which appears to be just as slow from my tests) and multi device hotplug support, though I only tested with my single board. Anyone having multiple ztex boards and wanting to experiment with MPBM, please give this a try and let me know.

I also created a pull request to get this into TheSeven's repo: https://github.com/TheSeven/Modular-Python-Bitcoin-Miner/pull/5

[O_Shovah]

I have finally gotten my ARM board + icarus + x6500 board  running with bitcoin-qt p2pool and MPBM.

I am currently getting somewhere around 13% stales.

How are your experiences regarding optimised values to improve efficiency on p2pool using MPBM ?

[shad]

I have finally gotten my ARM board + icarus + x6500 board  running with bitcoin-qt p2pool and MPBM.

I am currently getting somewhere around 13% stales.

How are your experiences regarding optimised values to improve efficiency on p2pool using MPBM ?

do you run bitcoin-qt on your arm device?
what OS are you using? Openwrt?

[O_Shovah]

I am using a modified Ubuntu 11.04 natty.
I recompiled both bitcoind and bitcoin-qt for the plattform.

[julz]

I just got this running with Icarus on Windows 7.. nice!
Bitcoin donated.

Is there a proper way to shut down MPBM? I've just been killing the command window as things like ctrl-c ctrl-z don't seem to have any effect.

[TheSeven]

I just got this running with Icarus on Windows 7.. nice!
Bitcoin donated.

Thank you!

Is there a proper way to shut down MPBM? I've just been killing the command window as things like ctrl-c ctrl-z don't seem to have any effect.

Hm, usually Ctrl+C should kill it. If it doesn't, there's probably something that locks up while attempting to shut things down. The Icarus worker module is currently kind of a 5 minute hack to make it work somehow, so it might suffer from that kind of thing. This will be improved in the upcoming major release.
Just killing the miner should be fine, it doesn't do any real cleanup anyway (yet).

[antirack]

I just got this running with Icarus on Windows 7.. nice!
Bitcoin donated.

Thank you!

Is there a proper way to shut down MPBM? I've just been killing the command window as things like ctrl-c ctrl-z don't seem to have any effect.

Hm, usually Ctrl+C should kill it. If it doesn't, there's probably something that locks up while attempting to shut things down. The Icarus worker module is currently kind of a 5 minute hack to make it work somehow, so it might suffer from that kind of thing. This will be improved in the upcoming major release.
Just killing the miner should be fine, it doesn't do any real cleanup anyway (yet).

Ctrl-C doesn't work on my Windows 7/Icarus setup neither. Ctrl-Break works.

I now also have 3 ZTEX boards, but I will be traveling for the next 10 days so I can't try it using the new code in MPBM.

[O_Shovah]

I am using a modified Ubuntu 11.04 natty.
I recompiled both bitcoind and bitcoin-qt for the plattform.

Have you done a writeup on this anywhere?

I will create thread for Bitcoin on ARM devices as soon as i find the time.
Maybe this weekend.
I will post descriptions  downloadable system images, build instructions....
For the "Gumstix" ARM , the panda board , and maybe for a rasberry pi
if i get one.

So you will have to be a little patient.
I would appreciate anybody else who has created systems on ARM devices
to post his system there.

And again:
what are your experience values  with MPBM and p2pool ( stales,rates, ..)
And wich values do you adjust to improve performance.

[TheSeven]

And again:
what are your experience values  with MPBM and p2pool ( stales,rates, ..)
And wich values do you adjust to improve performance.

Should be between 10% and 20% stales if p2pool performs well (which my installation clearly doesn't, due to a sluggish bitcoind).
And you can't really adjust any values to improve that, maybe increase the polling frequency if it's a x6500/ztex board.
The only thing you could do about it would be latency optimizing the code, which will break the modular approach (and multi-worker/multi-pool capabilities).

[TheSeven]

I've uploaded an MPBM v0.1.x prototype to the testing branch of the git repository.
It's working sufficiently to fetch work from the demo work sources somehow, so people can start porting worker interfaces. The SimpleRS232Worker module has already been ported, and might serve as kind of a porting guide.

Please not that this is not ready for production use yet. At the moment it supports neither long polling nor any other getwork extensions, and it's likely that there's still quite a lot of bugs. There also is no UI for the statistics yet (but the statistics should be collected in the background).

Tasks left to be done before the beta release:

• Clean up BCJSONRPC module
• Implement getwork extensions
• Possibly implement UBBP module
• Implement statistics UI (still needs some cosmetic improvements)
• Add work queue debug UI
• Possibly port CursesUI (likely not to be done)
• Port Icarus worker module (mostly working, minor statistics issues)
• Implement out of process worker support (might make sense for the X6500/ztex interfaces)
• Port X6500 worker modules (untested)
• Possibly implement support for uploading firmware to multiple X6500 boards at once, with only a single shared preparation process (will not be implemented)
• Possibly implement BFL worker module
• Decide what to do about the ztex worker module
• Possibly implement GPU mining (cherrypick from poclbm?) (not anytime soon)
• Fix bugs
• Fix even more bugs
• Improve documentation
• Clean up the code
• Do some benchmarks and identify/fix bottlenecks

[Turbor]

Finally working no curses, no miner... donation sent !

[TheSeven]

Finally working no curses, no miner... donation sent !

Are you sure that you prefer CursesUI over WebUI+StderrLogger with ANSI coding enabled?

[Turbor]

Finally working no curses, no miner... donation sent !

Are you sure that you prefer CursesUI over WebUI+StderrLogger with ANSI coding enabled?

  i'm a carpenter, no idea what you talk about

[Turbor]

Runs stable at 0.1% rejects. No more sleep mode. Only thing that's not working after some time is the average MH/s rate and the efficiency. It says 600+ MH for one board. Restart helps.

[TheSeven]

I'd like to invite everyone who's willing to take a small risk of deadlocks and other annoyances, to check out the testing branch of MPBM.
It isn't quite ready for production use that's unattended over months yet, but it has at least survived a week of testing on my own low-hashrate board.
It would be especially interesting for me to collect testing results from bigger FPGA rigs (>2GH/s, or even >10GH/s), but please only do that if you can react quickly in case of failure, to prevent wasting valuable hashing time.
Please note that latencies seem to be too high to work reasonably well with P2Pool. This seems to be the price you have to pay for flexibility. We'll see if that can be improved.

One of the most important new features is the web based user interface, which allows to configure the miner on the fly, and removes the need for a configuration file.

If you want to check it out, do this:

• Check out the testing branch (git clone git://github.com/TheSeven/Modular-Python-Bitcoin-Miner.git -b testing)
• Run "python run-mpbm.py"
• Open http://localhost:8832
• Log in with user name "admin", password "mpbm"
• Configure the WebUI frontend to set your own password
• Add your own work sources to the "user work sources" group
• Right click on the created work sources and assign them to the "Bitcoin" blockchain (or whichever blockchain they belong to), to enable long poll aggregation
• When you're done, hit "save configuration" (will be done during a clean shutdown anyway, but it can't hurt to do it manually after lots of changes to reduce the risk of data loss)

Btw. the correct way to shut down MPBM is just hitting ctrl+c on the console.

Known issues:

• Hashrate/priority settings do reach their targets after days of runtime, but shortly after starting the miner the hashrate distribution depends more on X-Roll-NTime support of the work source than anything else.
• The WebUI doesn't play nice with Firefox. Anyone got a fix for that?

If you hit some issues, or just want to provide some feedback, feel free to post here or contact me at #mpbm on irc.freenode.net.
In case you should hit a deadlock, please obtain a thread state dump from the debug menu on WebUI and pastebin that.

[coblee]

I can't seem to get my X6500 workers to work.

2012-03-18 17:17:44.525   [100]   X6500 worker - AH00WI18: Proxy: Exception caught: Traceback (most recent call last):
  File "/Users/charlie/Modular-Python-Bitcoin-Miner-testing/modules/fpgamining/x6500/boardproxy.py", line 77, in run
    else: self.device = FT232R(FT232R_PyUSB(self.serial, self.takeover))
  File "/Users/charlie/Modular-Python-Bitcoin-Miner-testing/modules/fpgamining/x6500/util/ft232r.py", line 314, in __init__
    for bus in usb.busses():
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/usb/legacy.py", line 333, in busses
    return (Bus(),)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/usb/legacy.py", line 329, in __init__
    self.devices = [Device(d) for d in core.find(find_all=True)]
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/usb/core.py", line 851, in find
    return [d for d in device_iter(k, v)]
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/usb/core.py", line 821, in device_iter
    for dev in backend.enumerate_devices():
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/usb/backend/libusb10.py", line 457, in enumerate_devices
    return _DevIterator()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/usb/backend/libusb10.py", line 445, in __init__
    byref(self.dev_list))
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/usb/backend/libusb10.py", line 403, in _check
    raise USBError(_str_error[ret], ret, _libusb_errno[ret])
USBError: [Errno None] Unknown error
2012-03-18 17:17:44.528   [100]   X6500 worker - AH00WI18: Traceback (most recent call last):
  File "/Users/charlie/Modular-Python-Bitcoin-Miner-testing/modules/fpgamining/x6500/x6500worker.py", line 211, in main
    elif data[0] == "dying": raise Exception("Proxy died!")
Exception: Proxy died!

[TheSeven]

USBError: [Errno None] Unknown error

Now that's the kind of libusb errors that I like most.
What OS is that? Mac? Does this affect only the testing branch or both? Does it work with the D2XX driver?