Modular Python Bitcoin Miner - Official Thread

[antirack]

Is this a bad board?

If your other two boards work, and switching around the cables, changing the socket on the USB hub, switching around power supply etc doesn't work, then that would be my guess.

I'd mail to support@fpgamining.com and see if they have any other ideas what the problem might be.

I haven't personally used the X6500 yet, but it may just be something stupid. I forgot to switch my Icarus boards ON since I was used to Ztex boards which don't have a on/off switch. You plug them in and they start mining (if the software is already running). This reminds me, I should test if Ztex hotplug really works with MPBM (edit: working very well).

[antirack]

I've had my Icarus + Ztex combo up and running with MPBM for 1D 12h now.

I'd like to understand some of the numbers.

For instance, why are cancelled jobs on the Ztex boards 11.50%, 14.58% and 13.61% vs. only between 1.11% and 1.16% on my Icarus boards?

The 'GHashes total' on the Icarus seems lower per FPGA (ie. Ztex 28290 vs. Icarus 25045) than on the Ztex, but then the jobs per hour seems higher (Ztex 240-250 Icarus 308-318).

Do any other numbers seem off in the screenshot? I realize that Slush has a lot of upload errors. The timeout is 5 now (default), would increasing that possibly solve that problem? 5 seems already very long to me.

[antirack]

So far the only thing that shows up in my log file (log level 300) with 'long poll' or 'longpoll' is those error messages. How can I actually confirm that long polling is working? Canceled Jobs = long polling works?

Code:

2012-03-26 23:23:20.042000 [200]: ABCPool long poll failed: Traceback (most recent call last):
2012-03-26 23:23:20.042000 [200]:   File "C:\Modular-Python-Bitcoin-Miner\modules\theseven\bcjsonrpc\bcjsonrpcworksource.py", line 181, in _longpollingworker
2012-03-26 23:23:20.042000 [200]:     response = conn.getresponse()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 1046, in getresponse
2012-03-26 23:23:20.042000 [200]:     response.begin()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 346, in begin
2012-03-26 23:23:20.042000 [200]:     version, status, reason = self._read_status()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 316, in _read_status
2012-03-26 23:23:20.042000 [200]:     raise BadStatusLine(line)
2012-03-26 23:23:20.042000 [200]: http.client.BadStatusLine: ''

I have two other pools (slush + eclipse) but as said nothing that confirms any long polling in my log file.

And long polling with ABCPool works with Ztex BTCMiner.

I'd like to see a message in the logs when a long poll is received. Is that possible?

[TheSeven]

There are two libusb implementations, both called the same thing but incompatible with each other.

This actually shouldn't matter for MPBM. PyUSB (which MPBM relies on) supports both libusb versions, and something called openusb, which I think is apple's own libusb equivalent. So you might possibly get away without any libusb at all.

Gotta run now... Antirack, I'll explain the stats to you later today.

[TheSeven]

So far the only thing that shows up in my log file (log level 300) with 'long poll' or 'longpoll' is those error messages. How can I actually confirm that long polling is working? Canceled Jobs = long polling works?

Canceled jobs do not necessarily mean that long polling actually works, because MPBM tries to detect new blocks by other means as well.

Code:

2012-03-26 23:23:20.042000 [200]: ABCPool long poll failed: Traceback (most recent call last):
2012-03-26 23:23:20.042000 [200]:   File "C:\Modular-Python-Bitcoin-Miner\modules\theseven\bcjsonrpc\bcjsonrpcworksource.py", line 181, in _longpollingworker
2012-03-26 23:23:20.042000 [200]:     response = conn.getresponse()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 1046, in getresponse
2012-03-26 23:23:20.042000 [200]:     response.begin()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 346, in begin
2012-03-26 23:23:20.042000 [200]:     version, status, reason = self._read_status()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 316, in _read_status
2012-03-26 23:23:20.042000 [200]:     raise BadStatusLine(line)
2012-03-26 23:23:20.042000 [200]: http.client.BadStatusLine: ''

I have two other pools (slush + eclipse) but as said nothing that confirms any long polling in my log file.

And long polling with ABCPool works with Ztex BTCMiner.

This message does not mean that long polling does not work at all. It just means that a long poll connection died without getting a response. MPBM will automatically reconnect it, so if you only get that message occasionally, this is probably just a sign of a slightly unstable pool, connection issues (bad router? have seen this before...) or just someone restarting something on the pool side.

I'd like to see a message in the logs when a long poll is received. Is that possible?

If you increase the loglevel to 500, you should occasionally see these messages:

Code:

<Pool name>: Got <job number> jobs from long poll response

[TheSeven]

I've had my Icarus + Ztex combo up and running with MPBM for 1D 12h now.

I'd like to understand some of the numbers.

For instance, why are cancelled jobs on the Ztex boards 11.50%, 14.58% and 13.61% vs. only between 1.11% and 1.16% on my Icarus boards?

A Ztex board takes about twice as long to process a job than an Icarus. This means that the probability that a job is hit by a long poll is twice as high.
The actual probability for a job to be canceled on an Icarus should be around 1.3% on average, twice that on Ztex. So your Icarus numbers look perfectly fine to me.
The reason that your Ztex boards show higher numbers is a bit more complicated: MPBM fetches new work as soon as the number of jobs in the queue that expire in more than 10 seconds goes below some threshold. However, if jobs are received in bursts due to X-Roll-NTime, it's rather likely that all jobs in the queue have the same expiry time, and that this is somewhere between 10 (fetcher threshold) and 15 (Ztex board job processing time) seconds in the future. Because of this, there are no jobs with a remaining life of >15 seconds in the queue, and the Ztex board is forced to work on a job that it can't possibly finish in time. As soon as the remaining jobs cross the 10 second boundary, MPBM will fetch new jobs, and as soon as the remaining life of the accepted job hits zero, the board will behave as if a long poll came in, cancel its current job and get a new one (with longer life). To account for network latencies, a safety margin of about 5 seconds is subtracted from the expiry time reported by the pool before inserting the jobs into the queue.
TL;DR: This is caused by MPBMs inner workings, and nothing to worry about.

The 'GHashes total' on the Icarus seems lower per FPGA (ie. Ztex 28290 vs. Icarus 25045) than on the Ztex, but then the jobs per hour seems higher (Ztex 240-250 Icarus 308-318).

This is partially caused by the effect I described above, and partially by the fact that MPBM pushes new work to the boards about 1 second before the board actually exhausts the noncerange of the previous job, because none of the boards that we have today support real queueing of jobs (which means we need to do this to prevent the board from idling briefly between jobs). This absolute time value doesn't depend on the hashrate, and thus offsets the number of jobs against the number of gigahashes.

Do any other numbers seem off in the screenshot? I realize that Slush has a lot of upload errors. The timeout is 5 now (default), would increasing that possibly solve that problem? 5 seems already very long to me.

5 Seconds should be plenty. If a pool fails to respond within this timeframe, I'd avoid that pool. I wouldn't increase that timeout, because that would decrease MPBM's aggressiveness to deliver a share, thereby possibly causing more stales, especially if the timeout was not caused by the pool software being overloaded, but rather by network packet loss (retrying is likely to be faster than waiting for the protocol layer to detect the packet loss, trigger retransmissions, and finally get a response). However it seems like stales are still within a reasonable margin on this instance, so I wouldn't care too much about those retries.

[BR0KK]

Somehow i'm unable to get it running.... Ztex told me how to revert my "serial number problem" and every board has back its unique one. When i start mpbm (the windows packaged) on my VM the hot plug worker recognizes the 5 Boards. But they are in some kind of constant reconnection loop. The don't start mining since they are kicked out of the work immediately when the connect to it.

even one board doesn't function with it (and it did before)?

Some Screens (I'm on an iPad so I'm only able to make screenshots):



It could be som VM related problems .... so its not neccecary that this works:)

I'm only curious whats the cause of this is?


Ill hook them to my gnu miner if i have some time tomorrow.

[TheSeven]

Somehow i'm unable to get it running.... Ztex told me how to revert my "serial number problem" and every board has back its unique one. When i start mpbm (the windows packaged) on my VM the hot plug worker recognizes the 5 Boards. But they are in some kind of constant reconnection loop. The don't start mining since they are kicked out of the work immediately when the connect to it.

even one board doesn't function with it (and it did before)?

Some Screens (I'm on an iPad so I'm only able to make screenshots):

http://www.abload.de/thumb/bildschirmfoto2012-03hx5c4.png
http://www.abload.de/thumb/bildschirmfoto2012-038x07u.png

It could be som VM related problems .... so its not neccecary that this works:)

I'm only curious whats the cause of this is?


Ill hook them to my gnu miner if i have some time tomorrow.

That's the result of a typo in the ztex bitstream upload code. Apparently nobody has ever tested this before, and I can't do it myself because I don't have a board and ztex doesn't want to cooperate with me.
I have pushed a fix for that particular bug to the testing branch, however I'm not sure if this was the only problem with that code. There's likely more to it, if this indeed was never tested before.

If you haven't done that yet, you'll probably need to extract the bitstream files from BTCMiner and put them into the directory that you specified as the firmware path for that worker.

As a workaround, you can let another miner software upload the bitstream for now, and then switch to MPBM. This will be neccessary after every power cycle of the boards.

[antirack]

BR0KK, before starting MPBM, fire up BTCMiner via command line and wait till all the boards are configured and it starts mining.

Then press Control-C to exit. Now start MPBM. This should fix the problem.

[TheSeven]

BR0KK, before starting MPBM, fire up BTCMiner via command line and wait till all the boards are configured and it starts mining.

Then press Control-C to exit. Now start MPBM. This should fix the problem.

Well, it's a workaround. A fix would be if I'd finally get a board from ztex to test this kind of stuff properly.

[antirack]

I just pulled the latest code and tried it with the .bit file in the MPBM directory and the path (not filename, since the fn is set automatically it seems) set in the hotplug manager. Got this error (in addition to the "Can not open the specified device, possibly because it is already in use"):

Code:

2012-03-31 09:46:20.445000 [100]:   File "C:\Modular-Python-Bitcoin-Miner\modules\theseven\ztex\boardproxy.py", line 75, in run
2012-03-31 09:46:20.445000 [100]:     self.device = ZtexDevice(self, self.serial, self.takeover, self.firmware)
2012-03-31 09:46:20.445000 [100]:   File "C:\Modular-Python-Bitcoin-Miner\modules\theseven\ztex\driver.py", line 124, in __init__
2012-03-31 09:46:20.445000 [100]:     if sig2 < 0 or (sig1 >= 0 and sig1 < sig2): raise Exception("Signature not found in bitstream, wrong bit order?")
2012-03-31 09:46:20.445000 [100]: Exception: Signature not found in bitstream, wrong bit order?
2012-03-31 09:46:20.446000 [100]: Ztex board 04A32E00E9: Traceback (most recent call last):
2012-03-31 09:46:20.446000 [100]:   File "C:\Modular-Python-Bitcoin-Miner\modules\theseven\ztex\ztexworker.py", line 170, in main
2012-03-31 09:46:20.446000 [100]:     elif data[0] == "dying": raise Exception("Proxy died!")
2012-03-31 09:46:20.446000 [100]: Exception: Proxy died!

[BR0KK]

The Seven where do you come from? Not from Germany or somewhere near that?

[TheSeven]

The Seven where do you come from? Not from Germany or somewhere near that?

From exactly that

[BR0KK]

Hm Ztex won't give u one for free for this ..... Would at least remote access to one help u?

May be Somerset of us Ztex owners could provide a board for testing? Kickstarter ?

Also mein VM Miner leuft 24/7 Und ich have Zugriff via teamviewer darauf. Wenns hilft kann ich dich drauf lassen?  

[TheSeven]

Hm Ztex won't give u one for free for this ..... Would at least remote access to one help u?

I still don't quite understand why he doesn't give me a free one for this, because he once stated he would offer free boards to mining software developers, but apparently he has changed his mind and tries to even prevent third party software support these days.

And yes, remote access would help, but I'll probably need some kind of a "board power cycling agent" at the other end of the tunnel to test the bitstream upload part.

May be Somerset of us Ztex owners could provide a board for testing? Kickstarter ?

Someone actually tried to raise a non-negligible bounty for cgminer ztex support, but as of now I don't know of any efforts to do that for MPBM. cgminer is just much more popular.

[BR0KK]

And yes, remote access would help, but I'll probably need some kind of a "board power cycling agent" at the other end of the tunnel to test the bitstream upload part.

Remote Access its easy to give but the remote shutdown isn't. I currently do not own an IP - switchable power plug (IP Steckdose). Maybe i can find something the next time i'm at saturn or mediamarkt?. Couldn't be that expensive?

My Setup is somewhat complicated (Mac --> Parallels 7 --> Windows 7). I hope that won't have an effect on your programing? (I only have a windows VM, but i could install a Linux for that?).

Doing it directly on the mac isn't possible because mac osx LION does have some wierd problems with teamviewer 7 beta (all the other versions i've tried are not working properly to) and i literally lack any experience with UNIX or linux at all.


Tell me where u are from with a pm. Maybe we can work something out

I'm trying to get a new board soon (or more if i could raise the money)......

 

[antirack]

The BountyChest effort to get a ztex board for Con in Australia fr cgminer development didn't go anywhere. They are going to "refund" the people at one point as far as I know from mjt (bountychest dev).

If I didn't understand this wrong, with ztex's discount program you would have to buy your ztex board, then write the software, and then once done he would decide how much credit you get in form of a refund to your bank account.

http://www.ztex.de/os_discount_program.e.html

You will get the money refunded (via bank wire transfer to a European EUR account or via Paypal) after the project is published and rated.

You'd also have to publish it on his wiki etc and link back to ztex on the MPBM website/thread to qualify.

And the way his discount program sounds, a "simple" miner (which effectively duplicates functionality that's already there with his own open source project) wouldn't score very high on his 15% to >50% discount table IMHO.

I have to be absolutely honest here, if I would only have ztex boards I would probably never have tried MPBM. Now that I am using it, I see a couple of great and unique features (ie the way it handles work sources/blockchains). But I can also understand ztex for discouraging third party miner use, he has to cover his ass for the 2 years warranty and if somebody fries the FPGA chips it will be very costly to replace the boards.

The main advantage (to me) in having a third party miner would actually be to get it to work on platforms with a very low resource/power footprint. But I don't know enough about Python to say if that would be possible. I always liked xiangfu's approach of running his cgminer with 40+ Icarus in a mini dlink router that's not much bigger than a pack of cigarettes. This won't happen with Java. But would this be possible with python for instance?

[antirack]

Remote Access its easy to give but the remote shutdown isn't. I currently do not own an IP - switchable power plug (IP Steckdose). Maybe i can find something the next time i'm at saturn or mediamarkt?. Couldn't be that expensive?

BROKK, Remote Power Switch made in Germany (retail price Euro 399.00, I had a few of their 8 port a few years ago in a server cabinet):
http://www.ebay.de/itm/Leunig-Remote-Power-Switch-4G-/160772462474?pt=DE_Computer_Peripherie_Netzwerk&hash=item256ec9178a

This one is also cool (but probably dodgy and low quality). Control your power socket with a SIM card via a mobile phone.

http://www.ebay.de/itm/New-Fashion-Cellphone-Phone-PDA-GSM-RC-Remote-Control-Socket-Power-Switch-DD-/150764040585?pt=LH_DefaultDomain_77&hash=item231a3cb189

It's kind of universal, but it doesn't show what kind of male plugs it has on the back, so you may need to use a travel adapter to make the setup even more dodgy and the risk of burning down your house even higher.

PS: Although I am not from Germany and actually far far away, I am another native German speaker

[BR0KK]

there is something similar from allnet (or other companies) thats far cheaper than this (i can't afford to pay 600€ for one purpose, if i had that money i could buy two and send one to TheSeven; but I'm out of money since i invested in 5 Ztex Boards last month).

Had one at my old workplace and its was controllable via Browser IP. I remember it costs a blot 60 -100€ or even less.

[antirack]

I checked their price list. New price Euro 399. The one on eBay costs Euro 150 and is 4 port. But I get your point. Probably overdoing it for a home setup with a few FPGA boards. If you control servers you need something robust, that's why they are still in business even with those prices.

(The allnet thing is 8A max, the Leunig one 16A continues and 120A tops).