Bitcoin Forum
December 08, 2016, 12:13:56 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 [11] 12 13 14 15 16 17 18 19 20 »  All
  Print  
Author Topic: Modular Python Bitcoin Miner - Official Thread  (Read 69221 times)
antirack
Hero Member
*****
Offline Offline

Activity: 491


Immersionist


View Profile
March 28, 2012, 03:50:29 AM
 #201

Is this a bad board?

If your other two boards work, and switching around the cables, changing the socket on the USB hub, switching around power supply etc doesn't work, then that would be my guess.

I'd mail to support@fpgamining.com and see if they have any other ideas what the problem might be.

I haven't personally used the X6500 yet, but it may just be something stupid. I forgot to switch my Icarus boards ON since I was used to Ztex boards which don't have a on/off switch. You plug them in and they start mining (if the software is already running). This reminds me, I should test if Ztex hotplug really works with MPBM (edit: working very well).




1481156036
Hero Member
*
Offline Offline

Posts: 1481156036

View Profile Personal Message (Offline)

Ignore
1481156036
Reply with quote  #2

1481156036
Report to moderator
1481156036
Hero Member
*
Offline Offline

Posts: 1481156036

View Profile Personal Message (Offline)

Ignore
1481156036
Reply with quote  #2

1481156036
Report to moderator
1481156036
Hero Member
*
Offline Offline

Posts: 1481156036

View Profile Personal Message (Offline)

Ignore
1481156036
Reply with quote  #2

1481156036
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481156036
Hero Member
*
Offline Offline

Posts: 1481156036

View Profile Personal Message (Offline)

Ignore
1481156036
Reply with quote  #2

1481156036
Report to moderator
1481156036
Hero Member
*
Offline Offline

Posts: 1481156036

View Profile Personal Message (Offline)

Ignore
1481156036
Reply with quote  #2

1481156036
Report to moderator
1481156036
Hero Member
*
Offline Offline

Posts: 1481156036

View Profile Personal Message (Offline)

Ignore
1481156036
Reply with quote  #2

1481156036
Report to moderator
antirack
Hero Member
*****
Offline Offline

Activity: 491


Immersionist


View Profile
March 28, 2012, 04:07:31 AM
 #202

I've had my Icarus + Ztex combo up and running with MPBM for 1D 12h now.

I'd like to understand some of the numbers.

For instance, why are cancelled jobs on the Ztex boards 11.50%, 14.58% and 13.61% vs. only between 1.11% and 1.16% on my Icarus boards?

The 'GHashes total' on the Icarus seems lower per FPGA (ie. Ztex 28290 vs. Icarus 25045) than on the Ztex, but then the jobs per hour seems higher (Ztex 240-250 Icarus 308-318).

Do any other numbers seem off in the screenshot? I realize that Slush has a lot of upload errors. The timeout is 5 now (default), would increasing that possibly solve that problem? 5 seems already very long to me.

catfish
Sr. Member
****
Offline Offline

Activity: 270


teh giant catfesh


View Profile
March 28, 2012, 11:56:50 AM
 #203

Just a tip from the guy who managed to build the entire Ztex SDK from scratch on Mac OS X.

There are two libusb implementations, both called the same thing but incompatible with each other. With the Mac, which is Unix underneath but left mostly to the Mac hackers, no package management system is installed by default (for FOSS and CLI stuff, at least - the Apple Installer system is pretty sophisticated).

Most Mac hackers who bought OS X because it promised 'Linux power and Mac usability and Microsoft apps' (like me) use MacPorts, which is a BSD-type package management system for the CLI.

Of course, sudo port install libusb will get you the latest libusb. And the Ztex Java bindings are built on the previous legacy libusb.

Not sure what OS X itself comes packaged with because I'd started compiling my own before checking (OS X 'unix' libs often either lag version-wise or have dependencies that aren't necessary on non-Apple systems (Kerberos turns up a lot, for example)) - but if separate modules require both incompatible libusb implementations concurrently, then I'd consider statically linking the 'modules' to avoid confusion.

Mac OS X also has a habit of caching what it thinks it 'knows' each USB device is, and seems to periodically enumerate the bus on various signals. I wouldn't be surprised if the main issue behind this is iTunes and a daemon called usbmuxd, which handles the proprietary interface of iPods and iPhones over USB (and ramps up the current available to iPads, which can't charge well off 500mA).

However, get the right libraries and OS X is like any other Unix. I'm doing insane amounts of idiotic things with my Mac Pro right now, and the USB buses have hubs and high-bandwidth devices galore (I don't need 4 high-res webcams, microphones and sets of speakers on ONE workstation, but when they're integrated into your displays...) - yet my single Ztex FPGA, running Stefan's code with my mods to his Java USB bindings (which use the *legacy* libusb), is happily using the 64-bit jnilib and hashing away merrily. It hasn't broken - ever - I've had to bork the USB bus by sticking completely fecked mass storage devices in (cue 'uninterruptible wait' - which the Penguinboys get *just* as bad - knackered hardware is knackered hardware) to duff up the BTCMiner.

If you're running a farm though - give each board a different 'serial number' - it doesn't change the USB vendor / device ID and Stefan's code requires it. I've only got one so far, so can't thoroughly test the ramifications of ignoring this. It shows up nicely in the Mac System Profiler though Smiley


Make no mistake - when my 25 other boards arrive, I will certainly give this software a try, and if everyone on OS X is still having trouble then I'll work out a way round it. Hope this info is helpful and not already redundant... when I found out that there were two incompatible libusb implementations, I wanted to force the developers to program in x86 assembler for the rest of the year  Angry

...so I give in to the rhythm, the click click clack
I'm too wasted to fight back...


BTC: 1A7HvdGGDie3P5nDpiskG8JxXT33Yu6Gct
antirack
Hero Member
*****
Offline Offline

Activity: 491


Immersionist


View Profile
March 28, 2012, 12:01:01 PM
 #204

So far the only thing that shows up in my log file (log level 300) with 'long poll' or 'longpoll' is those error messages. How can I actually confirm that long polling is working? Canceled Jobs = long polling works?

Code:
2012-03-26 23:23:20.042000 [200]: ABCPool long poll failed: Traceback (most recent call last):
2012-03-26 23:23:20.042000 [200]:   File "C:\Modular-Python-Bitcoin-Miner\modules\theseven\bcjsonrpc\bcjsonrpcworksource.py", line 181, in _longpollingworker
2012-03-26 23:23:20.042000 [200]:     response = conn.getresponse()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 1046, in getresponse
2012-03-26 23:23:20.042000 [200]:     response.begin()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 346, in begin
2012-03-26 23:23:20.042000 [200]:     version, status, reason = self._read_status()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 316, in _read_status
2012-03-26 23:23:20.042000 [200]:     raise BadStatusLine(line)
2012-03-26 23:23:20.042000 [200]: http.client.BadStatusLine: ''

I have two other pools (slush + eclipse) but as said nothing that confirms any long polling in my log file.

And long polling with ABCPool works with Ztex BTCMiner.

I'd like to see a message in the logs when a long poll is received. Is that possible?
TheSeven
Hero Member
*****
Offline Offline

Activity: 504


FPGA Mining LLC


View Profile WWW
March 28, 2012, 01:52:02 PM
 #205

There are two libusb implementations, both called the same thing but incompatible with each other.

This actually shouldn't matter for MPBM. PyUSB (which MPBM relies on) supports both libusb versions, and something called openusb, which I think is apple's own libusb equivalent. So you might possibly get away without any libusb at all.

Gotta run now... Antirack, I'll explain the stats to you later today.

My tip jar: 13kwqR7B4WcSAJCYJH1eXQcxG5vVUwKAqY
TheSeven
Hero Member
*****
Offline Offline

Activity: 504


FPGA Mining LLC


View Profile WWW
March 28, 2012, 02:28:39 PM
 #206

So far the only thing that shows up in my log file (log level 300) with 'long poll' or 'longpoll' is those error messages. How can I actually confirm that long polling is working? Canceled Jobs = long polling works?

Canceled jobs do not necessarily mean that long polling actually works, because MPBM tries to detect new blocks by other means as well.

Code:
2012-03-26 23:23:20.042000 [200]: ABCPool long poll failed: Traceback (most recent call last):
2012-03-26 23:23:20.042000 [200]:   File "C:\Modular-Python-Bitcoin-Miner\modules\theseven\bcjsonrpc\bcjsonrpcworksource.py", line 181, in _longpollingworker
2012-03-26 23:23:20.042000 [200]:     response = conn.getresponse()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 1046, in getresponse
2012-03-26 23:23:20.042000 [200]:     response.begin()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 346, in begin
2012-03-26 23:23:20.042000 [200]:     version, status, reason = self._read_status()
2012-03-26 23:23:20.042000 [200]:   File "C:\python32\lib\http\client.py", line 316, in _read_status
2012-03-26 23:23:20.042000 [200]:     raise BadStatusLine(line)
2012-03-26 23:23:20.042000 [200]: http.client.BadStatusLine: ''

I have two other pools (slush + eclipse) but as said nothing that confirms any long polling in my log file.

And long polling with ABCPool works with Ztex BTCMiner.

This message does not mean that long polling does not work at all. It just means that a long poll connection died without getting a response. MPBM will automatically reconnect it, so if you only get that message occasionally, this is probably just a sign of a slightly unstable pool, connection issues (bad router? have seen this before...) or just someone restarting something on the pool side.

I'd like to see a message in the logs when a long poll is received. Is that possible?

If you increase the loglevel to 500, you should occasionally see these messages:
Code:
<Pool name>: Got <job number> jobs from long poll response

My tip jar: 13kwqR7B4WcSAJCYJH1eXQcxG5vVUwKAqY
TheSeven
Hero Member
*****
Offline Offline

Activity: 504


FPGA Mining LLC


View Profile WWW
March 28, 2012, 02:56:50 PM
 #207

I've had my Icarus + Ztex combo up and running with MPBM for 1D 12h now.

I'd like to understand some of the numbers.

For instance, why are cancelled jobs on the Ztex boards 11.50%, 14.58% and 13.61% vs. only between 1.11% and 1.16% on my Icarus boards?

A Ztex board takes about twice as long to process a job than an Icarus. This means that the probability that a job is hit by a long poll is twice as high.
The actual probability for a job to be canceled on an Icarus should be around 1.3% on average, twice that on Ztex. So your Icarus numbers look perfectly fine to me.
The reason that your Ztex boards show higher numbers is a bit more complicated: MPBM fetches new work as soon as the number of jobs in the queue that expire in more than 10 seconds goes below some threshold. However, if jobs are received in bursts due to X-Roll-NTime, it's rather likely that all jobs in the queue have the same expiry time, and that this is somewhere between 10 (fetcher threshold) and 15 (Ztex board job processing time) seconds in the future. Because of this, there are no jobs with a remaining life of >15 seconds in the queue, and the Ztex board is forced to work on a job that it can't possibly finish in time. As soon as the remaining jobs cross the 10 second boundary, MPBM will fetch new jobs, and as soon as the remaining life of the accepted job hits zero, the board will behave as if a long poll came in, cancel its current job and get a new one (with longer life). To account for network latencies, a safety margin of about 5 seconds is subtracted from the expiry time reported by the pool before inserting the jobs into the queue.
TL;DR: This is caused by MPBMs inner workings, and nothing to worry about.

The 'GHashes total' on the Icarus seems lower per FPGA (ie. Ztex 28290 vs. Icarus 25045) than on the Ztex, but then the jobs per hour seems higher (Ztex 240-250 Icarus 308-318).

This is partially caused by the effect I described above, and partially by the fact that MPBM pushes new work to the boards about 1 second before the board actually exhausts the noncerange of the previous job, because none of the boards that we have today support real queueing of jobs (which means we need to do this to prevent the board from idling briefly between jobs). This absolute time value doesn't depend on the hashrate, and thus offsets the number of jobs against the number of gigahashes.

Do any other numbers seem off in the screenshot? I realize that Slush has a lot of upload errors. The timeout is 5 now (default), would increasing that possibly solve that problem? 5 seems already very long to me.

5 Seconds should be plenty. If a pool fails to respond within this timeframe, I'd avoid that pool. I wouldn't increase that timeout, because that would decrease MPBM's aggressiveness to deliver a share, thereby possibly causing more stales, especially if the timeout was not caused by the pool software being overloaded, but rather by network packet loss (retrying is likely to be faster than waiting for the protocol layer to detect the packet loss, trigger retransmissions, and finally get a response). However it seems like stales are still within a reasonable margin on this instance, so I wouldn't care too much about those retries.

My tip jar: 13kwqR7B4WcSAJCYJH1eXQcxG5vVUwKAqY
BR0KK
Hero Member
*****
Offline Offline

Activity: 742



View Profile
March 30, 2012, 10:59:41 PM
 #208

Somehow i'm unable to get it running.... Ztex told me how to revert my "serial number problem" and every board has back its unique one. When i start mpbm (the windows packaged) on my VM the hot plug worker recognizes the 5 Boards. But they are in some kind of constant reconnection loop. The don't start mining since they are kicked out of the work immediately when the connect to it.

even one board doesn't function with it (and it did before)?

Some Screens (I'm on an iPad so I'm only able to make screenshots):



It could be som VM related problems .... so its not neccecary that this works:)

I'm only curious whats the cause of this is?


Ill hook them to my gnu miner if i have some time tomorrow.

TheSeven
Hero Member
*****
Offline Offline

Activity: 504


FPGA Mining LLC


View Profile WWW
March 30, 2012, 11:52:31 PM
 #209

Somehow i'm unable to get it running.... Ztex told me how to revert my "serial number problem" and every board has back its unique one. When i start mpbm (the windows packaged) on my VM the hot plug worker recognizes the 5 Boards. But they are in some kind of constant reconnection loop. The don't start mining since they are kicked out of the work immediately when the connect to it.

even one board doesn't function with it (and it did before)?

Some Screens (I'm on an iPad so I'm only able to make screenshots):

http://www.abload.de/thumb/bildschirmfoto2012-03hx5c4.png
http://www.abload.de/thumb/bildschirmfoto2012-038x07u.png

It could be som VM related problems .... so its not neccecary that this works:)

I'm only curious whats the cause of this is?


Ill hook them to my gnu miner if i have some time tomorrow.

That's the result of a typo in the ztex bitstream upload code. Apparently nobody has ever tested this before, and I can't do it myself because I don't have a board and ztex doesn't want to cooperate with me.
I have pushed a fix for that particular bug to the testing branch, however I'm not sure if this was the only problem with that code. There's likely more to it, if this indeed was never tested before.

If you haven't done that yet, you'll probably need to extract the bitstream files from BTCMiner and put them into the directory that you specified as the firmware path for that worker.

As a workaround, you can let another miner software upload the bitstream for now, and then switch to MPBM. This will be neccessary after every power cycle of the boards.

My tip jar: 13kwqR7B4WcSAJCYJH1eXQcxG5vVUwKAqY
antirack
Hero Member
*****
Offline Offline

Activity: 491


Immersionist


View Profile
March 31, 2012, 12:38:21 AM
 #210

BR0KK, before starting MPBM, fire up BTCMiner via command line and wait till all the boards are configured and it starts mining.

Then press Control-C to exit. Now start MPBM. This should fix the problem.


TheSeven
Hero Member
*****
Offline Offline

Activity: 504


FPGA Mining LLC


View Profile WWW
March 31, 2012, 12:55:05 AM
 #211

BR0KK, before starting MPBM, fire up BTCMiner via command line and wait till all the boards are configured and it starts mining.

Then press Control-C to exit. Now start MPBM. This should fix the problem.

Well, it's a workaround. A fix would be if I'd finally get a board from ztex to test this kind of stuff properly.

My tip jar: 13kwqR7B4WcSAJCYJH1eXQcxG5vVUwKAqY
antirack
Hero Member
*****
Offline Offline

Activity: 491


Immersionist


View Profile
March 31, 2012, 01:50:30 AM
 #212

I just pulled the latest code and tried it with the .bit file in the MPBM directory and the path (not filename, since the fn is set automatically it seems) set in the hotplug manager. Got this error (in addition to the "Can not open the specified device, possibly because it is already in use"):

Code:
2012-03-31 09:46:20.445000 [100]:   File "C:\Modular-Python-Bitcoin-Miner\modules\theseven\ztex\boardproxy.py", line 75, in run
2012-03-31 09:46:20.445000 [100]:     self.device = ZtexDevice(self, self.serial, self.takeover, self.firmware)
2012-03-31 09:46:20.445000 [100]:   File "C:\Modular-Python-Bitcoin-Miner\modules\theseven\ztex\driver.py", line 124, in __init__
2012-03-31 09:46:20.445000 [100]:     if sig2 < 0 or (sig1 >= 0 and sig1 < sig2): raise Exception("Signature not found in bitstream, wrong bit order?")
2012-03-31 09:46:20.445000 [100]: Exception: Signature not found in bitstream, wrong bit order?
2012-03-31 09:46:20.446000 [100]: Ztex board 04A32E00E9: Traceback (most recent call last):
2012-03-31 09:46:20.446000 [100]:   File "C:\Modular-Python-Bitcoin-Miner\modules\theseven\ztex\ztexworker.py", line 170, in main
2012-03-31 09:46:20.446000 [100]:     elif data[0] == "dying": raise Exception("Proxy died!")
2012-03-31 09:46:20.446000 [100]: Exception: Proxy died!
BR0KK
Hero Member
*****
Offline Offline

Activity: 742



View Profile
March 31, 2012, 11:56:02 AM
 #213

The Seven where do you come from? Not from Germany or somewhere near that?

TheSeven
Hero Member
*****
Offline Offline

Activity: 504


FPGA Mining LLC


View Profile WWW
March 31, 2012, 12:04:01 PM
 #214

The Seven where do you come from? Not from Germany or somewhere near that?
From exactly that Smiley

My tip jar: 13kwqR7B4WcSAJCYJH1eXQcxG5vVUwKAqY
BR0KK
Hero Member
*****
Offline Offline

Activity: 742



View Profile
March 31, 2012, 12:43:59 PM
 #215

Hm Ztex won't give u one for free for this ..... Would at least remote access to one help u?

May be Somerset of us Ztex owners could provide a board for testing? Kickstarter ?

Also mein VM Miner leuft 24/7 Und ich have Zugriff via teamviewer darauf. Wenns hilft kann ich dich drauf lassen?  

TheSeven
Hero Member
*****
Offline Offline

Activity: 504


FPGA Mining LLC


View Profile WWW
March 31, 2012, 04:04:31 PM
 #216

Hm Ztex won't give u one for free for this ..... Would at least remote access to one help u?

I still don't quite understand why he doesn't give me a free one for this, because he once stated he would offer free boards to mining software developers, but apparently he has changed his mind and tries to even prevent third party software support these days.

And yes, remote access would help, but I'll probably need some kind of a "board power cycling agent" at the other end of the tunnel to test the bitstream upload part.

May be Somerset of us Ztex owners could provide a board for testing? Kickstarter ?

Someone actually tried to raise a non-negligible bounty for cgminer ztex support, but as of now I don't know of any efforts to do that for MPBM. cgminer is just much more popular.

My tip jar: 13kwqR7B4WcSAJCYJH1eXQcxG5vVUwKAqY
BR0KK
Hero Member
*****
Offline Offline

Activity: 742



View Profile
March 31, 2012, 10:54:16 PM
 #217

Quote
And yes, remote access would help, but I'll probably need some kind of a "board power cycling agent" at the other end of the tunnel to test the bitstream upload part.

Remote Access its easy to give but the remote shutdown isn't. I currently do not own an IP - switchable power plug (IP Steckdose). Maybe i can find something the next time i'm at saturn or mediamarkt?. Couldn't be that expensive?

My Setup is somewhat complicated (Mac --> Parallels 7 --> Windows 7). I hope that won't have an effect on your programing? (I only have a windows VM, but i could install a Linux for that?).

Doing it directly on the mac isn't possible because mac osx LION does have some wierd problems with teamviewer 7 beta (all the other versions i've tried are not working properly to) and i literally lack any experience with UNIX or linux at all.


Tell me where u are from with a pm. Maybe we can work something out Smiley

I'm trying to get a new board soon (or more if i could raise the money)......

 

antirack
Hero Member
*****
Offline Offline

Activity: 491


Immersionist


View Profile
March 31, 2012, 11:37:59 PM
 #218

The BountyChest effort to get a ztex board for Con in Australia fr cgminer development didn't go anywhere. They are going to "refund" the people at one point as far as I know from mjt (bountychest dev).

If I didn't understand this wrong, with ztex's discount program you would have to buy your ztex board, then write the software, and then once done he would decide how much credit you get in form of a refund to your bank account.

http://www.ztex.de/os_discount_program.e.html

You will get the money refunded (via bank wire transfer to a European EUR account or via Paypal) after the project is published and rated.

You'd also have to publish it on his wiki etc and link back to ztex on the MPBM website/thread to qualify.

And the way his discount program sounds, a "simple" miner (which effectively duplicates functionality that's already there with his own open source project) wouldn't score very high on his 15% to >50% discount table IMHO.

I have to be absolutely honest here, if I would only have ztex boards I would probably never have tried MPBM. Now that I am using it, I see a couple of great and unique features (ie the way it handles work sources/blockchains). But I can also understand ztex for discouraging third party miner use, he has to cover his ass for the 2 years warranty and if somebody fries the FPGA chips it will be very costly to replace the boards.

The main advantage (to me) in having a third party miner would actually be to get it to work on platforms with a very low resource/power footprint. But I don't know enough about Python to say if that would be possible. I always liked xiangfu's approach of running his cgminer with 40+ Icarus in a mini dlink router that's not much bigger than a pack of cigarettes. This won't happen with Java. But would this be possible with python for instance?


antirack
Hero Member
*****
Offline Offline

Activity: 491


Immersionist


View Profile
March 31, 2012, 11:48:23 PM
 #219


Remote Access its easy to give but the remote shutdown isn't. I currently do not own an IP - switchable power plug (IP Steckdose). Maybe i can find something the next time i'm at saturn or mediamarkt?. Couldn't be that expensive?

BROKK, Remote Power Switch made in Germany (retail price Euro 399.00, I had a few of their 8 port a few years ago in a server cabinet):
http://www.ebay.de/itm/Leunig-Remote-Power-Switch-4G-/160772462474?pt=DE_Computer_Peripherie_Netzwerk&hash=item256ec9178a

This one is also cool (but probably dodgy and low quality). Control your power socket with a SIM card via a mobile phone.

http://www.ebay.de/itm/New-Fashion-Cellphone-Phone-PDA-GSM-RC-Remote-Control-Socket-Power-Switch-DD-/150764040585?pt=LH_DefaultDomain_77&hash=item231a3cb189

It's kind of universal, but it doesn't show what kind of male plugs it has on the back, so you may need to use a travel adapter to make the setup even more dodgy and the risk of burning down your house even higher.

PS: Although I am not from Germany and actually far far away, I am another native German speaker Wink
BR0KK
Hero Member
*****
Offline Offline

Activity: 742



View Profile
March 31, 2012, 11:57:32 PM
 #220

there is something similar from allnet (or other companies) thats far cheaper than this Smiley (i can't afford to pay 600€ for one purpose, if i had that money i could buy two and send one to TheSeven; but I'm out of money since i invested in 5 Ztex Boards last month).

Had one at my old workplace and its was controllable via Browser IP. I remember it costs a blot 60 -100€ or even less.


Pages: « 1 2 3 4 5 6 7 8 9 10 [11] 12 13 14 15 16 17 18 19 20 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!