zefir (OP)
Donator
Hero Member
Offline
Activity: 919
Merit: 1000
|
|
July 06, 2013, 03:59:46 PM |
|
Developers, miners and geeks, I need to have a getwork proxy functionality added to cgminer and am willing to pay 5BTC for its development. OverviewLast year I had 5 computers distributed all over my flat running cgminer to run different types of mining rig. Since each instance operated its own set of pools and work items, I thought it would be a nice feature to have only one of them operating as proxy, i.e. it on one side connects to pools and handles work items and on the other side allow other cgminer instances to connect and get served. At that time I asked Con how feasible such a feature would be. His response was that it is doable, but since I was the only one requesting it, maybe not worth the work. With the subsequent feature advances in cgminer, I was able to reduce the number of instances to two and gradually refrained from my idea. Until recently, when it became hot again: I started operating Asicminer BE blades that are interfaced over Ethernet and use getwork as work API. The best way to operate them is to use a Stratum proxy. While generally doable, I miss lots of cgminer features (most relevant: pool backup / recovery, stability, scalability). With that, I realize that the proxy functionality I had in mind before to cascade multiple cgminer instances would also be useful to support BE blades. I am familiar with cgminer myself and have already a concept of how it could be realized (similar to hotplug_thread()). If I had 5 days of free time in a row, I'd do it myself - but obviously in today's interesting Bitcoin times I don't have it. Therefore I am offering a 5 BTC bounty to the skilled and motivated developer capable to develop that proxy functionality. RequirementsFrom a functional point, the requirements should be clear by the title. I need cgminer to be able to - act as a getwork proxy at a configurable port
- allow getwork based miners to connect (example: other cgminer instances, BE blades)
- keep track of the getwork clients (identified by username)
Initial testing can be accomplished by connecting multiple cgminer instances to the proxy. If required, I can provide wireshark traces of the communication between blades and Stratum proxy; also I will do qualification testing and performance evaluation with a larger scale BE blades setup myself. Beside the functional requirements, I am asking for - a cgminer compliant open source license to be used
- the coding style to be kept consistent (wish: Linux kernel coding style)
- sufficient code quality to get integrated into cgminer upstream
Alternative: stand alone proxyIf it makes development and testing easier for you, you are free to derive a getwork proxy component from existing cgminer. That is, this component would only use the pool and work management from cgminer, but not support mining devices itself (which btw. equals a hotplug-enabled cgminer with no active devices). cgminer's main source file is not modular enough to really support such an approach, just noting it here for completeness. ProcessBefore you start hacking into your keyboard, please contact me for further details via email (visible from my forum profile). Thanks, zefir
|
|
|
|
zefir (OP)
Donator
Hero Member
Offline
Activity: 919
Merit: 1000
|
|
July 08, 2013, 09:39:35 PM |
|
200+ views, zero feedback Is this too challenging, or am I missing something?
|
|
|
|
turtle83
|
|
July 09, 2013, 09:02:48 AM |
|
200+ views, zero feedback Is this too challenging, or am I missing something? You scoped it well. There is nothing to feedback.
|
|
|
|
zefir (OP)
Donator
Hero Member
Offline
Activity: 919
Merit: 1000
|
|
July 09, 2013, 12:50:42 PM |
|
200+ views, zero feedback Is this too challenging, or am I missing something? You scoped it well. There is nothing to feedback. I was hoping for feedback of type: "Yeah, that's something I always wanted to do - and now I can get even some coins for it!" Maybe after the recent exchange rate drop, 5 coins for 5 days of work is not enough as compensation. But anyhow, I have one develper already looking at it, hope it can be contributed soon to the community.
|
|
|
|
Luke-Jr
Legendary
Offline
Activity: 2576
Merit: 1186
|
|
August 18, 2013, 11:51:31 AM |
|
I've got the basic concept working as a new driver for BFGMiner: CPU 0: | 3.71/ 2.52/ 0.00Mh/s | A: 0 R:0+0(none) HW:0/none SGW 0: | 82.29/89.74/100.6Gh/s | A:2035 R:0+0(none) HW:0/none SGW 1: | 12.05/22.13/22.13Mh/s | A: 1 R:0+0(none) HW:0/none Each username creates a new SGW virtual device, and hashrate is measured by shares. In this case, I have my main dev miner running on SGW 0, and a lucky CPU miner on SGW 1.
|
|
|
|
zefir (OP)
Donator
Hero Member
Offline
Activity: 919
Merit: 1000
|
|
August 18, 2013, 06:53:09 PM |
|
I've got the basic concept working as a new driver for BFGMiner: CPU 0: | 3.71/ 2.52/ 0.00Mh/s | A: 0 R:0+0(none) HW:0/none SGW 0: | 82.29/89.74/100.6Gh/s | A:2035 R:0+0(none) HW:0/none SGW 1: | 12.05/22.13/22.13Mh/s | A: 1 R:0+0(none) HW:0/none Each username creates a new SGW virtual device, and hashrate is measured by shares. In this case, I have my main dev miner running on SGW 0, and a lucky CPU miner on SGW 1. Looks good! So you have an Avalon unit pointing to BFGminer and serving as GW proxy? Then it is basically done I'd need to check if the AM BE blades' GW implementation is working similarly well, so please let me know when the branch is ready for testing. Thanks for your time and efforts.
|
|
|
|
zefir (OP)
Donator
Hero Member
Offline
Activity: 919
Merit: 1000
|
|
August 19, 2013, 11:13:41 AM |
|
Luke-Jr implemented the getwork server / proxy functionality, which is now upstream in BFGminer master branch. Bounty paid. Proxy works as specified and was tested with multiple cascading instances of BFGminer. There are some specific issues with the BE blade open, which will be addressed separately (since they require access to a blade). Thanks a lot!
|
|
|
|
hanti
|
|
August 20, 2013, 10:49:18 PM |
|
can i ask if one of those issues with blades are low speed ? im getting like 2.8gh/s from one blade. On stratum proxy at same PC it works ok (max speed)
|
|
|
|
zefir (OP)
Donator
Hero Member
Offline
Activity: 919
Merit: 1000
|
|
August 21, 2013, 06:51:26 AM |
|
can i ask if one of those issues with blades are low speed ? im getting like 2.8gh/s from one blade. On stratum proxy at same PC it works ok (max speed)
Yes, exactly. I am currently looking at it, but it is a PITA to go through protocol traces of JSON-over-HTTP-over-TCP traffic
|
|
|
|
zefir (OP)
Donator
Hero Member
Offline
Activity: 919
Merit: 1000
|
|
August 22, 2013, 03:30:33 PM |
|
Update: ASICMINER BE blades now fully supported by bfgminerThe problem with the blades observed earlier was caused by the blades requiring a minimum size of the response packets and was fixed with the latest commits to bfgminer. With that, you can operate your blade pointing to a bfgminer instance ran with ./bfgminer --http-port <port> and enjoy all the unbeatable pool management and job scheduling features provided by bfg/cgminer. The GW server feature is built on top of Libmicrohttpd. On Debian based Linux systems install it with sudo apt-get install libmicrohttpd-dev and rebuild bfgminer from GIT sources. BE blades (or GW clients generally) are identified by their username. If you have more than one blade and configure individual usernames, each will have its own statistics. If otherwise all have the same name, they will show up as one SGW device with cumulative statistics. Happy Mining!
|
|
|
|
ewibit
Legendary
Offline
Activity: 2955
Merit: 1050
|
|
August 23, 2013, 12:30:38 PM |
|
and rebuild bfgminer from GIT sources.
In file included from httpsrv.c:15:0: /usr/include/microhttpd.h:497:3: Fehler: unbekannter Typname: »intptr_t« /usr/include/microhttpd.h:830:5: Fehler: unbekannter Typname: »uint64_t« /usr/include/microhttpd.h:868:46: Fehler: unbekannter Typname: »uint64_t« /usr/include/microhttpd.h:893:55: Fehler: unbekannter Typname: »va_list« /usr/include/microhttpd.h:1085:57: Fehler: unbekannter Typname: »uint64_t« /usr/include/microhttpd.h:1087:57: Fehler: unbekannter Typname: »MHD_ContentReaderCallback« /usr/include/microhttpd.h:1197:54: Fehler: unbekannter Typname: »MHD_PostDataIterator« make[2]: *** [bfgminer-httpsrv.o] Fehler 1
|
|
|
|
zefir (OP)
Donator
Hero Member
Offline
Activity: 919
Merit: 1000
|
|
August 23, 2013, 04:52:15 PM |
|
and rebuild bfgminer from GIT sources.
In file included from httpsrv.c:15:0: /usr/include/microhttpd.h:497:3: Fehler: unbekannter Typname: »intptr_t« /usr/include/microhttpd.h:830:5: Fehler: unbekannter Typname: »uint64_t« /usr/include/microhttpd.h:868:46: Fehler: unbekannter Typname: »uint64_t« /usr/include/microhttpd.h:893:55: Fehler: unbekannter Typname: »va_list« /usr/include/microhttpd.h:1085:57: Fehler: unbekannter Typname: »uint64_t« /usr/include/microhttpd.h:1087:57: Fehler: unbekannter Typname: »MHD_ContentReaderCallback« /usr/include/microhttpd.h:1197:54: Fehler: unbekannter Typname: »MHD_PostDataIterator« make[2]: *** [bfgminer-httpsrv.o] Fehler 1
Hard to say what's the issue without knowing your system, but the compiler not knowing standard integer types looks more like a build environment issue than a bfgminer related problem. Are you aware of the build process required to follow (pre-reqs, autogen, configure, make)? If not, be sure to read the READMEs. On a side note: there is lots of activity with bfgminer currently as preparation for the upcoming release. As a result, the HEAD in fact does not compile right now, so better wait until the release.
|
|
|
|
ewibit
Legendary
Offline
Activity: 2955
Merit: 1050
|
|
August 23, 2013, 04:55:46 PM |
|
Are you aware of the build process required to follow (pre-reqs, autogen, configure, make)? If not, be sure to read the READMEs.
thx yes I have done all pre-reqs
|
|
|
|
liyingfei
Newbie
Offline
Activity: 46
Merit: 0
|
|
August 29, 2013, 07:21:46 PM |
|
Are you aware of the build process required to follow (pre-reqs, autogen, configure, make)? If not, be sure to read the READMEs.
thx yes I have done all pre-reqs 我也遇到了这个问题 ubuntu server ,恳请知道 如何解决? CC bfgminer-httpsrv.o In file included from httpsrv.c:15:0: /usr/include/microhttpd.h:497:3: error: unknown type name ‘intptr_t’ /usr/include/microhttpd.h:830:5: error: unknown type name ‘uint64_t’ /usr/include/microhttpd.h:868:46: error: unknown type name ‘uint64_t’ /usr/include/microhttpd.h:893:55: error: unknown type name ‘va_list’ /usr/include/microhttpd.h:1085:57: error: unknown type name ‘uint64_t’ /usr/include/microhttpd.h:1087:57: error: unknown type name ‘MHD_ContentReaderCallback’ /usr/include/microhttpd.h:1197:54: error: unknown type name ‘MHD_PostDataIterator’ make[2]: *** [bfgminer-httpsrv.o] Error 1 make[2]: Leaving directory `/root/bfgminer' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/bfgminer' make: *** [all] Error 2
|
|
|
|
zefir (OP)
Donator
Hero Member
Offline
Activity: 919
Merit: 1000
|
|
August 30, 2013, 10:10:39 PM |
|
Are you aware of the build process required to follow (pre-reqs, autogen, configure, make)? If not, be sure to read the READMEs.
thx yes I have done all pre-reqs 我也遇到了这个问题 ubuntu server ,恳请知道 如何解决? CC bfgminer-httpsrv.o In file included from httpsrv.c:15:0: /usr/include/microhttpd.h:497:3: error: unknown type name ‘intptr_t’ /usr/include/microhttpd.h:830:5: error: unknown type name ‘uint64_t’ /usr/include/microhttpd.h:868:46: error: unknown type name ‘uint64_t’ /usr/include/microhttpd.h:893:55: error: unknown type name ‘va_list’ /usr/include/microhttpd.h:1085:57: error: unknown type name ‘uint64_t’ /usr/include/microhttpd.h:1087:57: error: unknown type name ‘MHD_ContentReaderCallback’ /usr/include/microhttpd.h:1197:54: error: unknown type name ‘MHD_PostDataIterator’ make[2]: *** [bfgminer-httpsrv.o] Error 1 make[2]: Leaving directory `/root/bfgminer' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/bfgminer' make: *** [all] Error 2 I am working with Linux. $ lsb_release -irc Distributor ID: Ubuntu Release: 13.04 Codename: raring
$ uname -a Linux PC 3.8.0-27-generic #40-Ubuntu SMP Tue Jul 9 00:17:05 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
$ gcc --version gcc (Ubuntu/Linaro 4.7.3-1ubuntu1) 4.7.3 Copyright (C) 2012 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Just tried with a fresh clone of the GIT repository and it worked. Step-by-Step: 1. ensure you have the required libs installed sudo apt-get install libjansson-dev libmicrohttpd-dev libncurses5-dev uthash-dev libcurl4-openssl-dev
2. clone the GIT repository git clone https://github.com/luke-jr/bfgminer.git
3. prepare and build cd bfgminer ./autogen.sh ./configure --disable-opencl --disable-bitforce --disable-adl --disable-modminer --disable-ztex --disable-avalon --disable-icarus --disable-modminer --disable-x6500 make
If this fails for you, and you have a different system, please post a bug report at https://github.com/luke-jr/bfgminer/issues
|
|
|
|
hanti
|
|
September 03, 2013, 03:04:52 PM Last edit: September 03, 2013, 03:23:10 PM by hanti |
|
it works almost perfect but i have much more rejected shares than on stratum proxy and blades stats page shows speed like 12600-12700 when on stratum proxy it was 13xxx and efficiency is 96.5 now, on stratum proxy it was like 99 i think maybe its bcouse 2 blades im getting much rejected shares (stale) im trying same user on both blades edit: yes with same user on both blades it works ok no more rejected shares edit2: yes confirmed again 2 diferent users = again some rejected (stale) shares for now it works perfect only if u set same user/pw on all blades (no stale rejected shares)
|
|
|
|
zefir (OP)
Donator
Hero Member
Offline
Activity: 919
Merit: 1000
|
|
September 03, 2013, 04:17:12 PM |
|
it works almost perfect but i have much more rejected shares than on stratum proxy and blades stats page shows speed like 12600-12700 when on stratum proxy it was 13xxx and efficiency is 96.5 now, on stratum proxy it was like 99 i think maybe its bcouse 2 blades im getting much rejected shares (stale) im trying same user on both blades edit: yes with same user on both blades it works ok no more rejected shares edit2: yes confirmed again 2 diferent users = again some rejected (stale) shares for now it works perfect only if u set same user/pw on all blades (no stale rejected shares) Hm, sounds like an issue I observed when debug log displays work-item queue underruns. In my case it helped to increase the the work-item queue, you could try adding e.g. to the parameter list or put it into the config file.
|
|
|
|
hanti
|
|
September 03, 2013, 07:49:17 PM Last edit: September 03, 2013, 08:12:29 PM by hanti |
|
with --queue 10 had many rejected shares too (3%) i think its ok now with --no-submit-stale option. it still shows at bfgminer stats A:1733 R:0+87(2.2%) but at least im not sending that shares to the mine whats strange with that option stats on blade config panel showing normal stats like total mhz 13k and efficency 99
edit: after like 2h its again same as before 12700-12800 ;p so no diference too
dont know why so many stales on bfgminer but not with stratum proxy
|
|
|
|
zefir (OP)
Donator
Hero Member
Offline
Activity: 919
Merit: 1000
|
|
September 03, 2013, 08:34:45 PM |
|
with --queue 10 had many rejected shares too (3%) i think its ok now with --no-submit-stale option. it still shows at bfgminer stats A:1733 R:0+87(2.2%) but at least im not sending that shares to the mine whats strange with that option stats on blade config panel showing normal stats like total mhz 13k and efficency 99
My blades settled at 2.3% after running for a week, and to me it seems perfectly accurate. Since the blades do not support LP, each time a block is found you get up to 32 stales (since each chip processes its own full nonce range). At 336 MHz, each chip needs 12.8 seconds for a full nonce range, so on average wastes 6.4 seconds per block change. With currently one block found every 440s, the theoretical loss is somewhere at 1.5% already - not too far from the measured stats.
|
|
|
|
hanti
|
|
September 03, 2013, 11:15:28 PM |
|
ye from what i can see on bitminter site in "latest shifts" speed is the same like with stratum proxy only one whats bothering me is blade stats page with stratum total speed = 13xxx and 99 efficiency but with bfgminer its 12600-12800 with 96 efficiency ;p
|
|
|
|
|