Bitcoin Forum
December 14, 2017, 07:14:57 AM *
News: Latest stable version of Bitcoin Core: 0.15.1  [Torrent].
 
   Home   Help Search Donate Login Register  
Poll
Question: Would you use such a mining proxy?  (Voting closed: April 13, 2011, 10:26:05 PM)
Yes, it seems like a good idea. - 7 (63.6%)
Maybe. - 1 (9.1%)
No, I don't like the idea. - 3 (27.3%)
No, I use something similar already. - 0 (0%)
Total Voters: 11

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [All]
  Print  
Author Topic: Flexible mining proxy  (Read 87865 times)
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
April 06, 2011, 10:26:05 PM
 #1

Edit 2011-05-04: The software has been released! (View post)

----------

Hey all, I'm trying to gauge interest for this.

I've been hacking for about a week and a half on a mining proxy written in PHP using MySQL for the data store.  I've been running my own miners against it with no problems for the last week.  The basic idea is that you can run multiple miners against multiple pools (same or different, it doesn't matter) and miners can fail-over to other pools if something happens to its preferred pool.

Additionally, using the web interface, you can manage pool assignments from any physical location; your miners won't notice that anything has changed when you switch them between pools.  The information on the dashboard can also be used to help determine when a miner goes AWOL.

Here's the more detailed list of how it all works:

  • Multiple pools can be defined.  Pools can be globally enabled/disabled for all workers.
  • Multiple workers can be defined, each with their own credentials to be used against the proxy itself.
  • Each worker can associate with as many pools as you have defined, and can have its own credentials to be used with that pool.  (In other words, you can have worker A and worker B both working slush's pool, but each using their own worker accounts.)
  • Worker-pool associations can be individually enabled/disabled without affecting other workers, or other pools associated with the worker.
  • Worker-pool associations can be ranked by priority.  The highest priority association will be used -- unless that pool is down or not responding, then the next-highest will be tried.
  • All getwork requests are partially logged in the database, and all work submissions are logged as well.  This includes which worker sent the request, which pool ultimately handled the request, and (in the case of work submissions) whether the share was accepted or not.

All this is manageable from a web-based control panel.  Right now the project is not terribly polished -- not well enough for a release anyway -- but the core seems to be working great.  If there is any interest in such a project, I will probably release it under the AGPL.

I'm interested in the views and perspectives of my fellow miners as to whether this project would have any value to the wider community.

Mandatory screenshot:


Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1513235697
Hero Member
*
Offline Offline

Posts: 1513235697

View Profile Personal Message (Offline)

Ignore
1513235697
Reply with quote  #2

1513235697
Report to moderator
1513235697
Hero Member
*
Offline Offline

Posts: 1513235697

View Profile Personal Message (Offline)

Ignore
1513235697
Reply with quote  #2

1513235697
Report to moderator
xenon481
Sr. Member
****
Offline Offline

Activity: 406



View Profile
April 06, 2011, 10:32:39 PM
 #2

I really like the idea of being able to do failovers, but the best part is that it is a great starting place for doing a really sophisticated implementation of Raulo's Pool Hopping Exploit!  Cry

Tips Appreciated: 171TQ2wJg7bxj2q68VNibU75YZB22b7ZDr
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
April 06, 2011, 10:40:59 PM
 #3

I really like the idea of being able to do failovers, but the best part is that it is a great starting place for doing a really sophisticated implementation of Raulo's Pool Hopping Exploit!  Cry
Just to be clear, I take no liability for any forks of the project.  Wink

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
slush
Legendary
*
Offline Offline

Activity: 1372



View Profile WWW
April 06, 2011, 10:53:18 PM
 #4

You did first metapool. It was just matter of the time, but still - congratz  Grin

Edit: Does this solve long polling somehow?

jgarzik
Legendary
*
Offline Offline

Activity: 1484


View Profile
April 06, 2011, 10:55:33 PM
 #5

IMO it makes much more sense to add multi-pool support to each mining client.

That way it doesn't break long-polling, and you can more easily utilize pool-specific features as they appear (such as using BDP).

A meta-pool is an additional point of failure.

Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
April 06, 2011, 11:49:13 PM
 #6

You did first metapool. It was just matter of the time, but still - congratz  Grin
Thanks.  Smiley

Edit: Does this solve long polling somehow?
The current revision does not, but this is going to be implemented pretty soon.  Obviously it will only work on pools that support LP themselves; it will just proxy the LP request.

IMO it makes much more sense to add multi-pool support to each mining client.
I agree.  But I don't have the desire to hack on every client out there to implement something like this.  Further, this approach also gives me other benefits like the ability to manage miners remotely, retarget their pool assignment without restarting them, etc.

If someone implements a consistent multi-pool interface in all the mining clients, I would probably deprecate this project.  But in the meantime it fills the gap and also gives you some other nifty features that client-side multi-pool support alone wouldn't.

That way it doesn't break long-polling, and you can more easily utilize pool-specific features as they appear (such as using BDP).
LP is only "broken" in that I have not yet implemented it.  There's no technical reason it can't be done, I've just been focused on other aspects of the proxy.

A similar BDP proxy could be implemented alongside the existing HTTP-based proxy, with minimal database schema changes.  Once that is done, you could theoretically run HTTP-only miners against a BDP pool efficiently, by having the HTTP proxy get work from the local BDP hub process.

A meta-pool is an additional point of failure.
To a degree, yes.  I run my proxy on my LAN, so network failure is extremely unlikely.  A software bug is the only thing I can think of that would cause a problem (other than the whole box going down, at which point I lose DNS too), and with (... checking DB ...) 82,000 getwork requests processed in the last 2.5 days, my confidence in the proxy code is very high at this point.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
April 12, 2011, 06:42:43 PM
 #7

Long-polling proxying is now implemented.  The only remaining feature on my list is connection pooling to take advantage of HTTP 1.1 keep-alive connections, but I'm not sure how feasible this is in PHP without using some external connection-pooling daemon.  I might make a release before this feature is implemented.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
mlg.odk
Newbie
*
Offline Offline

Activity: 5


View Profile
April 18, 2011, 03:35:58 PM
 #8

This project appears to be very interesting and is in fact exactly what I've been looking for to connect all the machines I have here to one single worker account.
Is there any release date set already?
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
April 19, 2011, 05:09:16 PM
 #9

This project appears to be very interesting and is in fact exactly what I've been looking for to connect all the machines I have here to one single worker account.

If you are using e.g. slush's pool, you should still have a separate account for each worker.  My proxy allows multiple miners to authenticate to it with separate credentials, and the proxy will then authenticate to pools using credentials stored for that worker.  In other words, each worker-pool assignment has its own pool credentials.

Is there any release date set already?

It's "as soon as I clean up the Git repo."  Smiley  I hope to get to that this week.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
mlg.odk
Newbie
*
Offline Offline

Activity: 5


View Profile
April 19, 2011, 08:27:53 PM
 #10

If you are using e.g. slush's pool, you should still have a separate account for each worker.  My proxy allows multiple miners to authenticate to it with separate credentials, and the proxy will then authenticate to pools using credentials stored for that worker.  In other words, each worker-pool assignment has its own pool credentials.

Generally this is how I do it. However, let's take slush's pool and a GPU cluster which is not dedicated to generating Bitcoins and does this only when being idle as example: The amount of GPUs and machines available constantly changes. While there is a max. limit of machines, I'd still require a software to dynamically associate a worker account to each machine. This is because (here's where slush's pool comes in) slush's reward calculating formula includes the time at which the last share was submitted. Of course this is a great way to prevent cheating, however it also means that if one machine goes offline or switches to a different task, the reward it would usually have made quickly shrinks to zero. Therefore using a proxy tool like yours for such a cluster is more effective and way easier to manage and automate.


It's "as soon as I clean up the Git repo."  Smiley  I hope to get to that this week.

Great! I'll see to drop you some coins when it is out Smiley
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
April 20, 2011, 12:12:34 PM
 #11

Generally this is how I do it. However, let's take slush's pool and a GPU cluster which is not dedicated to generating Bitcoins and does this only when being idle as example: The amount of GPUs and machines available constantly changes. While there is a max. limit of machines, I'd still require a software to dynamically associate a worker account to each machine. This is because (here's where slush's pool comes in) slush's reward calculating formula includes the time at which the last share was submitted. Of course this is a great way to prevent cheating, however it also means that if one machine goes offline or switches to a different task, the reward it would usually have made quickly shrinks to zero. Therefore using a proxy tool like yours for such a cluster is more effective and way easier to manage and automate.

The way I understand the math, all of your recently-submitted shares are considered, no matter which worker they were submitted through -- so running several workers on one worker account or several should make no difference to the amount of reward you get when the block is solved.  Even if one machine drops out... since the shares are considered collectively, it doesn't matter which account they were submitted through.  They will age just like every other share you submit.  (Correct me if I'm wrong, slush.)  In other words, the older shares you submitted from the still-active machines have aged and become worthless too, you just can't tell because they are submitting enough new shares to keep the reward up.

Now, having said that, my proxy doesn't "dynamically associate a worker account to each machine."  You still need to set up worker accounts in my proxy script.  The difference is that you can assign those worker accounts to more than one pool.  (Although you could probably hack it to do what you want. Wink)

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
mlg.odk
Newbie
*
Offline Offline

Activity: 5


View Profile
April 20, 2011, 01:40:25 PM
 #12

The way I understand the math, all of your recently-submitted shares are considered, no matter which worker they were submitted through -- so running several workers on one worker account or several should make no difference to the amount of reward you get when the block is solved.  Even if one machine drops out... since the shares are considered collectively, it doesn't matter which account they were submitted through.  They will age just like every other share you submit.  (Correct me if I'm wrong, slush.)  In other words, the older shares you submitted from the still-active machines have aged and become worthless too, you just can't tell because they are submitting enough new shares to keep the reward up.

This is interesting, in that case there still is the clear advantage of being able to quickly switch to a different pool in case that the current one goes offline for whatever reason.


Now, having said that, my proxy doesn't "dynamically associate a worker account to each machine."  You still need to set up worker accounts in my proxy script.  The difference is that you can assign those worker accounts to more than one pool.  (Although you could probably hack it to do what you want. Wink)

Ah I think you misunderstood me there, sorry if I was not clear enough. I ment to say that the alternative to what I said was your software; I understood the purpose of your software correctly.
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
April 23, 2011, 10:36:39 PM
 #13

What is the status of this?  I'd like to start testing and maybe contributing code if you're ready to release it into the wild.
BitLex
Hero Member
*****
Offline Offline

Activity: 588


View Profile
April 23, 2011, 11:30:33 PM
 #14

i would also like to test this, so please keep us up to date.  Wink

anisoptera
Member
**
Offline Offline

Activity: 98



View Profile
May 04, 2011, 06:39:21 PM
 #15

+1 for wanting to test this please Smiley
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 04, 2011, 08:10:12 PM
 #16

After recent strangeness with Deepbit, I'm putting a 10 BTC bounty on this, contingent upon ultimate release of the working product and sourcecode before I'm forced to write my own.  Looking forward to getting up and running.
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 04, 2011, 11:04:44 PM
 #17

Alright, I've cleaned up the repo and finished adding the final touches (for now...).  Grab the code: https://github.com/cdhowie/Bitcoin-mining-proxy

Setup directions are in the INSTALL file.  Please read the whole thing before asking questions; I took a long time writing it and proofread it several times to make sure it's complete and correct.  Let me know if anything is unclear or broken.  And please feel free to use GitHub's issue tracking to report issues.  Smiley

If you like the software and find it useful, there is a GPG-signed Bitcoin address on the about page where you can send donations.  Any amount would be appreciated.

For those interested in reliability information, I've been running one miner against it consistently for a full month, and a few other miners against it sporadically.  The getwork proxy has always worked 100% for me in my deployed copy, except during power failures.  Wink

@pwnyboy: Thanks for offering the bounty!  I was already getting it ready for release when you posted, so you don't have to send the bounty if you don't want (since it really wasn't responsible for motivating me; I just finally got off my lazy butt since everyone's been waiting for it).  But I won't turn it down either.  Wink  If you still want to send, feel free to use the donation address.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 06, 2011, 04:32:47 AM
 #18

Thanks for the release!  I'll dedicate some time to pulling and installing it tomorrow, and pass that money along once it's up.  After all, a promise is a promise Smiley  Thanks again.
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 06, 2011, 04:36:29 AM
 #19

Thanks for the release!  I'll dedicate some time to pulling and installing it tomorrow, and pass that money along once it's up.  After all, a promise is a promise Smiley  Thanks again.

Cool, thanks!  Hope the software works for you.

If you run into any trouble, let me know.  I intend to support the software I publish.  (That goes for everyone.)

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
aahzmundus
Hero Member
*****
Offline Offline

Activity: 644


Invest & Earn: https://cloudthink.io


View Profile
May 06, 2011, 04:40:52 AM
 #20

If you run into any trouble, let me know.  I intend to support the software I publish.  (That goes for everyone.)

He is serious about this, worked with me on and off for a good hour helping me figure stuff out step by step.  Already made a small donation, well worth it.

BitLex
Hero Member
*****
Offline Offline

Activity: 588


View Profile
May 06, 2011, 04:52:28 AM
 #21

i only get an xml-parsing error when trying to access /admin/
Code:
XML Parsing Error: XML or text declaration not at start of entity
Location: http://blah/admin/
Line Number 2, Column 1:<?xml version="1.0" encoding="UTF-8"?>

guess it's just a blank line hiding somewhere, but i can't find it.

pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 06, 2011, 05:38:03 AM
 #22

If you run into any trouble, let me know.  I intend to support the software I publish.  (That goes for everyone.)

He is serious about this, worked with me on and off for a good hour helping me figure stuff out step by step.  Already made a small donation, well worth it.

Sweet!  Maybe you can advise on the following.  I broke down and decided to stay up past my bedtime to pull your code and put it online.  I took your advice and put this on a separate vhost in apache, because my mining client (phoenix) would not hit http://domain.com/some/big/url as you pointed out.  I threw this on a CentOS box that already has working php/mysql sites on it, so any dependencies other than those two are hopefully met already.  When I connect Phoenix, it tells me 'failed to connect, retrying'.  When I examine my apache error log, I see this:

[Fri May 06 01:30:05 2011] [error] [client xxx.113] PHP Notice:  Undefined index:  lpurl in /home/rrb/bitproxy/index.php on line 70
[Fri May 06 01:30:05 2011] [error] [client xxx.113] PHP Notice:  Undefined variable: http_response_header in /home/rrb/bitproxy/common.inc.php on line 117
[Fri May 06 01:30:05 2011] [error] [client xxx.113] PHP Warning:  Invalid argument supplied for foreach() in /home/rrb/bitproxy/index.php on line 215

Additionally, you might find this of interest.  When I hit the admin url, the admin interface seems to work fine but merely hitting /admin/ throws the following into the apache error log:

[Fri May 06 01:31:15 2011] [error] [client xxx.252] PHP Notice:  Undefined index:  format in /home/rrb/bitproxy/mvc.inc.php on line 38
[Fri May 06 01:31:15 2011] [error] [client xxx.252] PHP Notice:  Undefined index:  tempdata in /home/rrb/bitproxy/common.inc.php on line 177
[Fri May 06 01:31:15 2011] [error] [client xxx.252] PHP Notice:  Undefined index:  tempdata in /home/rrb/bitproxy/common.inc.php on line 177

(IPs redacted).  I'm pretty sure I followed your instructions to the letter, but have not yet taken a look at the code to try and debug this (maybe tomorrow).  Any ideas?
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 06, 2011, 07:32:12 AM
 #23

So insomnia set in, and I decided to dig deeper.  I found a couple more things:

1) In your database schema, in the worker_pool table, the type for the 'enabled' field is a tinyint and should be boolean it looks like, unless you have some special state info to store that's not 0 or 1?
2) I discovered #1 while attempting to go back and edit the workers and pool info that I'd setup the first time, in an effort to debug what was happening with the behavior I reported above.  While using the various "edit" dialogs, I'm hitting blank pages being returned, i.e. in the "edit pool" dialog at http://xxx/admin/pool.php?id=1&action=edit - it returns nothing in View Source as well. 

Having said that, and after editing the databases by hand (because it looks like I had the port number to the pool AND my login credentials entered incorrectly), I got further with the miner now (I'm guessing before it would not connect because the credentials at the pool were incorrect.  Here's what happens now: 

[06/05/2011 03:18:17] Connected to server
[06/05/2011 03:18:23] Disconnected from server
[06/05/2011 03:18:23] Result: 5b433c62 rejected
[06/05/2011 03:18:24] Connected to server
[06/05/2011 03:18:26] Disconnected from server
[06/05/2011 03:18:26] Result: 9e17d0db rejected
[06/05/2011 03:18:27] Connected to server
[06/05/2011 03:18:31] Disconnected from server

My first instinct was to turn on keepalives in the webserver, I did so and set them to allow an unlimited number with a 30 second timeout, but it only prolonged the generation of the same errors by 10 or 20 seconds.  In the end it still disconnected my client from the server.  I tried m0mchil's miner with similar results:

06/05/2011 03:25:22, Unexpected error:
Traceback (most recent call last):
  File "BitcoinMiner.pyo", line 141, in mine
  File "BitcoinMiner.pyo", line 168, in sendResult
  File "BitcoinMiner.pyo", line 177, in getwork
  File "BitcoinMiner.pyo", line 204, in request
TypeError: string indices must be integers
248779 khash/s

This after running 10 to 20 seconds also.  Not seeing anything in the logs that would give me any indication of the problem.  Without cranking up a sniffer, I'm out of ideas; any suggestions?
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 06, 2011, 11:25:36 AM
 #24

When I connect Phoenix, it tells me 'failed to connect, retrying'.  When I examine my apache error log, I see this:

[Fri May 06 01:30:05 2011] [error] [client xxx.113] PHP Notice:  Undefined index:  lpurl in /home/rrb/bitproxy/index.php on line 70
This notice is expected.  PHP likes to whine if you access array keys that aren't defined, but it's impractical to always check if they're set first.  Smiley

[Fri May 06 01:30:05 2011] [error] [client xxx.113] PHP Notice:  Undefined variable: http_response_header in /home/rrb/bitproxy/common.inc.php on line 117
This is a bit troubling, as $http_response_header is a reserved variable.  Which version of PHP are you using?  Did you make sure to enter "http://" at the beginning of each pool URL?  (Something like "deepbit.net:8332" will not work; you need "http://deepbit.net:8332".)  You may also get this error if the HTTP host can't be reached at all, for example due to a DNS lookup failure.

[Fri May 06 01:30:05 2011] [error] [client xxx.113] PHP Warning:  Invalid argument supplied for foreach() in /home/rrb/bitproxy/index.php on line 215
This is related to the previous issue; fixing it should make this one go away.

Additionally, you might find this of interest.  When I hit the admin url, the admin interface seems to work fine but merely hitting /admin/ throws the following into the apache error log:

[Fri May 06 01:31:15 2011] [error] [client xxx.252] PHP Notice:  Undefined index:  format in /home/rrb/bitproxy/mvc.inc.php on line 38
[Fri May 06 01:31:15 2011] [error] [client xxx.252] PHP Notice:  Undefined index:  tempdata in /home/rrb/bitproxy/common.inc.php on line 177
[Fri May 06 01:31:15 2011] [error] [client xxx.252] PHP Notice:  Undefined index:  tempdata in /home/rrb/bitproxy/common.inc.php on line 177
Yes, this is PHP being picky again.  You can ignore these.  Smiley

1) In your database schema, in the worker_pool table, the type for the 'enabled' field is a tinyint and should be boolean it looks like, unless you have some special state info to store that's not 0 or 1?
MySQL does not have a real boolean type; BOOLEAN is an alias for TINYINT(1).

2) I discovered #1 while attempting to go back and edit the workers and pool info that I'd setup the first time, in an effort to debug what was happening with the behavior I reported above.  While using the various "edit" dialogs, I'm hitting blank pages being returned, i.e. in the "edit pool" dialog at http://xxx/admin/pool.php?id=1&action=edit - it returns nothing in View Source as well.
Hmm, this is almost certainly a bug.  Can you edit other pools, or is it just this one?  Does anything show up in the Apache logs when you hit this page?

Having said that, and after editing the databases by hand (because it looks like I had the port number to the pool AND my login credentials entered incorrectly), I got further with the miner now (I'm guessing before it would not connect because the credentials at the pool were incorrect.  Here's what happens now: 

[06/05/2011 03:18:17] Connected to server
[06/05/2011 03:18:23] Disconnected from server
[06/05/2011 03:18:23] Result: 5b433c62 rejected
[06/05/2011 03:18:24] Connected to server
[06/05/2011 03:18:26] Disconnected from server
[06/05/2011 03:18:26] Result: 9e17d0db rejected
[06/05/2011 03:18:27] Connected to server
[06/05/2011 03:18:31] Disconnected from server

My first instinct was to turn on keepalives in the webserver, I did so and set them to allow an unlimited number with a 30 second timeout, but it only prolonged the generation of the same errors by 10 or 20 seconds.  In the end it still disconnected my client from the server.  I tried m0mchil's miner with similar results:

06/05/2011 03:25:22, Unexpected error:
Traceback (most recent call last):
  File "BitcoinMiner.pyo", line 141, in mine
  File "BitcoinMiner.pyo", line 168, in sendResult
  File "BitcoinMiner.pyo", line 177, in getwork
  File "BitcoinMiner.pyo", line 204, in request
TypeError: string indices must be integers
248779 khash/s

This after running 10 to 20 seconds also.  Not seeing anything in the logs that would give me any indication of the problem.  Without cranking up a sniffer, I'm out of ideas; any suggestions?
This is definitely weird.  I use poclbm without any issues, so you should be able to use it too.  I'm mostly worried about the rejected shares in the first example.  Were all shares rejected or just those two?

Using a sniffer might be a good idea, since it's hard to tell just from these logs exactly what's going wrong.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 06, 2011, 11:30:30 AM
 #25

i only get an xml-parsing error when trying to access /admin/
Code:
XML Parsing Error: XML or text declaration not at start of entity
Location: http://blah/admin/
Line Number 2, Column 1:<?xml version="1.0" encoding="UTF-8"?>

guess it's just a blank line hiding somewhere, but i can't find it.
Yeah, there's definitely a blank line being output prior to the XML header.  I don't see the same behavior on my install, in fact the dashboard validates as XHTML strict (as should every other page, but I just checked the dashboard right now and it validates).  I can't find anywhere in the code where the blank line would be getting written.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 06, 2011, 11:33:50 PM
 #26

Thanks for the replies.  I should start by saying that I think those errors I was seeing in the errorlog were related to my incorrect entering of the parameters for deepbit (username/password, and the port number).  Although I did have it entered in the correct format as you mentioned.  In a future version you might want to do some more extensive error handling to let the user know in the admin panel that the "last result" from the pool is that it can't connect because of x or y, and the actual error sent back from the pool was z.

I sniffed out the communication going from Phoenix to the proxy only, and this is what it looks like (including timestamps above):

1) Initial connection:

19:14:20.136363
POST / HTTP/1.1
Connection: close
Content-Length: 44
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": [], "id": 1}

2) One more (not sure why this was necessary):

19:14:20.693905
POST / HTTP/1.1
Connection: close
Content-Length: 44
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": [], "id": 1}

3) Another for long polling?:

19:14:20.717374
GET /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=1 HTTP/1.1
Connection: close
Host: bitproxy.xxx.com
Authorization: Basic xxx
User-Agent: phoenix/v1.2

4) And one more.. not sure why this was necessary, the client didn't say it had seen a new block:

19:14:21.213894
POST / HTTP/1.1
Connection: close
Content-Length: 302
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": [], "id": 1}

5) Ahh now we're getting somewhere....14 seconds later when we have some work done:

19:14:35.187166
POST / HTTP/1.1
Connection: close
Content-Length: 302
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": ["00000001856410222014682e1d4361839ddae0adf1f7415584a4e6d8000006d20000000086ed4af 00f553972b297b1cdbd24d6b99e4720df26ff41229dbd2384c1d0cab44dc480c81b0098fa35cafe bd00000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000"], "id": 1}

At this point, the client disconnects from the proxy and throws the following in the client:

[06/05/2011 19:15:02] Disconnected from server
[06/05/2011 19:15:02] Result: a8b4a195 rejected

Then immediately reconnects to the server:

[06/05/2011 19:15:03] Connected to server

And sends the same message as in 1) above.  I should also note that the apache logs are consistent with the above and I do not see any signs of an error related to these http transactions in the error log.  Hope that helps.
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 07, 2011, 01:08:36 AM
 #27

In a future version you might want to do some more extensive error handling to let the user know in the admin panel that the "last result" from the pool is that it can't connect because of x or y, and the actual error sent back from the pool was z.
Yes, better validation and logging are on the cards.  I was more concerned with getting the code ready for a proper release than adding features, so this should be coming pretty soon.

I sniffed out the communication going from Phoenix to the proxy only, and this is what it looks like (including timestamps above):
Can you post some more logs, but include the replies from the proxy?  It's hard to diagnose problems when I can only see one half of the conversation.

Authorization: Basic xxx
If I were you I would sanitize these headers too; your username and password are exposed to anyone who understands HTTP basic authentication.

3) Another for long polling?:
Yes, this is a long polling request.

4) And one more.. not sure why this was necessary, the client didn't say it had seen a new block:
It's possible that your client is pre-fetching the next chunk of work so that when it's done with the current chunk it can move on to the next right away.

5) Ahh now we're getting somewhere....14 seconds later when we have some work done:

...

At this point, the client disconnects from the proxy and throws the following in the client:

[06/05/2011 19:15:02] Disconnected from server
[06/05/2011 19:15:02] Result: a8b4a195 rejected
This is where response logs from the proxy would be most helpful, since I can't really tell why it's failing without knowing what the proxy is saying in response.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 07, 2011, 02:05:27 AM
 #28

Thanks for the reply, I've sanitized things further in the last post, even though the worker password is of no consequence for me.  Here is the work being sent and the corresponding error:

POST / HTTP/1.1
Connection: close
Content-Length: 302
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": ["0000000129d09b9af4d37ae0641aeaf3618fb62692477035e4f4a17300009697000000004a84ffa 3bb5f69b95dd4656b1267f99f0675fca59a2899b31a07f3eb31992d8e4dc4a7561b0098fa7958ce a500000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000"], "id": 1}

Date: Sat, 07 May 2011 01:59:14 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=6r9lc3ejja26ok9fv5ve84dc44; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Length: 66
Connection: close
Content-Type: application/json-rpc

{"error":"Work not found in proxy database.","result":null,"id":1}
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 07, 2011, 02:53:36 AM
 #29

Here is the work being sent and the corresponding error:
Hmm.  Can you post the replies to the previous getworks too?  Somehow the work either isn't getting saved in the database correctly, or the worker isn't sending the work back correctly, or something.  I'd have to see the getwork responses and the work submission requests to diagnose this further.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 07, 2011, 06:33:15 AM
 #30

Here is a complete exchange including both sides from tcpdump.  The IPs were masked with "xxx" on the first 3 octets as you'll see:


Code:
# tcpdump -n -s 0 -A dst port 80 or src port 80 > /tmp/dump.txt
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes

02:21:27.632988 IP xxx.113.52908 > xxx.170.http: S 3391654312:3391654312(0) win 8192 <mss 1460,nop,wscale 8,nop,nop,sackOK>
E..4..@.~.d.D..qD......P.(........ ..S..............
02:21:27.633083 IP xxx.170.http > xxx.113.52908: S 713576223:713576223(0) ack 3391654313 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 4>
E..4..@.@...D...D..q.P..*.O..(......0...............
02:21:27.633239 IP xxx.113.52908 > xxx.170.http: . ack 1 win 256
E..(..@.~.d.D..qD......P.(..*.O P....n..
02:21:27.633921 IP xxx.113.52908 > xxx.170.http: P 1:230(229) ack 1 win 256
E..
..@.~.d D..qD......P.(..*.O P....i..POST / HTTP/1.1
Connection: close
Content-Length: 44
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": [], "id": 1}
02:21:27.633942 IP xxx.170.http > xxx.113.52908: . ack 230 win 432
E..(.p@.@..pD...D..q.P..*.O .(..P.......
02:21:28.175445 IP xxx.170.http > xxx.113.52908: P 1:1133(1132) ack 230 win 432
E....q@.@...D...D..q.P..*.O .(..P....u..HTTP/1.1 200 OK
Date: Sat, 07 May 2011 06:21:27 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=ag0fmlipr94a1n4roi99qrs232; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Long-Polling: /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=1
Content-Length: 596
Connection: close
Content-Type: application/json-rpc

{"error":null,"result":{"midstate":"c409bb59cd614fb25f69b942af4e8f20fb512572dff7bf7e679085671bbd7f03","data":"000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e000000000c431765aafaef90465a4ce8ddad3b4e89ca4a618e205acd5a841ea348a09eeb4dc4e4e31b0098fa00000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000","hash1":"00000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000010000","target":"ffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000"},"id":"json"}
02:21:28.175565 IP xxx.170.http > xxx.113.52908: F 1133:1133(0) ack 230 win 432
E..(.r@.@..nD...D..q.P..*.S..(..P....l..
02:21:28.175875 IP xxx.113.52908 > xxx.170.http: . ack 1134 win 252
E..(..@.~.d.D..qD......P.(..*.S.P.... ..
02:21:28.192774 IP xxx.113.52908 > xxx.170.http: F 230:230(0) ack 1134 win 252
E..(..@.~.d.D..qD......P.(..*.S.P.......
02:21:28.192782 IP xxx.170.http > xxx.113.52908: . ack 231 win 432
E..(.s@.@..mD...D..q.P..*.S..(..P....k..
02:21:28.194594 IP xxx.113.52909 > xxx.170.http: S 2285988338:2285988338(0) win 8192 <mss 1460,nop,wscale 8,nop,nop,sackOK>
E..4..@.~.d.D..qD......P.Ai....... .................
02:21:28.194608 IP xxx.170.http > xxx.113.52909: S 703577298:703577298(0) ack 2285988339 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 4>
E..4..@.@...D...D..q.P..)....Ai.....%Q..............
02:21:28.194895 IP xxx.113.52909 > xxx.170.http: . ack 1 win 256
E..(..@.~.d.D..qD......P.Ai.)...P...{...
02:21:28.195439 IP xxx.113.52909 > xxx.170.http: P 1:230(229) ack 1 win 256
E..
..@.~.d.D..qD......P.Ai.)...P...v...POST / HTTP/1.1
Connection: close
Content-Length: 44
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": [], "id": 1}
02:21:28.195454 IP xxx.170.http > xxx.113.52909: . ack 230 win 432
E..(..@.@...D...D..q.P..)....Aj.P...z[..
02:21:28.199436 IP xxx.113.52910 > xxx.170.http: S 2903052052:2903052052(0) win 8192 <mss 1460,nop,wscale 8,nop,nop,sackOK>
E..4..@.~.d.D..qD......P.       ........ .9...............
02:21:28.199449 IP xxx.170.http > xxx.113.52910: S 709956609:709956609(0) ack 2903052053 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 4>
E..4..@.@...D...D..q.P..*Q...   ......................
02:21:28.199742 IP xxx.113.52910 > xxx.170.http: . ack 1 win 256
E..(..@.~.d.D..qD......P.       ..*Q..P...Zu..
02:21:28.200281 IP xxx.113.52910 > xxx.170.http: P 1:203(202) ack 1 win 256
E.....@.~.d.D..qD......P.       ..*Q..P...O...GET /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=1 HTTP/1.1
Connection: close
Host: bitproxy.xxx.com
Authorization: Basic xxx
User-Agent: phoenix/v1.2


02:21:28.200298 IP xxx.170.http > xxx.113.52910: . ack 203 win 432
E..(..@.@...D...D..q.P..*Q...   ..P...X...
02:21:28.687472 IP xxx.170.http > xxx.113.52909: P 1:1133(1132) ack 230 win 432
E.....@.@..eD...D..q.P..)....Aj.P....u..HTTP/1.1 200 OK
Date: Sat, 07 May 2011 06:21:28 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=opqid2ulnmonksj5qkdo6thuj4; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Long-Polling: /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=1
Content-Length: 596
Connection: close
Content-Type: application/json-rpc

{"error":null,"result":{"midstate":"5ecbfe50f947ebfbd67ff0c71e54658c17f517d965c763d86ac3868a66a2aae7","data":"000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e00000000eee1830529617e61922c5268219db6a18b6f2f8ec6f0b06f5a1d894705f88e2b4dc4e4e31b0098fa00000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000","hash1":"00000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000010000","target":"ffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000"},"id":"json"}
02:21:28.687592 IP xxx.170.http > xxx.113.52909: F 1133:1133(0) ack 230 win 432
E..(..@.@...D...D..q.P..)..?.Aj.P...u...
02:21:28.687907 IP xxx.113.52909 > xxx.170.http: . ack 1134 win 252
E..(
.@.~.d.D..qD......P.Aj.)..@P...v...
02:21:29.080529 IP xxx.113.52909 > xxx.170.http: F 230:230(0) ack 1134 win 252
E..(
.@.~.d.D..qD......P.Aj.)..@P...v...
02:21:29.080547 IP xxx.170.http > xxx.113.52909: . ack 231 win 432
E..(..@.@...D...D..q.P..)..@.Aj.P...u...
02:21:42.166553 IP xxx.113.52911 > xxx.170.http: S 3058214165:3058214165(0) win 8192 <mss 1460,nop,wscale 8,nop,nop,sackOK>
E..4
.@.~.d8D..qD......P.H........ .................
02:21:42.166588 IP xxx.170.http > xxx.113.52911: S 720814152:720814152(0) ack 3058214166 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 4>
E..4..@.@...D...D..q.P..*..H.H......................
02:21:42.166882 IP xxx.113.52911 > xxx.170.http: . ack 1 win 256
E..(
.@.~.dCD..qD......P.H..*..IP....G..
02:21:42.167610 IP xxx.113.52911 > xxx.170.http: P 1:230(229) ack 1 win 256
E..

.@.~.c]D..qD......P.H..*..IP... B..POST / HTTP/1.1
Connection: close
Content-Length: 44
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": [], "id": 1}
02:21:42.167631 IP xxx.170.http > xxx.113.52911: . ack 230 win 432
E..(}.@.@.2.D...D..q.P..*..I.H..P.......
02:21:42.666881 IP xxx.170.http > xxx.113.52911: P 1:1133(1132) ack 230 win 432
E...}.@.@.-.D...D..q.P..*..I.H..P....u..HTTP/1.1 200 OK
Date: Sat, 07 May 2011 06:21:42 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=htn9oirmvclsnrcjf9efdic5r4; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Long-Polling: /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=1
Content-Length: 596
Connection: close
Content-Type: application/json-rpc

{"error":null,"result":{"midstate":"fc9385dbdc6f066c4f06d7b720f0022cf38015b5895eed8f9517d4cc158cf8c6","data":"000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa00000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000","hash1":"00000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000010000","target":"ffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000"},"id":"json"}
02:21:42.667032 IP xxx.170.http > xxx.113.52911: F 1133:1133(0) ack 230 win 432
E..(}.@.@.2.D...D..q.P..*....H..P....E..
02:21:42.667198 IP xxx.113.52911 > xxx.170.http: . ack 1134 win 252
E..(
.@.~.d@D..qD......P.H..*...P.......
02:21:43.040604 IP xxx.113.52911 > xxx.170.http: F 230:230(0) ack 1134 win 252
E..(
.@.~.d=D..qD......P.H..*...P.......
02:21:43.040628 IP xxx.170.http > xxx.113.52911: . ack 231 win 432
E..(}.@.@.2.D...D..q.P..*....H..P....D..
02:21:56.115507 IP xxx.113.52912 > xxx.170.http: S 3963094368:3963094368(0) win 8192 <mss 1460,nop,wscale 8,nop,nop,sackOK>
E..4
.@.~.d.D..qD......P.8.`...... .................
02:21:56.115536 IP xxx.170.http > xxx.113.52912: S 743911290:743911290(0) ack 3963094369 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 4>
E..4..@.@...D...D..q.P..,W/z.8.a....................
02:21:56.115842 IP xxx.113.52912 > xxx.170.http: . ack 1 win 256
E..(
.@.~.d.D..qD......P.8.a,W/{P....y..
02:21:56.118465 IP xxx.113.52912 > xxx.170.http: P 1:230(229) ack 1 win 256
E..

.@.~.c*D..qD......P.8.a,W/{P....t..POST / HTTP/1.1
Connection: close
Content-Length: 44
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": [], "id": 1}
02:21:56.118487 IP xxx.170.http > xxx.113.52912: . ack 230 win 432
E..(.[@.@...D...D..q.P..,W/{.8.FP.......
02:21:56.630388 IP xxx.170.http > xxx.113.52912: P 1:1133(1132) ack 230 win 432
E....\@.@...D...D..q.P..,W/{.8.FP....u..HTTP/1.1 200 OK
Date: Sat, 07 May 2011 06:21:56 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=nvq8v0tcrit13dl2gaf3rp6vb0; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Long-Polling: /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=1
Content-Length: 596
Connection: close
Content-Type: application/json-rpc

{"error":null,"result":{"midstate":"8905e709a985e89297c0702f3cdff5f2389c5b0d9010e8ded39ac4c898b0464d","data":"000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e000000000e824081865a44bfa0d370807db026d218a820bc7571c5fda5bab44f85a85a5b4dc4e4ff1b0098fa00000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000","hash1":"00000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000010000","target":"ffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000"},"id":"json"}
02:21:56.630500 IP xxx.170.http > xxx.113.52912: F 1133:1133(0) ack 230 win 432
E..(.]@.@...D...D..q.P..,W3..8.FP....w..
02:21:56.630842 IP xxx.113.52912 > xxx.170.http: . ack 1134 win 252
E..(
.@.~.d.D..qD......P.8.F,W3.P....+..
02:21:56.991797 IP xxx.113.52912 > xxx.170.http: F 230:230(0) ack 1134 win 252
E..(
.@.~.d  D..qD......P.8.F,W3.P....*..
02:21:56.991819 IP xxx.170.http > xxx.113.52912: . ack 231 win 432
E..(.^@.@...D...D..q.P..,W3..8.GP....v..
02:22:01.355768 IP xxx.113.52913 > xxx.170.http: S 2483722273:2483722273(0) win 8192 <mss 1460,nop,wscale 8,nop,nop,sackOK>
E..4
.@.~.c.D..qD......P.
.!...... .................
02:22:01.355795 IP xxx.170.http > xxx.113.52913: S 744601149:744601149(0) ack 2483722274 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 4>
E..4..@.@...D...D..q.P..,a.=.
.".....w..............
02:22:01.355981 IP xxx.113.52913 > xxx.170.http: . ack 1 win 256
E..(
.@.~.c.D..qD......P.
.",a.>P...F...
02:22:01.357701 IP xxx.113.52913 > xxx.170.http: P 1:489(488) ack 1 win 256
E...
.@.~.b.D..qD......P.
.",a.>P.......POST / HTTP/1.1
Connection: close
Content-Length: 302
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": ["000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fafb878b5c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"], "id": 1}
02:22:01.357723 IP xxx.170.http > xxx.113.52913: . ack 489 win 432
E..(f.@.@.I.D...D..q.P..,a.>.
.
P...C...
02:22:01.368776 IP xxx.170.http > xxx.113.52913: P 1:513(512) ack 489 win 432
E..(f.@.@.G.D...D..q.P..,a.>.
.
P....   ..HTTP/1.1 200 OK
Date: Sat, 07 May 2011 06:22:01 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=jjnaeqibh1069oa4e750uov3v1; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Length: 66
Connection: close
Content-Type: application/json-rpc

{"error":"Work not found in proxy database.","result":null,"id":1}
02:22:01.368886 IP xxx.170.http > xxx.113.52913: F 513:513(0) ack 489 win 432
E..(f.@.@.I.D...D..q.P..,a.>.
.
P...A~..
02:22:01.369164 IP xxx.113.52913 > xxx.170.http: . ack 514 win 254
E..(
.@.~.c.D..qD......P.
.
,a.?P...B0..
02:22:01.800364 IP xxx.113.52913 > xxx.170.http: F 489:489(0) ack 514 win 254
E..(
.@.~.c.D..qD......P.
.
,a.?P...B/..
02:22:01.800388 IP xxx.170.http > xxx.113.52913: . ack 490 win 432
E..(f.@.@.I.D...D..q.P..,a.?.
..P...A}..
02:22:02.236887 IP xxx.113.52914 > xxx.170.http: S 3434189684:3434189684(0) win 8192 <mss 1460,nop,wscale 8,nop,nop,sackOK>
E..4
.@.~.c.D..qD......P...t...... .................
02:22:02.236914 IP xxx.170.http > xxx.113.52914: S 742812174:742812174(0) ack 3434189685 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 4>
E..4..@.@...D...D..q.P..,Fj....u....................
02:22:02.237217 IP xxx.113.52914 > xxx.170.http: . ack 1 win 256
E..(
.@.~.c.D..qD......P...u,Fj.P...^f..
02:22:02.237772 IP xxx.113.52914 > xxx.170.http: P 1:230(229) ack 1 win 256
E..

.@.~.b.D..qD......P...u,Fj.P...Ya..POST / HTTP/1.1
Connection: close
Content-Length: 44
Host: bitproxy.xxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": [], "id": 1}
02:22:02.237787 IP xxx.170.http > xxx.113.52914: . ack 230 win 432
E..(..@.@...D...D..q.P..,Fj....ZP...\...
02:22:02.726166 IP xxx.170.http > xxx.113.52914: P 1:1133(1132) ack 230 win 432
E.....@.@..^D...D..q.P..,Fj....ZP....u..HTTP/1.1 200 OK
Date: Sat, 07 May 2011 06:22:02 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=0ecg4k48echds2qcnhfs1tr8q3; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Long-Polling: /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=1
Content-Length: 596
Connection: close
Content-Type: application/json-rpc

{"error":null,"result":{"midstate":"8a305c3e93669380e4075f4dcd8785990b60953362b1bf9b2e825aa5b287571f","data":"000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e00000000787bdec7bc46a3ebeca98e8d7f74ced0eb66583fb1ded4b164d0bc18b502f9164dc4e5051b0098fa00000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000","hash1":"00000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000010000","target":"ffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000"},"id":"json"}
02:22:02.726319 IP xxx.170.http > xxx.113.52914: F 1133:1133(0) ack 230 win 432
E..(..@.@...D...D..q.P..,Fn{...ZP...Xd..
02:22:02.726488 IP xxx.113.52914 > xxx.170.http: . ack 1134 win 252
E..(..@.~.c.D..qD......P...Z,Fn|P...Y...
02:22:03.114950 IP xxx.113.52910 > xxx.170.http: F 203:203(0) ack 1 win 256
E..(..@.~.c.D..qD......P.       ..*Q..P...Y...
02:22:03.115170 IP xxx.113.52914 > xxx.170.http: F 230:230(0) ack 1134 win 252
E..(..@.~.c.D..qD......P...Z,Fn|P...Y...
02:22:03.115182 IP xxx.170.http > xxx.113.52914: . ack 231 win 432
E..(..@.@...D...D..q.P..,Fn|...[P...Xc..
02:22:03.154389 IP xxx.170.http > xxx.113.52910: . ack 204 win 432
E..(..@.@...D...D..q.P..*Q...   ..P...X...
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 07, 2011, 05:44:41 PM
 #31

Here is a complete exchange including both sides from tcpdump.  The IPs were masked with "xxx" on the first 3 octets as you'll see:
Perfect, that's exactly the data I needed.

The error you are getting back from the work submission should indeed not be happening; the submitted work does match one of the getwork requests.  Can you execute this query against your database and see if it gets any results?  If it doesn't, can you verify that your work_data table has any records in it at all?

Code:
SELECT * FROM `work_data` WHERE `data` = '000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa';

Edit: I just checked the code and if the INSERT into the work_data table fails, it will do so silently, still returning the work to the client.  I need to patch this, since the client shouldn't be given work that can't be registered, or the proxy won't know which pool to route it too (I think this is the problem you are seeing).  We still need to figure out why the INSERT is failing though.

Edit 2: I think I found the problem with work submissions.  Please update to the latest code from Git.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 07, 2011, 11:44:26 PM
 #32

My work_data table had 192 entries.  I cloned the tree again and replaced the contents of htdocs, copying over the same config file, and the error is still the same on the client side.  Let me know if there is anything else I can do to help troubleshoot.
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 07, 2011, 11:47:56 PM
 #33

My work_data table had 192 entries.  I cloned the tree again and replaced the contents of htdocs, copying over the same config file, and the error is still the same on the client side.  Let me know if there is anything else I can do to help troubleshoot.
Hmm.  Can you run the query I gave and see what it returns?  Also, you shouldn't need to clone anything again -- just git pull from inside the git root.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 08, 2011, 12:21:12 AM
 #34

Yes, it's in there.

mysql> SELECT * FROM `work_data` WHERE `data` = '000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9 a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa';
+-----------+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+
| worker_id | pool_id | data                                                                                                                                                     | time_requested      |
+-----------+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+
|         1 |       1 | 000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9 a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa | 2011-05-07 06:21:42 |
+-----------+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+
1 row in set (0.00 sec)

mysql>
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 08, 2011, 04:25:54 AM
 #35

Yes, it's in there.
Hmm, weird.  Can you put the contents of this script in a PHP file alongside index.php (in other words, directly in the web root) and paste me its output?  Of course, sanitize the username and password if they get displayed.

Code:
<?php

require_once(dirname(__FILE__) . '/common.inc.php');

header('Content-Type: text/plain');

$pdo db_connect();

$q $pdo->prepare('
    SELECT
        p.id AS pool_id,
        wp.pool_username AS username,
        wp.pool_password AS password,
        p.url AS url

    FROM
        work_data d,
        worker_pool wp,
        pool p

    WHERE d.data = :data
      AND d.worker_id = :worker_id

      AND d.pool_id = p.id

      AND wp.worker_id = :worker_id
      AND wp.pool_id = p.id
'
);

if (
$q === FALSE) {
    echo 
"Unable to prepare statement: ";
    
var_export($pdo->errorInfo());
    exit;
}

$result $q->execute(array(
    
':worker_id'    => 1,
    
':data'         => '000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa'));

if (!
$result) {
    echo 
"Unable to execute statement: ";
    
var_export($q->errorInfo());
    exit;
}

$row $q->fetch();
$q->closeCursor();

if (
$row === FALSE) {
    echo 
"No rows found.";
    exit;
}

echo 
"Row found: ";
var_export($row);

?>

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 08, 2011, 07:10:59 AM
 #36

That script returns "No rows found."  Preemptively I'm going to send you the contents of tables "worker", "worker_pool" and "pool" with passwords removed, since it looks like that's the information you're looking for with the aforementioned php script:

mysql> select * from worker;
+----+-------+-----------+
| id | name  | password  |
+----+-------+-----------+
|  1 | w0113 | xxxxx |
+----+-------+-----------+
1 row in set (0.00 sec)

mysql> select * from worker_pool
    -> ;
+---------+-----------+------------------------+---------------+----------+---------+
| pool_id | worker_id | pool_username          | pool_password | priority | enabled |
+---------+-----------+------------------------+---------------+----------+---------+
|       1 |         1 | xxxxxxxxx@xxxxx.com_10 | xxxxxxxxxxx     |        1 |       1 |
+---------+-----------+------------------------+---------------+----------+---------+
1 row in set (0.00 sec)

mysql> select * from pool;
+----+---------+-------------------------+---------+
| id | name    | url                     | enabled |
+----+---------+-------------------------+---------+
|  1 | deepbit | http://deepbit.net:8332 |       1 |
+----+---------+-------------------------+---------+
1 row in set (0.00 sec)

Based on my lay knowledge of php, I can't imagine why it doesn't return anything from the script you sent.  Is it possible that my installation (default php 5.1 from CentOS 5.x with php-* installed, mysql-* installed, and the add-in json libraries) is different, misconfigured, or missing a dependency?  For reference, it already works fine with other popular MySQL/PHP off the shelf applications.

Also sent you 5 BTC to the address in your signature for all the work you're putting in; this is in addition to my previous commitment to send you 10 when it appears that this works and is viable.  Thanks again for your help.
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 08, 2011, 07:20:13 AM
 #37

One more that I meant to send:

mysql> select * from work_data where data = '000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9 a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa';
+-----------+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+
| worker_id | pool_id | data                                                                                                                                                     | time_requested      |
+-----------+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+
|         1 |       1 | 000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9 a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa | 2011-05-07 06:21:42 |
+-----------+---------+----------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------+
1 row in set (0.00 sec)

mysql>
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 08, 2011, 01:50:32 PM
 #38

That script returns "No rows found."

....

Based on my lay knowledge of php, I can't imagine why it doesn't return anything from the script you sent.
#$%&.  Me either... though I do have a few ideas...

Is it possible that my installation (default php 5.1 from CentOS 5.x with php-* installed, mysql-* installed, and the add-in json libraries) is different, misconfigured, or missing a dependency?  For reference, it already works fine with other popular MySQL/PHP off the shelf applications.
It should work fine.  I'm on PHP 5.3.3, but all my code should be compatible with most PHP5 versions (and probably some PHP4 versions too).

To further diagnose this, can you try this script now?

Code:
<?php

require_once(dirname(__FILE__) . '/common.inc.php');

header('Content-Type: text/plain');

$pdo db_connect();

function 
try_query($sql$args) {
    global 
$pdo;

    
$q $pdo->prepare($sql);

    if (
$q === FALSE) {
        return 
"Could not prepare statement (" var_export($pdo->errorInfo(), true) . ")";
    }

    
$result $q->execute($args);

    if (!
$result) {
        return 
"Could not execute statement (" var_export($q->errorInfo(), true) . ")";
    }

    
$row $q->fetch();

    if (
$row === FALSE) {
        return 
"No rows found";
    }

    
$q->closeCursor();

    return 
"Success (" var_export($rowtrue) . ")";
}

$queries = array(
    array(
'name' => 'original',
          
'args' => array(':worker_id'    => 1,
                          
':data'         => '000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa'),
          
'query' => '
                SELECT
                    p.id AS pool_id,
                    wp.pool_username AS username,
                    wp.pool_password AS password,
                    p.url AS url

                FROM
                    work_data d,
                    worker_pool wp,
                    pool p

                WHERE d.data = :data
                  AND d.worker_id = :worker_id

                  AND d.pool_id = p.id

                  AND wp.worker_id = :worker_id
                  AND wp.pool_id = p.id
          '
),
    array(
'name' => 'hardcoded',
          
'args' => array(),
          
'query' => "
                SELECT
                    p.id AS pool_id,
                    wp.pool_username AS username,
                    wp.pool_password AS password,
                    p.url AS url

                FROM
                    work_data d,
                    worker_pool wp,
                    pool p

                WHERE d.data = '000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa'
                  AND d.worker_id = 1

                  AND d.pool_id = p.id

                  AND wp.worker_id = 1
                  AND wp.pool_id = p.id
          "
),
    array(
'name' => 'nodup',
          
'args' => array(':worker_id'     => 1,
                          
':worker_id_two' => 1,
                          
':data'          => '000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa'),
          
'query' => '
                SELECT
                    p.id AS pool_id,
                    wp.pool_username AS username,
                    wp.pool_password AS password,
                    p.url AS url

                FROM
                    work_data d,
                    worker_pool wp,
                    pool p

                WHERE d.data = :data
                  AND d.worker_id = :worker_id

                  AND d.pool_id = p.id

                  AND wp.worker_id = :worker_id_two
                  AND wp.pool_id = p.id
          '
),
    array(
'name' => 'no underscore',
          
'args' => array(':workerid'    => 1,
                          
':data'         => '000000015bb9a0f983558324beead35e719c4e1e4f3e3d0a8f6550bb00003a5e0000000035cfac9a5865f1e5500714aa46e327a3727fd5a481f0b708820f794f6e0dada84dc4e4f11b0098fa'),
          
'query' => '
                SELECT
                    p.id AS pool_id,
                    wp.pool_username AS username,
                    wp.pool_password AS password,
                    p.url AS url

                FROM
                    work_data d,
                    worker_pool wp,
                    pool p

                WHERE d.data = :data
                  AND d.worker_id = :workerid

                  AND d.pool_id = p.id

                  AND wp.worker_id = :workerid
                  AND wp.pool_id = p.id
          '
)
);

foreach (
$queries as $query) {
    echo 
$query['name'];
    echo 
': ';
    echo 
try_query($query['query'], $query['args']);
    echo 
"\n";
}

?>

Also sent you 5 BTC to the address in your signature for all the work you're putting in; this is in addition to my previous commitment to send you 10 when it appears that this works and is viable.  Thanks again for your help.
Cool, thanks!  Hopefully we can get this working soon.  Obscure bugs are annoying.  Sad

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
Cdecker
Hero Member
*****
Offline Offline

Activity: 487



View Profile WWW
May 08, 2011, 05:18:39 PM
 #39

I love this one. It make switching really painless, especially after todays outage of slush's pool this proved a timesaver. What I noticed however that the failing pool will still be asked for work, and only he doesn't return any, the next one will be asked. Are there plans for exponential backoff in order to avoid the extra request and the extra idle time? Just a mechanism to automatically blacklist a pool should it fail, and increase the blacklist time on each subsequent failure would be cool.
The problem today is that slush sometimes gives me work, but does not accept the result (timeout/502 errors/...).

Also I get a lot of these:
Quote
192.168.3.5 - pilum [08/May/2011:19:21:43 +0200] "POST /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=2 HTTP/1.1" 200 1181 "-" "Java/1.6.0_22"
192.168.3.5 - pilum [08/May/2011:19:21:45 +0200] "POST /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=2 HTTP/1.1" 200 1181 "-" "Java/1.6.0_22"
192.168.3.5 - pilum [08/May/2011:19:21:47 +0200] "POST /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=2 HTTP/1.1" 200 1181 "-" "Java/1.6.0_22"
192.168.3.5 - pilum [08/May/2011:19:21:49 +0200] "POST /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=2 HTTP/1.1" 200 1181 "-" "Java/1.6.0_22"
192.168.3.5 - pilum [08/May/2011:19:21:49 +0200] "POST /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=2 HTTP/1.1" 200 1181 "-" "Java/1.6.0_22"
Which makes me think that the diablo miner does not really take advantage of the long polling feature, and I start feeling bad for bombing pools with all those long poll requests. Any ideas?

Want to see what developers are chatting about? http://bitcoinstats.com/irc/bitcoin-dev/logs/
Bitcoin-OTC Rating
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 08, 2011, 07:17:20 PM
 #40


To further diagnose this, can you try this script now?

This is the result, sanitized to remove portions of my email/password at deepbit:

original: No rows found
hardcoded: Success (array (
  'pool_id' => '1',
  0 => '1',
  'username' => 'xxx@xxx.com_10',
  1 => 'xxx@xxx.com_10',
  'password' => 'xxx',
  2 => 'xxx',
  'url' => 'http://deepbit.net:8332',
  3 => 'http://deepbit.net:8332',
))
nodup: Success (array (
  'pool_id' => '1',
  0 => '1',
  'username' => 'xxx@xxx.com_10',
  1 => 'xxx@xxx.com_10',
  'password' => 'xxx',
  2 => 'xxx',
  'url' => 'http://deepbit.net:8332',
  3 => 'http://deepbit.net:8332',
))
no underscore: No rows found
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 08, 2011, 07:54:12 PM
 #41

This is the result, sanitized to remove portions of my email/password at deepbit:
Thanks, that's all I need to know to fix the problem.  Apparently the older PDO library doesn't support using the same query argument twice in a query.  A fix is forthcoming.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 08, 2011, 08:04:18 PM
 #42

This is the result, sanitized to remove portions of my email/password at deepbit:
Thanks, that's all I need to know to fix the problem.  Apparently the older PDO library doesn't support using the same query argument twice in a query.  A fix is forthcoming.
Fix just got pushed to the Git repository; please pull and see if the problem goes away.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 08, 2011, 08:15:49 PM
 #43

I love this one. It make switching really painless, especially after todays outage of slush's pool this proved a timesaver.
I'm glad you like it. Smiley

What I noticed however that the failing pool will still be asked for work, and only he doesn't return any, the next one will be asked. Are there plans for exponential backoff in order to avoid the extra request and the extra idle time? Just a mechanism to automatically blacklist a pool should it fail, and increase the blacklist time on each subsequent failure would be cool.
This is a good idea.  I've added it to the issue tracker.

The problem today is that slush sometimes gives me work, but does not accept the result (timeout/502 errors/...).
Hmm, I see.  I'll try to implement something so that if a pool doesn't respond normally to a share submission (e.g. it doesn't accept or reject the share, but gives some other HTTP or JSON-level error) the software will retry a few times, and disabled the pool for some time if it still can't reach the server.  This will depend on the other feature being implemented first.  (This one has also been added to the issue tracker.)

Also I get a lot of these:
Quote
192.168.3.5 - pilum [08/May/2011:19:21:43 +0200] "POST /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=2 HTTP/1.1" 200 1181 "-" "Java/1.6.0_22"
192.168.3.5 - pilum [08/May/2011:19:21:45 +0200] "POST /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=2 HTTP/1.1" 200 1181 "-" "Java/1.6.0_22"
192.168.3.5 - pilum [08/May/2011:19:21:47 +0200] "POST /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=2 HTTP/1.1" 200 1181 "-" "Java/1.6.0_22"
192.168.3.5 - pilum [08/May/2011:19:21:49 +0200] "POST /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=2 HTTP/1.1" 200 1181 "-" "Java/1.6.0_22"
192.168.3.5 - pilum [08/May/2011:19:21:49 +0200] "POST /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=2 HTTP/1.1" 200 1181 "-" "Java/1.6.0_22"
Which makes me think that the diablo miner does not really take advantage of the long polling feature, and I start feeling bad for bombing pools with all those long poll requests. Any ideas?
Hmm, that is definitely weird.  Can you sniff those requests and paste the HTTP conversation (sanitizing authentication details) so that I can see what's going on?

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 09, 2011, 12:32:27 AM
 #44

Quote from: cdhowie
Fix just got pushed to the Git repository; please pull and see if the problem goes away.

Thanks, that gets connected and accepting work now, but after maybe 3 minutes of mining, I see a random disconnect:

[08/05/2011 20:27:46] Connected to server
[08/05/2011 20:28:30] Result: 9f0a0a6e accepted
[08/05/2011 20:28:31] Result: 6731f6a4 accepted
[08/05/2011 20:28:55] Result: a0527e09 accepted
[08/05/2011 20:28:59] Result: 196f2c04 accepted
[08/05/2011 20:29:17] Result: 65173661 accepted
[08/05/2011 20:29:21] Result: 564ba46a accepted
[08/05/2011 20:29:47] Disconnected from server
[08/05/2011 20:29:48] Connected to server
[08/05/2011 20:30:05] Result: 40071f20 accepted
[08/05/2011 20:30:24] Result: a6206ab4 accepted
[08/05/2011 20:30:33] Result: 73d3021d accepted
[08/05/2011 20:30:34] Disconnected from server
[307.16 Mhash/sec] [9 Accepted] [0 Rejected] [RPC (+LP)]

Let me know how I can help diagnose this.
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 09, 2011, 03:03:54 AM
 #45

Thanks, that gets connected and accepting work now, but after maybe 3 minutes of mining, I see a random disconnect:
The long-polling request is probably getting terminated.  This should only happen if you have PHP's safe mode enabled -- set_time_limit() has no effect when running under safe mode, so the most likely cause here is that PHP is simply killing the script doing the long-polling request because it is running for too long.  (Note that the "disconnection" message comes almost exactly two minutes after the "connection" message.)

Try disabling safe mode.  Safe mode can only be disabled in php.ini or your Apache configuration; .htaccess files can not disable safe mode.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 09, 2011, 03:27:18 AM
 #46

Yes it's already disabled in php.ini:

;
; Safe Mode
;
safe_mode = Off

Though I think it would be a good idea to go back and mention as dependencies (in the INSTALL file) that safe_mode should be disabled, and that this (quite obviously) requires php json support, which is an add-on for PHP versions prior to 5.3.

Anyway this could very well have something to do with long polling.  I tried setting my max execution time (php.ini) to 1500 seconds, up from 30 seconds.  It did the following:

[08/05/2011 23:21:44] Connected to server
[08/05/2011 23:21:49] Result: be0189d3 accepted
[08/05/2011 23:21:54] Result: 62cffe98 accepted
[08/05/2011 23:21:58] Result: 65b66b0e accepted
[08/05/2011 23:22:31] Result: 2ca96088 accepted
[08/05/2011 23:22:38] Result: 8bd7c27e accepted
[08/05/2011 23:22:39] LP: New work pushed
[08/05/2011 23:22:47] Result: c662da7b accepted
[08/05/2011 23:23:20] Result: b9b3a17f accepted
[08/05/2011 23:23:30] Result: 889e14f2 accepted
[08/05/2011 23:23:31] Result: 1b237f90 accepted
[08/05/2011 23:23:38] Result: 9da72831 accepted
[08/05/2011 23:24:33] Result: ca84878c accepted
[08/05/2011 23:24:40] Disconnected from server
[08/05/2011 23:24:41] Connected to server
[08/05/2011 23:24:43] Result: 1b1964ce accepted
[08/05/2011 23:24:57] Result: 6b5df998 accepted
[08/05/2011 23:25:41] Result: 86510be4 accepted

Any other ideas?
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 09, 2011, 04:04:30 AM
 #47

Anyway this could very well have something to do with long polling.  I tried setting my max execution time (php.ini) to 1500 seconds, up from 30 seconds.  It did the following:

....

Any other ideas?
Hmm... run a packet sniffer and see what's going on with the LP request?

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
Cdecker
Hero Member
*****
Offline Offline

Activity: 487



View Profile WWW
May 09, 2011, 11:54:33 AM
 #48

Hmm, that is definitely weird.  Can you sniff those requests and paste the HTTP conversation (sanitizing authentication details) so that I can see what's going on?
Will do ^^

Want to see what developers are chatting about? http://bitcoinstats.com/irc/bitcoin-dev/logs/
Bitcoin-OTC Rating
Cdecker
Hero Member
*****
Offline Offline

Activity: 487



View Profile WWW
May 09, 2011, 12:32:44 PM
 #49

Will do ^^
I should have seen it immediately: Diablo uses POST request while well behaving clients (phoenix, poclbm, ...) use GET requests. Should be a simple matter of routing POST to the same handler Smiley
In the meantime I'm working on a hashrate estimate for the workers, which I'd find quite usefull. Do you accept patches/pull requests?

Want to see what developers are chatting about? http://bitcoinstats.com/irc/bitcoin-dev/logs/
Bitcoin-OTC Rating
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 09, 2011, 02:02:22 PM
 #50

I should have seen it immediately: Diablo uses POST request while well behaving clients (phoenix, poclbm, ...) use GET requests. Should be a simple matter of routing POST to the same handler Smiley
Ah, good catch.  I should have followed the "be lenient in what you accept" rule when coding...

In the meantime I'm working on a hashrate estimate for the workers, which I'd find quite usefull. Do you accept patches/pull requests?
Absolutely, as long as you release your patches under the AGPLv3.  Have you patched it to accept POST for long-polling requests, or should I fix that?

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
Cdecker
Hero Member
*****
Offline Offline

Activity: 487



View Profile WWW
May 09, 2011, 02:16:08 PM
 #51

I'm not really familiar with the framework you're using, so I started by implementing the hash speed for the workers. As for the routing of the post request, I guess you'd be faster than me :-)

Want to see what developers are chatting about? http://bitcoinstats.com/irc/bitcoin-dev/logs/
Bitcoin-OTC Rating
Cdecker
Hero Member
*****
Offline Offline

Activity: 487



View Profile WWW
May 09, 2011, 02:35:10 PM
 #52

Just issued a pull request for the hashing speed and shares in the last hour on the dashboard for each worker. Hope you like it ^^

Want to see what developers are chatting about? http://bitcoinstats.com/irc/bitcoin-dev/logs/
Bitcoin-OTC Rating
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 09, 2011, 02:39:21 PM
 #53

I'm not really familiar with the framework you're using, so I started by implementing the hash speed for the workers. As for the routing of the post request, I guess you'd be faster than me :-)
It's a custom lightweight MVC framework that doesn't do terribly fancy routing, but still allows me to separate my code better.  Smiley

I've pushed a fix to the diablo-lp-fix branch.  Can you try that and see if it fixes the issue for you?

Just issued a pull request for the hashing speed and shares in the last hour on the dashboard for each worker. Hope you like it ^^
Thanks!  I'll have a look later today.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
Cdecker
Hero Member
*****
Offline Offline

Activity: 487



View Profile WWW
May 09, 2011, 02:50:48 PM
 #54

Works like a charm, and it fixed my 100% problem with Diablo too ^^

Want to see what developers are chatting about? http://bitcoinstats.com/irc/bitcoin-dev/logs/
Bitcoin-OTC Rating
pharaon
Full Member
***
Offline Offline

Activity: 129


View Profile
May 09, 2011, 03:14:44 PM
 #55

How to set up a "no admin ask password" to test it
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 09, 2011, 03:22:20 PM
 #56

Works like a charm, and it fixed my 100% problem with Diablo too ^^
Neat, I'll merge the fix into master.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 09, 2011, 03:22:32 PM
 #57

How to set up a "no admin ask password" to test it
I'm not sure what you mean.  Can you elaborate?

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pharaon
Full Member
***
Offline Offline

Activity: 129


View Profile
May 09, 2011, 03:39:24 PM
 #58

How to set up a "no admin ask password" to test it
I'm not sure what you mean.  Can you elaborate?


yes sorry....

In your Install file you say:
2.  Create a MySQL user for the proxy, giving it the privileges SELECT, INSERT, UPDATE, DELETE, and LOCK TABLES on the proxy's database only.  (Optional, but recommended.)

Is it the same user than:

3.  Set the admin user and password.  This will be used when logging in to the web management console.


and an other question is, when i connect on the "/" of my proxy, it ask me a login and password.
Is it the same than in the admin section ?

sorry for my bad english
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 09, 2011, 03:59:08 PM
 #59

In your Install file you say:
2.  Create a MySQL user for the proxy, giving it the privileges SELECT, INSERT, UPDATE, DELETE, and LOCK TABLES on the proxy's database only.  (Optional, but recommended.)

Is it the same user than:

3.  Set the admin user and password.  This will be used when logging in to the web management console.
No.  The first is a user in the MySQL database; all of the workers as well as the admin account will cause the proxy to authenticate to the database server with these credentials.  The admin user is a separate set of credentials that you use to log in to the proxy management console.  You can pick whatever you want for the admin credentials, but the MySQL credentials must match a MySQL user.

and an other question is, when i connect on the "/" of my proxy, it ask me a login and password.
Is it the same than in the admin section ?
No.  When you use the "/admin" URI, it will ask for admin credentials.  When you use the "/" URI it will ask for worker credentials.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 09, 2011, 05:50:16 PM
 #60

Ok further to my issue, I captured one of those sessions per below:

Code:
13:43:46.455793 IP xxx.113.52046 > xxx.170.http: S 744407596:744407596(0) win 8192 <mss 1460,nop,wscale 2,nop,nop,sackOK>
E..4.^@.~.ewD..qD....N.P,^.,...... .    ...............
13:43:46.455904 IP xxx.170.http > xxx.113.52046: S 3067826941:3067826941(0) ack 744407597 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 4>
E..4..@.@...D...D..q.P.N..R.,^.-....    B..............
13:43:46.456045 IP xxx.113.52046 > xxx.170.http: . ack 1 win 16425
E..(._@.~.e.D..qD....N.P,^.-..R.P.@) ...
13:43:46.456723 IP xxx.113.52046 > xxx.170.http: P 1:489(488) ack 1 win 16425
E....`@.~.c.D..qD....N.P,^.-..R.P.@)....POST / HTTP/1.1
Connection: close
Content-Length: 302
Host: bitproxy.xxxxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": ["00000001bc66976a156c69045f8108574a73efd268ee2faed010d66f00000672000000003d9dd833b0ac46504204b3be7b9dfa2ce1295409970ac51eab2b8e0ed66d12b34dc827ba1b0098fa02ec2550000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"], "id": 1}
13:43:46.456743 IP xxx.170.http > xxx.113.52046: . ack 489 win 432
E..(W.@.@.X.D...D..q.P.N..R.,^..P...]I..
13:43:46.956705 IP xxx.170.http > xxx.113.52046: P 1:484(483) ack 489 win 432
E...W.@.@.V4D...D..q.P.N..R.,^..P.......HTTP/1.1 200 OK
Date: Mon, 09 May 2011 17:43:46 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=badcm71qo331pfnu8b9420med3; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Length: 37
Connection: close
Content-Type: application/json-rpc

{"result":true,"error":null,"id":"1"}
13:43:46.956767 IP xxx.170.http > xxx.113.52046: F 484:484(0) ack 489 win 432
E..(W.@.@.X.D...D..q.P.N..T.,^..P...[e..
13:43:46.956948 IP xxx.113.52046 > xxx.170.http: . ack 485 win 16304
E..(.c@.~.e~D..qD....N.P,^....T.P.?..e..
13:43:47.330166 IP xxx.113.52046 > xxx.170.http: F 489:489(0) ack 485 win 16304
E..(.e@.~.e|D..qD....N.P,^....T.P.?..d..
13:43:47.330189 IP xxx.170.http > xxx.113.52046: . ack 490 win 432
E..(W.@.@.X.D...D..q.P.N..T.,^..P...[d..
13:43:55.614063 IP xxx.113.52047 > xxx.170.http: S 3321044043:3321044043(0) win 8192 <mss 1460,nop,wscale 2,nop,nop,sackOK>
E..4..@.~.eSD..qD....O.P...K...... ..I..............
13:43:55.614082 IP xxx.170.http > xxx.113.52047: S 3074415693:3074415693(0) ack 3321044044 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 4>
E..4..@.@...D...D..q.P.O.?.M...L....................
13:43:55.614375 IP xxx.113.52047 > xxx.170.http: . ack 1 win 16425
E..(..@.~.e^D..qD....O.P...L.?.NP.@).N..
13:43:55.615008 IP xxx.113.52047 > xxx.170.http: P 1:230(229) ack 1 win 16425
..@.~.dxD..qD....O.P...L.?.NP.@).I..POST / HTTP/1.1
Connection: close
Content-Length: 44
Host: bitproxy.xxxxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": [], "id": 1}
13:43:55.615027 IP xxx.170.http > xxx.113.52047: . ack 230 win 432
E..(.f@.@..zD...D..q.P.O.?.N...1P.......
13:43:56.104782 IP xxx.170.http > xxx.113.52047: P 1:1133(1132) ack 230 win 432
D...D..q.P.O.?.N...1P....u..HTTP/1.1 200 OK
Date: Mon, 09 May 2011 17:43:55 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=qt55pon5dn780ipddvv5b5b6i3; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Long-Polling: /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=1
Content-Length: 596
Connection: close
Content-Type: application/json-rpc

{"error":null,"result":{"midstate":"dfbbd14c8d2c37fbcc47bc795c40fe6c8785fe34516942c390fd3fc51ad5eda5","data":"00000001bc66976a156c69045f8108574a73efd268ee2faed010d66f0000067200000000684f3b8dea52cfbfb263679b2f60b4852a54003f9a515ba1fd86c967ea4c79c54dc827d61b0098fa00000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000","hash1":"00000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000010000","target":"ffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000"},"id":"json"}
13:43:56.104936 IP xxx.170.http > xxx.113.52047: F 1133:1133(0) ack 230 win 432
E..(.h@.@..xD...D..q.P.O.?.....1P....u..
13:43:56.105108 IP xxx.113.52047 > xxx.170.http: . ack 1134 win 16142
E..(..@.~.e\D..qD....O.P...1.?..P.?.....
13:43:56.487939 IP xxx.113.52047 > xxx.170.http: F 230:230(0) ack 1134 win 16142
E..(..@.~.eZD..qD....O.P...1.?..P.?.....
13:43:56.487966 IP xxx.170.http > xxx.113.52047: . ack 231 win 432
E..(.i@.@..wD...D..q.P.O.?.....2P....t..
13:43:59.944022 IP xxx.170.http > xxx.113.52026: P 2946665288:2946665812(524) ack 1221764226 win 432
E..4*\@.@..yD...D..q.P.:...HH...P.......HTTP/1.1 200 OK
Date: Mon, 09 May 2011 17:42:03 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=dtmsljts7gfebhij4jhkt76e75; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Length: 78
Connection: close
Content-Type: application/json-rpc

{"error":"Invalid response from long-poll request.","result":null,"id":"json"}
13:43:59.944091 IP xxx.170.http > xxx.113.52026: F 524:524(0) ack 1 win 432
E..(*]@.@...D...D..q.P.:...TH...P...-^..
13:43:59.944370 IP xxx.113.52026 > xxx.170.http: . ack 525 win 16294
E..(..@.~.eND..qD....:.PH......UP.?..g..
13:43:59.981456 IP xxx.113.52026 > xxx.170.http: F 1:1(0) ack 525 win 16294
E..(..@.~.eMD..qD....:.PH......UP.?..f..
13:43:59.981474 IP xxx.170.http > xxx.113.52026: . ack 2 win 432
E..(*^@.@...D...D..q.P.:...UH...P...-]..
13:44:00.417881 IP xxx.113.52048 > xxx.170.http: S 387224838:387224838(0) win 8192 <mss 1460,nop,wscale 2,nop,nop,sackOK>
E..4..@.~.e4D..qD....P.P.......... .Ll..............
13:44:00.417910 IP xxx.170.http > xxx.113.52048: S 3089436255:3089436255(0) ack 387224839 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 4>
E..4..@.@...D...D..q.P.P.%._........................
13:44:00.418212 IP xxx.113.52048 > xxx.170.http: . ack 1 win 16425
E..(..@.~.e?D..qD....P.P.....%.`P.@).z..
13:44:00.418829 IP xxx.113.52048 > xxx.170.http: P 1:230(229) ack 1 win 16425
..@.~.dYD..qD....P.P.....%.`P.@).u..POST / HTTP/1.1
Connection: close
Content-Length: 44
Host: bitproxy.xxxxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": [], "id": 1}
13:44:00.418849 IP xxx.170.http > xxx.113.52048: . ack 230 win 432
E..(..@.@..)D...D..q.P.P.%.`....P.......
13:44:00.948140 IP xxx.170.http > xxx.113.52048: P 1:1133(1132) ack 230 win 432
E.....@.@...D...D..q.P.P.%.`....P....u..HTTP/1.1 200 OK
Date: Mon, 09 May 2011 17:44:00 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=63481nkntcud9f6m4pagb9b1t7; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Long-Polling: /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=1
Content-Length: 596
Connection: close
Content-Type: application/json-rpc

{"error":null,"result":{"midstate":"b182bf7d65f2d20ac82d0d4b565784c2068e9d74540395d99371d401a3771bd3","data":"000000010bf5756fc40209bedd2d96da3a11c0c58e17ead100c5b17f000092cc000000008580d382a53b5854437590690a3931aef8099d83a94942bf6394ac2fc7a09a9b4dc827de1b0098fa00000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000","hash1":"00000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000010000","target":"ffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000"},"id":"json"}
13:44:00.948266 IP xxx.170.http > xxx.113.52048: F 1133:1133(0) ack 230 win 432
E..(..@.@..'D...D..q.P.P.%......P.......
13:44:00.948593 IP xxx.113.52048 > xxx.170.http: . ack 1134 win 16142
E..(..@.~.e=D..qD....P.P.....%..P.?..C..
13:44:01.297377 IP xxx.113.52048 > xxx.170.http: F 230:230(0) ack 1134 win 16142
E..(..@.~.e;D..qD....P.P.....%..P.?..B..
13:44:01.297402 IP xxx.170.http > xxx.113.52048: . ack 231 win 432
E..(..@.@..&D...D..q.P.P.%......P.......
13:44:01.734129 IP xxx.113.52049 > xxx.170.http: S 3138450324:3138450324(0) win 8192 <mss 1460,nop,wscale 2,nop,nop,sackOK>
E..4..@.~.e D..qD....Q.P.......... .I...............
13:44:01.734154 IP xxx.170.http > xxx.113.52049: S 3082134796:3082134796(0) ack 3138450325 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 4>
E..4..@.@...D...D..q.P.Q.............:..............
13:44:01.734452 IP xxx.113.52049 > xxx.170.http: . ack 1 win 16425
...)..@.~.e+D..qD....Q.P.......
13:44:01.734985 IP xxx.113.52049 > xxx.170.http: P 1:203(202) ack 1 win 16425
P.@)....GET /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=1 HTTP/1.1
Connection: close
Host: bitproxy.xxxxx.com
Authorization: Basic xxx
User-Agent: phoenix/v1.2


13:44:01.734999 IP xxx.170.http > xxx.113.52049: . ack 203 win 432
..._P...K`..D...D..q.P.Q...
13:44:02.170888 IP xxx.113.52050 > xxx.170.http: S 2138553343:2138553343(0) win 8192 <mss 1460,nop,wscale 2,nop,nop,sackOK>
...............qD....R.P.w........ ..
13:44:02.170915 IP xxx.170.http > xxx.113.52050: S 3093645464:3093645464(0) ack 2138553344 win 5840 <mss 1460,nop,nop,sackOK,nop,wscale 4>
E..4..@.@...D...D..q.P.R.eH..w.......,..............
13:44:02.171211 IP xxx.113.52050 > xxx.170.http: . ack 1 win 16425
E..(..@.~.e'D..qD....R.P.w...eH.P.@)....
13:44:02.171933 IP xxx.113.52050 > xxx.170.http: P 1:230(229) ack 1 win 16425
..@.~.dAD..qD....R.P.w...eH.P.@)....POST / HTTP/1.1
Connection: close
Content-Length: 44
Host: bitproxy.xxxxx.com
Content-Type: application/json
Authorization: Basic xxx
User-Agent: phoenix/v1.2

{"method": "getwork", "params": [], "id": 1}
13:44:02.171954 IP xxx.170.http > xxx.113.52050: . ack 230 win 432
E..(\.@.@.S.D...D..q.P.R.eH..w..P....7..
13:44:02.670896 IP xxx.170.http > xxx.113.52050: P 1:1133(1132) ack 230 win 432
E...\.@.@.O^D...D..q.P.R.eH..w..P....u..HTTP/1.1 200 OK
Date: Mon, 09 May 2011 17:44:02 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=m3et5o4vie44qhibp48aol5ge0; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Long-Polling: /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=1
Content-Length: 596
Connection: close
Content-Type: application/json-rpc

{"error":null,"result":{"midstate":"50c28d3c275afafd87eed724bd0d5b1847e9dbdda7e79e12ed0e6418ff49766f","data":"000000010bf5756fc40209bedd2d96da3a11c0c58e17ead100c5b17f000092cc00000000cb9cc4fb77da766429a30281b126ccec2aa1e6675f45a1067d56ff2c3bf1be654dc827de1b0098fa00000000000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000","hash1":"00000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000010000","target":"ffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000"},"id":"json"}
13:44:02.671014 IP xxx.170.http > xxx.113.52050: F 1133:1133(0) ack 230 win 432
E..(\.@.@.S.D...D..q.P.R.eM..w..P.......
13:44:02.671333 IP xxx.113.52050 > xxx.170.http: . ack 1134 win 16142
E..(..@.~.e.D..qD....R.P.w...eM.P.?..k..
13:44:03.047103 IP xxx.113.52050 > xxx.170.http: F 230:230(0) ack 1134 win 16142
E..(..@.~.e.D..qD....R.P.w...eM.P.?..j..
13:44:03.047124 IP xxx.170.http > xxx.113.52050: . ack 231 win 432
E..(\.@.@.S.D...D..q.P.R.eM..w..P.......
13:44:03.483509 IP xxx.113.52049 > xxx.170.http: F 203:203(0) ack 1 win 16425
P.@)....~.e     D..qD....Q.P..._...
13:44:03.520512 IP xxx.170.http > xxx.113.52049: . ack 204 win 432
...`P...K_..D...D..q.P.Q...

This is mid-stream, i.e. I let the miner run for maybe 2 minutes, then turned on the sniffer when I thought it would throw the error, so it sent in maybe 2 or 3 successful shares, then the miner threw a disconnection error per below:

[09/05/2011 13:43:22] Result: f780500d accepted
[09/05/2011 13:43:53] Result: 98a41184 accepted
[09/05/2011 13:44:06] Disconnected from server
[09/05/2011 13:44:07] Connected to server

The capture looks completely normal to me, based on my (very lay) knowledge of the protocol. 
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 09, 2011, 05:56:57 PM
 #61

Ok further to my issue, I captured one of those sessions per below:
If I'm reading this right, the miner is actually closing the long-polling connection of its own accord, mid-stream.  What miner are you using?  Perhaps I can reproduce the issue locally.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 09, 2011, 11:34:38 PM
 #62

Just wanted to update this to let you know that Phoenix 1.2 (version I've run for the last couple weeks) seemed to exhibit the problem, while version 1.45 has not yet exhibited the problem during several hours of testing.  I'm going to let this thing run for several more hours, then switch over a few more miners to the proxy and see how it goes.

Two additional feature requests:

1) Can you add in code (maybe configured by config.inc.php) to adjust the timestamp according to a user-defined timezone?  I've become accustomed to looking at UTC and knowing the offset, but still would be a nice thing to have.

2) The purpose of my push for your software, and what makes it valuable to me, is being able to know if a miner stops pulling and submitting shares (in addition to all of the wonderful features you've coded in so far).  Is this a feature you plan to add in the future, i.e. with simply listing all of the miners in red/green status like deepbit does, and having the option to act on it somehow, like with a shell script?

Thanks again for all of your hard work!
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 10, 2011, 03:19:09 AM
 #63

Just wanted to update this to let you know that Phoenix 1.2 (version I've run for the last couple weeks) seemed to exhibit the problem, while version 1.45 has not yet exhibited the problem during several hours of testing.  I'm going to let this thing run for several more hours, then switch over a few more miners to the proxy and see how it goes.
Cool, keep me posted on that.

1) Can you add in code (maybe configured by config.inc.php) to adjust the timestamp according to a user-defined timezone?  I've become accustomed to looking at UTC and knowing the offset, but still would be a nice thing to have.
Sure!  I planned to do this at one point but forgot about it later.  Thanks for reminding me.

2) The purpose of my push for your software, and what makes it valuable to me, is being able to know if a miner stops pulling and submitting shares (in addition to all of the wonderful features you've coded in so far).  Is this a feature you plan to add in the future, i.e. with simply listing all of the miners in red/green status like deepbit does, and having the option to act on it somehow, like with a shell script?
Yeah, this is probably something that could be done.  The red display is pretty easy to do; the shell script will require DB schema changes and a bit more work, but will still be possible.  I assume this would be used to, for example, email you if one of the miners stops requesting work?

I could define three new configuration options: one for the amount of time a miner must have not requested work for it to display differently, one for the amount of time a miner must have not requested work for the script to be run, and one for the script to run.

Thanks again for all of your hard work!
No problem!  I'm glad you find the software useful.  Smiley

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 10, 2011, 06:19:10 AM
 #64

Yeah, this is probably something that could be done.  The red display is pretty easy to do; the shell script will require DB schema changes and a bit more work, but will still be possible.  I assume this would be used to, for example, email you if one of the miners stops requesting work?

Correct.  One could specify a shell script in the database, and perhaps two other user-defined arguments which would be called like:

`$script $minerip $timedelta $userdefinedarg1 $userdefinedarg2`

And would hopefully cause the script to send me an email letting me know that miner 1.2.3.4 has been down for 300 seconds.

Quote
I could define three new configuration options: one for the amount of time a miner must have not requested work for it to display differently, one for the amount of time a miner must have not requested work for the script to be run, and one for the script to run.

Or the script could be a field in the database for workers, i.e. if the miner is a Windows box you might want a simple email, but if it's Linux, you might want to execute a different script that runs 'expect' and attempts to restart the miner by logging in via ssh and giving it a kick.  For that matter, the other two could be database arguments, but if it's too difficult to be practical from a programmatical standpoint, no worries.

Thanks again for all of your hard work!
No problem!  I'm glad you find the software useful.  Smiley
[/quote]

Useful indeed, if only I could make it function as flawlessly in my environment as it does in yours.  So as I mentioned, Phoenix 1.2 prints periodic "disconnected" messages.  Phoenix 1.45 seems to sporadically connect to the long-polling URL.  Phoenix 1.2 will connect, but it disconnects after a period of time.  I think I see why.  I hit my local mining proxy and fetched the long polling URL.  The first time it worked, and actually sent me info on a new block.  But the next time I tried it, I encountered a problem that almost surely explains what I'm seeing in Phoenix:

GET /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=1 HTTP/1.1
Connection: close
Host: bitproxy.xxxxx.com
Authorization: Basic xxx
User-Agent: phoenix/v1.2


(((2 minutes elapse, and then...)))


HTTP/1.1 200 OK
Date: Tue, 10 May 2011 06:00:12 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=4j2r53dsse5afoa5u9k3oratn0; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Length: 78
Connection: close
Content-Type: application/json-rpc

{"error":"Invalid response from long-poll request.","result":null,"id":"json"}Connection closed by foreign host.


Based on the 3 or 4 times I've done this, it seems to happen when there is in fact a new block.  This should be easily duplicable for you.  If not, I can PM you the info on my proxy and let you see for yourself.  Hope that helps.
Cdecker
Hero Member
*****
Offline Offline

Activity: 487



View Profile WWW
May 10, 2011, 12:02:58 PM
 #65

Just wanted to update this to let you know that Phoenix 1.2 (version I've run for the last couple weeks) seemed to exhibit the problem, while version 1.45 has not yet exhibited the problem during several hours of testing.  I'm going to let this thing run for several more hours, then switch over a few more miners to the proxy and see how it goes.
Cool, keep me posted on that.

1) Can you add in code (maybe configured by config.inc.php) to adjust the timestamp according to a user-defined timezone?  I've become accustomed to looking at UTC and knowing the offset, but still would be a nice thing to have.
Sure!  I planned to do this at one point but forgot about it later.  Thanks for reminding me.

2) The purpose of my push for your software, and what makes it valuable to me, is being able to know if a miner stops pulling and submitting shares (in addition to all of the wonderful features you've coded in so far).  Is this a feature you plan to add in the future, i.e. with simply listing all of the miners in red/green status like deepbit does, and having the option to act on it somehow, like with a shell script?
Yeah, this is probably something that could be done.  The red display is pretty easy to do; the shell script will require DB schema changes and a bit more work, but will still be possible.  I assume this would be used to, for example, email you if one of the miners stops requesting work?

I could define three new configuration options: one for the amount of time a miner must have not requested work for it to display differently, one for the amount of time a miner must have not requested work for the script to be run, and one for the script to run.
Being a stats junky I'm working on a simple page that dumps the data in the dashboard to JSON so that I can then write a munin plugin to monitor the status of my miners, I guess we could use that one with a cronjob to kick off custom scripts. It would make it much easier and more flexible since the web server user is usually quite constrained in what he can do ^^

BTW: htdocs/index.php@71 is throwing a lot of errors since in getwork requests the lpurl parameter is not set, and since we already check 2 lines below and then set $lpurl again it is redundant and should be removed ^^

Edit: My first attempt on a JSON stats dump is here: https://github.com/cdecker/Bitcoin-mining-proxy/commit/82f8ad9e352352cdbe81d84a8939b817204cb59d

Edit2: hope you didn't yet look at the stats and the worker-hashing branch because I was missing two GROUP BYs in the shares subquery which resulted in the hashing rate of all miners to be reported on a single miner instead of distinguishing. I fixed it in the branches ^^

Want to see what developers are chatting about? http://bitcoinstats.com/irc/bitcoin-dev/logs/
Bitcoin-OTC Rating
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 10, 2011, 02:25:02 PM
 #66

Correct.  One could specify a shell script in the database, and perhaps two other user-defined arguments which would be called like:

`$script $minerip $timedelta $userdefinedarg1 $userdefinedarg2`

And would hopefully cause the script to send me an email letting me know that miner 1.2.3.4 has been down for 300 seconds.
I'm not sure I could include the IP since that's not logged anywhere.  But I could include the worker's database ID as well as the worker name.

Or the script could be a field in the database for workers, i.e. if the miner is a Windows box you might want a simple email, but if it's Linux, you might want to execute a different script that runs 'expect' and attempts to restart the miner by logging in via ssh and giving it a kick.  For that matter, the other two could be database arguments, but if it's too difficult to be practical from a programmatical standpoint, no worries.
To keep database schema changes to a minimum, I'm not sure if I would want to do this.  But if I pass the worker's ID to the script, you could have your script look at the ID and figure out what to do.

Useful indeed, if only I could make it function as flawlessly in my environment as it does in yours.  So as I mentioned, Phoenix 1.2 prints periodic "disconnected" messages.  Phoenix 1.45 seems to sporadically connect to the long-polling URL.  Phoenix 1.2 will connect, but it disconnects after a period of time.  I think I see why.  I hit my local mining proxy and fetched the long polling URL.  The first time it worked, and actually sent me info on a new block.  But the next time I tried it, I encountered a problem that almost surely explains what I'm seeing in Phoenix:

...

Based on the 3 or 4 times I've done this, it seems to happen when there is in fact a new block.  This should be easily duplicable for you.  If not, I can PM you the info on my proxy and let you see for yourself.  Hope that helps.
Hmm.  This capture is helpful, but without seeing the long-polling request and response that the proxy itself sends out, I can't really say why it's doing this.  It is possible that deepbit itself is returning an error from the long-polling request and I'm just forwarding it on to the worker.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 10, 2011, 02:29:05 PM
 #67

Being a stats junky I'm working on a simple page that dumps the data in the dashboard to JSON so that I can then write a munin plugin to monitor the status of my miners, I guess we could use that one with a cronjob to kick off custom scripts. It would make it much easier and more flexible since the web server user is usually quite constrained in what he can do ^^
There is support in the MVC framework for JSON output.  The dashboard view page already implements IJsonView so try visiting /admin/?format=json.  This works for me.

BTW: htdocs/index.php@71 is throwing a lot of errors since in getwork requests the lpurl parameter is not set, and since we already check 2 lines below and then set $lpurl again it is redundant and should be removed ^^
You're right, I'm not sure why that's there.  That being said, PHP is a whiny little bitch and it likes to complain about things that aren't problems.  Wink

Edit: My first attempt on a JSON stats dump is here: https://github.com/cdecker/Bitcoin-mining-proxy/commit/82f8ad9e352352cdbe81d84a8939b817204cb59d

Edit2: hope you didn't yet look at the stats and the worker-hashing branch because I was missing two GROUP BYs in the shares subquery which resulted in the hashing rate of all miners to be reported on a single miner instead of distinguishing. I fixed it in the branches ^^
I haven't looked at it in-depth yet.  Let me know when you are done with it and have tested it and I'll review the pull request.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
Cdecker
Hero Member
*****
Offline Offline

Activity: 487



View Profile WWW
May 10, 2011, 04:21:56 PM
 #68

Being a stats junky I'm working on a simple page that dumps the data in the dashboard to JSON so that I can then write a munin plugin to monitor the status of my miners, I guess we could use that one with a cronjob to kick off custom scripts. It would make it much easier and more flexible since the web server user is usually quite constrained in what he can do ^^
There is support in the MVC framework for JSON output.  The dashboard view page already implements IJsonView so try visiting /admin/?format=json.  This works for me.
Great, it works for me too, just had to change the URL in my scripts. I think using such an API to run cronjobs against is more useful than having it crammed into the web interface.

BTW: htdocs/index.php@71 is throwing a lot of errors since in getwork requests the lpurl parameter is not set, and since we already check 2 lines below and then set $lpurl again it is redundant and should be removed ^^
You're right, I'm not sure why that's there.  That being said, PHP is a whiny little bitch and it likes to complain about things that aren't problems.  Wink
Agreed Cheesy

Edit: My first attempt on a JSON stats dump is here: https://github.com/cdecker/Bitcoin-mining-proxy/commit/82f8ad9e352352cdbe81d84a8939b817204cb59d

Edit2: hope you didn't yet look at the stats and the worker-hashing branch because I was missing two GROUP BYs in the shares subquery which resulted in the hashing rate of all miners to be reported on a single miner instead of distinguishing. I fixed it in the branches ^^
I haven't looked at it in-depth yet.  Let me know when you are done with it and have tested it and I'll review the pull request.
I guess I can drop the stats branch then, since it already works out of the box from your dashboard. On the worker-hashing branch I made the average interval configurable because it strongly depends on the hashing speed (high hashing speeds can get a more accurate reading with shorter intervals, while slow miners will have the average jump wildly around for small values). I think it is ready to be pulled. Once again sorry for all the noise I created in here ^^

Want to see what developers are chatting about? http://bitcoinstats.com/irc/bitcoin-dev/logs/
Bitcoin-OTC Rating
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 10, 2011, 04:30:38 PM
 #69

Great, it works for me too, just had to change the URL in my scripts. I think using such an API to run cronjobs against is more useful than having it crammed into the web interface.
Yup, that's why it's there.  Smiley  Not all of the pages have proper JSON support implemented, simply because for most of them it wouldn't be terribly useful.  But at some point, the proxy should allow full manipulation of the database objects via JSON, allowing you to code an alternative interface (like a desktop app) against the JSON API.  That probably wouldn't be a good thing to do, but it would at least be possible.

I guess I can drop the stats branch then, since it already works out of the box from your dashboard. On the worker-hashing branch I made the average interval configurable because it strongly depends on the hashing speed (high hashing speeds can get a more accurate reading with shorter intervals, while slow miners will have the average jump wildly around for small values). I think it is ready to be pulled. Once again sorry for all the noise I created in here ^^
Alright, I'll have a look at it at some point later.  Thanks for the patch!

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
molecular
Donator
Legendary
*
Offline Offline

Activity: 2436



View Profile
May 10, 2011, 10:34:13 PM
 #70

hey cdhowie,

great proxy, just checking it out.

Had problems with the php notices spamming the pages to the point where chrome refused to render them (they get inserted before DOCTYPE decl):

Example:
Code:
<br />
<b>Notice</b>:  Undefined variable: viewdata in <b>/var/www/localhost/htdocs/bmproxy/admin/pool.php</b> on line <b>55</b><br />
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
    <head>
...

I simply put
Code:
error_reporting(E_ERROR);
in my config.inc.php, that fixed the issue nicely.

Maybe you want to set the error_reporting level somewhere centrally to avoid this problem happening to others and also to avoid having to explain all the notices.

PGP key molecular F9B70769 fingerprint 9CDD C0D3 20F8 279F 6BE0  3F39 FC49 2362 F9B7 0769
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 10, 2011, 11:45:40 PM
 #71

Maybe you want to set the error_reporting level somewhere centrally to avoid this problem happening to others and also to avoid having to explain all the notices.
Hmm, I thought I did.  Maybe I just did that in my own php.ini or something.  Anyway, thanks for the comment.  I'll add this in a future push.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 11, 2011, 04:15:14 AM
 #72

Quote from: cdhowie
Hmm.  This capture is helpful, but without seeing the long-polling request and response that the proxy itself sends out, I can't really say why it's doing this.  It is possible that deepbit itself is returning an error from the long-polling request and I'm just forwarding it on to the worker.

Ahhh I see the problem.  From your proxy to deepbit:

GET /listenChannel HTTP/1.0
Authorization: Basic youcanthavemypassword;]
Host: deepbit.net:8332

I watched it send this to deepbit once and it worked, but the next time it didn't.  The request was the same, but it appears to be failing because you're sending the request as HTTP 1.0.  Based on the reply from the pool, you can see that the pool wants to serve HTTP/1.1.  So my guess is we're hitting nginx, which is setup as a loadbalancer, and one of the backend machines is configured to work with HTTP 1.0 and the other is not.  You can also see from my previous post that the client normally asks in HTTP/1.1 (though in the long polling specification at deepbit's website, they don't mention that it needs HTTP/1.1).  If that all looks sensible, can you commit a fix to make the long polling requests in HTTP/1.1?

Thanks!
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 11, 2011, 06:49:17 AM
 #73

I've just done a push.  I recommend that everyone update from master.

This update fixes some PHP warnings that would get triggered on a failover, effectively breaking failover entirely on configurations where PHP warnings are emitted.  The pool timeout has also been lowered from 5 seconds to 2 since the default configuration of some miners is to request work every 5 seconds.  These miners will break if the proxy takes too long to respond, terminating the active getwork request and issuing another.  (This timeout will be made configurable in a later push so that you can customize it to your configuration.)

An error_reporting() call has also been added to the config.inc.php.sample file.  It's strongly recommended that you copy this into your config.inc.php to mitigate any possible future issues caused by PHP warnings getting mixed into the JSON response for workers.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 11, 2011, 06:54:27 AM
 #74

Ahhh I see the problem.  From your proxy to deepbit:

GET /listenChannel HTTP/1.0
Authorization: Basic youcanthavemypassword;]
Host: deepbit.net:8332

I watched it send this to deepbit once and it worked, but the next time it didn't.  The request was the same, but it appears to be failing because you're sending the request as HTTP 1.0.  Based on the reply from the pool, you can see that the pool wants to serve HTTP/1.1.  So my guess is we're hitting nginx, which is setup as a loadbalancer, and one of the backend machines is configured to work with HTTP 1.0 and the other is not.  You can also see from my previous post that the client normally asks in HTTP/1.1 (though in the long polling specification at deepbit's website, they don't mention that it needs HTTP/1.1).  If that all looks sensible, can you commit a fix to make the long polling requests in HTTP/1.1?

Thanks!
Hmm... I'm not going to discount this explanation, but I don't see any evidence that sending an HTTP 1.0 request is causing problems.  My workers are issuing long-polling requests to deepbit through the proxy and they appear to be working just fine.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 11, 2011, 10:12:39 AM
 #75

Hmm... I'm not going to discount this explanation, but I don't see any evidence that sending an HTTP 1.0 request is causing problems.  My workers are issuing long-polling requests to deepbit through the proxy and they appear to be working just fine.

I just cranked up a new kind of sniffer, one that logs each unique tuple into a separate file.  What I found is that upon receiving my input text (fed to your proxy via telnet) as follows:

Quote
GET /index.php?lpurl=http%3A%2F%2Fdeepbit.net%3A8332%2FlistenChannel&pool=1 HTTP/1.1
Connection: close
Host: bitproxy.xxxxx.com
Authorization: Basic xxx
User-Agent: phoenix/v1.2
(and one more trailing CR)

Your proxy actually connected to deepbit twice.  The first time was apparently to setup the actual long-polling session for my telnet to the proxy.  The second communication was completely blank - neither side said anything.  After a couple of minutes, my telnet session spat back the following:

Quote
HTTP/1.1 200 OK
Date: Wed, 11 May 2011 09:59:01 GMT
Server: Apache/2.2.3 (CentOS)
X-Powered-By: PHP/5.1.6
X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy
Set-Cookie: PHPSESSID=bd8s2r4gnb80pgl6n3hd3ne606; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Length: 78
Connection: close
Content-Type: application/json-rpc

{"error":"Invalid response from long-poll request.","result":null,"id":"json"}

This was not in response to anything being sent from deepbit to the proxy - deepbit never at any time uttered another word.  I ran the same testing iteration again and got a successful long poll when the pool jumped to the next block, but on the next iteration it was exactly the same behavior as above.  (note that all of this is still based on a 2-day old pull, I did not pull the code again per your mention above).

Does that help?
pharaon
Full Member
***
Offline Offline

Activity: 129


View Profile
May 16, 2011, 02:58:57 PM
 #76

can you give me a name of web hoster where your proxy can work ?

All i test, had 'php_auth_user' disable ......

Sorry for my poor english
BurningToad
Full Member
***
Offline Offline

Activity: 207


View Profile
May 17, 2011, 04:27:25 AM
 #77

Took me a while, but I got it set up! Thanks!

1 BTC heading your way

BurningToad
Full Member
***
Offline Offline

Activity: 207


View Profile
May 17, 2011, 11:27:00 AM
 #78

Actually this just didn't work very well for me Sad  Set it up with BTC Guild as primary and Slush as secondary.  BTC Guild is having intermittent connection issues, so I wanted a backup.  However, overnight all of my GUIMiner clients got stuck with a "connection problem".  My one phoenix client (which is also on same machine as my proxy) continued working fine however.

Cdecker
Hero Member
*****
Offline Offline

Activity: 487



View Profile WWW
May 17, 2011, 12:18:29 PM
 #79

Actually this just didn't work very well for me Sad  Set it up with BTC Guild as primary and Slush as secondary.  BTC Guild is having intermittent connection issues, so I wanted a backup.  However, overnight all of my GUIMiner clients got stuck with a "connection problem".  My one phoenix client (which is also on same machine as my proxy) continued working fine however.
Sounds strange. The switch should occur either upon work request or work submission. The latter isn't very nice because a share might be lost. We are working on exponential backoff to temporarily blacklist flapping pools (intermittent connection issues). Should it occur again please provide a tcpdump (just let tcpdump run along capturing traffic from/to the pool proxy port 80, and use temporary passwords between pool proxy and the workers).

Want to see what developers are chatting about? http://bitcoinstats.com/irc/bitcoin-dev/logs/
Bitcoin-OTC Rating
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 17, 2011, 02:11:18 PM
 #80

The switch should occur either upon work request or work submission.
Switches will never occur on work submission.  Submissions always route back to the pool that the work came from.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
leepfrog
Sr. Member
****
Offline Offline

Activity: 252


View Profile
May 22, 2011, 09:07:50 PM
 #81

Great peace of software. Managed to get it running in under 10 minutes.
I've always had some cases in the past where there were connection problems and the miner was in a stale state after connectivity has been restored. Looking forward to see if this is solved when using the proxy.

It also would be an interesting idea to take such a service and offer it for hosting, so that the average miner does not have to worry about pool failover any more as he can simply point to some public proxy which handles the requests.
leepfrog
Sr. Member
****
Offline Offline

Activity: 252


View Profile
May 23, 2011, 04:22:25 AM
 #82

Actually this did not run as well as expected. I've got a very high rate of rejected (about 10%) and after running for two hours phoenix stalled again:

Traceback (most recent call last):
  File "phoenix.py", line 125, in <module>
  File "twisted\internet\base.pyc", line 1162, in run
  File "twisted\internet\base.pyc", line 1171, in mainLoop
--- <exception caught here> ---
  File "twisted\internet\base.pyc", line 793, in runUntilCurrent
  File "minerutil\RPCProtocol.pyc", line 127, in callback
  File "minerutil\RPCProtocol.pyc", line 319, in handleWork
exceptions.TypeError: argument of type 'NoneType' is not iterable
[23/05/2011 01:04:42] Result: 4b18edf7 rejected

My connection has some packet loss (~2-4%) but the latency in general is good (under 150ms). Could this be a problem?
oVPN
Member
**
Offline Offline

Activity: 105


View Profile WWW
May 23, 2011, 10:43:54 AM
 #83

hi

there are problems with php-authentication, if php is running as fast-cgi, but after switching to mod_php, login to admin-panel is working.

i believe ufa cpu-miner is not supported at the moment? all my tests ran in timeouts Sad


oVPN.to Anonymous Services
twmz
Hero Member
*****
Offline Offline

Activity: 673



View Profile
May 26, 2011, 06:38:51 AM
 #84

I really like the idea of this tool, but have failed to get it working well.

I use phoenix as my miner on ubuntu 11.04.  I have tried a couple different web hosts including my own ubuntu 11.04 box, and I have the following symptoms:

1. lots more stale shares than when connected directly.  long polling does not seem to work with phoenix through the proxy but works fine when connected directly.
2. phoenix miner *hammers* the web server with requests to index.php.  Something like 10-20 requests per second as seen in the apache access.log.  When connected directly and observed with tcpdump it behaves much more normally (1 request every 5 seconds or so).
3. lower than normal hash rate when using the proxy.  I suspect this may be because the miner is so busy doing #2 as fast as it can.

I also tried poclbm on my Mac and it had similar problems with stale shares.  I did see an occasional LP message, but not frequently enough as compared to blocks being found in the network.  I also regularly got errors about "NoneType" not being indexable.

From this thread, though, it sounds like some people are having great success with it, so I wish I knew what to do differently.

Was I helpful?  1TwmzX1wBxNF2qtAJRhdKmi2WyLZ5VHRs
WoT, GPG

Bitrated user: ewal.
datathe1st
Jr. Member
*
Offline Offline

Activity: 30


View Profile
May 26, 2011, 08:34:16 AM
 #85

I'd like to set this up to manage a mining for computers I have in different locations, but I am hesitant because of reports by some users of stale shares.

Thank you so very much cdhowie for even releasing what looks like an amazing piece of software.


Btw if I have access to computers at school and they are behind a firewall and so cannot mine, can I use this software to get around that?

All donations (even 0.01 BTC) are much appreciated! Thank you!
13PNeVP5hNbmXVccfeC73BENtKdht2aEVe
Dusty
Hero Member
*****
Offline Offline

Activity: 725


Libertas a calumnia


View Profile WWW
May 26, 2011, 10:15:36 AM
 #86

Watching (very nice project btw)

Articoli bitcoin: Il portico dipinto
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 26, 2011, 04:26:46 PM
 #87

Just pushed some updates.

  • Applied patch from cdecker implementing hash speed estimation for workers (issue 5).
  • Implemented timezone conversion (issue 7).

These changes require modifications to your config.inc.php.  If you pull without updating your config.inc.php, the software will break.

To address some posts:

Actually this just didn't work very well for me Sad  Set it up with BTC Guild as primary and Slush as secondary.  BTC Guild is having intermittent connection issues, so I wanted a backup.  However, overnight all of my GUIMiner clients got stuck with a "connection problem".  My one phoenix client (which is also on same machine as my proxy) continued working fine however.
Some miners will bail on a connection if it doesn't return work within 3-5 seconds.  This is not always enough time for the proxy to detect a fail condition, and so the miner gives up before the proxy has a chance to try another pool.

Actually this did not run as well as expected. I've got a very high rate of rejected (about 10%) and after running for two hours phoenix stalled again:
I have started seeing similar problems on my install.  This is being actively researched.

I did see an occasional LP message, but not frequently enough as compared to blocks being found in the network.  I also regularly got errors about "NoneType" not being indexable.
I have been seeing the same with poclbm lately.  I'm not sure what changed to make this happen, but I suspect it is related to the stale shares problem.

i believe ufa cpu-miner is not supported at the moment? all my tests ran in timeouts Sad
I haven't used that particular miner and so I'm not sure if the proxy supports it.  If the miner isn't doing anything too wacky it should work...  If I get some free time I might investigate this.

long polling does not seem to work with phoenix through the proxy but works fine when connected directly.
There are some issues with long polling that are not easily addressed by the current model.  Long polling should in theory work just fine if your preferred pool doesn't go down, but during a failover scenario it can get a little bit wonky.  Some miners behave differently with respect to how they handle changing long-polling URLs, and so this is something that will take a lot of engineering to fix.

In the meantime I may supply a config option to disable long polling entirely; broken long polling is worse than none at all.

phoenix miner *hammers* the web server with requests to index.php.  Something like 10-20 requests per second as seen in the apache access.log.  When connected directly and observed with tcpdump it behaves much more normally (1 request every 5 seconds or so).
Sounds like a bug in Phoenix.  Can you take some sniffs of this behavior with tcpdump -w bmp.pcap port 80 and email me the pcap file(s)?  This will help me figure out what's going on.  (Please compress them, and you can optionally encrypt them to my GPG key.  This should not expose your mining account passwords, only your local proxy worker and possibly admin passwords.)

Btw if I have access to computers at school and they are behind a firewall and so cannot mine, can I use this software to get around that?
As long as the computers can make outbound connections on whatever port you are running your web server on, yes.  Transparent (or non-transparent) HTTP proxies should be able to process these requests too, though they might strip long-polling info.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
twmz
Hero Member
*****
Offline Offline

Activity: 673



View Profile
May 28, 2011, 11:18:24 PM
 #88

Hey, cdhowie.  I'm still not clear on what was happening with the proxy, but I have a couple clues.

I did determine that the latest phoenix behavior is to re-request the long polling URL immediatlly if the LP does not return a valid getwork response.  So, I think the issue with phoenix hammering the web server was because the proxied long-poll address must have been immediately returning, which caused phoenix to request it again right away, which caused...  well, a very fast loop of LP requests to the proxy.

In the end, I only know enough about PHP to be dangerous, and less about how to run an apache web server.  In the end, I was so inspired by the idea of this proxy that I ended up recoding it in ASP.NET, not because it is better than ASP.NET but just because I have 10 years of experience with it and know how to debug.  For the actual proxying of the JSON-RPC requests, I followed your algorithm almost exactly, just in C#, ASP.NET MVC, IIS and SQL Server.  I now have it working and long polling is working as well although the whole design of long polling tie-ing up a web service thread for 10-20 minutes doesn't play all nicely with IIS, but it doesn't matter too much for my 4 measly miners.  A side bonus is that ASP.NET has a very capable charting component and so I was able to quickly whip up some graphs for the dashboard as well.

To be clear, I doubt anyone will care about my implementation.  An IIS web server and SQL Server are much harder to come by than a Linux, Apache, MySQL, and PHP server.  I just happen to have a suitable windows server in my house already, so it worked well for me.  That said, as soon as I polish it a little more, I will post the source code somewhere given that I did borrow the idea and some of the assets from your project.


Was I helpful?  1TwmzX1wBxNF2qtAJRhdKmi2WyLZ5VHRs
WoT, GPG

Bitrated user: ewal.
anisoptera
Member
**
Offline Offline

Activity: 98



View Profile
May 29, 2011, 06:21:39 AM
 #89

To be clear, I doubt anyone will care about my implementation. 

I do, I definitely want to see the code. Smiley
Cdecker
Hero Member
*****
Offline Offline

Activity: 487



View Profile WWW
May 29, 2011, 09:53:33 AM
 #90

Just seeing now that you have a payout monitor in there, that would be a cool feature, though it adds a lot of problems since some pools don't provide that info and even worse, they don't adhere to a consistent standard for publishing them.

Want to see what developers are chatting about? http://bitcoinstats.com/irc/bitcoin-dev/logs/
Bitcoin-OTC Rating
twmz
Hero Member
*****
Offline Offline

Activity: 673



View Profile
May 29, 2011, 12:38:36 PM
 #91

Just seeing now that you have a payout monitor in there, that would be a cool feature, though it adds a lot of problems since some pools don't provide that info and even worse, they don't adhere to a consistent standard for publishing them.

Yes, there is no standard.  But it only turned out to be a couple lines of code specific to each type of pool to fish the balance out of each pools JSON API.

It actually doesn't work that well and is the main thing I am working on right now before releasing the code in the next day or two.  Making all of the JSON API calls to retrieve current balances is making the dashboard load slowly, so I'm moving balance updating to a background thread.

Was I helpful?  1TwmzX1wBxNF2qtAJRhdKmi2WyLZ5VHRs
WoT, GPG

Bitrated user: ewal.
0xlemming
Newbie
*
Offline Offline

Activity: 2


View Profile
May 29, 2011, 06:34:33 PM
 #92

Is the stuff in work_data needed for something importent? I added to my runnning script a function which remove all entries older than 10h.
twmz
Hero Member
*****
Offline Offline

Activity: 673



View Profile
May 29, 2011, 06:51:47 PM
 #93

10h should be ok.  It is needed when a miner eventually submits a share.  The proxy uses the information in the work_data table to look up which pool the miner got that work from so that it gets submitted back to the right place.  So don't delete everything in the table, but it is not needed as soon as your miners have finished working on their current work task.

Was I helpful?  1TwmzX1wBxNF2qtAJRhdKmi2WyLZ5VHRs
WoT, GPG

Bitrated user: ewal.
error
Hero Member
*****
Offline Offline

Activity: 574



View Profile
May 30, 2011, 03:40:03 AM
 #94

Thanks for the proxy. Seems to work pretty well so far.

I did have to make one change to fix a problem with the proxy out of the box. Whenever I submitted a form, I'd be redirected to some other web site (localhost) which has no server running. It appears you're reading SERVER_NAME, which may or may not be correct. It's more reliable to read HTTP_HOST:

Code:
--- a/htdocs/common.inc.php    2011-05-29 22:20:55.000000000 -0400
+++ b/htdocs/common.inc.php      2011-05-29 23:29:49.000000000 -0400
@@ -154,7 +154,7 @@
     $port = ($default_port == $_SERVER['SERVER_PORT']) ? "" :
         ":" . $_SERVER['SERVER_PORT'];
 
-    $base = "$scheme://{$_SERVER['SERVER_NAME']}$port" . get_site_uri();
+    $base = "$scheme://{$_SERVER['HTTP_HOST']}$port" . get_site_uri();
 
     return $base . $uri;
 }

15UFyv6kfWgq83Pp3yhXPr8rknv9m6581W
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 30, 2011, 03:58:35 AM
 #95

For those of you affected by deepbit/slush outages today, how well did this code hold up for you?
error
Hero Member
*****
Offline Offline

Activity: 574



View Profile
May 30, 2011, 04:10:30 AM
 #96

For those of you affected by deepbit/slush outages today, how well did this code hold up for you?

It's pretty transparently switching between pools. The miner never notices a thing.

15UFyv6kfWgq83Pp3yhXPr8rknv9m6581W
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 30, 2011, 04:29:08 AM
 #97

Sweet.  I have what looks like a working implementation, minus the aforementioned difficulties with long-polling that me and several others have experienced.  But it wasn't in use today for that reason.  I'm hoping to find a simple http proxying script and just change the LP URL handed out by cdhowie's script, to make the miners hit a different script to handle long polling in a non-problematic way.  Or so I hope.
d3c0n808
Member
**
Offline Offline

Activity: 115

Not Goxed


View Profile
May 30, 2011, 05:09:35 AM
 #98

I'm sorry but I need to ask a question.  Currently for pools I have deepbit and solo, deepbit has a higher priority but doesnt connect for some reason.  I rebooted, the computer can resolve the dns but none of the miners connect to deepbit just the solo.  Perhaps this has been asked before.  Im wondering if there is some type of cache that needs to be reset or what.  Im using poclbm gui and poclbm on my dedicated miners(slackware)  The admin pages seems to be working fine but its just not kicking over to deepbit.  Thank you in advance and the address i ahve in the pool config is http://pit.deepbit.net:8332

60 GH/s BFL Single SC - Pre-Order Yours Today!
`````` Only $1299.99 - butterflylabs.com ``````
CubedRoot
Sr. Member
****
Offline Offline

Activity: 295


View Profile
May 30, 2011, 07:20:16 AM
 #99

Going to look into this one for myself.  I am having a problem mining from work Smiley  I have my own co-located server, but dont want to setup a pool on it.  I might could use this to connect my miners to my co-located server on a port that isnt blocked by our firewall, and then use my co-located server to connect to the pools.
Yngwie
Newbie
*
Offline Offline

Activity: 3


View Profile
May 30, 2011, 09:04:41 AM
 #100

I can't configure this proxy((
Admin panel works fine, i add pools (deepbit: http://deepbit.net:8332 and btcguild: http://btcguild.com:8332) then created worker (login q password 1) add pool workers credentials for it (email_5 password_of_worker_in_pool for deepbit login_2 password_of_worker_in_pool for btc guild). But when i started miner (i tried poclbm, bitcoin-miner, diablo miner windows with my servers ip as url port 80 login and password from proxy worker) proxy returned
HTTP/1.1 200 OK

Date: Mon, 30 May 2011 07:46:57 GMT

Server: Apache

X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy

Set-Cookie: PHPSESSID=kph1h3dib6nnad4okc3k29kp85; path=/

Expires: Thu, 19 Nov 1981 08:52:00 GMT

Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0

Pragma: no-cache

Content-Length: 85

Content-Type: application/json-rpc



{"error":"No enabled pools responded to the work request.","result":null,"id":"json"}

How can i solve this problem?
error
Hero Member
*****
Offline Offline

Activity: 574



View Profile
May 30, 2011, 09:28:11 AM
 #101

Add more pools?

15UFyv6kfWgq83Pp3yhXPr8rknv9m6581W
d3c0n808
Member
**
Offline Offline

Activity: 115

Not Goxed


View Profile
May 30, 2011, 10:04:55 AM
 #102

I can't configure this proxy((
Admin panel works fine, i add pools (deepbit: http://deepbit.net:8332 and btcguild: http://btcguild.com:8332) then created worker (login q password 1) add pool workers credentials for it (email_5 password_of_worker_in_pool for deepbit login_2 password_of_worker_in_pool for btc guild). But when i started miner (i tried poclbm, bitcoin-miner, diablo miner windows with my servers ip as url port 80 login and password from proxy worker) proxy returned
HTTP/1.1 200 OK

Date: Mon, 30 May 2011 07:46:57 GMT

Server: Apache

X-Source-Code: https://github.com/cdhowie/Bitcoin-mining-proxy

Set-Cookie: PHPSESSID=kph1h3dib6nnad4okc3k29kp85; path=/

Expires: Thu, 19 Nov 1981 08:52:00 GMT

Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0

Pragma: no-cache

Content-Length: 85

Content-Type: application/json-rpc



{"error":"No enabled pools responded to the work request.","result":null,"id":"json"}

How can i solve this problem?
 


I'm getting this exact same error.   Ive added slush's pool, deepbit, btcmine, and solo and solo is the only one that works....

60 GH/s BFL Single SC - Pre-Order Yours Today!
`````` Only $1299.99 - butterflylabs.com ``````
error
Hero Member
*****
Offline Offline

Activity: 574



View Profile
May 30, 2011, 11:27:16 AM
 #103

Sorry, I meant to say: ENABLE your pools!

15UFyv6kfWgq83Pp3yhXPr8rknv9m6581W
d3c0n808
Member
**
Offline Offline

Activity: 115

Not Goxed


View Profile
May 30, 2011, 11:35:33 AM
 #104

I followed the instructions to the T, and yes all 4 pools are enabled. 

60 GH/s BFL Single SC - Pre-Order Yours Today!
`````` Only $1299.99 - butterflylabs.com ``````
0xlemming
Newbie
*
Offline Offline

Activity: 2


View Profile
May 30, 2011, 05:34:34 PM
 #105


I'm getting this exact same error.   Ive added slush's pool, deepbit, btcmine, and solo and solo is the only one that works....
Same problem here. On my site it's currently only deepbit with this error.
d3c0n808
Member
**
Offline Offline

Activity: 115

Not Goxed


View Profile
May 30, 2011, 10:46:59 PM
 #106


I'm getting this exact same error.   Ive added slush's pool, deepbit, btcmine, and solo and solo is the only one that works....
Same problem here. On my site it's currently only deepbit with this error.

This seems like its some kind of dependancy.  I reinstalled the database using phpmysql to see if perhaps mysql admin didnt install the scheme properly.  That did not resolve the issue.  So I loaded it on my suse box at another location and it works without a hitch.  It would seem to me there is some dependy that slackware and your distro don't have that are required.  PHP magic quotes is is disabled so I know its not that I also tried to install php 5.2 on my slackware box and that did not resolve the issue either. 

60 GH/s BFL Single SC - Pre-Order Yours Today!
`````` Only $1299.99 - butterflylabs.com ``````
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 31, 2011, 01:14:32 AM
 #107

Slackware?  Heh I'd hang it up.  I don't even bother trying to run anything meaningful on it (i.e. above basic apache, php and mail).  I love slack, been an avid user since version 3.3 (circa 1996).  But it has very little utility.

I can also confirm that this package, as it sits, is very distribution-specific.  It runs out of the box on Cent 5.6 but is broken on FC14x64, have not had the time to troubleshoot further.  Out of curiosity, have you installed json for php?
d3c0n808
Member
**
Offline Offline

Activity: 115

Not Goxed


View Profile
May 31, 2011, 01:23:37 AM
 #108

Slackware?  Heh I'd hang it up.  I don't even bother trying to run anything meaningful on it (i.e. above basic apache, php and mail).  I love slack, been an avid user since version 3.3 (circa 1996).  But it has very little utility.

I can also confirm that this package, as it sits, is very distribution-specific.  It runs out of the box on Cent 5.6 but is broken on FC14x64, have not had the time to troubleshoot further.  Out of curiosity, have you installed json for php?

I prefer slackware to just about any other distro.   Its fast, simple, efficient, sbopkg has come along way and i build most of my own modules, kernels etc etc.  I wasn't sure if slackware's php package had json rpc built in, so in installed it, no go.  Rebuilt php with json argument, installed pkg(after removing old pkg ofcourse)  still no go.  I don't think its a json issue.  I could be wrong and maybe its  a version issue.  Im just going to keep using it on my suse box as the slackware box seems to have enough to do already

60 GH/s BFL Single SC - Pre-Order Yours Today!
`````` Only $1299.99 - butterflylabs.com ``````
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 31, 2011, 01:32:40 AM
 #109

I agree, I'd use Slack for everything if I could.  I mentioned JSON because in PHP versions prior to 5.3 you have to install it separately.  I asked cdhowie to mention this somewhere as a dependency, as to cut down on potential support issues, but I guess it's still not in the docs.
d3c0n808
Member
**
Offline Offline

Activity: 115

Not Goxed


View Profile
May 31, 2011, 01:45:51 AM
 #110

I agree, I'd use Slack for everything if I could.  I mentioned JSON because in PHP versions prior to 5.3 you have to install it separately.  I asked cdhowie to mention this somewhere as a dependency, as to cut down on potential support issues, but I guess it's still not in the docs.

All my miners run slack, sdk 2.1, cat 11.4 no problem.  Its just this darn proxy hahaha.  Slack 13.37 ships with 5.3.6 so json should be installed by default and I believe it is as installing it manually created a json error stating that the module was already loaded.  Im wondering if 5.3.6 just doesnt play well with the proxy oh well eventually i imagine ill figure it out.  Suse in runlevel 3 isnt too bad, just not as slick as slack hahaha. 

60 GH/s BFL Single SC - Pre-Order Yours Today!
`````` Only $1299.99 - butterflylabs.com ``````
kjj
Legendary
*
Offline Offline

Activity: 1302



View Profile
May 31, 2011, 02:29:55 AM
 #111

I'm running the proxy on Slackware.  Slackware 10.2, no less.

Works great with BTC Guild, Deepbit, Eligius, a local bitcoind (last resort), and a local pushpool (for testing, that was a much bigger hassle to build on Slack).  Works great with poclbm and phoenix, but tends to cause DiabloMiner to get hung.

Just finished setting up a cron job to SNMP-diddle an APC networked PDU to reboot my boxes if they ever go more than 15 minutes without a work request.

p2pcoin: a USB/CD/PXE p2pool miner - 1N8ZXx2cuMzqBYSK72X4DAy1UdDbZQNPLf - todo
I routinely ignore posters with paid advertising in their sigs.  You should too.
d3c0n808
Member
**
Offline Offline

Activity: 115

Not Goxed


View Profile
May 31, 2011, 02:32:41 AM
 #112

I'm running the proxy on Slackware.  Slackware 10.2, no less.

Works great with BTC Guild, Deepbit, Eligius, a local bitcoind (last resort), and a local pushpool (for testing, that was a much bigger hassle to build on Slack).  Works great with poclbm and phoenix, but tends to cause DiabloMiner to get hung.

Just finished setting up a cron job to SNMP-diddle an APC networked PDU to reboot my boxes if they ever go more than 15 minutes without a work request.


Full install or did you have to build your own php package?  Also is there any special configurations you used?  Im still trying to get my slack to work hahaah

60 GH/s BFL Single SC - Pre-Order Yours Today!
`````` Only $1299.99 - butterflylabs.com ``````
kjj
Legendary
*
Offline Offline

Activity: 1302



View Profile
May 31, 2011, 02:37:13 AM
 #113

I custom build PHP and Apache (and openssl, and mod_ssl, and mm, and mysql, and...).  Wrote a script for it many, many years ago.

I did have to add --with-pdo-mysql to the file that stores my PHP config options to get this working.

p2pcoin: a USB/CD/PXE p2pool miner - 1N8ZXx2cuMzqBYSK72X4DAy1UdDbZQNPLf - todo
I routinely ignore posters with paid advertising in their sigs.  You should too.
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 31, 2011, 02:57:07 AM
 #114

Just finished setting up a cron job to SNMP-diddle an APC networked PDU to reboot my boxes if they ever go more than 15 minutes without a work request.

Nice!  You should open-source that.. I was just pondering making such a thing for my AP9211s Smiley
kjj
Legendary
*
Offline Offline

Activity: 1302



View Profile
May 31, 2011, 03:15:21 AM
 #115

It'll take some work to adapt to your location.  Oh, and this is not pretty.

First file is a PHP script that runs out of cron every 10 minutes.  Second file is a BASH CGI that lives on another box.  If your flex proxy can see the PDU directly, you can cut out the middle man and either use snmpget() and snmpsend() PHP functions, if you have them, or exec the binaries directly.

My worker IDs are 3 and 4, so I plugged them into ports 3 and 4 on the PDU.  That isn't necessary, unless you are OCD like me.  The arg to the CGI is pasted directly onto the end of the OID passed to SNMP, which in this case is the outlet number on the strip.  Oh, and you should totally add some input checking to the CGI, because as it is now, you are asking to get pwn3d by executing attacker-supplied variables.

Search for __ (double underlines), those are things that you need to change.

Tested and working on a APC 885-1890 (1U, 8 outlet, 15Amp, 120v).  Yours may differ.  Thank god I was able to find the OIDs for SNMP, I wasn't looking forward to writing an expect script to navigate the telnet menu.

Code: (cron_job)
#!/bin/php
<?php

$restart_3
=TRUE;
$restart_4=TRUE;

$con=mysql_connect("__HOST__","__USER__","__PASS__");
mysql_select_db("__DATABASE__",$con);

$q="SELECT DISTINCT worker_id FROM work_data WHERE time_requested  > (NOW() - INTERVAL 15 MINUTE)";

$r=mysql_query($q);
if(
0!=mysql_errno())echo mysql_error()."\n";
while(
$row=mysql_fetch_assoc($r)){
 if(
3==$row['worker_id'])$restart_3=FALSE;
 if(
4==$row['worker_id'])$restart_4=FALSE;
// echo $row['worker_id']."\n";
}

if(
$restart_3){
// echo "Restarting 3\n";
 
$junk=file("http://__CGI_HOST__/cgi-bin/apc_restart.cgi?3");
}
if(
$restart_4){
// echo "Restarting 4\n";
 
$junk=file("http://__CGI__HOST__/cgi-bin/apc_restart.cgi?4");
}

?>


Code: (bash_cgi)
#!/bin/bash
echo "Content-type: text/html"
echo ""
echo ""
echo "<html>"
echo "<head><title>APC restart</title></head>"
echo "<body>"
snmpset -v1 -c __WRITE_COMMUNITY__ __PDU_IP__ .1.3.6.1.4.1.318.1.1.12.3.3.1.1.4.$QUERY_STRING i 3
echo "<br>"
snmpget -v1 -c __READ_COMMUNITY__ __PDU_IP__ .1.3.6.1.4.1.318.1.1.12.3.3.1.1.4.$QUERY_STRING
echo "<br>"
echo "</body>"
echo "</html>"

p2pcoin: a USB/CD/PXE p2pool miner - 1N8ZXx2cuMzqBYSK72X4DAy1UdDbZQNPLf - todo
I routinely ignore posters with paid advertising in their sigs.  You should too.
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
May 31, 2011, 03:44:55 AM
 #116

Exxxcellent.  This will come in handy when I get all of the long-polling implementation bugs worked out.  Thanks!
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 31, 2011, 03:23:35 PM
 #117

I've pushed a series of changes that should clear up several issues.  Long-polling should be much more stable now; I've been running my miner all night and have seen plenty of new block notifications in the miner log, and a significant reduction in rejected shares.

To address Deepbit specifically, it has started returning an absolute URL in the X-Long-Polling header, and the proxy did not support this.  Now it does.  Note that Deepbit is currently infinitely redirecting long-polling requests for some reason; I am going to contact Tycho about this and see what's up.  In the meantime, long-polling against Deepbit will not work -- but it will appear to!  The proxy will now sleep for 30 minutes after a failed long-polling request and then pretend that it was a getwork request.  This is because some miners will give up on long-polling altogether if any long-polling request ever fails.  By pretending that the request succeeded, the miner will continue to send long-polling requests, so when long-polling starts working again (or the proxy fails-over to a different pool) the miner will begin benefiting from long-polling again without needing to be restarted.

I believe that some of these problems were the cause of Phoenix hammering Apache with long-polling requests.  Those of you who were having trouble with Phoenix, consider giving the proxy another go with the latest code.

Please thoroughly stress-test the code in the repository and let me know what other issues you find.  Smiley

UPDATE: Tycho fixed long-polling.  The proxy should now work correctly with Deepbit.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
kjj
Legendary
*
Offline Offline

Activity: 1302



View Profile
May 31, 2011, 03:31:20 PM
 #118

Any idea why my one Diablo miner occasionally goes idle when using the proxy?  Seems to happen at random, sometimes a few minutes, sometimes many hours.

p2pcoin: a USB/CD/PXE p2pool miner - 1N8ZXx2cuMzqBYSK72X4DAy1UdDbZQNPLf - todo
I routinely ignore posters with paid advertising in their sigs.  You should too.
twmz
Hero Member
*****
Offline Offline

Activity: 673



View Profile
May 31, 2011, 05:19:36 PM
 #119

I believe that some of these problems were the cause of Phoenix hammering Apache with long-polling requests.  Those of you who were having trouble with Phoenix, consider giving the proxy another go with the latest code.

There was also a bug in Phoenix with LP urls that used query strings. Not sure if that may also make a difference.

http://forum.bitcoin.org/index.php?topic=6458.msg152263#msg152263


Was I helpful?  1TwmzX1wBxNF2qtAJRhdKmi2WyLZ5VHRs
WoT, GPG

Bitrated user: ewal.
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
May 31, 2011, 06:11:15 PM
 #120

There was also a bug in Phoenix with LP urls that used query strings. Not sure if that may also make a difference.

http://forum.bitcoin.org/index.php?topic=6458.msg152263#msg152263
Good catch.  Yeah, that would cause problems.  I might work around this by using PATH_INFO to convey the required data.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
kjj
Legendary
*
Offline Offline

Activity: 1302



View Profile
May 31, 2011, 06:28:26 PM
 #121

So, I got sick of MySQL's buggy time zone handling and made an improvement to my script.  For this to work, you need to add apc_port as an INTEGER DEFAULT 0 column to the worker table.  Set the port to toggle when reboot is requested, or leave at zero to ignore that worker.

Code: (improved reboot script)
#!/bin/php
<?php

$threshold
=15*60;

$con=mysql_connect("__DB_HOST__","__DB_USER__","__DB_PASS__");
mysql_select_db("__DATABASE__",$con);

$restarts=array();

$q="SELECT * FROM worker WHERE apc_port!=0";
$r=mysql_query($q);
while(
$row=mysql_fetch_assoc($r)){
 
$restarts[$row['id']]=$row['apc_port'];
}

reset($restarts);
while(list(
$key,$val)=each($restarts)){
 
$q="SELECT * FROM work_data WHERE worker_id=".$key." ORDER BY time_requested DESC LIMIT 1";
 
$r=mysql_query($q);
 if(
0!=mysql_num_rows($r)){
  
$row=mysql_fetch_assoc($r);
  
$lastts=strtotime($row['time_requested']." GMT");
  if((
time()-$lastts)>$threshold){
   
$url="http://__CGI_HOST__/cgi-bin/apc_restart.cgi?".$val;
   
$junk=file($url);
  }
 }
}

?>


p2pcoin: a USB/CD/PXE p2pool miner - 1N8ZXx2cuMzqBYSK72X4DAy1UdDbZQNPLf - todo
I routinely ignore posters with paid advertising in their sigs.  You should too.
error
Hero Member
*****
Offline Offline

Activity: 574



View Profile
June 01, 2011, 01:38:14 AM
 #122

There was also a bug in Phoenix with LP urls that used query strings. Not sure if that may also make a difference.

http://forum.bitcoin.org/index.php?topic=6458.msg152263#msg152263
Good catch.  Yeah, that would cause problems.  I might work around this by using PATH_INFO to convey the required data.

Thanks. It was because of this proxy that I caught it. Smiley

15UFyv6kfWgq83Pp3yhXPr8rknv9m6581W
twmz
Hero Member
*****
Offline Offline

Activity: 673



View Profile
June 01, 2011, 05:25:03 AM
 #123

I do, I definitely want to see the code. Smiley

My alternate implementation in ASP.NET, as it exists today is now published here: http://code.google.com/p/btcproxy/.  It has been functioning well for me over the past 4-5 days, but I have only tested it with phoenix miner, poclbm, and the pools shown in the screenshot.

Right now, I have not written any documentation on how to install it, so you'll need to be comfortable with ASP.NET development and SQL to get it deployed.  I'll work on a brief readme.txt over the next couple days that spells out the prerequisites and general installation steps.

As I make additional improvements or refinements, I'll update the code on that website and not hijack this thread any further.

Was I helpful?  1TwmzX1wBxNF2qtAJRhdKmi2WyLZ5VHRs
WoT, GPG

Bitrated user: ewal.
burtyb
Jr. Member
*
Offline Offline

Activity: 45



View Profile WWW
June 01, 2011, 11:56:19 AM
 #124

I'm not using PHP > 5.3 yet (CentOS 5.x) so I had to make a little change to get it to run without the DateTime class.

Basically I've replaced (format_date function in common.inc.php)
Code:
$obj = new DateTime($date, new DateTimeZone('UTC'));
$obj->setTimezone(new DateTimeZone($BTC_PROXY['timezone']));
return $obj->format($BTC_PROXY['date_format']);

with
Code:
return date($BTC_PROXY['date_format'],strtotime($date));

I didn't bother with the timezone as it looks like it's only used to display but that might be fixed with something like

Code:
date_default_timezone_set($BTC_PROXY['timezone']);

ChrisB.

ciuciu
Donator
Hero Member
*
Offline Offline

Activity: 588


View Profile
June 01, 2011, 06:45:00 PM
 #125

Hi,

I try to setup the proxy but I can't connect to it using the phoenix miner. I did setup the pools and workers.
I'm using start /D C:\phoenix phoenix -u http://user:pass@mywebspace.com:80/ -k poclbm device=1 VECTORS BFI_INT FASTLOOP AGGRESSION=7 without success.

Can somebody show me an example of the phoenix command line for connecting it correctly to the web proxy?

Thanks.

ciuciu
Donator
Hero Member
*
Offline Offline

Activity: 588


View Profile
June 01, 2011, 08:08:20 PM
 #126

Tried now and still not working.

Thanks.

btcLeger
Member
**
Offline Offline

Activity: 98


View Profile
June 01, 2011, 10:15:12 PM
 #127

nice idea but im getting lots of blank pages when handling the backend. This happens when
a) editing a pool
b) saving worker-pool managemant goes to worker-pool.php seems missing parameters (would be nice to jump back to actual woker overview page e.g. ?id=1&action=index

xampp with PHP 5.2.4

I use TradeHill instant BTC trading: http://www.tradehill.com/?r=TH-R11320

donations: 12BdMNcn7VQitFGMhtRojvwXbrX3Jcs9U6
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
June 02, 2011, 12:39:56 AM
 #128

nice idea but im getting lots of blank pages when handling the backend.

xampp with PHP 5.2.4
I have not tested with many versions, but others have reported that at least PHP 5.3 is required to use the proxy.  I will add this to the readme at some point.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
June 02, 2011, 11:12:02 AM
 #129

nice idea but im getting lots of blank pages when handling the backend.

xampp with PHP 5.2.4
I have not tested with many versions, but others have reported that at least PHP 5.3 is required to use the proxy.  I will add this to the readme at some point.

I can also confirm many blank pages and strangeness under Cent 5.x with the default PHP version 5.1.x.  A stated dependency on PHP 5.3 would go a long way I'm sure.
carlo
Full Member
***
Offline Offline

Activity: 132


View Profile
June 02, 2011, 01:50:46 PM
 #130

Hello everybody,

i love your proxy its nice to have a central control unit.

There are some minor problems:
It seems a bit slow for me, often my miners (poclbm) report this:  warning: job finished, miner is idle

Ideas:
Maybe i need to pimp the apache.conf a bit. Not sure right now, my System: Debian 5 / apache
- Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny9 with Suhosin-Patch mod_python/3.3.1 Python/2.5.2 mod_ssl/2.2.9 OpenSSL/0.9.8g mod_wsgi/2.5

Theres a new problem with they nice proxy, it happens with different pools, this results in a lot of failed work submissions:
Code:
02/06/2011 08:56:10, Unexpected error:
Traceback (most recent call last):
  File "BitcoinMiner.pyo", line 165, in mine
  File "BitcoinMiner.pyo", line 208, in getwork
  File "BitcoinMiner.pyo", line 235, in request
TypeError: string indices must be integers

Maybe its all apaches fault Smiley

I hope i will figure it out soon, thanks for your time spending in this project.

burtyb
Jr. Member
*
Offline Offline

Activity: 45



View Profile WWW
June 02, 2011, 11:21:32 PM
 #131

I have not tested with many versions, but others have reported that at least PHP 5.3 is required to use the proxy.  I will add this to the readme at some point.

Using my changes above I've been using it with PHP 5.1.6 with no apparent problems, do you know of any other reasons why it requires PHP 5.3?

ChrisB.

kjj
Legendary
*
Offline Offline

Activity: 1302



View Profile
June 03, 2011, 03:18:12 PM
 #132

This join makes me sad.

Code: ("sad join")
SELECT
                w.name AS worker,
                w.id AS worker_id,

                worked.pool_name AS active_pool,
                worked.latest - INTERVAL 5 HOUR AS active_time,

                submitted.pool_name AS last_accepted_pool,
                submitted.latest - INTERVAL 5 HOUR AS last_accepted_time,

                w.last_reboot AS last_reboot

            FROM worker w

            LEFT OUTER JOIN (
                SELECT
                    wd.worker_id AS worker_id,
                    wd.time_requested AS latest,
                    p.name AS pool_name

                FROM work_data wd

                INNER JOIN (
                    SELECT
                        worker_id,
                        MAX(time_requested) AS latest

                    FROM work_data

                    GROUP BY worker_id
                ) wd2
                    ON wd.worker_id = wd2.worker_id
                   AND wd.time_requested = wd2.latest

                INNER JOIN pool p
                    ON p.id = wd.pool_id

                GROUP BY wd.worker_id
            ) worked

            ON worked.worker_id = w.id

            LEFT OUTER JOIN (
                SELECT
                    sw.worker_id AS worker_id,
                    sw.time AS latest,
                    p.name AS pool_name

                FROM submitted_work sw

                INNER JOIN (
                    SELECT
                        worker_id,
                        MAX(time) AS latest

                    FROM submitted_work

                    WHERE result = 1

                    GROUP BY worker_id
                ) sw2
                    ON sw.worker_id = sw2.worker_id
                   AND sw.result = 1
                   AND sw.time = sw2.latest

                INNER JOIN pool p
                    ON p.id = sw.pool_id

                GROUP BY sw.worker_id
            ) submitted

            ON submitted.worker_id = w.id

            ORDER BY w.name

Solution?

Code: ("delta.sql")
ALTER TABLE work_data DROP PRIMARY KEY;
ALTER TABLE work_data ADD id INT NOT NULL PRIMARY KEY AUTO_INCREMENT FIRST;
ALTER TABLE work_data ADD KEY (`worker_id`, `data`);

CREATE TABLE `work_data_history` (
  `id` int(11) NOT NULL PRIMARY KEY AUTO_INCREMENT,
  `worker_id` int(11) NOT NULL,
  `pool_id` int(11) NOT NULL,
  `data` char(152) character set ascii collate ascii_bin NOT NULL,
  `time_requested` datetime NOT NULL,
  KEY  (`worker_id`,`data`),
  KEY `worker_time` (`worker_id`,`time_requested`)
);

CREATE TABLE `submitted_work_history` (
  `id` int(11) NOT NULL auto_increment,
  `worker_id` int(11) NOT NULL,
  `pool_id` int(11) NOT NULL,
  `result` tinyint(1) NOT NULL,
  `time` datetime NOT NULL,
  PRIMARY KEY  (`id`),
  KEY `dashboard_status_index` (`worker_id`,`result`,`time`)
);

Code: ("new cron job, hourly")
<?php

$con
=mysql_connect("__DB_HOST__","__DB_USERNAME__","__DB_PASSWORD__");
mysql_select_db("__DB_DATABASE__",$con);

$q="SELECT MAX(id) AS mid FROM work_data_history";
$r=mysql_query($q);
if(
mysql_errno()!=0)echo mysql_error()."\n";
$row=mysql_fetch_assoc($r);
$mid=$row['mid'];

$q="INSERT INTO work_data_history SELECT * FROM work_data WHERE id>".$mid;
mysql_query($q);
if(
mysql_errno()!=0)echo mysql_error()."\n";

$q="DELETE FROM work_data WHERE time_requested < (NOW() - INTERVAL 12 HOUR)";
mysql_query($q);
if(
mysql_errno()!=0)echo mysql_error()."\n";

$q="SELECT MAX(id) AS mid FROM submitted_work_history";
$r=mysql_query($q);
if(
mysql_errno()!=0)echo mysql_error()."\n";
$row=mysql_fetch_assoc($r);
$mid=$row['mid'];

$q="INSERT INTO submitted_work_history SELECT * FROM submitted_work WHERE id>".$mid;
mysql_query($q);
if(
mysql_errno()!=0)echo mysql_error()."\n";

$q="DELETE FROM submitted_work WHERE time < (NOW() - INTERVAL 12 HOUR)";
mysql_query($q);
if(
mysql_errno()!=0)echo mysql_error()."\n";
?>


p2pcoin: a USB/CD/PXE p2pool miner - 1N8ZXx2cuMzqBYSK72X4DAy1UdDbZQNPLf - todo
I routinely ignore posters with paid advertising in their sigs.  You should too.
Ersan
Newbie
*
Offline Offline

Activity: 5


View Profile
June 03, 2011, 05:09:37 PM
 #133

Any news on the high rejection rate?  That's the only thing keeping me from using this software, the idea behind it is fantastic!
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
June 04, 2011, 05:50:28 AM
 #134

Any news on the high rejection rate?  That's the only thing keeping me from using this software, the idea behind it is fantastic!

You're saying the long polling apparatus is still problematic, even with the latest code?
Ersan
Newbie
*
Offline Offline

Activity: 5


View Profile
June 05, 2011, 01:09:08 AM
 #135

I think it's a problem related to BTCGuild specifically.  Slush's pool seems alright - I'll keep you posted.
Disposition
Full Member
***
Offline Offline

Activity: 122


View Profile
June 05, 2011, 01:28:50 AM
 #136

everything is running fine except after awhile it just times out for all my clients, feels like there's some apache config problem.

I noticed earlier post you mentioned something about safe mode, and that's off by default so I'm a bit confused to why this is happening,

appreciated if you can look into it

edit: it seems like this only applies to btcguild for some reason.
Ersan
Newbie
*
Offline Offline

Activity: 5


View Profile
June 05, 2011, 06:49:20 AM
 #137

I think btcguild is just problematic.  I've moved on to other pools until they get their act together.
gigabytecoin
Sr. Member
****
Offline Offline

Activity: 280


View Profile
June 05, 2011, 06:51:45 AM
 #138

Keep up the great work OP!
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
June 05, 2011, 07:12:40 PM
 #139

everything is running fine except after awhile it just times out for all my clients, feels like there's some apache config problem.

I noticed earlier post you mentioned something about safe mode, and that's off by default so I'm a bit confused to why this is happening,

appreciated if you can look into it

edit: it seems like this only applies to btcguild for some reason.

For long polling to work correctly through the proxy, you'd need (at minimum) to have a long execution time in php.ini.  Something like this:

Code:
max_execution_time=7500

That assumes it would take 2 hours (probably a maximum of maximums) to find a block, so I added an extra 5 minutes for additional padding.  In actuality I've never seen it take more than 40 or 50 minutes to find a block though, so 7500 should be perfectly fine.  Note that modifying php.ini is a global setting, so you might be able to/might want to modify this only for the virtualhost that runs your proxy, depending on your specific OS and apache/php implementation.  Don't forget to restart httpd to make the setting take effect.

Also there might be other settings that need to be modified, as it's really not normal to have an http connection open for 2 hours.  But this one is the most obvious.
Disposition
Full Member
***
Offline Offline

Activity: 122


View Profile
June 06, 2011, 12:25:55 AM
 #140

@pwnyboy

Thanks, will stress test n run this for a bit.
Disposition
Full Member
***
Offline Offline

Activity: 122


View Profile
June 07, 2011, 04:57:11 AM
 #141

much better now, except I'm still getting a weird unable to auth error some times (I assume due to the time out between proxy and server)

the proxy(NJ based), btcmine(NY) based, Me(NYC) is really close, the ping is great so I think it's something with this script.

pwnyboy any other php.ini configuration I should know about it?
btcLeger
Member
**
Offline Offline

Activity: 98


View Profile
June 07, 2011, 06:04:35 PM
 #142

Ok with php 5.3.5 things work better but there is still a blank page when editing a worker. Changed name or password is not saved.

I've let it run for 2 hours with 4 workers and 1 Ghash/s total and got 3% rejected shares with guiminer using poclbm no matter which pool I join. The rate of stale shares is 10 times higher on deepbit when using the proxy.

Besides of that I like the idea and the concept. I will use the proxy and donate if the stale shares problem gets solved.

Features I'd love to have:
- autorefresh for dashboard
- workerstatus with configureable timeout display (green/red row background)

I use TradeHill instant BTC trading: http://www.tradehill.com/?r=TH-R11320

donations: 12BdMNcn7VQitFGMhtRojvwXbrX3Jcs9U6
Disposition
Full Member
***
Offline Offline

Activity: 122


View Profile
June 08, 2011, 04:53:04 AM
 #143

bump, seriously if this technical problem is solved and timing-out auth issue is fixed, I put a 5btc bounty on it.

edit: nvm I fixed it.
exfuga
Newbie
*
Offline Offline

Activity: 1


View Profile
June 08, 2011, 05:41:00 AM
 #144

Love, love, love the idea!

One thing, on the main admin page I get this error, and it only displays half of the table.

"This page contains the following errors:

error on line 37 at column 13: Extra content at the end of the document
Below is a rendering of the page up to the first error."

I am on PHP 5.3.2
Papipapito
Newbie
*
Offline Offline

Activity: 8



View Profile
June 08, 2011, 04:38:28 PM
 #145

Hello

Have a problem with a miner Ufasoft, with diablo and poclbm work fine

The miner connecting to proxy and start a work, but  in 5 minutes generate a timeout and miner stop

the log of apache:


 - - [08/Jun/2011:18:06:51 +0200] "POST / HTTP/1.1" 401 484 "-" "Ufasoft bitcoin-miner/0.10 CPU/Intel(R) Pentium(R) 4 CPU 3.00GHz/f43 PC=2 CC=1"
 - 2 [08/Jun/2011:18:06:51 +0200] "POST / HTTP/1.1" 400 392 "-" "Ufasoft bitcoin-miner/0.10 CPU/Intel(R) Pentium(R) 4 CPU 3.00GHz/f43 PC=2 CC=1"
 - 2 [08/Jun/2011:18:07:11 +0200] "POST / HTTP/1.1" 200 1015 "-" "Ufasoft bitcoin-miner/0.10 CPU/Intel(R) Pentium(R) 4 CPU 3.00GHz/f43 PC=2 CC=1"
 And now work and in 10 minutes, timeout and not appears nothing in logs

Any ideas?



Sorry for  my bad english

 *Link Removed*
tito13kfm
Jr. Member
*
Offline Offline

Activity: 42



View Profile
June 09, 2011, 06:06:48 AM
 #146

So awesome!  Donation sent!

I look forward to future updates.  Would love graphs!  I don't even know what kind, but I love graphs!

pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
June 09, 2011, 06:30:39 AM
 #147

bump, seriously if this technical problem is solved and timing-out auth issue is fixed, I put a 5btc bounty on it.

edit: nvm I fixed it.

For the benefit of others, and because it's unclear whether a long polling problem still exists with the proxy code, would you care to share the nature of your problem and the solution?
Azelphur
Jr. Member
*
Offline Offline

Activity: 43


View Profile
June 10, 2011, 07:49:25 PM
 #148

Anyone else care to check the contents of their /var/lib/php5? after running this for a while I started getting no disk space errors, upon further investigation I found out that PHP had created 15.5 million session files in /var/lib/php5. Fun.

Should anyone else bump into this problem, you'll probably notice that nothing can delete the files (rm, ls, find etc all hang) after much fiddling, the solution is to use ls -U1, which will print the file list unsorted, and then pipe the output to rm, like so. ls -U1 /var/lib/php5 | xargs rm -f

Gonna take me 5 days of rm to clean up all the session files, fun fun Cheesy

Besides this bug (which may just be some issue on my end) it seems great, I love how I can have pool redundancy in case one goes down.
btcLeger
Member
**
Offline Offline

Activity: 98


View Profile
June 11, 2011, 06:12:25 PM
 #149

Anyone else care to check the contents of their /var/lib/php5? after running this for a while I started getting no disk space errors, upon further investigation I found out that PHP had created 15.5 million session files in /var/lib/php5. Fun.

Should anyone else bump into this problem, you'll probably notice that nothing can delete the files (rm, ls, find etc all hang) after much fiddling, the solution is to use ls -U1, which will print the file list unsorted, and then pipe the output to rm, like so. ls -U1 /var/lib/php5 | xargs rm -f

Gonna take me 5 days of rm to clean up all the session files, fun fun Cheesy

Besides this bug (which may just be some issue on my end) it seems great, I love how I can have pool redundancy in case one goes down.
cannot confirm this. have the proxy running under xampp. Yes, there are loads of sessions generated in the tmp-folder (about 10 per minute per miner) but expired session are getting deleted by the system. Must be a misconfigured serverside variable. Its not uncommon, Ive seen this often on some webservers when millions of sessionfiles slow down webservices. But dont ask me for a solution. Thats work for those who configure servers...  Grin

I use TradeHill instant BTC trading: http://www.tradehill.com/?r=TH-R11320

donations: 12BdMNcn7VQitFGMhtRojvwXbrX3Jcs9U6
btcLeger
Member
**
Offline Offline

Activity: 98


View Profile
June 11, 2011, 06:23:36 PM
 #150

Another suggestion for improved usability: I find it a bit unhandy to add new workers and pools when already having lots of workers & pools in the database. Would be nice to have a worker-edit page to configure all existing pools at once without the need to edit them all individually. Same with the pools. when adding a new pool, would be nice to update all existing workers for the pool on one page instead of picking them individually. Thats 30+ mouseclicks, where 3 would be sufficient...

I use TradeHill instant BTC trading: http://www.tradehill.com/?r=TH-R11320

donations: 12BdMNcn7VQitFGMhtRojvwXbrX3Jcs9U6
jondecker76
Full Member
***
Offline Offline

Activity: 238


View Profile
June 12, 2011, 10:43:43 AM
 #151

I just finished setting up the bitcoin-mining-proxy.  I have the mysql database working, and have the pools and workers set up.  Now I have a couple of questions!

1)  When I go to my <ip_of_proxy> it asks for username and password, but they don't work there like they do at <ip_of_proxy>/admin.  Is this normal?

2)  How do I point my miner to the proxy?  I couldn't find anything in the documentation??  Since its on the same host, I'm sure its going to be 127.0.0.1 - but what port??  If I use 8332, it will interfere with the locally running bitcoind, correct?

So far very impressive!  Help me to point my miners (running phoenix r100) to the proxy and I'll post back results!

thanks again

RollerBot Advanced Trading Platform
https://bitcointalk.org/index.php?topic=447727.0
BTC Donations for development: 1H36oTJsi3adFh68wwzz95tPP2xoAoTmhC
Cdecker
Hero Member
*****
Offline Offline

Activity: 487



View Profile WWW
June 12, 2011, 10:46:49 AM
 #152

I just finished setting up the bitcoin-mining-proxy.  I have the mysql database working, and have the pools and workers set up.  Now I have a couple of questions!

1)  When I go to my <ip_of_proxy> it asks for username and password, but they don't work there like they do at <ip_of_proxy>/admin.  Is this normal?

2)  How do I point my miner to the proxy?  I couldn't find anything in the documentation??  Since its on the same host, I'm sure its going to be 127.0.0.1 - but what port??  If I use 8332, it will interfere with the locally running bitcoind, correct?

So far very impressive!  Help me to point my miners (running phoenix r100) to the proxy and I'll post back results!

thanks again
1) the /admin you use the user and pass from your config, while / is for the workers to contact, so there you'd use the miners username and password
2) Since you run the proxy on a webserver you'll probably have to point the workers to http://<ip>:80/ Cheesy

HTH,
cdecker

Want to see what developers are chatting about? http://bitcoinstats.com/irc/bitcoin-dev/logs/
Bitcoin-OTC Rating
jondecker76
Full Member
***
Offline Offline

Activity: 238


View Profile
June 12, 2011, 10:54:44 AM
 #153

I thank you, one Decker to another Tongue

Worked great!  Seamless failover for me as deepbit is priority 1, but it is down so its working great with the next pool in line - excellent!
Hash rate is normal - so far so good

RollerBot Advanced Trading Platform
https://bitcointalk.org/index.php?topic=447727.0
BTC Donations for development: 1H36oTJsi3adFh68wwzz95tPP2xoAoTmhC
jondecker76
Full Member
***
Offline Offline

Activity: 238


View Profile
June 12, 2011, 11:54:52 AM
 #154

is priority 1 higher than priority 2, or is priority 2 higher than priority 1?

RollerBot Advanced Trading Platform
https://bitcointalk.org/index.php?topic=447727.0
BTC Donations for development: 1H36oTJsi3adFh68wwzz95tPP2xoAoTmhC
jondecker76
Full Member
***
Offline Offline

Activity: 238


View Profile
June 12, 2011, 12:10:47 PM
 #155

didn't work well for me at all......  Had 3 pools set up, and 3 workers set up (I have 3 5850's, so a worker for each)..

priorities...
Worker1:
1 bitcoin.cz
2 btcguild.com
3 deepbit.net

worker 2:
1 btcguild.com
2 deepbit.net
3 bitcoin.cz

worker 3
1 deepbit.net
2 bitcoin.cz
3 btcguild.com


All 3 of my cards were hashing for bitcoin.cz for some reason, even though all 3 pools were up...  So each card should have been going to a separate pool........
The other problem is, after a few minutes, phoenix would display that the work queue was empty and hashes would drop to 0 (on all 3 cards/workers) and hang there....

So, so far, not so good unfortunately Sad
I now have each card manually hashing each of the 3 pools listed above, and they all work fine..... Just not through the proxy.
Wil experiment some more later

RollerBot Advanced Trading Platform
https://bitcointalk.org/index.php?topic=447727.0
BTC Donations for development: 1H36oTJsi3adFh68wwzz95tPP2xoAoTmhC
kjj
Legendary
*
Offline Offline

Activity: 1302



View Profile
June 12, 2011, 03:02:21 PM
 #156

is priority 1 higher than priority 2, or is priority 2 higher than priority 1?

2 will be used before 1.

Better idea is to space them out by 10s, or even 20s, so that you can squeeze pools in between without having to renumber them all.

p2pcoin: a USB/CD/PXE p2pool miner - 1N8ZXx2cuMzqBYSK72X4DAy1UdDbZQNPLf - todo
I routinely ignore posters with paid advertising in their sigs.  You should too.
jgarzik
Legendary
*
Offline Offline

Activity: 1484


View Profile
June 12, 2011, 04:14:15 PM
 #157

See also https://github.com/kylegibson/poclbm for a poclbm fork that offers fallback support.  A bit easier than a proxy.

Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
kjj
Legendary
*
Offline Offline

Activity: 1302



View Profile
June 12, 2011, 04:20:57 PM
 #158

didn't work well for me at all......  Had 3 pools set up, and 3 workers set up (I have 3 5850's, so a worker for each)..

priorities...
Worker1:
1 bitcoin.cz
2 btcguild.com
3 deepbit.net

worker 2:
1 btcguild.com
2 deepbit.net
3 bitcoin.cz

worker 3
1 deepbit.net
2 bitcoin.cz
3 btcguild.com


All 3 of my cards were hashing for bitcoin.cz for some reason, even though all 3 pools were up...  So each card should have been going to a separate pool........
The other problem is, after a few minutes, phoenix would display that the work queue was empty and hashes would drop to 0 (on all 3 cards/workers) and hang there....

So, so far, not so good unfortunately Sad
I now have each card manually hashing each of the 3 pools listed above, and they all work fine..... Just not through the proxy.
Wil experiment some more later


Check your URLs, usernames, and passwords.

Every time that I've run into a problem like this, it was because one or more pools was down, or because I had botched my configuration.

p2pcoin: a USB/CD/PXE p2pool miner - 1N8ZXx2cuMzqBYSK72X4DAy1UdDbZQNPLf - todo
I routinely ignore posters with paid advertising in their sigs.  You should too.
jondecker76
Full Member
***
Offline Offline

Activity: 238


View Profile
June 12, 2011, 05:37:50 PM
 #159

very strange..  it works fine if there is only one pool, but when I add multiples, it simply doesn't work.  I have also noticed that my stale/rejected shares went up extremely high...  In deepbit, for the last month I have been under 1% stales.  Using the proxy, I'm at almost 20% rejected shares Sad

I was really hoping that this would work

RollerBot Advanced Trading Platform
https://bitcointalk.org/index.php?topic=447727.0
BTC Donations for development: 1H36oTJsi3adFh68wwzz95tPP2xoAoTmhC
pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
June 12, 2011, 10:13:39 PM
 #160

very strange..  it works fine if there is only one pool, but when I add multiples, it simply doesn't work.  I have also noticed that my stale/rejected shares went up extremely high...  In deepbit, for the last month I have been under 1% stales.  Using the proxy, I'm at almost 20% rejected shares Sad

I was really hoping that this would work

The stale shares you're referring to are because of the aforementioned long-polling issues.  Nobody is sure what makes it work, or not work, through the current codebase.  The best suggestion I can give you is to run something like PHProxy at http://url.of.your.minining.proxy/phproxy/ and edit the code to send out this URL to your miners, rather than trying to use the in-built broken proxying code.  At least I think that'll work, from what I recall of the code and the way that LP works.  If you go this route, report back and let us know how it works out.
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
June 13, 2011, 04:58:20 AM
 #161

Anyone else care to check the contents of their /var/lib/php5? after running this for a while I started getting no disk space errors, upon further investigation I found out that PHP had created 15.5 million session files in /var/lib/php5. Fun.

Should anyone else bump into this problem, you'll probably notice that nothing can delete the files (rm, ls, find etc all hang) after much fiddling, the solution is to use ls -U1, which will print the file list unsorted, and then pipe the output to rm, like so. ls -U1 /var/lib/php5 | xargs rm -f

Gonna take me 5 days of rm to clean up all the session files, fun fun Cheesy

Besides this bug (which may just be some issue on my end) it seems great, I love how I can have pool redundancy in case one goes down.
Thanks for the report.  This was indeed a bug, and has been fixed in master.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
kjj
Legendary
*
Offline Offline

Activity: 1302



View Profile
June 13, 2011, 05:07:42 AM
 #162

If you aren't using /var/lib/php5 for your session files, do "php -i | grep session.save_path" on the command line, or make a file with:

Code:
<?php
phpinfo
();
?>


In your document tree, and load it in your browser, then look for session.save_path.

An alternative removal method is:

Code:
find /tmp/ -name sess_\* -exec rm {} \;

p2pcoin: a USB/CD/PXE p2pool miner - 1N8ZXx2cuMzqBYSK72X4DAy1UdDbZQNPLf - todo
I routinely ignore posters with paid advertising in their sigs.  You should too.
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
June 13, 2011, 05:09:00 AM
 #163

very strange..  it works fine if there is only one pool, but when I add multiples, it simply doesn't work.  I have also noticed that my stale/rejected shares went up extremely high...  In deepbit, for the last month I have been under 1% stales.  Using the proxy, I'm at almost 20% rejected shares Sad

I was really hoping that this would work
That's weird.  Sad  The code path isn't any different if you have one or more pools configured...

If any of you are still having stale share issues, would you mind taking a capture of the network traffic?  On the proxy host, issue this as root:

Code:
tcpdump -w proxy.pcap -s 65535 'port 80 or portrange 8332-8337'

Then email me the proxy.pcap file after an hour or two.  Note that this file will contain all of the traffic on these ports and this means that I will have access to your proxy worker passwords as well as your pool worker passwords.  You may certainly encrypt the file to my GPG key if you are worried about the file being intercepted.  (Specifically, note that if you talk to a bitcoind from the web server, this will expose your RPC password!  So be especially careful in this case.  I'm not going to misuse this information, but... well, if I were you, I wouldn't trust me blindly either. Wink)

If possible, capture traffic from the workers to the pools directly, without the proxy, and email me that as a separate file.  Comparing the two files will help me determine what's going on.

Even better, if you're using a pool supporting long-polling, have one miner going through the proxy and at the same time another one not using the proxy, and mail me both files.  Seeing how the long-polling requests return in both cases (they should be identical...) will help me diagnose this.

Thanks all for the feedback, donations, and support!  It's heartening to know that the software is fulfilling its purpose and that people find it helpful.  Smiley

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
tito13kfm
Jr. Member
*
Offline Offline

Activity: 42



View Profile
June 13, 2011, 05:24:49 AM
 #164

Rebooted after crash and now Worker Status table on main dashboard is completely blank.  Recent work submissions and recent failed submissions still show up just fine in the top 2 tables, but the bottom table is blank.

Any ideas?

Edit:  I was also having connection issues.  I exported my workers and pools settings.  Imported the original schema back, and imported those saved settings.  All is well now.

cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
June 13, 2011, 02:08:53 PM
 #165

Rebooted after crash and now Worker Status table on main dashboard is completely blank.  Recent work submissions and recent failed submissions still show up just fine in the top 2 tables, but the bottom table is blank.

Any ideas?

Edit:  I was also having connection issues.  I exported my workers and pools settings.  Imported the original schema back, and imported those saved settings.  All is well now.
Sounds like the MySQL database got corrupted.  Which is weird, since table logs are supposed to keep that from happening.  :/

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
teknohog
Sr. Member
****
Offline Offline

Activity: 471


555


View Profile WWW
June 13, 2011, 09:54:12 PM
 #166

Has anyone got this working with Lighttpd (under Gentoo)? I am trying to rule out wider system-level errors before digging deeper into this code. Basically, my clients (Phoenix) are connecting to the proxy, but they are not getting any work. My PHP setup should be OK as I use it for other web content, and the admin interface seems to work fine. I have set this up in a separate port so there are no subdirectory issues for clients.

Freakin
Full Member
***
Offline Offline

Activity: 154


View Profile
June 13, 2011, 09:59:27 PM
 #167

Parse error: syntax error, unexpected T_STRING, expecting T_OLD_FUNCTION or T_FUNCTION or T_VAR or '}' in /home/REDACTED/public_html/admin/index.php on line 28

Hmm, not sure why i'm getting this.  This is on the admin page.  It complains about line 125 on the root
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
June 14, 2011, 02:05:30 PM
 #168

Parse error: syntax error, unexpected T_STRING, expecting T_OLD_FUNCTION or T_FUNCTION or T_VAR or '}' in /home/REDACTED/public_html/admin/index.php on line 28

Hmm, not sure why i'm getting this.  This is on the admin page.  It complains about line 125 on the root
Which version of PHP?  It's possible that your version does not support function access modifiers.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
Freakin
Full Member
***
Offline Offline

Activity: 154


View Profile
June 14, 2011, 04:23:10 PM
 #169

Parse error: syntax error, unexpected T_STRING, expecting T_OLD_FUNCTION or T_FUNCTION or T_VAR or '}' in /home/REDACTED/public_html/admin/index.php on line 28

Hmm, not sure why i'm getting this.  This is on the admin page.  It complains about line 125 on the root
Which version of PHP?  It's possible that your version does not support function access modifiers.

4.4.8

5.x required?
exahash
Sr. Member
****
Offline Offline

Activity: 274



View Profile
June 14, 2011, 05:20:41 PM
 #170

Do you want patches?  I'm testing putting 50 miners through this proxy and was running into stability issues.  I have mysql on a separate machine and am used to that being a bottleneck, so I looked into making it use persistent connections.  I am not familiar with the PDO extension, but found the instantiator in common.inc.php on line 31; I added array(PDO::ATTR_PERSISTENT => true) to the new call and this took care of my problems.

The whole line now looks like this:

return new PDO($BTC_PROXY['db_connection_string'], $BTC_PROXY['db_user'], $BTC_PROXY['db_password'], array(PDO::ATTR_PERSISTENT => true) );

I now have 32 connections from the proxy web server staying open (which is what I want).

Hopefully this helps someone.


btcLeger
Member
**
Offline Offline

Activity: 98


View Profile
June 14, 2011, 08:23:40 PM
 #171

4.4.8

5.x required?
5.3 at least.

I use TradeHill instant BTC trading: http://www.tradehill.com/?r=TH-R11320

donations: 12BdMNcn7VQitFGMhtRojvwXbrX3Jcs9U6
Freakin
Full Member
***
Offline Offline

Activity: 154


View Profile
June 14, 2011, 11:17:57 PM
 #172

Cool got it working on a Centos 5.6 VM w/ 5.3.6

had a bit of difficulty getting PDO installed but it went ok.

looking forward to getting this working!
Freakin
Full Member
***
Offline Offline

Activity: 154


View Profile
June 14, 2011, 11:45:26 PM
 #173

There should be a way to copy worker/pool configs

so instead of setting up 10x pools on 10x workers you can set it up once then copy it to the rest, then go in and change the logins as necessary.  Or a way to use a variable in the config so you build X workers with the suffix X changed on each one

krom
Newbie
*
Offline Offline

Activity: 12


View Profile
June 14, 2011, 11:46:42 PM
 #174

All working "quite" well with 1 miner and 1 pool.

Some things i've noticed:
  • I have a lot of "Listener for "Proxy": 15/06/2011 00:23:30, Unexpected error:" lines on GUIMiner's console, using poclbm and the proxy.
  • Long polls seem to get through sometimes, but not all the time - i get a lot more of these when connecting to the pool directly, but maybe this was random.
  • Fall back to other pools does not work for me, nor will the miner work on any other pool than the first one i created, even if i disable that pool overall or remove it from the worker's list.
  • A second worker i added is able to get work (only from the first pool i defined as well), but the returned shares never show up anywhere - not on the dashboard nor the pool's stats page. "Last accepted submission" is "Never" for that worker all the time as well.

I'll dig into the problem further, but have no time to do so yet.
Besides that: Good work and keep it up! When i'll have more than some fractions of BC, i'll donate to you. Smiley

Update:
Long poll seem to be okay.
Problem 3 was solved by adding 'http://' to the pool's url. Won't work without.
Freakin
Full Member
***
Offline Offline

Activity: 154


View Profile
June 15, 2011, 03:25:38 AM
 #175

Really wish there was a way to monitor anything on solo mining besides getwork w/ the proxy
kripz
Full Member
***
Offline Offline

Activity: 182



View Profile
June 17, 2011, 01:45:14 PM
 #176

Testing out now, looks fairly solid.

What sort of values can i set for 'average_interval'? and what effects will it have?

Also, please add rejected per hour and efficiency.

Edit: I get alot of miner idle using this...

 Merged mining, free SMS notifications, PayPal payout and much more.
http://btcstats.net/sig/JZCODg2
Freakin
Full Member
***
Offline Offline

Activity: 154


View Profile
June 17, 2011, 04:11:46 PM
 #177

I had problems with this once i connected 4 machines to it (6 total miners).

Tried it soloing and on bitparking and while I slept and found that I wasn't getting what I was supposed to.

turned off both namecoin options and was hitting slush and deepbit and getting about 2/3 of the shares I should have

connections to the proxy started timing out, so restarting apache fixed it, but i didn't trust it enough to keep going

This was on a VM w/ 1GB of ram

What kind of hardware are you guys running on?


pwnyboy
Full Member
***
Offline Offline

Activity: 125


View Profile
June 17, 2011, 06:34:21 PM
 #178

Really wish there was a way to monitor anything on solo mining besides getwork w/ the proxy

There is indeed, it's called pushpool.  But it's a real PITA to build.  Once you have it going, it'll log shares to a text file, which you can use to dig out the info you want/need.  You can also make it log shares to the database backend (SQLite, MySQL etc).
Freakin
Full Member
***
Offline Offline

Activity: 154


View Profile
June 17, 2011, 08:05:35 PM
 #179

Really wish there was a way to monitor anything on solo mining besides getwork w/ the proxy

There is indeed, it's called pushpool.  But it's a real PITA to build.  Once you have it going, it'll log shares to a text file, which you can use to dig out the info you want/need.  You can also make it log shares to the database backend (SQLite, MySQL etc).

yeah I think I'm going to set up a POC pool on a VM for personal use and play around.

After 3 days soloing namecoin with 2GH and not finding a single block I don't plan on doing that again anyways... Wouldn't be so bitter if I hadn't found 23 blocks in the week before (half at ~3k difficulty) for BitParking
carlo
Full Member
***
Offline Offline

Activity: 132


View Profile
June 17, 2011, 10:00:50 PM
 #180

Im testing your proxy now since a week. they main problem seems to be the apache.

i figured out that it does reject lesser shares if i run this settings for the apache:
KeepAlive=On
KeepAliveTimeout=60
MaxKeepAliveRequests=250
MaxClients=100

I run 8 GPU miners.

But its still not the same reject rate if i connect directly to a pool. Im happy if somebody has other ideas, cause the fail-over is so nice in days of ddos.

Problems during the last 2 days:
Code:
ERROR: Cannot connect to Bitcoin: Bitcoin returned error message: Work not found in proxy database.
ERROR: Cannot connect to Bitcoin: Bitcoin returned error message: <?xml version="1.0" encoding

cheers
kripz
Full Member
***
Offline Offline

Activity: 182



View Profile
June 18, 2011, 02:10:38 AM
 #181

Is it safe to drop the submitted work data? I want to reset my stats so i can use it for benchmarking.

Also what is average_interval?

EDIT: Once a pool goes down, does it auto switch as a fail over and then switch back once it's back up?
EDIT: How do you test if the pool is down? Mine seems to be constantly cycling through the pools even though they seem to be ok.

 Merged mining, free SMS notifications, PayPal payout and much more.
http://btcstats.net/sig/JZCODg2
krom
Newbie
*
Offline Offline

Activity: 12


View Profile
June 19, 2011, 12:29:35 PM
 #182

EDIT: Once a pool goes down, does it auto switch as a fail over and then switch back once it's back up?
EDIT: How do you test if the pool is down? Mine seems to be constantly cycling through the pools even though they seem to be ok.
If a pool does not respond within a given time, which is about 3 seconds, the proxy will continue with the next pool in the priority list.
kripz
Full Member
***
Offline Offline

Activity: 182



View Profile
June 19, 2011, 12:31:02 PM
 #183

So when does it switch back to the highest priority?

 Merged mining, free SMS notifications, PayPal payout and much more.
http://btcstats.net/sig/JZCODg2
krom
Newbie
*
Offline Offline

Activity: 12


View Profile
June 19, 2011, 12:53:25 PM
 #184

Every time the client requests new work from the proxy, so every 40-50 seconds, this will happen:

The proxy asks the first pool in his priority list for new work. If that pool does not respond within X seconds, the proxy goes on and asks the next pool in his priority list.
This will repeat until a pool does respond and delivers new work which the proxy can relay to the client, or the end of the prioriry list is reached.
teknohog
Sr. Member
****
Offline Offline

Activity: 471


555


View Profile WWW
June 21, 2011, 03:26:13 PM
 #185

Has anyone got this working with Lighttpd (under Gentoo)? I am trying to rule out wider system-level errors before digging deeper into this code. Basically, my clients (Phoenix) are connecting to the proxy, but they are not getting any work.

I got exactly the same error with Apache, and I even did some debugging via tcpdump. It seemed like the proxy was not even asking for any work from the pools. The solution turned out simple, I had to set

allow_url_fopen = On

in php.ini.

So, lighttpd (which uses PHP via fastcgi) seems to work. As I just got it working, I have no long-term data which might be important for long polling, but so far things are OK Smiley

lachesis
Full Member
***
Offline Offline

Activity: 210


View Profile
June 22, 2011, 03:03:05 AM
 #186

I'm getting the response
Code:
{"error":"No enabled pools responded to the work request.","result":null,"id":1}
with Multipool. I just verified -- Multipool is up. In fact, when I make the exact same work request which I made to the proxy (with updated credentials) to multipool.hpc.tw, I get a response in under a second.

How can I troubleshoot this?

Bitcoin Calculator | Scallion | GPG Key | WoT Rating | 1QGacAtYA7E8V3BAiM7sgvLg7PZHk5WnYc
lebuen
Full Member
***
Offline Offline

Activity: 125


View Profile
June 22, 2011, 01:25:00 PM
 #187

Unfortunately, I get extreme stale rates (>10%) when using the proxy on my local mining-rig. But I'm behind a firewall - could it be that the Mining Proxy needs to be listening on port 80 from the "outside" to be able to receive LP-queries?
kripz
Full Member
***
Offline Offline

Activity: 182



View Profile
June 22, 2011, 02:25:26 PM
 #188

Randomly get this using DiabloMiner. Hashkill will just crash. Phoenix seems to be 100% stable, never crashes but i dont doubt it runs into the same problem behind the scenes.

Code:
[23/06/11 12:20:47 AM] DEBUG: Attempt 26 found on Juniper (#1)
[23/06/11 12:20:47 AM] Accepted block 26 found on Juniper (#1)
[23/06/11 12:21:08 AM] ERROR: Cannot connect to Bitcoin: Bitcoin returned error message: No enabled pools responded to the work request.
[23/06/11 12:21:09 AM] ERROR: Cannot connect to Bitcoin: Bitcoin returned error message: No enabled pools responded to the work request.
[23/06/11 12:21:09 AM] ERROR: Cannot connect to Bitcoin: Bitcoin returned error message: No enabled pools responded to the work request.
[23/06/11 12:21:17 AM] DEBUG: Attempt 27 found on Juniper (#1)
[23/06/11 12:21:23 AM] Rejected block 1 found on Juniper (#1)
[23/06/11 12:21:46 AM] DEBUG: Attempt 28 found on Juniper (#1)
[23/06/11 12:21:46 AM] Rejected block 2 found on Juniper (#1)
[23/06/11 12:22:43 AM] DEBUG: Attempt 29 found on Juniper (#1)
[23/06/11 12:22:43 AM] Accepted block 27 found on Juniper (#1)

192.168.1.10 - beasty [23/Jun/2011:00:20:13 +1000] "POST / HTTP/1.1" 200 394 "-" "Java/1.6.0_18"
192.168.1.10 - beasty [23/Jun/2011:00:20:47 +1000] "POST / HTTP/1.1" 200 394 "-" "Java/1.6.0_18"
192.168.1.10 - beasty [23/Jun/2011:00:21:06 +1000] "POST / HTTP/1.1" 200 372 "-" "Java/1.6.0_18"
192.168.1.10 - beasty [23/Jun/2011:00:21:06 +1000] "POST / HTTP/1.1" 200 373 "-" "Java/1.6.0_18"
192.168.1.10 - beasty [23/Jun/2011:00:21:06 +1000] "POST / HTTP/1.1" 200 373 "-" "Java/1.6.0_18"
192.168.1.10 - beasty [23/Jun/2011:00:21:17 +1000] "POST / HTTP/1.1" 200 395 "-" "Java/1.6.0_18"
192.168.1.10 - beasty [23/Jun/2011:00:21:46 +1000] "POST / HTTP/1.1" 200 395 "-" "Java/1.6.0_18"
192.168.1.10 - beasty [23/Jun/2011:00:22:08 +1000] "POST / HTTP/1.1" 200 951 "-" "Java/1.6.0_18"
192.168.1.10 - beasty [23/Jun/2011:00:22:09 +1000] "POST / HTTP/1.1" 200 952 "-" "Java/1.6.0_18"
192.168.1.10 - beasty [23/Jun/2011:00:22:09 +1000] "POST / HTTP/1.1" 200 952 "-" "Java/1.6.0_18"
192.168.1.10 - beasty [23/Jun/2011:00:22:43 +1000] "POST / HTTP/1.1" 200 394 "-" "Java/1.6.0_18"
192.168.1.10 - beasty [23/Jun/2011:00:23:09 +1000] "POST / HTTP/1.1" 200 951 "-" "Java/1.6.0_18"
192.168.1.10 - beasty [23/Jun/2011:00:23:09 +1000] "POST / HTTP/1.1" 200 952 "-" "Java/1.6.0_18"
192.168.1.10 - beasty [23/Jun/2011:00:23:09 +1000] "POST / HTTP/1.1" 200 952 "-" "Java/1.6.0_18"

 Merged mining, free SMS notifications, PayPal payout and much more.
http://btcstats.net/sig/JZCODg2
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
June 22, 2011, 10:17:37 PM
 #189

Do you want patches?  I'm testing putting 50 miners through this proxy and was running into stability issues.  I have mysql on a separate machine and am used to that being a bottleneck, so I looked into making it use persistent connections.  I am not familiar with the PDO extension, but found the instantiator in common.inc.php on line 31; I added array(PDO::ATTR_PERSISTENT => true) to the new call and this took care of my problems.

I now have 32 connections from the proxy web server staying open (which is what I want).
Sounds like a good idea.  I'll test this on my setup and commit if there are no issues.

There should be a way to copy worker/pool configs

so instead of setting up 10x pools on 10x workers you can set it up once then copy it to the rest, then go in and change the logins as necessary.  Or a way to use a variable in the config so you build X workers with the suffix X changed on each one
I might add one or more of these ideas in the future.  Right now I'm trying to keep the number of features low and work on the ones that are having trouble.

Problem 3 was solved by adding 'http://' to the pool's url. Won't work without.
Yup.  Smiley  I'll add something like this to the readme, and maybe add some validation too so that it will bail unless it sees a scheme in the URL.

Really wish there was a way to monitor anything on solo mining besides getwork w/ the proxy
I've no clue what you're asking for here.

What sort of values can i set for 'average_interval'? and what effects will it have?
When reporting on the number of shares submitted and the miner speed, this is the window of time it will look back to gather data.  If you set it longer, the query will take longer and will generally even out more; if you set it shorter, the query will run faster but will result in wild fluctuations of the worker speed (for example) as the worker has lucky and unlucky periods.

It is purely used for reports; it will have no effect on mining.

Also, please add rejected per hour and efficiency.
Another user has a nearly-ready patch that adds a lot of this information.

connections to the proxy started timing out, so restarting apache fixed it, but i didn't trust it enough to keep going

This was on a VM w/ 1GB of ram

What kind of hardware are you guys running on?
I'm running with 768MB, most of which is in use by a Minecraft server...

Try increasing the number of worker children in your Apache configuration.  With more miners, you need more workers to keep up with the demand.  If Apache hits its limit then it will simply stop responding to requests.

I'm getting the response
Code:
{"error":"No enabled pools responded to the work request.","result":null,"id":1}
with Multipool. I just verified -- Multipool is up. In fact, when I make the exact same work request which I made to the proxy (with updated credentials) to multipool.hpc.tw, I get a response in under a second.

How can I troubleshoot this?
If you can, run a sniffer on the web server and see if it even tries to connect.  If it does, see if you can diagnose the problem from the content of the HTTP conversation.

If it does not try to connect:

  • Verify that you have pools assigned to the worker you are using.
  • Verify that the php allow_url_fopen configuration flag is set to On.

Unfortunately, I get extreme stale rates (>10%) when using the proxy on my local mining-rig. But I'm behind a firewall - could it be that the Mining Proxy needs to be listening on port 80 from the "outside" to be able to receive LP-queries?
No, the proxy needs no ports open except for the miners themselves.  It may be worth running a packet sniffer and seeing if the LP requests are actually making it through.  If you are not running on Apache, some config tweaks may be necessary to get LP to work at all.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
lachesis
Full Member
***
Offline Offline

Activity: 210


View Profile
June 22, 2011, 11:22:14 PM
 #190

I'm getting the response
Code:
{"error":"No enabled pools responded to the work request.","result":null,"id":1}
with Multipool. I just verified -- Multipool is up. In fact, when I make the exact same work request which I made to the proxy (with updated credentials) to multipool.hpc.tw, I get a response in under a second.

How can I troubleshoot this?
If you can, run a sniffer on the web server and see if it even tries to connect.  If it does, see if you can diagnose the problem from the content of the HTTP conversation.

If it does not try to connect:

  • Verify that you have pools assigned to the worker you are using.
  • Verify that the php allow_url_fopen configuration flag is set to On.
My client tried to connect twice. The proxy tried to connect a number of times and received several responses, but forwarded none on to my miner.

I wonder if there is an issue here: this server has two IPs on the same subnet. Is there a way to prioritize your proxy to choose one of them exclusively? I'm afraid it's expecting an answer on eth0 but receiving one on eth1 and not properly handling it.

Bitcoin Calculator | Scallion | GPG Key | WoT Rating | 1QGacAtYA7E8V3BAiM7sgvLg7PZHk5WnYc
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
June 23, 2011, 01:40:13 PM
 #191

My client tried to connect twice. The proxy tried to connect a number of times and received several responses, but forwarded none on to my miner.
If you can save a pcap file and email it to me, maybe I could have a look and see what's going on.

I wonder if there is an issue here: this server has two IPs on the same subnet. Is there a way to prioritize your proxy to choose one of them exclusively? I'm afraid it's expecting an answer on eth0 but receiving one on eth1 and not properly handling it.
The kernel should take care of routing stuff correctly.  If traffic is coming back on the wrong interface, that would be a problem with an upstream router and not any particular configuration on your box.  If you can load other websites fine then the network config shouldn't be interfering.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
carlo
Full Member
***
Offline Offline

Activity: 132


View Profile
June 23, 2011, 01:51:22 PM
 #192

Hello,

the Dashboard is getting slower and slower with the whole data.

Does sombody have a quick and dirty soulution to erase the "submitted_work" and the "work_data" every 24h ?
I thougt about dropping every 24h the data from hour 0 -> 23 ... so i keep some stats (like hashrate of the last hour).

If i make my cronjob working ...  i let you know ... if somebody does allready have one please let me know Smiley

cheers
wyze
Newbie
*
Offline Offline

Activity: 28



View Profile WWW
June 23, 2011, 02:48:44 PM
 #193

Hello,

the Dashboard is getting slower and slower with the whole data.

Does sombody have a quick and dirty soulution to erase the "submitted_work" and the "work_data" every 24h ?
I thougt about dropping every 24h the data from hour 0 -> 23 ... so i keep some stats (like hashrate of the last hour).

If i make my cronjob working ...  i let you know ... if somebody does allready have one please let me know Smiley

cheers

How big is your database? Mine is at 19MB and my dashboard still loads in under a second. I have about 4-5 days worth of data in there. This is the same speed I get when accessing the dashboard from work as well, while it is hosted on a little web server in my basement.
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
June 23, 2011, 03:29:16 PM
 #194

Also, please add rejected per hour and efficiency.
Another user has a nearly-ready patch that adds a lot of this information.
I misread you the first time -- the rejected-per-hour stats are there, but efficiency is not.  How do you define efficiency, and how would you compute it?

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
cdhowie
Full Member
***
Offline Offline

Activity: 182



View Profile WWW
June 23, 2011, 03:33:50 PM
 #195

the Dashboard is getting slower and slower with the whole data.
There are some indexes missing from the schema.  I need to add those and provide a migration script for older installs, but this will take a bit of work and I haven't finished it yet.  Once this is done, running the upgrade script will create the indexes and the dashboard will load much faster.

Does sombody have a quick and dirty soulution to erase the "submitted_work" and the "work_data" every 24h ?
I thougt about dropping every 24h the data from hour 0 -> 23 ... so i keep some stats (like hashrate of the last hour).
See the "Database maintenance" section of the readme.

Tips are always welcome and can be sent to 1CZ8QgBWZSV3nLLqRk2BD3B4qDbpWAEDCZ

Thanks to ye, we have the final piece.

PGP key fingerprint: 2B7A B280 8B12 21CC 260A  DF65 6FCE 505A CF83 38F5

SerajewelKS @ #bitcoin-otc
Salain
Newbie
*
Offline Offline

Activity: 18


View Profile
June 23, 2011, 04:34:39 PM
 #196

Also, please add rejected per hour and efficiency.
Another user has a nearly-ready patch that adds a lot of this information.
I misread you the first time -- the rejected-per-hour stats are there, but efficiency is not.  How do you define efficiency, and how would you compute it?

Based on context, I think he's referring to a % good/stale shares stat.  So (#good shares) / (#good + #stale shares).  Any better efficiency metric would require you to load the current difficulty...

Sharecoin: SUtRDGuqYGJHevMWTLnPaurFDdzJqJcDvg
PulsedMedia
Sr. Member
****
Offline Offline

Activity: 402


View Profile WWW
June 23, 2011, 06:34:46 PM
 #197

Can this include some actual proxy kind of features, ie. caching?
So that this could be used behind a flaky internet connection to keep miners 100% at work, if the flakyness is in the seconds range?

Or does this try to connect the pool directly on the same instance as the miner connects to etc.?

What i'm wondering, would this enable me to run miners behind a 3G connection Wink

I'll try this out soon, and i will then optimize the mysql db for you after it starts to slow off (one my of specialties is optimization, esp. mysql), to increase the performance.
Not sure how well you've done performance wise, but we can make it work even with Tb sized datasets if you want to. Contact me via PM or freenode #PulsedMedia if you want to chat about it.

I have a mining hosting idea (in the mining hw section) for which using this as intermediary could be PERFECT. Just need more stats etc. and eventually tagging, grouping nodes/gpus etc. "advanced workflow features", and RESTful API (or more like my own relaxed version using of which is way easier than full REST spec).

Sorry for the basic questions, just yet is not time for me to research into this, i got other dev. tasks i need to finish first before i get to play around with this Smiley

http://PulsedMedia.com - Semidedicated rTorrent seedboxes
PulsedMedia
Sr. Member
****
Offline Offline

Activity: 402


View Profile WWW
June 24, 2011, 01:07:36 AM
 #198

Installed and seems to work. Tho weighting seems to be quite a bit off (haven't looked at that portion of code yet).

Quick glance at DB shows no reason why it should be slow, if queries match. So those who are having speed issues probably have bad MySQL config just. MySQL scales really efficiently and i don't see immediate reasons why this would be slow, but have to wait for data set to increase.

If anyone got say 8G+ dataset they don't mind sharing, i would be willing to look into it. Or any size with which someone is having some serious perf issues.

Code should be PLENTY more commented btw Wink

http://PulsedMedia.com - Semidedicated rTorrent seedboxes
kjj
Legendary
*
Offline Offline

Activity: 1302



View Profile
June 24, 2011, 01:10:22 AM
 #199

Installed and seems to work. Tho weighting seems to be quite a bit off (haven't looked at that portion of code yet).

Quick glance at DB shows no reason why it should be slow, if queries match. So those who are having speed issues probably have bad MySQL config just. MySQL scales really efficiently and i don't see immediate reasons why this would be slow, but have to wait for data set to increase.

If anyone got say 8G+ dataset they don't mind sharing, i would be willing to look into it. Or any size with which someone is having some serious perf issues.

Code should be PLENTY more commented btw Wink

Check out the join that creates the status display.

p2pcoin: a USB/CD/PXE p2pool miner - 1N8ZXx2cuMzqBYSK72X4DAy1UdDbZQNPLf - todo
I routinely ignore posters with paid advertising in their sigs.  You should too.
PulsedMedia
Sr. Member
****
Offline Offline

Activity: 402


View Profile WWW
June 24, 2011, 01:21:03 AM
 #200

Installed and seems to work. Tho weighting seems to be quite a bit off (haven't looked at that portion of code yet).

Quick glance at DB shows no reason why it should be slow, if queries match. So those who are having speed issues probably have bad MySQL config just. MySQL scales really efficiently and i don't see immediate reasons why this would be slow, but have to wait for data set to increase.

If anyone got say 8G+ dataset they don't mind sharing, i would be willing to look into it. Or any size with which someone is having some serious perf issues.

Code should be PLENTY more commented btw Wink

Check out the join that creates the status display.

LOL! Yeah that would cause some serious issues (first query in admin/index.php) 3rd query is a monstrosity.

Well there is the problem, using dynamic (on the fly created) tables etc.

These queries is almost like SELECT *, wonder if they ever hit any indexes ...

In any case need bigger data set before i can optimize them properly.

But on first query FROM ( XXXX XXXX ) sw should be change to just choose the damn table, moving the LIMIT 10 at the whole query.
Inner joins -> just choose from multiple tables, use same p.id = sw.pool_id

So something like

SELECT w.name AS worker, p.name AS pool, p.pool_id AS poolId, p.id AS poolId, sw.worker_id AS workerId  [AND SO ON FOR ALL FIELDS REQUIRED]
FROM submitted_work sw, pool p, woker
WHERE pool.id = sw.pool_id AND worker.id=sw.workerid
ORDER BY
LIMIT

INNER JOINS are like SELECT *, if i recall right Tongue

Bottomline is that carefully crafted queries can do a full text match scored on a 100G dataset with multiple text matches, joining multiple tables (not using JOIN clause tho, but still called joining), on a Quad Core Xeon with 16G ram (pre-i7 xeon) in well under 100ms. (Target was 15searches/sec, achieved peak was above that, and bottleneck was actually the PHP syntax parsing for real world scenario which transformed our simplified custom query language into a mysql query, for easier use for the end users)

http://PulsedMedia.com - Semidedicated rTorrent seedboxes
kjj
Legendary
*
Offline Offline

Activity: 1302



View Profile
June 24, 2011, 01:47:12 AM
 #201

Meh.  Took me like 5 minutes to modify the work_data table, add new history tables, write a cron job to rotate the records out, and post my scripts.  Bonus: I didn't even have to think of any clever SQL tricks.

p2pcoin: a USB/CD/PXE p2pool miner - 1N8ZXx2cuMzqBYSK72X4DAy1UdDbZQNPLf - todo
I routinely ignore posters with paid advertising in their sigs.  You should too.
nick5429
Member
**
Offline Offline

Activity: 77


View Profile
June 24, 2011, 03:15:04 AM
 #202

Just as another datapoint: loading Multipool into the proxy doesn't work for me, either.  My miner sits there with 0Mhash/sec.  Directly mining at Multipool directly works fine for me.

Also, I get weird authentication issues with hashkill.  About half the time it will fail the initial auth, the other half it authenticates, does a little bit of work, and then claims it had authentication issues:
Code:
[hashkill] Version 0.2.5
[hashkill] Plugin 'bitcoin' loaded successfully
[hashkill] Found GPU device: Advanced Micro Devices, Inc. - Barts
[hashkill] GPU0: AMD Radeon HD 6800 Series [busy:0%] [temp:49C]
[hashkill] Temperature threshold set to 90 degrees C
[hashkill] This plugin supports GPU acceleration.
[hashkill] Initialized hash indexes
[hashkill] Initialized thread mutexes
[hashkill] Spawned worker threads
[hashkill] Successfully connected and authorized at my.bitcoinminingproxy.com:80
[hashkill] Compiling OpenCL kernel source (amd_bitcoin.cl)
[hashkill] Binary size: 349376
[hashkill] Doing BFI_INT magic...

Mining statistics...
Speed: 273 MHash/sec [proc: 4] [subm: 1] [stale: 0] [eff: 25%]     [error] (ocl_bitcoin.c:141) Cannot authenticate!

Phoenix miner works fine for me with the same proxy settings.
nick5429
Member
**
Offline Offline

Activity: 77


View Profile
June 24, 2011, 03:35:12 AM
 #203

In case anyone is interested, I was able to get the mining proxy working on my Dreamhost shared server with a bit of tweaking.  This should be equally applicable to other hosts that may impose similar restrictions.

Since Dreamhost forces PHP-CGI (rather than mod_php) on its shared hosting users, the .htaccess tricks don't work and the PHP_AUTH_USER / PHP_AUTH_PW variables in the scripts are empty.

First, follow all the basic setup instructions in the standard Proxy guide.

Then, you need to make sure that magic_quotes_gpc and allow_url_fopen are set to the defaults that we need, otherwise I don't think there's anything you can do.  To do this, create a file called phpinfo.php in your htdocs with the following code:
Code:
<?php
phpinfo
();
?>

And browse to it in your browser.

Search for magic_quotes_gpc (it needs to be off) and allow_url_fopen (it needs to be on).  If these settings don't match, getting the proxy working is beyond the scope of this tweak.


If those settings are okay, then we can start with the tweaking.  The way the script is written won't allow you to authenticate, but everything else works fine.  To fix this...

Replace the contents of your .htaccess file with the following:
Code:
Options -Indexes
RewriteEngine on
RewriteRule .* - [env=HTTP_AUTHORIZATION:%{HTTP:Authorization},last]

Then, edit your common.inc.php file to include this code inside the "do_admin_auth()" function:
Code:
if (preg_match('/Basic\s+(.*)$/i', $_SERVER['HTTP_AUTHORIZATION'], $matches))
    {
        list($name, $password) = explode(':', base64_decode($matches[1]));
        $_SERVER['PHP_AUTH_USER'] = strip_tags($name);
        $_SERVER['PHP_AUTH_PW'] = strip_tags($password);
    }

This function will now look something like:
Code:
function do_admin_auth() {
    global $BTC_PROXY;
    if (preg_match('/Basic\s+(.*)$/i', $_SERVER['HTTP_AUTHORIZATION'], $matches))
    {
        list($name, $password) = explode(':', base64_decode($matches[1]));
        $_SERVER['PHP_AUTH_USER'] = strip_tags($name);
        $_SERVER['PHP_AUTH_PW'] = strip_tags($password);
    }
    if (!isset($_SERVER['PHP_AUTH_USER'])) {
        auth_fail();
    }

    if (    $_SERVER['PHP_AUTH_USER'] != $BTC_PROXY['admin_user'] ||
            $_SERVER['PHP_AUTH_PW']   != $BTC_PROXY['admin_password']) {
        auth_fail();
    }
}

EDIT: You also need to include this code snippet near the top of index.php
PulsedMedia
Sr. Member
****
Offline Offline

Activity: 402


View Profile WWW
June 24, 2011, 06:53:57 AM
 #204

Meh.  Took me like 5 minutes to modify the work_data table, add new history tables, write a cron job to rotate the records out, and post my scripts.  Bonus: I didn't even have to think of any clever SQL tricks.

I looked at the linked post... There is a bunch of bad things i could now say about that cron job ...

Slow, prone to fault gum h4x to fix performance issues? People! This is how we create bloatware!

Fix the problem, not the symptoms!

http://PulsedMedia.com - Semidedicated rTorrent seedboxes
PulsedMedia
Sr. Member
****
Offline Offline

Activity: 402


View Profile WWW
June 24, 2011, 08:07:43 AM
 #205

wow, there is many kinds of bad things going on code QA wise.

This code is what i call confused newbie abstraction.

Using functions to output HTML in a view (already outputting HTML)? Yup!

Using abstration function to output a simple form image button? Yup!

Doing insane joined and dynamic tables query for a few fields of simple data? Yup!

Anyways, 2 first queries optimized, one less table to lookup, indices being actually hit. Far from being completely optimized (still a filesort happening!), but should proof to be an order of magnitude faster, DESPITE hitting more rows. I have NO way to test however, nor profile correctly due to lack of dataset size, so measurements would be less than error of margin.

Replace admin/index.php 2 first queries with those found at: http://pastebin.com/hwncLV1w

NOTE: I have not done proper testing, results seem to be correct tho Smiley

EDIT: Looking into 3rd query now, it's worse than expected. Will rework complete query and accompanying view portion. It actually checks all rows multiple times, assuming MySQL realizes how to optimize. It makes 5 dynamic (on-the-fly created) tables, hits *ALL* rows on submitted work, creates temp tables (lack of suitable index), 3 times filesort, 13 tables total, and if i interpret correctly total rows to go through is in the range of hundreds of thousands rows or more with my 2526 shares submitted test data! :O

http://PulsedMedia.com - Semidedicated rTorrent seedboxes
PulsedMedia
Sr. Member
****
Offline Offline

Activity: 402


View Profile WWW
June 24, 2011, 10:38:55 AM
 #206

well, i fooled around, according to profiling my changes are beneficial to the 3rd query but due to the really tiny sample data set i got the actual measurements mean nothing, the spent time might just be all in parsing the query and optimization engine, not the actual query.

So take even the earlier ones with a grain of salt: I've got no clue of the impact as i can't actually measure the difference, even tho i'm using for testing a ancient Athlon XP 1900+ ...

As soon as i got enough data i will see again how my changed queries affect the performance.

http://PulsedMedia.com - Semidedicated rTorrent seedboxes
kripz
Full Member
***
Offline Offline

Activity: 182



View Profile
June 24, 2011, 11:21:39 AM
 #207

I will test, got a fix for the 3rd?

 Merged mining, free SMS notifications, PayPal payout and much more.
http://btcstats.net/sig/JZCODg2
PulsedMedia
Sr. Member
****
Offline Offline

Activity: 402


View Profile WWW
June 24, 2011, 11:28:56 AM
 #208

I will test, got a fix for the 3rd?

Yes, but approach was wrong so it might actually perform worse.

Try also how these affect: http://pastebin.com/kcWN9gPH

How many rows in your submitted_work and work_data tables?

http://PulsedMedia.com - Semidedicated rTorrent seedboxes
kripz
Full Member
***
Offline Offline

Activity: 182



View Profile
June 24, 2011, 11:38:52 AM