Has anyone successfully used Multipool as a target pool through the Flexible Mining Proxy? cdhowie, have you tried it? I've seen at least 2-3 reports in addition to mine claiming it doesn't work, and nobody saying it works for them. No signup or setup is needed -- just connect to http://multipool.hpc.tw:8337 with a Bitcoin address as your username, and anything for your password. I'd be interested to hear if multipool is broken for everyone.... I will take a look at this for you over the weekend. Damn, beat me to it, wyze.  Yes, if nobody can connect to multipool then it's likely some communication issue caused by the proxy code. If wyze figures it out then he'll probably fix it (he has a fork over on Github you could use until I merge his fix) and if not I'll have a look sometime this weekend too. |
|
|
|
Installed and seems to work. Tho weighting seems to be quite a bit off (haven't looked at that portion of code yet).
There is no such thing as "weighting." Did you read the readme? Particularly the section on how priority works? Code should be PLENTY more commented btw  Probably, particularly index.php. The rest is fairly well-separated and should be readable as-is. My philosophy on comments is that if you have to comment a lot, your code doesn't explain itself well enough and should be refactored. |
|
|
|
Note that a few indexes that are required to prevent table scans are not present in the schema; these were added later and I don't have a database migration script just yet, so it's expected that these queries will run a bit slow unless you've manually created the needed indexes.
I just took a few minutes to knuckle-down and get this done. I recommend that all users get the latest from master and apply the DB migration script. Dashboard performance should significantly improve (to sub-second response times). |
|
|
|
Can this include some actual proxy kind of features, ie. caching?
So that this could be used behind a flaky internet connection to keep miners 100% at work, if the flakyness is in the seconds range?
A non-PHP getwork backend is planned to resolve this and other LP issues. PHP is not well-suited to this kind of task. Check out the join that creates the status display.
LOL! Yeah that would cause some serious issues (first query in admin/index.php) 3rd query is a monstrosity. Well there is the problem, using dynamic (on the fly created) tables etc. These queries is almost like SELECT *, wonder if they ever hit any indexes ... On my database, an EXPLAIN against that query shows zero table scans and very little processing. I spent a lot of time optimizing that query. Note that a few indexes that are required to prevent table scans are not present in the schema; these were added later and I don't have a database migration script just yet, so it's expected that these queries will run a bit slow unless you've manually created the needed indexes. Well there is the problem, using dynamic (on the fly created) tables etc.
This in particular made me lol. Subqueries can be an effective optimization technique if you know how to use them correctly, and any DBA knows that. In the "last 10 submissions" cases, MySQL creates a plan that executes the LIMIT after the JOIN, which results in a full table scan of work_data/submitted_work. Querying those tables in a subquery with LIMIT forces it to execute the limit first, which results in a very fast join. This was a pain point until I refactored the query to use a subquery to derive the data tables. Please know WTF you are talking about and use EXPLAIN, kthx. EDIT: I just checked, and interestingly doesn't hit index, which is really wierd. Oh well, i check why at better time.
Exactly. MySQL doesn't use the indexes in this case because it has decided to apply the LIMIT after the joins. So it does a table scan. And you don't need indexes to do a table scan, now do you? Essentially, MySQL's query analyzer sucks, and the subquery is the workaround. So let's do some investigation: mysql> SELECT COUNT(*) FROM work_data;
+----------+
| COUNT(*) |
+----------+
| 76422 |
+----------+
1 row in set (0.01 sec)
mysql> SELECT COUNT(*) FROM submitted_work;
+----------+
| COUNT(*) |
+----------+
| 126715 |
+----------+
1 row in set (0.00 sec) After executing the dashboard status query: EXPLAIN on the dashboard status query: +----+-------------+----------------+--------+------------------------------------------------+-------------------------+---------+---------------------------------+------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+----------------+--------+------------------------------------------------+-------------------------+---------+---------------------------------+------+----------------------------------------------+
| 1 | PRIMARY | w | ALL | NULL | NULL | NULL | NULL | 3 | Using temporary; Using filesort |
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 1 | |
| 1 | PRIMARY | <derived4> | ALL | NULL | NULL | NULL | NULL | 2 | |
| 1 | PRIMARY | <derived6> | ALL | NULL | NULL | NULL | NULL | 1 | |
| 6 | DERIVED | sw | range | dashboard_status_index2 | dashboard_status_index2 | 8 | NULL | 136 | Using where; Using temporary; Using filesort |
| 4 | DERIVED | <derived5> | ALL | NULL | NULL | NULL | NULL | 2 | Using temporary; Using filesort |
| 4 | DERIVED | sw | ref | dashboard_status_index,dashboard_status_index2 | dashboard_status_index | 13 | sw2.worker_id,sw2.latest | 1 | |
| 4 | DERIVED | p | eq_ref | PRIMARY | PRIMARY | 4 | bitcoin-mining-proxy.sw.pool_id | 1 | |
| 5 | DERIVED | submitted_work | range | NULL | dashboard_status_index | 5 | NULL | 3 | Using where; Using index for group-by |
| 2 | DERIVED | <derived3> | system | NULL | NULL | NULL | NULL | 1 | |
| 2 | DERIVED | wd | ref | PRIMARY,worker_time | worker_time | 12 | const,const | 1 | |
| 2 | DERIVED | p | eq_ref | PRIMARY | PRIMARY | 4 | bitcoin-mining-proxy.wd.pool_id | 1 | |
| 3 | DERIVED | work_data | range | NULL | worker_time | 4 | NULL | 3 | Using index for group-by |
+----+-------------+----------------+--------+------------------------------------------------+-------------------------+---------+---------------------------------+------+----------------------------------------------+ Sorry, the query is fine. A bit big, but it's attempting to reduce quite a bit of data down to three rows. So meh. You do better and I'll take a patch. |
|
|
|
the Dashboard is getting slower and slower with the whole data.
There are some indexes missing from the schema. I need to add those and provide a migration script for older installs, but this will take a bit of work and I haven't finished it yet. Once this is done, running the upgrade script will create the indexes and the dashboard will load much faster. Does sombody have a quick and dirty soulution to erase the "submitted_work" and the "work_data" every 24h ?
I thougt about dropping every 24h the data from hour 0 -> 23 ... so i keep some stats (like hashrate of the last hour).
See the "Database maintenance" section of the readme. |
|
|
|
Also, please add rejected per hour and efficiency.
Another user has a nearly-ready patch that adds a lot of this information. I misread you the first time -- the rejected-per-hour stats are there, but efficiency is not. How do you define efficiency, and how would you compute it? |
|
|
|
My client tried to connect twice. The proxy tried to connect a number of times and received several responses, but forwarded none on to my miner.
If you can save a pcap file and email it to me, maybe I could have a look and see what's going on. I wonder if there is an issue here: this server has two IPs on the same subnet. Is there a way to prioritize your proxy to choose one of them exclusively? I'm afraid it's expecting an answer on eth0 but receiving one on eth1 and not properly handling it.
The kernel should take care of routing stuff correctly. If traffic is coming back on the wrong interface, that would be a problem with an upstream router and not any particular configuration on your box. If you can load other websites fine then the network config shouldn't be interfering. |
|
|
|
Do you want patches? I'm testing putting 50 miners through this proxy and was running into stability issues. I have mysql on a separate machine and am used to that being a bottleneck, so I looked into making it use persistent connections. I am not familiar with the PDO extension, but found the instantiator in common.inc.php on line 31; I added array(PDO::ATTR_PERSISTENT => true) to the new call and this took care of my problems.
I now have 32 connections from the proxy web server staying open (which is what I want).
Sounds like a good idea. I'll test this on my setup and commit if there are no issues. There should be a way to copy worker/pool configs
so instead of setting up 10x pools on 10x workers you can set it up once then copy it to the rest, then go in and change the logins as necessary. Or a way to use a variable in the config so you build X workers with the suffix X changed on each one
I might add one or more of these ideas in the future. Right now I'm trying to keep the number of features low and work on the ones that are having trouble. Problem 3 was solved by adding 'http://' to the pool's url. Won't work without.
Yup. I'll add something like this to the readme, and maybe add some validation too so that it will bail unless it sees a scheme in the URL. Really wish there was a way to monitor anything on solo mining besides getwork w/ the proxy
I've no clue what you're asking for here. What sort of values can i set for 'average_interval'? and what effects will it have?
When reporting on the number of shares submitted and the miner speed, this is the window of time it will look back to gather data. If you set it longer, the query will take longer and will generally even out more; if you set it shorter, the query will run faster but will result in wild fluctuations of the worker speed (for example) as the worker has lucky and unlucky periods. It is purely used for reports; it will have no effect on mining. Also, please add rejected per hour and efficiency.
Another user has a nearly-ready patch that adds a lot of this information. connections to the proxy started timing out, so restarting apache fixed it, but i didn't trust it enough to keep going
This was on a VM w/ 1GB of ram
What kind of hardware are you guys running on?
I'm running with 768MB, most of which is in use by a Minecraft server... Try increasing the number of worker children in your Apache configuration. With more miners, you need more workers to keep up with the demand. If Apache hits its limit then it will simply stop responding to requests. I'm getting the response {"error":"No enabled pools responded to the work request.","result":null,"id":1}with Multipool. I just verified -- Multipool is up. In fact, when I make the exact same work request which I made to the proxy (with updated credentials) to multipool.hpc.tw, I get a response in under a second. How can I troubleshoot this? If you can, run a sniffer on the web server and see if it even tries to connect. If it does, see if you can diagnose the problem from the content of the HTTP conversation. If it does not try to connect: - Verify that you have pools assigned to the worker you are using.
- Verify that the php allow_url_fopen configuration flag is set to On.
Unfortunately, I get extreme stale rates (>10%) when using the proxy on my local mining-rig. But I'm behind a firewall - could it be that the Mining Proxy needs to be listening on port 80 from the "outside" to be able to receive LP-queries?
No, the proxy needs no ports open except for the miners themselves. It may be worth running a packet sniffer and seeing if the LP requests are actually making it through. If you are not running on Apache, some config tweaks may be necessary to get LP to work at all. |
|
|
|
OK, my time is up. Send me a quote for a 60-day WoW card. Instead of doing gift cards I'm now buying time online and sending you the code. The quote for a 30-day code is here: http://bitcoin-otc.com/vieworder.php?id=3666. Double it for 60 days. I'd prefer to do the transaction in realtime (over IRC for example) since the market moves around pretty quickly. Note that these purchases are strictly non-refundable. I will optionally GPG-encrypt the code(s) to whatever key you specify when you place the order. |
|
|
|
Parse error: syntax error, unexpected T_STRING, expecting T_OLD_FUNCTION or T_FUNCTION or T_VAR or '}' in /home/REDACTED/public_html/admin/index.php on line 28
Hmm, not sure why i'm getting this. This is on the admin page. It complains about line 125 on the root
Which version of PHP? It's possible that your version does not support function access modifiers. |
|
|
|
Do you sell wow eu game cards?  I don't think the card codes are localized... are they? |
|
|
|
Rebooted after crash and now Worker Status table on main dashboard is completely blank. Recent work submissions and recent failed submissions still show up just fine in the top 2 tables, but the bottom table is blank.
Any ideas?
Edit: I was also having connection issues. I exported my workers and pools settings. Imported the original schema back, and imported those saved settings. All is well now.
Sounds like the MySQL database got corrupted. Which is weird, since table logs are supposed to keep that from happening. :/ |
|
|
|
very strange.. it works fine if there is only one pool, but when I add multiples, it simply doesn't work. I have also noticed that my stale/rejected shares went up extremely high... In deepbit, for the last month I have been under 1% stales. Using the proxy, I'm at almost 20% rejected shares  I was really hoping that this would work That's weird. The code path isn't any different if you have one or more pools configured... If any of you are still having stale share issues, would you mind taking a capture of the network traffic? On the proxy host, issue this as root: tcpdump -w proxy.pcap -s 65535 'port 80 or portrange 8332-8337' Then email me the proxy.pcap file after an hour or two. Note that this file will contain all of the traffic on these ports and this means that I will have access to your proxy worker passwords as well as your pool worker passwords. You may certainly encrypt the file to my GPG key if you are worried about the file being intercepted. (Specifically, note that if you talk to a bitcoind from the web server, this will expose your RPC password! So be especially careful in this case. I'm not going to misuse this information, but... well, if I were you, I wouldn't trust me blindly either. ) If possible, capture traffic from the workers to the pools directly, without the proxy, and email me that as a separate file. Comparing the two files will help me determine what's going on. Even better, if you're using a pool supporting long-polling, have one miner going through the proxy and at the same time another one not using the proxy, and mail me both files. Seeing how the long-polling requests return in both cases (they should be identical...) will help me diagnose this. Thanks all for the feedback, donations, and support! It's heartening to know that the software is fulfilling its purpose and that people find it helpful. |
|
|
|
Anyone else care to check the contents of their /var/lib/php5? after running this for a while I started getting no disk space errors, upon further investigation I found out that PHP had created 15.5 million session files in /var/lib/php5. Fun. Should anyone else bump into this problem, you'll probably notice that nothing can delete the files (rm, ls, find etc all hang) after much fiddling, the solution is to use ls -U1, which will print the file list unsorted, and then pipe the output to rm, like so. ls -U1 /var/lib/php5 | xargs rm -f Gonna take me 5 days of rm to clean up all the session files, fun fun Besides this bug (which may just be some issue on my end) it seems great, I love how I can have pool redundancy in case one goes down. Thanks for the report. This was indeed a bug, and has been fixed in master. |
|
|
|
Ok, I'm not selling them, but my dad has expressed some interest in selling paintings for BTC. Samples of his work: paintings, drawings (charcoal, etc.) (click each picture to go to a gallery of that section). I think he'll be willing to sell prints and some originals, as well as commission pieces. If y'all like his work and would be interested in buying some, let me know and I'll see what he thinks. |
|
|
|
Me either... why wouldn't they use double precision?
Why wouldn't they use fixed-point? Floating point is for when you don't know how many decimal places you need. With Bitcoin you know exactly how many you need; Mt. Gox should not use floating point at all. |
|
|
|
nice idea but im getting lots of blank pages when handling the backend.
xampp with PHP 5.2.4
I have not tested with many versions, but others have reported that at least PHP 5.3 is required to use the proxy. I will add this to the readme at some point. |
|
|
|
If there is enough disruption in the pools for a couple weeks would the difficulty drop?
Probably not by much. People will either switch to another pool or solo mine until the pools come back up. The network doesn't care if the blocks come from pools or not. |
|
|
|
Good catch. Yeah, that would cause problems. I might work around this by using PATH_INFO to convey the required data. |
|
|
|
|
Show Posts
