Bitcoin Forum

Announcing RollProxy, a mining proxy... with a twist!

RollProxy takes advantage of the X-Roll-NTime extension to minimize the number of getwork requests you send to pools.  By keeping more requests local, it minimizes latency, while drastically reducing the load on upstream servers.
The proxy will prove especially useful to those who run many miners on a single local network.

Features

• Works with Bitcoin as well as with Litecoin and other cryptocurrencies that use the getwork protocol
• Makes persistent HTTP connections
• Supports long polling
• Per-worker statistics available via web interface
• Round-robin failover mechanism (more advanced strategies may be implemented in the future)
• Ability to connect through an additional proxy server

Please note that your miners need not support the X-Roll-NTime extension; it's the pools you connect to that need to. If you connect to a pool that doesn't support the extension, the proxy will still work, but you will see no increase in getwork efficiency. In that case, RollProxy will warn you with a "Work cannot be reused" message.
Major pools that are known to support the extension include:

• For Bitcoin: Eclipse, Eligius, MaxBTC, Mt. Red, OzCoin, P2Pool, Slush
• For Litecoin: burnside's, LitecoinCash, notroll.in, OzCoin, P2Pool, Pool-X, Xurious's

RollProxy is written in Python and is distributed under the terms of the AGPLv3. Please refer to the readme for details about setting it up.
https://github.com/pooler/rollproxy

This first public release (version 0.5.0) should already be pretty stable; I wish to thank Pontius, who helped me by testing preliminary versions of the program.

Nice, will be using this  :)

Nice work.

Great work, pooler! Thanks a lot for sharing this.

And for the windows dummies out there (like me) :)?

how do i connect a miner to that proxy?

Normaly i give my mining program the credentials to my pool. For what i understand is that i give them to your proxy and connect my miner to it instead?

Note that cgminer does this already intrinsically and adding this proxy with it serves no real purpose. I can't speak for all other mining software though.

That's if you only have one computer mining. If you have more than one, no miner can do what this proxy does.

Last time I used Windows was about 10 years ago, so it's best if I let someone else answer this question...

Yes. You start the proxy specifying the address and credentials for one or more pools; then you connect your miners to the proxy, which by default runs on port 8345. Miners need not authenticate, but if they provide a username you will see per-worker stats in the web interface of RollProxy (which is also served on port 8345 by default).

So i connect them to  localhost:port x ?

You connect miners to the computer running the proxy. If a miner is on the same computer as the proxy, you can just connect it to localhost:8345. If the proxy is running on, say, 192.168.1.123, connect miners to 192.168.1.123:8345.

Great idea. I will try it out

say, pooler, would it be possible to add support for socks proxies in addition to http? this could quite possibly be the best thing since cgminer, if only it had socks support :)

What about just using p2pool?

I realize that SOCKS support would be a very nice feature to have. The problem is that the master branch of urllib3 doesn't support SOCKS proxies yet, although developers have been working on it for some time now (see this pull request (https://github.com/shazow/urllib3/pull/68)).
One option would be to rewrite RollProxy to use PycURL instead of urllib3, but the problem is that urllib3 seems to be the only decent HTTP library that supports connection pooling.
Another option would be to use SocksiPy, which however is buggy, old, and unmaintained.
For the above reasons, it's probably better to wait till the urllib3 guys finish implementing SOCKS support.

oh. Yeah, you're right. No point in rewriting the thing for another lib.

In any case, it seems my debian squeeze server is stuck on python 2.6.6 anyway, which neither offers urllib3 packages, nor satisfies rollproxy's version requirement. Oh well.

Rollproxy on squeeze works fine for me:

oooh, interesting. and here i tried installing the wheezy packages for urllib3 ::)
thank you very much, it is working now.

and thank you, too, pooler. rollproxy is great!

cgminer does:
--socks-proxy <arg> Set socks4 proxy (host:port)

Unfortunately that doesn't help. What is needed is a way for RollProxy to connect to the Internet through a SOCKS4/5 proxy, not a way for the miners to connect to RollProxy through an additional local proxy.

Use socat.
 
for mining through tor,

socat tcp-listen:18332,fork,reuseaddr socks4a:127.0.0.1:pool.com:8332,socksport=9050

But that would still leak DNS requests, wouldn't it? That has always been the issue with tools like socat or tsocks.

Your miner connects to 127.0.0.1 - no dns there
if you use the socks4a option on socat, it does the dns lookup for the pool through the proxy.

As I understand, this helps for situation where miner don't support ntime rolling and pool does, right? What miners don't support ntime rolling yet?

I lived in the opinion that ntime rolling is widely supported everywhere. Then I found that some big pools still don't support such simple and important extension, like Deepbit or even 50BTC. But as I understand, this proxy don't help in such situation - pool have to support ntime rolling itself.

Maybe the best thing is just to switch to miner and pool which supports ntime rolling, or even give a try to Stratum which implements mining protocol in quite innovative way (yes, this is kind of advertisement for my pool and BtcGuild :).

Hey, that's nifty! Thanks for the tip!

The way I understand it, a miner can only do rolling within itself. If you have several miners on different machines, this proxy will help a lot. But please correct me if this is nonsense!

Besides, even without this feature, mining proxies are great tools to have if you use many machines. It makes monitoring, maintenance, setup of new machines or changing the pool and propagating that change to all clients so much easier.

I especially like Rollproxy because it's so small and efficient. I've used another proxy before, but it required apache, php, mysql, regular database cleanups, and while it worked well, it was a bit cumbersome to use.

Yes, macavity is right, that's the true raison d'être of this program, and why it was developed.
RollProxy allows multiple miners to reuse the same work unit. And this can bring huge bandwidth savings.

pooler, looks like you're concerned about mining efficiency. Maybe you would be interested in supporting Stratum in your proxy as well? :)

I'm really busy with other stuff in this period, so no, or at least not any time soon.

Just for the fun - cascading the proxies (miner -> rollproxy -> stratum-proxy -> pool) works.

Yes, it's quite expected, because stratum proxy provides full getwork API, including long polling and ntime rolling.

Getting this endlessly:
https://i.imgur.com/p2xI4.png

however the proxy still seems to be redirecting shares perfectly, it just has high CPU usage now...
Any ideas?

Do you plan to add support for ntime rolling to the miners ? I'm asking because my config is a little odd and I have to cascade 2 rollproxy and I notice that second one does not see any support for ntime rolling from the first proxy.

Do you plan to support the new getblocktemplate protocol in the proxy ?

If miners were allowed to do ntime rolling themselves, they could end up working on the exact same work unit, so it's not a good idea. Maybe it is possible to allow for little variations using a very low "expire" parameter, assuming of course that miners interpret that correctly, but honestly I can't see why you need the miners to roll the work themselves when they can just ask the proxy to do it.

Support for additional protocols is not planned at this time. I have very limited time to dedicate to this project, so right now I can only fix bugs and maybe add a few minor features.

This is pretty slick.  Thanks!!

For those windows users out there like me:

I couldn't get ez_setup to install urllib.  It complained about a config file be missing.
So instead of I copied the urllib3 folder into my python\lib folder, and voila, it works.

For those that aren't used to the somewhat odd URL format this uses (I've seen it elsewhere, only reason I could figure it out), it's:

http://username:password@server:port

M

I have the same issue (although the error message in rollproxy varies a bit).  Whenever rollproxy has a connection issue between itself and a pool, CPU usage goes crazy (it's stuck in a loop, I'd imagine).  With enough rigs running against it, rollproxy can no longer keep up with providing work while this is happening.

Yeah, I had to stop using it on my server, It was taking up half my CPU time on just 7Gh/s  :D

What's this "cannot decode byte"-message? Happens everytime I try to access rollproxy with a browser... the webinterface is not displayed.

http://666kb.com/i/c7za3i7803jnsq42f.gif

I have added a 1-second pause between retries as a quick-and-dirty fix for this issue.

After exchanging a few private messages with ShadesOfMarble, this turned out to be a OS-and-locale-dependent issue; it should be fixed now.

Version 0.5.2 should also support miners that (like Phoenix) issue long polling requests using the GET method.

Thanks again!

Rolled back to v0.5.1. On pool errors v0.5.2 started rotating over all configured pools in a endless loop...

I just had a look at the 0.5.2 commit. All I did is adding a 1-second pause between retries; I don't think I've changed anything else in the failover mechanism.
So I'd say if the problem you describe exists in 0.5.2 it should also exist in 0.5.1.
Can you please send me some logs if the issue shows up again?

Sure, no problem. Just switched back to 0.5.2 and started logging to a file. If anything shows up I'll send it to you.

or just use proxychains? ;)

Ok, so after six days of using v0.5.2 (again) everything is fine. Whatever my problem was I'm unable to reproduce it...

Hi,

is it possible to connect with different workernames to a pool ? i lose controll which worker is down.

I could start a bunch of rollproxies on different ports, but that looks for me like a waste of recources.

Miner_1 =>> Rollproxy ==> worker1@abcpool
Miner_2 =>> same Rollproxy ==> worker2@abcpool

Thats what i mean.

Thanks

AFAIK that's not possible but RollProxy provides a webinterace where you can check your miners status...

help me :

Looks like you've downloaded the github website but not the python code...

Try this:

I'm getting noticeably higher rejects when mining through rollproxy. Long poll is enabled on both cgminer and rollproxy, but my rejects are still 1-2% compared to the 0.5% max when directly connecting. There's no additional network delay since rollproxy is running on localhost. Has anyone else noticed higher reject rates? What might be the cause?

any way to log mining data like shares submitted per worker etc ? like in a text file or something?

Done. There's now a --sharelog option that allows you to specify a file for logging submissions.

Sweet thanks !

Hey. I need a small code addon. Is it possible?

Basically, I would like for the proxy to restart if there are >5% stale shares.

I think it's just a few lines of code. I'd do it myself, but I don't know Python.
Thanks a bunch!

You could turn this into a simple pool software that could mine on litecoind if you did the 2 following things:

#1) Allow the rollproxy to rewrite target difficulty

#2) Add the option to only submit shares that meet difficulty upstream.

Would you be willing to implement this for solo miners?

I used rollproxy to force reuse of work units when solo-mining ppcoin for a few days. It worked very well, fixing my problem with running out of work when pointing miners directly to ppcoind. I found 2 blocks and those were the only shares submitted to rollproxy. Why would you want any other difficulty "shares" on a setup like that, or are you thinking of something different?

When you have multiple machines accessing the same pool server/proxy, it's nice to know that the miners are online and functioning properly. If they submit lower difficulty shares I will be able to tell this (even if they aren't difficulty 1.. 1 share every 5 or 10 minutes would be fine.)

 Hello. i setup pushpool + bitcoind.   is there appropriate mechanism to proxy to bigger mining pull to share profit with them? we are young and it will be difficult at first time to earn something.
it is possible to create something like that:
miners->  pushpool -> proxy -> another_pool? 

will i be able to gather statistics from Rollproxy via SimpleBTC?   now simplebtc connects to bitcoind via rpc and get all stat.

i looked at rollproxy and it seems it can help me.

  Thanks in advance.

Its working but it auto crashes at random times (10 mins or 30 mins) by itself for me

I am running in a centos dedicated server with python 2.6

Is this proxy works with quark algorithm?

Love the program, I too wish to see a stratum version soon, as well.

One annoyance at the moment I would like to know how to solve, if possible, is getting the connection error to reset to normal after the miner disconnects. Right now it keeps saying:

"[Errno 10054] An existing connection was forcibly closed by the remote host"

After the miner is disconnected but I know it is running just fine after I reconnect. The current solution is to simply just close and reopen Rollproxy but I will lose my collected stats if I do so. So it's either lose my stats or not have a log in the mean time.

Thank you.