Bitcoin Forum
December 11, 2016, 06:29:11 AM *
News: To be able to use the next phase of the beta forum software, please ensure that your email address is correct/functional.
 
   Home   Help Search Donate Login Register  
Pages: « 1 2 3 4 5 6 7 8 9 10 11 [12] 13 14 15 16 17 18 19 »  All
  Print  
Author Topic: Multipool - the pool mining pool (with source code)  (Read 44990 times)
mf
Newbie
*
Offline Offline

Activity: 25


View Profile
June 29, 2011, 11:36:38 PM
 #221

What license is the code under?

I've downloaded it and started rewriting some ugly bits, removing embedded defaults, using JSON for the config files, etc.

I would like to make my changes available, but would rather know the license they'd be supposed to be under.

I'd then be putting the changes on Github as soon as I get them done, and I know which license they're supposed to be under.

Donations gladly accepted at 1QA1K3Ghz9AuJ8na6JKJVudEko3WKx1xC2
1481437751
Hero Member
*
Offline Offline

Posts: 1481437751

View Profile Personal Message (Offline)

Ignore
1481437751
Reply with quote  #2

1481437751
Report to moderator
1481437751
Hero Member
*
Offline Offline

Posts: 1481437751

View Profile Personal Message (Offline)

Ignore
1481437751
Reply with quote  #2

1481437751
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1481437751
Hero Member
*
Offline Offline

Posts: 1481437751

View Profile Personal Message (Offline)

Ignore
1481437751
Reply with quote  #2

1481437751
Report to moderator
Zoomer
Hero Member
*****
Offline Offline

Activity: 644



View Profile
June 30, 2011, 12:03:24 AM
 #222

Multipool, are you going to be bringing the pool at hpc.tw back up or will you defer to Multiclone? I need some help to make up for the lost shares.
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
June 30, 2011, 12:16:07 AM
 #223

What license is the code under?

I've downloaded it and started rewriting some ugly bits, removing embedded defaults, using JSON for the config files, etc.

I would like to make my changes available, but would rather know the license they'd be supposed to be under.

I'd then be putting the changes on Github as soon as I get them done, and I know which license they're supposed to be under.

Nice Smiley

Thought about forking a pool version and a stand alone client? The stand alone wouldn't need as much in config and might be easier for some folks (ahem) to set up.

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
iopq
Hero Member
*****
Offline Offline

Activity: 644


View Profile
June 30, 2011, 02:30:57 AM
 #224

is multiclone up? I'm getting connection errors

nick5429
Member
**
Offline Offline

Activity: 70


View Profile
June 30, 2011, 02:51:56 AM
 #225

is multiclone up? I'm getting connection errors

Working for me and at least 5 other people right now.

Post your full miner string?

Computer Engineering professional by day, tinkerer and Bitcoin miner by night.
Multiclone operator -- a Multipool clone
1GaZUsCAdUNbvdwFToZenkDDxAPi6ULavA
iopq
Hero Member
*****
Offline Offline

Activity: 644


View Profile
June 30, 2011, 03:11:52 AM
 #226

is multiclone up? I'm getting connection errors

Working for me and at least 5 other people right now.

Post your full miner string?
poclbm.exe --user=1HHroZyBvQQtsp6bSyQ9rpYgDpN2RoTrUM --pass=pass -o multiclone.us.to -p 18080 --device=1 --platform=0 --verbose -v -w128 -f30

organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
June 30, 2011, 03:16:13 AM
 #227

is multiclone up? I'm getting connection errors

Working for me and at least 5 other people right now.

Post your full miner string?
poclbm.exe --user=1HHroZyBvQQtsp6bSyQ9rpYgDpN2RoTrUM --pass=pass -o multiclone.us.to -p 18080 --device=1 --platform=0 --verbose -v -w128 -f30

mining port is 18337, not 18080(that's the website port)

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
iopq
Hero Member
*****
Offline Offline

Activity: 644


View Profile
June 30, 2011, 03:27:00 AM
 #228

is multiclone up? I'm getting connection errors

Working for me and at least 5 other people right now.

Post your full miner string?
poclbm.exe --user=1HHroZyBvQQtsp6bSyQ9rpYgDpN2RoTrUM --pass=pass -o multiclone.us.to -p 18080 --device=1 --platform=0 --verbose -v -w128 -f30

mining port is 18337, not 18080(that's the website port)
thanks, that fixed it

JDog
Newbie
*
Offline Offline

Activity: 13


View Profile
June 30, 2011, 04:12:50 AM
 #229

Hey guys, I hope you can help with this one.

I'm trying to set up Multipool on a Win7 machine (yes, I'm a masochist), and things seem to be more or less working.  I can see Multipool connecting to the regular pools and get shares.  I can see my miner connect to it, but for some reason the miner never receives any shares from Multipool.  The miner sits connected, but idle and no errors on either side.  I've played around with ports and I'm sure it's connecting properly (firewall deactivated and such), but I just can't get the miner to get any shares.

I'm a coding 'hobbiest', but I've learned Perl in the last 48 hours for this project.  I understand about 1/2 of what the code does, but I can't find the section that assigns shares to the workers.  It could be a porting issue (had to change several lines to make it work with wget for windows), could be a config error(IP addresses or such), could be who the heck knows what??

I don't know what other kind of information anyone would need to help me, but I'm grasping at straws at this point.
Multipool
Newbie
*
Offline Offline

Activity: 24


View Profile
June 30, 2011, 04:46:52 AM
 #230

I had thrown the 'production' switch, which caused the work queue entries to start out at 30, but was only testing with about 300MHash/sec.  Occasionally (presumably after a new round began on the current pool or something), I would get dozens of invalid shares in a row from that pool.  Decreasing the work queue size seemed to mitigate the issue somewhat.

I occasionally saw work queue 'purge' messages going by in the log, but never noticed a non-zero number of entries being purged during these rejection streaks.

If you feel like explaining it, how is the work queue purging determined?  I know that Multipool-the-server implements Long Polling.  What about Multipool-the-client? 

For anyone running Multipool locally/with a fairly small hash rate, I might suggest decreasing the 'production' work queue size which defaults to 30 queue entries (around line 43) to avoid similar issues.  OTOH, maybe this is a non-issue and I was just having problems with Mt. Red.
Anyone running multipool locally will be satisfied leaving the $production switch set to false. Setting it to true does precisely the things like increasing the work queue size and rewriting all the addresses to external, rather than 127.0.0.1.

The work queue size expands or shrinks slowly to meet demand - it stores about 15 seconds worth of shares. When work is received from a pool with a different prev_block hash than before (substr($work->{data}, 16, Cool), all the work in the queue with the previous hash (including work from all other pools) is purged. About this time is also when longpoll is sent. The exact timing is a bit complicated, because while the pools typically, but not always, all work on the same block, the switching from one block to the next can happen up to 20 seconds apart between pools. So while you might have received a new share (and purged all old shares) from one pool, you might still receive shares with the old block hash from another pool for up to 20 seconds. Most of these shares will be rejected as stale once they are actually solved, so there is still some room for improvement in this area. Also, you don't want to send a longpoll signal ten times in 20 seconds, so you need a way to prevent double longpolls. For now, Multipool only sends longpoll when its own bitcoind reports a block hash change, but this could also be based on a cooldown timer.

I'm trying to set up Multipool on a Win7 machine (yes, I'm a masochist), and things seem to be more or less working.  I can see Multipool connecting to the regular pools and get shares.  I can see my miner connect to it, but for some reason the miner never receives any shares from Multipool.  The miner sits connected, but idle and no errors on either side.  I've played around with ports and I'm sure it's connecting properly (firewall deactivated and such), but I just can't get the miner to get any shares.

I'm a coding 'hobbiest', but I've learned Perl in the last 48 hours for this project.  I understand about 1/2 of what the code does, but I can't find the section that assigns shares to the workers.  It could be a porting issue (had to change several lines to make it work with wget for windows), could be a config error(IP addresses or such), could be who the heck knows what??

I don't know what other kind of information anyone would need to help me, but I'm grasping at straws at this point.
Amazing that it actually runs! I had imagined the biggest problem with the porting would have been the sockets. Perl on Windows possibly doesn't implement all socket features, like maybe nonblocking mode, or the can_read function of IO::Select. Look in the rpc_server_alt function that handles all client connections. There is a big while loop which iterates over all connected clients, reads any request data available up to that point, and then sends response data when sufficient request data (such as basic authentication) has been obtained. Trace the execution of this loop to see what runs normally and at what exact point the execution gets stuck.
M4v3R
Hero Member
*****
Offline Offline

Activity: 607



View Profile
June 30, 2011, 04:54:11 AM
 #231

Hm... I'm still left with some questions. One thing is - is it normal that I see *many* rpc calls per second? I mean, calls like this:

Code:
got work 6c1ea670 b5d42a84 from deepbit (t=0.070/0.082 s/g=0/14 old=32/2209 p=0.00)
got work 6c1ea670 259bd031 from mtred (t=0.118/1.508 s/g=0/12 old=3/44 p=0.00)
got work 6c1ea670 dde84185 from btcmine (t=0.154/0.157 s/g=0/8 old=30/3192 p=0.00)
rpc connection opened from 83.26.173.182:52750
rpc authorization from 83.26.173.182:52747: user=--- pass=x
rpc getwork from 83.26.173.182:52747 (deepbit 6c1ea670 dddc356d) queue: 15/103w 0so 0sh
got work 6c1ea670 3128bc4f from deepbit (t=0.082/0.082 s/g=0/15 old=32/2209 p=0.00)
rpc connection opened from 83.26.173.182:33185
rpc authorization from 83.26.173.182:52750: user=--- pass=x
rpc getwork from 83.26.173.182:52750 (btcmine 6c1ea670 c25c0819) queue: 15/103w 0so 0sh
got work 6c1ea670 6eda19aa from mtred (t=0.106/1.368 s/g=0/13 old=3/44 p=0.00)

I only use two workers so I assumed that it would not generate much traffic, and then it does Smiley.

My other question is - will you still be working on this project? I already saw an error in the log while parsing rewards, probably for BTCMine. I've looked at the rewards functions, but they are somewhat complicated and I doubt I'll manage to fix that myself without breaking something Smiley.
rb2k
Member
**
Offline Offline

Activity: 109


View Profile
June 30, 2011, 05:12:54 AM
 #232

Another thing I notice is a lot of those:

Code:
connection timeout to eligius-eu (p=0.49)
connection timeout to btcmine (p=30.00)
connection timeout to btcguild (p=30.00)


Seems a bit strange
M4v3R
Hero Member
*****
Offline Offline

Activity: 607



View Profile
June 30, 2011, 05:14:32 AM
 #233

BTC Guild is offline sometimes, depending on which server do you use (they have several). Eligius also likes being down from time to time. Don't know about BTCMine.
organofcorti
Donator
Legendary
*
Offline Offline

Activity: 1960


Poor impulse control.


View Profile WWW
June 30, 2011, 05:32:16 AM
 #234

And eligius-eu is no more. The new url is in the eligius thread in Pools.

Bitcoin network and pool analysis 12QxPHEuxDrs7mCyGSx1iVSozTwtquDB3r
follow @oocBlog for new post notifications
M4v3R
Hero Member
*****
Offline Offline

Activity: 607



View Profile
June 30, 2011, 05:47:41 AM
 #235

One more thing: I noticed that there is no reward function for bitcoins-lc, even though it is supported in the pools.conf. Does that mean that the script won't be able to distribute payments from bitcoins-lc to my workers?
Multipool
Newbie
*
Offline Offline

Activity: 24


View Profile
June 30, 2011, 05:51:46 AM
 #236

Hm... I'm still left with some questions. One thing is - is it normal that I see *many* rpc calls per second? I mean, calls like this:

Code:
got work 6c1ea670 b5d42a84 from deepbit (t=0.070/0.082 s/g=0/14 old=32/2209 p=0.00)
got work 6c1ea670 259bd031 from mtred (t=0.118/1.508 s/g=0/12 old=3/44 p=0.00)
got work 6c1ea670 dde84185 from btcmine (t=0.154/0.157 s/g=0/8 old=30/3192 p=0.00)
rpc connection opened from 83.26.173.182:52750
rpc authorization from 83.26.173.182:52747: user=--- pass=x
rpc getwork from 83.26.173.182:52747 (deepbit 6c1ea670 dddc356d) queue: 15/103w 0so 0sh
got work 6c1ea670 3128bc4f from deepbit (t=0.082/0.082 s/g=0/15 old=32/2209 p=0.00)
rpc connection opened from 83.26.173.182:33185
rpc authorization from 83.26.173.182:52750: user=--- pass=x
rpc getwork from 83.26.173.182:52750 (btcmine 6c1ea670 c25c0819) queue: 15/103w 0so 0sh
got work 6c1ea670 6eda19aa from mtred (t=0.106/1.368 s/g=0/13 old=3/44 p=0.00)

I only use two workers so I assumed that it would not generate much traffic, and then it does Smiley.

My other question is - will you still be working on this project? I already saw an error in the log while parsing rewards, probably for BTCMine. I've looked at the rewards functions, but they are somewhat complicated and I doubt I'll manage to fix that myself without breaking something Smiley.

A work queue size of 103? Do you see like ten requests per second? That's not supposed to happen - usually it's one request every five seconds per miner. I've seen this before though - what miner version are you using? Does rebooting the server help?
And yes, I will work on the project.

One more thing: I noticed that there is no reward function for bitcoins-lc, even though it is supported in the pools.conf. Does that mean that the script won't be able to distribute payments from bitcoins-lc to my workers?
There isn't a function for bitcoins.lc yet - they don't show per-round earnings and it will require some magic to sort the rounds out.
M4v3R
Hero Member
*****
Offline Offline

Activity: 607



View Profile
June 30, 2011, 05:56:18 AM
 #237

A work queue size of 103? Do you see like ten requests per second? That's not supposed to happen - usually it's one request every five seconds per miner. I've seen this before though - what miner version are you using? Does rebooting the server help?
And yes, I will work on the project.

Yes, that's exactly what I'm seeing. And I have the exact opposite - over time the request rate decreases. Restarting the server brings it back up.

EDIT: Now I restarted the server and the log suggests that I have queue size of 5, still I see like 10 requests per second on the screen.

Oh, and I use Multiminer. Maybe that's the culprit.

One more thing: I noticed that there is no reward function for bitcoins-lc, even though it is supported in the pools.conf. Does that mean that the script won't be able to distribute payments from bitcoins-lc to my workers?
There isn't a function for bitcoins.lc yet - they don't show per-round earnings and it will require some magic to sort the rounds out.

I guess there is now, at URL: https://www.bitcoins.lc/transactions



And yes, I will work on the project.

That's great news! Smiley. A big thank you from me.
JDog
Newbie
*
Offline Offline

Activity: 13


View Profile
June 30, 2011, 06:36:23 AM
 #238

Quote
Amazing that it actually runs! I had imagined the biggest problem with the porting would have been the sockets. Perl on Windows possibly doesn't implement all socket features, like maybe nonblocking mode, or the can_read function of IO::Select. Look in the rpc_server_alt function that handles all client connections. There is a big while loop which iterates over all connected clients, reads any request data available up to that point, and then sends response data when sufficient request data (such as basic authentication) has been obtained. Trace the execution of this loop to see what runs normally and at what exact point the execution gets stuck.

So I've gone through the rpc_server_alt function and it doesn't appear to be hanging anywhere, the loop runs normally whether the worker is connected or not.  But what I did find that seemed odd, was the it moves into code that appears to process the getwork request *as the worker disconnects*.  Here's a short section of code I'm referring too.  (Can't give line numbers as the unix2dos converter added a ton of whitespace, in theory it's after line 1400)

Code:
   } else {
if ($vars->{longpoll}){
   $vars->{rec}=2;
   last;
}
$user->{getworks}++;
my $user_ip=$user->{host}; $user_ip=~s/:.*//;
my $limiter=$user_limiter{$user_ip};
my $lawful=user_lawful($limiter);
my $pair;
if ($lawful){ #only give good shares to lawful users                       <-indicative comment
   $pair = $work_queue->dequeue_nb;                                       <-confirmed code is executed only when worker is disconnecting
}
if (!$pair){
   $pair=$solo_queue->dequeue;
}
$limiter->{got_works}++ if $limiter;;
my ($work, $pool_name) = @{$pair};
{
   lock $work_dequeues;
   $work_dequeues++;
}
logd("rpc getwork from $vars->{host} ($pool_name ".substr($work->{data}, 16, 8)." ".substr($work->{data}, 72, 8).") queue: ".$work_queue->pending."/".$WORK_QUEUE_SIZE."w ".$solo_queue->pending."so ".$submit_queue->pending."sh\n");

The rpc getwork message does appear, but only when the worker disconnects.

Again, I'm not strong with my Perl, so maybe this code is only supposed to run during disconnection, but the comment and the rest of the code implies it's removing a getwork from the internal queue and giving it to the worker.  Would this be accurate?

Multipool
Newbie
*
Offline Offline

Activity: 24


View Profile
June 30, 2011, 07:10:12 AM
 #239

The rpc getwork message does appear, but only when the worker disconnects.
This would be the exact result if nonblocking mode on the socket doesn't work. Blocking is turned off at line 1717:
Code:
$incoming->blocking(0);
Then check out line 1583:
Code:
while (<$c>){ #nonblocking
In nonblocking mode, this statement will slurp a whole line, or as much of a line as the server has received from the client. In blocking mode, this statement will wait for more data from the client until a whole line is available, or the socket is closed. The magic trick is that json rpc requests are not terminated by line breaks! So the loop hangs here waiting for an end-of-line that never comes, until the client closes the connection. At that point, since no more data will be available, the <$c> statement returns, the while loop iterates one final time, and the server tries to send a response through the closed socket. To make this work without nonblocking mode, you would need to use the perl read function on the socket - either read content-length characters after receiving a single empty line (the separator between the header and the content in http packets), or read a single character at a time until you get a valid full json object string. The dangers here is that some pools use chunked encoding instead of content length, and that you must trust your miners to send valid requests. A single malicious request (or even accidental network-problem-related) with a content-length header, but only partial content, will hang the entire rpc server!
rb2k
Member
**
Offline Offline

Activity: 109


View Profile
June 30, 2011, 11:51:30 AM
 #240

@M4v3R on your multipool, all of the "earned" columns seem stuck on "pending", how do the payouts work with your multipool? Smiley
Pages: « 1 2 3 4 5 6 7 8 9 10 11 [12] 13 14 15 16 17 18 19 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!