Bitcoin Forum

Bitcoin => Mining => Topic started by: jgarzik on February 01, 2011, 08:22:06 AM



Title: New mining pool for testing (now closed)
Post by: jgarzik on February 01, 2011, 08:22:06 AM
Over the next week or two, I am testing some new mining pool software.  If you can spare a miner for an hour or day or three, testing is greatly appreciated.

Worker registration and basic pool details at http://pool.yyz.us/register.html

This is alpha quality software right now -- still have not found a single block.  Payments are made on a best effort basis; this is brand new software and bugs are guaranteed.  The server will go up and down as software updates are applied, so don't be surprised if you occasionally see connection errors.

Please send bug reports and other feedback to jgarzik@gmail.com.  Thanks!


Title: Re: New mining pool for testing
Post by: ElectricGoat on February 01, 2011, 09:09:04 AM
I registered a worker and found a share, so that part works. Having a statistics page would be nice, but I suppose that's already on the todo list.


Title: Re: New mining pool for testing
Post by: motherhumper on February 01, 2011, 09:32:09 AM
registered a worker, mining now...

address should be without http:// shouldnt it?

Connect your miner to http://pool.yyz.us:8334/ using the username/password you registered in the above form.


Title: Re: New mining pool for testing
Post by: ColdHardMetal on February 01, 2011, 10:10:32 AM
In the name of science I fired up my GPU and connected to the pool. I'm not sure if it's connected properly but I have a hash rate so something must be going right.

edit: I'm getting "accepted" messages so it looks like its doing things just fine.


Title: Re: New mining pool for testing
Post by: BitCoinPurse on February 01, 2011, 01:31:50 PM
Over the next week or two, I am testing some new mining pool software.  If you can spare a miner for an hour or day or three, testing is greatly appreciated.

Happy to help...


Title: Re: New mining pool for testing
Post by: jgarzik on February 01, 2011, 06:41:26 PM
A quick status update:

Initial testing seems to show the pool database working correctly.  Shares are being found, and correctly being credited to workers.

Still waiting on that first found block (and associated code), so more test workers are still wanted!

I should have a stats page up in 24-48 hours.  Thanks to everyone participating in this test.


Title: Re: New mining pool for testing
Post by: ElectricGoat on February 01, 2011, 06:57:16 PM
I get "Problems communicating with bitcoin RPC" errors after a short while. Does it mean that you restarted the pool, or is it something else ?


Title: Re: New mining pool for testing
Post by: jgarzik on February 01, 2011, 09:15:51 PM
I get "Problems communicating with bitcoin RPC" errors after a short while. Does it mean that you restarted the pool, or is it something else ?

The pool will probably see many restarts in the first 24-48 hours of its lifetime.

Additionally, I am currently using python's BaseHTTPServer, which seems to sometimes pause for several seconds before giving up on a socket, and processing more.  As soon as this is turned into a multi-threaded server, that should not be a problem anymore.


Title: Re: New mining pool for testing
Post by: Cdecker on February 01, 2011, 10:08:20 PM
Any chances of an open source pool?


Title: Re: New mining pool for testing
Post by: jgarzik on February 02, 2011, 12:20:40 AM
The pool server was upgraded to a more robust multi-threaded model, which I'm hoping solves some of the timeout issues reported.

Email (or post here) any questions or issues.  Still searching for that first block, so the payout code has not yet been exercised.  If you can spare some GPU/CPU power for testing this pool, it is appreciated.

Right now the pool is around 9 getwork/second.

Stats are definitely on the todo list.


Title: Re: New mining pool for testing
Post by: LZ on February 02, 2011, 01:06:49 AM
What is the current hash/s value?


Title: Re: New mining pool for testing
Post by: jgarzik on February 02, 2011, 01:35:02 AM
What is the current hash/s value?

How does one calculate that?


Title: Re: New mining pool for testing
Post by: slush on February 02, 2011, 02:08:04 AM
How does one calculate that?

shares * 2**32 / round time


Title: Re: New mining pool for testing
Post by: BitCoinPurse on February 02, 2011, 02:15:16 PM
Running much more smoothly today...


Title: Re: New mining pool for testing
Post by: cdb000 on February 02, 2011, 03:23:54 PM
I have added 1 thread on a 2.6GHz i7 -- about 1k hash/second.




Title: Re: New mining pool for testing
Post by: jgarzik on February 02, 2011, 04:40:28 PM
The pool found its first block last night, block # 105839 (http://blockexplorer.com/b/105839).

Now that a block is in the database, work on statistics may commence.


Title: Re: New mining pool for testing
Post by: davout on February 02, 2011, 04:58:04 PM
Still running?
 i get a problem communicating with bitcoin RPC
Just changed the address, port, user and pass from the script I use to pool with slush


Title: Re: New mining pool for testing
Post by: jgarzik on February 02, 2011, 05:46:05 PM
Still running?
 i get a problem communicating with bitcoin RPC
Just changed the address, port, user and pass from the script I use to pool with slush

Yes, still running with plenty of miners attached.


Title: Re: New mining pool for testing
Post by: baitzor on February 02, 2011, 07:46:21 PM
Any chances of an open source pool?


I would very much be interested in this!
I'm trying to setup a mining pool for me and my friends who've I've converted to bitcoin!

Where does one get started on writing a pooled server? Considering coding in any language is allready known. (c/c++ are preferred)

^_^


Title: Re: New mining pool for testing
Post by: jgarzik on February 02, 2011, 08:39:34 PM
This is GPL'd: http://yyz.us/bitcoin/poold.py

Though it's not necessarily exactly what I'm running on the pool at any given time, as I'm constantly tweaking and fixing the server.


Title: Re: New mining pool for testing
Post by: slush on February 02, 2011, 09:06:30 PM
This is GPL'd: http://yyz.us/bitcoin/poold.py

It would be nice to say to potential users that this version is vulnerable and may not be used in production.


Title: Re: New mining pool for testing
Post by: zipslack on February 03, 2011, 03:16:33 AM
It would be nice to say to potential users that this version is vulnerable and may not be used in production.

Do you care to explain that statement? It resembles a baseless accusation.  ;)


Title: Re: New mining pool for testing
Post by: jgarzik on February 03, 2011, 04:02:38 AM
It would be nice to say to potential users that this version is vulnerable and may not be used in production.

Do you care to explain that statement? It resembles a baseless accusation.  ;)

slush is right, but he's being terse because we discussed the vulnerabilities in private...  Once my further changes survive testing on this new mining pool, I'll update the code again.


Title: Re: New mining pool for testing
Post by: ColdHardMetal on February 03, 2011, 05:51:32 AM
The pool found its first block last night, block # 105839 (http://blockexplorer.com/b/105839).

Now that a block is in the database, work on statistics may commence.

My miner has been connected for a few days so should have some shares in that block. It's slow so it wouldn't have been a large %. Do you have a minimum BTC level threshold before you send coins out?


Title: Mining pool test results
Post by: jgarzik on February 03, 2011, 11:11:34 PM
This mining pool test has been a smashing success.  Thanks to all who helped out with their CPU/GPU cycles.  Here's a quick briefing.


1. What was being tested?
--------------------------------------------------------------------
I'm working on a suite of bitcoin-related services.  Mining pool capability is a small part of that.  This was a live test of code that will be deployed in production later on.  We found many bugs that a static, in-house test could have never found.  slush was also very generous with advice and cautionary warnings.

To thank the community for the help, the core server source code has been GPL'd and released:  http://yyz.us/bitcoin/poold.py   However, it should be noted that that server is missing several data validations that need to occur (to prevent cheating).



2. What is the current status of the pool?
--------------------------------------------------------------------
Only one block was ever generated by the pool (which IMO is fantastic, given current difficulty, short notice and informal nature of this new pool)

You can get real-time stats at http://pool.yyz.us/stats.cgi

As I consider this a concluded test, I will be shutting down the pool next week -- hopefully enough time to generate another block for the remaining folks who contributed shares.  I have switched my own GPU miners back to slush's pool, though my CPU miners are still pointed at pool.yyz.us.



3. What is the status of the payout -- where my money?
-------------------------------------------------------------------------------------------
We have one block (50 BTC), which was confirmed 87 blocks ago as of this writing.

Based on a technical reading of slush's site, I wrote the software so that only miners who got shares in the block we generated got credit:  "When the pool mints a block, only users who worked on that block are rewarded, and only for work they did on that block."  But apparently I wrote the software incorrectly, because people expected to be rewarded based on shares contributed to all blocks up to and including the newly minted 50BTC block.

So according to my software, this is the winning distribution based on those who worked on block 105839,
     mine319.pit00     211
     mine300pit04     48
     BitCoinPurse     7
     worker102     2
     immaminer     2
     Raulo     2
     wtf     2
     test     1
     mine319.pit00     1

That excludes shares contributed on blocks prior to 105839, and some ~18 hours of shares in the current round, and those records cannot easily be recreated due to since-fixed software bugs.

So, since it's only 50 BTC and I don't want to argue endlessly about it, as a compromise I am going send 1.00 BTC to each of 41 registered users, and then divide the remaining 9 BTC up according to the distribution above.

If the mining pool mints another 50 BTC block in the next ~5 days or so, those share records are present, and I can do a real, slush-like distribution of the reward.  After that, the pool server will be shut down.



4.  What comes next?
---------------------------------------------
Here are some tantalizing hints:
  • push-based mining (current RPC miners poll for new work every few seconds)
  • geographically distributed mining servers in the US, Eur and Asia, for best possible latency
  • additional share rewards -- sometimes we might want to compensate you above the 50 BTC reward itself, because your mining increases the security and value of our network


Title: Re: Mining pool test results
Post by: jgarzik on February 04, 2011, 12:28:29 AM
So, since it's only 50 BTC and I don't want to argue endlessly about it, as a compromise I am going send 1.00 BTC to each of 41 registered users, and then divide the remaining 9 BTC up according to the distribution above.

This payout has been made.


Title: Re: New mining pool for testing
Post by: LZ on February 04, 2011, 12:39:49 AM
Got it, thanks! By the way, what indices have I to create for users, shares and credits tables in the database?


Title: Re: New mining pool for testing
Post by: Anonymous on February 04, 2011, 01:12:02 AM
Added my miniscule khashes to the pool lol.


Protip - a pentium D cpu is not the best for mining  :D




Title: Re: New mining pool for testing
Post by: jgarzik on February 04, 2011, 01:22:21 AM
Got it, thanks! By the way, what indices have I to create for users, shares and credits tables in the database?

Not sure I want to give out all the secret sauce ;)  Left as an exercise to the reader.


Title: Re: Mining pool test results
Post by: slush on February 04, 2011, 09:26:30 AM
and those records cannot easily be recreated

So, shortly said, this is a scam. gene, where are you???

(edit: I hope it is obvious that I'm kidding.)


Title: Re: New mining pool for testing
Post by: BitCoinPurse on February 04, 2011, 01:51:39 PM
I got my 1.23 BTC, which will be worth $2,856,092,198.89 someday! 

I enjoyed helping the experiment.


Title: Re: Mining pool test results
Post by: Raulo on February 04, 2011, 02:04:57 PM
Based on a technical reading of slush's site, I wrote the software so that only miners who got shares in the block we generated got credit:  "When the pool mints a block, only users who worked on that block are rewarded, and only for work they did on that block."  But apparently I wrote the software incorrectly, because people expected to be rewarded based on shares contributed to all blocks up to and including the newly minted 50BTC block.

But what you programmed is actually a perfect pool-abuse countermeasure that I realized after reading about your method:
http://bitcointalk.org/index.php?topic=3165.0

Instead of switching to slush's method, you should keep this one.


Title: Re: New mining pool for testing
Post by: jgarzik on February 04, 2011, 11:44:14 PM
The pool will be shut down Monday at 11:59pm EST.  Until then, I'm keeping my CPU miners going on it, at least :)


Title: Re: New mining pool for testing
Post by: motherhumper on February 05, 2011, 11:16:38 AM
 :D lol $2,856,092,198.89 per bitcoin?  how did you com up with that number?  ;D


Title: Re: New mining pool for testing
Post by: jgarzik on February 08, 2011, 04:59:00 AM
The pool has been shut down.  Thanks to all who helped test!  Only one block was found during this test, from beginning to end.


Title: Re: New mining pool for testing
Post by: baitzor on February 10, 2011, 03:41:51 AM
I'm interested in starting my own mining server, got most of it down, all I have to implement is the getwork method, for geting new hashes, is the latest poold.py the one I should be looking at?

Where did you get starting on reading about getwork, or did you just look up the code from the bitcoin client?


Title: Re: New mining pool for testing
Post by: jgarzik on February 10, 2011, 04:48:46 AM
I'm interested in starting my own mining server, got most of it down, all I have to implement is the getwork method, for geting new hashes, is the latest poold.py the one I should be looking at?

Where did you get starting on reading about getwork, or did you just look up the code from the bitcoin client?

You will need to intimately understand the mining process, notably the binary layout of the 80-byte bitcoin block headers.  The best reference is the C++ source code for the official bitcoin client.


Title: Re: New mining pool for testing
Post by: Cdecker on February 10, 2011, 11:27:27 AM
The pool has been shut down.  Thanks to all who helped test!  Only one block was found during this test, from beginning to end.
I assume it didn't go all that well, I'm sorry to hear it. It's always been my opinion that if we really must have pools (instead of running hundreds of independent miners) we should have many of them to chose from, as to avoid central points of failure. Should slush turn out to be taken down or replaced by a malicious entity we'd have a problem at hand.

Meanwhile, thank you for publishing part of your code, I'll use it in my own mining rig if that's ok, and if someone can confirm that it actually works :D


Title: Re: New mining pool for testing
Post by: jgarzik on February 10, 2011, 07:37:44 PM
The pool has been shut down.  Thanks to all who helped test!  Only one block was found during this test, from beginning to end.
I assume it didn't go all that well, I'm sorry to hear it.

Scroll up for test results ("smashing success").

Quote
It's always been my opinion that if we really must have pools (instead of running hundreds of independent miners) we should have many of them to chose from, as to avoid central points of failure. Should slush turn out to be taken down or replaced by a malicious entity we'd have a problem at hand.

I agree, the more pools, the better for bitcoin.  That's why I published my code!


Title: Re: New mining pool for testing
Post by: dishwara on February 11, 2011, 08:08:16 AM
U saying u closed the mining pool?

coz i just registered & try to run & got error

problem communicating with RPC....


Title: Re: New mining pool for testing
Post by: adv on February 13, 2011, 05:32:55 AM
When will next run a pool? This is particularly relevant in the context of the problems with the Slush's pool.


Title: Re: New mining pool for testing
Post by: Cdecker on February 13, 2011, 02:05:17 PM
I'm trying to build my own small pool which I'd use to gather statistics about the miners I'm deploying. Would that script work for that purpose? I guess that validation is not necessary if I trust my miners, right?


Title: Re: New mining pool for testing
Post by: jgarzik on February 13, 2011, 04:36:07 PM
I'm trying to build my own small pool which I'd use to gather statistics about the miners I'm deploying. Would that script work for that purpose?

Yes.

Quote
I guess that validation is not necessary if I trust my miners, right?

Correct.


Title: Re: New mining pool for testing
Post by: Txyru on February 21, 2011, 10:50:06 PM
Maybe someone can help me here to get poold.py setup.

I recreated the database thats needed, but the pool server keeps rejecting all the CHECKWORKs for my own pc using m0mchill's gpu miner. It works fine with bitpenny, but I dont know why its getting rejected. In my client it says work accepted, but on the server in the cmd window it says that it has been rejected, doesnt give a reason, and doesnt give credit.

Here are some screenshots...

https://i.imgur.com/LU8y1.png

EDIT: I confirmed that this is actually CORRECT operation and that credits are only given when a block is found. False alarm.

Also, for anyone who wants to use poold.py, here is the SQL you need to recreate the database to get started...

Code:
BEGIN TRANSACTION;
CREATE TABLE credits (username TEXT, height NUMERIC, type TEXT, v TEXT, datetime NUMERIC);
CREATE TABLE shares (username TEXT, height NUMERIC, datetime NUMERIC, accepted TEXT, datastr TEXT);
CREATE TABLE users (username TEXT, password TEXT);
COMMIT;


Title: Re: New mining pool for testing
Post by: geebus on March 12, 2011, 02:03:32 AM
PATCH FOR POOLD.PY TO ALLOW SUPPORT FOR UFASOFT'S CPU MINER:

Original - Lines 319 - 323:
Code:
	auth = s.headers.get('Authorization')
auth_user = checkauth(db, auth)
if auth_user is None:
s.send_error(401, "not authorized")
return

Change to:
Code:
                auth = s.headers.get('Authorization')
                if auth is None:
                        s.send_response(401)
                        s.send_header('WWW-Authenticate', 'Basic realm=\"poold\"')
                        s.send_header('Content-type', 'text/html')
                        s.end_headers()
                        auth = s.headers.get('Authorization')
                auth_user = checkauth(db, auth)
                if auth_user is None:
                        s.send_error(401, "not authorized")
                        return


Title: Re: New mining pool for testing
Post by: snedie on March 13, 2011, 07:49:03 PM
You still looking for testers? I'm willing to donate a single miner running a 5970 core. What method are you using to share the bitcoins earned?


Title: Re: New mining pool for testing
Post by: jgarzik on March 13, 2011, 08:29:18 PM
You still looking for testers? I'm willing to donate a single miner running a 5970 core. What method are you using to share the bitcoins earned?

The pool test has been closed for a few weeks.  I'll update the top post.