Bitcoin Forum

Bitcoin => Project Development => Topic started by: Zagitta on June 18, 2011, 10:13:14 PM



Title: Interrest for a new pool backend?
Post by: Zagitta on June 18, 2011, 10:13:14 PM
Would anyone be interrested in a pool backend written in C# with support for proxying work so i both can act as a public pool and a private home pool for multiply rigs?

The reason i'm asking is because I used Flexible mining proxy (http://forum.bitcoin.org/index.php?topic=5506.0) for some time and i really like the concept but i don't like having 20% stales (im also a php noob so i can't help with the development) so i decided to make my own implementation in C# and i figured it wouldn't require too much extra work to add support the slightly different features a public pool would require as most of the difference lies in the frontend...

All of the needed functions would be exposed with both Json-rpc and MySQL to ease frontend development, hell i might even do some WCF and named pipes if a ASP.net dev wanted to make a frontend :)

However as i said before i go out and invest my summervacation in developing features i don't need I'd just like to hear if there's any interrest :)

oh yeah i plan to keep it opensource and mono compatible so it can run on a linux server :)


---edit---

now on github: https://github.com/Zagitta/Zetlon.Coinster


Title: Re: Interrest for a new pool backend?
Post by: Zagitta on June 20, 2011, 12:24:54 AM
Just sucessfully redirected packets to and from BTCGuild:

http://img607.imageshack.us/img607/2752/progressh.jpg (http://imageshack.us/photo/my-images/607/progressh.jpg/)

:)


Title: Re: Interrest for a new pool backend?
Post by: Coaster on June 20, 2011, 05:00:52 AM
Hells yeah man! I'd love to see this. Not only am I interested in running my own pool, but the concept of multi-pooling interests me a ton.


Title: Re: Interrest for a new pool backend?
Post by: codler on June 20, 2011, 07:47:58 AM
Just sucessfully redirected packets to and from BTCGuild:

http://img607.imageshack.us/img607/2752/progressh.jpg (http://imageshack.us/photo/my-images/607/progressh.jpg/)

:)

How do you redirect packets?


Title: Re: Interrest for a new pool backend?
Post by: Zagitta on June 20, 2011, 04:56:41 PM
@Codler System.Net.HttpListener (http://msdn.microsoft.com/en-us/library/system.net.httplistener.aspx)


Title: Re: Interrest for a new pool backend?
Post by: codler on June 21, 2011, 09:41:46 AM
thanks


Title: Re: Interrest for a new pool backend?
Post by: Coaster on June 21, 2011, 03:41:10 PM
any chance of bringing this to a release?
maybe even working with the pushpool frontends others have previously developed?


Title: Re: Interrest for a new pool backend?
Post by: Zagitta on June 21, 2011, 06:59:40 PM
It's now here done yet but i'll release it one i feel comfortable with releasing it as an alpha :)

And i'll see what i can do on the pushpool frontend part, it probably won't be a first priority but maybe i'll create it as an extension later when the core gets up and running.

BTW if anyone want's to help coding just poke me and i'll set up SVN/Git :)


Title: Re: Interrest for a new pool backend?
Post by: Sukrim on June 21, 2011, 09:17:17 PM
I'd love to see it done in Python, but I guess there would be something to learn from your code too, so I'm all for you putting it on Github! ;)


Title: Re: Interrest for a new pool backend?
Post by: Zagitta on June 21, 2011, 11:32:33 PM
I'd love to see it done in Python, but I guess there would be something to learn from your code too, so I'm all for you putting it on Github! ;)

I'll see if i can get it done tomorrow after my last exam :)


Title: Re: Interrest for a new pool backend?
Post by: Zagitta on June 22, 2011, 08:53:13 PM
I'd love to see it done in Python, but I guess there would be something to learn from your code too, so I'm all for you putting it on Github! ;)

Did i hear Github? oh yeah finally got it up and running: https://github.com/Zagitta/Zetlon.Coinster :)

Right now the code is ugly as hell and hackish but i'll start cleaning it up soon and make it more modular so it supports different protocols ETC


Title: Re: Interrest for a new pool backend?
Post by: Coaster on June 23, 2011, 04:48:31 PM
very awesome. thanks for putting it on git.


Title: Re: Interrest for a new pool backend?
Post by: Xenland on June 23, 2011, 05:03:25 PM
Keep me posted and ill make it an option in my front end software


Title: Re: Interrest for a new pool backend?
Post by: Zagitta on June 26, 2011, 12:51:40 PM
First let me appologise for not having made any progress the past few days, just graduated from highschool so there's been a lot partying and stuff haha (and yes i'm only 18 but don't worry, i've been actively learning programming myself programming for the past 3-4 years)

Keep me posted and ill make it an option in my front end software

Awesome! Prehaps we should talk on IRC/IM at some point to establish wich way the backend and frontend should communicate (mysql db schemes etc)


Title: Re: Interrest for a new pool backend?
Post by: Zagitta on July 01, 2011, 10:25:07 PM
Okay i finalely managed to clean it up proplerly and extract interfaces for modularity aswell as some simple IoC that should help easing up configuration :)

check it out!

There's a bug atm tough that results in the miner not getting any work after having finished the first round of work, i'll have to look into that :3


Title: Re: Interrest for a new pool backend?
Post by: Capitan on July 02, 2011, 07:06:47 AM
I'm pretty interested in helping with the coding. I'm a pro dev and very good at C#, high performance code, and database architecting and tuning. Only problem is I have no idea what the underpinnings of a mining pool look like. For instance what are stale shares?  Why are you redirecting packets for an existing pool?  Are you setting up a proxy that miners like pobclm or phoenix would point to, and the proxy would manage getting work assigned for different pools?

I guess, can you explain in more general detail, to someone who hasn't reallly learned about how mining internals work yet, what problem you are trying to solve / what you are creating / how it will solve it?


Title: Re: Interrest for a new pool backend?
Post by: Zagitta on July 02, 2011, 12:50:45 PM
To be honest i don't know all that much about it either but as i'm a quick learner i just learn it as i go... Stale shares as far as i know are shares that was potential solutions to the previous block (remember that the network aims for a block to be solved every 10 minutes) but since these no longer are valid to the pool they'll be counted as "stale shares", this is where longpoll comes into the picture because it's essentially a command for the miners to drop the hashing they're doing on the work they recieved previously and start on new work data.

The reason i'm redirecting stuff to an existing pool atm is for ease of testing as solving block to test the pool with my 70 mhash/s would take years and i'm too lazy to set up bitcoind with the test network :)
It's also in the plans that the backend should be able to work as not only a public pool but also a private proxy pool that much like you described would manage wich pools the miners get work from etc...

The main problem i'm trying to solve is that it's a real PITA to get started with pushpool (the current pool backend) and that there currently is no easy way to redirect your local miners from a webinterface :)

You're more than welcome to joining the team altough i'll have to warn you about my coding skills ;) I'm only selfthought as my highschool didn't offer any programming courses so the code is probably quite ugly to a professional dev ;D


Title: Re: Interrest for a new pool backend?
Post by: Capitan on July 03, 2011, 08:15:42 PM
Ah, thanks for that explanation on stale shares. That makes sense.

I'm still a little confused about what you are ultimately trying to do. So two questions:

1. What is ultimate goal? Are you trying to run a public pool?

2. What do you consider the "backend" of a pool? Is it the part that manages assigning work, and keeping track of what work is done, etc? I am confused by this because of the stuff you mentioned with redirecting packets. I thought backend would be on the "server" side of the pool. But if that's the case, why would you have to mess with sending packets to BTCGuild? That would imply that you are doing some work on the mining client side, and not the server side. I'm sure i'm missing something, can you please explain what?

Don't worry about your quality of code. Everyone has to learn somewhere. I'm sure we could learn from each other.


Title: Re: Interrest for a new pool backend?
Post by: markm on July 03, 2011, 08:49:08 PM
I would like to be able to have miners connect to a pool and not have to concern themselves at all about what the pool happens to be mining from moment to moment.

A whole bunch of different blockchains could thus avail themselves of mining power without the miners having to worry about whether any of those chains are ever going to amount to anything. The miners can simply do work for pay, being paid in whatever currency they like to be paid in, leaving the whole problem of which blockchain might or might be worth how much in terms of whatever currency the miner pays rent and groceries and electricity in to the pool operators. All the miner need care about is whether the pool is offering enough of the currency they want per unit of work they do.

Maybe pool isn't the best word for that business, but pool software or "load balancing software for miners" both seem like they are relatively close to the kinds of functions such a business would need.

-MarkM-


Title: Re: Interrest for a new pool backend?
Post by: Zagitta on July 04, 2011, 12:36:49 AM
Ah, thanks for that explanation on stale shares. That makes sense.

I'm still a little confused about what you are ultimately trying to do. So two questions:

1. What is ultimate goal? Are you trying to run a public pool?

2. What do you consider the "backend" of a pool? Is it the part that manages assigning work, and keeping track of what work is done, etc? I am confused by this because of the stuff you mentioned with redirecting packets. I thought backend would be on the "server" side of the pool. But if that's the case, why would you have to mess with sending packets to BTCGuild? That would imply that you are doing some work on the mining client side, and not the server side. I'm sure i'm missing something, can you please explain what?

Don't worry about your quality of code. Everyone has to learn somewhere. I'm sure we could learn from each other.

1: No if i were trying to run a pool I'd just run the already existing pool software, pushpool. The ultimate goal is to have a competing pool backend that's easy to setup and expand with new protocols and functions.

2: I think you need to know some more about how the bitcoin system works so i'll try and explain it:

A simple flow chart of it would look like this:

-> miner requests work
<- bitcoin client responds with workdata wich consists of midstate, data, hash1 and target
    
|   miner now keeps working on the data by hashing it and then increasing some specific bytes in the data field untill it overflows, then requests new work. Inside this "loop" it checks if the resulting hash is below the specified target and if that's true it returns the work to the server where it's verified and if correct a block get solved.

-> so miner either returns result or request new work
<- server responds accordingly

The process for a miner connected to a pool is very similar, the only difference is that the a few things in the block is changed. For one the payout address in the solved block is set to the pool's, secondly the target (AKA difficulty) is set to 0 because it allows everyone to participate, even CPU miners wich can't realisticly solve a block at the current difficulty alone.
Another significant change is that when results are submitted back to the pool, it stores the work as a "share" if it's valid and if it's above the target specified by the network it's submitted to the bitcoin client running behind the pool software so the pool gets rewareded the credits for the block.

Now as for the part that's confusing you: basically instead of my pool software getting data from a bitcoin client in server mode i get it from a pool because to the outside world they work the exact same way except that as mentioned before the difficulty is set to 0 wich lets me debug the recieving, storing and sending of work much quicker than if i had to solve a real block instead of just a share :)
I guess what's confusing you the most is my way of coding, because instead of writing lots of code knowing that it will work and then later write unittests to prove it, i "shotgun debug" my way through the writing  ;D

Also i'm glad you look at the code quality that way haha :) anyway PM me if you want to exchange instant messenger informations of some kind like skype or MSN as it's a bit easier to get you up and running that way :)

---------------

Markm, I'm not entirely sure what your point is? is it a feature request of some kind?


Title: Re: Interrest for a new pool backend?
Post by: Capitan on July 04, 2011, 04:29:17 AM
I've downloaded the code and am looking around it. So far it seems pretty clean. This is not the type of development i'm use to doing (i do more enterprise type stuff, so a lot of database, console apps, websites, windows services). Plus, my company is stuck using tech and practices that are about 8 years old, so one of the things I'm looking to get out of this is using more modern stuff like IoC, Json, which you are already using.

Sending you a PM now...


Title: Re: Interrest for a new pool backend?
Post by: Zagitta on July 04, 2011, 01:47:43 PM
How about a decent FRONTEND???

If everything goes according to the plan it should be compatible with all existing pushpool frontends :)


Title: Re: Interrest for a new pool backend?
Post by: Zagitta on July 05, 2011, 03:03:34 AM
http://img808.imageshack.us/img808/9485/progressv.jpg (http://imageshack.us/photo/my-images/808/progressv.jpg/)

The software running out of the box on an Amazon EC2 micro instance with mono... It's beautiful isn't it?  :D