Bitcoin Forum

Bitcoin => Pools => Topic started by: bernardk on December 01, 2015, 12:31:06 PM



Title: I'm working on a pool software and wanted to check if I'm on the right track
Post by: bernardk on December 01, 2015, 12:31:06 PM
Hi,

being fairly new to mining pools I wanted to clear some doubts about the approach I'm taking.
I hope that experienced pool developers will be able to point me in the right directions.

Here is my set of questions:

1. I would prefer to use the getblocktemplate protocol instead of stratum. Are there any non-obvious drawbacks in doing so? Do all popular miners support it correctly?

2. For the beginning I would just use a regular bitcoin node and a proxy before it. The proxy would ask the bitcoin node for work and update all miners. Does this sound like a good approach?

3. I guess the bitcoin node also provides the current network difficulty so I would need to change the getblocktemplate in the proxy, lowering the difficulty so miners can actually produce shares. In this case I would keep track of submitted shares with the proxy. Does this sound good?

4. As in BIP23 (https://en.bitcoin.it/wiki/BIP_0023#Basic_Pool_Extensions) explained, the miner can request a new target. Do miners use this? Are they good at choosing the perfect target? Would it be better to adjust the target on the server depending on the shares received from the miner and ignore this?

5. Playing around with bitcoin-qt I noticed when using getblocktemplate and submitting coinbasetxn as one of the capabilities the server answers me with a coinbasevalue and not coinbasetxn. Contrary to the documentation in BIP22 (https://en.bitcoin.it/wiki/BIP_0022#Block_Template_Request). Is this a bug or am I missing something? And it's not really clear to me if I just forward the information without coinbasetxn to miners will they be able to do their work. This whole process of generating coinbasetxn in getblocktemplate is a bit unclear and confusing. Would be great if somebody took the time to explain it.

And of course I would appreciate all other suggestions you have and stuff I maybe missed/overlooked.

Cheers,
Bernard


Title: Re: I'm working on a pool software and wanted to check if I'm on the right track
Post by: bernardk on December 02, 2015, 12:07:27 AM
Regarding question 5, I guess the bitcoin client will always answer with coinbasevalue because he expects you to solo mine and you need to provide your own address. Now I built a little proxy to intercept the answer and populate the value coinbasetxn data field and remove coinbasevalue. But if I run cgminer against it he complains with "No BTC address specified, unable to mine solo on http://127.0.0.1". Shouldn't he already have an address that is part of coinbasetxn. Why would I need to specify another one?


Title: Re: I'm working on a pool software and wanted to check if I'm on the right track
Post by: philipma1957 on December 02, 2015, 12:49:34 AM
As A heads up  we just had a guy open a pool and run off with the first block it hit.

So if you are going to open this pool to the public I will make this offer send me 25btc I will hold in escrow for no charge and I will add .25 btc to the first block payout.

 Most of us are tired of the cons that have happened .  If you don't want to use me as an escrow ask another trusted person.  But frankly no one will want to be in your pool due to the recent theft.


Title: Re: I'm working on a pool software and wanted to check if I'm on the right track
Post by: bernardk on December 02, 2015, 01:14:34 AM
I don't think I will be running a pool. I'm far from having anything working yet, as you can see from my questions.
I just have some technical questions and this discussion is already diverging.

Now I realise that maybe I'm posting in the wrong subforum.
Would Development & Technical Discussion be more appropriate?


Title: Re: I'm working on a pool software and wanted to check if I'm on the right track
Post by: kipper01 on December 02, 2015, 01:35:09 AM
These scammers are getting more creative everyday.   ::)


Title: Re: I'm working on a pool software and wanted to check if I'm on the right track
Post by: jonnybravo0311 on December 02, 2015, 01:54:21 AM
As A heads up  we just had a guy open a pool and run off with the first block it hit.

So if you are going to open this pool to the public I will make this offer send me 25btc I will hold in escrow for no charge and I will add .25 btc to the first block payout.

 Most of us are tired of the cons that have happened .  If you don't want to use me as an escrow ask another trusted person.  But frankly no one will want to be in your pool due to the recent theft.
I actually got so angry that I opened up my own pool.  It's nothing fancy and I'm not offering crazy promotions or promises.  Straight up BTC mining.  Anybody who wants to can mine on it.  So far, it's only me.  Every now and then I'll rent some hash and throw it at it to see if I get lucky - after all, I found that block on the scammer's pool.

bravo-mining.com


Title: Re: I'm working on a pool software and wanted to check if I'm on the right track
Post by: bctmke on December 02, 2015, 02:15:19 AM
As A heads up  we just had a guy open a pool and run off with the first block it hit.

So if you are going to open this pool to the public I will make this offer send me 25btc I will hold in escrow for no charge and I will add .25 btc to the first block payout.

 Most of us are tired of the cons that have happened .  If you don't want to use me as an escrow ask another trusted person.  But frankly no one will want to be in your pool due to the recent theft.
I actually got so angry that I opened up my own pool.  It's nothing fancy and I'm not offering crazy promotions or promises.  Straight up BTC mining.  Anybody who wants to can mine on it.  So far, it's only me.  Every now and then I'll rent some hash and throw it at it to see if I get lucky - after all, I found that block on the scammer's pool.

bravo-mining.com

Looks like fun.  At least you have a history here along with a bit of trust amongst others


Title: Re: I'm working on a pool software and wanted to check if I'm on the right track
Post by: jonnybravo0311 on December 02, 2015, 02:19:45 AM
Yeah, I've been around a while :P.  Been running my own p2pool node for almost 2 years.  I just got fed up with all the scams and jackasses ripping people off, and wanted to do something about it.  Since people seem to be scared of p2pool for some reason, I thought I'd offer up the choice of a "typical" mining pool.

Edit: @OP - sorry for hijacking your thread.


Title: Re: I'm working on a pool software and wanted to check if I'm on the right track
Post by: bernardk on December 02, 2015, 03:07:33 PM
After spending 2 days debugging getblocktemplate requests my conclusion is:

Most mining software implements a subset of GBT to make it work for them and doesn't follow the standard strictly.
Miners also introduce some other rules in agreement with pools that are not documented.
If you are working on a pool you will have to manually test it against miners to be sure it works (good luck).

I couldn't get cgminer to work with my pool proxy. The problem is that it differentiates between solo and pool mining (what I don't think is part of the getblocktemplate specification). After reading the source code I found this https://github.com/ckolivas/cgminer/blob/master/cgminer.c#L6512. So, cgminer will only treat you as a pool if you submit "coinbase/append" and "submit/coinbase" as mutables. This is not really explained anywhere and it took me some time to figure it out. I can understand the idea behind this choice, but it still looks a bit arbitrary to me. It will just reject you with a message that you are not a pool without clarification. I can see users breaking their head over this.

To answer some of the questions myself after 2 days of researching:

1. Stratum is much better defined and smaller. It is also more popular between pools. This makes it easier to implement and find information about. It's straight forward, there is basically only one way to do it. You loose some flexibility, but at least you don't need to deal with the mess of BIP22 and BIP23. I will try to write a tutorial how to implement GBT correctly if I find time.

2. Because of cgminer behavior the proxy needs to do much more. Now it needs to understand "submit/coinbase" and keep track of the transactions submited to the miner. It makes it more complicated. But using a bitcoin node to get information from the network and not implementing your own is still a good idea.

I didn't get to experiment with setting the difficulty. I hope I will soon have a proof of concept pool running on testnet to try more advanced stuff.

P.S. What is wrong with you guys? Accusing me of scaming. Casually requesting me to send you 25btc. Promoting your own pools. From my perspective this doesn't look like a welcoming community.


Title: Re: I'm working on a pool software and wanted to check if I'm on the right track
Post by: itsnogood on December 02, 2015, 03:58:16 PM
Some of us have had very bad experience recently with a new pool owner doing a runner after finding its first block.
Trust is in short supply at the moment.

The scam card gets pulled first then its up to you to prove otherwise. That's just the way it is.

If you are on the level and honest then I wish you the best of luck with your project.


Title: Re: I'm working on a pool software and wanted to check if I'm on the right track
Post by: pj40 on December 02, 2015, 04:13:48 PM

P.S. What is wrong with you guys? Accusing me of scaming. Casually requesting me to send you 25btc. Promoting your own pools. From my perspective this doesn't look like a welcoming community.

Quite simple.  New user account.  Take a look at the Nexious pool thread https://bitcointalk.org/index.php?topic=1135806.0 to see how responsive that new pool was.  Eager to get people mining at his pool, bending over backwards with support and incentives to people to mine there....then once the block was found, he was never seen or heard from again.

And simply put, the request for you to send over 25 btc is to escrow it in case your pool DOES find a block and you won't take off but pay out the people who spent their money and effort to get that block.  All those people were cheated and it makes us all leery of new pools popping up.


Title: Re: I'm working on a pool software and wanted to check if I'm on the right track
Post by: jonnybravo0311 on December 02, 2015, 04:18:01 PM
After spending 2 days debugging getblocktemplate requests my conclusion is:

Most mining software implements a subset of GBT to make it work for them and doesn't follow the standard strictly.
Miners also introduce some other rules in agreement with pools that are not documented.
If you are working on a pool you will have to manually test it against miners to be sure it works (good luck).

I couldn't get cgminer to work with my pool proxy. The problem is that it differentiates between solo and pool mining (what I don't think is part of the getblocktemplate specification). After reading the source code I found this https://github.com/ckolivas/cgminer/blob/master/cgminer.c#L6512. So, cgminer will only treat you as a pool if you submit "coinbase/append" and "submit/coinbase" as mutables. This is not really explained anywhere and it took me some time to figure it out. I can understand the idea behind this choice, but it still looks a bit arbitrary to me. It will just reject you with a message that you are not a pool without clarification. I can see users breaking their head over this.

To answer some of the questions myself after 2 days of researching:

1. Stratum is much better defined and smaller. It is also more popular between pools. This makes it easier to implement and find information about. It's straight forward, there is basically only one way to do it. You loose some flexibility, but at least you don't need to deal with the mess of BIP22 and BIP23. I will try to write a tutorial how to implement GBT correctly if I find time.

2. Because of cgminer behavior the proxy needs to do much more. Now it needs to understand "submit/coinbase" and keep track of the transactions submited to the miner. It makes it more complicated. But using a bitcoin node to get information from the network and not implementing your own is still a good idea.

I didn't get to experiment with setting the difficulty. I hope I will soon have a proof of concept pool running on testnet to try more advanced stuff.

P.S. What is wrong with you guys? Accusing me of scaming. Casually requesting me to send you 25btc. Promoting your own pools. From my perspective this doesn't look like a welcoming community.
I never accused you of scamming - I was responding to the comments made by Phil.  I think it's cool that you're trying to understand the processes behind the scenes and are experimenting with your own software.  Personally, I'd recommend taking a look at some of the existing implementations (ckpool, NOMP, UNOMP, stratum-mining, eloipool, etc).  See how those guys tackled the problems you're facing.  Who knows, maybe you'll be inspired and write the next great thing :).

Also, I apologized previously, but let me do so again in case you missed it.  I didn't mean to hijack your thread chatting about my pool.  A lot of us were just scammed by some guy running a long con.  He opened up a pool this past summer.  Was active on the boards and responsive to queries and support on his pool.  Right up until I found the pool's first block.  Within minutes, the pool went down, the website went dark, email servers were taken down.  The pool operator took the 25BTC and ran.  So, as a gesture of goodwill, I am providing a place for miners to point their hash.  That's all.

Both Phil and -ck have made the same offer - which I personally think is very generous of them.  If you're a newbie account and want to open up a pool, they are willing to escrow the 25BTC for the first block.  That way, miners will know that they'll at least get paid for the first block.  It's not a perfect solution - but at least it's something to try and prevent the scammers from continually taking advantage of folks.


Title: Re: I'm working on a pool software and wanted to check if I'm on the right track
Post by: .anto. on December 02, 2015, 05:51:02 PM
I actually got so angry that I opened up my own pool.  It's nothing fancy and I'm not offering crazy promotions or promises.  Straight up BTC mining.  Anybody who wants to can mine on it.  So far, it's only me.  Every now and then I'll rent some hash and throw it at it to see if I get lucky - after all, I found that block on the scammer's pool.

bravo-mining.com

You don't want to be beaten by a newbie/noob like me, do you? So you have to be ahead of them. :)

I am sorry. Off topic but I really couldn't resist.