[CLOSED] BTC Guild - Pays TxFees+NMC, Stratum, VarDiff, Private Servers

[runlinux]

Every time a miner performs a hash, it is creating a share.  The difficulty of that share is based on the quality of the hash (leading 0-bits for the most part).  If the first 32 bits of the hash are 0s, then you have a difficulty 1 hash.  If 33 are 0, it's difficulty 2.  34 = 4, 35 = 8, etc., etc.

I learned something new today! Thanks! I never really "knew" how that worked.

[1714261737]

1714261737

[1714261737]

1714261737

[1714261737]

1714261737

[1714261737]

1714261737

[1714261737]

1714261737

[eleuthria]

Glad to have helped.  I enjoy answering some of the more "how stuff works" questions on the general level.  Especially compared to my normal daily pool support questions .

[matt4054]

Glad to have helped.  I enjoy answering some of the more "how stuff works" questions on the general level.  Especially compared to my normal daily pool support questions .

This post was a very interesting read for me too.

I think, but I'm not sure, that I am somewhat familiar with the basic concepts of PoW in solo mining:

• Your mempool contains unconfirmed transactions, that makes the content for the next block being "generated"
• In order to have this block accepted as the next one in the chain, you need to find an SHA256d hash for this content + a nonce
• The hash is used in the header, and it needs to be below a defined target, i.e. difficulty, for the pushed block to be accepted by other nodes

Now I'm more confused about pooled mining. I admit that I never took the time to read the Stratum protocol RFC, spec or whatever.

What I'm not understanding here is how the pool manages to hide block content from pool miner (that's something I'm used to read when starting the miner, i.e. something like "pool is hiding block contents from us"). I understand that the hiding is necessary to prevent the pool miner from pushing the block solo, bypassing the pool and getting the block reward for himself instead of the pool. But then what exactly is the (partial?) content that is used to hash, and more importantly, how can it be useful to the pool, since the hash sent by the miner will not be block_content + nonce but other_content + nonce (+ extra nonce etc?).

In short: how can sha256d(block_content + nonce) = jackpot_hash be derived from sha256d(f(block_content) + nonce and/or whatever) = jackpot_hash? Most likely, it isn't and I'm asking the wrong question, and you're gonna explain to me why

I'm not sure that I've made myself clear, but I promise that I've tried to... I can rephrase the question if necessary.

Thank you in advance for your enlightenment!

[eleuthria]

This post was a very interesting read for me too.

I think, but I'm not sure, that I am somewhat familiar with the basic concepts of PoW in solo mining:

• Your mempool contains unconfirmed transactions, that makes the content for the next block being "generated"
• In order to have this block accepted as the next one in the chain, you need to find an SHA256d hash for this content + a nonce
• The hash is used in the header, and it needs to be below a defined target, i.e. difficulty, for the pushed block to be accepted by other nodes

Now I'm more confused about pooled mining. I admit that I never took the time to read the Stratum protocol RFC, spec or whatever.

What I'm not understanding here is how the pool manages to hide block content from pool miner (that's something I'm used to read when starting the miner, i.e. something like "pool is hiding block contents from us"). I understand that the hiding is necessary to prevent the pool miner from pushing the block solo, bypassing the pool and getting the block reward for himself instead of the pool. But then what exactly is the (partial?) content that is used to hash, and more importantly, how can it be useful to the pool, since the hash sent by the miner will not be block_content + nonce but other_content + nonce (+ extra nonce etc?).

What you're missing is that the hash used for proof of work is *not* a hash of the block contents, only a hash of the block header.  The block header contains:

1) Block Version (version 2 currently)
2) Hash of the previous block
3) MerkleRoot hash of the transactions in the block
4) Timestamp
5) Bits (difficulty)
6) Nonce

#3 is the key part.  A MerkleRoot is essentially a hash of hashes.  So lets say you have 4 transactions, A, B, C, D.  You take the hash of them, and call them A1, B1, C1, and D1.  You would hash A1 and B1 together to get A2.  You would hash C1 and D1 together to get B2.  You then hash A2 and B2 together.  You're now out of transactions to hash together, which means you have the MerkleRoot.  It creates a tree/pyramid shape as you go.  A1 would be your Coinbase transaction, which is why each pool/miner has a different MerkleRoot/transaction list.

The key for Stratum is that to rebuild a MerkleRoot, you only need one side of the branches in this tree of hashes.  You can change A1 (coinbase), rehash it with B1 for a new A2, and hash it with the same B2, without redoing the C1+D1 hash.  As such, the miner can constantly alter a field in the Coinbase transaction (ExtraNonce), to get a new A1, rehash it with B2, B3, B4, B5, B6, etc., until it is done and has a fresh MerkleRoot, without knowing the actual transactions.

[BRADLEYPLOOF]

The above post makes me wonder if Eleuthria had a hand in the development of bitcoin...

[eleuthria]

The above post makes me wonder if Eleuthria had a hand in the development of bitcoin...

None at all.  I believe Luke-Jr is the only one involved in development that also ended up making a pool (slush may have, I'm not sure).

I know the overall concepts of mining very well now that I've been running BTC Guild for just under 3 years.  General concepts, certain terms, eventually I've understood them well because I try to explain these things to my family over time, and need to learn how to break them up into "normal English" as best as I can.  However, the deeper design of Bitcoin is still very mysterious to me when it comes to how the code itself functions.  

It's very different from the style of programming I'm used to, and deals in a lot of raw byte data/hex data, whereas I'm used to dealing in ASCII and integers.  My experience in programming has always been hobbyist level.  My C/C++ experience is entirely based on MUDs.  I'm still not quite sure how I managed to apply my limited programming experience and ended up making a very stable and high traffic volume custom pool backend.

[rgiskard]

Just a general question. Does ping time have any bearing on the effectiveness of mining in a pool? I'm on the opposite side of the planet, so have a ping of circa 260 ms. Are there any noticeable disadvantages to this? Not that i can do a whole lot about it, but curious none the less.   

[mithrandi]

Just a general question. Does ping time have any bearing on the effectiveness of mining in a pool? I'm on the opposite side of the planet, so have a ping of circa 260 ms. Are there any noticeable disadvantages to this? Not that i can do a whole lot about it, but curious none the less.   

I would expect latency to have an impact primarily in two areas:
1) The time taken to initially start mining
2) Quantity of stale shares submitted

The first point should hopefully be obvious (can't start mining until you have some work from the pool); the second actually stems from the same point, if the pool needs to send you a new work template, the window for stale shares is exactly how long it takes for you to receive the new work template and switch to it.

[matt4054]

What you're missing is that the hash used for proof of work is *not* a hash of the block contents, only a hash of the block header.

(...)

As such, the miner can constantly alter a field in the Coinbase transaction (ExtraNonce), to get a new A1, rehash it with B2, B3, B4, B5, B6, etc., until it is done and has a fresh MerkleRoot, without knowing the actual transactions.

Thank you very much for your crystal clear explanation. That makes so much more sense now!

Unrelated, less technical question: several people have requested features that are just not feasible at pool level for many reasons that you have already explained (hashrate drop notifications, for example). However these features seem not that complicated to implement for third parties or miners themselves, by running a watcher monitoring workers through the API. Are you aware of already existing OSS code or services (free/tips-based or commercial)? Would it stress the API too much if many miners started doing so? If not, maybe I'll feel like writing some basic code tonight...

I know about cgwatcher and the advantages of monitoring the mining software directly, but setting up NAT traversal on a wide scale can be tricky, especially (sadly unavoidable under some circumstances) double-NATs, native IPv6 support is still scarce, etc. Basically, the pool API is often a more convenient way to collect overall worker stats from different locations / LANs.

Just a general question. Does ping time have any bearing on the effectiveness of mining in a pool? I'm on the opposite side of the planet, so have a ping of circa 260 ms. Are there any noticeable disadvantages to this? Not that i can do a whole lot about it, but curious none the less.    

Answer above is good IMO. I just wanted to add that it should only be a minor effect (i.e. for a 200 ms RTT I would expect much less than 1% drop of efficiency due to the stales), on a chain like Bitcoin with 10 minutes avg block time. However, optimizing is always a good thing and I would just ping the US and the EU servers to check for the best round trip time

[rgiskard]

Thanks for this. I can ping the US server ok, but the EU server seems to be dropping the ICMP, so unsure of my ping time there.

Cheers.

[hurricandave]

At 00:41 CST I look up and see Network Diff changed to 2.62G(18.77 Ph/s) PPS rate drops too. DANG!

[eleuthria]

Unrelated, less technical question: several people have requested features that are just not feasible at pool level for many reasons that you have already explained (hashrate drop notifications, for example). However these features seem not that complicated to implement for third parties or miners themselves, by running a watcher monitoring workers through the API. Are you aware of already existing OSS code or services (free/tips-based or commercial)? Would it stress the API too much if many miners started doing so? If not, maybe I'll feel like writing some basic code tonight...

There is a service somebody was starting where you could enter your pool API key and they would send you notices if your hash rate dropped below a defined amount.  I can not recall it at this time.

This type of feature is *possible* built in to BTC Guild, the problem is efficiency.  This is one of the many features I'm planning to use ScryptGuild as a test bed for, things like worker speed estimates being built into the pool backend itself, rather than a frontend/website side calculation.

BTC Guild is about to hit its 3 year anniversary, and while it uses a completely custom backend these days, it was originally built using the open source solutions at the time (pushpool, then PoolServerJ).  As such, many of the database structures have simply been carried a long for consistency, rather than rebuilt for efficiency.  ScryptGuild will be giving me a chance to break free of old DB schema, and hopefully some of those changes get rolled back into the BTC Guild system after they have proven themselves on ScryptGuild.

[MoreBloodWine]

New diff as of 2/5/14 2:15am EST: 2,621,404,453 (+ 427,556,583)

On side note,
   Eleuthria... do you ever see sub pools in BTC Guilds future. Not too sure on how they work but as I think I understand it, they sorta work like a referral system for people that bring hash in. They get a bump of sorts in pay or w/e.

Also, someone able to explain to me why I saw this in CGMiner when there was still one block left before the diff jump. I know this because I was watching my node stats at he same time and the diff didn't change til 2:05, 35mins after what I saw in CGMiner.

[2014-02-05 01:40:44] Accepted 45aa1ebf Diff 4/2 AMU 0 pool 0
[2014-02-05 01:40:52] Network diff set to 2.62G
[2014-02-05 01:40:52] Stratum from pool 0 detected new block
[2014-02-05 01:40:55] Accepted 027c0b0a Diff 103/2 AMU 2 pool 0

----------------------------------------------------------------------------------------------------------------------------------------------

Want a sig like mine for your BTC Guild stats, then check out this post !

[eleuthria]

New diff as of 2/5/14 2:15am EST: 2,621,404,453 (+ 427,556,583)

On side note,
   Eleuthria... do you ever see sub pools in BTC Guilds future. Not too sure on how they work but as I think I understand it, they sorta work like a referral system for people that bring hash in. They get a bump of sorts in pay or w/e.

Also, someone able to explain to me why I saw this in CGMiner when there was still one block left before the diff jump. I know this because I was watching my node stats at he same time and the diff didn't change til 2:05, 35mins after what I saw in CGMiner.

[2014-02-05 01:40:44] Accepted 45aa1ebf Diff 4/2 AMU 0 pool 0
[2014-02-05 01:40:52] Network diff set to 2.62G
[2014-02-05 01:40:52] Stratum from pool 0 detected new block
[2014-02-05 01:40:55] Accepted 027c0b0a Diff 103/2 AMU 2 pool 0

The network difficulty actually jumps up before the first block at that difficulty is found.  So while the first block to be mined at the new difficulty was at 2:15, the difficulty was known when the block before it was found at 1:40.

[ak49er]

ScryptGuild? ? ?  Tell me more!  I haven't been on the forums in a while and missed any reference to this.  Is there a thread I can search for?

I also have a question about the Dashboard page.  Is PPS Rate really relevant any more?  I'd like to see that replaced with something like current difficulty / blocks to next difficulty. (OK, not something like, but actually those two bits of information exactly!)  I know that you're keen not to have a cluttered page, and I wholeheartedly agree with that concept, but the only reason I check bitcoincharts is to get that information, and they're not always up to date.

Just my contribution to the suggestion box.

[eleuthria]

ScryptGuild? ? ?  Tell me more!  I haven't been on the forums in a while and missed any reference to this.  Is there a thread I can search for?

I also have a question about the Dashboard page.  Is PPS Rate really relevant any more?  I'd like to see that replaced with something like current difficulty / blocks to next difficulty. (OK, not something like, but actually those two bits of information exactly!)  I know that you're keen not to have a cluttered page, and I wholeheartedly agree with that concept, but the only reason I check bitcoincharts is to get that information, and they're not always up to date.

Just my contribution to the suggestion box.

PPS rate is relevant for PPS users, though they are a minority audience these days.  It's useful for PPLNS users when eyeballing their paid rate on recent shifts to get a general idea on luck.  I may change the tooltip on the PPS rate to show the stats you're asking for though.

ScryptGuild is expected to launch *alpha* testing on Friday, at 5 PM PST.  There's a countdown timer at www.scryptguild.com .  When it launches, the front page will include a rough timeline of what features will be added during the different phases of testing.  During alpha, it will be 0% fee.  During beta, the new features will be 0% fee, and the "known stable" features will be 2% PPLNS.

[ak49er]

PPS rate is relevant for PPS users, though they are a minority audience these days.  It's useful for PPLNS users when eyeballing their paid rate on recent shifts to get a general idea on luck.  I may change the tooltip on the PPS rate to show the stats you're asking for though.

ScryptGuild is expected to launch *alpha* testing on Friday, at 5 PM PST.  There's a countdown timer at www.scryptguild.com .  When it launches, the front page will include a rough timeline of what features will be added during the different phases of testing.  During alpha, it will be 0% fee.  During beta, the new features will be 0% fee, and the "known stable" features will be 2% PPLNS.

My apologies, I didn't know anyone was still using PPS.  For some reason I thought you had discontinued that as an option.  I did know that it was used as a basis for luck, but since I can't do anything about that I largely ignore it.

Off to check out scryptguild.com!  Thanks!

[alanwgeorge]

Hello,

Quick question.

It looks like one of my miners solved a block recently.  I was wondering if there was a way to determine which one it was?

Thanks!
-alan

[bobcaticus]

Unrelated, less technical question: several people have requested features that are just not feasible at pool level for many reasons that you have already explained (hashrate drop notifications, for example). However these features seem not that complicated to implement for third parties or miners themselves, by running a watcher monitoring workers through the API. Are you aware of already existing OSS code or services (free/tips-based or commercial)? Would it stress the API too much if many miners started doing so? If not, maybe I'll feel like writing some basic code tonight...

There is a service somebody was starting where you could enter your pool API key and they would send you notices if your hash rate dropped below a defined amount.  I can not recall it at this time.

Bitcoinium has that option..

https://play.google.com/store/apps/details?id=com.veken0m.cavirtex&hl=en

[eleuthria]

Hello,

Quick question.

It looks like one of my miners solved a block recently.  I was wondering if there was a way to determine which one it was?

Thanks!
-alan

At this time, the specific block a user solved is not recorded.  This is something that is on the list of modifications to the BTC Guild stratum backend, and will likely be added after I have a chance to evaluate the changes that are being made for ScryptGuild.  As stated before, ScryptGuild is giving me a good way to test new modifications to the backend software for additional statistics tracking.