iProduct                2 NanoFury NF1 v0.7

{"id": 0, "method": "mining.get_transactions", "params":["545198de00000000"]}

 Summary

The Stratum protocol reduces client-server communications to levels that are usable with very low bandwidth, and reduces the strain on servers drastically.
The key concept behind Stratum based mining is "push" based work, where the server pushes work to the miner, and the miner is able to utilize that work until the next push, regardless of the hash rate.
This is done by allowing the miner to increase a counter in the coinbase transaction, and build a new merkleroot for the block header, which effectively means the miner generates new work continuously without contacting the server.
Additionally, this new protocol utilizes a single asynchronous socket connection for all communication, rather than opening/closing HTTP connections with each communication.



Building Local Work

The mining pool provides a miner with two pieces of information upon connecting. The first is ExtraNonce1. The second piece of information is ExtraNonce2_size. This is how many bytes should be used for the ExtraNonce2 counter. ExtraNonce2 is a hexadecimal counter, and should be padded to fill up the number of bytes identified as the ExtraNonce2_size.

When the pool pushes work, it provides the coinbase transaction in two pieces. When the miner is building the block header to hash, it builds a merkleroot by creating a coinbase transaction. This is done by combining: Coinbase1 + ExtraNonce1 + ExtraNonce2 (padded) + Coinbase2.

This transaction is then passed through a double SHA256 to get the coinbase tx hash. You then combine that with first merkle branch provided by the work (if any), and double SHA256 the combined string. You then take that result, and do the same with the next merkle branch, repeating this process until all merkle branch hashes have been combined with previous results.

This process creates a unique merkle root for the new block header, which can then be pushed through the miner using the standard 32-bit nonce counter.



Initial Connection Communication

Client: {"id": 1, "method": "mining.subscribe", "params": []}\n
Server:  {"id": 1, "result": [["mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f"], "08000002", 4], "error": null}\n

Connection uses 'mining.subscribe' to subscribe to work from the server.
Server result response contains:
result[0] = "mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f" - Unique string used for the subscription
result[1] = ExtraNonce1, used for building the coinbase.
result[2] = Extranonce2_size, the number of bytes that the miner users for its ExtraNonce2 counter



Server Work Communication

Server: {"params": ["bf", "4d16b6f85af6e2198f44ae2a6de67f78487ae5611b77c6c0440b921e00000000",
"01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff20020862062f503253482f04b8864e5008",
"072f736c7573682f000000000100f2052a010000001976a914d23fcdf86f7e756a64a7a9688ef9903327048ed988ac00000000",
["c5bd77249e27c2d3a3602dd35c3364a7983900b64a34644d03b930bfdb19c0e5","049b4e78e2d0b24f7c6a2856aa7b41811ed961ee52ae75527df9e80043fd2f12"],
"00000002", "1c2ac4af", "504e86b9", false], "id": null, "method": "mining.notify"}\n

This is sent at regular intervals by the server. It should be sent almost immediately after an acknowledge subscription.
Work pushes contain:
params[0] = Job ID. This is included when miners submit a results so work can be matched with proper transactions.
params[1] = Hash of previous block. Used to build the header.
params[2] = Coinbase (part 1). The miner inserts ExtraNonce1 and ExtraNonce2 after this section of the coinbase.
params[3] = Coinbase (part 2). The miner appends this after the first part of the coinbase and the two ExtraNonce values.
params[4][] = List of merkle branches. The coinbase transaction is hashed against the merkle branches to build the final merkle root.
params[5] = Bitcoin block version, used in the block header.
params[6] = nBit, the encoded network difficulty. Used in the block header.
params[7] = nTime, the current time. nTime rolling should be supported, but should not increase faster than actual time.
params[8] = Clean Jobs. If true, miners should abort their current work and immediately use the new job. If false, they can still use the current job, but should move to the new one after exhausting the current nonce range.



Worker Authorization

Client: {"params": ["eleuthria_miner1", "password"], "id": 2, "method": "mining.authorize"}\n
Server: {"error": null, "id": 2, "result": true}\n

Client authorizes a worker using the method "mining.authorize". Client request contains:
params[0] = Worker Name
params[1] = Worker Password (can be "" or omitted if server does not require passwords)

Server response is "result": true for successful authorization, "result": false for unsuccessful authorization.



Work Submissions

Client: {"params": ["eleuthria_miner1", "bf", "00000001", "504e86ed", "b2957c02"], "id": 4, "method": "mining.submit"}\n
Server: {"error": null, "id": 4, "result": true}\n

Miners submit shares using the method "mining.submit".
Client submissions contain:
params[0] = Worker Name
params[1] = Job ID
params[2] = ExtraNonce 2
params[3] = nTime
params[4] = nonce
Server response is result: true for accepted, false for rejected.



Difficulty Adjustment

Server: { "id": null, "method": "mining.set_difficulty", "params": [2]}\n

The server can adjust the difficulty required for miner shares with the "mining.set_difficulty" method.
The miner should begin enforcing the new difficulty on the next job received. Some pools may force a new job out when set_difficulty is sent, using clean_jobs to force the miner to begin using the new difficulty immediately.



Cheat Sheet

Methods:
mining.subscribe     : Used to subscribe to work from a server, required before all other communication.
mining.authorize     : Used to authorize a worker, required before any shares can be submitted.
mining.notify        : Used to push new work to the miner.  Previous work should be aborted if Clean Jobs = true!
mining.submit        : Used to submit shares
mining.set_difficulty: Used to signal the miner to stop submitting shares under the new difficulty.

For mining software developers

Stratum protocol is based on JSON-RPC 2.0. In this chapter I expect that you're familiar with this protocol and you understand terms like "request", "response" and "notification". Please read JSON-RPC specification for more details.

For high level image of the Stratum protocol concept, please read Stratum protocol specification on Google docs. This document needs some care, but give you the basic examples how to connect to Stratum server.
Exception handling

Stratum defines simple exception handling. Example of rejected share looks like:

{"id": 10, "result": null, "error": (21, "Job not found", null)}

Where error field is defined as (error_code, human_readable_message, traceback). Traceback may contain additional information for debugging errors.

Proposed error codes for mining service are:

    20 - Other/Unknown
    21 - Job not found (=stale)
    22 - Duplicate share
    23 - Low difficulty share
    24 - Unauthorized worker
    25 - Not subscribed

Real-world example

This chapter contains real log of miner-pool communication which solved testnet3 block 000000002076870fe65a2b6eeed84fa892c0db924f1482243a6247d931dcab32
Miner connects the server

On the beginning of the session, client subscribes current connection for receiving mining jobs:

{"id": 1, "method": "mining.subscribe", "params": []}\n
{"id": 1, "result": [[["mining.set_difficulty", "b4b6693b72a50c7116db18d6497cac52"], ["mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f"]], "08000002", 4], "error": null}\n

Reminder: The newline character \n is a part of the message and must be added to the end of *every* JSON message. Server may wait to this magic character to start processing the message. This is the most common mistake which people implementing line-based clients do!

The result contains three items:

    Subscriptions details - 2-tuple with name of subscribed notification and subscription ID. Teoretically it may be used for unsubscribing, but obviously miners won't use it.
    Extranonce1 - Hex-encoded, per-connection unique string which will be used for coinbase serialization later. Keep it safe!
    Extranonce2_size - Represents expected length of extranonce2 which will be generated by the miner.

Authorize workers

Now let authorize some workers. You can authorize as many workers as you wish and at any time during the session. In this way, you can handle big basement of independent mining rigs just by one Stratum connection.

{"params": ["slush.miner1", "password"], "id": 2, "method": "mining.authorize"}\n
{"error": null, "id": 2, "result": true}\n

Server start sending notifications with mining jobs

Server sends one job *almost* instantly after the subscription.

Small engineering note: There's a good reason why first job is not included directly in subscription response - miner will need to handle one response type in two different way; firstly as a subscription response and then as a standalone notification. Hook job processing just to JSON-RPC notification sounds a bit better to me.

{"params": ["bf", "4d16b6f85af6e2198f44ae2a6de67f78487ae5611b77c6c0440b921e00000000",
"01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff20020862062f503253482f04b8864e5008",
"072f736c7573682f000000000100f2052a010000001976a914d23fcdf86f7e756a64a7a9688ef9903327048ed988ac00000000", [],
"00000002", "1c2ac4af", "504e86b9", false], "id": null, "method": "mining.notify"}

Now we finally have some interesting stuff here! I'll descibe every field of the notification in the particular order:

    job_id - ID of the job. Use this ID while submitting share generated from this job.
    prevhash - Hash of previous block.
    coinb1 - Initial part of coinbase transaction.
    coinb2 - Final part of coinbase transaction.
    merkle_branch - List of hashes, will be used for calculation of merkle root. This is not a list of all transactions, it only contains prepared hashes of steps of merkle tree algorithm. Please read some materials for understanding how merkle trees calculation works. Unfortunately this example don't have any step hashes included, my bad!
    version - Bitcoin block version.
    nbits - Encoded current network difficulty
    ntime - Current ntime/
    clean_jobs - When true, server indicates that submitting shares from previous jobs don't have a sense and such shares will be rejected. When this flag is set, miner should also drop all previous jobs, so job_ids can be eventually rotated.

How to build coinbase transaction

Now miner received all data required to serialize coinbase transaction: Coinb1, Extranonce1, Extranonce2_size and Coinb2. Firstly we need to generate Extranonce2 (must be unique for given job_id!). Extranonce2_size tell us expected length of binary structure. Just be absolutely sure that your extranonce2 generator always produces extranonce2 with correct length! For example my pool implementation sets extranonce2_size=4, which mean this is valid Extranonce2 (in hex): 00000000.

To produce coinbase, we just concatenate Coinb1 + Extranonce1 + Extranonce2 + Coinb2 together. That's all!

For following calculations we have to produce double-sha256 hash of given coinbase. In following snippets I'm using Python, but I'm sure you'll understand the concept even if you're a rubyist! It is as simple as:

import hashlib
import binascii
coinbase_hash_bin = hashlib.sha256(hashlib.sha256(binascii.unhexlify(coinbase)).digest()).digest()

How to build merkle root

Following Python snippet will generate merkle root for you. Use merkle_branch from broadcast and coinbase_hash_bin from previous snippet as an input:

import binascii

def build_merkle_root(self, merkle_branch, coinbase_hash_bin):
    merkle_root = coinbase_hash_bin
    for h in self.merkle_branch:
        merkle_root = doublesha(merkle_root + binascii.unhexlify(h))
    return binascii.hexlify(merkle_root)

How to build block header?

Now we're almost done! We have to put all together to produce block header for hashing:

version + prevhash + merkle_root + ntime + nbits + '00000000' +
'000000800000000000000000000000000000000000000000000000000000000000000000000000000000000080020000'

First zeroes are blank nonce, the rest is padding to uint512 and it is always the same.

Note that merkle_root must be in reversed byte order. If you're a miner developer, you already have util methods there for doing it. For some example in Python see Stratum mining proxy source codes.
Server can occasionally ask miner to change share difficulty

Default share difficulty is 1 (big-endian target for difficulty 1 is 0x00000000ffff0000000000000000000000000000000000000000000000000000), but server can ask you anytime during the session to change it:
{ "id": null, "method": "mining.set_difficulty", "params": [2]}

This means that difficulty 2 will be applied to every next job received from the server.
How to submit share?

When miner find the job which meets requested difficulty, it can submit share to the server:

{"params": ["slush.miner1", "bf", "00000001", "504e86ed", "b2957c02"], "id": 4, "method": "mining.submit"}
{"error": null, "id": 4, "result": true}

Values in particular order: worker_name (previously authorized!), job_id, extranonce2, ntime, nonce.