Bitcoin Forum

SMF won't format tables properly (surprise!) ... I will reformat it if it's too confusing.

Here is a first draft proposal for a JSON API standards for Bitcoin Pools:

Generic keys:

JSON Key	Description
version	Version of the API in use

Pool keys:

JSON Key	Description
round_duration	Duration of the current round
round_shares	Number of shares submitted in current round
hashrate	Total hashrate of the pool

User keys:

JSON Key	Description
round_shares

Total shares submitted by user for current round

estimated_reward

Estimated reward the user will receive on next payout in BTC

hashrate	Total hashrate of the user
last_activity

Last miner activity in GMT

btc_balance	Current available balance for the user in BTC
btc_unconfirmed	Unconfirmed/unavailable balance for the user in BTC
btc_paid	Total amount paid to the user in BTC
last_payout_amount	Last amount paid to the user in BTC
last_payout_time	Time user was paid in GTM

what do you think about adding a field about the payout-method used?
and i would love to see standardized uris too.

Uris? 

What do you mean by payout method exactly?

I really like these suggestions, I will soon be adding proper json and may aswell make use of the suggested options once "finalised" in this thread I assume.

I assume he mean reward method (PPS, SMPSS, DGM, etc)?

Including things like fee %, payout tx fees, merge mining would make it possible to keep something like this actually up to date. :)
https://en.bitcoin.it/wiki/Comparison_of_mining_pools

yes, thanks
would be easy to develop an app to monitor multiple pools then

https://en.bitcoin.it/wiki/BIP_PoolAPI WIP

p_shep approves this message :)

A standardised api for pool stat history would be nice.

"rewardalgo" may have to go per worker. Some pools you can select what worker does what... EMC & Ozcoin allows this. Perhaps this field can be used to list available algorithms.
I quite liked my idea for a round/block structure. Allows for block history to use the same structure repeated. And everyone likes re-using code :)

For a pool history API, for pools like deepbit where pool history will be thousands of blocks there might need to be a protocol limiting the number of blocks to 100 recent blocks, and then have a slightly different url for blocks further back.

I'd like to see a JSON or other API present a pool history table like this:

Data should be standardised too. 'Round duration' should be in seconds and the timestamp should be in the standard unix  %Y-%m-%d %H:%M:%S. Current difficulty is important to include. It's easily available elsewhere, but this being a block history API it's a pain for a miner to search the difficulty history for each and every block.

The field for <Payout method specific field> can be used for any results specific to a payout method. I had SMPPS in mind (a field for buffer), but I'm sure other payout methods would find a use for it.

standardisation is slowly creeping into bitcoin - which is good

Ozcoin shows per worker hashrate as well as total, this was on user request - I think it needs to be included for people that use API to monitor rigs up/down. e.g. our desktop widget will not work without separated worker hashrates.
At Ozcoin payout method is by user not worker (this might change in the future)
Currently we are doing some heavy development on the backend and site - whilst a great idea a standardised API is not high on our priorities.

Agree


Agreed (as in integer unformated # of seconds)

In my opinon formatting has no place in an API that is a presentation layer task.  Unix/Posix time is better since it is an integer and every language has tools to format it.

For example as of writing Unix/Posix time is 1337632227

Still the larger point is that data format should be part of the spec.

Agreed.

I would definitely prefer it to be unixtime for my own purposes, although I get a seamless conversion when I need it. However I proposed the above format because I thought that the average miner might not be able to use unixtime so easily.
Yes, that's the main thing. Anyone wanting to - for example - create a website to list miner api results or pool history will need all data standardised if she or he's going to list all pools equally easily. Btc-poolwatch.com was a constant job for koo (the owner/writer) since adding an api meant writing a new code to interpret the api, and any api changes broke the service.

I think UNIX timestamp is the way to go.  If a pool wants to add extra "human" time, then that's easily done.  I think the standard as UNIX time is the right choice.  Really, all time should be given in unix time stamps and extra fields added for human time if desired.

So my proposal for the standard:

All time displays presented as UNIX epoc.  This will also do away with the need for timezoning questions/issues.
All hash rates presented as hashes per second.
All BTC's presented as Satoshis

Sounds good to me

Good proposal Inaba, and thanks to Inaba and P_Shep for getting this going.

Can I add:

• All fields to be integer (no char, no added commas etc, no double)
• Apis should all be the same format (preferably JSON).
  
• The number and order of fields in each api should be standardised.
• The api call should be standardised -
  eg the following could call worker stats:

• Any api call that includes historical data should include an option for selection of amount of data. Eg if X is the last X rounds of historical data:[\li]
  
  Code:

  poolname.com/api/username/history?X

  
  

One more thing - a pool history request shouldn't require a key (Ahem maxbtc, I'm looking at you :))

What if user (or few of his workers) are PPS? null value?


UTC, not GMT! GMT has summer/winter time, which is always pain in applications.


Let's rename it to balance, balance_unconfirmed, balance_paid. Otherwise we'll rename those fields for NMC? :-)

I'm not against this initiative if it will need just to rename columns and URLs in my own API (which was - as Tycho mentioned - the first API, and there's no general issue with that). That means - I don't see any reason to change token management etc.

Also, we should define some standard on ask rate from clients. I'm caching API calls for 30 seconds, but still I see some stupid users fetching JSON data every second. That's not only annoying, that also needs much more transfers than mining itself of those users (especially when they're CPU miners).

JSON makes both of these immaterial.  No need to have them ordered and it would impose an undue burden on the pool side.  The number of fields would also be immaterial, just discard the fields you don't want to use.  That way it allows a pool to provide additional information beyond the standard.


This is nice in theory, but I'm not sure it can work in practice, given all the different systems out there. In your example it would create a supremely difficult problem for me to provide since I'm not setup to handle calls in that fashion.  If we are going to standardize in this fashion, HTTP GET is probably the best solution, since it should be universally compatible.


Zero would be the logical amount, since it would be an accurate value.


Yes!  Sorry I meant UTC not GMT!


I did that specifically so you could add nmc_balance, nmc_unconfirmed, and nmc_paid.  Although I suppose we could make them a sub array of BTC: or NMC: (or whatever other currency) and then just use balance, unconfirmed, paid.


Yes, I absolutely agree.  I see the same things.

Have you guys seen Luke's BIP?

https://en.bitcoin.it/wiki/BIP_PoolAPI

Some interesting ideas on displaying balances. Very generic, so can be used for any currency.

Things like the API call and call rate I should think can be totally up to the pool. I don't think there's any need to standardise these.

Having currency name in variable doesn't look smart. Then I prefer - separate calls completely (and have an api for every currency) or have separate section for every currency, like ...'balance': {'BTC': {'unconfirmed': 0, 'confirmed': 0, 'paid': 0}, 'NMC': {...}} ...

Yes and I see it overcomplicated, especially that "collab" construction. Better to have more APIs than one huge monster, isn't it? I even think that current API should be split into more calls like account state (balances, address, etc) and worker states.

Yes it is rather. I didn't intend such complexity.


That was my original suggestion :)

but having structure:
balance =
{
  type = BTC
  confirmed = xxx
  unconfimed = xxx
  ...etc...
}

As an array inside the main structure also works for me.

user =
{
    ...
    balances = {array of balance}
    ...
}

I'd suggest providing 2 API's/url's

One API that requires an API key, showing all the information related to that user.

A second api/url containing the pool data, global data that is not related to a specific user, so it can be retrieved by any user without an API key.  Examples are global hashrate, found blocks history and such.

Seperating them would make it easy for people to get full data on any pool without an account...

Perhaps a third url can be made like /standard_api_locations.txt where the location of both those url's are placed, so the user doesn't have to enter anything except the main pool url.

I think pool round history is probably better as a .csv than in JSON. It's much easier to use a dataframe when working with aggregates of data. Also much easier for the poor sods using excel to get a feel for their pool's recent history.

As Inaba said the number and order of fields in JSON is immaterial, but that's not the case in a dataframe, especially if you want to perform exactly the same operations on different dataframes from different pools.

To be exact, this is true only for keys of an array. Lists (and history of blocks should be definitely stored in list) are ordered. But I see your point. Actually I'm for defining JSON API as an standard, because it's usually easier to handle in applications, and provide alternate CSV API with block history only (from pool side it's just an iteration over original JSON and columns with expected order).

I would agree, historical data is better provided as CSV.  The API should be for relevant current information that changes frequently.

Json has overhead compared to csv.  However, csv is not extensible, and the information isn't that big, so the overhead doesn't matter that much.  I would prefer json over csv.  A simple (online) convertor can still be made for those that prefer to work with csv instead of json.  If it is a standard, such a convertor would be easy to write for all pools at once...

csv is only considered for historical data that would be easier to manipulate as a dataframe and needs updating only when a block is found.

CSV for block data sounds reasonable to me.

shares =
{
    "submitted" = <shares submitted>,
    "stale" = <shares deemed stale by pool>,
    "invalid" = <shares deemed invalid by pool>
}

block =
{
    "currency" = <BTC, NMC, etc...>
    "id" = <block ID>,
    "duration" = <seconds>,
    "shares_round" = {<'shares' for entire pool for the round (as defined above)>},
    "shares_user" = {<'shares' for all workers of the user for the round (as defined above)>}
}

balance =
{
    "currency" = <BTC, NameCoin, Ix, iO, etc.>,
    "confirmed" = <confirmed reward (>=120 valid blocks) in Satoshis>,
    "unconfirmed" = <unconfirmed reward (<120 valid blocks) in Satoshis>,
    "estimate" = <estimated reward for current round>,
    "last_pay" = <value of last payout in Satoshis>,
    "last_pay_time" = <time of last payout in unix time>,
    "total_pay" = <total value of payout in Satoshis>,
    "threshold" = <min confirmed reward before auto payout in Satoshis. 0 for no auto payment>
}

worker =
{
    "id" = <unique identifier>,
    "name" = <name / description>,
    "hashrate" = <worker hashrate>,
    "last_activity" = <last submitted share time in unix time>,
    "shares_round" = {<'shares' for current round (as defined above)>},
    "shares_reset" = {<'shares' since last user reset (as defined above)>},
    "shares_total" = {<'shares' for all time (as defined above)>},
    "reward_algo" = <reward algorithm identifier>,
    "fee" = <fee multiplier (1% = 0.01)>
}

user =
{
    "API" = 1.0.0
    "pool_MOTD" = <whatever the pool wants to tell us>,
    "pool_hashrate" = <entire pool current hashrate>,
    "pool_users" = <number of active users on the pool>,
    "pool_workers" = <number of active workers on the pool>,
    "current_round" = {<array of active 'block' (as defined above)>},
    "user_hashrate" = <current hashrate in H/s for all workers - implementation pool dependent>,
    "last_activity" = <last submitted share time in unix time>,
    "balances" = {<array of 'balance' (as defined above)>},
    "workers" = {<array of 'worker' (as defined above)>}
}

Where:
All time displays presented as UNIX epoc.
All hash rates presented as hashes per second.
'Satoshis' is the smallest value for the associated currency. i.e. For USD this would be 1c.

+1

I would love to see this implemented.

I (we!) missed a very important metric up there, invalid/stale shares. So I've tweaked it a bit.
Getting rather complicated now though! Didn't intend for it to go this way, but at least it's pretty complete!

https://bitcointalk.org/index.php?topic=83027.msg921816#msg921816

Did you update the wiki page with the specification?

I should probably do that...

Reminds me of

https://i.imgur.com/M0LYX.png

Certainly does :D

but no standard exists for this yet... all have been proprietary.

Hoppable prop pools will not be wanting to give out that much information... of course that's their problem.

That's why everything is optional.

Word.

Please try to edit the existing specification draft, rather than replace it entirely. I combined the two, but please double-check that I didn't miss anything. Note that JSON does not have Float or Time types, just Number.

I also made a quick pass over it to try to make it sensible for Bitcoin address mining.

A suggestion (or 2) ... welcome to be ignored :)


Elapsed should 'almost' never be used unless there is an actual specific reason for it
How long has a block been around is a good example of when it shouldn't be used - that should be a timestamp
(and saying it makes it easier for recipient XYZ simply means you got it wrong)
Timestamp things and return those integer sec timestamps or float timeval if higher accuracy is required
Also, as I did in the cgminer API, include a 'now' with any data set that has timestamps
(the cgminer API STATUS header always has 'When' for that reason ... once I realised it was missing)
This resolves any issue of ever creating correct elapsed values if anyone wants to display them


Also - I like float MH/s not H/s since that is what everyone thinks in nowadays and yeah in the future it will be even bigger
1 TH/s is 1000000000000 H/s - that number is just way too big already ...
1000000.000000 MH/s is 'nicer' IMO :)

It is a timestamp...

1e12 is valid JSON.