Pool API Standard

[Inaba]

The number and order of fields in each api should be standardised.

JSON makes both of these immaterial.  No need to have them ordered and it would impose an undue burden on the pool side.  The number of fields would also be immaterial, just discard the fields you don't want to use.  That way it allows a pool to provide additional information beyond the standard.

The api call should be standardised -
eg the following could call worker stats:

This is nice in theory, but I'm not sure it can work in practice, given all the different systems out there. In your example it would create a supremely difficult problem for me to provide since I'm not setup to handle calls in that fashion.  If we are going to standardize in this fashion, HTTP GET is probably the best solution, since it should be universally compatible.

What if user (or few of his workers) are PPS? null value?

Zero would be the logical amount, since it would be an accurate value.

UTC, not GMT! GMT has summer/winter time, which is always pain in applications.

Yes!  Sorry I meant UTC not GMT!

Let's rename it to balance, balance_unconfirmed, balance_paid. Otherwise we'll rename those fields for NMC?

I did that specifically so you could add nmc_balance, nmc_unconfirmed, and nmc_paid.  Although I suppose we could make them a sub array of BTC: or NMC: (or whatever other currency) and then just use balance, unconfirmed, paid.

Also, we should define some standard on ask rate from clients. I'm caching API calls for 30 seconds, but still I see some stupid users fetching JSON data every second. That's not only annoying, that also needs much more transfers than mining itself of those users (especially when they're CPU miners).

Yes, I absolutely agree.  I see the same things.

[P_Shep]

Have you guys seen Luke's BIP?

https://en.bitcoin.it/wiki/BIP_PoolAPI

Some interesting ideas on displaying balances. Very generic, so can be used for any currency.

Things like the API call and call rate I should think can be totally up to the pool. I don't think there's any need to standardise these.

[slush]

I did that specifically so you could add nmc_balance, nmc_unconfirmed, and nmc_paid.  Although I suppose we could make them a sub array of BTC: or NMC: (or whatever other currency) and then just use balance, unconfirmed, paid.

Having currency name in variable doesn't look smart. Then I prefer - separate calls completely (and have an api for every currency) or have separate section for every currency, like ...'balance': {'BTC': {'unconfirmed': 0, 'confirmed': 0, 'paid': 0}, 'NMC': {...}} ...

[slush]

Have you guys seen Luke's BIP?

Yes and I see it overcomplicated, especially that "collab" construction. Better to have more APIs than one huge monster, isn't it? I even think that current API should be split into more calls like account state (balances, address, etc) and worker states.

[P_Shep]

Yes and I see it overcomplicated, especially that "collab" construction.

Yes it is rather. I didn't intend such complexity.

Better to have more APIs than one huge monster, isn't it? I even think that current API should be split into more calls like account state (balances, address, etc) and worker states.

That was my original suggestion

but having structure:
balance =
{
  type = BTC
  confirmed = xxx
  unconfimed = xxx
  ...etc...
}

As an array inside the main structure also works for me.

user =
{
    ...
    balances = {array of balance}
    ...
}

[kinlo]

I'd suggest providing 2 API's/url's

One API that requires an API key, showing all the information related to that user.

A second api/url containing the pool data, global data that is not related to a specific user, so it can be retrieved by any user without an API key.  Examples are global hashrate, found blocks history and such.

Seperating them would make it easy for people to get full data on any pool without an account...

Perhaps a third url can be made like /standard_api_locations.txt where the location of both those url's are placed, so the user doesn't have to enter anything except the main pool url.

[organofcorti]

I think pool round history is probably better as a .csv than in JSON. It's much easier to use a dataframe when working with aggregates of data. Also much easier for the poor sods using excel to get a feel for their pool's recent history.

As Inaba said the number and order of fields in JSON is immaterial, but that's not the case in a dataframe, especially if you want to perform exactly the same operations on different dataframes from different pools.

[slush]

As Inaba said the number and order of fields in JSON is immaterial

To be exact, this is true only for keys of an array. Lists (and history of blocks should be definitely stored in list) are ordered. But I see your point. Actually I'm for defining JSON API as an standard, because it's usually easier to handle in applications, and provide alternate CSV API with block history only (from pool side it's just an iteration over original JSON and columns with expected order).

[Inaba]

I would agree, historical data is better provided as CSV.  The API should be for relevant current information that changes frequently.

[kinlo]

I would agree, historical data is better provided as CSV.  The API should be for relevant current information that changes frequently.

Json has overhead compared to csv.  However, csv is not extensible, and the information isn't that big, so the overhead doesn't matter that much.  I would prefer json over csv.  A simple (online) convertor can still be made for those that prefer to work with csv instead of json.  If it is a standard, such a convertor would be easy to write for all pools at once...

[organofcorti]

I would agree, historical data is better provided as CSV.  The API should be for relevant current information that changes frequently.

Json has overhead compared to csv.  However, csv is not extensible, and the information isn't that big, so the overhead doesn't matter that much.  I would prefer json over csv.  A simple (online) convertor can still be made for those that prefer to work with csv instead of json.  If it is a standard, such a convertor would be easy to write for all pools at once...

csv is only considered for historical data that would be easier to manipulate as a dataframe and needs updating only when a block is found.

[P_Shep]

CSV for block data sounds reasonable to me.

[P_Shep]

shares =
{
    "submitted" = <shares submitted>,
    "stale" = <shares deemed stale by pool>,
    "invalid" = <shares deemed invalid by pool>
}

block =
{
    "currency" = <BTC, NMC, etc...>
    "id" = <block ID>,
    "duration" = <seconds>,
    "shares_round" = {<'shares' for entire pool for the round (as defined above)>},
    "shares_user" = {<'shares' for all workers of the user for the round (as defined above)>}
}

balance =
{
    "currency" = <BTC, NameCoin, Ix, iO, etc.>,
    "confirmed" = <confirmed reward (>=120 valid blocks) in Satoshis>,
    "unconfirmed" = <unconfirmed reward (<120 valid blocks) in Satoshis>,
    "estimate" = <estimated reward for current round>,
    "last_pay" = <value of last payout in Satoshis>,
    "last_pay_time" = <time of last payout in unix time>,
    "total_pay" = <total value of payout in Satoshis>,
    "threshold" = <min confirmed reward before auto payout in Satoshis. 0 for no auto payment>
}

worker =
{
    "id" = <unique identifier>,
    "name" = <name / description>,
    "hashrate" = <worker hashrate>,
    "last_activity" = <last submitted share time in unix time>,
    "shares_round" = {<'shares' for current round (as defined above)>},
    "shares_reset" = {<'shares' since last user reset (as defined above)>},
    "shares_total" = {<'shares' for all time (as defined above)>},
    "reward_algo" = <reward algorithm identifier>,
    "fee" = <fee multiplier (1% = 0.01)>
}

user =
{
    "API" = 1.0.0
    "pool_MOTD" = <whatever the pool wants to tell us>,
    "pool_hashrate" = <entire pool current hashrate>,
    "pool_users" = <number of active users on the pool>,
    "pool_workers" = <number of active workers on the pool>,
    "current_round" = {<array of active 'block' (as defined above)>},
    "user_hashrate" = <current hashrate in H/s for all workers - implementation pool dependent>,
    "last_activity" = <last submitted share time in unix time>,
    "balances" = {<array of 'balance' (as defined above)>},
    "workers" = {<array of 'worker' (as defined above)>}
}

Where:
All time displays presented as UNIX epoc.
All hash rates presented as hashes per second.
'Satoshis' is the smallest value for the associated currency. i.e. For USD this would be 1c.

[JinTu]

block =
{
    "currency" = <BTC, NMC, etc...>
    "id" = <block ID>,
    "duration" = <seconds>,
    "shares_total" = <total shares submitted so far for round>,
    "shares_submitted" = <shares submitted by user (all workers)>
}

balance =
{
    "currency" = <BTC, NameCoin, Ix, iO, etc.>,
    "confirmed" = <confirmed reward (>=120 valid blocks) in Satoshis>,
    "unconfirmed" = <unconfirmed reward (<120 valid blocks) in Satoshis>,
    "estimate" = <estimated reward for current round>,
    "last_pay" = <value of last payout in Satoshis>,
    "last_pay_time" = <time of last payout in unix time>,
    "total_pay" = <total value of payout in Satoshis>,
    "threshold" = <min confirmed reward before auto payout in Satoshis. 0 for no auto payment>
}

worker =
{
    "id" = <unique identifier>,
    "name" = <name / description>,
    "hash_rate" = <worker hash rate>,
    "last_activity" = <last submitted share time in unix time>,
    "shares" = <shares submitted by worker in current round>,
    "shares_total" = <total shares submitted by worker (may be resetable by pool?)>,
    "reward_algo" = <reward algorithm identifier>,
    "fee" = <fee multiplier (1% = 0.01)>
}

user =
{
    "API" = 1.0.0
    "pool_MOTD" = <whatever the pool wants to tell us>,
    "pool_hash_rate" = <entire pool current hash rate>,
    "pool_users" = <number of active users on the pool>,
    "pool_workers" = <number of active workers on the pool>,
    "current_round" = {<array of active 'block' (as defined above)>},
    "hash_rate" = <current hash rate in H/s for all workers - implementation pool dependent>,
    "last_activity" = <last submitted share time in unix time>,
    "balances" = {<array of 'balance' (as defined above)>},
    "workers" = {<array of 'worker' (as defined above)>}
}

Where:
All time displays presented as UNIX epoc.
All hash rates presented as hashes per second.
'Satoshis' is the smallest value for the associated currency. i.e. For USD this would be 1c.

+1

I would love to see this implemented.

[P_Shep]

I (we!) missed a very important metric up there, invalid/stale shares. So I've tweaked it a bit.
Getting rather complicated now though! Didn't intend for it to go this way, but at least it's pretty complete!

https://bitcointalk.org/index.php?topic=83027.msg921816#msg921816

[Luke-Jr]

I (we!) missed a very important metric up there, invalid/stale shares. So I've tweaked it a bit.
Getting rather complicated now though! Didn't intend for it to go this way, but at least it's pretty complete!

https://bitcointalk.org/index.php?topic=83027.msg921816#msg921816

Did you update the wiki page with the specification?

[P_Shep]

I (we!) missed a very important metric up there, invalid/stale shares. So I've tweaked it a bit.
Getting rather complicated now though! Didn't intend for it to go this way, but at least it's pretty complete!

https://bitcointalk.org/index.php?topic=83027.msg921816#msg921816

Did you update the wiki page with the specification?

I should probably do that...

[Tittiez]

Reminds me of

[P_Shep]

Reminds me of

Certainly does

but no standard exists for this yet... all have been proprietary.

[-ck]

Hoppable prop pools will not be wanting to give out that much information... of course that's their problem.