why JSON RPC values not use INT64 instead of float string?

[genjix]

This would solve a bunch of problems.

Why not just return the int64 and let the client cast it to a float & divide 10^8 for display?

Currently since the JSON RPC returns floats, any library you use will return the values as floats, not strings. So to get the int64 value you need to multiply the float by 10^8 and cast to an int for internal usage.

Also by returning the value as int64, it will be enforcing good practice on clients instead of them unwittingly using floats.

[Gavin Andresen]

Because JavaScript doesn't have a 64-bit integer type (all Numbers in JavaScript are double-precision floating point).

[ribuck]

all Numbers in JavaScript are double-precision floating point

Fortunately for Bitcoin, double-precision floating point represents integers exactly up to 9,007,199,254,740,992 which is above the number of bitcoin base units i.e. 2,100,000,000,000,000.

[Gavin Andresen]

Can we not beat this dead horse?

I think there are MUCH more important things to work on / worry about than whether or not "send 1 BTC" is expressed as "sendtoaddress FOO 1.00" or "sendtoaddress FOO 100000000" in the JSON-RPC.

How about we (I'll start) write a "Proper Money Handling" page for the Wiki that discusses the issue and gives code example of how to convert to/from JSON double-precision floating point and 64-bit integer?

[genjix]

Python uses floats for it's JSON library. Herein lies the problems.

$ python
>>> import json
>>> json.dumps(10.001)
'10.000999999999999'
>>> json.loads('{"blaa": 0.333331}')
{u'blaa': 0.33333099999999999}
>>> type(json.loads('{"blaa": 0.333331}')['blaa'])
<type 'float'>

This is unacceptable.

[Gavin Andresen]

Wiki page created:  https://en.bitcoin.it/wiki/Proper_Money_Handling_(JSON-RPC)

genjix:  You should be calling json.loads(..., parse_float=decimal.Decimal) and use a custom JSON encoder class to convert decimals to JSON strings with no loss of precision...

[genjix]

Yep, just noticed that.

But then the Python JSON-RPC library does not and I couldn't find one for the PHP RPC library either.

Code:

>>> from jsonrpc import ServiceProxy
>>> access = ServiceProxy("http://user:password@127.0.0.1:8332")
>>> type(access.getbalance())
<type 'float'>

BTW, in that wiki page why did you use those lambdas instead of simply using decimal.Decimal? Multiplying by e8 will cause everything like version numbers or difficulty to be multiplied.

[Luke-Jr]

Currently since the JSON RPC returns floats, any library you use will return the values as floats, not strings. So to get the int64 value you need to multiply the float by 10^8 and cast to an int for internal usage.

It's one of the many JSON-RPC design flaws. Instead of trying to fix it, I have moved on to working on a new protocol to address all the problems: https://www.bitcoin.org/smf/index.php?topic=3757.0

[Gavin Andresen]

genjix: here is how to do it right in Python2.6 :

Code:

import decimal
import json

# From http://stackoverflow.com/questions/1960516/python-json-serialize-a-decimal-object
class DecimalEncoder(json.JSONEncoder):
  def _iterencode(self, o, markers=None):
    if isinstance(o, decimal.Decimal):
      return (str(o) for o in [o])
    return super(DecimalEncoder, self)._iterencode(o, markers)

decimal.setcontext(decimal.Context(prec=8))

print json.dumps(decimal.Decimal('10.001'), cls=DecimalEncoder)
print json.dumps({ "decimal" : decimal.Decimal('1.1'), "float" : 1.1, "string" : "1.1" }, cls=DecimalEncoder)
print json.loads('{"blaa": 0.333331}', parse_float=decimal.Decimal)

Produces output:

Code:

10.001
{"decimal": 1.1, "float": 1.1000000000000001, "string": "1.1"}
{u'blaa': Decimal('0.333331')}

Note that EVEN IF YOU PASSED THE 'WRONG' strings to Bitcoin, Bitcoin would do the right thing. That is, these two are equivalent once they are parsed by bitcoin:

Code:

sendtoaddress FOO 10.000999999999999
sendtoaddress FOO 10.001

... because bitcoin does proper rounding.

On the bitcoin side, this is a non-issue.  And if code on the other end of the JSON-RPC connection does the wrong thing (truncates values like 10.000999999999999 instead of rounding them to the nearest 8'th decimal place) then that's a bug in that code.

[genjix]

That's code to parse JSONs. There's a Python library to work with JSON-RPC. There's also a PHP library to do JSON-RPC. Both use floats.

Solution A: everybody that wishes to interface with Bitcoin in Python/PHP must write their own (potentially buggy) RPC http code because the default libs for those languages uses floats.
Solution B: a small change is made to Bitcoin.

B is a much better solution

[Gavin Andresen]

That's code to parse JSONs. There's a Python library to work with JSON-RPC.

Huh?  See that 'import json' statement at the top?  That would be the standard (as of python 2.6) JSON parsing library.

The code I posted tells the standard JSON parsing library to read JSON Numbers as Decimal.  If you are doing monetary calculations in python, then you should be using Decimal.  That is what it is for.

[genjix]

That's code to parse JSONs. There's a Python library to work with JSON-RPC.

Huh?  See that 'import json' statement at the top?  That would be the standard (as of python 2.6) JSON parsing library.

The code I posted tells the standard JSON parsing library to read JSON Numbers as Decimal.  If you are doing monetary calculations in python, then you should be using Decimal.  That is what it is for.

Not json but json-rpc as recommended by json-rpc themselves.
http://json-rpc.org/wiki/python-json-rpc

[genjix]

This is stubborness... Now I'm trying to integrate Bitcoin into a website but the JSON-RPC library only returns floats.

Floating point numbers have limited precision. Although it depends on the system, PHP typically uses the IEEE 754 double precision format, which will give a maximum relative error due to rounding in the order of 1.11e-16. Non elementary arithmetic operations may give larger errors, and, of course, error progragation must be considered when several operations are compounded.

Additionally, rational numbers that are exactly representable as floating point numbers in base 10, like 0.1 or 0.7, do not have an exact representation as floating point numbers in base 2, which is used internally, no matter the size of the mantissa. Hence, they cannot be converted into their internal binary counterparts without a small loss of precision. This can lead to confusing results: for example, floor((0.1+0.7)*10) will usually return 7 instead of the expected 8, since the internal representation will be something like 7.9999999999999991118....

So never trust floating number results to the last digit, and never compare floating point numbers for equality. If higher precision is necessary, the arbitrary precision math functions and gmp functions are available.

Why can't Bitcoin return strings?

You're deliberately breaking with all the languages (python JSON-RPC library, PHP JSON-RPC and Perl JSON-RPC). Using floats anywhere in financial transactions is unacceptable.

php's json_decode DOESN'T support anyway to return floats as strings. The option doesn't exist.
PHP solution: write/maintain my own JSON parser.

Neither does Python's JSON-RPC. Have to write my own JSON-RPC lib using json module instead of using the one that already exists.

Same for Perl.

Really, why is it such a big deal? Bitcoin is broken and this needs fixing.

[Gavin Andresen]

PHP solution: write/maintain my own JSON parser.

Why can't you just multiply the numbers by 1.0e8 and then round to the nearest integer?  That integer WILL ALWAYS BE EXACTLY RIGHT (assuming you're not running PHP on some really weird hardware).

According to the PHP manual:

The size of a float is platform-dependent, although a maximum of ~1.8e308 with a precision of roughly 14 decimal digits is a common value (the 64 bit IEEE format).

I added a Python JSON-RPC library example on the Proper Money Handling wiki page.

[ShadowOfHarbringer]

This is stubborness... Now I'm trying to integrate Bitcoin into a website but the JSON-RPC library only returns floats.

Why can't Bitcoin return strings?

+ 1
Floats are a royal pain in the ass. Every bank application programmer will probably tell you that.
Especially RPC-like services should operate on strings - it makes a lot of stuff easier and allows infinite precision.

According to the PHP manual:

The size of a float is platform-dependent, although a maximum of ~1.8e308 with a precision of roughly 14 decimal digits is a common value (the 64 bit IEEE format).

For operations of extreme precision, PHP has many sets of mathematical libraries which also operate on strings, not floats.

For example, BC-Math or GMP.

http://php.net/manual/en/book.bc.php
http://www.php.net/manual/en/book.gmp.php

[ShadowOfHarbringer]

One more thing:

Additionally I think that in PHP, result of float <-> integer calculations may differ on 32Bit & 64Bit platforms, however i may be wrong (and I am too lazy to check with Google).

There may be some bugs or php-specific "features" involved.

[genjix]

ShadowOfHarbringer: Do you know how to divide numbers in PHP GMP and obtain a decimal number (not quotient + remainder)?

[ShadowOfHarbringer]

ShadowOfHarbringer: Do you know how to divide numbers in PHP GMP and obtain a decimal number (not quotient + remainder)?

Well it should be fairly easy using GMP, however i have never done that before.

You will probably need to study usage of following functions:

http://www.php.net/manual/en/function.gmp-init.php
http://www.php.net/manual/en/function.gmp-div-q.php
http://www.php.net/manual/en/function.gmp-strval.php

Example from PHP.net:

Code:

<?php

$div1 = gmp_div_q("100", "5");
echo gmp_strval($div1) . "\n";

$div2 = gmp_div_q("1", "3");
echo gmp_strval($div2) . "\n";

$div3 = gmp_div_q("1", "3", GMP_ROUND_PLUSINF);
echo gmp_strval($div3) . "\n";

$div4 = gmp_div_q("-1", "4", GMP_ROUND_PLUSINF);
echo gmp_strval($div4) . "\n";

$div5 = gmp_div_q("-1", "4", GMP_ROUND_MINUSINF);
echo gmp_strval($div5) . "\n";

And result:

Code:

20
0
1
0
-1

[genjix]

Yeah those are integer values,

var_dump(gmp_strval(gmp_div_q(gmp_init("5"), gmp_init("2"))));

output:
string(1) "2" 

Here's my solution, http://codepad.viper-7.com/tbZ9oD

Code:

<?php

$quot = gmp_init("5");
$divis = gmp_init("2");
# number of decimals
$precision = 2;

$shift = gmp_pow("10", $precision);
$quot = gmp_mul($quot, $shift);

$res = gmp_div_q($quot, $divis);
$repr = gmp_strval($res);
$dotpos = strlen($repr) - $precision;
$repr = substr($repr, 0, $dotpos) . "." . substr($repr, $dotpos);

echo "Number is $repr";
$res = gmp_init($repr);

Multiply the quotient by 10^p (p = precision), perform the integer division, convert to string, insert the decimal point, convert back to GMP.

[ShadowOfHarbringer]

Yeah those are integer values,

var_dump(gmp_strval(gmp_div_q(gmp_init("5"), gmp_init("2"))));

output:
string(1) "2" 

Here's my solution, http://codepad.viper-7.com/tbZ9oD

Code:

<?php

$quot = gmp_init("5");
$divis = gmp_init("2");
# number of decimals
$precision = 2;

$shift = gmp_pow("10", $precision);
$quot = gmp_mul($quot, $shift);

$res = gmp_div_q($quot, $divis);
$repr = gmp_strval($res);
$dotpos = strlen($repr) - $precision;
$repr = substr($repr, 0, $dotpos) . "." . substr($repr, $dotpos);

echo "Number is $repr";
$res = gmp_init($repr);

Multiply the quotient by 10^p (p = precision), perform the integer division, convert to string, insert the decimal point, convert back to GMP.

Can't you do everything of that iside GMP (resources) ?
It will be much faster than operating on strings.

Moving the decimal point should be also possible inside GMP.