Bitcoin Forum
March 29, 2024, 01:32:50 AM *
News: Latest Bitcoin Core release: 26.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: [1] 2 3 4 5 »  All
  Print  
Author Topic: why JSON RPC values not use INT64 instead of float string?  (Read 18410 times)
genjix (OP)
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1071


View Profile
March 03, 2011, 12:49:34 PM
 #1

This would solve a bunch of problems.

Why not just return the int64 and let the client cast it to a float & divide 10^8 for display?

Currently since the JSON RPC returns floats, any library you use will return the values as floats, not strings. So to get the int64 value you need to multiply the float by 10^8 and cast to an int for internal usage.

Also by returning the value as int64, it will be enforcing good practice on clients instead of them unwittingly using floats.
Transactions must be included in a block to be properly completed. When you send a transaction, it is broadcast to miners. Miners can then optionally include it in their next blocks. Miners will be more inclined to include your transaction if it has a higher transaction fee.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1711675970
Hero Member
*
Offline Offline

Posts: 1711675970

View Profile Personal Message (Offline)

Ignore
1711675970
Reply with quote  #2

1711675970
Report to moderator
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 2164


Chief Scientist


View Profile WWW
March 03, 2011, 01:15:26 PM
 #2

Because JavaScript doesn't have a 64-bit integer type (all Numbers in JavaScript are double-precision floating point).

How often do you get the chance to work on a potentially world-changing project?
ribuck
Donator
Hero Member
*
Offline Offline

Activity: 826
Merit: 1039


View Profile
March 03, 2011, 01:23:16 PM
 #3

all Numbers in JavaScript are double-precision floating point
Fortunately for Bitcoin, double-precision floating point represents integers exactly up to 9,007,199,254,740,992 which is above the number of bitcoin base units i.e. 2,100,000,000,000,000.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 2164


Chief Scientist


View Profile WWW
March 03, 2011, 01:33:48 PM
 #4

Can we not beat this dead horse?

I think there are MUCH more important things to work on / worry about than whether or not "send 1 BTC" is expressed as "sendtoaddress FOO 1.00" or "sendtoaddress FOO 100000000" in the JSON-RPC.

How about we (I'll start) write a "Proper Money Handling" page for the Wiki that discusses the issue and gives code example of how to convert to/from JSON double-precision floating point and 64-bit integer?

How often do you get the chance to work on a potentially world-changing project?
genjix (OP)
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1071


View Profile
March 03, 2011, 02:17:12 PM
Last edit: March 03, 2011, 02:34:54 PM by genjix
 #5

Python uses floats for it's JSON library. Herein lies the problems.

$ python
>>> import json
>>> json.dumps(10.001)
'10.000999999999999'
>>> json.loads('{"blaa": 0.333331}')
{u'blaa': 0.33333099999999999}
>>> type(json.loads('{"blaa": 0.333331}')['blaa'])
<type 'float'>

This is unacceptable.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 2164


Chief Scientist


View Profile WWW
March 03, 2011, 02:33:21 PM
 #6

Wiki page created:  https://en.bitcoin.it/wiki/Proper_Money_Handling_(JSON-RPC)

genjix:  You should be calling json.loads(..., parse_float=decimal.Decimal) and use a custom JSON encoder class to convert decimals to JSON strings with no loss of precision...

How often do you get the chance to work on a potentially world-changing project?
genjix (OP)
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1071


View Profile
March 03, 2011, 02:43:17 PM
 #7

Yep, just noticed that.

But then the Python JSON-RPC library does not and I couldn't find one for the PHP RPC library either.

Code:
>>> from jsonrpc import ServiceProxy
>>> access = ServiceProxy("http://user:password@127.0.0.1:8332")
>>> type(access.getbalance())
<type 'float'>

BTW, in that wiki page why did you use those lambdas instead of simply using decimal.Decimal? Multiplying by e8 will cause everything like version numbers or difficulty to be multiplied.
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2576
Merit: 1186



View Profile
March 03, 2011, 02:54:55 PM
 #8

Currently since the JSON RPC returns floats, any library you use will return the values as floats, not strings. So to get the int64 value you need to multiply the float by 10^8 and cast to an int for internal usage.
It's one of the many JSON-RPC design flaws. Instead of trying to fix it, I have moved on to working on a new protocol to address all the problems: https://www.bitcoin.org/smf/index.php?topic=3757.0

Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 2164


Chief Scientist


View Profile WWW
March 03, 2011, 03:26:16 PM
 #9

genjix: here is how to do it right in Python2.6 :

Code:

import decimal
import json

# From http://stackoverflow.com/questions/1960516/python-json-serialize-a-decimal-object
class DecimalEncoder(json.JSONEncoder):
  def _iterencode(self, o, markers=None):
    if isinstance(o, decimal.Decimal):
      return (str(o) for o in [o])
    return super(DecimalEncoder, self)._iterencode(o, markers)

decimal.setcontext(decimal.Context(prec=8))

print json.dumps(decimal.Decimal('10.001'), cls=DecimalEncoder)
print json.dumps({ "decimal" : decimal.Decimal('1.1'), "float" : 1.1, "string" : "1.1" }, cls=DecimalEncoder)
print json.loads('{"blaa": 0.333331}', parse_float=decimal.Decimal)
Produces output:
Code:
10.001
{"decimal": 1.1, "float": 1.1000000000000001, "string": "1.1"}
{u'blaa': Decimal('0.333331')}

Note that EVEN IF YOU PASSED THE 'WRONG' strings to Bitcoin, Bitcoin would do the right thing. That is, these two are equivalent once they are parsed by bitcoin:
Code:
sendtoaddress FOO 10.000999999999999
sendtoaddress FOO 10.001
... because bitcoin does proper rounding.

On the bitcoin side, this is a non-issue.  And if code on the other end of the JSON-RPC connection does the wrong thing (truncates values like 10.000999999999999 instead of rounding them to the nearest 8'th decimal place) then that's a bug in that code.

How often do you get the chance to work on a potentially world-changing project?
genjix (OP)
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1071


View Profile
March 03, 2011, 03:34:56 PM
 #10

That's code to parse JSONs. There's a Python library to work with JSON-RPC. There's also a PHP library to do JSON-RPC. Both use floats.

Solution A: everybody that wishes to interface with Bitcoin in Python/PHP must write their own (potentially buggy) RPC http code because the default libs for those languages uses floats.
Solution B: a small change is made to Bitcoin.

B is a much better solution Smiley
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 2164


Chief Scientist


View Profile WWW
March 03, 2011, 08:06:34 PM
 #11

That's code to parse JSONs. There's a Python library to work with JSON-RPC.
Huh?  See that 'import json' statement at the top?  That would be the standard (as of python 2.6) JSON parsing library.

The code I posted tells the standard JSON parsing library to read JSON Numbers as Decimal.  If you are doing monetary calculations in python, then you should be using Decimal.  That is what it is for.

How often do you get the chance to work on a potentially world-changing project?
genjix (OP)
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1071


View Profile
March 03, 2011, 09:37:28 PM
 #12

That's code to parse JSONs. There's a Python library to work with JSON-RPC.
Huh?  See that 'import json' statement at the top?  That would be the standard (as of python 2.6) JSON parsing library.

The code I posted tells the standard JSON parsing library to read JSON Numbers as Decimal.  If you are doing monetary calculations in python, then you should be using Decimal.  That is what it is for.


Not json but json-rpc as recommended by json-rpc themselves.
http://json-rpc.org/wiki/python-json-rpc
genjix (OP)
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1071


View Profile
March 11, 2011, 08:53:38 PM
 #13

This is stubborness... Now I'm trying to integrate Bitcoin into a website but the JSON-RPC library only returns floats.

Quote
Floating point numbers have limited precision. Although it depends on the system, PHP typically uses the IEEE 754 double precision format, which will give a maximum relative error due to rounding in the order of 1.11e-16. Non elementary arithmetic operations may give larger errors, and, of course, error progragation must be considered when several operations are compounded.

Additionally, rational numbers that are exactly representable as floating point numbers in base 10, like 0.1 or 0.7, do not have an exact representation as floating point numbers in base 2, which is used internally, no matter the size of the mantissa. Hence, they cannot be converted into their internal binary counterparts without a small loss of precision. This can lead to confusing results: for example, floor((0.1+0.7)*10) will usually return 7 instead of the expected 8, since the internal representation will be something like 7.9999999999999991118....

So never trust floating number results to the last digit, and never compare floating point numbers for equality. If higher precision is necessary, the arbitrary precision math functions and gmp functions are available.

Why can't Bitcoin return strings?

You're deliberately breaking with all the languages (python JSON-RPC library, PHP JSON-RPC and Perl JSON-RPC). Using floats anywhere in financial transactions is unacceptable.

php's json_decode DOESN'T support anyway to return floats as strings. The option doesn't exist.
PHP solution: write/maintain my own JSON parser.

Neither does Python's JSON-RPC. Have to write my own JSON-RPC lib using json module instead of using the one that already exists.

Same for Perl.

Really, why is it such a big deal? Bitcoin is broken and this needs fixing.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 2164


Chief Scientist


View Profile WWW
March 11, 2011, 09:17:22 PM
 #14

PHP solution: write/maintain my own JSON parser.

Why can't you just multiply the numbers by 1.0e8 and then round to the nearest integer?  That integer WILL ALWAYS BE EXACTLY RIGHT (assuming you're not running PHP on some really weird hardware).

According to the PHP manual:
Quote
The size of a float is platform-dependent, although a maximum of ~1.8e308 with a precision of roughly 14 decimal digits is a common value (the 64 bit IEEE format).

I added a Python JSON-RPC library example on the Proper Money Handling wiki page.

How often do you get the chance to work on a potentially world-changing project?
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1005


Bringing Legendary Har® to you since 1952


View Profile
March 11, 2011, 09:19:46 PM
 #15

This is stubborness... Now I'm trying to integrate Bitcoin into a website but the JSON-RPC library only returns floats.

Why can't Bitcoin return strings?

+ 1
Floats are a royal pain in the ass. Every bank application programmer will probably tell you that.
Especially RPC-like services should operate on strings - it makes a lot of stuff easier and allows infinite precision.

According to the PHP manual:
Quote
The size of a float is platform-dependent, although a maximum of ~1.8e308 with a precision of roughly 14 decimal digits is a common value (the 64 bit IEEE format).

For operations of extreme precision, PHP has many sets of mathematical libraries which also operate on strings, not floats.

For example, BC-Math or GMP.

http://php.net/manual/en/book.bc.php
http://www.php.net/manual/en/book.gmp.php

ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1005


Bringing Legendary Har® to you since 1952


View Profile
March 11, 2011, 09:33:09 PM
 #16

One more thing:

Additionally I think that in PHP, result of float <-> integer calculations may differ on 32Bit & 64Bit platforms, however i may be wrong (and I am too lazy to check with Google).

There may be some bugs or php-specific "features" involved.

genjix (OP)
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1071


View Profile
March 11, 2011, 10:00:55 PM
 #17

ShadowOfHarbringer: Do you know how to divide numbers in PHP GMP and obtain a decimal number (not quotient + remainder)?
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1005


Bringing Legendary Har® to you since 1952


View Profile
March 11, 2011, 10:12:47 PM
Last edit: March 11, 2011, 10:52:20 PM by ShadowOfHarbringer
 #18

ShadowOfHarbringer: Do you know how to divide numbers in PHP GMP and obtain a decimal number (not quotient + remainder)?

Well it should be fairly easy using GMP, however i have never done that before.

You will probably need to study usage of following functions:

http://www.php.net/manual/en/function.gmp-init.php
http://www.php.net/manual/en/function.gmp-div-q.php
http://www.php.net/manual/en/function.gmp-strval.php

Example from PHP.net:

Code:
<?php

$div1 
gmp_div_q("100""5");
echo 
gmp_strval($div1) . "\n";

$div2 gmp_div_q("1""3");
echo 
gmp_strval($div2) . "\n";

$div3 gmp_div_q("1""3"GMP_ROUND_PLUSINF);
echo 
gmp_strval($div3) . "\n";

$div4 gmp_div_q("-1""4"GMP_ROUND_PLUSINF);
echo 
gmp_strval($div4) . "\n";

$div5 gmp_div_q("-1""4"GMP_ROUND_MINUSINF);
echo 
gmp_strval($div5) . "\n";

And result:

Code:
20
0
1
0
-1

genjix (OP)
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1071


View Profile
March 11, 2011, 10:35:33 PM
 #19

Yeah those are integer values,

var_dump(gmp_strval(gmp_div_q(gmp_init("5"), gmp_init("2"))));

output:
string(1) "2" 

Here's my solution, http://codepad.viper-7.com/tbZ9oD
Code:
<?php

$quot 
gmp_init("5");
$divis gmp_init("2");
# number of decimals
$precision 2;

$shift gmp_pow("10"$precision);
$quot gmp_mul($quot$shift);

$res gmp_div_q($quot$divis);
$repr gmp_strval($res);
$dotpos strlen($repr) - $precision;
$repr substr($repr0$dotpos) . "." substr($repr$dotpos);

echo 
"Number is $repr";
$res gmp_init($repr);

Multiply the quotient by 10^p (p = precision), perform the integer division, convert to string, insert the decimal point, convert back to GMP.
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1005


Bringing Legendary Har® to you since 1952


View Profile
March 11, 2011, 10:51:31 PM
 #20

Yeah those are integer values,

var_dump(gmp_strval(gmp_div_q(gmp_init("5"), gmp_init("2"))));

output:
string(1) "2" 

Here's my solution, http://codepad.viper-7.com/tbZ9oD
Code:
<?php

$quot 
gmp_init("5");
$divis gmp_init("2");
# number of decimals
$precision 2;

$shift gmp_pow("10"$precision);
$quot gmp_mul($quot$shift);

$res gmp_div_q($quot$divis);
$repr gmp_strval($res);
$dotpos strlen($repr) - $precision;
$repr substr($repr0$dotpos) . "." substr($repr$dotpos);

echo 
"Number is $repr";
$res gmp_init($repr);

Multiply the quotient by 10^p (p = precision), perform the integer division, convert to string, insert the decimal point, convert back to GMP.

Can't you do everything of that iside GMP (resources) ?
It will be much faster than operating on strings.

Moving the decimal point should be also possible inside GMP.

Pages: [1] 2 3 4 5 »  All
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!