Bitcoin Forum
January 25, 2020, 02:56:03 AM *
News: Latest Bitcoin Core release: 0.19.0.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: 1 2 3 4 5 [All]
  Print  
Author Topic: why JSON RPC values not use INT64 instead of float string?  (Read 18214 times)
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 03, 2011, 12:49:34 PM
 #1

This would solve a bunch of problems.

Why not just return the int64 and let the client cast it to a float & divide 10^8 for display?

Currently since the JSON RPC returns floats, any library you use will return the values as floats, not strings. So to get the int64 value you need to multiply the float by 10^8 and cast to an int for internal usage.

Also by returning the value as int64, it will be enforcing good practice on clients instead of them unwittingly using floats.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 03, 2011, 01:15:26 PM
 #2

Because JavaScript doesn't have a 64-bit integer type (all Numbers in JavaScript are double-precision floating point).

How often do you get the chance to work on a potentially world-changing project?
ribuck
Donator
Hero Member
*
Offline Offline

Activity: 826
Merit: 1006


View Profile
March 03, 2011, 01:23:16 PM
 #3

all Numbers in JavaScript are double-precision floating point
Fortunately for Bitcoin, double-precision floating point represents integers exactly up to 9,007,199,254,740,992 which is above the number of bitcoin base units i.e. 2,100,000,000,000,000.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 03, 2011, 01:33:48 PM
 #4

Can we not beat this dead horse?

I think there are MUCH more important things to work on / worry about than whether or not "send 1 BTC" is expressed as "sendtoaddress FOO 1.00" or "sendtoaddress FOO 100000000" in the JSON-RPC.

How about we (I'll start) write a "Proper Money Handling" page for the Wiki that discusses the issue and gives code example of how to convert to/from JSON double-precision floating point and 64-bit integer?

How often do you get the chance to work on a potentially world-changing project?
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 03, 2011, 02:17:12 PM
Last edit: March 03, 2011, 02:34:54 PM by genjix
 #5

Python uses floats for it's JSON library. Herein lies the problems.

$ python
>>> import json
>>> json.dumps(10.001)
'10.000999999999999'
>>> json.loads('{"blaa": 0.333331}')
{u'blaa': 0.33333099999999999}
>>> type(json.loads('{"blaa": 0.333331}')['blaa'])
<type 'float'>

This is unacceptable.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 03, 2011, 02:33:21 PM
 #6

Wiki page created:  https://en.bitcoin.it/wiki/Proper_Money_Handling_(JSON-RPC)

genjix:  You should be calling json.loads(..., parse_float=decimal.Decimal) and use a custom JSON encoder class to convert decimals to JSON strings with no loss of precision...

How often do you get the chance to work on a potentially world-changing project?
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 03, 2011, 02:43:17 PM
 #7

Yep, just noticed that.

But then the Python JSON-RPC library does not and I couldn't find one for the PHP RPC library either.

Code:
>>> from jsonrpc import ServiceProxy
>>> access = ServiceProxy("http://user:password@127.0.0.1:8332")
>>> type(access.getbalance())
<type 'float'>

BTW, in that wiki page why did you use those lambdas instead of simply using decimal.Decimal? Multiplying by e8 will cause everything like version numbers or difficulty to be multiplied.
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1022



View Profile
March 03, 2011, 02:54:55 PM
 #8

Currently since the JSON RPC returns floats, any library you use will return the values as floats, not strings. So to get the int64 value you need to multiply the float by 10^8 and cast to an int for internal usage.
It's one of the many JSON-RPC design flaws. Instead of trying to fix it, I have moved on to working on a new protocol to address all the problems: https://www.bitcoin.org/smf/index.php?topic=3757.0

Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 03, 2011, 03:26:16 PM
 #9

genjix: here is how to do it right in Python2.6 :

Code:

import decimal
import json

# From http://stackoverflow.com/questions/1960516/python-json-serialize-a-decimal-object
class DecimalEncoder(json.JSONEncoder):
  def _iterencode(self, o, markers=None):
    if isinstance(o, decimal.Decimal):
      return (str(o) for o in [o])
    return super(DecimalEncoder, self)._iterencode(o, markers)

decimal.setcontext(decimal.Context(prec=8))

print json.dumps(decimal.Decimal('10.001'), cls=DecimalEncoder)
print json.dumps({ "decimal" : decimal.Decimal('1.1'), "float" : 1.1, "string" : "1.1" }, cls=DecimalEncoder)
print json.loads('{"blaa": 0.333331}', parse_float=decimal.Decimal)
Produces output:
Code:
10.001
{"decimal": 1.1, "float": 1.1000000000000001, "string": "1.1"}
{u'blaa': Decimal('0.333331')}

Note that EVEN IF YOU PASSED THE 'WRONG' strings to Bitcoin, Bitcoin would do the right thing. That is, these two are equivalent once they are parsed by bitcoin:
Code:
sendtoaddress FOO 10.000999999999999
sendtoaddress FOO 10.001
... because bitcoin does proper rounding.

On the bitcoin side, this is a non-issue.  And if code on the other end of the JSON-RPC connection does the wrong thing (truncates values like 10.000999999999999 instead of rounding them to the nearest 8'th decimal place) then that's a bug in that code.

How often do you get the chance to work on a potentially world-changing project?
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 03, 2011, 03:34:56 PM
 #10

That's code to parse JSONs. There's a Python library to work with JSON-RPC. There's also a PHP library to do JSON-RPC. Both use floats.

Solution A: everybody that wishes to interface with Bitcoin in Python/PHP must write their own (potentially buggy) RPC http code because the default libs for those languages uses floats.
Solution B: a small change is made to Bitcoin.

B is a much better solution Smiley
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 03, 2011, 08:06:34 PM
 #11

That's code to parse JSONs. There's a Python library to work with JSON-RPC.
Huh?  See that 'import json' statement at the top?  That would be the standard (as of python 2.6) JSON parsing library.

The code I posted tells the standard JSON parsing library to read JSON Numbers as Decimal.  If you are doing monetary calculations in python, then you should be using Decimal.  That is what it is for.

How often do you get the chance to work on a potentially world-changing project?
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 03, 2011, 09:37:28 PM
 #12

That's code to parse JSONs. There's a Python library to work with JSON-RPC.
Huh?  See that 'import json' statement at the top?  That would be the standard (as of python 2.6) JSON parsing library.

The code I posted tells the standard JSON parsing library to read JSON Numbers as Decimal.  If you are doing monetary calculations in python, then you should be using Decimal.  That is what it is for.


Not json but json-rpc as recommended by json-rpc themselves.
http://json-rpc.org/wiki/python-json-rpc
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 11, 2011, 08:53:38 PM
 #13

This is stubborness... Now I'm trying to integrate Bitcoin into a website but the JSON-RPC library only returns floats.

Quote
Floating point numbers have limited precision. Although it depends on the system, PHP typically uses the IEEE 754 double precision format, which will give a maximum relative error due to rounding in the order of 1.11e-16. Non elementary arithmetic operations may give larger errors, and, of course, error progragation must be considered when several operations are compounded.

Additionally, rational numbers that are exactly representable as floating point numbers in base 10, like 0.1 or 0.7, do not have an exact representation as floating point numbers in base 2, which is used internally, no matter the size of the mantissa. Hence, they cannot be converted into their internal binary counterparts without a small loss of precision. This can lead to confusing results: for example, floor((0.1+0.7)*10) will usually return 7 instead of the expected 8, since the internal representation will be something like 7.9999999999999991118....

So never trust floating number results to the last digit, and never compare floating point numbers for equality. If higher precision is necessary, the arbitrary precision math functions and gmp functions are available.

Why can't Bitcoin return strings?

You're deliberately breaking with all the languages (python JSON-RPC library, PHP JSON-RPC and Perl JSON-RPC). Using floats anywhere in financial transactions is unacceptable.

php's json_decode DOESN'T support anyway to return floats as strings. The option doesn't exist.
PHP solution: write/maintain my own JSON parser.

Neither does Python's JSON-RPC. Have to write my own JSON-RPC lib using json module instead of using the one that already exists.

Same for Perl.

Really, why is it such a big deal? Bitcoin is broken and this needs fixing.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 11, 2011, 09:17:22 PM
 #14

PHP solution: write/maintain my own JSON parser.

Why can't you just multiply the numbers by 1.0e8 and then round to the nearest integer?  That integer WILL ALWAYS BE EXACTLY RIGHT (assuming you're not running PHP on some really weird hardware).

According to the PHP manual:
Quote
The size of a float is platform-dependent, although a maximum of ~1.8e308 with a precision of roughly 14 decimal digits is a common value (the 64 bit IEEE format).

I added a Python JSON-RPC library example on the Proper Money Handling wiki page.

How often do you get the chance to work on a potentially world-changing project?
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 11, 2011, 09:19:46 PM
 #15

This is stubborness... Now I'm trying to integrate Bitcoin into a website but the JSON-RPC library only returns floats.

Why can't Bitcoin return strings?

+ 1
Floats are a royal pain in the ass. Every bank application programmer will probably tell you that.
Especially RPC-like services should operate on strings - it makes a lot of stuff easier and allows infinite precision.

According to the PHP manual:
Quote
The size of a float is platform-dependent, although a maximum of ~1.8e308 with a precision of roughly 14 decimal digits is a common value (the 64 bit IEEE format).

For operations of extreme precision, PHP has many sets of mathematical libraries which also operate on strings, not floats.

For example, BC-Math or GMP.

http://php.net/manual/en/book.bc.php
http://www.php.net/manual/en/book.gmp.php

ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 11, 2011, 09:33:09 PM
 #16

One more thing:

Additionally I think that in PHP, result of float <-> integer calculations may differ on 32Bit & 64Bit platforms, however i may be wrong (and I am too lazy to check with Google).

There may be some bugs or php-specific "features" involved.

genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 11, 2011, 10:00:55 PM
 #17

ShadowOfHarbringer: Do you know how to divide numbers in PHP GMP and obtain a decimal number (not quotient + remainder)?
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 11, 2011, 10:12:47 PM
Last edit: March 11, 2011, 10:52:20 PM by ShadowOfHarbringer
 #18

ShadowOfHarbringer: Do you know how to divide numbers in PHP GMP and obtain a decimal number (not quotient + remainder)?

Well it should be fairly easy using GMP, however i have never done that before.

You will probably need to study usage of following functions:

http://www.php.net/manual/en/function.gmp-init.php
http://www.php.net/manual/en/function.gmp-div-q.php
http://www.php.net/manual/en/function.gmp-strval.php

Example from PHP.net:

Code:
<?php

$div1 
gmp_div_q("100""5");
echo 
gmp_strval($div1) . "\n";

$div2 gmp_div_q("1""3");
echo 
gmp_strval($div2) . "\n";

$div3 gmp_div_q("1""3"GMP_ROUND_PLUSINF);
echo 
gmp_strval($div3) . "\n";

$div4 gmp_div_q("-1""4"GMP_ROUND_PLUSINF);
echo 
gmp_strval($div4) . "\n";

$div5 gmp_div_q("-1""4"GMP_ROUND_MINUSINF);
echo 
gmp_strval($div5) . "\n";

And result:

Code:
20
0
1
0
-1

genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 11, 2011, 10:35:33 PM
 #19

Yeah those are integer values,

var_dump(gmp_strval(gmp_div_q(gmp_init("5"), gmp_init("2"))));

output:
string(1) "2" 

Here's my solution, http://codepad.viper-7.com/tbZ9oD
Code:
<?php

$quot 
gmp_init("5");
$divis gmp_init("2");
# number of decimals
$precision 2;

$shift gmp_pow("10"$precision);
$quot gmp_mul($quot$shift);

$res gmp_div_q($quot$divis);
$repr gmp_strval($res);
$dotpos strlen($repr) - $precision;
$repr substr($repr0$dotpos) . "." substr($repr$dotpos);

echo 
"Number is $repr";
$res gmp_init($repr);

Multiply the quotient by 10^p (p = precision), perform the integer division, convert to string, insert the decimal point, convert back to GMP.
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 11, 2011, 10:51:31 PM
 #20

Yeah those are integer values,

var_dump(gmp_strval(gmp_div_q(gmp_init("5"), gmp_init("2"))));

output:
string(1) "2" 

Here's my solution, http://codepad.viper-7.com/tbZ9oD
Code:
<?php

$quot 
gmp_init("5");
$divis gmp_init("2");
# number of decimals
$precision 2;

$shift gmp_pow("10"$precision);
$quot gmp_mul($quot$shift);

$res gmp_div_q($quot$divis);
$repr gmp_strval($res);
$dotpos strlen($repr) - $precision;
$repr substr($repr0$dotpos) . "." substr($repr$dotpos);

echo 
"Number is $repr";
$res gmp_init($repr);

Multiply the quotient by 10^p (p = precision), perform the integer division, convert to string, insert the decimal point, convert back to GMP.

Can't you do everything of that iside GMP (resources) ?
It will be much faster than operating on strings.

Moving the decimal point should be also possible inside GMP.

error
Hero Member
*****
Offline Offline

Activity: 588
Merit: 500



View Profile
March 11, 2011, 10:55:52 PM
 #21

You can do everything in GMP except decode the JSON, which is what I think OP was complaining about.

3KzNGwzRZ6SimWuFAgh4TnXzHpruHMZmV8
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 11, 2011, 10:57:50 PM
 #22

GMP is an integer only library,

http://codepad.viper-7.com/pj58SK
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 11, 2011, 10:59:19 PM
 #23

You can do everything in GMP except decode the JSON, which is what I think OP was complaining about.

This is a misunderstanding.
I only meant do the calculations in GMP, and only convert from/to string on input/output .

GMP is an integer only library,

http://codepad.viper-7.com/pj58SK

Ah sorry then, my mistake.

genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 11, 2011, 11:00:04 PM
 #24

You can do everything in GMP except decode the JSON, which is what I think OP was complaining about.

No, I was complaining about the fact that the Bitcoin JSON-RPC Api uses numbers (that nearly all libraries convert to floats and have no option to change) when using strings would only be minorly impactful.
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 11, 2011, 11:07:10 PM
 #25

OK, i have found the base for my allegations about dangers of using floats:

Warning
Floating point precision

Floating point numbers have limited precision. Although it depends on the system, PHP typically uses the IEEE 754 double precision format, which will give a maximum relative error due to rounding in the order of 1.11e-16. Non elementary arithmetic operations may give larger errors, and, of course, error progragation must be considered when several operations are compounded.

Additionally, rational numbers that are exactly representable as floating point numbers in base 10, like 0.1 or 0.7, do not have an exact representation as floating point numbers in base 2, which is used internally, no matter the size of the mantissa. Hence, they cannot be converted into their internal binary counterparts without a small loss of precision. This can lead to confusing results: for example, floor((0.1+0.7)*10) will usually return 7 instead of the expected 8, since the internal representation will be something like 7.9999999999999991118....

So never trust floating number results to the last digit, and never compare floating point numbers for equality. If higher precision is necessary, the arbitrary precision math functions and gmp functions are available.

If this doesn't shout "floats are bad", then i don't know what else to say about that.

So everything i said was true. "Precision depends on the system". So it may be different on 32bit and 64bit systems, and perhaps even on windows / Linux / different types of Unix/BSD.

ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 11, 2011, 11:15:04 PM
 #26

GMP is an integer only library,

http://codepad.viper-7.com/pj58SK

You can still use bcmath, may be faster than working strings:

http://www.php.net/manual/en/book.bc.php

Also, somebody on the forums told me that there are many other libraries supporting arbitrary precision mathematics for PHP, but i don't know them myself because i never needed that.

Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1022



View Profile
March 12, 2011, 02:05:17 AM
 #27

This isn't the only (nor biggest) problem with JSON-RPC. Help create a new standard protocol fixing this and other problems: https://en.bitcoin.it/wiki/Wallet_protocol

genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 12, 2011, 04:09:03 AM
 #28

Code:
import urllib
import decimal
import json

class JSONRPCException(Exception):
    def __init__(self, rpcError):
        Exception.__init__(self)
        self.error = rpcError
       
class ServiceProxy(object):
    def __init__(self, serviceURL, serviceName=None):
        self.__serviceURL = serviceURL
        self.__serviceName = serviceName

    def __getattr__(self, name):
        if self.__serviceName != None:
            name = "%s.%s" % (self.__serviceName, name)
        return ServiceProxy(self.__serviceURL, name)

    def __call__(self, *args):
         postdata = json.dumps({"method": self.__serviceName, 'params': args, 'id':'jsonrpc'})
         respdata = urllib.urlopen(self.__serviceURL, postdata).read()
         resp = json.loads(respdata, parse_float=decimal.Decimal)
         if resp['error'] != None:
             raise JSONRPCException(resp['error'])
         else:
             return resp['result']         

This should work in Python is anybody ever needs it. I've updated the wiki,
https://en.bitcoin.it/wiki/API_tutorial_%28JSON-RPC%29#Python

It's based off the old lib.
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 12, 2011, 05:52:18 AM
Last edit: March 12, 2011, 02:10:54 PM by genjix
 #29

Anyone that prefers a saner API (because they're using PHP) should check out my branch,
https://github.com/genjix/bitcoin/tree/strrpc

Values are returned as int64 strings.
Code:
function numstr_to_internal($numstr)
{
    return bcmul($numstr, pow(10, 8), 0);
}
function internal_to_numstr($num, $precision=8)
{
    $repr = gmp_strval($num);
    $repr = bcdiv($repr, pow(10, 8), $precision);
    # now tidy output...
    # trim trailing 0s
    $repr = rtrim($repr, '0');
    # and a trailing . if it exists
    $repr = rtrim($repr, '.');
    return $repr;
}

Those are the 2 functions I use to convert from internal values to display/user input values (numstr).

I updated the wiki, https://en.bitcoin.it/wiki/API_tutorial_%28JSON-RPC%29#PHP
mizerydearia
Hero Member
*****
Offline Offline

Activity: 574
Merit: 504



View Profile
March 12, 2011, 05:59:10 AM
 #30

Also see http://meta.witcoin.com/p/348/Transactions---float-conversion-issue
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 12, 2011, 03:24:51 PM
Last edit: March 14, 2011, 11:49:04 AM by gavinandresen
 #31

Is there a PHP implementation that does not use double-precision floating point?
After doing a little googling I couldn't figure out the answer to that.  I will be MUCH more sympathetic to changing the JSON-RPC api if there is.

And mizerydearia:  re: the witcoin issue:  You say:
"I see the transaction as 0.94 However, http://json-rpc.org/ retrieves the data as 0.93999999999999994671"

So why when you display that value are you truncating it instead of rounding it to 8 decimal places?
For example:
Quote
> php -r "printf('%.8f', 0.94);"    #CORRECT
0.94000000
> php -r "printf('%.16f', 0.94);"   #WRONG
0.9399999999999999

... or to convert to an integer-number-of-base-unit:
Quote
> php -r '$val=0.94; printf("%d", round(1e8*$val));'
94000000

All of that assume that your php support double-precision floating point, which brings me back to my question:  are there any php implementations that do not?

How often do you get the chance to work on a potentially world-changing project?
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 12, 2011, 03:57:26 PM
 #32

Code:
<?php
$json 
'{"a":1,"b":2,"c":3,"d":4.08,"e":5}';
var_dump(json_decode($json));
?>


Output:

Code:
object(stdClass)#1 (5) { ["a"]=> int(1) ["b"]=> int(2) ["c"]=> int(3) ["d"]=> float(4.08) ["e"]=> int(5) }

Why don't why put in a new number like

nAPIVersion and call the current API 0.

Then we can make nAPIVersion 0.5 supporting the int64 with appended .0 as luke-jr has done in his gitorious branch.

Then I want to make an API version 1.0 which puts in a strict naming convention for the methods (getblaa, action instead of the current messy names) and use namespaces.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 12, 2011, 04:12:07 PM
 #33

Then we can make nAPIVersion 0.5 supporting the int64 with appended .0 as luke-jr has done in his gitorious branch.

That's just dumb.

If the problem is jsonrpc-supporting-environments that don't support double-precision floats, then multiplying and slapping a zero on the end won't fix the problem-- you'll just either lose precision as it gets squeezed into a 32-bit float or get a 32-bit-integer-overflow error.


How often do you get the chance to work on a potentially world-changing project?
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 12, 2011, 04:37:03 PM
 #34

You're right. In that case, it seems that going with strings is the only option.

Still it might be a good idea to have API versions (bitcoin.getinfo()["rpcversion"]) and take an optional -rpcversion command line argument. When you create a new API, then the old one is deprecated for at least one release cycle, allowing compatibility.

When a new RPC version is created (lets call new version 1 and old version 0 for example) then:

bitcoind
The RPC version (0) in this release is DEPRECATED! Re-run bitcoind -rpcversion=1 for the new version. RPC changes can be found at http://bitcoin.org/rpcapi.html
bitcoind stop
bitcoind -rpcversion=1
bitcoind getinfo
...
"rpcversion": 1
...

The policy would be to deprecate only when the API becomes backwards incompatible. Otherwise it's silently upgraded and becomes the default.
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 12, 2011, 05:29:31 PM
 #35

You're right. In that case, it seems that going with strings is the only option.

All serious institutions (like banks) never use floats for currency calculations because they know the dangers.
If we want to be seen as "serious", float is not an option.

genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 12, 2011, 05:31:43 PM
 #36

You're right. In that case, it seems that going with strings is the only option.

All serious institutions (like banks) never use floats for currency calculations because they know the dangers.
If we want to be seen as "serious", float is not an option.

Yep but it's a problem with the json decoding in PHP/Perl/Python. The actual precision for the number in a JSON is accurate AFAIK.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 12, 2011, 05:35:32 PM
 #37

All serious institutions (like banks) never use floats for currency calculations because they know the dangers.
If we want to be seen as "serious", float is not an option.

You know, I was looking at the PayPal payment API yesterday, and $1.01 is sent as... 1.01

"transmitting" != "calculations"

How often do you get the chance to work on a potentially world-changing project?
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 12, 2011, 06:24:26 PM
 #38

All serious institutions (like banks) never use floats for currency calculations because they know the dangers.
If we want to be seen as "serious", float is not an option.

You know, I was looking at the PayPal payment API yesterday, and $1.01 is sent as... 1.01

"transmitting" != "calculations"

Of course you're right.
I was merely stating the fact.

j16sdiz
Newbie
*
Offline Offline

Activity: 37
Merit: 0


View Profile
March 14, 2011, 07:50:25 AM
 #39

....
Quote
> php -r "printf('%.8f', 0.94);"    #CORRECT
0.94000000
> php -r "printf('%.16f', 0.94);"   #WRONG
0.9399999999999999
....


It does not work that way ..  printf("%.8f") behaviour is inconsistent across platform. If you don't believe, try

Code:
printf("%.2f", -999.04)

-- some platform give  -999.03, some give -999.04.

Of course, you can fix this.... in your code. (But never expect all 3rd party apps do it right)
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 14, 2011, 11:55:07 AM
 #40

It does not work that way ..  printf("%.8f") behaviour is inconsistent across platform. If you don't believe, try

Code:
printf("%.2f", -999.04)

-- some platform give  -999.03, some give -999.04.

I tried it... got -999.04 on my Mac and Linux machines.  What platform gives the wrong answer?

How often do you get the chance to work on a potentially world-changing project?
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 14, 2011, 04:10:18 PM
Last edit: March 14, 2011, 05:41:31 PM by genjix
 #41

Try this:

Code:
<?php
$json 
'{"e":5.01}';
var_dump(json_decode($json));
?>


Output:

Code:
object(stdClass)#1 (1) { ["e"]=> float(5.01) }

ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 14, 2011, 05:10:25 PM
 #42

My result:

Code:
object(stdClass)#1 (1) {
  ["e"]=>
  float(5.01)
}

Checked on 2 separate PHP Versions : 5.3.5 and 5.2.14

Different versions, different results ?

Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 14, 2011, 05:13:56 PM
 #43

PHP 5.3.3. on my mac gives:
Code:
object(stdClass)#1 (1) {
  ["e"]=>
  float(5.01)
}

Please be specific about what platform you're running on, what version of PHP are you running, are you running with a standard php.ini or have you tweaked it, etc.

How often do you get the chance to work on a potentially world-changing project?
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 14, 2011, 05:40:59 PM
 #44

sorry it was a typo. supposed to say 5.01

The point was that PHP is casting to floats in their JSON decoder and there's no option to use doubles or strings.
BitterTea
Sr. Member
****
Offline Offline

Activity: 294
Merit: 250



View Profile
March 14, 2011, 06:15:15 PM
 #45

The PHP manual says that float=double.

Wouldn't gavin's solution of "float * 1e8" work?
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 14, 2011, 07:26:08 PM
 #46

sorry it was a typo. supposed to say 5.01

Probably a buggy version of PHP.

Anyway, on a lot of hosting providers have old versions of PHP installed (there are older versions in many distros' repositories), so it may be dangerous to use floats.
This is one of the reasons why i said initially that floats are dangerous as hell.

Alex Beckenham
Full Member
***
Offline Offline

Activity: 154
Merit: 100


View Profile
March 30, 2011, 04:14:18 PM
 #47

Anyone that prefers a saner API (because they're using PHP) should check out my branch,
https://github.com/genjix/bitcoin/tree/strrpc

Values are returned as int64 strings.
Code:
function numstr_to_internal($numstr)
{
    return bcmul($numstr, pow(10, 8), 0);
}
function internal_to_numstr($num, $precision=8)
{
    $repr = gmp_strval($num);
    $repr = bcdiv($repr, pow(10, 8), $precision);
    # now tidy output...
    # trim trailing 0s
    $repr = rtrim($repr, '0');
    # and a trailing . if it exists
    $repr = rtrim($repr, '.');
    return $repr;
}

Those are the 2 functions I use to convert from internal values to display/user input values (numstr).

I updated the wiki, https://en.bitcoin.it/wiki/API_tutorial_%28JSON-RPC%29#PHP


Reading this at the moment... https://en.bitcoin.it/wiki/API_reference_(JSON-RPC)

Under 'Precision' you wrote:

You will need to get a saner branch and compile it.
https://github.com/genjix/bitcoin/tree/strrpc

Does 'saner branch' mean get a good version of GMP? I don't quite get what the phrase refers to.

Nefario
Hero Member
*****
Offline Offline

Activity: 602
Merit: 502


GLBSE Support support@glbse.com


View Profile WWW
March 30, 2011, 04:29:53 PM
 #48

Saner branch is a genjix fork of bitcoind that returns int64 strings instead of floats or doubles in the json-api

This prevents any rounding errors associated with floats.

PGP key id at pgp.mit.edu 0xA68F4B7C

To get help and support for GLBSE please email support@glbse.com
Alex Beckenham
Full Member
***
Offline Offline

Activity: 154
Merit: 100


View Profile
March 30, 2011, 04:34:54 PM
 #49

Saner branch is a genjix fork of bitcoind that returns int64 strings instead of floats or doubles in the json-api

This prevents any rounding errors associated with floats.

Cheers, still getting used to terms such as 'fork' and 'branch'.

So is this still the best way to go about it for a 32-bit PHP install? (Change seems to happen quickly in this community at the moment).


Nefario
Hero Member
*****
Offline Offline

Activity: 602
Merit: 502


GLBSE Support support@glbse.com


View Profile WWW
March 30, 2011, 04:47:33 PM
 #50

I don't know if it's the best way, I'm using ruby, and the whole float thing is a pain in my ass. I'm working with genjix anyway so I use his fork and avoid the entire float thing altogether.

PGP key id at pgp.mit.edu 0xA68F4B7C

To get help and support for GLBSE please email support@glbse.com
Alex Beckenham
Full Member
***
Offline Offline

Activity: 154
Merit: 100


View Profile
March 30, 2011, 04:53:42 PM
 #51

I don't know if it's the best way, I'm using ruby, and the whole float thing is a pain in my ass. I'm working with genjix anyway so I use his fork and avoid the entire float thing altogether.

Okay thanks... It looks like my host is able to up my PHP install to 64bit anyway.

Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 30, 2011, 05:11:31 PM
 #52

Can you'all educate me about these mythical rounding errors that require using GMP?

I can see, maybe, if you're computing interest down to the penny on a 30-year mortgage you might conceivably be off by a penny if you use 64-bit floats instead of 64-bit integers, although even there you're going to have to think hard about rounding as you get integer remainders.

And I can see being really careful if you're writing a bitcoin exchange site or bitcoin bank that deals in thousands of internal transactions that must all balance exactly.

But for the typical PHP website that is just going to add up 10 items in a shopping cart using plain-old PHP Numbers will be just fine.   I don't see PayPal recommending that PHP users of it's APIs install GMP.  Recommending that any website dealing with bitcoins compile genjix' fork and use GMP is a really good way to ensure that nobody accepts bitcoins.


How often do you get the chance to work on a potentially world-changing project?
burtyb
Newbie
*
Offline Offline

Activity: 45
Merit: 0



View Profile WWW
March 30, 2011, 05:36:57 PM
 #53

I've seen problems when porting C test functions to ARM a few years ago, you definitely can't trust a floats to behave the same on all platforms.
Nefario
Hero Member
*****
Offline Offline

Activity: 602
Merit: 502


GLBSE Support support@glbse.com


View Profile WWW
March 30, 2011, 05:55:11 PM
 #54

Can you'all educate me about these mythical rounding errors that require using GMP?

Recommending that any website dealing with bitcoins compile genjix' fork and use GMP is a really good way to ensure that nobody accepts bitcoins.



Whoa there cowboy, I wasn't recommending anything, I just ansered a question, what was the genjix fork, and when asked what the best thing to do simply said what I was doing, and why.

I'm working with genjix on my project so it's not an issue for me if I use his fork, but I can't say the same for anyone else.

PGP key id at pgp.mit.edu 0xA68F4B7C

To get help and support for GLBSE please email support@glbse.com
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1582
Merit: 1006


View Profile
March 30, 2011, 05:57:26 PM
 #55

I've seen problems when porting C test functions to ARM a few years ago, you definitely can't trust a floats to behave the same on all platforms.

Actually, as long as you make sure all your IEEE compliance options are turned on...

Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 30, 2011, 06:16:15 PM
 #56

I've seen problems when porting C test functions to ARM a few years ago, you definitely can't trust a floats to behave the same on all platforms.

Actually, as long as you make sure all your IEEE compliance options are turned on...

...but probably sometimes for some reason they won't be turned on...

ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 30, 2011, 06:17:55 PM
 #57

But for the typical PHP website that is just going to add up 10 items in a shopping cart using plain-old PHP Numbers will be just fine.   I don't see PayPal recommending that PHP users of it's APIs install GMP.  Recommending that any website dealing with bitcoins compile genjix' fork and use GMP is a really good way to ensure that nobody accepts bitcoins.

Can't we just have 2 separate APIs - one float and one string in the official client ? Or maybe perhaps one API with "string mode" and "float mode" ?

BTW, are the floats really so necessary and important that you defend them ?
No offense, but while reading this topic it seems that almost nobody here likes floats except you.

Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 30, 2011, 06:22:16 PM
 #58

Apologies to Nefario, I was reacting to the wiki pages written by genjix on how to use PHP with bitcoind that started with:

+ First, compile my fork.
+ Next, install the GMP and BCMath libraries...

And why do I defend floats:  because simple things should be simple.  Using GMP/BCMATH is overkill for 98% of what bitcoin JSON-RPC users will be doing.

And because certain people keep beating this dead horse.  I have said that I am PERFECTLY WILLING to support strings in the JSON-RPC interface if somebody can demonstrate to me someplace where it is actually a real problem (that isn't trivially solved using something like round(value*1e8+0.5) or printf("%.08", value)).


How often do you get the chance to work on a potentially world-changing project?
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1582
Merit: 1006


View Profile
March 30, 2011, 06:22:43 PM
 #59

If we want to be taken remotely seriously by the banking and merchant communities, floats need to go.  Whether the end result is "1000000000" or "10.00000000" is largely cosmetic.  But I do agree that -- long term -- JSON's "number" and internal floats fall short of what is needed for proper money handling software.

But....    it just hurts to change the API right now, because we will see very little payback for a large amount of pain.


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1582
Merit: 1006


View Profile
March 30, 2011, 06:26:13 PM
 #60

And why do I defend floats:  because simple things should be simple.  Using GMP/BCMATH is overkill for 98% of what bitcoin JSON-RPC users will be doing.

As an aside:   Using GMP (or similar) is overkill for two big reasons:

  • We already link with and use OpenSSL's bignum (CBigNum), so there is no need for an additional library for Big Maths
  • It's just silly to want GMP when we already handle money properly internally, as int64.  The only issues present are interface issues, and you don't need to link with GMP in order to display an int64 with a decimal point somewhere in the middle.

Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
Nefario
Hero Member
*****
Offline Offline

Activity: 602
Merit: 502


GLBSE Support support@glbse.com


View Profile WWW
March 30, 2011, 06:28:11 PM
 #61

The api doesn't need to be changed, we can just add an option to the command line that will specify int64 with float being the default.

Merge genjix's changes to be the ones used when int64 is passed in.

PGP key id at pgp.mit.edu 0xA68F4B7C

To get help and support for GLBSE please email support@glbse.com
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 30, 2011, 06:29:02 PM
 #62

If we want to be taken remotely seriously by the banking and merchant communities, floats need to go.

People keep claiming that, and yet I just did YET ANOTHER google search for "banking apis", clicked through to the Open Financial Exchange standard (from Microsoft and Quicken), and what do you know!  Money amounts look like floats:

Code:
          <STMTTRN>
              <TRNTYPE>CREDIT
              <DTPOSTED>20070315
              <DTUSER>20070315
              <TRNAMT>200.00
              <FITID>980315001
              <NAME>DEPOSIT
              <MEMO>automatic deposit
            </STMTTRN>
http://www.ofx.net/OFXExamplesPage/OFXExamples.aspx


How often do you get the chance to work on a potentially world-changing project?
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1582
Merit: 1006


View Profile
March 30, 2011, 06:30:04 PM
 #63

The api doesn't need to be changed, we can just add an option to the command line that will specify int64 with float being the default.

Thus creating a new, second parallel API, conditionally present or absent?  Ugh, no thanks.




Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1582
Merit: 1006


View Profile
March 30, 2011, 06:34:12 PM
 #64

People keep claiming that, and yet I just did YET ANOTHER google search for "banking apis", clicked through to the Open Financial Exchange standard (from Microsoft and Quicken), and what do you know!  Money amounts look like floats:

Don't assume a decimal (".") implies floating point.

Explanation by example:  Remember the difference in SQL between a FLOAT and DECIMAL.  Float is Approximate-number data type, which means that not all values in the data type range can be represented exactly.  Decimal/Numeric is Fixed-Precision data type, which means that all the values in the data type range can be represented exactly with precision and scale.

Anyone who deals with money demands DECIMAL-like behavior.  Bitcoin's internal use of int64 for values provides that fixed-precision, DECIMAL-like behavior.

Decimals (".") are common in monetary software.  Floating point data types are not.


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 30, 2011, 06:38:57 PM
 #65

All righty, I hereby state:

All money values in the bitcoin JSON-RPC interface Are and Shall Be treated as Decimal, with 8 digits of precision after the decimal point.

If you're writing a banking application in a language that doesn't support Decimal types from JSON, then you should pack up your bags and go home.

How often do you get the chance to work on a potentially world-changing project?
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1582
Merit: 1006


View Profile
March 30, 2011, 06:43:41 PM
 #66

If you're writing a banking application in a language that doesn't support Decimal types from JSON, then you should pack up your bags and go home.

Is there a language that does produce fixed-precision from JSON?

jansson (C) and python's json both produce float, from a JSON number containing "."


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
error
Hero Member
*****
Offline Offline

Activity: 588
Merit: 500



View Profile
March 30, 2011, 06:46:15 PM
 #67

The main problem is that floating point is not sufficient to accurately store a Bitcoin balance or to do simple math and achieve a correct answer. Discussions of this problem are all over the Internet. A slightly oversimplified example from a Python FAQ hints at the real problem.

Ultimately a Bitcoin balance isn't a floating point number, it's a fixed-point number, and such numbers should not be manipulated with floating point functions, ever, lest Bitcoins start showing up in unexpected wallets.

Oh, and some people need pictures, so here's one.


3KzNGwzRZ6SimWuFAgh4TnXzHpruHMZmV8
error
Hero Member
*****
Offline Offline

Activity: 588
Merit: 500



View Profile
March 30, 2011, 06:47:46 PM
 #68

If you're writing a banking application in a language that doesn't support Decimal types from JSON, then you should pack up your bags and go home.

Is there a language that does produce fixed-precision from JSON?

jansson (C) and python's json both produce float, from a JSON number containing "."

PHP will soon have an option to json_decode() to treat large numbers as strings, though this code hasn't yet been released.

3KzNGwzRZ6SimWuFAgh4TnXzHpruHMZmV8
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
March 30, 2011, 07:02:04 PM
 #69

And why do I defend floats:  because simple things should be simple.  Using GMP/BCMATH is overkill for 98% of what bitcoin JSON-RPC users will be doing.

As an aside:   Using GMP (or similar) is overkill for two big reasons:

  • We already link with and use OpenSSL's bignum (CBigNum), so there is no need for an additional library for Big Maths
  • It's just silly to want GMP when we already handle money properly internally, as int64.  The only issues present are interface issues, and you don't need to link with GMP in order to display an int64 with a decimal point somewhere in the middle.

My Bitcoin branch doesn't use GMP. It's simply a change to AmountFromValue, ValueFromAmount and other functions that weren't using those helper methods. They're referring to me using GMP in PHP which I use for a range of reasons. Namely PHP not always having 64 bit ints:
http://stackoverflow.com/questions/864058/how-to-have-64-bit-integer-on-php
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 30, 2011, 07:18:46 PM
 #70

The main problem is that floating point is not sufficient to accurately store a Bitcoin balance

An IEEE double-precision floating point number has 53 bits of precision, which IS sufficiently accurate to store a bitcoin balance.

Every single possible bitcoin value can be converted to and from an IEEE 64-bit float with no loss of precision.

I agree that if you're going to be performing lots of calculations on bitcoin values you need a Decimal type (and ClearCoin stores and uses python's decimal.Decimal(precision=8) for all bitcoin values)-- if you don't, floating point errors can accumulate and eventually cause you to gain or lose .00000001 of a coin.

But really the main problem with storing monetary values as any floating point type is you're likely to be embarrassed by mistakes like error's cash register receipt if you truncate values instead of rounding before printing.

How often do you get the chance to work on a potentially world-changing project?
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1022



View Profile
March 30, 2011, 07:24:27 PM
 #71

I agree that if you're going to be performing lots of calculations on bitcoin values you need a Decimal type
If you're going to be doing *anything* with Bitcoin values, you should be using an int64.

error
Hero Member
*****
Offline Offline

Activity: 588
Merit: 500



View Profile
March 30, 2011, 07:38:04 PM
 #72

The main problem is that floating point is not sufficient to accurately store a Bitcoin balance

An IEEE double-precision floating point number has 53 bits of precision, which IS sufficiently accurate to store a bitcoin balance.

Ha. I had to go check this for myself. Indeed, the maximum number of Bitcoins that can currently exist, 21000000.00000000 requires only 51 bits of data to store.

Eventually, perhaps not in our lifetimes, Bitcoin will be working with such "tiny" values as 0.00000001, and then it becomes a big issue. So I think it's better to deal with the issue now, so our kids don't have to. Cheesy

3KzNGwzRZ6SimWuFAgh4TnXzHpruHMZmV8
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 30, 2011, 08:13:27 PM
 #73

The api doesn't need to be changed, we can just add an option to the command line that will specify int64 with float being the default.

Thus creating a new, second parallel API, conditionally present or absent?  Ugh, no thanks.

Is adding a "string mode" creating a second API ? I don't think so. This is just a switch, how can you call this a second api ?
Also, an option for selecting the format of output data through XMLRPC/Command line/Whatever already exists in many modern applications.

jgarzik
Legendary
*
qt
Offline Offline

Activity: 1582
Merit: 1006


View Profile
March 30, 2011, 08:36:14 PM
 #74

Is adding a "string mode" creating a second API ? I don't think so. This is just a switch, how can you call this a second api ?

Because it changes the data format of every number in every RPC call.


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 30, 2011, 10:01:18 PM
 #75

Is adding a "string mode" creating a second API ? I don't think so. This is just a switch, how can you call this a second api ?

Because it changes the data format of every number in every RPC call.

Well, OK. You've got a point.
Still, no big deal.

After everybody starts using strings instead of floats (and they will, because floats are seen as VERY unprofessional in the industry), we may remove the old float mode completely.
However no hurry there, it may wait a few years.

BitterTea
Sr. Member
****
Offline Offline

Activity: 294
Merit: 250



View Profile
March 30, 2011, 10:15:23 PM
 #76

Well, OK. You've got a point.
Still, no big deal.

After everybody starts using strings instead of floats (and they will, because floats are seen as VERY unprofessional in the industry), we may remove the old float mode completely.
However no hurry there, it may wait a few years.

If this change was made, any code using the API must first determine the API version, and then handle numbers in two different ways. No thanks.
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1022



View Profile
March 31, 2011, 12:52:35 AM
 #77

No, they would just have their single "JSON-RPC value to raw bitcoins" function check the type of the argument and react appropriately. However, using strings is even MORE unprofessional than floats.

error
Hero Member
*****
Offline Offline

Activity: 588
Merit: 500



View Profile
March 31, 2011, 12:58:31 AM
 #78

No, they would just have their single "JSON-RPC value to raw bitcoins" function check the type of the argument and react appropriately. However, using strings is even MORE unprofessional than floats.

The language may not give you a choice in the matter. Hello?

3KzNGwzRZ6SimWuFAgh4TnXzHpruHMZmV8
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 31, 2011, 01:55:54 AM
 #79

Well, OK. You've got a point.
Still, no big deal.

After everybody starts using strings instead of floats (and they will, because floats are seen as VERY unprofessional in the industry), we may remove the old float mode completely.
However no hurry there, it may wait a few years.

If this change was made, any code using the API must first determine the API version, and then handle numbers in two different ways. No thanks.

We can use different names for parameters, like

SendCoinsString 100.00000000
Instead of
SendCoins 100.00000000

That would eleminate such problems forever.

If you want to operate on string, just add suffix "String" to every parameter/attribute name.

ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 31, 2011, 02:12:05 AM
 #80

Ok.... so Gavin asked me (many times before and now) for the proof that the banking/financial industry avoids floats.

And i have found it !!

http://en.wikipedia.org/wiki/Floating_point#Minimizing_the_effect_of_accuracy_problems

Quote
Binary floating-point arithmetic is at its best when it is simply being used to measure real-world quantities over a wide range of scales (such as the orbital period of a moon around Saturn or the mass of a proton), and at its worst when it is expected to model the interactions of quantities expressed as decimal strings that are expected to be exact. An example of the latter case is financial calculations. For this reason, financial software tends not to use a binary floating-point number representation.[7] The "decimal" data type of the C# and Python programming languages, and the IEEE 754-2008 decimal floating-point standard, are designed to avoid the problems of binary floating-point representations when applied to human-entered exact decimal values, and make the arithmetic always behave as expected when numbers are printed in decimal.

The [7] reference leads here:
http://speleotrove.com/decimal/

--------------
I think we can call the topic closed.
Let's go string baby !

BitterTea
Sr. Member
****
Offline Offline

Activity: 294
Merit: 250



View Profile
March 31, 2011, 02:13:17 AM
 #81

If you want to operate on string, just add suffix "String" to every parameter/attribute name.

It's not a matter of what you "want" to operate on, it's a matter of the API version, and the Bitcoin version, of the user. Any code using the API would have to detect that and modify its behavior.
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1582
Merit: 1006


View Profile
March 31, 2011, 02:25:15 AM
 #82

We can use different names for parameters, like

SendCoinsString 100.00000000
Instead of
SendCoins 100.00000000

That would eleminate such problems forever.

If you want to operate on string, just add suffix "String" to every parameter/attribute name.

Which is, again, a second API to support on top of the first.  Twice bugs, twice the pain.


Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
March 31, 2011, 12:04:07 PM
 #83

Ok.... so Gavin asked me (many times before and now) for the proof that the banking/financial industry avoids floats.

HOW MANY TIMES DO I HAVE TO YELL THIS?Huh??

Of COURSE you shouldn't use floats internally (unless you are doing something trivial like adding up items in a shopping cart).

We are talking about the JSON-RPC api.  Which is an api for communicating between bitcoin and other applications, in which all values are turned into strings.

So:  what are the best practices in the banking world for representing monetary values in strings?  As far as I can tell, the answer is "write them out as decimal values and convert them to Decimal() or integer as you read in or write out."

Which is exactly what Bitcoin does, and which is what I think we should recommend to people.

How often do you get the chance to work on a potentially world-changing project?
BitterTea
Sr. Member
****
Offline Offline

Activity: 294
Merit: 250



View Profile
March 31, 2011, 02:54:11 PM
 #84

So the difference mainly lies in that the banking world isn't using JSON to shuttle data back and forth.
ShadowOfHarbringer
Legendary
*
Offline Offline

Activity: 1470
Merit: 1003


Bringing Legendary Har® to you since 1952


View Profile
March 31, 2011, 03:28:27 PM
 #85

Ok.... so Gavin asked me (many times before and now) for the proof that the banking/financial industry avoids floats.

HOW MANY TIMES DO I HAVE TO YELL THIS?Huh??

Of COURSE you shouldn't use floats internally (unless you are doing something trivial like adding up items in a shopping cart).

We are talking about the JSON-RPC api.  Which is an api for communicating between bitcoin and other applications, in which all values are turned into strings.

Ok, my mistake.

So:  what are the best practices in the banking world for representing monetary values in strings?

Unfortunately, I do not posess this information.

error
Hero Member
*****
Offline Offline

Activity: 588
Merit: 500



View Profile
March 31, 2011, 11:55:38 PM
 #86

So the difference mainly lies in that the banking world isn't using JSON to shuttle data back and forth.

This is the heart of the issue. JSON is very weakly typed, while much or most financial software is written in strongly typed languages.

When a weakly typed language attempts to handle JSON, that large number, with or without a decimal point, may be converted into a floating-point number by the language. So, really, it's the language losing, but we have to somehow deal with it.

3KzNGwzRZ6SimWuFAgh4TnXzHpruHMZmV8
BitterTea
Sr. Member
****
Offline Offline

Activity: 294
Merit: 250



View Profile
April 01, 2011, 12:04:07 AM
 #87

So the difference mainly lies in that the banking world isn't using JSON to shuttle data back and forth.

This is the heart of the issue. JSON is very weakly typed, while much or most financial software is written in strongly typed languages.

When a weakly typed language attempts to handle JSON, that large number, with or without a decimal point, may be converted into a floating-point number by the language. So, really, it's the language losing, but we have to somehow deal with it.

Then I say in the short term, we deal with JSON's quirks and leave the number as a number (which some languages may treat as a float instead of decimal), but take this into account when designing Bitcoin RPC v2.
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1582
Merit: 1006


View Profile
April 01, 2011, 01:22:22 AM
 #88

FWIW, this is the python recipe, straight from Python documentation:

Code:
>>> import decimal
>>> json.loads('1.1', parse_float=decimal.Decimal)
Decimal('1.1')

Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1022



View Profile
April 01, 2011, 06:00:04 AM
 #89

Then I say in the short term, we deal with JSON's quirks and leave the number as a number (which some languages may treat as a float instead of decimal), but take this into account when designing Bitcoin RPC v2.
Exactly: https://en.bitcoin.it/wiki/Wallet_protocol

genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
April 01, 2011, 10:56:42 AM
 #90

FWIW, this is the python recipe, straight from Python documentation:

Code:
>>> import decimal
>>> json.loads('1.1', parse_float=decimal.Decimal)
Decimal('1.1')


Except Python's JSON-RPC has no support for that. You have to use my (non-standard) modified version:
https://en.bitcoin.it/wiki/API_reference_%28JSON-RPC%29#Python

I prefer wherever possible that people stick with official libraries.

Bitcoin now is effectively saying that all other languages are broken therefore we won't fix our API.

Oh but there's a hack you can use to workaround... It's mostly accurate to the 8th decimal place.

Not the point!

By exposing an API which nearly all languages interpret as floats, you are expressing a statement of intent that it's fine to use floats for clients (API users). It should be passed as strings (without the decimal point preferably) that users have to explicitly cast to mutable objects. Then they realise that since these values are in INT51 strings (since we can't use large ints in JSON), they'll hopefully understand to deal with the values using ints.

You're encouraging users to deal with floats ATM. Bad practice, and shouldn't be encouraged.

I am really paranoid about leakage, and for me to even touch a float when dealing with people's money is unacceptable. On Britcoin, there's a whole system of balances and checks at every stage of the various transactions to ensure consistency to all decimal places, and I would not want to throw a spanner in the works because of a tiny rounding error (which would throw up flags and warnings everywhere). Even hearing the words round(...) is like heresy.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1027


Chief Scientist


View Profile WWW
April 01, 2011, 11:58:01 AM
 #91

I am really paranoid about leakage, and for me to even touch a float when dealing with people's money is unacceptable. On Britcoin, there's a whole system of balances and checks at every stage of the various transactions to ensure consistency to all decimal places, and I would not want to throw a spanner in the works because of a tiny rounding error (which would throw up flags and warnings everywhere). Even hearing the words round(...) is like heresy.

Am I the only person here who looks at our documentation for how to use bitcoin from PHP and thinks "people are going to run away screaming" ?  See:  https://en.bitcoin.it/wiki/API_reference_(JSON-RPC)

I strongly believe you are making the common cases (simple shopping carts or "hold a bitcoin balance for a customer and let them spend it") much, much more complicated than necessary.

What do other people think?  I'd especially like to hear from people who have prior experience using PHP to implement shopping carts and other simple applications that deal with money.  Did you use BCMATH/GMP?

How often do you get the chance to work on a potentially world-changing project?
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1000


View Profile
April 01, 2011, 12:52:59 PM
 #92

Instead of removing all of this:

https://en.bitcoin.it/w/index.php?title=PHP_developer_intro&action=historysubmit&diff=6404&oldid=5737

Maybe we should add it to the bottom of the page, with an opening paragraph about how it isn't strictly necessary, but can help avoid problems. Because I did take time to write that for people, and doubtless others will find it useful, but not everyone may wish to use it Wink
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1022



View Profile
April 01, 2011, 03:35:06 PM
 #93

round(100000000 * amount) works fine in PHP. The new Wallet protocol will use ints properly, so this should be good enough.

jgarzik
Legendary
*
qt
Offline Offline

Activity: 1582
Merit: 1006


View Profile
April 02, 2011, 06:28:21 AM
 #94

See this thread for a python class that handles bitcoin's JSON-RPC, including proper decoding of JSON float-like numbers into python Decimal.

Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
bytemaster
Hero Member
*****
Offline Offline

Activity: 770
Merit: 555

BitShares


View Profile WWW
May 21, 2011, 05:57:02 AM
 #95

https://en.bitcoin.it/wiki/Original_Bitcoin_client/API_Calls_list says that the sendtoaddress and sendfrom "rounds to the nearest 0.01"   

I am guessing that this documentation is out of date?   Is the a new, "non original",  RPC documentation?

https://steemit.com  Blogging is the new Mining
wumpus
Hero Member
*****
qt
Offline Offline

Activity: 812
Merit: 1000

No Maps for These Territories


View Profile
May 21, 2011, 06:06:31 AM
 #96

OP is right in that financial software generally uses fixed point instead of floating point (the SQL/Python DECIMAL type). This is because you can guarantee that every picocent is accounted for, when handled correctly there will never be any loss due to precision issues.

Twitter had the same problem with their TweetIDs running out of the 53 bit range that JSON is able to represent with numbers. They opted with using strings:

http://blog.programmableweb.com/2010/10/19/the-twitter-id-shuffle-text-vs-numbers/

Bytemaster: yes, it is out of date. No rounding happens in AmountFromValue.

Bitcoin Core developer [PGP] Warning: For most, coin loss is a larger risk than coin theft. A disk can die any time. Regularly back up your wallet through FileBackup Wallet to an external storage or the (encrypted!) cloud. Use a separate offline wallet for storing larger amounts.
Pages: 1 2 3 4 5 [All]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!