Bitcoin Forum
June 22, 2021, 11:02:45 AM *
News: Latest Bitcoin Core release: 0.21.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: 1 2 [All]
  Print  
Author Topic: 21million BTC is just over 51 bits of precision...  (Read 4524 times)
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1085


Chief Scientist


View Profile WWW
January 29, 2011, 02:28:26 AM
 #1

luke-jr's patches got me thinking about whether or not passing double-precision values over the JSON-RPC api would ever cause problems.  I've convinced myself it isn't an issue:

JSON numbers are 64-bit double-precision floating point values, which have 53 bits of precision.

21 million bitcoins is actually 2,100,000,000,000,000 (2.1 quadrillion) of the smallest possible unit.  That's a bit over 251 -- you need just over 51 bits to represent them.

So, unless your JSON library is buggy, you should never run into rounding errors converting to/from JSON, even if you're sending 0.00000001 bitcoins.

How often do you get the chance to work on a potentially world-changing project?
1624359765
Hero Member
*
Offline Offline

Posts: 1624359765

View Profile Personal Message (Offline)

Ignore
1624359765
Reply with quote  #2

1624359765
Report to moderator
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
theymos
Administrator
Legendary
*
Offline Offline

Activity: 4144
Merit: 8490


View Profile
January 29, 2011, 07:27:13 AM
 #2

I've observed float-type errors (0.9999999, etc.) from bitcoind output (when using bitcoind as the RPC client), though this was in a very old release. Does Bitcoin still do this? (I patched my version to use strings for all numbers.)

1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD
mrb
Legendary
*
Offline Offline

Activity: 1512
Merit: 1019


View Profile WWW
January 29, 2011, 08:11:14 AM
 #3

gavinandresen: Murphy's Law states that some JSON libs are broken. Use integers, don't use floating point.

Smiley
ribuck
Donator
Hero Member
*
Offline Offline

Activity: 826
Merit: 1008


View Profile
January 29, 2011, 12:05:36 PM
 #4

Murphy's Law states that some JSON libs are broken. Use integers, don't use floating point.
If you have enough bits of floating point precision, you can represent integers safely and exactly. But you can't do the same for decimal fractions, due to the binary representation within the floating point value.

So if you represent the bitcoins as bitdust (2,100,000,000,000,000) rather than as coins (21,000,000.00000000) it should work unless the library is extremely broken (in which case it's likely to be extremely broken for integers too).

But ... what about when the whole world is using bitcoin and the precision gets extended beyond 51 bits?
Mike Hearn
Legendary
*
expert
Offline Offline

Activity: 1526
Merit: 1018


View Profile
January 29, 2011, 12:31:29 PM
 #5

I really don't think BitCoins should ever be represented as floats except for presentation to humans.

At some point it's quite possible that we'll want to move the decimal place around. If anything it'd be good to start thinking about how to organize that. Having the point hard-coded into APIs would cause issues.
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1026



View Profile
January 29, 2011, 03:17:45 PM
 #6

But ... what about when the whole world is using bitcoin and the precision gets extended beyond 51 bits?
As I understand it, precision cannot be extended without rewriting the whole system and starting from a new genesis (which might be a copy of the old "final block" I suppose). In any case, 51 bits of precision does seem to be enough, even for the whole world: Tonal BitCoin is based on 1,0000 (65,536 decimal) base units, which yields a maximum of 7,750,54.00 (32 billion decimal) TBC-- enough for 4 TBC per human, if every human (unlikely the entire world adopts BitCoin anyway) had the same number (which is itself unlikely too). From there, there are still 16 bits of precision left for division into 1,0000 (65,536 decimal) TBCᵇ pieces. Worst case, each of these TBCᵇ should still be small enough for any significant spending, and smaller divisions could be done with a separate micro-transaction network trading 1000:1 with these.

Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1085


Chief Scientist


View Profile WWW
January 29, 2011, 03:27:44 PM
 #7

ribuck:

Accuracy/precision is a red-herring unless you're treating numbers as strings, since JSON-RPC numbers ARE ALWAYS double-precision floating point numbers (according to the JavaScript/ECMAScript spec).  Bitcoin could send a number that looks like 2100000000000001, but the code that interprets that JSON-RPC number will convert it into an inexact double-precision floating-point equivalent.  And then the code that displays that number will have to decide how to round and format that inexact floating point number and display it to the user.

When we need more than 51 bits of precision (wouldn't THAT be a fantastic problem to have!), then we'd HAVE to send numbers as strings, and have the JavaScript (or whatever) on the other end feed them into a bignum-type library to handle them.

How often do you get the chance to work on a potentially world-changing project?
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1026



View Profile
January 29, 2011, 03:31:36 PM
 #8

JSON-RPC numbers ARE ALWAYS double-precision floating point numbers (according to the JavaScript/ECMAScript spec).
JSON is not defined by the ECMAScript standard. The actual JSON standard does not define any precision, or even that numbers are floating-point.

ribuck
Donator
Hero Member
*
Offline Offline

Activity: 826
Merit: 1008


View Profile
January 29, 2011, 06:21:41 PM
 #9

Bitcoin could send a number that looks like 2100000000000001, but the code that interprets that JSON-RPC number will convert it into an inexact double-precision floating-point equivalent.  And then the code that displays that number will have to decide how to round and format that inexact floating point number and display it to the user.

Gavin, my point is this:

The integer 2100000000000001 is within the range that is represented EXACTLY in double-precision floating point. On the other hand, the number 2100000.000000001 is not (and cannot be) represented exactly in double precision floating point (because it has a decimal fraction part that cannot be exactly represented in the binary encoding used for floating point).

The limit for exact integer representation in a double precision floating point number is 9,007,199,254,740,992 which is sufficient for Bitcoin's needs.

Perhaps you understood that I was saying that, and are pointing out that we are at the mercy of JSON libraries that might not correctly handle the conversion from integer to double-precision.

If the libraries are that buggy, it's pretty sad.
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1026



View Profile
January 29, 2011, 08:01:03 PM
 #10

The integer 2100000000000001 is within the range that is represented EXACTLY in double-precision floating point. On the other hand, the number 2100000.000000001 is not (and cannot be) represented exactly in double precision floating point (because it has a decimal fraction part that cannot be exactly represented in the binary encoding used for floating point).

The limit for exact integer representation in a double precision floating point number is 9,007,199,254,740,992 which is sufficient for Bitcoin's needs.
But so long as we are working in base units, there is no reason to use floating-point rather than a simple int64...

ribuck
Donator
Hero Member
*
Offline Offline

Activity: 826
Merit: 1008


View Profile
January 29, 2011, 08:36:17 PM
 #11

...there is no reason to use floating-point rather than a simple int64...
JSON-RPC doesn't support an Integer type. So it's floating-point or string.
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1026



View Profile
January 29, 2011, 08:52:12 PM
 #12

...there is no reason to use floating-point rather than a simple int64...
JSON-RPC doesn't support an Integer type. So it's floating-point or string.
Actually, JSON-RPC doesn't support integer nor floating-point. It supports "number", which is not necessarily either.

jgarzik
Legendary
*
qt
Offline Offline

Activity: 1596
Merit: 1015


View Profile
January 29, 2011, 09:12:22 PM
 #13

Actually, JSON-RPC doesn't support integer nor floating-point. It supports "number", which is not necessarily either.

Irrelevant distinction.  The definition of "number" requires support for decimal numbers.

That said, many libraries are smart -- such as jansson -- and will evaluate a number directly into an integer.

Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1085


Chief Scientist


View Profile WWW
January 29, 2011, 09:30:37 PM
 #14

That said, many libraries are smart -- such as jansson -- and will evaluate a number directly into an integer.
From the jansson docs:

"integer numbers whose absolute values are too large to be represented in the int type will result in an overflow error"

As I said in the thread about possibly changing the ECDSA curve bitcoin uses, programmers like to solve "problems" that they know they can solve and that they think will make things a little bit better.  But, in my humble opinion, unless you're solving an important problem changing things because you think you know how is often a bad idea.

This is a perfect example:  change the RPC to spit out 64-bit integers (or move to a new RPC that spits out integers) and anybody using jansson on a 32-bit-int platform will get an overflow error.

I kind of like tcatm's suggestion to define new RPC methods that specify a base unit using strings... but then I thought more about it:

We could use scientific notation, so 1 BTC would be 1e08 base units; then if we ever needed more precision the JSON interface wouldn't change, you could just specify 1e-03 as a value....
... but that's exactly what we have now.  1 BTC is 1e00, 1 base unit is 1e-08, and if we ever needed more precision the JSON interface is ready.

How often do you get the chance to work on a potentially world-changing project?
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1026



View Profile
January 29, 2011, 11:52:51 PM
 #15

As I said in the thread about possibly changing the ECDSA curve bitcoin uses, programmers like to solve "problems" that they know they can solve and that they think will make things a little bit better.  But, in my humble opinion, unless you're solving an important problem changing things because you think you know how is often a bad idea.

This is a perfect example:  change the RPC to spit out 64-bit integers (or move to a new RPC that spits out integers) and anybody using jansson on a 32-bit-int platform will get an overflow error.
OR, anyone using jansson would simply have to use (non-fractional) floats. As mentioned earlier, float types can represent the full range of base bitcoins fine, so long as they're not represented as decimal values.
I kind of like tcatm's suggestion to define new RPC methods that specify a base unit using strings... but then I thought more about it:

We could use scientific notation, so 1 BTC would be 1e08 base units; then if we ever needed more precision the JSON interface wouldn't change, you could just specify 1e-03 as a value....
... but that's exactly what we have now.  1 BTC is 1e00, 1 base unit is 1e-08, and if we ever needed more precision the JSON interface is ready.
1e8 is a perfectly legal JSON number. 1 BTC shouldn't be 1e00, it should be 1e8. And no matter how it is written (0.001 or 1e-03), the fractional-floating-point issues remain.

This works fine with the neutral branch: ./bitcoind -rpcversion=1 sendtoaddress 1KczVqwopWXQdFLe5sNQbpCq7yGSmXx2oo 1e8

mrb
Legendary
*
Offline Offline

Activity: 1512
Merit: 1019


View Profile WWW
January 30, 2011, 12:06:40 AM
 #16

ribuck makes a valid point why double-precision floating point does not work. To make my own example, if the exact value 3000000.00000001 (about 3M) is stored in a double-precision floating point number, it will be stored as approximately as 3000000.0000000098. Therefore doing any math on this number risks amplifying the rounding error (eg. calculating compounded interest, etc).

To quote an adage from the embedded development world: "if you are using floating point, you probably have not yet understood the problem you are trying to solve".

Do not use floating point to ever store BTC quantities. Use a single 64-bit integer. Or two 32-bit integers to represent the decimal and fractional parts.

Don't store non-fractional values in floating point either. It's not even more developer-friendly than using integers. And it is a dangerous way to either accidentally forget to use double-precision everywhere, or to entice developers receiving such a value from an API to divide by 100000000 to get the decimal point in the right place, leading to inaccurate results.
jgarzik
Legendary
*
qt
Offline Offline

Activity: 1596
Merit: 1015


View Profile
January 30, 2011, 12:09:57 AM
 #17

If I were to rewrite the JSON-RPC interface from scratch, I would probably use strings to represent the full int64 bitcoin value, without any decimals.

But perfect is the enemy of good.

Our current interface works just fine, and all this is making a mountain out of a molehill.  I do not see the value in changing what is currently in use.

Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
ribuck
Donator
Hero Member
*
Offline Offline

Activity: 826
Merit: 1008


View Profile
January 30, 2011, 10:06:48 AM
 #18

ribuck makes a valid point why double-precision floating point does not work. To make my own example, if the exact value 3000000.00000001 (about 3M) is stored in a double-precision floating point number, it will be stored as approximately as 3000000.0000000098. Therefore doing any math on this number risks amplifying the rounding error (eg. calculating compounded interest, etc).
Sure that's true, but it's trivially worked around by storing it within the double-precision as 300000000000001 and inserting the decimal when displaying the number.

Cute quote, but the embedded world has true integers to work with; JSON-RPC does not.
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1026



View Profile
January 30, 2011, 06:42:26 PM
 #19

Our current interface works just fine, and all this is making a mountain out of a molehill.  I do not see the value in changing what is currently in use.
It doesn't work just fine. It is impossible to send values with sub-cent precision, and therefore also impossible to send almost any TBC coins. And just because you personally don't care about TBC is not an excuse to make life more difficult for those who do. It's no different than Windows developers trying to prevent a Linux (or Mac) port of their open source software simply by stating "it works fine for us on Windows" as an excuse to not merge patches that make it work on both platforms. The BitCoin community is growing, and not everyone is going to have the same ideals or even basic reasons for being part of the community; don't force your narrow subset of interests on everyone else whose interests might or might not overlap with yours.

jgarzik
Legendary
*
qt
Offline Offline

Activity: 1596
Merit: 1015


View Profile
January 30, 2011, 06:43:34 PM
 #20

Problem is, "TBC" is your own invention, no matter how many times you repeat it.

Jeff Garzik, Bloq CEO, former bitcoin core dev team; opinions are my own.
Visit bloq.com / metronome.io
Donations / tip jar: 1BrufViLKnSWtuWGkryPsKsxonV2NQ7Tcj
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1026



View Profile
January 30, 2011, 07:04:49 PM
 #21

Problem is, "TBC" is your own invention, no matter how many times you repeat it.
No problem at all. TBC is an adaptation of the BitCoin currency to the Tonal number system, in the same way as a port of software is an adaptation to a new platform. The only problem is when people work against innovation and progress.

0x6763
Guest

February 07, 2011, 08:00:16 PM
 #22

Floating point is not good for dealing with money.

Subtracting two Doubles on the JVM:
Code:
user> (- 0.8846 0.8524)
0.032200000000000006

Subtracting two Floats on the JVM:
Code:
user> (- (float 0.8846) (float 0.8524))
0.03219998

Neither operation got the accurate answer of 0.0322.

Why floating point is being considered at all is a mystery to me.
Gavin Andresen
Legendary
*
qt
Offline Offline

Activity: 1652
Merit: 1085


Chief Scientist


View Profile WWW
February 07, 2011, 08:40:11 PM
 #23

Why floating point is being considered at all is a mystery to me.

Huh?  Nobody is suggesting that any math be done with floating-point coins.

But even if we were, your first example gives the correct result to the 8-places-of-precision that Bitcoin deals with:
 (round 0.032200000000000006  to 8 places and you get 0.03220000)

How often do you get the chance to work on a potentially world-changing project?
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1026



View Profile
February 07, 2011, 11:20:57 PM
 #24

Floating point is not good for dealing with money.

Subtracting two Doubles on the JVM:
Code:
user> (- 0.8846 0.8524)
0.032200000000000006

Subtracting two Floats on the JVM:
Code:
user> (- (float 0.8846) (float 0.8524))
0.03219998

Neither operation got the accurate answer of 0.0322.

Why floating point is being considered at all is a mystery to me.
Those are problems with decimal fractions in floats. Tonal fractions have no such problem, nor do integer values (which I was recommending). A floating-point implementation that cannot handle all possible BitCoin values as integers, is a buggy implementation. Wink

0x6763
Guest

February 08, 2011, 03:18:11 AM
 #25

Perhaps my examples are not the best, and admittedly this is outside of my area of expertise, but my main concern is that I think it goes against best practices to use floating point numbers for money.  Perhaps this is not an issue with the bitcoin client, I don't know.  Using floating point numbers still doesn't sound like a good idea to me, though.
genjix
Legendary
*
expert
Offline Offline

Activity: 1232
Merit: 1039


View Profile
February 08, 2011, 10:31:38 PM
 #26

If it's just the JSON api then why not fix it when it becomes a problem? Upgrading an API to a new one is not a problem as much as upgrading a protocol.

The real question is whether the precision should be made larger from int64 to int128

I'm sure the inventors of the internet didn't guess that ipv4 addresses would become exhausted. It's not a big deal to double the precision and much easier to change now than in the future.
jon_smark
Member
**
Offline Offline

Activity: 90
Merit: 10


View Profile
February 08, 2011, 11:20:24 PM
 #27

Why does this discussion have to made over and over again?  Over the ages countless people have been burned by the practice of storing currency values in floating point, which is why nowadays it is conventional wisdom that any application handling money should never, ever use floating point types to store currency. You may think you are saving some space, or being more "efficient", or that you have enough precision, or whatever.  Resist the temptation, seriously. Just say NO!  If you don't, sooner or later that decision will come back to bite you in the ass.

Ever wonder why DBMSs have these awkward looking "numeric" types (sometimes also called "decimal"), where you explicitly specify the precision and the number of decimal places?  It's not because the DBMS people don't know of floating point.  Quite on the contrary.  They know floating point all too well, which is why they don't use it for currency.
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1026



View Profile
February 09, 2011, 02:24:25 AM
 #28

If it's just the JSON api then why not fix it when it becomes a problem? Upgrading an API to a new one is not a problem as much as upgrading a protocol.
It is a problem now.

The real question is whether the precision should be made larger from int64 to int128
It is technically impossible for more than 21 million bitcoins to ever exist. int64 holds the entire quantity of base units. int128 makes no sense on a technical level. Besides, JSON only has "numbers", not int-sizes or floats. Buggy JSON implementations might try to read number-without-a-decimal-point as an int32, which is why the JSON API should require a ".0" after every number. Base units should be used because 1) valid JSON implementations might try making an imprecise float (all are, for 0.1) even when precision is needed, 2) human-readable values should be kept on the GUI, not in low-level internals like APIs, and 3) Not all Bitcoin users want to use the Decimal BitCoin (BTC) divisions.

casascius
Mike Caldwell
VIP
Legendary
*
Offline Offline

Activity: 1386
Merit: 1073


The Casascius 1oz 10BTC Silver Round (w/ Gold B)


View Profile WWW
February 09, 2011, 11:55:56 PM
 #29

Problem is, "TBC" is your own invention, no matter how many times you repeat it.

Even though I think "TBC" is silly, if the RPC accepted strings and could either be decimal ones (1234.567) as well as hexadecimal (0x1234.567) then he'd have his Tonal Bitcoins.  It would be up to him to fork it to put those silly 19th-century Mormon Deseret Alphabet looking digits in instead of the hexadecimal the rest of us understand, but anyone who shared the legitimate concern about bitcoins being strictly divisible by ten would have it solved, this could be used to represent exact amounts from IEEE floating point as well.

Companies claiming they got hacked and lost your coins sounds like fraud so perfect it could be called fashionable.  I never believe them.  If I ever experience the misfortune of a real intrusion, I declare I have been honest about the way I have managed the keys in Casascius Coins.  I maintain no ability to recover or reproduce the keys, not even under limitless duress or total intrusion.  Remember that trusting strangers with your coins without any recourse is, as a matter of principle, not a best practice.  Don't keep coins online. Use paper or hardware wallets instead.
Luke-Jr
Legendary
*
expert
Offline Offline

Activity: 2506
Merit: 1026



View Profile
February 10, 2011, 04:53:38 AM
Last edit: February 10, 2011, 09:12:39 PM by Luke-Jr
 #30

Even though I think "TBC" is silly, if the RPC accepted strings and could either be decimal ones (1234.567) as well as hexadecimal (0x1234.567) then he'd have his Tonal Bitcoins.  It would be up to him to fork it to put those silly 19th-century Mormon Deseret Alphabet looking digits in instead of the hexadecimal the rest of us understand, but anyone who shared the legitimate concern about bitcoins being strictly divisible by ten would have it solved, this could be used to represent exact amounts from IEEE floating point as well.
1 BTC != 1 TBC. 0x1234.567 BTC is 6816.287 TBC (6C81C6.E9287 in hexadecimal; it is also impossible, given it contains a fractional 0.7 (Tonal/hex; 0.4375 decimal) base units)

In any case, I don't want RPC to use TBC. I want it to use base units, like all internals/protocols should. Adding a ".0" to the end is only to workaround a known buggy JSON implementation.
 

casascius
Mike Caldwell
VIP
Legendary
*
Offline Offline

Activity: 1386
Merit: 1073


The Casascius 1oz 10BTC Silver Round (w/ Gold B)


View Profile WWW
February 10, 2011, 07:31:30 AM
 #31

Even though I think "TBC" is silly, if the RPC accepted strings and could either be decimal ones (1234.567) as well as hexadecimal (0x1234.567) then he'd have his Tonal Bitcoins.  It would be up to him to fork it to put those silly 19th-century Mormon Deseret Alphabet looking digits in instead of the hexadecimal the rest of us understand, but anyone who shared the legitimate concern about bitcoins being strictly divisible by ten would have it solved, this could be used to represent exact amounts from IEEE floating point as well.
1 BTC != 1 TBC. 0x1234.567 BTC is 6816.9287 TBC (6C81C6.E9287 in hexadecimal; it is also impossible, given it contains a fractional 0.7 (Tonal/hex; 0.4375 decimal) base units)

In any case, I don't want RPC to use TBC. I want it to use base units, like all internals/protocols should. Adding a ".0" to the end is only to workaround a known buggy JSON implementation.
 
6816.9287 TBC ==6C81C6.E9287 in hex?
See look you even got your own 9 confused. Isn't 9 in this tonal gibberish really a ten (0xa)?  You have used a 9 in the same spot in your tonal quantity as in hex. If the purported fact that 5+5=9 in bonal trips up even a tonal genius, imagine how much the rest of us want nothing to do with it.

Companies claiming they got hacked and lost your coins sounds like fraud so perfect it could be called fashionable.  I never believe them.  If I ever experience the misfortune of a real intrusion, I declare I have been honest about the way I have managed the keys in Casascius Coins.  I maintain no ability to recover or reproduce the keys, not even under limitless duress or total intrusion.  Remember that trusting strangers with your coins without any recourse is, as a matter of principle, not a best practice.  Don't keep coins online. Use paper or hardware wallets instead.
Pages: 1 2 [All]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!