Developers - Best practises for decimal handling

[Seal]

I'd like to open up a discussion about the different way you as developers have approached the problems of decimal handling when working on bitcoin projects.

Because bitcoin is divisible down to 8 decimal places, calculations on float / decimal numbers can be a pain in a number of languages (as numbers may never appear exact even if displayed correctly).

When working on the MtGox api, they have marked the decimal value representations as legacy. It appears they are moving away from anything decimal related and using integers (which is multiplied by 1e-8 when displayed for bitcoin). This makes calculations simpler in whatever language you are programming in, however work will be required in the display logic of a site / program.

What are your programming practises for handling the many decimals that bitcoin provides?

I'd like to hear from you if you run a bitcoin site or even a wallet program, btcjam / satoshidice / electrum etc... ?

[deepceleron]

One should always calculate and store in integer base units when dealing with Bitcoin amounts. Pool operators and other sites that have previously stored values in floats have been shamed out of existence.

The only time it would be even close to acceptable would be if you are dividing or multiplying by a non-integer and want to maintain an even higher degree of accuracy internally in accounts (such as my $0.00000 in mtgox), and then you must ensure that your platform and any other that may run your code will store in IEEE 754 double floats or greater.

Another way bitcoin was more thought out than you may understand: There will be a maximum of 0x775F05A074000 bitcoin base units ever in existence, which is a 51 bit number. From wikipedia "'double precision' (64-bit) binary floating-point number has a coefficient of 53 bits (one of which is implied), an exponent of 11 bits, and one sign bit.".

[Scrat Acorns]

however work will be required in the display logic of a site / program.

I don't see how this is a problem. You're already using extra logic to escape user input, format timestamps, generate urlencoded links, etc.

Storing satoshi values in integer format is the sensible thing to do. Just like storing unix timestamps instead of the date string.

You should never store BTC values as floats. You cannot guarantee that your favorite language / web framework / database will handle them the way you expect them to be handled.

[Zeilap]

One point that hasn't been mentioned that I've seen be a problem is rounding. I've seen posts on the forum where some site has said they have X amount, but it turns out that really they have ever so slightly less than that amount and it's been rounded up, so when they try to send, they get an error saying 'not enough funds!' which is very confusing. So the moral of this story is to round *down* if you aren't going to display the whole balance down to 8 decimal places.

[CIYAM]

There are multiple ways to *round* and any decent *numeric* implementation will let you choose the one that suits you best.

If you use a 64 bit integer type and create a *numeric* class around that then you will have no problems with the range of values required for Bitcoin (and pretty much all currencies). As has already been pointed out *binary* floating point types are *not* suitable for financial software.

Welcome to take a look at the numeric class that CIYAM uses: https://github.com/ciyam/ciyam/blob/master/src/numeric.cpp

[Xenland]

If you use my library (Bitcoin Dev Kit) forces you to use integers for math/proccessing while displaying is in your choice dynamically between decimal and integers. Aswell as other helpfull features like make bitcoind communication smoother and wont block your website and make it render blank on a minor bitcoin error (like low balance,etc).

[Seal]

One should always calculate and store in integer base units when dealing with Bitcoin amounts. Pool operators and other sites that have previously stored values in floats have been shamed out of existence.

There is consensus that from all the replies so far that integers are the way forward, and I would totally agree, from a IT/developer's perspective this makes sense.

I've worked in my real jobs extensively in retail and investment banking on a range of trading and reporting systems. Not a single one I have come across represents monetary value in integer format. (hence this discussion thread) Rounding within all banking systems (in my experience at least) occurs programatically in functions.

however work will be required in the display logic of a site / program.

I don't see how this is a problem. You're already using extra logic to escape user input, format timestamps, generate urlencoded links, etc.

Agreed, it isn't an issue however the logic would either have to be built into the calculation layer (ie your programming) or the visual presentation layer if you were to store values in decimal or integer format respectively. I'd like to know what everybody's preference is and why.

Second to this, looking at the mtgox api, values are stored in not one, not two, but THREE formats:

• Display value
• Float/Double/Decimal value
• Integer value

[CIYAM]

I've worked in my real jobs extensively in retail and investment banking on a range of trading and reporting systems. Not a single one I have come across represents monetary value in integer format. (hence this discussion thread) Rounding within all banking systems (in my experience at least) occurs programatically in functions.

That's interesting - certainly all the banking software that I've seen (am talking *bank end* not UI) is still written in COBOL (developed in the 70's and patched in the 90's for the Y2K bug) where you use PIC variables which let you describe the numeric as a number of digits and number of decimal places.

Am pretty certain that *under the covers* these are actually represented as integers (just not seen that way by the programmer even) as they don't suffer from the "binary floating point problems".

[BitcoinRate.com]

Storing as integers is the most common way I think. Depending on your framework/database it can differ however. For example in Django you could use DecimalField.

[Seal]

I've worked in my real jobs extensively in retail and investment banking on a range of trading and reporting systems. Not a single one I have come across represents monetary value in integer format. (hence this discussion thread) Rounding within all banking systems (in my experience at least) occurs programatically in functions.

That's interesting - certainly all the banking software that I've seen (am talking *bank end* not UI) is still written in COBOL (developed in the 70's and patched in the 90's for the Y2K bug) where you use PIC variables which let you describe the numeric as a number of digits and number of decimal places.

Am pretty certain that *under the covers* these are actually represented as integers (just not seen that way by the programmer even) as they don't suffer from the "binary floating point problems".

My experience has mostly been with back end systems so raw data in SQL databases and mainframes. Data inputs for their front end systems will only accept 2 decimal places as valid data. Anything else input from either a data feed or directly would be rejected as a bad data input.

I don't believe that it would matter whether you represent values in integer or in floating point as long as you have programmed your controls correctly. For calculations and rounding, there should can be a few issues. Eg:

Using Integers:

• Ensure that all calculations do not accept any decimal value and rounding is performed at integer level
• Ensure you have strict controls in your UI layer to process raw figures into the correct decimal representations
• * the tricky aspect here is knowing whether a figure read is a currency figure or a btc figure

Using float/decimals:

• Ensure that all calculations are subject to the correct decimal rounding (eg if BTC, then always round every calculation to 8dp.
• Ensure that every language used represents decimals correctly

[CIYAM]

Using Integers:

• Ensure that all calculations do not accept any decimal value and rounding is performed at integer level
• Ensure you have strict controls in your UI layer to process raw figures into the correct decimal representations
• * the tricky aspect here is knowing whether a figure read is a currency figure or a btc figure

Using float/decimals:

• Ensure that all calculations are subject to the correct decimal rounding (eg if BTC, then always round every calculation to 8dp.
• Ensure that every language used represents decimals correctly

Although I would still very much recommend using integers over binary FP if you do the rounding at all the right points (and this is harder with FP if you are going to be doing multiple calculations) then you should of course end up with the same result.

[davout]

Use a decimal type of which you explicitly set the precision to whatever required.

Don't confuse floats with decimals, they are completely different, a float is by definition an approximation, a decimal stores an exact value.

Don't ever listen to anyone suggesting floating point for financial calculations, they don't know what they're doing, no matter how much precautions and care you take floats are and remain approximations that are not suited for monetary amounts.

If you know the difference and know what you're doing there's really no need to use integers as storage type and handling type. Using an integer to store a decimal value in a high level language would show that you don't know exactly what you're doing.

The reason MtGox will return integers in their API responses is because there is no way to discriminate between decimal and float types in JSON, they are represented the same way. Therefore, if your parsing lib parses these as floats instead of proper decimals you're boned. But if it parses them as integers you have an opportunity to post-process these as proper decimals before actually using them.

[Seal]

Use a decimal type of which you explicitly set the precision to whatever required.

Don't confuse floats with decimals, they are completely different, a float is by definition an approximation, a decimal stores an exact value.

Don't ever listen to anyone suggesting floating point for financial calculations, they don't know what they're doing, no matter how much precautions and care you take floats are and remain approximations that are not suited for monetary amounts.

If you know the difference and know what you're doing there's really no need to use integers as storage type and handling type. Using an integer to store a decimal value in a high level language would show that you don't know exactly what you're doing.

The reason MtGox will return integers in their API responses is because there is no way to discriminate between decimal and float types in JSON, they are represented the same way. Therefore, if your parsing lib parses these as floats instead of proper decimals you're boned. But if it parses them as integers you have an opportunity to post-process these as proper decimals before actually using them.

Understood davout. Most languages these days can handle the different number types, decimal or float won't be an issue. Certainly in my specific cases using MySQL and PHP, these numbers are handled very well and very precisely.

The issue arises with Javascript which is widely used these days.

http://stackoverflow.com/questions/588004/is-javascripts-floating-point-math-broken

Javascript treats floats and decimals as the same type. When performing number manipulations for visual display then some issues might arise. The tricky aspect is... how do i use an Integer number format and have javascript run calculations whilst returning numbers in correct format? Can any javascript developers out there share their thoughts?

[davout]

Javascript treats floats and decimals as the same type. When performing number manipulations for visual display then some issues might arise. The tricky aspect is... how do i use an Integer number format and have javascript run calculations whilst returning numbers in correct format? Can any javascript developers out there share their thoughts?

Interesting.

If it's purely for display purposes I'd say a float is an acceptable approximation.
But of course it really depends on the specifics of your problem, what works for me might not work for you.

You'd probably better in-depth advice if you gave a bit more context

[TheKoziTwo]

Use a decimal type of which you explicitly set the precision to whatever required.

Don't confuse floats with decimals, they are completely different, a float is by definition an approximation, a decimal stores an exact value.

+1

I use only decimals, and bcmath in php will ensure correct calculations.

[Xenland]

Use a decimal type of which you explicitly set the precision to whatever required.

Don't confuse floats with decimals, they are completely different, a float is by definition an approximation, a decimal stores an exact value.

+1

I use only decimals, and bcmath in php will ensure correct calculations.

I might note that in my experience shows any decimal math especially bcmath will result in fun explanations to your customers....

[TheKoziTwo]

Use a decimal type of which you explicitly set the precision to whatever required.

Don't confuse floats with decimals, they are completely different, a float is by definition an approximation, a decimal stores an exact value.

+1

I use only decimals, and bcmath in php will ensure correct calculations.

I might note that in my experience shows any decimal math especially bcmath will result in fun explanations to your customers....

Why? It is 100% accurate as long as you have enough decimals.

[Xenland]

Use a decimal type of which you explicitly set the precision to whatever required.

Don't confuse floats with decimals, they are completely different, a float is by definition an approximation, a decimal stores an exact value.

+1

I use only decimals, and bcmath in php will ensure correct calculations.

I might note that in my experience shows any decimal math especially bcmath will result in fun explanations to your customers....

Why? It is 100% accurate as long as you have enough decimals.

Probably the way binary is done(hint). Binary doesn’t know a "decimal" and computers(CPU) can only "auto" guess the required precision or their calculations could take as long as infinity to compute(hence why we have a math symbol for repeating decimals...) thus most "Test" cases will look like its working correctly but it will break at some point.

[davout]

Some folks still don't get it apparently.

Floats are represented with a sum of powers of 2, any integer can safely be represented in binary since 2^0 = 1.

So for example 0.75 can be exactly represented in floating point representation as : 2^-1 * 2^-2

But some other numbers can not be represented exactly in floating point format, no matter how much bits you use.

Use a decimal type, it's internally represented by an integer times a negative power of ten.

0.75 in decimal is represented as 75 * 10^-2.

Any decimal number can be exactly represented this way.

[TheKoziTwo]

Use a decimal type of which you explicitly set the precision to whatever required.

Don't confuse floats with decimals, they are completely different, a float is by definition an approximation, a decimal stores an exact value.

+1

I use only decimals, and bcmath in php will ensure correct calculations.

I might note that in my experience shows any decimal math especially bcmath will result in fun explanations to your customers....

Why? It is 100% accurate as long as you have enough decimals.

Probably the way binary is done(hint). Binary doesn’t know a "decimal" and computers(CPU) can only "auto" guess the required precision or their calculations could take as long as infinity to compute(hence why we have a math symbol for repeating decimals...) thus most "Test" cases will look like its working correctly but it will break at some point.

BCMath uses arbitrary precision and will never fail as long as you round and use a big enough scale. If you think otherwise I would love to see proof / example of how it can fail.