mtgox, please fix your rounding bugs

[davout]

See screenshot, this kind of stuff happens a lot.

[1715410806]

1715410806

[ShadowOfHarbringer]

I don't know if mtgox knows or uses this, but PHP has special set of functions for this kind of things - it's called BC_MATH.

http://www.php.net/manual/en/book.bc.php

With it, it is possible to count any number with any needed precision.

[davout]

Also *never* use float column sql data types, when storing money values decimal is the way to go.
Edit : <troll>Also never use PHP</troll>

[ribuck]

<troll>Also never use PHP</troll>

Indeed that's a troll. PHP offers all the tools you need to program sound decimal arithmetic.

PHP is inelegant, but it's capable and ubiquitous. A good software engineer can implement great systems with PHP if they need to.

[ShadowOfHarbringer]

Also *never* use float column sql data types, when storing money values decimal is the way to go.

Precisely, i believe that in PHP it may be best to store floats as pure strings, and convert them when necessary. There were some nasty bugs in float-to-float comparisons in PHP, i don't remember if they fixed them all.

[davout]

Also *never* use float column sql data types, when storing money values decimal is the way to go.

Precisely, i believe that in PHP it may be best to store floats as pure strings, and convert them when necessary. There were some nasty bugs in float-to-float comparisons in PHP, i don't remember if they fixed them all.

You have it wrong, floats are not supposed to be used to store exact values, decimal types are designed for that purpose

<troll>Also never use PHP</troll>

Indeed that's a troll. PHP offers all the tools you need to program sound decimal arithmetic.

PHP is inelegant, but it's capable and ubiquitous. A good software engineer can implement great systems with PHP if they need to.

Just as a good math student can compute a block's hash with a pencil and some paper if he needs to.
Anyway, that's absolutely not the point, I respect PHP for what it is, a performant language that's really easy to learn. Personnally I switched for Rails years ago.

Anyway, I think the rounding issues probably would have been the same if coded in RoR without careful design and knowledge of the exact values handling good practices.

[mtgox]

davout: It is just a display issue. You really have something like $11.487 in your account. So just withdraw one penny less. I'm not using floats to store the account balances.   I suppose I should just floor the amount displayed.

[ribuck]

I suppose I should just floor the amount displayed.

No, you should display the actual balance. If the real balance has three digits after the decimal, you should show three digits.

[kiba]

Also, can you please automatically calculate the fee?

[nanotube]

+1 on showing actual balances without rounding.

+1 on pre-calculating the fee that would be charged, and showing it to the user to see before submitting an order.

[bitcoinex]

+1 on showing actual balances without rounding.

are you ready to see the numbers like this:

12.3234453404553525454656

?

In a system that can trade fractional offerings such numbers are always presented. Simple rounding on sending and receiving sides negate it to the fixed-precise and beautiful numbers.

By the way, btcex.com do not use the decimal rounding. We rounding to the smallest possible size of the quantum of currency. Usually it is 0.01 but not always, it can be exotic such as 0.001 or 0.025.

[tcatm]

you could limit all prices to 4..6 decimal places.

[ribuck]

are you ready to see the numbers like this:

12.3234453404553525454656

No. MtGox already calculates the transactions to a certain level of precision (I think it might be 4). All we want is for our real MtGox balance to be exposed through the interface.

Without that, we can't withdraw our entire balance. If it says 80 bitcoins, but rejects our request to withdraw 80 bitcoins, how are we to know that we really need to ask for 79.9775?

[tcatm]

Without that, we can't withdraw our entire balance. If it says 80 bitcoins, but rejects our request to withdraw 80 bitcoins, how are we to know that we really need to ask for 79.9775?

Click on "Account History".

[bitcoinex]

are you ready to see the numbers like this:

12.3234453404553525454656

No. MtGox already calculates the transactions to a certain level of precision (I think it might be 4).

It's awful. On the set of small transactions rounding error can reach substantial quantities! Trading robots resent!

[ribuck]

Click on "Account History".

Thanks! I see three decimal places for all the calculated values.

[ShadowOfHarbringer]

Also *never* use float column sql data types, when storing money values decimal is the way to go.

Precisely, i believe that in PHP it may be best to store floats as pure strings, and convert them when necessary. There were some nasty bugs in float-to-float comparisons in PHP, i don't remember if they fixed them all.

You have it wrong, floats are not supposed to be used to store exact values, decimal types are designed for that purpose

No, you have it wrong, because You are talking about databases, not PHP. PHP does not have something like "decimal types" as far as i know.

But PHP has the strings, which can store almost anything with any precision you need.

[ShadowOfHarbringer]

+1 on showing actual balances without rounding.

+1 on pre-calculating the fee that would be charged, and showing it to the user to see before submitting an order.

It's awful. On the set of small transactions rounding error can reach substantial quantities! Trading robots resent!

I agree with all of these suggestions.
Trading should be done with (almost ?) absolute rounding precision, if possible.

[The Madhatter]

PHP's float precision can be adjusted in the php.ini file too.

[wumpus]

Financial programming lesson one: never use floats for money!

If there is no built-in fixed-point decimal type in your language it's best to use large integers.... or indeed, strings, although that makes computation somewhat harder.

[bitcoinex]

Financial programming lesson one: never use floats for money!

If there is no built-in fixed-point decimal type in your language it's best to use large integers.... or indeed, strings, although that makes computation somewhat harder.

A bit more complicated in the exchange software because it uses non-integer multipliers.
It is important that the sum of inflows and outflows accurately matched. But the relation between them can be a tad bit inaccurate.

[ShadowOfHarbringer]

or indeed, strings, although that makes computation somewhat harder.

...not in PHP.

[wumpus]

Financial programming lesson one: never use floats for money!

If there is no built-in fixed-point decimal type in your language it's best to use large integers.... or indeed, strings, although that makes computation somewhat harder.

A bit more complicated in the exchange software because it uses non-integer multipliers.
It is important that the sum of inflows and outflows accurately matched. But the relation between them can be a tad bit inaccurate.

Fixed-point arithmetic with non-integer multipliers ain't rocket science either. Before the times of GPUs and fast floating point units, we used to do that a lot in games.
But in this case it is probably best to use a reliable and well-tested library, instead of reinventing another wheel. Isn't there some open source project implementing a Decimal() type for PHP?

[bitcoinex]

Financial programming lesson one: never use floats for money!

If there is no built-in fixed-point decimal type in your language it's best to use large integers.... or indeed, strings, although that makes computation somewhat harder.

A bit more complicated in the exchange software because it uses non-integer multipliers.
It is important that the sum of inflows and outflows accurately matched. But the relation between them can be a tad bit inaccurate.

Fixed-point arithmetic with non-integer multipliers ain't rocket science either. Before the times of GPUs and fast floating point units, we used to do that a lot in games.
But in this case it is probably best to use a reliable and well-tested library, instead of reinventing another wheel. Isn't there some open source project implementing a Decimal() type for PHP?

Each SQL database engine contains a types what you mean. For example, btcex.com provides 1000 digits of precision for you!

btcex.com btcex.com btcex.com btcex.com btcex.com

[nanotube]

+1 on showing actual balances without rounding.

are you ready to see the numbers like this:

12.3234453404553525454656

?

Why, yes, yes i am. Bring them on, please. As long as they accurately represent the balance in my account, it's all good.

Though note that it should never get to so many decimal places, since there's a limit to how many decimal places bid/ask offer inputs can take. But even if it does - I'd rather see the real amount than a rounded one.

[bitcoinex]

+1 on showing actual balances without rounding.

are you ready to see the numbers like this:

12.3234453404553525454656

?

Why, yes, yes i am. Bring them on, please. As long as they accurately represent the balance in my account, it's all good.

Though note that it should never get to so many decimal places, since there's a limit to how many decimal places bid/ask offer inputs can take. But even if it does - I'd rather see the real amount than a rounded one.

Even the price and volume of this small accuracy generates such large precision numbers when you using "limit" orders type.

Do not want to seem intrusive, but we have no such restriction. More precisely, now you can write approx 15 characters of price.

[nanotube]

Even the price and volume of this small accuracy generates such large precision numbers when you using "limit" orders type.

Still, as i said, no problem for me. It doesn't cost my eyes anything to see extra digits.

Do not want to seem intrusive, but we have no such restriction. More precisely, now you can write approx 15 characters of price.

checked out your exchange site... a few comments for you, which i hope may be helpful. more competition in exchanges is always good.

1. it seems that there are no outstanding asks on the site? you need to get some market makers in there, at least until you have a bigger population of traders.

2. your fees seem to be rather large... 6% one way, 50rub the other way? if you want to compete with established exchanges like bcm and mtgox, you have to offer competitive fees. or possibly even start with zero fees, to attract clientele

3. don't know how possible it is... but if you allowed USD exchange on your site as well as RUB, you'd have more interested clientele.

3.1. To that end, you need to finish your translation to EN, there's a bunch of russian in places even if EN is selected. The 'rules' page is in fact completely in russian.

Well, that's it for now. hope that helps.

[davout]

Also *never* use float column sql data types, when storing money values decimal is the way to go.

Precisely, i believe that in PHP it may be best to store floats as pure strings, and convert them when necessary. There were some nasty bugs in float-to-float comparisons in PHP, i don't remember if they fixed them all.

You have it wrong, floats are not supposed to be used to store exact values, decimal types are designed for that purpose

No, you have it wrong, because You are talking about databases, not PHP. PHP does not have something like "decimal types" as far as i know.

But PHP has the strings, which can store almost anything with any precision you need.

Yes, of course I'm talking about databases, as stated earlier I don't do PHP (anymore).
Suggesting to use strings says a lot about your coding skills/experience.

Extra question for mtgox : correct me if i'm wrong but i don't think LR allows more than two decimal places when expressing LRUSD amounts, so, what's the point of storing more decimal places for LRUSD balance ? Rounding should be done at trade time, not display time, so no-one ends up with unusable dollar crumbs.

Also, an easy solution for the which precision should be used for displaying i see two easy answers :
 - "You don't have that much USD" error message changed into "You don't have that much USD, your balance is XX.XXXXX"
 - Display full precision when amount is hovered by the mouse, pretty easy with title attribute

[wumpus]

Each SQL database engine contains a types what you mean. For example, btcex.com provides 1000 digits of precision for you!

btcex.com btcex.com btcex.com btcex.com btcex.com

We are not talking about SQL engines here, I know every SQL since 92 has had a built-in decimal type with configurable precision. I also know that many other languages (Java, Python,...) have Decimal support in the standard library.

The problem is that PHP has no built-in support for computing with these, so my question was if PHP has a (open source) library to do this. Because converting money amounts to floating point is a very dangerous idea. Mtgox is written in PHP so we have to do with PHP, the fact that you don't do PHP anymore is not important.

[bitcoinex]

Each SQL database engine contains a types what you mean. For example, btcex.com provides 1000 digits of precision for you!

btcex.com btcex.com btcex.com btcex.com btcex.com

We are not talking about SQL engines here,

Very vain

I know every SQL since 92 has had a built-in decimal type with configurable precision. I also know that many other languages (Java, Python,...) have Decimal support in the standard library.

The problem is that PHP has no built-in support for computing with these, so my question was if PHP has a (open source) library to do this. Because converting money amounts to floating point is a very dangerous idea. Mtgox is written in PHP so we have to do with PHP, the fact that you don't do PHP anymore is not important.

btcex.com also written on the PHP and it does not prevent him from working as it should in the matter of rounding and precision.

Trying to do the main computations in PHP-frontend is a bad design.

[wumpus]

Well, doing all computations in the SQL engine isn't very good design either. Especially not those that are only used for showing numbers in the UI. It means a lot of unnecessary round-trips to your database server...
Of course, for updates to the accounts it's a different story.

[bitcoinex]

Even the price and volume of this small accuracy generates such large precision numbers when you using "limit" orders type.

Still, as i said, no problem for me. It doesn't cost my eyes anything to see extra digits.

Imagine that all the numbers associated with money, will look this way?
12.3234453404553525454656

(While this number did not give an absolute accuracy!)

Do not want to seem intrusive, but we have no such restriction. More precisely, now you can write approx 15 characters of price.

checked out your exchange site... a few comments for you, which i hope may be helpful. more competition in exchanges is always good.

1. it seems that there are no outstanding asks on the site? you need to get some market makers in there, at least until you have a bigger population of traders.

This chicken and egg problem in some way. I'm not interested in the trading, I was more interested in the technical implementation. Maybe someone will be a marketmaker?

2. your fees seem to be rather large... 6% one way, 50rub the other way? if you want to compete with established exchanges like bcm and mtgox, you have to offer competitive fees. or possibly even start with zero fees, to attract clientele

This is only for RUB, and this is because of taxes.

WM* deposits is free and withdraw only 3%.
(Forgot to write, thank you for info.)

3. don't know how possible it is... but if you allowed USD exchange on your site as well as RUB, you'd have more interested clientele.

In this topic I am looking for people who agree to maintain accounts for other currencies:
https://www.bitcoin.org/smf/index.php?topic=1485.0

Now we are preparing for the EUR and JPY.

3.1. To that end, you need to finish your translation to EN, there's a bunch of russian in places even if EN is selected. The 'rules' page is in fact completely in russian.

Ok

Well, that's it for now. hope that helps.

Thanks for comments!

[bitcoinex]

Well, doing all computations in the SQL engine isn't very good design either. Especially not those that are only used for showing numbers in the UI. It means a lot of unnecessary round-trips to your database server...
Of course, for updates to the accounts it's a different story.

Each page of any complexity is shown by one call to the database

[wumpus]

Ehh yeah sure, if you store everything in one big table...

[bitcoinex]

Ehh yeah sure, if you store everything in one big table...

 

[davout]

Each page of any complexity is shown by one call to the database

Yea right

[ShadowOfHarbringer]

The problem is that PHP has no built-in support for computing with these

Actually, i think that BC_MATH extension is compiled-in PHP in most Linux distributions. If I am wrong, then You will probably have to compile a version with that yourself.
I don't remember ever selecting "bc_math" extension in ubuntu/debian when installing PHP anyway, sho it should be built-in. I cannot confirm this for sure, because I'm not running debian/ubuntu now. Can somebody else confirm this ?

   * bcadd — Add two arbitrary precision numbers
    * bccomp — Compare two arbitrary precision numbers
    * bcdiv — Divide two arbitrary precision numbers
    * bcmod — Get modulus of an arbitrary precision number
    * bcmul — Multiply two arbitrary precision number
    * bcpow — Raise an arbitrary precision number to another
    * bcpowmod — Raise an arbitrary precision number to another, reduced by a specified modulus
    * bcscale — Set default scale parameter for all bc math functions
    * bcsqrt — Get the square root of an arbitrary precision number
    * bcsub — Subtract one arbitrary precision number from another

[ribuck]

Actually, i think that BC_MATH extension is compiled-in PHP in most Linux distributions.

BC_MATH is widely available, and if it isn't in your distro (or at your ISP) then it's likely that GMP is available. Either of these libraries is suitable for arbitrary precision decimal arithmetic without glitches.

[ShadowOfHarbringer]

Actually, i think that BC_MATH extension is compiled-in PHP in most Linux distributions.

BC_MATH is widely available, and if it isn't in your distro (or at your ISP) then it's likely that GMP is available. Either of these libraries is suitable for arbitrary precision decimal arithmetic without glitches.

WOW, it's even better than i though.

PHP has actually **TWO** different libraries just for doing arbitrary precision arithmetics ! This is awesome. PHP FTW.

[ribuck]

WOW ... PHP has actually **TWO** different libraries just for doing arbitrary precision arithmetics ! This is awesome. PHP FTW.

I'll let you into a secret.

I only mentioned two of them. But PHP actually has **ELEVEN**! Ruby only goes up to ten.

[ribuck]

Ehh yeah sure, if you store everything in one big table...

Or if you know the mysteries of SQL "Join".

[jib]

Account history screenshot attached:



Things that concern me:

• Rounding is inconsistent. e.g most of the site's UI and the Description column of this table show 2 decimal places, and other columns of this table show 3.
• When the rest of the site showed I had 47.79 BTC (which was actually 47.787) I was allowed to sell it all and get to -0.003 BTC. This is much more serious, as it shows that the inconsistent rounding and related problems affect the application's behaviour, not just the UI.
• My current balance (-0.003 BTC) is shown as -0 by the API. This is bad; the API shouldn't be hiding information.
• The transaction fees aren't shown anywhere in this table; they're hidden by tweaking the displayed price or something. Mt Gox should be more transparent about what the actual price was at each trade and what the transaction fee was for each trade.
• We're never told what precision the numbers are calculated to internally. Mt. Gox should be more transparent about how many bitcoins I actually own. Is it -0, or -0.003, or -0.0032567 or something else?