weird hex calculation

[citb0in]

I have two numbers which I want to do simple addition math with.

Dec / Hex

Number 1 is:
67890123456789012345 / 3AE2A2B962F52DF79

Number 2 is:
35000000000000 / 1FD512913000

If I add decimal values of nr.1 and nr.2 the result is:
67890123456789012345 + 35000000000000 = 67890158460000000000

When I convert this decimal value to hex I get as result:
3AE2A4B6C0147D800

Now I try the same thing but only using the hex values. I use 'bc' tool on Linux for this calculation:

Code:

$ bc

bc 1.07.1
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
obase=16;ibase=16
3AE2A2B962F52DF79+1FD512913000
3AE2A4B6B41E40F79

As you see the result differs from the 1th method.
3AE2A4B6C0147D800 != 3AE2A4B6B41E40F79

Then I thought I check online with any of the available hex calculators <https://www.calculators.tech/hex-calculator>
I enter HexValueA=3AE2A2B962F52DF79 plus (+) HexValueB=1FD512913000 and the
Result = 3AE2A4B6B41E40000[/color]
This same result was output by another online hexcalc (https://purecalculators.com/de/hex-calculator)

I got three different results. What the #!R(T$F2M?!?

[1714916603]

1714916603

[hosseinimr93]

You are adding the two decimal numbers together incorrectly.

67890123456789012345 + 35000000000000 = 67890158460000000000

67890123456789012345 plus 35000000000000 doesn't equal 67890158460000000000. It equals 67890158456789012345. That's 3AE2A4B6B41E40F79 in hexadecimal format and has been calculated correctly by the tool you used on Linux.

[citb0in]

There you go   
I used the calculator on my Linux desktop and when adding those two numbers, the result was displayed as:

6,789015846×10¹⁹

so I just adjusted it myself to show the whole number without expression in power of. The thing is: my 'real' numbers were not as clear as this example here. So I just copied and pasted from one tool to another. Conclusion is: some tools are not capable in calculating and displaying very high numbers. I guess the same is true for the mentioned online hex calculators.

Lesson learned --> I will stick with 'bc' toll for further calculations.

Thanks for pointing out!

[citb0in]

That anyway, yes. But the problem in my case was not the missing specification of the base the problem, but the fact that gnome-calculator cannot handle large numbers. I just found after the fact the function where you can set gnome-calculator also in "programmer mode". There you can simply add hex values. Also here the error occurs, see the following screenshot which makes this clear.



Everything after B4 is just simply omitted and replaced by zeros.

[PowerGlove]

@citb0in: You ran into a pretty interesting situation (3 different answers, in 3 different tools), here's what happened:

The first attempt

It looks like you used GNOME Calculator for this attempt, and the reason you got the limited-precision answer that you did is because of the default preferences in that tool:



With that level of precision: 67890123456789012345 + 35000000000000 = 67890158460000000000 = 0x3ae2a4b6c0147d800

Btw, you can crank the precision all the way up to 100 decimals, which is something not many people seem to be aware of.

The second attempt

This time you used bc, which supports arbitrary precision, so it gave you the correct (i.e. fully precise) answer: 67890123456789012345 + 35000000000000 = 67890158456789012345 = 0x3ae2a4b6b41e40f79

The third attempt

This time you used an online calculator, which did its job in JavaScript, so the calculation was carried out in double precision: 67890123456789012345 + 35000000000000 = 67890158456789008384 = 0x3ae2a4b6b41e40000

I hope that fully solves the mystery for you!

[citb0in]

Btw, you can crank the precision all the way up to 100 decimals, which is something not many people seem to be aware of.

Hi PowerGlove and thanks for your reply. Yes, that's why I wrote on my last post. It was due to precision. Didn't know that I can modify gnome-calculator's precision. Changed it to maximum possible value = 100 and now the result is displayed correct. One can use gnome-calculator in "programming mode" to perform arithmetics with hex values.



Nice to have, now I don't need to open a terminal shell and use bc for my calculations.

Thank you.

[NotATether]

If you have any fixed point library, it is not hard to create an arbitrary-precision calculator. All you have to do is code an AST (abstract syntax tree) to arrange the operators in a tree form according to their order of precedence, and this supports operators with any operant count, and cool stuff like parentheses and even nested functions.

[brainless]

use this calc
https://www.calculator.net/big-number-calculator.html

and
https://www.calculator.net/hex-calculator.html

[citb0in]

Thanks for providing the links for those online calculators, however I wanted something to use locally in Python. It's part of an educational process

[brainless]

Thanks for providing the links for those online calculators, however I wanted something to use locally in Python. It's part of an educational process

use math library

[brainless]

use gmpy2 lib to large number multiply or divide etc