Title: weird hex calculation Post by: citb0in on November 17, 2022, 06:04:44 PM I have two numbers which I want to do simple addition math with.
Dec / Hex Number 1 is: 67890123456789012345 / 3AE2A2B962F52DF79 Number 2 is: 35000000000000 / 1FD512913000 If I add decimal values of nr.1 and nr.2 the result is: 67890123456789012345 + 35000000000000 = 67890158460000000000 When I convert this decimal value to hex I get as result: 3AE2A4B6C0147D800 Now I try the same thing but only using the hex values. I use 'bc' tool on Linux for this calculation: Code: $ bc Quote bc 1.07.1 Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free Software Foundation, Inc. This is free software with ABSOLUTELY NO WARRANTY. For details type `warranty'. obase=16;ibase=16 3AE2A2B962F52DF79+1FD512913000 3AE2A4B6B41E40F79 As you see the result differs from the 1th method. 3AE2A4B6C0147D800 != 3AE2A4B6B41E40F79 Then I thought I check online with any of the available hex calculators <https://www.calculators.tech/hex-calculator> I enter HexValueA=3AE2A2B962F52DF79 plus (+) HexValueB=1FD512913000 and the Result = 3AE2A4B6B41E40000[/color] This same result was output by another online hexcalc (https://purecalculators.com/de/hex-calculator) I got three different results. What the #!R(T$F2M?!? :D Title: Re: weird hex calculation Post by: hosseinimr93 on November 17, 2022, 07:00:29 PM You are adding the two decimal numbers together incorrectly.
67890123456789012345 + 35000000000000 = 67890158460000000000 67890123456789012345 plus 35000000000000 doesn't equal 67890158460000000000. It equals 67890158456789012345. That's 3AE2A4B6B41E40F79 in hexadecimal format and has been calculated correctly by the tool you used on Linux. Title: Re: weird hex calculation Post by: citb0in on November 17, 2022, 08:25:21 PM There you go ;D :D
I used the calculator on my Linux desktop and when adding those two numbers, the result was displayed as: 6,789015846×10¹⁹ so I just adjusted it myself to show the whole number without expression in power of. The thing is: my 'real' numbers were not as clear as this example here. So I just copied and pasted from one tool to another. Conclusion is: some tools are not capable in calculating and displaying very high numbers. I guess the same is true for the mentioned online hex calculators. Lesson learned --> I will stick with 'bc' toll for further calculations. Thanks for pointing out! Title: Re: weird hex calculation Post by: citb0in on November 18, 2022, 12:15:23 PM That anyway, yes. But the problem in my case was not the missing specification of the base the problem, but the fact that gnome-calculator (https://en.wikipedia.org/wiki/GNOME_Calculator) cannot handle large numbers. I just found after the fact the function where you can set gnome-calculator also in "programmer mode". There you can simply add hex values. Also here the error occurs, see the following screenshot which makes this clear.
https://i.ibb.co/ZLJw719/gnome-calculator-not-capable-big-numbers.png Everything after B4 is just simply omitted and replaced by zeros. Title: Re: weird hex calculation Post by: PowerGlove on November 19, 2022, 02:44:10 AM @citb0in: You ran into a pretty interesting situation (3 different answers, in 3 different tools), here's what happened:
The first attempt It looks like you used GNOME Calculator (https://en.wikipedia.org/wiki/GNOME_Calculator) for this attempt, and the reason you got the limited-precision answer that you did is because of the default preferences in that tool: https://i.postimg.cc/jtDg2VZm/Image.png With that level of precision: 67890123456789012345 + 35000000000000 = 67890158460000000000 = 0x3ae2a4b6c0147d800 Btw, you can crank the precision all the way up to 100 decimals, which is something not many people seem to be aware of. The second attempt This time you used bc (https://en.wikipedia.org/wiki/Bc_(programming_language)), which supports arbitrary precision, so it gave you the correct (i.e. fully precise) answer: 67890123456789012345 + 35000000000000 = 67890158456789012345 = 0x3ae2a4b6b41e40f79 The third attempt This time you used an online calculator, which did its job in JavaScript, so the calculation was carried out in double precision (https://en.wikipedia.org/wiki/Double-precision_floating-point_format): 67890123456789012345 + 35000000000000 = 67890158456789008384 = 0x3ae2a4b6b41e40000 I hope that fully solves the mystery for you! ;) Title: Re: weird hex calculation Post by: citb0in on November 19, 2022, 07:54:52 AM Btw, you can crank the precision all the way up to 100 decimals, which is something not many people seem to be aware of. Hi PowerGlove and thanks for your reply. Yes, that's why I wrote on my last post. It was due to precision. Didn't know that I can modify gnome-calculator's precision. Changed it to maximum possible value = 100 and now the result is displayed correct. One can use gnome-calculator in "programming mode" to perform arithmetics with hex values. https://i.ibb.co/W2LWQhM/gnome-calc-100precision-hex-calc.png Nice to have, now I don't need to open a terminal shell and use bc for my calculations. Thank you. Title: Re: weird hex calculation Post by: NotATether on November 19, 2022, 08:55:20 AM If you have any fixed point library, it is not hard to create an arbitrary-precision calculator. All you have to do is code an AST (abstract syntax tree) to arrange the operators in a tree form according to their order of precedence, and this supports operators with any operant count, and cool stuff like parentheses and even nested functions.
Title: Re: weird hex calculation Post by: brainless on November 21, 2022, 09:59:43 PM use this calc
https://www.calculator.net/big-number-calculator.html and https://www.calculator.net/hex-calculator.html Title: Re: weird hex calculation Post by: citb0in on November 22, 2022, 07:53:36 AM Thanks for providing the links for those online calculators, however I wanted something to use locally in Python. It's part of an educational process :)
Title: Re: weird hex calculation Post by: brainless on November 22, 2022, 01:42:57 PM Thanks for providing the links for those online calculators, however I wanted something to use locally in Python. It's part of an educational process :) use math library Title: Re: weird hex calculation Post by: brainless on November 22, 2022, 01:46:36 PM use gmpy2 lib to large number multiply or divide etc
|