cancel
Showing results for 
Search instead for 
Did you mean: 

double type calculation

Kim.Andy
Associate III

Hello.

At the next double type calculation,

double a = 0.7;
char b = 12;
double c;
 
c = a * b;

c is saved to 8.39999 in the register

But 0.7*12 is 8.4

How can I save 8.4 to the 'c' register?

Thanks.

2 REPLIES 2

8.4 is not representable by binary floating-point (there's no N so that 8.4*2^N is integer).

https://en.wikipedia.org/wiki/Floating-point_arithmetic#Representable_numbers,_conversion_and_rounding :

Any rational with a denominator that has a prime factor other than 2 will have an infinite binary expansion. This means that numbers that appear to be short and exact when written in decimal format may need to be approximated when converted to binary floating-point.

JW

S.Ma
Principal

Always be careful when calculating using different variable type format. Cast to make sure.