Page 1 of 2
Truncating Doubles
Posted: Thu Feb 23, 2023 6:34 am
by Centripidity
Here's a technical question for those who have a much deeper knowledge of Java than I (most of you). I honestly have no idea what the answer is.
If a double is cast to an int the entire fractional part is discarded and the truncated whole number is returned.
So,
would set k = 2.
But, is it at all possible that an integer stored as a double could have a precision/rounding error such, for example, that 3 is actually stored as 2.99999999999999999, and if so would casting this to an int return 2 rather than 3?
We were always taught to avoid equality conditions with doubles for just this reason I think. For example, never write:
Code: Select all
double x = 1.;
double y = SomeFunction(x)
if(y == 10.1)
{
// Do Something Clever.
}
Re: Truncating Doubles
Posted: Thu Feb 23, 2023 10:14 am
by utdgrant
Add 0.5 to the double BEFORE casting, and that will perform a rounding operation.
Re: Truncating Doubles
Posted: Thu Feb 23, 2023 10:59 am
by Centripidity
utdgrant wrote: ↑Thu Feb 23, 2023 10:14 am
Add 0.5 to the double BEFORE casting, and that will perform a rounding operation.
But what if the the value is 2.5 but rounding errors store it as 2.49999999999999999999999? That would still give an unexpected result.
Re: Truncating Doubles
Posted: Thu Feb 23, 2023 11:26 am
by utdgrant
Centripidity wrote: ↑Thu Feb 23, 2023 10:59 am
utdgrant wrote: ↑Thu Feb 23, 2023 10:14 am
Add 0.5 to the double BEFORE casting, and that will perform a rounding operation.
But what if the the value is 2.5 but rounding errors store it as 2.49999999999999999999999? That would still give an unexpected result.
I can't think of any practical situation (especially within the VM environment) where that would be a problem.
My advice would be not to overthink it.
Re: Truncating Doubles
Posted: Thu Feb 23, 2023 11:38 am
by Centripidity
Just curious really.
Re: Truncating Doubles
Posted: Thu Feb 23, 2023 11:39 am
by ColinP
Doubles can store a good range of integers with complete precision.
The significand is 52 bits so if you cast to a 32 bit integer then no problems at all.
One very important thing to be aware of though is that casting a double to an int truncates towards zero rather than the lowest int. Use Math.floor() to get the lowest int.
(int) 3.5 produces 3
(int) -3.5 produces -3
floor( 3.5 ) produces 3
floor( -3.5 ) produces -4
So casting to int can move things in two different directions on the number line which might not be what you expect! It's the source of some mysterious bugs.
Re: Truncating Doubles
Posted: Thu Feb 23, 2023 12:02 pm
by utdgrant
ColinP wrote: ↑Thu Feb 23, 2023 11:39 am
One very important thing to be aware of though is that casting a double to an int
truncates towards zero rather than the lowest int. Use Math.floor() to get the lowest int.
Excellent point. Thanks, Colin.
Re: Truncating Doubles
Posted: Thu Feb 23, 2023 12:45 pm
by Centripidity
utdgrant wrote: ↑Thu Feb 23, 2023 12:02 pm
ColinP wrote: ↑Thu Feb 23, 2023 11:39 am
One very important thing to be aware of though is that casting a double to an int
truncates towards zero rather than the lowest int. Use Math.floor() to get the lowest int.
Excellent point. Thanks, Colin.
Indeed. Something to watch out for.
Re: Truncating Doubles
Posted: Thu Feb 23, 2023 12:48 pm
by Centripidity
As always, thanks for your input. Much appreciated.
It's almost midnight now and I'm no longer young enough to work into the night so...
Good night, gentlemen, and thank you once again.
Cheers,
Peter
Re: Truncating Doubles
Posted: Thu Feb 23, 2023 1:39 pm
by seal58
In some cases I've got a converting problem even when double value is greater than zero. That's why I use to add a very small amount as 0.00000001 to a cv before quantizing or truncating it.