"Practical" double rounding in Java
So, I've read a lot about this, so please know that I get that a number like 0.6 cannot be represented absolutely exactly like a Java double, but I know there is a version of the double that represents the number 0. 6 is close enough for the number to appear as 0.6 instead of 0.6000000000000001 in most interpretations of the end of that number (e.g. including toString () or marshaling JSON using a library like Jackson is my goal).
I am looking for a mechanism to coerce a double to a number that is interpreted as relative truncated precision. For example, if I want to truncate to 0.1 precision, I want something that will represent 0.6975265613 truncated to 0.1 with 0.6 precision, not 0.6000000000000001
So basically, do the following test work:
@Test
public void testRounding() {
// prove there is a value that displays cleanly
Double cleanValue = 0.6d;
assertEquals("0.6", cleanValue.toString());
Double value = 0.6975265613d;
Double precision = 0.1d;
value = value / precision;
value = value.longValue() * precision;
// this fails with 0.6000000000000001
assertEquals("0.6", value.toString());
}
I am using longValue () to truncate for speed, but you get the same result with Math.floor ().
Looking at the raw double hex values, the pureValue above is: 3fe3333333333333
While rounding result (values): 3fe3333333333334
I need a technique that will consistently give me the intended version (the first), regardless of the rounding precision - as long as the precision doesn't chase the precision limit for the type. Converting to String or BigDecimal would be too slow - this is an analytical application where the x10 cost of these methods would be very measurable.
Again, it's clear that no hexadecimal number really represents a real number 0.6, I just want Java to count it as 0.6 (i.e. 3fe3333333333333) :)
Thank!
source to share
It's not sure what you want, but
@Test
public void testRounding() {
// prove there is a value that displays cleanly
Double cleanValue = 0.6d;
assertEquals("0.6", cleanValue.toString());
Double value = 0.6975265613d;
// get double value truncated to 1 decimal place
value = BigDecimal.valueOf(value).setScale(1,BigDecimal.ROUND_DOWN).doubleValue();
// this fails with 0.6000000000000001
assertEquals("0.6", value.toString());
}
source to share