Oddities of Computer Math in Java

Here Be Dragons

[pic of dragon] [pic of dragon]

Fixed Point Oddities

Example of post-increment side effect: int i = 1; i = i++ + i + i; System.output.print( "i = " + i ); Output: i = 5
Example of overflow: byte b = 100; b += 100; System.output.print( "b = " + b ); Output: b = -56
Example of conversion error: byte b = -100; char c = (char) b; int i = c; System.out.print( "b=" + b + ", c=" + c + "i=" + i ); Output: b=-100, c=ワ, i=65436
Example of division: int i = 9 / 10; System.output.print( "i = " + i ); Output: i = 0

Floating Point Oddities

Example of round-off error: double d1 = 0.1, d2 = 0.3, d3; d3 = d1 + d1 + d1; if ( d2 == d3 ) System.out.print( "d2 equals d3" ); else System.out.print( "d3 - d2 = " + (d3 - d2) ); Output: d3 - d2 = 5.551115123125783E-17 To work around this use fixed point, the "BigDecimal" class, or code like this: double delta = 1E-14; ... if ( d2 >= d3 - delta && d2 <= d3 + delta ) System.out.print( "d2 equals d3" ); else System.out.print( "d3 - d2 = " + (d3 - d2) ); Output: d2 equals d3 The similar looking test below isn't quite the same and should be avoided: if ( Math.abs(d3 - d2) < delta )
Examples of loss of precision: float delta = 0.1f; float f1 = 50000000.0f; float f2 = f1 + 2.0f; if ( f2 - f1 < delta ) System.out.print( "f1 equals f2" ); else System.out.print( "f2 - f1 = " + (f2 - f1) ); Output: f1 equals f2 float f3 = f1 + 5.0f + 5.0f + 5.0f + 5.0f; float f4 = 5.0f + 5.0f + 5.0f + 5.0f + f1; NumberFormat nf = NumberFormat.getInstance(); nf.setMinimumFractionDigits( 1 ); System.out.println( "f3 = " + nf.format(f3) + ", f4 = " + nf.format(f4) + "\n" ); System.out.println( "f4 - f3 = " + (f4 - f3) ); Output: f3 = 50,000,016.0, f4 = 50,000,020.0 f4 - f3 = 4.0 Modern Java allows printf instead of NumberFormat: System.out.printf("%,f%n", f3);
Example of overflow: double d1 = 1.6e308; double d2 = d1 * 2.0 / 2.0; System.out.println( "d1 = " + d1 ); System.out.println( "d2 = " + d2 ); Output: d1 = 1.6E308 d2 = Infinity
Example of floating point division by zero: double d1 = 0.0 / 0.0; double d2 = 0.0 / 0.0; double d3 = d1; System.out.println( "d1 = " + d1 ); System.out.println( "d2 = " + d2 ); System.out.println( "d3 = " + d3 ); System.out.println( "d1 == d2 is " + (d1 == d2) ); System.out.println( "d1 == d3 is " + (d1 == d3) ); Output: d1 = NaN d2 = NaN d3 = NaN d1 == d2 is false d1 == d3 is false
Negative zero and infinity: double d1 = 1.0 / 0.0; double d2 = -1.0 / 0.0; double d3 = 0.0 / 1.0; double d4 = 0.0 / -1.0; double d5 = -0.0 / 0.0; System.out.println( "d1 = " + d1 ); System.out.println( "d2 = " + d2 ); System.out.println( "d3 = " + d3 ); System.out.println( "d4 = " + d4 ); System.out.println( d3 == d4 ); System.out.println( "d5 = " + d5 ); Output: d1 = Infinity d2 = -Infinity d3 = 0.0 d4 = -0.0 true d5 = NaN

See also What Every Computer Scientist Should Know About Floating-Point Arithmetic