Floating point: between blissful ignorance and unnecesssary fear
Most programmers are at one extreme or another when it comes to floating point arithmetic. Some are blissfully ignorant that anything can go wrong, while others believe that danger lurks around every corner when using floating point.
The limitations of floating point arithmetic are something to be aware of, and ignoring these limitations can cause problems, like crashing airplanes. On the other hand, floating point arithmetic is usually far more reliable than the application it is being used in.
It's well known that if you multiply two floating point numbers, a and b, then divide by b, you might not get exactly a back. Your result might differ from a by one part in ten quadrillion (10^16). To put this in perspective, suppose you have a rectangle approximately one kilometer on each side. Someone tells you the exact area and the exact length of one side. If you solve for the length of the missing side using standard (IEEE 754) floating point arithmetic, your result could be off by as much as the width of a helium atom.
Most of the problems attributed to "round off error" are actually approximation error. As Nick Trefethen put it,
If rounding errors vanished, 90% of numerical analysis would remain.
Still, despite the extreme precision of floating point, in some contexts rounding error is a practical problem. You have to learn in what context floating point precision matters and how to deal with its limitations. This is no different than anything else in computing.
For example, most of the time you can imagine that your computer has an unlimited amount of memory, though sometimes you can't. It's common for a computer to have enough memory to hold the entire text of Wikipedia (currently around 12 gigabytes). This is usually far more memory than you need, and yet for some problems it's not enough.
More on floating point computing:
- Floating point numbers are a leaky abstraction
- Anatomy of a floating point number
- Math library functions that seem unnecessary
- Floating point error is the least of my worries