Improving Floating Point Precision

Programming Languages and Software Engineering
The University of Washington

  1. Pavel Panchekha
  2. Alex Sanchez-Stern
  3. Zachary Tatlock

Floating-point arithmetic can be surprising. Correct-looking code produce incorrect results due to Round-off error. Numerical methods can fix these problems, but they’re complex and require specialized knowledge. Floating-point code is written by scientists, engineers, and analysts who are not numerical analysis experts. They need better tools.

Casio automatically fixes numerical precision problems. Casio uses a heuristic search to find program transformations that improve numerical precision. It automatically discovers and applies classic numerical analysis techniques, leading to numerically precise programs without requiring any specialized knowledge from the user. Casio’s output is guaranteed to compute the same mathematical function as the original; it just computes that function more accurately.


The implementation is currently under heavy development. It succeeds on many textbook examples, and on some examples drawn from numerical libraries, scientific publications, and mathematics. Results on 28 test cases drawn from Richard Hamming’s Numerical Methods for Scientists and Engineers are below. In the plot, each rectangle measures the average number of correct bits for random single-precision inputs, with the left end measuring the original program, and the right end measuring Casio’s output. Wider rectangles are better. Casio improves all results, many to almost-perfect precision.

Casio will become free software once the project is published. For now, if you’d like access, please email Pavel Panchekha.

Blog posts


By . Share it—it's CC-BY-SA licensed.