Herbie in 2019

A lot has happened in Herbie over the last year: new team members (Oliver Flatt), dramatic speed improvements, a plugin system, and more.

First, David Thien continued his incredible work on splitting the notions of type (like real or complex) and representation (like float64 and complex32) in Herbie. This enabled posit support in Herbie 1.3 and could be used as a platform for exploring new floating point representations, as is increasingly common in machine learning. This work continues, but Herbie is increasingly agnostic to varying or even mixed representations. Furthermore, I did some work to make posit support entirely optional, loadable as a Herbie plugin. The plugin interface is very much under development, but we hope to eventually see it possible to load Herbie support for new libraries, new types, and new representations.

Second, simplification, a core Herbie component got dramatically faster: 300× from Dec 2018 to Dec 2019! This largely happened thanks to Max Willsey and his work on faster equivalence graph algorithms. First, he recommended switching from streaming to batch mode for Herbie's simplification, a simple change that produced a dramatic 5× speed-up. Then, he proposed tweaks that eliminated make-work inside the Herbie simplifier. Finally, he wrote a Rust implementation of equivalence graphs, Egg, that brought a further 60× speed-up. Herbie now uses Egg, thanks to some impressive refactoring and packaging by Oliver Flatt, and Herbie's old equivalence graph implementation is now a separate package that lives outside Herbie. The upshot for users? If you add in some big performance improvements in Racket 7.5, our standard benchmark suite has gone from 30 minutes to 5 minutes.

Third, there's been the usual mixed bag of minor fixes and improvements. Precisions and preconditions can be chosen from the GUI. :spec and let* are supported. Infinite loops are shortener. Dependencies were cleaned up. A variety of subtle memory exhaustion bugs with simplification have also been fixed. The speed-ups above helped with all of this, since they made it possible to regularly run our biggest and baddest test suites.

There's a lot to look forward to in 2020. There's not much left to do to speed up Herbie, simply because it is already so fast. So now we can turn to improving Herbie's results, expanding its scope, and making it ever easier to run. If you'd like to help, please use Herbie, submit bug reports, and email us about cases where Herbie wasn't able to help.