Fast tanh approximations trade precision for speed using integer cast tricks, polynomial refinements, and lookup tables – relevant for real-time DSP and ML inference.
Key Takeaways
tanh is a core activation function and audio sigmoid; fast approximations matter in neural network inference and signal processing hot paths.
Three broad implementation strategies surface: sqrt-based sigmoid + polynomial refinement, Schraudolph-style integer cast on floats, and LUT interpolation over mantissa bits.
The Schraudolph exponential approximation is a known baseline – computing exp() via int cast – and tanh can be derived from it.
Worst-case error is the practical benchmark that separates approximation strategies, not just average error.
Hacker News Comment Review
Commenters independently converge on the Schraudolph exp trick as foundational context, suggesting the article may underexplain this prior art.
No shared benchmarks appear; claims about which method has better worst-case error are asserted, not measured in this thread.
HN commenter @mjcohen notes the article doesn’t define hyperbolic tangent until roughly two-thirds through, after a long discussion of exp(x) – a structural critique that affects accessibility for newcomers.
Notable Comments
@raphlinus: sqrt-based sigmoid refined with a polynomial may have better worst-case error than generic fast approximations; references his “a few of my favorite sigmoids” post.
@AlotOfReading: cast float input to int, use the top 2 mantissa bits to index a 5-entry LUT, then lerp or poly-approximate with remaining mantissa bits.