E.g.

julia> BetterExp.exp(0f8)
1.0f0

julia> BetterExp.exp(0.8)
2.225540928492468

@JuliaRegistrator register

You can further reduce the polynomial error using minimax polynomials instead of Taylor series. Using the Remez.jl package:

using Remez
N,D,E,X = ratfn_minimax(expm1, (-1/512,1/512), 4, 0)

and rounding the first coefficient to zero gives the coefficients:

 P = (0.0, 0.9999999999999621, 0.49999999999998357, 0.16666670640310027, 0.04166667593850114)

This has maximal approximation error on the interval [-1/512,1/512] of 1.5e-17, vs 2.1e-16 of the Taylor series.

Thanks for implementing this! This error sorta confuses me. I would have thought the AD would go through into the meat of your exp implementation.

ERROR: MethodError: no method matching exp(::ForwardDiff.Dual{ForwardDiff.Tag{TravelingWaveSimulationsPlotting.var"#loss#27"{TravelingWaveSimulationsPlotting.var"#sigmoid_fn#26"{Float64,Float64}},Float64},Float64,2})
You may have intended to import Base.exp
Closest candidates are:
  exp(::T) where T<:Float64 at /home/graham/.julia/packages/BetterExp/DaCOJ/src/exp.jl:108
  exp(::T) where T<:Float32 at /home/graham/.julia/packages/BetterExp/DaCOJ/src/exp.jl:131

Thanks for working on this problem!

Not sure if you are aware, but I wanted to call your attention to a difference in behavior with Base.exp at the infinities. The following goes for both Float64 and Float32. Whereas Base.exp returns 0.0 and Inf at respectively -Inf and Inf, BetterExp.exp returns NaN in both cases.

julia> using BetterExp

# different
julia> Base.exp(-Inf)
0.0

julia> Base.exp(Inf)
Inf

julia> BetterExp.exp(-Inf)
NaN

julia> BetterExp.exp(Inf)
NaN

# this seems fine however
julia> BetterExp.exp(-999.0) == Base.exp(-999.0) == 0.0
true

julia> BetterExp.exp(999.0) == Base.exp(999.0) == Inf
true