Comments (5)
Sure Miles, let's move them here. I will look at your code today to think of how we will organize the interface.
from forwarddiff.jl.
I thought about what you suggested Miles. Here are my thoughts. I would suggest that we merge your code asap. You can then call it from Optim
with using ForwardDiff
. I would further suggest that we delay for a bit more the unification of our interface since the ForwarDiff
package is still under heavy development. Let me share with you the background reading I am doing since we work together on forward mode and dual number-based autodiff:
- I found these two wonderful blog posts for computing the exact n-th derivatives (easy to implement for function f:R->R) rather than just the first order derivative:
http://jliszka.github.io/2013/10/24/exact-numeric-nth-derivatives.html
http://duaeliststudios.com/automatic-differentiation-with-dual-numbers/ - I am reading this article which was published by some IBM researches, explaining how to exploit matrix algebra for efficient automatic differentiation (I am focusing on the forward mode implementation, which is within the scope of the current package):
http://link.springer.com/chapter/10.1007%2F978-3-642-30023-3_7
This paper seems amazing to me as it offers efficient matrix-centric autodiff algorithms (we can use BLAS here). In any case, although I haven't added much code yet, I am doing work on this in silence, so let's spare us some more time so that I can implement the above methods. Then we will have a broader view of the available routines in order to better design the interface :)
I will tidy up the tests for the existing code (related to issue #9) over the weekend, and you can add your functions now without worrying so much about unifying the API at this stage, what do you think?
from forwarddiff.jl.
I can see these three approaches:
- using the DualNumbers package directly
- higher-order derivatives (using one of the polynomial packages?)
- matrix-based forward AD
as coexisting in ForwardDiff. All three are useful in different contexts.
from forwarddiff.jl.
I agree with you Miles. We can have all these three coexisting in ForwardDiff. Do you have push access to ForwardDiff? If not, I can add you as collaborator when I get back to my computer. Is it a good plan to add your autodiff
functions here with an optional named argument dtype::Symbol=:dual
? The other values for the dtype
option would by matrix
, polynomial
(possibly classic
too if we maintain the existing old fashioned approach for comparison purposes as a fourth option).
from forwarddiff.jl.
Miles, I reorganized the file hierarchy based on our discussion here, allowing for 4 forward AD approaches to coexist (the 3 you mentioned plus the existing one in the package based on types specifically for forward AD). Each of the three approaches goes on a separate folder in src
. Your work on forward AD using dual numbers resides in src/dual_fad
. I kept your function names as autodiff
. If later on we see that there is overlap with the other approaches, then we can rethink if we need to change your function names.
You will notice that your code is in two files, the univariate_range.jl
wich holds functions that map on univariate ranges (f:R^n->R), which is your code taken from Optim.jl/src/autodiff.jl
and the multivariate_range.jl
which holds functions that map on multivariate ranges (f:R^n->R^m), which is your code taken from NLsolve.jl/src/ autodiff.jl
.
Your code has been copied and is intact apart from one change I had to make; the second autodiff
function in each of these files return f, g!
and f!, g!
instead of the respective DifferentiableFunction(f,g!)
and DifferentiableMultivariateFunction(f!,g!)
. The reason for this choice is to avoid making Optim
and NLSolve
package dependencies (so as to call DifferentiableFunction
and DifferentiableMultivariateFunction
). It is an one-liner to add to Optim
and NLSolve
a wrapper that wraps the output of autodiff
.
I will close this issue now, since it seems resolved, but if you want to make any changes please do not hesitate to do so.
from forwarddiff.jl.
Related Issues (20)
- AD in-place instead of broadcast HOT 1
- Is the mutating code the problem here? How could I debug this? HOT 2
- Rationals and Modulo
- `NaNMath` (and `SpecialFunctions`) as extensions? HOT 5
- Broken external link
- `construct_seeds` for types where `typeof(one(T)) !=T` is broken HOT 1
- incorrect 2nd derivative of complex exponential HOT 2
- Can you take derivative of complicated function whose symbolic form is not explicit or not known?
- Cancellation with sparse arrays HOT 5
- Implement hessian! for scalar x
- Implement gammalogccdf for ForwardDiff HOT 1
- `ForwardDiff.jacobian` throws error for `fft` HOT 1
- Correctly forming nested dual numbers. HOT 8
- Derivative of a function of derivatives HOT 7
- Symbolics.jl compatibility HOT 1
- Support derivative(f, ::Complex)
- `ForwardDiff` fails to compute correct derivative HOT 3
- Incorrect Hessian by `exp` function HOT 1
- Method ambiguities reported by Aqua HOT 3
- Document internals? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from forwarddiff.jl.