Is there some package or built-in function in Mathematica for algorithmic/automatic differentiation? I’m thinking something with the capabilities of JAX or Autograd for Python. For example, if I have some complicated Python function involving compositions of mathematical functions, control flow statements, etc., JAX and Autograd can compute its derivative at different inputs. Is there some way to do this in Mathematica short of writing a package for it myself?
There have been some past discussions about this:
You can implement various forms of automatic differentiation in the language:
-
Using dual numbers: https://mathematica.stackexchange.com/questions/1873/is-there-any-automatic-differentiation-package
-
Using tensor networks: Tensor networks, Einsum optimizers and the path towards autodiff
If you are referring to autodifferentiation of neural networks…the current system (as of 14.0) based on MXNet does not seem to support doing this.
- Discussion on StackExchange: https://mathematica.stackexchange.com/questions/256323/neural-network-automatic-differentiation-autograd
- Discussion about MXNet and its limits: About MxNet as Mathematica backend choice
Much appreciated; the dual number package was very helpful.