Can Mathematica do symbolic matrix calculus? This is such an important notation and almost all the Statistics, Machine Learning and Deep Learning literature uses this notation.
Precisely, can we define functions of symbolic vectors and matrices, take derivatives with respect to vectors, matrices and have the final result be also expressed in the same form? For example, differentiating A*x with respect to x should yield A where A is a matrix and x is a vector. This is a very simple function, but it illustrates the idea.
I found an online tool which can do it, so Mathematica should in principle be also able to do it. This is the link to the tool.
However, these links appear to be busted. I am trying to determine how to upload my copy of the original ZIP file of the package (Community no longer allows ZIP files). . Meanwhile, attached is a PDF of its demo notebook, illustrating its capabilities. If these are appealing to anyone, I might be motivated to convert into and share as a Paclet (on the Paclet Repository).