Thank you all for your replies!

Yes, my question arose exactly from Python - my colleague works on the ANN-based eikonal equation solver implemented with Tensorflow - and I wanted to play with this problem using WL.

A particular thanks to Sam's reply: interesting approach to the problem!

That post involves turning the solution into a supervised learning problem by building a dataset of finalized renders of the pde solution generated by NDSolve, then training a network on the dataset.

I am not exactly sure how that method ideally compares to something like the techniques described here (paper1) or here (paper2), but it could potentially be an acceptable approach. The creator of that post claims an 8% error, but this could be a result of hyperparameter selection of the model.

Is NDSolve fast enough at solving high-dimensional differential equations to build a training dataset of thousands of results to train on? If NDSolve took a long enough time, it would probably be faster to directly optimize a black-box PINN learner that can directly use the gradients to train on (as mentioned in the papers linked above) rather than a distance function to the 'correct' NDSolve solution.

I am very new to this area, so any insight regarding the tradeoffs would be greatly appreciated.

Also, it seems MXNet has had a high-priority ticket to support second-derivative computations for several years now, but it does not appear any progress has been made. Perhaps this is not even possible with the MXNet backend Wolfram is using.

• MXNet Jira

• MXNet GitHub issue

Theoretically, the reply to this other issue on the MXNet GitHub is how such a computation would be done in MXNet if all layers in your network have supported higher-order derivatives implemented:

Tensorflow supports it, and I believe PyTorch has a similar method for computing higher-order derivatives as well. This page shows the tf method.

Perhaps ONNYX transformation could be done to bring the model into another library for training and back to Wolfram...