Oh, interesting! Now we just need a utility that would rewrite the original Mathematica snippet into GPU-runnable version automatically :P
Yes exactly! And I imaging that's a job very suitable for a rule substitution system like what we have in WL. :)
OTOH, I just remember a new feature: CompiledLayer has got back-propagation ability, which sounds promising:
BTW, the main downside of moving to Python is the lack of good visualization APIs. They keep changing, and the one that is stable (matplotlib) is very limiting.
