I had almost given up, but reading this conversation I got the idea to install and configure a local VPN server and connect the remote machine (an AWS instance in my case) to the VPN. And... it works, because front-end and remote kernel are now part of the same network. I tested also a Manipulate and a trivial NetTrain that are known to require the "dynamic features" between the kernel and the front-end.

The setup is not so complicated: the most laborious part (let's say) is to configure the VPN server and prepare the client profile for the remote machine. For the moment, I also have to start the VPN client on the AWS machine manually, but I'm sure it can be done automatically with a systemd service; I just have to look up how.

In my case I could do it directly on my router (an ASUS with the Merlin Wrt firmware) that can run an OpenVPN server. The kernel configuration is identical, except for the different remote IP address which is now part of the local subnet created by VPN.

Hello. As several people have already reported (see for example this search results on Mathematica Stackexchange), connecting to a remote kernel is a rather obscure thing. It seems that nowhere in the docs is written clearly how the front-end (FE) actually connects to a non-local kernel. For example, if you inspect a working remote connection you will find that the ports the FE is listening on are random; and if they are not really random, they aren't fixed (see here).

That being said, if the remote kernel is running on a local machine, then I'm able to connect from a FE running Wolfram Desktop 12.1.1 with the following steps. Note that my FE is on macOS (10.15) and the remote machine runs GNU/Linux (Arch Linux, but that shouldn't matter at all).

1. Make sure you can connect via ssh with a private/public key pair. You should be able to run a simple ssh -i /path/to/your/private_key user@host wolfram and play around with a bare kernel.
2. Go to "Evaluation > Kernel Configuration Options", select "Advanced Options", and set
   • Arguments to MLOpen: -LinkMode Listen -LinkProtocol TCPIP -LinkOptions MLDontInteract
   • Launch command: $(which ssh) -i /path/to/your/private_key user@host "wolfram" -wstp -LinkMode Connect -LinkProtocol TCPIP -LinkName "`linkname`" -LinkHost `ipaddress`
3. You should now be able to open a new notebook, select your newly created remote kernel (whichever name you gave it), and evaluate some commands. For example, here's a screenshot of a few successful command I ran on FE connected to a remote machine:

A completely different story is launching a remote kernel on a non-local machine (e.g., an Amazon EC2 instance). That is, to my understanding, not straightforward and perhaps not even possible. But I hope someone more expert will contradict me and find a working solution.

I hope this helps.

SSH is configured correctly, as a matter of fact it's the only way I can access the remote machine. I don't have that math link, though. None was created during the installation of the Mathematica kernel. As far as I know, I only have "/usr/local/bin/WolframKernel" and "/usr/bin/wolframscript". Both work and behave similarily but are different binary files (I checked with /usr/bin/diff).

'find /usr/{local,}/bin -name math' returns nothing.

Should I create a symbolic link myself, and if so, to WolframKernel or wolframscript ?

Also, the command in the 'Kernel Properties Windows' of Mathematica is (apart from username and IP) :

"`java`" -jar "`wolframssh`" username@ip-address "/usr/local/bin/WolframKernel" -wstp -LinkMode Connect -LinkProtocol TCPIP -LinkName "`linkname`"

So it seems to use what you call the old method (LinkProtocol TCPIP? ). How do I change it to use the shell log-in ?