I find myself in the situation of having to implement the equivalent of LinkMode -> Launch, for MathLink (WSTP?) connections. How can this be done in a robust manner without actually resorting to LinkLaunch/LinkOpen?

Let me explain in more detail.

There are two processes that will communicate over MathLink. Eventually it will be Mathematica and MATLAB, but for now I'm looking for answers using two Mathematica kernels and pure Mathematica code. Let's call them K1 and K2. These are the steps I need:

1. K1 creates a link with link mode Listen.

2. K1 launches another program (in this case K2) asynchronously (which is in itself a program because Run blocks by default until the other program returns).

   I do not want to pass standard MathLink command line option to the other process in this case, as this is inconvenient. This is one reason I don't want to simply LinkLaunch. Another reason is that eventually I will want to monitor the state of the K2 process somehow and see if it really did start up successfully.

3. K1 needs to wait until K2 connects to its like. This may take a long time (a few 10s of seconds in the worst case---recent versions of MATLAB are slow to start).

4. After a connection is established successfully, K1 will run Install on the link.

What is a reliable way to implement this? In particular, how can it be implemented in a way that it is robust against failures in K2? K1 should wait for an incoming connection, but it must be able to time out eventually. This timeout should not break things for the next attempt at launching K2 and connecting. Nor should interrupting K1 while it's waiting.

This basic attempt with LinkActivate seems to work, but it is not robust to interruptions. Interruptions can break some internal state used by MathLink and cause various problems on the next attempts. Also, I do no actually understand what LinkActivate (a semi-documented function) does, or when it needs to be called.

I think to get a good understanding of how to do this properly, I'd need to understand:

1. What does LinkActivate actually do, and when does it need to be called? When does it block? LinkActivate is defined in terms of LinkConnect which is defined in terms of LinkOpen. How do these Mathematica functions map to the C API functions (MLActivate/MLConnect and MLOpen...)?

2. What sort of internal state does a MathLink link have and how does it change while setting up the link and connecting to it?

Documentation on these things is rather sparse.