I am looking to do some quite intensive batch processing for a large number of stock symbols.
My plan is to use a Wolfram Language script to call multiple instances of the Mathematica program, each with different input parameters.
My questions are:
1) Would a batch script execute the Mathematica program calls sequentially, or concurrently (as I hope)?
2) Are there limitations as to the # of instances that can be run concurrently? E.g. on a 20 core server can I run 20 instances on 20 kernels concurrently?
There might be a better way of doing this (I have considered parallelization) . Any suggestions appreciated.
The main limiting factor you could run into (that isn't hardware related) is the number of controlling processes allowed to you by your license. That is, the type of license you have may limit the number of simultaneous instances of Mathematica you can run at once. You can evaluate $MaxLicenseProcesses from within a Mathematica notebook on your computer to get this value.
Up to that limit, you can start multiple instances of Mathematica, either as Front Ends (the notebook interface), or in simultaneous scripts.
Instead of doing this in parallel yourself (running multiple scripts at once), you might be able to take advantage of Mathematica's built-in parallel tools. This would allow you to let Mathematica handle a lot of the parallelization specifics, as well as potentially unlock more computing power with computing processes.
The different methods may require slightly different techniques in programming and how you approach the problem, but both would likely be possible ways to tackle your project.