Message Boards Message Boards

0
|
1058 Views
|
2 Replies
|
2 Total Likes
View groups...
Share
Share this post:

$ProcessorCount isn't equal to the real number of CPUs

Shepard WQC
Posted 8 months ago

I'm running Mathematica13.2 on a CentOS7 machine with two AMD Epyc 9754 but $ProcessorCount=12, which is much less.

The shell command "lscpu" and C function sysconf(SCNPROCESSORS_ONLN) in <unistd.h> both give that the number of cpus is 256, and I'm sure the program is on the computing node.

I made some checks, such as SetSystemOptions["ParallelOptions" -> "ParallelThreadNumber" -> 256], and $MaxLicenseProcesses gives Infinity. But they don't work.

This problem is complicated, maybe the key is how does Mathematica know the number of cpus. I guess it may be associated with the MKL lib, but I don't know how to test.

Thanks for help and discussion.
POSTED BY: Shepard WQC
2 Replies
Sort By:

Yes, the automatic detection is done through MKL and a very common reason for an unexpected result is having an environment variable override (such as OMP_NUM_THREADS or MKL_NUM_THREADS).

If that does not resolve the issue, you may define $ProcessorCount in any desired way, for example

Unprotect[$ProcessorCount];
$ProcessorCount = 256;
Protect[$ProcessorCount];

in initialization code like init.m. The value does not have any impact on parallel computation except for deciding how many parallel kernels to launch by default via LaunchKernels[].

Other important parameters would be "ParallelThreadNumber" and "MKLThreadNumber" in SystemOptions["ParallelOptions"], which again can be set to any desired values.
POSTED BY: Ilian Gachevski
Shepard WQC
Shepard WQC
Posted 8 months ago

Thank you very much! The two environment variables exactly as you mentioned are set to 12 (maybe done by the root), and it works fine after changing them to 256.
POSTED BY: Shepard WQC
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract