So it does mostly work except for CUDAFluidDynamics (apparently it also works if you will enable dynamics beforehand and also CUDAVolumetricRender works https://mathematica.stackexchange.com/a/22654 ).

See: https://www.wolframcloud.com/obj/13f45cd5-6d93-4bab-8c82-e0e08d824984

I hope installing CUDA toolkit 11.5 update 1 will not going to break anything.

Hi Greg,

No worries. Thanks for the heads-up on the 11.3 CUDA toolkit. The above now works on my machine also. However, it still isn't using GPU, even when you specify TargetDevice -> "GPU". It just reverts to using CPU, as in other cases.

Jonathan

Gregory,

Do you want to post (a link to) some WL code that I can benchmark on my Ryzen 3090 machine, to confirm your findings?

Jonathan

Jonathan,

depending on batch size, I've had the GPU crunch over 100k samples per second, but 85-90k is typical. 3-5s GPU time is typical, 5-6MIN CPU time is typical.

I'm running an EVGA RTX 3090 air cooled at 1995Mhz stable OC. My RAM is 3600Mhz, which probably also helps. There's about 35-37% CPU utilization still showing during those 3-4 seconds of GPU training.

-Greg

Attachments:

CUDA Screenshot ...png

Here's another GPU fail, this time with AnomalyDetection. The error message may shed some light on the cause of the issue. It seems to be a missing dependency...

Are you able to replicate this Greg?

Stefan,

I updated the latest driver and toolkit, so I am now on 466.11 and 11.3, all latest paclets from Wolfram, NetTrain works.... kinda... 3090 core count isn't being recognized, max BatchSize is limited to 10000 batches, and when I run NetTrain, it shows my GPU utilization at 10-15% at all times. So I'm using a 10th of the CUDA power I have available. NetTrain on CPU correctly shows 98-99% utilization, though seems like 10k batches is still too small even for CPU, given the very fast RAM and PCIe4 bus speeds.
Most curiously, I'm showing +/-50% CPU utilization while NetTrain is targeted to use GPU! Why? Either way, 16s GPU time vs 19-20s CPU time is absolutely a TERRIBLE result, given the power of this GPU.

Attachments:

CUDA Screenshot ...png

The current situation of Mathematica CUDA support for recent NVIDIA GPUs is really terrible. Try to compare with recent MATLAB R2021a support, where overall CUDA performance on similar tasks (discussed here) is far more better.

Wolfram Research should really solve this situation as high priority task!!! Especially in a case of new Apple M1 HW, where the NVIDIA GPUs are completely discontinued.

Hello, I am also facing a similar problem; I am using Mathematica 12.1 (can't use 12.2 at this moment as university procedure for update takes a while, so I have to work with 12.1), CUDA version 11.2, windows server 2019, visual studio 2019. I ran the following code:

Needs["CUDALink`"];
cudaActiveContourFunc = 
  CUDAFunctionLoad[src, 
   "CUDAActiveContour", {{ _Real, "Input"}, { _Real, 
     "Input"}, { _Real, "Input"}, { _Real, "Input"}, { _Real, 
     "Input"}, { _Real, "Input"}, { _Real, "Input"}, {_Real, 
     "Output"}, {_Real, 
     "Output"}, _Real, _Real, _Real, _Integer, _Integer, _Integer}, 
   256, "CompilerInstallation" -> 
    "C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v" <> 
     ToString[DecimalForm[CUDAversion, {2, 1}]] <> "\\"];

gave error: CUDAFunctionLoad::invxpth: The "XCompilerInstallation" option set to $Failed is not valid. "XCompilerInstallation" must be a string pointing to the C compiler directory.

I ran the following commands and got the following output:

CUDAQ[]
o/p: true

CUDADriverVersion[]
o/p: 461.09

CUDAInformation[]
o/p: {1 -> {"Name" -> "GeForce GTX 1080 Ti", "Clock Rate" -> 1582000, 
   "Compute Capabilities" -> 6.1, "GPU Overlap" -> 1, 
   "Maximum Block Dimensions" -> {1024, 1024, 64}, 
   "Maximum Grid Dimensions" -> {2147483647, 65535, 65535}, 
   "Maximum Threads Per Block" -> 1024, 
   "Maximum Shared Memory Per Block" -> 49152, 
   "Total Constant Memory" -> 65536, "Warp Size" -> 32, 
   "Maximum Pitch" -> 2147483647, 
   "Maximum Registers Per Block" -> 65536, "Texture Alignment" -> 512,
    "Multiprocessor Count" -> 28, "Core Count" -> 3584, 
   "Execution Timeout" -> 1, "Integrated" -> False, 
   "Can Map Host Memory" -> True, "Compute Mode" -> "Default", 
   "Texture1D Width" -> 131072, "Texture2D Width" -> 131072, 
   "Texture2D Height" -> 65536, "Texture3D Width" -> 16384, 
   "Texture3D Height" -> 16384, "Texture3D Depth" -> 16384, 
   "Texture2D Array Width" -> 32768, 
   "Texture2D Array Height" -> 32768, 
   "Texture2D Array Slices" -> 2048, "Surface Alignment" -> 512, 
   "Concurrent Kernels" -> True, "ECC Enabled" -> False, 
   "TCC Enabled" -> False, "Total Memory" -> 11811160064}, 
 2 -> {"Name" -> "GeForce GTX 1080 Ti", "Clock Rate" -> 1582000, 
   "Compute Capabilities" -> 6.1, "GPU Overlap" -> 1, 
   "Maximum Block Dimensions" -> {1024, 1024, 64}, 
   "Maximum Grid Dimensions" -> {2147483647, 65535, 65535}, 
   "Maximum Threads Per Block" -> 1024, 
   "Maximum Shared Memory Per Block" -> 49152, 
   "Total Constant Memory" -> 65536, "Warp Size" -> 32, 
   "Maximum Pitch" -> 2147483647, 
   "Maximum Registers Per Block" -> 65536, "Texture Alignment" -> 512,
    "Multiprocessor Count" -> 28, "Core Count" -> 3584, 
   "Execution Timeout" -> 1, "Integrated" -> False, 
   "Can Map Host Memory" -> True, "Compute Mode" -> "Default", 
   "Texture1D Width" -> 131072, "Texture2D Width" -> 131072, 
   "Texture2D Height" -> 65536, "Texture3D Width" -> 16384, 
   "Texture3D Height" -> 16384, "Texture3D Depth" -> 16384, 
   "Texture2D Array Width" -> 32768, 
   "Texture2D Array Height" -> 32768, 
   "Texture2D Array Slices" -> 2048, "Surface Alignment" -> 512, 
   "Concurrent Kernels" -> True, "ECC Enabled" -> False, 
   "TCC Enabled" -> False, "Total Memory" -> 11811160064}, 
 3 -> {"Name" -> "Quadro K420", "Clock Rate" -> 0, 
   "Compute Capabilities" -> 3., "GPU Overlap" -> 1, 
   "Maximum Block Dimensions" -> {1024, 1024, 64}, 
   "Maximum Grid Dimensions" -> {2147483647, 65535, 65535}, 
   "Maximum Threads Per Block" -> 1024, 
   "Maximum Shared Memory Per Block" -> 49152, 
   "Total Constant Memory" -> 65536, "Warp Size" -> 32, 
   "Maximum Pitch" -> 2147483647, 
   "Maximum Registers Per Block" -> 65536, "Texture Alignment" -> 512,
    "Multiprocessor Count" -> 1, "Core Count" -> 192, 
   "Execution Timeout" -> 1, "Integrated" -> False, 
   "Can Map Host Memory" -> True, "Compute Mode" -> "Default", 
   "Texture1D Width" -> 65536, "Texture2D Width" -> 65536, 
   "Texture2D Height" -> 65536, "Texture3D Width" -> 4096, 
   "Texture3D Height" -> 4096, "Texture3D Depth" -> 4096, 
   "Texture2D Array Width" -> 16384, 
   "Texture2D Array Height" -> 16384, 
   "Texture2D Array Slices" -> 2048, "Surface Alignment" -> 512, 
   "Concurrent Kernels" -> True, "ECC Enabled" -> False, 
   "TCC Enabled" -> False, "Total Memory" -> 2147483648}}

CCompilers[Full]
o/p: {{"Name" -> "Intel Compiler", 
  "Compiler" -> CCompilerDriver`IntelCompiler`IntelCompiler, 
  "CompilerInstallation" -> None, 
  "CompilerName" -> Automatic}, {"Name" -> "Generic C Compiler", 
  "Compiler" -> CCompilerDriver`GenericCCompiler`GenericCCompiler, 
  "CompilerInstallation" -> None, 
  "CompilerName" -> Automatic}, {"Name" -> "NVIDIA CUDA Compiler", 
  "Compiler" -> NVCCCompiler, 
  "CompilerInstallation" -> 
   "C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.2\\bin\
\\", "CompilerName" -> Automatic}}

cudaFun = 
 CUDAFunctionLoad[code, 
  "addTwo", {{_Integer, _, "Input"}, {_Integer, _, 
    "Output"}, _Integer}, 256]
o/p: 
CUDAFunctionLoad::invprog: CUDALink encountered an invalid program.
CUDAFunctionLoad[code, "addTwo", {{_Integer, _, 
   "Input"}, {_Integer, _, "Output"}, _Integer}, 256]

CUDAResourcesInformation[]
o/p: {{"Name" -> "CUDAResources", "Version" -> "12.1.0", 
  "WolframVersion" -> "12.1", "Qualifier" -> "Win64", 
  "SystemID" -> {"Windows-x86-64"}, 
  "Description" -> "{ToolkitVersion -> v10.2, MinimumDriver -> 290}", 
  "Category" -> Missing["NotAvailable"], 
  "Keywords" -> Missing["NotAvailable"], 
  "UUID" -> Missing["NotAvailable"], 
  "Creator" -> Missing["NotAvailable"], 
  "URL" -> Missing["NotAvailable"], "Internal" -> False, 
  "Context" -> {}, "Loading" -> Manual, "AutoUpdating" -> False, 
  "Enabled" -> True, 
  "Location" -> 
   "C:\\Users\\P70072599\\AppData\\Roaming\\Mathematica\\Paclets\\\
Repository\\CUDAResources-Win64-12.1.0", 
  "Hash" -> "3357678c60aa79e333fe04fbb5d04dd7"}}

CUDAResourcesInstall[]
o/p: {
PacletObject[
Association[
  "Name" -> "CUDAResources", "Version" -> "12.1.0", 
   "MathematicaVersion" -> "12.1", 
   "Description" -> "{ToolkitVersion -> v10.2, MinimumDriver -> 290}",
    "SystemID" -> {"Windows-x86-64"}, "Qualifier" -> "Win64", 
   "Extensions" -> {{
     "Resource", 
      "Resources" -> {
       "CUDAToolkit", "ExampleData", "LibraryResources"}}}, 
   "Location" -> "C:\\Users\\P70072599\\AppData\\Roaming\\Mathematica\
\\Paclets\\Repository\\CUDAResources-Win64-12.1.0"]]}


vec = Range[1., 10];
CUDAFourier[vec]
o/p:{17.3925 + 0. I, -1.58114 - 4.86624 I, -1.58114 - 
  2.17625 I, -1.58114 - 1.14876 I, -1.58114 - 0.513743 I, -1.58114 + 
  0. I, -1.58114 + 0.513743 I, -1.58114 + 1.14876 I, -1.58114 + 
  2.17625 I, -1.58114 + 4.86624 I}

InstallCUDA[]
o/p: InstallCUDA[]

CUDADot[Table[i, {i, 10}, {j, 10}], 
  Table[i, {i, 10}, {j, 10}]] // MatrixForm
o/p: {
 {55, 55, 55, 55, 55, 55, 55, 55, 55, 55},
 {110, 110, 110, 110, 110, 110, 110, 110, 110, 110},
 {165, 165, 165, 165, 165, 165, 165, 165, 165, 165},
 {220, 220, 220, 220, 220, 220, 220, 220, 220, 220},
 {275, 275, 275, 275, 275, 275, 275, 275, 275, 275},
 {330, 330, 330, 330, 330, 330, 330, 330, 330, 330},
 {385, 385, 385, 385, 385, 385, 385, 385, 385, 385},
 {440, 440, 440, 440, 440, 440, 440, 440, 440, 440},
 {495, 495, 495, 495, 495, 495, 495, 495, 495, 495},
 {550, 550, 550, 550, 550, 550, 550, 550, 550, 550}
}

Any help here would be very very helpful, as I am stuck here for quite sometime. Thank you! Looking forward !!

I just installed Cuda from Nvidia on my Windows 10 Lenovo X1 no problem:

 In[1]:= Needs["CUDALink`"]

In[2]:= CUDAInformation[]

Out[2]= {1 -> {"Name" -> "GeForce GTX 1650 with Max-Q Design", 
   "Clock Rate" -> 1245000, "Compute Capabilities" -> 7.5, 
   "GPU Overlap" -> 1, "Maximum Block Dimensions" -> {1024, 1024, 64},
    "Maximum Grid Dimensions" -> {2147483647, 65535, 65535}, 
   "Maximum Threads Per Block" -> 1024, 
   "Maximum Shared Memory Per Block" -> 49152, 
   "Total Constant Memory" -> 65536, "Warp Size" -> 32, 
   "Maximum Pitch" -> 2147483647, 
   "Maximum Registers Per Block" -> 65536, "Texture Alignment" -> 512,
    "Multiprocessor Count" -> 16, "Core Count" -> 1024, 
   "Execution Timeout" -> 1, "Integrated" -> False, 
   "Can Map Host Memory" -> True, "Compute Mode" -> "Default", 
   "Texture1D Width" -> 131072, "Texture2D Width" -> 131072, 
   "Texture2D Height" -> 65536, "Texture3D Width" -> 16384, 
   "Texture3D Height" -> 16384, "Texture3D Depth" -> 16384, 
   "Texture2D Array Width" -> 32768, 
   "Texture2D Array Height" -> 32768, 
   "Texture2D Array Slices" -> 2048, "Surface Alignment" -> 512, 
   "Concurrent Kernels" -> True, "ECC Enabled" -> False, 
   "TCC Enabled" -> False, "Total Memory" -> 4294967296}}

In[3]:= InstallCUDA[]

Out[3]= Success["CUDALinkLoaded", 
Association[
 "MessageTemplate" :> "CUDALink installation complete.", 
  "CUDAVersion" -> 11.2, 
  "DefaultDevice" -> "GeForce GTX 1650 with Max-Q Design", 
  "Toolkit" -> "C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\
\\v11.2", 
  "NVCC" -> "C:\\Program Files\\NVIDIA GPU Computing \
Toolkit\\CUDA\\v11.2\\bin\\nvcc.exe", 
  "LibrariesLoaded" -> {
   "C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.2\\bin\
\\cublas64_11.dll", 
    "C:\\Program Files\\NVIDIA GPU Computing \
Toolkit\\CUDA\\v11.2\\bin\\cublasLt64_11.dll", 
    "C:\\Program Files\\NVIDIA GPU Computing \
Toolkit\\CUDA\\v11.2\\bin\\cudart64_110.dll", 
    "C:\\Program Files\\NVIDIA GPU Computing \
Toolkit\\CUDA\\v11.2\\bin\\cufft64_10.dll", 
    "C:\\Program Files\\NVIDIA GPU Computing \
Toolkit\\CUDA\\v11.2\\bin\\cufftw64_10.dll", 
    "C:\\Program Files\\NVIDIA GPU Computing \
Toolkit\\CUDA\\v11.2\\bin\\curand64_10.dll", 
    "C:\\Windows\\System32\\nvcuda.dll"}]]

In[4]:= CUDADot[Table[i, {i, 10}, {j, 10}], 
  Table[i, {i, 10}, {j, 10}]] // MatrixForm


Out[4]//MatrixForm= \!\(
TagBox[
RowBox[{"(", "", GridBox[{
{"55", "55", "55", "55", "55", "55", "55", "55", "55", "55"},
{"110", "110", "110", "110", "110", "110", "110", "110", "110", "110"},
{"165", "165", "165", "165", "165", "165", "165", "165", "165", "165"},
{"220", "220", "220", "220", "220", "220", "220", "220", "220", "220"},
{"275", "275", "275", "275", "275", "275", "275", "275", "275", "275"},
{"330", "330", "330", "330", "330", "330", "330", "330", "330", "330"},
{"385", "385", "385", "385", "385", "385", "385", "385", "385", "385"},
{"440", "440", "440", "440", "440", "440", "440", "440", "440", "440"},
{"495", "495", "495", "495", "495", "495", "495", "495", "495", "495"},
{"550", "550", "550", "550", "550", "550", "550", "550", "550", "550"}
},
GridBoxAlignment->{"Columns" -> {{Center}}, "Rows" -> {{Baseline}}},
GridBoxSpacings->{"Columns" -> {
Offset[0.27999999999999997`], {
Offset[0.7]}, 
Offset[0.27999999999999997`]}, "Rows" -> {
Offset[0.2], {
Offset[0.4]}, 
Offset[0.2]}}], "", ")"}],
Function[BoxForm`e$, 
MatrixForm[BoxForm`e$]]]\)

NB: CUDA no longer works on older iMACs as the graphics card support for the GTX 660 is limited to CUDA ver 10.1 by NVIDIA.

You would have to revert to Mathematica V11.x to run GPU functionality on one of these cards.

As of today, my previously working setup (Mathematica 12.2.0.0 + GeForce GTX 1060 6GB on Windows 10) has stopped working after an update from the Wolfram servers replaced the MXNetLink and MXNetResources paclets I had before.

CUDA seems to work fine (CUDAQ[] gives True, InstallCUDA[] is successful, CUDADot[] example works but CUDAFourier doesn't, probably due to the unrelated issue), but I can't run any neural network functions at this point.

Any suggestions?

It all depends a somewhat on the order in which things are done, but here's some more information:

Running

<< CUDALink`
CUDAQ[]

gives True, then CudaInformation[] recognizes the GPU

{1 -> {"Name" -> "GeForce GTX 1060 6GB", "Clock Rate" -> 1708500,
"Compute Capabilities" -> 6.1, "GPU Overlap" -> 1,
"Maximum Block Dimensions" -> {1024, 1024, 64},
"Maximum Grid Dimensions" -> {2147483647, 65535, 65535},
"Maximum Threads Per Block" -> 1024,
"Maximum Shared Memory Per Block" -> 49152,
"Total Constant Memory" -> 65536, "Warp Size" -> 32,
"Maximum Pitch" -> 2147483647,
"Maximum Registers Per Block" -> 65536, "Texture Alignment" -> 512,
"Multiprocessor Count" -> 10, "Core Count" -> 1280,
"Execution Timeout" -> 1, "Integrated" -> False,
"Can Map Host Memory" -> True, "Compute Mode" -> "Default",
"Texture1D Width" -> 131072, "Texture2D Width" -> 131072,
"Texture2D Height" -> 65536, "Texture3D Width" -> 16384,
"Texture3D Height" -> 16384, "Texture3D Depth" -> 16384,
"Texture2D Array Width" -> 32768,
"Texture2D Array Height" -> 32768,
"Texture2D Array Slices" -> 2048, "Surface Alignment" -> 512,
"Concurrent Kernels" -> True, "ECC Enabled" -> False,
"TCC Enabled" -> False, "Total Memory" -> 6442450944}}

CUDADriverVersion[] identifies my drivers as version 465.21, which is correct (note that these are beta drivers needed to run Docker with GPU support on WSL 2, as explained here https://www.docker.com/blog/wsl-2-gpu-support-is-here/ ; they were working just fine with Mathematica up until yesterday).

After doing all of this, NetTrain trains fine on CPU but fails on GPU with the error

NetTrain::badtrgdevgpu: TargetDevice -> GPU could not be used. Please ensure that you have a compatible NVIDIA graphics card and have installed the latest drivers from http://www.nvidia.com/Download/index.aspx.

Strangely enough, if I now try CUDAQ[] I get a pop-up error box saying (note the DLL file is certainly there):

The procedure entry point cufftloadwisdom could not be located in the dynamic link library C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\bin\cufftw64_10.dll.

Then afterwards, even when running neural network functions on the CPU, for example before NetTrain I get (again, the DLL file is present)

LibraryFunction::load: The library C:\Users\fidel\AppData\Roaming\Mathematica\Paclets\Repository\MXNetResources-WIN64-12.2.404\LibraryResources\Windows-x86-64\cublas64_11.dll cannot be loaded.

and also a pop-up error box saying

The procedure entry point cublasLtZZZMatmulAlgoGetHeuristic could not be located in the dynamic link library C:\Users\fidel\AppData\Roaming\Mathematica\Paclets\Repository\MXNetResources-WIN64-12.2.404\LibraryResources\Windows-x86-64\cublas64_11.dll

Thanks for the driver update suggestion! Mathematica is now back to working with the GPU, after I:

1) Uninstalled CUDA Toolkit v11.0 and installed v11.2 2) Updated the driver to v465.42 3) Uninstalled and reinstalled latest Mathematica paclets for MXNetLink and MXNResources (not sure how much this is necessary, I did it only because I was trying before to roll-back the updates)

Note that skipping step (1) above and keeping CUDA Toolkit 11.0 resulted in a working set-up, but I still got an error when running neural net computations after loading CUDALink (or the other way around). I'm guessing this was due to CUDALink and MXNLink being confused about toolkit versions, so I just updated everything to run on v11.2... but note that TensorFlow doesn't play nicely with v11.2, so this required extra tinkering on the side.

What is the output or error messages you're seeing in your setups that aren't working?

What does InstallCUDA[] return?

Could you try this?

Needs["CUDALink`"]
GPUTools`Utilities`VerboseLogPrinter = 1;
CUDAQ[]
CUDADot[Table[i,{i,10},{j,10}],Table[i,{i,10},{j,10}]]

and post the result?

That command should have printed a bunch of debug information as well, I'm mostly interested in that. For example, on my Windows machine I get:

Needs["CUDALink`"]
GPUTools`Utilities`VerboseLogPrinter=1;
CUDAQ[]
CUDADot[Table[i,{i,10},{j,10}],Table[i,{i,10},{j,10}]]
LOG:   ==== Loading Library Files ==== 
LOG:  Loading CUDA Library Files:  C:\WINDOWS\System32\nvcuda.dll, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin\cudart64_110.dll, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin\cufft64_10.dll, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin\cufftw64_10.dll, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin\cublasLt64_11.dll, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin\cublas64_11.dll, C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\bin\curand64_10.dll
LOG:  NVIDIA driver library is located in  C:\WINDOWS\System32\nvapi64.dll
LOG:  NVIDIA Driver Library is   Valid
LOG:  CUDA Library is   Valid
LOG:   ==== Loading Library Functions ==== 
LOG:   ==== Initializing  CUDA  ==== 
True

What do you see on your machine?

I tested on my dell precision 7750 with RTX3000 + Win 10 + WL12.2. Your code works for me.

Could you please contact Wolfram Support (https://www.wolfram.com/support/contact/) with the details?

I spoke too soon!

Needs["CUDALink`"]

CUDADot[Table[i, {i, 10}, {j, 10}], 
  Table[i, {i, 10}, {j, 10}]] // MatrixForm

(837977408  1    1  1    1  1    1  1    1  1
2           2 2   2 2   2 2   2 2   2
3           3 3   3 3   3 3   3 3   3
4           4 4   4 4   4 4   4 4   4
5           5 5   5 5   5 5   5 5   5
6           6 6   6 6   6 6   6 6   6
7           7 7   7 7   7 7   7 7   7
8           8 8   8 8   8 8   8 8   8
9           9 9   9 9   9 9   9 9   9
10         10   10    10 10  10   10    10 10  10

)

Same problem here as above. CUDAResourcesInformation[] gives error, while CUDAQ[] returns true. CUDA functions do not execute. CUDA Version 11.2. Mathematica version 12.2. OS Version Windows 10 (Insider Build), GPU : Nvidia 2070 RTX. Can't get to work any of the functions like CUDAFold, CUDAFourier etc.

Hello. Here are the outputs for commands you asked:

In[2]:= CUDAQ[]

Out[2]= True

In[3]:= InstallCUDA[]

Out[3]= Success["CUDALinkLoaded", 
Association[
 "MessageTemplate" :> "CUDALink installation complete.", 
  "CUDAVersion" -> 11.2, "DefaultDevice" -> "GeForce RTX 2070", 
  "Toolkit" -> "C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\
\\v11.2", 
  "NVCC" -> "C:\\Program Files\\NVIDIA GPU Computing \
Toolkit\\CUDA\\v11.2\\bin\\nvcc.exe", 
  "LibrariesLoaded" -> {
   "C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.2\\bin\
\\cublas64_11.dll", 
    "C:\\Program Files\\NVIDIA GPU Computing \
Toolkit\\CUDA\\v11.2\\bin\\cublasLt64_11.dll", 
    "C:\\Program Files\\NVIDIA GPU Computing \
Toolkit\\CUDA\\v11.2\\bin\\cudart64_110.dll", 
    "C:\\Program Files\\NVIDIA GPU Computing \
Toolkit\\CUDA\\v11.2\\bin\\cufft64_10.dll", 
    "C:\\Program Files\\NVIDIA GPU Computing \
Toolkit\\CUDA\\v11.2\\bin\\cufftw64_10.dll", 
    "C:\\Program Files\\NVIDIA GPU Computing \
Toolkit\\CUDA\\v11.2\\bin\\curand64_10.dll", 
    "C:\\Windows\\System32\\nvcuda.dll"}]]

Yes CUDAFourier[[] is now working. However CUDAFold was not working before but strangely enough it is now working! so is CUDAFoldList. I don't know what changed.

I would like to add that I used CUDALink on my PC (Windows 10, M1200) and on a server (2x K2 until recently, now 2x P4) prior to 12.2. Since I installed 12.2, CUDA did not work. The same issues are described several times in this thread.

I also removed Mathematica completely as described on the Wolfram website and re-installed Mathematica, Visual Studio 2019, nVidia GPU, CUDA toolkit 11.2 from scratch. No problem compiling GPU code in the VS2019 environment but CUDA in 12.2 does not work.

I rely heavily on CUDA.

I am working myself through various 12.2 issues (SerialLink?, R?, Julia?, ...). Unluckily, I cannot test everything from my well-equipped and well-connected home office but I thought I could solve the CUDA issues. I will give this up, until Wolfram provides a solution.

EDIT: Well so much about stopping.

(Inner[Rule, #, ToExpression@#, Association] &@
   Names["$*CUDA*"]) // Dataset

<< CUDALink`

CUDAQ[]
True

CUDAInformation[]
{1 -> {"Name" -> "Quadro M1200", "Clock Rate" -> 1148000, 
   "Compute Capabilities" -> 5., "GPU" ....

CUDADriverVersion[]
460.89

vec = Range[1., 10];
CUDAFourier[ vec]

CUDAFourier::internal: CUDALink experienced an internal error.
CUDAFourier[{1., 2., 3., 4., 5., 6., 7., 8., 9., 10.}]

Amazing. CUDADot and now even CUDAFourier and CUDAMemoryLoad and CUDAMemoryGet work. But

cudaFun = 
 CUDAFunctionLoad[code, 
  "addTwo", {{_Integer, _, "Input"}, {_Integer, _, 
    "Output"}, _Integer}, 256]

CUDAFunctionLoad::instl: The compiler installation directive "CompilerInstallation" -> $Failed does not indicate a usable installation of NVIDIA CUDA Compiler (executable: CCompilerDriver`CCompilerDriverBase`BaseDriver[ResolveCompilerName][Automatic]).

CUDACCompilers[]
{{"Name" -> "Visual Studio", 
  "Compiler" -> 
   CCompilerDriver`VisualStudioCompiler`VisualStudioCompiler, 
  "CompilerInstallation" -> 
   "C:\\Program Files (x86)\\Microsoft Visual \
Studio\\2019\\Community", 
  "CompilerName" -> Automatic}, {"Name" -> "Visual Studio", 
  "Compiler" -> 
   CCompilerDriver`VisualStudioCompiler`VisualStudioCompiler, 
  "CompilerInstallation" -> 
   "C:\\Program Files (x86)\\Microsoft Visual \
Studio\\2017\\BuildTools", "CompilerName" -> Automatic}}

There seems to be a development on my problem. Apparently the GPU calculations do work after all, the problem is one of initialization. The first time the GPUs are called for any calculation the kernel takes around 25 minutes to end the calculation and load the GPUs (this seems to be independent of what calculation involving the GPUs is being done). Once the GPU is loaded once by waiting these 25 minutes, all remaining calls to GPU have an instant response and everything works normally. This intial call happens for every new Kernel and everytime the kernel is re-initiated.

Therefore it seems to be an issue with loading the GPUs the first time. Not sure however how to resolve this. Clearly this is a big problem, since kernel resets happen all the time and one cannot be dependant on waiting 25 minutes per reset.

Thank you for the feedback! We are investigating and will push out fixes as soon as possible!

Yes, we pushed out a paclet update yesterday afternoon! If you restart the kernel and load CUDALink again it should automatically update. The paclet version is CUDALink-12.2.1.

It includes:

• Updated Setup tutorial
• New function InstallCUDA[] that checks if CUDALink is supported on the current machine.
• Functions CUDAResourcesInstall and CUDAResourcesUninstall are now marked Obsolete and CUDALink will no longer attempt to download the CUDAResources paclet automatically.
• Code for detecting CUDA Toolkit installations updated.

Correct, that is still in progress. The NeuralNetworks team is working on it as their highest priority.

Hello, I am on Windows 10 Pro edition, Mathematica 12.2, RTX 3090 GPU and I have installed NVIDIA's CUDA Toolkit 11.2.

CUDAQ[] returns True and CUDAInformation correctly recognises my GPU system. However whenever I run any GPU call the calculation will simply run forever up to the point where mathematica tells me it's not responding any more. I attach a snaposhot to demonstrate. I should mention that the same system on mathematica 12.1 run perfectly on my previous RTX 2080Ti GPU.

Attachments:

Captura2.PNG

Dear Stefan,

Indeed it appears Mathematica 12.2 uses an old version of MxNET which doesn't seem appropriate for RTX3090 GPU series. I think the MxNET paclet has to be updated.

Thank you.

Gianni

Stefan,

is there any light at the end of the tunnel to properly utilize the 3090? Mine works with NetTrain, but...
1) CUDAInformation[] returns Core Count -> Indeterminate (same as one of the posts above),
2) RT cores are determined correctly (82), but Tensor cores aren't displayed at all?
3) max batch size seems to be 10k in NetTrain, which is a travesty, as 3090 can benefit from greater batch sizes, as it has over 10k cores and huge PCIe4 bus.
4) Running the example given in NetTrain help file under TargetDevice option, but adding BatchSize->10000, WorkingPrecision->"Mixed" finishes the training in 13-14 seconds. Given that my 3960X CPU finishes in 20-22s with same batch size and Automatic WorkingPrecision, 13 seconds just seems woefully poor of performance for this monstrous GPU.

Thanks for your feedback and for developing such a great computing platform!

Attachments:

CUDA Screenshot ...png

CUDAResourcesInformation[], as well as the CUDAResources paclet itself, is no longer needed in Mathematica 12.2.0. We apologize for the documentation not having been updated to reflect this change.

What does CUDAQ[] return? Are you on Windows or Linux?

Hi Jean-Michel, Well, strange... I guess you are with Windows. So, your CUDAResourcesInformation[] must return something valid, pointing to the correct CUDA lib version ? (11.2). Could you share ? Thanks, J.

Hello, I just downloaded the package from NVDIA resources (11.2) toolkit , clicked on the installer and all ran well. I am not a magician but it worked. Best.

I am getting the exact same issue.

"As of Version 12.2, CUDA on macOS is no longer supported."

Seriously?! Is there any workaround to run NetTrain[] on macOS + NVIDIA GPU + MMA 12.2?

https://reference.wolfram.com/language/workflow/UseCUDAOnAnExternalGPUOnMac.html

CUDA worls perfectly with Mathematica 2.2 AND CUDA Toolkit 11.2 . this is sure I just installed it and tested. I repeat Toolkit 11.2 . You can find it on nvidia web. Excellent :) Greetings to all members.

Starting in V12.2 the CUDA Toolkit binaries are not supplied by Wolfram Research, but should be installed separately by the user. Do you have the CUDA Toolkit from NVIDIA installed?

It can be found here: https://developer.nvidia.com/cuda-toolkit

What a pity I have THE RTX 3090 with more than 10000 cuda cores. I will install cuda toolkit 11.* I will let you know here. Regards

CUDA Toolkit 11.x should work (but 10.x will not).

That is very strange... CUDADot does not require the CUDAResources paclet. Do the documentation examples for CUDAFunctionLoad work?

But in the official site, the newest Mathematica version of 12.1.0 is only compatible with CUDA Toolkit 10.2.89 but not cuda 11.1, the site is below: https://www.wolfram.com/CUDA/CUDAResources.html and sometiems cannot connect to wolfram research, maybe because censor or web block in China.

Yes, I use Chinese version. Thanks for your passional help. I can‘t wait to see Mathematica 12.2.

Yes, I can confirm that (Win10 + Quadro P1000 + Mathematica 12.2). CUDAlink and all subsequent CUDA functionalities looks like totally broken... :(