Improve neural network performance with Mathematica 11.3 ?

Questions
, , , , , ,
1

I look at the blog post with the 11.3 word cloud with ‘Blockchain’ as the BIG center and ask how important is that? As I run the exact same data science GPU code on identical hardware software configuration except for the change from Mathematica 11.2 to Mathematica 11.3 and see my neural network performance go from 295 seconds on 11.2 to 2038 seconds on 11.3. Again NO change other than Mathematica version. And then I see that Mathematica 11.3 still does not support current XCode LLVM/GCC compiler or NVIDIA for CUDA tools (watch it move back to old paclet for Mathematica 10.5 after you upgrade your XCode command line tools to current version, am I expected to pay money to figure this out?) .

This is my experience as I explore the value of Mathematica since release 10 to today for data science and at the same time see the massive improvements and quality of Python, Jupyter, NVIDIA, iOS CoreML, Vulkan, Tensorflow and core GPU computing on MacOS, iOS and Linux.

Really questioning the value proposition of Wolfram for data science going forward. Sad.

2

As I run the exact same data science GPU code on identical hardware software configuration except for the change from Mathematica 11.2 to Mathematica 11.3 and see my neural network performance go from 295 seconds on 11.2 to 2038 seconds on 11.3.

When training the net, could you set the BatchSize and MaxTrainingRounds options to the same value in both versions, and then report the training time again? (we changed the heuristic for choosing a value of MaxTrainingRounds which sometimes uses more training rounds for extra accuracy. And: the inputs/s is a better indicator of speed than the total training time as it doesn’t depend on MaxTrainingRounds). If there is still a speed difference, could you give information about the GPU you are using, the output of $Version and if possible the training script to reproduce this training?

And then I see that Mathematica 11.3 still does not support current XCode LLVM/GCC compiler or NVIDIA for CUDA tools

I am not sure exactly what you mean by this. Could you expand a bit about which functionality you are referring to that doesn’t support “current XCode LLVM/GCC compiler or NVIDIA”?

3

I reported to Wolfram Support:

“While version 11.2.0 was using CUDA paclet CUDAResources-Win64-11.2.22 (based on CUDA 8.0), version 11.3 is downloading paclet CUDAResources-Win64-10.5.0 (based on CUDA 7). I had hoped to get a paclet based on CUDA 9.1. But I get something very old.”

Their reply:

“It does appear that the latest officially supported CUDA paclet is the version 10.5.0, shipped by the product by default. In the past, we have provided the customers, such as yourself, with the beta versions of newer paclets. I have reported the issues with the older paclets, including lack of support for CUDA toolkit 9.0, to our developers. They are working on releasing newer paclet, and we will update you as soon as it happens.”

Please note that CUDA 7 does not support the latest GPU’s. So for sure 11.3 will perform much slower than 11.2. Why they released beta paclets to 11.2 and no longer to 11.3 is a mystery.

4

We had hoped to have updated CUDALink paclets ready for 11.3 release but we ran into some technical issues that had to be resolved first. We plan to release updated CUDALink paclets with CUDA 9.1 support in the next few days, assuming the lingering issues can be ironed out in time.

5

One thing that should also be mentioned: Neural Nets in the WL don’t use CUDALink` at all, so this issue has no effect on the neural net framework which uses CUDA 8 on OSX and CUDA 9.1 for Windows + Linux.

6

Do you support MacBook Pros with NVIDIA cards (from 2014 and before)?

For quite some time now (i.e. even in 11.2), TargetDevice -> "GPU" works for me only if I load CUDALink first, and then successfully evaluate CUDAQ[]. By “successfully” I mean that I’m lucky and it doesn’t cause a crash. Often, the kernel just crashes after CUDAQ[].

In earlier versions (11.0? 11.1? I don’t remember) both TargetDevice -> "GPU" and CUDALink used to work fine on this machine.

I have macOS 10.13, the NVIDIA web driver, and CUDA driver version 387.128.

7

We fixed this annoying issue in 11.3!

8

I must have mixed up my versions … I was certain that I just tried this in 11.3 ("ReleaseID" -> "11.3.0.0 (5944637, 2018030701)") and it did not work. Now I tried it again and TargetDevice -> "GPU" seems to be working fine.

9

One thing that should also be mentioned: Neural Nets in the WL don’t use CUDALink` at all, so this issue has no effect on the neural net framework which uses CUDA 8 on OSX and CUDA 9.1 for Windows + Linux.

So CUDA 9.1 is present in the installation of Mathematica 11.3.
Is this all precompiled code?
Or do you need a C++ compiler like would be needed with the CUDA paclet?

10

What about support for NVidia cards for option TargetDevice->"GPU" in such things as Classify and other neural net-related functions?

Given the long-time status of Mathematica as essentially platform-independent, it seems really constricting that only CUDA with NVidia is supported whereas OpenCL with AMD is not.

11

What about support for NVidia cards for option TargetDevice->“GPU” in such things as Classify and other neural net-related functions?

That is already supported out-of-the-box in 11.3.0.

Given the long-time status of Mathematica as essentially platform-independent, it seems really constricting that only CUDA with NVidia is supported whereas OpenCL with AMD is not.

That has more to do with CUDA being the de-facto standard for most of the state-of-the-art neural network libraries (like MXNet), which Mathematica uses under the hood. If the libraries we use add support for OpenCL then we’d certainly investigate adding support from our end.

12

What about support for NVidia cards for option TargetDevice->“GPU” in such things as Classify and other neural net-related functions?

That is already supported out-of-the-box in 11.3.0.

Given the long-time status of Mathematica as essentially platform-independent, it seems really constricting that only CUDA with NVidia is supported whereas OpenCL with AMD is not.

That has more to do with CUDA being the de-facto standard for most of the state-of-the-art neural network libraries (like MXNet), which Mathematica uses under the hood. If the libraries we use add support for OpenCL then we’d certainly investigate adding support from our end.

13

I meant: “What about support for AMD cards”. Sorry for the confusion.

14

Sorry to say Sebastian, but you are really out of touch with what got delivered to your customers with 11.3 release.

Your team member (and customers) had to tell you that the CUDA paclet was downgraded with 11.3 release and myself and others will tell you that 11.3 causes macOS to reboot with current NVIDIA drivers and CUDA support under current macOS 10.3.3. What kind testing does Wolfram do? In the past I was offered a chance to at least try beta release, not for 11.3. WFT?

Under prior release of MMA I could run CUDA function on this machine, yes rather limited CUDA function but it ran… now it crashes my Mac! I am not here to pay and debug your production product! If you don’t test it DON’T release it as production code!

NVIDIA Web Driver 387.10.10.10.25.161 (Up to date)
Last checked was 03/15/18, 19:55

CUDA Driver Version 387.128

macOS High Sierra 10.13.3 (17D102)
MacBook Pro (Retina, Mid 2012)
NVIDIA GeForce GT 650M:

Chipset Model: NVIDIA GeForce GT 650M
Type: GPU
Bus: PCIe
PCIe Lane Width: x8
VRAM (Total): 1 GB
Vendor: NVIDIA (0x10de)
Device ID: 0x0fd5
Revision ID: 0x00a2
ROM Revision: 3688
Automatic Graphics Switching: Supported
gMux Version: 3.2.19 [3.2.8]
Metal: Supported, feature set macOS GPUFamily1 v3

Mathematica 11.3.0.0

Just asking for System Information
–or–
running this code

Needs[“CUDALink`”]
TextGrid[CUDAInformation[]]

cause macOS to black screen.

That’s really bad!!!

15

Sorry to say Sebastian, but you are really out of touch with what got delivered to your customers with 11.3 release…
Your team member (and customers) had to tell you that the CUDA paclet was downgraded with 11.3 release

I am one of the developers of the neural net framework. I have nothing to do with CUDALink (and as mentioned in this thread, CUDALink is not used by the neural net framework at all), and indeed I had no idea what issues CUDALink faced in 11.3.

I can however speak authoritatively about the neural net framework, which is the title of this post (ie neural net slowdowns in 11.3, for which the discussion seems to have ended). You are replying to a subthread where I claimed that this is fixed:

For quite some time now (i.e. even in 11.2), TargetDevice → “GPU” works for me only if I load CUDALink first

This was a bug in the neural net framework, which I was involved with fixing. It seems your real issues are about CUDALink, for which I have nothing to add to the discussion.

16

Hi on Windows 10
It seems that 11.3 is not finding the NVidea and 11.2 does.
I have two Video cards on this Zbook (main Intel and second NVidea)

MM11.3

Needs["CUDALink`"] 
TextGrid[CUDAInformation[]]

Gives
CUDAInformation::invdevnm: CUDA is not supported on device Intel(R) HD Graphics 530. Refer to CUDALink System Requirements for system requirements.

With 11:2

TextGrid[{1 → {“Name” → “Quadro M1000M”, “Clock Rate” → 1071500,
“Compute Capabilities” → 5., “GPU Overlap” → 1,
“Maximum Block Dimensions” → {1024, 1024, 64},
“Maximum Grid Dimensions” → {2147483647, 65535, 65535},
“Maximum Threads Per Block” → 1024,
“Maximum Shared Memory Per Block” → 49152,
“Total Constant Memory” → 65536, “Warp Size” → 32,
“Maximum Pitch” → 2147483647,
“Maximum Registers Per Block” → 65536,
“Texture Alignment” → 512, “Multiprocessor Count” → 4,
“Core Count” → 128, “Execution Timeout” → 1,
“Integrated” → False, “Can Map Host Memory” → True,
“Compute Mode” → “Default”, “Texture1D Width” → 65536,
“Texture2D Width” → 65536, “Texture2D Height” → 65536,
“Texture3D Width” → 4096, “Texture3D Height” → 4096,
“Texture3D Depth” → 4096, “Texture2D Array Width” → 16384,
“Texture2D Array Height” → 16384,
“Texture2D Array Slices” → 2048, “Surface Alignment” → 512,
“Concurrent Kernels” → True, “ECC Enabled” → False,

"TCC Enabled" -> False, "Total Memory" -> 2147483648}}]

11.2.PNG
11.2.PNG476×477 7.48 KB

11.3.PNG
11.3.PNG715×286 4.62 KB

17

The issue I described above is fixed by upgrading the NVidia driver to version 391.03.
This release was mentioned by Nettrain (387+) that didn’t work in MM11.3 but did work with 11.2.
Windows claimed the driver was up to date but the NVidia site showed newer versions.

18

We have now released updated CUDALink paclets for Mathematica 11.3.
Please run CUDAResourcesInstall[Update -> True] to get the new version.
We apologize for the delay.

19

We have now released updated CUDALink paclets for Mathematica 11.3.
Please run CUDAResourcesInstall[Update -> True] to get the new version.
We apologize for the delay.

20

Thank you very much!

Finally CUDA 9.1 with Visual Studio 2017 compiler supported :slight_smile:

But CUDACCompilers is still like in Mathematica 11.2 not funtional.
But in SytemInformation Toolkit Version 4. is listed. Should this not be 9.1 ?
And the Core Count says 320 while my GeForce GTX1060 with Max-Q Design has 1280 cores!