I am training some data using my onw neural net (UNET) which has ran great in version 11.
Using version 12 some updates have been done to the NetTrain function which are giving me troubles.
During training i get these errors.
CompiledFunction::cfta: Argument {Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]]} at position 1 should be a rank 1 tensor of machine-size real numbers.
Transpose::nmtx: The first two levels of {{0.142857},If[Min[\[Infinity],Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]]]>0,Log10[{Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]]}],{Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]]}]} cannot be transposed.
CompiledFunction::cfta: Argument {Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]],Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]]} at position 1 should be a rank 1 tensor of machine-size real numbers.
Transpose::nmtx: The first two levels of {{0.142857,0.285714},If[Min[\[Infinity],Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]]]>0,Log10[{Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]],Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]]}],{Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]],Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]]}]} cannot be transposed.
CompiledFunction::cfta: Argument {Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]],Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]],Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]]} at position 1 should be a rank 1 tensor of machine-size real numbers.
General::stop: Further output of CompiledFunction::cfta will be suppressed during this calculation.
Transpose::nmtx: The first two levels of {{0.142857,0.285714,0.428571},If[Min[\[Infinity],Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]]]>0,Log10[{Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]],Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]],Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]]}],{Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]],Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]],Mean[LibraryFunctionError[LIBRARY_FUNCTION_ERROR,6]]}]} cannot be transposed.
General::stop: Further output of Transpose::nmtx will be suppressed during this calculation.
The errors only seem to happen when i uses large data. When training the netwrork with 2D data (128x128) i dont get any errors but when moving to 3D (40x128x128) the errors start popping up. The NetTrain still seems to run through its iterations but does not do anything.
Any experts around that know what is going on and how to fix it. Version 12 has some new features that i want to give a try so it would be a shame going back to version 11.
Thanks!