Thanks! your solution do solve my problem with data generator function when NetTrain, that is:

Image`ImportExportDump`DeleteCacheJPEG[]`

I can confirm almost the same problem. Out-of-core data eat all my 64GB memory. The generator function does not help. Consequently I have to reduce the volume of the training data and the max training round.

My platform is ubuntu 20.04， Mathematica 13.0.

Assuming your segmentation masks are images, there was a memory leak in the fast image import function used but the NN framework which should have been fixed for 13.0.1 so be on the lookout for that update.

In the meantime here are few things you can try:

1. If the segmentations masks are small and you have less then 257 classes, save them in-core as byte arrays, they will take less space than 64bit integer arrays as they automatically gets save as Integer8

   (mask = RandomInteger[10, {128, 128}]) // ByteCount
   BinarySerialize[mask] // ByteCount
   (*
   131280
   17512
   *)
   

2. Keep them out of core but export them as WXF files

   (* In[117] *)
   file1 = BinaryWrite["mask1", BinarySerialize[mask]];
   Close["mask1"];
   
   (* In[125] *)
   (mask1 = BinaryDeserialize@ReadByteArray[file1]) // MaxMemoryUsed // RepeatedTiming
   mask1 == mask
   
   (* Out[125] *)
   (* {0.000401338, 167864} *)
   
   (* Out[126] *)
   (* True *)
   

3. Not sure why the one above is not using the same encoding but you can force it to use one byte per class

   (* In[122] *)
   file2 = BinaryWrite["mask2", BinarySerialize[NumericArray[mask, "Integer8"]]];
   Close["mask2"];
   
   (* In[128] *)
   (mask2 = BinaryDeserialize@ReadByteArray[file2]) // MaxMemoryUsed // RepeatedTiming
   
   (* Out[128] *)
   (* {0.00037028, 110752} *)
   
   (* In[130] *)
   mask2 == mask
   
   (* Out[130] *)
   (* True *)
   

These solution would require the appropriate encoder on the segmentation port, e.g.

ElementwiseLayer[Sin, "Input" -> NetEncoder[{"Function", BinaryDeserialize, {128, 128}}]]

The low level PNG importer ImageReadPNG is automatically caching the result. If you use the internal function directly you must clean it up yourself with

Image`ImportExportDump`DeleteCachePNG[]

This should hopefully solve the memory leak problem with that approach.