Hello,

I am working with a large set (87K) of ~1MB RGB images. I wanted to share some observations and difficulties I have encountered. I have Mathematica Version 10.1 running on Win7 with 16GB of memory. To perform calculations I typically loop over all the files with

Output[ImageF[Import[]]]

Where Output is either Export or Write, ImageF is a series of Image functions like ImageCrop, ImageLevels etc. I have looped through the data several times so I know crashes are not data related. I have encountered problems when trying to run ComponentMeasurements on an AlphaMask. The offending code is below

Do[Write[str, qall[[j]], " ", ComponentMeasurements[Import[qall[[j]]], {"Count", "Circularity", "EquivalentDiskRadius"}, #1 > max & ], " "], {j, 1, Length[qall]}]

I monitor the process with ResourceMonitor and find the kernel's commit to get in the neighborhood of 50GB before it dies. I can perform the same calculation successfully if I resize the images to 1/8th size. Mathematica then uses 13/16 GB which is not freed until the program exits. Given the importance of ComponentMeasurements I am wondering if I am doing something wrong or if this could be a memory leak?

Secondly, I have been unsuccessful with Classify to build a classifier on these images, even with only 50 images in each class. I strongly suspect this is again related to the size of the images, which are not the tiny thumbnails typically used (such as in the logo classifier in this thread or the day/night classifier). I would generally consider these as sort of "toy" problems, especially in light of ImageNet sized classifiers and also Wolfram's ImageIdentify. I am wondering what Wolfram intends to provide to build serious image classifiers or if are there any best practices from the ImageIdentify project that can be shared.

Thanks