Hi, I'd like to use ImageAugmentationLayer in my binary image segmentation neural network. However, it seems like I can't get the ImageSegmentationLayer to do exactly the same transform on my input image as on my target mask. Is there a hidden way to do this that's not mentioned in the docs? It seems like every invocation of the layer will use a new random crop, but I need to do the exact same random crop on pairs of images. Cheers!
In the meantime I'm using this function to do it.
randomCrop[is_, sz_] := Module[{x = RandomReal[{-1, 1}], y = RandomReal[{-1, 1}]}, ImageCrop[#, sz, {x, y}] & /@ is ]
Hi Carl, unfortunately there is no way to link two ImageAugmentationLayers currently. NetMapOperator will also give you two different crops. You will have to perform the augmentation outside the Neural Net as a pre-processing step. You can either pre-compute you augmented dataset before traning, or you can have online augmentation during training (as ImageAugmentationLayer would give you) by using the "generator" syntax of NetTrain: NetTrain[net, f], as in the 5th signature of the NetTrain doc page:
ImageAugmentationLayer
NetTrain
NetTrain[net, f]
https://reference.wolfram.com/language/ref/NetTrain.html
Inside the generator function you will have to take the current minibatch, apply your linked random cropping and return the cropped minibatch.
@Matteo - was this limitation overcome in the latest versions? I can't find a way to lock the random cropping so that I can augment input and output images of the same random crop. Neither outside training as a pre-processing step. BlockRandom does not seem to lock the random crop parameters. Thanks
We haven't added any feature to ImageAugmentationLayer, but I don't see any reason why you shouldn't be able to achieve what you want by using the "generator" syntax of NetTrain as I mentioned in my old reply. Your generator function should grab a raw image together with its mask, generate random cropping parameters, apply the same cropping to both image and mask and return the crops.
