Message Boards Message Boards

ImageAugmentationLayer on image and target mask

Posted 6 years ago

Hi, I'd like to use ImageAugmentationLayer in my binary image segmentation neural network. However, it seems like I can't get the ImageSegmentationLayer to do exactly the same transform on my input image as on my target mask. Is there a hidden way to do this that's not mentioned in the docs? It seems like every invocation of the layer will use a new random crop, but I need to do the exact same random crop on pairs of images. Cheers!

POSTED BY: Carl Lange
4 Replies
Posted 6 years ago

In the meantime I'm using this function to do it.

randomCrop[is_, sz_] := 
 Module[{x = RandomReal[{-1, 1}], y = RandomReal[{-1, 1}]},
  ImageCrop[#, sz, {x, y}] & /@ is
  ]
POSTED BY: Carl Lange

Hi Carl, unfortunately there is no way to link two ImageAugmentationLayers currently. NetMapOperator will also give you two different crops. You will have to perform the augmentation outside the Neural Net as a pre-processing step. You can either pre-compute you augmented dataset before traning, or you can have online augmentation during training (as ImageAugmentationLayer would give you) by using the "generator" syntax of NetTrain: NetTrain[net, f], as in the 5th signature of the NetTrain doc page:

https://reference.wolfram.com/language/ref/NetTrain.html

Inside the generator function you will have to take the current minibatch, apply your linked random cropping and return the cropped minibatch.

Posted 1 year ago

@Matteo - was this limitation overcome in the latest versions? I can't find a way to lock the random cropping so that I can augment input and output images of the same random crop. Neither outside training as a pre-processing step. BlockRandom does not seem to lock the random crop parameters. Thanks

POSTED BY: Massimo Fazio

We haven't added any feature to ImageAugmentationLayer, but I don't see any reason why you shouldn't be able to achieve what you want by using the "generator" syntax of NetTrain as I mentioned in my old reply. Your generator function should grab a raw image together with its mask, generate random cropping parameters, apply the same cropping to both image and mask and return the crops.

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract