Message Boards Message Boards

Can RAISR be done within existing Machine Learning framework?

Suppose one has an image that is size m1 x n1 and one wishes to upsize it to size m2 x n2, where m2/m1 = n2/n1. Right now, one can do this in Mathematica by using ImageResize. But the process does not create any new information; it basically interpolates. Can we do better? Here is the idea. One takes a real photo (P1) that is originally m2 x n2. One downsizes it to m1 x n1 in a reasonable way. In the process, information is lost. One then "cheaply" upscales it back to m2 x n2 using traditional methods. Call this image P2. One can then make a map going from patches of P1 to patches of P2. That is, the 10th through 13th row of P1 and 35th through 39th column of P1 might contain one set of pixels and the the 10th through 13th row of P2 and 35th through 39th column of P2 would contain another set of pixels. You do this with A LOT of images and create a lot of patches in each. One then machine learns the transformations that best predict P2 given P1.

If this were possible one could then intelligently fill in the detail in a lot of low resolution images in a plausible way. Google is apparently fiddling with this in a project it calls RAISR. (See https://arxiv.org/abs/1606.01299 and http://www.pcmag.com/news/351027/google-raisr-intelligently-makes-low-res-images-high-quality)

So, my question is ... Is this possible within the existing Wolfram Language machine learning Neural Networks framework? Is it possible to do it efficiently? The various ConvolutionLayers appear to shrink an image, but perhaps there are options that would prevent that. Anyway, the whole idea seemed intriguing to me and might have a lot of applications in both imagery, music and other fields where lossy compression and transmission was useful.

POSTED BY: Seth Chandler
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract