Message Boards Message Boards

How to adjust an image by sculpting its gamut

Meet a colorful character, Mr. Helmet Hornbill:

enter image description here

Mr. Hornbill is likely pleased with how his pictures get adjusted day after day. Ah... artistically pleasant photo retouching. Well, sure, but not today, Mr. Hornbill.

This picture of Mr. Hornbill was already stretched so its Red, Green and Blue color values range from 0 to 1. Or maybe Mr. Hornbill did not need anything like that, because maybe he is naturally stretched:

img = Import@"https://cdn.pixabay.com/photo/2013/08/23/14/17/animal-175033_1280.jpg";
In[222]:= ImageMeasurements[img, {"Min", "Max"}]

Out[222]= {{0., 0., 0.}, {1., 1., 1.}}

Anyhow, things get nicer and sharper for Mr. Hornbill before they get worse. A composition of itself with its flattened channel version looks flattering:

ImageCompose[img, {HistogramTransform[img], .5}]

enter image description here

Switching colors with a rooster by transferring color distributions channel by channel: Not so nice. (Images' backgrounds contribute to the distributions.)

HistogramTransform[ColorConvert[img, "LAB"], ColorConvert[rooster, "LAB"]]

enter image description here

And now... Mr. Hornbill will serve my purpose of demonstrating how to change the 3D distribution of its colors in the Lab space. The target 3D distribution? A non-pleasing one: the sRGB colorspace gamut, treated as if all its colors are equiprobable. This is where we start. In the final image, colors will spread the entire volume (sRGB gamut) uniformly.

ChromaticityPlot3D[{"sRGB", img}]

enter image description here

How can I do that to Mr. Hornbill? Step by step. Like a sculptor, I reshape the color distribution in each of the 3D dimensions, then rotate randomly the distribution and reshape again. And again, until convergence. Mr. Hornbill might say is a victim of a multidimensional PDF transfer by iteratively reshaping random marginal histograms.

Before that, I hack my way to define a 3D, empirical, uniform distribution for the sRGB gamut. Once I grabbed points on the 6 "faces" of the gamut, I compute the convex hull, then sample on a grid the 3D space around the convex hull so as to only retain these 3D points inside the convex hull.

c = ChromaticityPlot3D["sRGB", "LAB"];
points3D = c[[1, 1, All, 1]];
chm = ConvexHullMesh[Flatten[points3D, 1]];
ptsinside = {};
mf = RegionMember[chm];
Do[
  If[mf[{l, a, b}],
       AppendTo[ptsinside, {l, a, b}]
   ],
  Evaluate[Sequence @@ pspec]
  ];

This gives about 62 thousands points. And now, the iterative sculpting:

multidimensionalReshape[source_, ref_, \[Epsilon]_: 10^-10, maxiterations_: 200] :=
  FixedPoint[
   With[{R = randomRotationMatrix[Length[source]]},
     Transpose[R].HistogramTransform[R.#, R.ref]] &,
   source, maxiterations, 
   SameTest -> (RootMeanSquare[Norm /@ Transpose[#1 - #2]] < \[Epsilon] &)];

randomRotationMatrix[n_] := 
  If[Det@# == 1, #, randomRotationMatrix[n]] &[
   Transpose@#.DiagonalMatrix[Sign@Diagonal@#2] & @@ 
    QRDecomposition@RandomReal[NormalDistribution[], {n, n}]];

pix = Flatten /@ ImageData[ColorConvert[img, "LAB"], Interleaving -> False];
new = multidimensionalReshape[pix, Transpose@ptsinside];
adjusted = Image[Partition[#, First@ImageDimensions@img] & /@ new, Interleaving -> False, ColorSpace -> "LAB"];

The result looks quite good enough, pixels are spread all around inside the volume. I.e. we sculpted the initial convex hull into the shape of the sRGB gamut. enter image description here

Mr. Hornbill stays cool under his new colors: enter image description here

POSTED BY: Matthias Odisio
4 Replies

Thanks for the link! Yes, style transfer can yield pretty cool results. Such deep learning methods are trainable with the Wolfram Language. Soon enough there will be a net model doing that in Mathematica.

POSTED BY: Matthias Odisio
Posted 8 years ago

You probably like to read this article and code at Github.

Maybe we can approach with Mathematica 11.1 enhanced neural network ?

Deep Photo Style Transfer enter image description here

Article PDF and source code links.

https://arxiv.org/abs/1703.07511

https://github.com/luanfujun/deep-photo-styletransfer

Attachments:
POSTED BY: Frederick Wu

enter image description here - Congratulations! This post is now a Staff Pick! Thank you for your wonderful contributions. Please, keep them coming!

POSTED BY: EDITORIAL BOARD

Very neat way of enhancing colors, nice trick!

POSTED BY: Sander Huisman
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract