What are your ideas how to build a robust automated from-photo sketching tool?

Making cartoon-like avatars from photos is popular now on social media. But it is actually pretty old trick and related to animation industry. A Scanner Darkly and Waking Life, two of my favorite animated films, employed rotoscoping technique. The creators needed first to shoot a scene with real actors and then use special techniques and technology to transform it into animation. If you watch a short behind the scenes documentary, you will learn that shooting was fun while animation part was "living hell". Creators were trying to bring out the quintessential sum of a character and used special manual guides for rotoscoping visual elements of real footage. As they said "...what makes Winona (Ryder) look like Winona?.." I wanted to see if I can achieve something close with automated computation.

"Rotoscoping is an animation technique used by animators to trace over motion picture footage, frame by frame, when realistic action is required. Originally, photographed live-action movie images were projected onto a glass panel and re-drawn by an animator. This projection equipment is referred to as a Rotoscope. Although this device was eventually replaced by computers, the process is still referred to as Rotoscoping." We have to start obviously by making a computation that we can apply to a single frame of a film that will produce a nice sketch. There are limitless number of ways to approach this problem. I suggest the following starting function:

sketch[i_]:=
    With[{
       OilComic=ImageAdjust[
          ImageMultiply[
              ImageEffect[i,{"OilPainting",5}],
              ImageEffect[i,{"Comics",{.1,.13}}]
          ],{0,.5,.5}]
       },
       ImageCompose[OilComic,{Blur[OilComic,5],.5}]
    ]

To see the effect, import a short .GIF from giphy.com of Robert Downey Jr.:

frames = RemoveAlphaChannel /@ Import["https://media.giphy.com/media/wvrn7MMemcOB2/giphy.gif"];

Then make a new .GIF by processing every frame. Note quite useful here parallel computation over CPUs (4 in my case) as frame processing is independent.

comic = ParallelMap[ImageAssemble[{{sketch[#]}, {#}}] &, frames];
Export["comic.gif", comic]

Of course, very far from "A Scanner Darkly" but has its own charm. Let's what it does. First of all notice there are 3 image effects:

• Oil Painting
• Comics
• Orton

First 2 are obvious, while the last one, Orton, is hidden inside ImageCompose part. "The Orton effect is a photographic technique that adds a unique artistic expression to images. Originally it was created by overlaying two transparency slides of the same image, one of them being slightly out of focus. In digital image processing, blending an image with a blurred copy of itself gives the same effect." For me it gives more "paper" and "watercolor" feel and adds a sort of glow. Learn more from this demonstration.