Message Boards Message Boards

5
|
12726 Views
|
3 Replies
|
9 Total Likes
View groups...
Share
Share this post:

Directional quantile envelopes

Posted 10 years ago

The idea of directional quantile envelopes is conceptually simple (and straightforward to come up with). The calculation is also relatively simple: over a set of uniformly distributed directions we find the lines that separate the data according a quantile parameter q, 0 < q < 1, and with those lines we approximate the enveloping curve for data that corresponds to q.

Here is a visual aid in support of this statement. The points in blue are the data points, the points in red are the quantile points, the points in black are the envelope approximation points determined by the intersections of the quantile lines in red.

40DirectionalQuantileLinesFor95

Below are couple of application examples. The grid lines of the plots are derived from the quantiles of the x and y coordinates.

enter image description here

enter image description here

Further discussion, algorithm details, references, and examples are given in my recent blog post "Directional quantile envelopes": http://mathematicaforprediction.wordpress.com/2014/11/03/directional-quantile-envelopes/

The function QuantileEnvelope is included in the package QuantileRegression.m available at GitHub: https://github.com/antononcube/MathematicaForPrediction/blob/master/QuantileRegression.m

I became interested in this type of 2D quantile regression because I was looking for a tool for finding 2D outliers. Obviously, the technique can be generalized in higher dimensions.

POSTED BY: Anton Antonov
3 Replies

Would it be possible to do quantiles from a Pareto frontier? In other words, find points which pareto dominate x % of all the points?

POSTED BY: Frank Kampas

I would say no. But I think directional quantiles can be used to restrict the candidates for Pareto frontier points.

POSTED BY: Anton Antonov

The algorithm can be easily extended to 3D and higher dimensions, and I decided to illustrate that with an example. The 3D implementation is really easy using Mathematica's geometric computation functions (i) for derivation of rotation matrices and (ii) for region manipulation. (The latter are new in version 10.)

Let us first generate some data.

npoints = 10000;
data = {RandomReal[{0, 2 \[Pi]}, npoints], 
   RandomReal[{-\[Pi], \[Pi]}, npoints], 
   RandomVariate[PoissonDistribution[4.8], npoints]};
data = MapThread[#3*{Cos[#1] Cos[#2], Sin[#1] Cos[#2], Sin[#2]} &, data];

Let us plot the data:

Block[{qs = 12},
 qs = Map[Quantile[#, Range[0, 1, 1/(qs - 1)]] &, Transpose[data]];
 ListPointPlot3D[data, PlotRange -> All, PlotTheme -> "Detailed", 
  FaceGrids -> {{{0, 0, -1}, Most[qs]}, {{0, 1, 0}, 
     qs[[{1, 3}]]}, {{-1, 0, 0}, Rest[qs]}}, ImageSize -> 700]
 ]

enter image description here

(On the plot the grid lines derived from the quantiles of x, y and z coordinates of the data set.)

Using the set of uniform angles:

nqs = 20;
dirs = N@Flatten[
    Table[{Cos[\[Theta]] Cos[\[Phi]], Sin[\[Theta]] Cos[\[Phi]], 
      Sin[\[Phi]]}, {\[Theta], 2 \[Pi]/(10 nqs), 2 \[Pi], 
      2 \[Pi]/nqs}, {\[Phi], -\[Pi], \[Pi], 2 \[Pi]/nqs}], 1];

enter image description here

we can compute the quantiles of the z-coordinates in each direction:

rmats = RotationMatrix[{{1, 0, 0}, #}] & /@ dirs;

qs = {0.95};
qDirPoints = 
  Flatten[Map[Function[{m}, Quantile[(m.Transpose[data])[[3]], qs]], rmats]];

and produce the quantile envelope surface using ImplicitRegion and BoundaryDiscretizeRegion:

enter image description here

More details and examples of usage are given in my blog post: "Directional quantile envelopes in 3D" . Also, see the attached notebook.

Attachments:
POSTED BY: Anton Antonov
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract