Group Abstract Group Abstract

Message Boards Message Boards

0
|
10.6K Views
|
6 Replies
|
4 Total Likes
View groups...
Share
Share this post:

Upper and lower bounds of data

Posted 10 years ago
POSTED BY: patrick
6 Replies
Posted 10 years ago
POSTED BY: patrick

Hi Patrick,

ok, I can see the problem now. I have a very different suggestion that involves the new and experimental function FindFormula. I must admit that this is an absolutely magic function for me, but it is very useful. In your example you have the data in the variable A and you "guess" the function that should approximate the data. Your guess is of the form:

a + b*(Cos[\[Theta]])^2 

Now, in general we don't know that and also FindFormula is able to find a much better approximation. After your parameter estimation you get something like this:

enter image description here

Now, let's try what FindFormula does:

FindFormula[A][x]

enter image description here

Looking at this shows that the polynomial approximation is surprisingly good.

Show[ListPlot[A], Plot[FindFormula[A][x], {x, 0, 1.6}]]

enter image description here

It is practically impossible to see deviations. Now let's try to find upper and lower bounds.

eqn = FindFormula[A][x];
deviation = Max[Abs[((eqn /. x -> #[[1]]) - #[[2]])] & /@ A];
Show[ListPlot[A], Plot[{eqn, eqn + deviation, eqn - deviation}, {x, 0, 1.6}]]

enter image description here

The important bit is the deviation formula. It looks for the largest deviation of all data points from the estimated curve. The figure contains four curves: the data, the estimated equation, the upper and the lower bound. You can be sure that the upper and lower bounds are larger/equal and lower/equal to the deviations.

I think that the crucial point is when you say "and it fits well with the following function". If you have any reason to believe that this should be the function to fit (i.e. you have a theory that tells you so) then you can proceed like you did. If you do not have a theory and just want a good upper/lower bound you might want to do what I suggest. Also, you could of course look for deviations from the eqn separately for points that are higher and the ones that are lower. You would get one of the two estimates slightly closer to eqn.

Does this help?

Cheers,

Marco

PS: The following is unrelated to your problem, but I hope it illustrates what I mean by magic. Suppose you have data from a sin function and apply FindFormula:

formula = FindFormula[Table[Sin[0.1 x] + 0.2, {x, 1, 100}]][x]

This gives:

enter image description here

which looks really horrible. Nothing at all like a Sine function; there is a Log function in there and we obtain a weird piecewise function. Now the magic is this:

Plot[{Sin[0.1 x] + 0.2, formula}, {x, 0, 100}]

enter image description here

which is utterly surprising for me. That strange piecewise function is a very accurate representation of the sin function in that interval. I would really like to see how the Wolfram Language comes up with that! If someone of the Wolfram team could comment on this and indicate what is going on, that would be really interesting.

POSTED BY: Marco Thiel
Posted 10 years ago
Attachments:
POSTED BY: patrick

Hi Patrick,

can you post the data an graphic of what you want to achieve or have achieved?

Cheers,

Marco

POSTED BY: Marco Thiel
Posted 10 years ago

Hi Marco,

Thanks for your answer. I think I should be more pricise in my question, as you said. The thing is that I have some random values, I found their maxima (and minima), and I want to find the curve(s) that is(are) upper(lower) bound of this(these) maxima(minima). By using "FindFit", I can have an idea of what I am looking for, but I want that all the points must be below/above the curves.

POSTED BY: patrick

Hi,

I am not sure whether I understand your question. If you have data and you take the max and the min you would get upper and lower bounds.

data = RandomVariate[NormalDistribution[], 100]; \[Epsilon] = 
 0.1*Variance[data];
ListPlot[data, 
 Epilog -> {Red, 
   Line[{{0, Max[data] + \[Epsilon]}, {Length[data], 
      Max[data] + \[Epsilon]}}], Red, 
   Line[{{0, Min[data] - \[Epsilon]}, {Length[data], 
      Min[data] - \[Epsilon]}}]}]

enter image description here

I believe that this technically solves your problem. I generate two curves, which bound the time series. But this appears to be too easy to be your problem. So probably your are interested in something they call "the envelop". Perhaps this post helps you. You might also like this demonstration, which contains this piece of code:

Manipulate[
 ListLinePlot[{data, 
   EstimatedBackground[data, \[Sigma], 
    Method -> {"SNIP", r}], -EstimatedBackground[-data, \[Sigma], 
     Method -> {"SNIP", r}]}, AspectRatio -> 1/2, ImageSize -> Medium,
   PlotStyle -> {Automatic, {Orange, Thick}, {Orange, Thick}}, 
  Filling -> {2 -> {3}}], {{\[Sigma], 10}, 2, 30, 1}, {r, 0, 20},
 Initialization :> (data = 
    WolframAlpha[
      "sunspot", {{"SunspotsPartialTimeSeries:SpaceWeatherData", 1}, 
       "TimeSeriesData"}][[All, 2]])]

enter image description here

This suggests that you need to be more specific regarding your objective. There are parameters in here ("how smooth do I want the envelope to be?"), which need to be included in the original question.

Note that the core of the animation is the EstimatedBackground function which is new in MMA10.

ListLinePlot[{data, 
  EstimatedBackground[data, 5, 
   Method -> {"SNIP", 0.1}], -EstimatedBackground[-data, 5, 
    Method -> {"SNIP", 0.1}]}, AspectRatio -> 1/2, ImageSize -> Large,
  PlotStyle -> {Automatic, {Orange, Thick}, {Orange, Thick}}, 
 Filling -> {2 -> {3}}]

enter image description here

I hope that this helps you to find a solution to your problem.

Cheers,

Marco

POSTED BY: Marco Thiel
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard