hyperreview
October 23, 2020, 1:32pm
1
I’m not very experienced with this part of Mathematica so please bear me.data set consisting of two columns, the first column is a date and the second is a numeric cumulative amount. I tried to create a time series, resample it and smooth, but I couldn’t manage to smooth it enough. I’m a noob with Mathematica. Can you help me to understand what I’m doing wrong?
data = Import[
"https://1drv.ms/x/s!AiF0MVfYzFaAi3cO044kGlJW0Oam?e=Mt7qTv"][[1]];
data2 = data[[2 ;;, {1, 2}]]
DateListPlot[data2]
ts = TimeSeries[data2,
ResamplingMethod -> {"Interpolation", InterpolationOrder -> 1}]
tr = TimeSeriesResample[ts, "Day",
ResamplingMethod -> {"Interpolation", InterpolationOrder -> 1}]
MovingMap[Median, tr, Quantity[3, "Months"]]
quotient[values_, times_] :=
First[Differences[values]/Differences[times]]
mm =
MovingMap[quotient[#BoundaryValues, #BoundaryTimes] &,
tr, {.1, Right}]
DateListPlot[mm]
mikebesso
October 23, 2020, 8:35pm
3
Duke:
I’ve been doing a bit of resampling lately. Though, oddly enough, my use case requires the resampling method that you are using. To get smoother, I suggest you try other ResamplingMethods.
Here is what I’ve learned from my use case:
{"Interpolation", InterpolationOrder -> 1}
Will give you a straight line between our observation points.
{"Interpolation", InterpolationOrder -> 0}
Will give assume that the observed values remain unchanged until your next observation. This gives you a stair step curve.
As I understand it, if you increase the InterpolationOrder, then you will start using curves of the order specified as opposed to lines.
You might also want to try other “ResamplingMethods”, but there is a good chance you just need to increase your InterpolationOrder.
I hope that helps.
Have a great and safe weekend.
hyperreview
October 24, 2020, 1:32am
4
Thanks, I will try with some different values.
danl
October 24, 2020, 2:01pm
5
That Import gives me an empty string. Perhaps you could add the x-y pairs to the post, or put them in a notebook and attach it to the post?
hyperreview
October 24, 2020, 3:58pm
6
Uncompress[“1:eJyt3X18z/X+x/
would work?
Here comes a simple approach using MeanFilter - I personally like this function because it preserves the values of the “x-axis” i.e. date values. Maybe this works for you (let data be your data from above):
ts = TimeSeries@data;
mfts = MeanFilter[ts, Quantity[4, "Weeks"]];
deriv0 = Differences[mfts]; (* quick and dirty method! *)
deriv1 = MeanFilter[deriv0, Quantity[4, "Weeks"]];
DateListPlot[{ts, mfts}, PlotStyle -> {Green, Red, Blue}, GridLines -> Automatic, ImageSize -> Large, PlotLabel -> "orig. values and smoothed values"]
DateListPlot[{deriv0, deriv1}, PlotStyle -> {Green, Red, Blue}, GridLines -> Automatic, ImageSize -> Large, PlotLabel -> "(deriv. of smoothed vals.) and (smoothed deriv. of smoothed vals.)"]
mikebesso
October 24, 2020, 8:20pm
8
Thanks Henrik. I had not learned the MeanFilter[] yet. It will likely help me in my next project.
hyperreview
October 24, 2020, 10:10pm
9
Thanks, this is really what I was looking for
Dear all,
in my code above I calculated the derivative like so:
deriv0 = Differences[mfts]; (* quick and dirty method! *)
and yes - maybe this is a bit too quick and dirty! But TimeSeries has a lot of nice properties to offer: E.g. “PathFunction” gives an InterpolatingFunction, from which easily the derivative can be calculated:
Plot[mfts["PathFunction"]'[t], {t, mfts["FirstTime"], mfts["LastTime"]}, ImageSize -> Large, GridLines -> Automatic]
Here the abscissa is in terms of AbsoluteTime, i.e. the number of seconds starting from 01.01.1900. Consequently the unit of the derivative is (unit of your data)/seconds.
This then can be resampled (here e.g. in 3-day-interval) and converted back into a TimeSeries for further smoothing:
sampleDates = DateRange[FromAbsoluteTime@mfts["FirstTime"], FromAbsoluteTime@mfts["LastTime"], Quantity[3, "Days"]];
deriv = TimeSeries[{#, mfts["PathFunction"]'[AbsoluteTime@#]} & /@ sampleDates];
derivSmooth = MeanFilter[deriv, Quantity[4, "Weeks"]];
DateListPlot[{deriv, derivSmooth}, ImageSize -> Large, GridLines -> Automatic, PlotStyle -> {Green, Red}]
This definitely is a cleaner solution. Regards – Henrik
Consider the answer to the same question posted on MathematicaStackExchange which uses kernel regression.
