Message Boards Message Boards

Plotting timeseries without gaps or interpolation

MODERATOR NOTE: This post contains parts written by kglr, which can be found here.


I am sharing this because it's a commonly occurring issue in financial time series, and other applications.

I have a time series sampled at 5-minute frequency for a financial instrument that trades from 08:30 to 15:15 CT. No trading takes place outside those hours.

I want to plot the series with the x-axis showing the series timestamps, but without gaps or interpolation. Neither DateListPlot nor ListLinePlot appears able to accomplish this.

The timeseries looks like this:

enter image description here

On a large enough timescale a DateListPlot looks fine:

enter image description here

However, on a shorter timescale the plot shows gaps that are either filled with (meaningless) interpolated values:

enter image description here

or else are plotted with a large gaps between days:

enter image description here

Nor is ListPLot much help:

enter image description here

What is required, of course, is to be able to restrict the plot range to the hours of 08:30 to 15:15, i.e. to redefine a "day" or use "session hours" rather than the standard 24-hour clock times.

A number of excellent solutions were offered on Stack Exchange that I reproduce in part here, as I believe they could be useful for fellow finance practitioners. I would encourage WR to encapsulate the functionality of at least one of them in a new, native WL function in a future release.


MODERATOR NOTE: The following part was written by kglr, and can be found here.

SOLUTIONS:

First an input dataset with a structure as described above

dates = Join @@ Table[DateRange[DateList[{2021, 1, i, 8, 30}], 
     DateList[{2021, 1, i, 17, 30}], Quantity[5, "Minutes"]], {i, 1, 4}];

SeedRandom[1]
data = Transpose[{dates, Accumulate[RandomReal[{-1, 3/2}, Length@dates]]}];

DateListPlot[data, ImageSize -> 600]

enter image description here

Use GroupBy to get a data set for each day:

datasets = Values[GroupBy[data, #[[1, ;; 3]] &]];

1) We can combine the separate DateListPlots for each day using Lukas Lang's ResourceFunction["PlotGrid"].

Use DateListPlot (specifying horizontal FrameTicks explicitly) for each dataset:

ClearAll[dateTicks]
dateTicks = Module[{minmax = MinMax[AbsoluteTime /@ #[[All, 1]]]}, 
    Select[First @ minmax <= #[[1]] <= Last @ minmax &]@
     System`DateListPlotDump`DateTicks[minmax, 6,
   {"YearShort", "/", "MonthShort", "/", "DayShort", " ", "Hour", ":", "Minute"}]] &;

yrange = MinMax @ data[[All, 2]];

dlps = Map[DateListPlot[#, PlotRange -> {Automatic, yrange}, 
     PlotRangePadding -> {Automatic, Scaled[.05]}, 
     FrameTicks -> {{Automatic, Automatic},
       {MapAt[Rotate[#, Pi/2] &, #, {All, 2}] & @ dateTicks @ #, 
        MapAt["" &, #, {All, 2}] & @ dateTicks @ #}}] &, datasets];

Combine the plots dlps with Lukas Lang's ResourceFunction["PlotGrid"]:

ResourceFunction["PlotGrid"][{dlps}, "MergeAxes" -> "Cut", 
 Spacings -> 10, ImageSize -> 600, AspectRatio -> 1/GoldenRatio]

enter image description here

2) Alternatively, we can use the function rowLayout from this answer:

    ClearAll[rowLayout]
    rowLayout[pad_: 0.02][opts : OptionsPattern[]] := 
      Module[{trFs, gridlines, xticks, 
         plts = Show[#, PlotRangePadding -> {Scaled[pad], Scaled[pad]}] & /@ #, 
         paddings = pad {-1, 1}.# & /@ (First /@ PlotRange /@ #), 
         shifts = {1., -1}.# & /@ Partition[Rest@Flatten[First /@ PlotRange /@ #], 2]}, 
        trFs = TranslationTransform[{#, 0}] & /@ 
          Accumulate[2 paddings + Prepend[shifts, 0]];
        gridlines = Most @ MapThread[#[{#2 + #3, 0}] &, {trFs, 
             PlotRange[#][[1, 2]] & /@ plts, paddings}][[All, 1]];
        xticks = Join @@ Table[MapAt[trFs[[i]][{#, 0}][[1]] &, (FrameTicks /. 
               Options[plts[[i]], FrameTicks])[[2, 1]], {All, 1}], {i,  Length@plts}];
        Graphics[MapThread[GeometricTransformation[#[[1]], #2] &, {plts, trFs}], 
         FrameTicks -> {{Automatic, All}, {xticks, MapAt["" &, xticks, {All, 2}]}}, opts, 
         GridLines -> {gridlines, None}, Frame -> True, 
         AspectRatio -> 1/3/GoldenRatio, ImageSize -> 800]] &;

    rowLayout[][ImageSize -> 600, AspectRatio -> 1/GoldenRatio] @ dlps

enter image description here

3) We can use TimeSeriesShift to shift the dates for each series in dataset and to construct custom date ticks:

    gap = DateDifference[datasets[[1, -1, 1]], datasets[[2, 1, 1]], 
        "Minutes"] - Quantity[60, "Minutes"];

    shiftedseries = Table[TimeSeriesShift[datasets[[i]], -(i - 1) gap], {i, 1, 4}];

    ticks = TimeSeries[dateTicks @ #] & /@ datasets;

    dticks = MapAt[Rotate[#, Pi/2] &, #, {All, 2}] &[
      Join @@ Table[TimeSeriesShift[ticks[[i]], -(i - 1) gap]["Path"], {i, 1, 4}]];

    DateListPlot[shiftedseries, FrameTicks -> {{Automatic, Automatic}, {dticks, Automatic}}]

enter image description here

POSTED BY: Jonathan Kinlay
4 Replies

Thanks Jonathan for sharing this approach with us. However on running the code I get a pink graph for solution (2) with the error messages: MapAt::argrx: MapAt called with 12 arguments; 3 arguments are expected.

MapAt::partw: Part {All,2} of MapAt[trFs$476065[[i]][{#1,0}][[1]]&,Charting`DateTicksFunction[None,{Ticks->{{3.81848*10^9,3.81849*10^9,3.8185*10^9,3.81851*10^9}},TickLabels->{{Rotate[21/1/1 10:00,Times[<<2>>]],Rotate[21/1/1 12:00,Times[<<2>>]],Rotate[21/1/1 14:00,Times[<<2>>]],Rotate[21/1/1 16:00,Times[<<2>>]]}},TickLengths->{Automatic},TicksStyle->{Automatic}}],{All,1},<<5>>,{All,1},trFs$476065[[i]][{#1,0}][[1]]&,<<2>>] does not exist.
POSTED BY: Michael Kelly
Posted 3 years ago

Hi Michael,

The solutions came from this post on MSE. The problem should be reported there.

POSTED BY: Rohit Namjoshi

enter image description here -- you have earned Featured Contributor Badge enter image description here Your exceptional post has been selected for our editorial column Staff Picks http://wolfr.am/StaffPicks and Your Profile is now distinguished by a Featured Contributor Badge and is displayed on the Featured Contributor Board. Thank you!

POSTED BY: EDITORIAL BOARD
Posted 3 years ago

Crossposted here.

POSTED BY: Rohit Namjoshi
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract