Message Boards Message Boards

Date-related functions: revision and performance comparison

Posted 3 years ago

GitHub repository: https://github.com/ben-izd/cDateFunctions

Recently, I was working on an infographic which heavily involves working with dates and times. After using DateObject and some date-related functions, it surprises me how slow they are (in the context of tens of thousands of samples). So I decided to rewrite the main one (DateDifference) in Wolfram language. But after seeing the performance boost gained by rewriting, I decided to write some other functions too. After a couple of months, I end up rewriting 19 functions in wolfram language and 9 in kernel-level (with Rust), in this post you'll see the result and interesting points I discovered in this journey.

Before we start, remember these notes:

  • Most of the implementations were done with minimum options, many of them do not support options that their built-in ones support (like TimeZone, TimeSystem, ...) and functions in the first section written in pure Wolfram language without using Compile or related functions

  • Default calendar is "Gregorian", which doesn't have year 0 (is also true for "ArithmeticPersian" calendar) unless it specified

  • In the 'Possible Issues/Bugs + Suggestions' section order is not important, some of them relate to the design decisions that developers decided (it would be awesome to shed some light on them)

  • All the implementations have the same name with 'c' added to start (DayName is cDayName, which has nothing to do with C language)

  • Some built-in functions were not Listable, which is understandable because of date formats like DateList, but some of my implementations are Listable

  • All the functions were tested with a wide range of random dates and numbers - Except some cases which will be discussed. The result of the functions with their built-in ones are equal. If you find any case with a wrong result, please comment it

  • Platform is Mathematica 13.0.0 for Microsoft Windows 10 20H2 (64-bit) on AMD Ryzen 1700 with 16 GB RAM with time-zone offset: +3.5

  • You can access the code in GitHub which also include the LibraryLink section (DLL files and their source code)

Performance Comparison

Here are the result of comparing c* functions with their built-in ones in terms of timing (RepeatedTiming were used). Tested on different input formats and the number shown here is the floored average of the result.

LeapYearQ

LeapYearQ Comparison

DayName

DayName Comparison

BusinessDayQ

BusinessDayQ Comparison

DayMatchQ

DayMatchQ Comparison

DayRound

DayRound Comparison

DateBounds

DateBounds Comparison

DateOverlapQ

DateOverlapQ Comparison

DateWithinQ

DateWithinQ Comparison

DayCount

DayCount Comparison

CurrentDate

CurrentDate Comparison

NextDate

NextDate Comparison

PreviousDate

PreviousDate Comparison

DateDifference

DateDifference Comparison

DayPlus

DayPlus Comparison

DatePlus

DayPlus Comparison DayPlus Comparison

DayRangee

DayRange Comparison

DateRange

DateRange Comparison

Possible Issues/Bugs + Suggestions

During testing and developing the code, I notice some strange/unexpected cases + some ideas which I'll discuss for each function.

LeapYearQ

1. "ArithmeticPersian" uses "Gregorian" formula

With CalendarType -> "ArithmeticPersian" input will be converted to "Gregorian" calendar and the result will be calculated like CalendarType -> "Gregorian" (note that they have different formulas):

LeapYearQ[{1403, 1, 1}, CalendarType -> "ArithmeticPersian"]
(* Out: True *)

LeapYearQ[{1403, 12, 1}, CalendarType -> "ArithmeticPersian"]
(* Out: False *)

The reason for above results is that date {1403, 1, 1} in "ArithmeticPersian" is {2024, 3, 20} in "Gregorian" which is a leap year in "Gregorian" but {1403, 12, 1} fall into {2025, 2, 19} which is not a leap year, according to "Gregorian" calendar.

More comprehensive test:

And @@ (
  LeapYearQ[{#}, CalendarType -> "ArithmeticPersian"] ===
     LeapYearQ[
      CalendarConvert[
       DateObject[{#}, CalendarType -> "ArithmeticPersian"], 
       "Gregorian"], CalendarType -> "Gregorian"] & /@ Range[1, 5000])

(* Out: True *)

Note that in "ArithmeticPersian" leap year gap (which has different methods to calculate, Mathematica uses 2820 period) is 4 and sometimes 5 years which is different than "Gregorian".

BusinessDayQ

1. Handling TimeObject without raising any message (related to AbsoluteTime-1 problem)

When using TimeObject in BusinessDayQ, it will be converted to a date and return a result! instead of raising an error:

DayName[]
(* Out: Thursday *)

BusinessDayQ[TimeObject[]]
(* Out: True *)

DayMatchQ

1. Missing/displace Veterans Day

Veterans Day which starts in 1938, on YYYY-11-11 except 1971-1978, which was on Oct 4th Monday [Source]. Considering observed holidays (holiday fall on Saturday/Sunday the day before/after will be observed day), a holiday could be either the specified day or a day before or after.

Select[Range[1938, 2050], Not@Or[DayMatchQ[{#, 11, 10}, "Holiday"],
    DayMatchQ[{#, 11, 11}, "Holiday"],
    DayMatchQ[{#, 11, 12}, "Holiday"]] &]

Result:

{1939, 1940, 1944, 1950, 1961, 1967, 1972, 1978, 1989, 1995, 2000, 2006, 2017, 2023, 2028, 2034, 2045}

which if you apply Differences, we'll get a pattern:

{1, 4, 6, 11, 6, 5, 6, 11, 6, 5, 6, 11, 6, 5, 6, 11}

Some years are missing the holiday and in the range of 1971-1978 it should not be on YYYY-11-11, but for some years it is.

Correct result (use cDayMatchQ instead of DayMatchQ):

{1971, 1972, 1973, 1974, 1975, 1976, 1977}

Also, it misses some dates in the future [Source]:

DayMatchQ[{2045, 11, 10}, "Holiday"]
(* Out: False *)

(* 1 day before/after also is not a holiday*)
DayMatchQ[{2045, 11, 9}, "Holiday"]
(* Out: False *)

DayMatchQ[{2045, 11, 11}, "Holiday"]
(* Out: False *)

2. Handling TimeObject (related to AbsoluteTime-1 problem)

When using TimeObject in DayMatchQ, it will convert it to a date and return the result ! instead of raising an error:

DayName[]
(* Out: Thursday *)

DayMatchQ[TimeObject[], Thursday]
(* Out: True*)

3. Documentation typo

Based on DayMatchQ documentation, default value for DayMatchQ is All while giving error to a single argument:

DayMatchQ[{2021, 11, 12}]

Error image

DateWithinQ

1. Does not support DateList format for input:

The documentation notes that input should be DateObject of any calendar, it's a good feature to support different calendars but what about DateList format? What's the reason behind it? Is it much different from DateDifference which supports this format?

DateWithinQ[{2021}, {2021, 1}]

Also, it should be noted that a *Q function in this example returned an Unevaluated expression instead of a Boolean, is this a normal behavior?

2. Returning Unevaluated expression without raising any message:

DateWithinQ[DateObject@{2020, 1, 1, 1, 1, 1}, DateObject@{2020, 1, 1}]

ERROR IMAGE

DayCount

1. Round results

If it's a day and more than half, it adds one day, otherwise, it doesn't (opposite of what documentation in Properties & Relations says, equal to the length of DayRange with some options).

DayCount[{2020, 1, 1}, {2020, 1, 1, 11}, All]
(* Out: 0 *)

DayCount[{2020, 1, 1}, {2020, 1, 1, 13}, All]
(* Out: 1 *)

cDayCount returns 0 for both of the above cases.

2. Wrong result in year 1

DayCount[{1, 12, 23}, {2, 3, 25}, "BeginningOfMonth"]
(* Out: 13 *)

NextDate

1. Inconsistent behavior

When using a granularity with an input that doesn't have enough precision, on most of the types, the upper-bound will be used as a start date to find the next occurrences but not for weekdays (not exactly the upper-bound is used).

NextDate[{2020, 1}, "Day"]
(* Out: DateObject[{2020,2,1}, "Day", "Gregorian", 3.5`] *)

NextDate[{2020, 1}, Sunday]
(* Out: DateObject[{2020,1,5}, "Day", "Gregorian", 3.5`] *)

Also, note that calendar type and time-zone offset was added to the output (input argument does not have those).

2. Handling TimeObject

NextDate support TimeObject. It can even give us the next Day of a TimeObject:

First@TimeObject[]
(* Out: {15, 38, 31.} *)

First@NextDate[TimeObject[], "Hour"]
(* Out: {2021, 11, 14, 16} *)

First@NextDate[TimeObject[], "Day"]
(* Out: {2021, 11, 15} *)

cNextDate result:

{16}

3. Wrong result in year 1 or -1

DateList@NextDate[{1, 12, 1}, "BeginningOfMonth"]
(* Out: {1, 2, 1, 0, 0, 0.} *)

DateList@NextDate[{-1, 12, 1, 0, 0, 0}, "Quarter"]
(* Out: {-1, 1, 1, 0, 0, 0.} *)

Correct result (cNextDate):

{2, 1, 1, 0, 0, 0.}

{1, 1, 1, 0, 0, 0.}

4. Inconsistent keyword in documentation

NextDate documentation uses "MonthFirstDay" and "MonthLastDay" instead of "BeginningOfMonth" and "EndOfMonth".

DateDifference

1. Type "quarter"/"Year" includes year 0

As noted in the beginning, the "Gregorian" calendar in Mathematica doesn't have year 0 but in these cases it does!

DateDifference[{-1, 9, 1}, {1, 9, 1}, "Quarter"]

(* Out: Quantity[9.96739, "QuarterYears"] *)

Correct Result (using cDateDifference):

Quantity[4., "QuarterYears"]

How many quarters do you see in this picture?

ERROR IMAGE

Another example:

DateDifference[{-1, 10, 1, 0, 0, 0}, {1, 1, 1, 0, 0, 0}, "Year"]

(* Out: Quantity[1.74795, "Years"] *)

2. Slightly different result

DateDifference[{2020, 3, 5, 15}, {2020, 7, 15, 5}, "Week"]
DateDifference[AbsoluteTime@{2020, 3, 5, 15}, AbsoluteTime@{2020, 7, 15, 5}, "Week"]

Result (see the last digits):

Quantity[18.797619047619047, "Weeks"]
Quantity[18.797619047619044, "Weeks"]

cDateDifference returns the first result. The real decimal can be calculated manually, which is 67/84 or:

0.7976190476190476

3. Raise error for multiple units including Decade /Century / Millennium

DateDifference[{2020, 1, 5}, {2120, 1, 4}, "Decade"]
DateDifference[{2020, 1, 5}, {2120, 1, 4}, {"Decade", "Year"}]
cDateDifference[{2020, 1, 5}, {2120, 1, 4}, {"Decade", "Year"}]

ERROR IMAGE

DayPlus

1. Does not support Week

It supports "BeginningOfMonth" and "EndOfMonth" but not "Week":

DayPlus[{2020, 1, 1}, 1, "Week"]

ERROR IMAGE

Possible with cDayPlus:

{2020, 1, 8}

DatePlus

1. Different results with Quantity and List

DatePlus[{2020, 1, 1}, {1, "Quarter"}]
(* Out: {2020, 4, 1} *)

DatePlus[{2020, 1, 1}, Quantity[1, "QuarterYears"]]
(* Out: {2020, 4, 1, 12} *)

2. Odd Behavior with year 1

DatePlus[{1, 8, 15}, {3, "EndOfMonth"}]
(* Out: {1, 3, 31} *)

Correct result (with cDatePlus):

{1, 10, 31}

Another example:

DatePlus[{1, 10, 5}, {2, "BeginningOfMonth"}]
(* Out: {1, 3, 1} *)

Correct result:

{1, 12, 1}

3. Strange result with an empty list

When we use an empty list ({}) as offset, it uses the first argument as day offset from now:

First@DatePlus[0, {}]
(* Out: {2021, 11, 13, 15, 7, 0.} *)

First@DatePlus[1, {}]
(* Out: {2021, 11, 14, 15, 7, 0.} *)

DatePlus[0, {1, "Day"}]
(* Out: 86400 *)

DayRange

1. Odd behavior with year 1 or -1

It starts from 1-1-31, just to avoid clutter, you'll see the length:

Length@DayRange[{1, 7, 9}, {2, 2, 15}, "EndOfMonth"]
(* Out: 13 *)

Length@DayRange[{-1, 7, 9}, {1, 2, 15}, "EndOfMonth"]
(* Out: 12 *)

Proper result (with cDayRange):

7

7

DateRange

1. The backward calculation is done using the forward formula

When the end date is larger than the start date, we add the granularity (should be positive) to the start until we reach the end, but if the start is bigger than the end and granularity is negative, we should subtract it from the start until we reach the end. It's true for all the types except "BeginningOfMonth" and "EndOfMonth".

DateRange[{2019, 12, 5}, {2019, 7, 1}, {-3, "BeginningOfMonth"}]
(* Out: {{2019, 10, 1, 0, 0, 0.}, {2019, 7, 1, 0, 0, 0.}} *)

Which should be (using cDateRange):

{{2019, 12, 1, 0, 0, 0.}, {2019, 9, 1, 0, 0, 0.}}

Pay attention to hour element:

(* Normal *)
DateRange[{2019, 12, 5, 12}, {2019, 7, 1}, {-3, "Month"}]
(* Out: {{2019, 12, 5, 12, 0, 0.}, {2019, 9, 5, 12, 0, 0.}} *)

(* Odd *)
DateRange[{2019, 12, 5, 12}, {2019, 7, 1}, {-3, "EndOfMonth"}]
(* Out: {{2019, 10, 31, 0, 0, 0.}, {2019, 7, 31, 0, 0, 0.}} *)

(* Odd *)
DateRange[{2019, 12, 5, 12}, {2019, 7, 1}, {-3, "BeginningOfMonth"}]
(* Out: {{2019, 10, 1, 0, 0, 0.}, {2019, 7, 1, 0, 0, 0.}} *)

In other terms, the result of the examples below are equal except in different ordering:

DateRange[{2019, 12, 5}, {2019, 7, 1}, {-3, "BeginningOfMonth"}]
(* Out: {{2019, 10, 1, 0, 0, 0.}, {2019, 7, 1, 0, 0, 0.}} *)

DateRange[{2019, 7, 1}, {2019, 12, 5}, {3, "BeginningOfMonth"}]
(* Out: {{2019, 7, 1, 0, 0, 0.}, {2019, 10, 1, 0, 0, 0.}} *)

AbsoluteTime

1. Handling TimeObject

AbsoluteTime convert TimeObject by adding Today value, which I think could be counterintuitive when somebody wants to compare AbsoluteTime of two TimeObject which was applied in different days, should use Mod[,86400] first.

DateList@AbsoluteTime@TimeObject[]
(* Out: {2021, 11, 13, 15, 42, 56.} *)

Why not return this result ( cAbsoluteTime):

{1, 1, 1, 15, 42, 56.}

UnixTime

1. Truncating sub-second precision:

Why UnixTime does not have floating numbers to handle more accurate dates?

DecimalForm[UnixTime[{2021, 11, 12, 12, 30, 30.35}], 30]
(* Out: 1636700430 *)

What it could be: 1636707630.35.

DateObject

1. Raise overlapping message for non-overlap inputs

For years, before 812 and after 2989 equality of two consecutive days raise Message while DateOverlapsQ returning False:

DateObject[{-1, 1, 1}] == DateObject[{-1, 1, 2}]

DateOverlapsQ[DateObject[{-1, 1, 1}], DateObject[{-1, 1, 2}]]
(* Out: False *)

DateOverlapsQ[DateObject[{-1, 1, 2}], DateObject[{-1, 1, 1}]]
(* Out: False *)

ERROR IMAGE

Far future example:

DateObject[{2988, 10, 12}] == DateObject[{2988, 10, 13}]
(* No Message, Out: False*)

(* But *)
DateObject[{2989, 10, 12}] == DateObject[{2989, 10, 13}]

ERROR IMAGE

2. Suggestion - Round 'Second' in Second granularity

Is this considered Second granularity or Sub-Second/Instant granularity ?

DateObject[{2020, 1, 1, 30, 45, 12.25}, "Second"]
(* Out: DateObject[List[2020,1,2,6,45,12.25`],"Second","Gregorian",3.5`] *)

3. Incorrect result with AbsoluteTime values with custom granularity

Consider date {2020, 12, 10} which is Thursday and every week starts on Monday (default option):

DateList@DateObject[3816547200, "Week"]
(* Out: {2020, 12, 10, 0, 0, 0.} *)

DateList@DateObject[DateList@3816547200, "Week"]
(* Out: {2020, 12, 7, 0, 0, 0.} *)

DateList@DateObject[FromAbsoluteTime@3816547200, "Week"]
(* Out: {2020, 12, 7, 0, 0, 0.} *)

This also applies to WeekBeginningSunday, Century, CenturyBeginning01, Millennium, MillenniumBeginning01.

TimeObject

1. Suggestion - Embed TimeZone offset to TimeObject like DateObject

Since DateObject uses $TimeZone and can be manipulated in a session while TimeObject uses system clock, why not use the time-zone offset from the system ? (My system time-zone offset is +3.5)

First@TimeObject[]
(* Out: {14, 4, 3.} *)

First@TimeZoneConvert[TimeObject[], "GMT"]
(* Out: {14, 4, 3.} *)

First@TimeZoneConvert[TimeObject[TimeZone -> 3.5], "GMT"]
(* Out: {10, 34, 3.} *)

(* Real answer *)
TimeZoneConvert[DateObject[], "GMT"][[1, -3 ;;]]
(* Out: {10, 34, 3.} *)

CalendarConvert

1. Negative/Zero in the month!

For far future years, conversion to "AstronomicalPersian" will give negative/zero months

First@CalendarConvert[DateObject@{9999997, 9, 24}, "AstronomicalPersian"]
(* Out: {9999386, -15, 5} *)

Another example (wrong output probably starts from this year):

First@CalendarConvert[DateObject@{82380, 7, 4}, "AstronomicalPersian"]
(* Out: {81758, 12, 29} *)

First@CalendarConvert[DateObject@{82380, 7, 5}, "AstronomicalPersian"]
(* Out: {81759, 0, 26} *)

If we don't apply First to the CalendarConvert you'll see:

ERROR IMAGE

2. Two consecutive leap years in the Persian calendar

Persian Calendar has a little complex leap year system, that's why it has an "ArithmeticPersian" and "AstronomicalPersian" option in calendar conversion. "AstronomicalPersian" type is the more accurate system. I don't think two consecutive leap years is correct. Also, I should mention, leap month in the Persian calendar is the 12th month which in normal years has 29 days, and leap years have 30 days.

First@CalendarConvert[DateObject@{622, 3, 21}, "AstronomicalPersian"]
First@CalendarConvert[DateObject@{621, 3, 21}, "AstronomicalPersian"]

If we calculate with cCalendarConvert:

{-1, 12, 30, 0, 0, 0.}

{-1, 1, 1, 0, 0, 0.}

TimeZoneConvert

1. Wrong result

For version 13 live stream, I want to schedule to watch it live, but I live in a different time-zone, so I notice this strange case:

First to show the real offset from my place to New_York:

TimeZoneOffset["America/New_York", "Asia/Tehran", 
 DateObject[{2021, 12, 13, 15, 30}, TimeZone -> "America/New_York"]]
(* Out: -8.5 *)

Now, with 8.5 hour offset, convert the date to the Persian calendar, then convert time zones:

First@TimeZoneConvert[
  CalendarConvert[
   DateObject[{2021, 12, 13, 15, 30}, TimeZone -> "America/New_York"],
    "AstronomicalPersian"], "Asia/Tehran"]

(* Out: {1400, 9, 22, 23, 51} *)

We get 8h 21m offset!

Further investigation, shows that the offset used in the calculation may differ from the real one:

First@TimeZoneConvert[
  DateObject[{1400, 9, 22, 3, 31, 15}, 
   CalendarType -> "AstronomicalPersian"], "Europe/London"]

(* Out: {1400, 9, 22, 0, 0, 0.} *)

If we change the destination to GMT the correct result shows up:

First@TimeZoneConvert[
  DateObject[{1400, 9, 22, 3, 30, 0}, 
   CalendarType -> "AstronomicalPersian"], "GMT"]

(* Out: {1400, 9, 22, 0, 0, 0.} *)

Bonus

If you have read this far, congratulations. These functions were absent in wolfram but are useful to have, so here you are:

cHolidayName

For getting the name of a holiday:

cHolidayName[{2021, 11, 25}]
(* Out: "Thanksgiving Day" *)

Observed Holidays, the original day is not considered a holiday:

cHolidayName[{2021, 7, 5}]
(* Out: "Independence Day' day off" *)

cHolidayName[{2021, 7, 4}]
(* Out: Missing["Holiday"] *)

Also if you're more interested in holidays, there are two related internal functions for getting next/previous holiday:

DateList@cDateFunctions`Private`cNextHoliday[{2021, 11, 12}]
(* Out: {2021, 11, 25, 0, 0, 0.} *)

DateList@cDateFunctions`Private`cPreviousHoliday[{2021, 11, 12}]
(* Out: {2021, 11, 11, 0, 0, 0.} *)

Also for getting the previous holiday, there is a lower limit:

cDateFunctions`Private`cPreviousHoliday[{1500, 11, 11}]

(* Message: "No official holiday exists before 1863-11-26." *)

cCalendarView

For getting a month/multiple month/custom range calendar views with Grid which supports highlighting and is only for "Gregorian" calendar (See Advance section for other calendars):

View a month at a glance:

cCalendarView[{2021, 11}]

CALENDAR VIEW

cCalendarView[{2021, 11}, {2021, 11, 12} -> Red]

CALENDAR VIEW

cCalendarView[{2021, 11}, 12 ;; 15]

CALENDAR VIEW

cCalendarView[{2021, 11}, {6, 7} ;; ;; 7]

CALENDAR VIEW

cCalendarView[{2021, 11}, "Holiday"]

CALENDAR VIEW

Use cCalendarMultipleView to view multiple months in a row. Could be positive or negative:

(* show 2021-11 and 1 month after that with highlighting *)
cCalendarMultipleView[{2021, 11}, 1, 6 ;; ;; 6]

CALENDAR VIEW

Use cCalendarRangeView to view a range:

cCalendarRangeView[{2021, 11}, {2021,12},
   {{{2021, 11, 14}, {2021, 11, 15}, {2021, 11, 16}} -> Gray,
    {2021, 11, 13} -> Green}]

CALENDAR VIEW

Note that cCalendarMultipleView and cCalendarRangeView automatically highlight the first day of each month.

Suggestion - N-th Weekday

How should we program to express 4th Thursday (for Thanksgiving ) / last Monday (for Memorial Day) ? Here is my proposal inspired by the C++ Chrono library, implemented via UpValues (minimum viable product):

(Unprotect[#];
    UpValues[#] = {};
    # /: {year_Integer, month_Integer, #[n_Integer?Positive]} := DatePlus[{year, month, 0}, {n, #}];
    # /: {year_Integer, month_Integer, #[n_Integer?Negative]} := DatePlus[{year, month + 1, 1}, {n, #}];
    # /: {year_Integer, month_Integer, #[First]} := DatePlus[{year, month, 0}, {1, #}];
    # /: {year_Integer, month_Integer, #[Last]} := DatePlus[{year, month + 1, 1}, {-1, #}];
    Protected[#];) & /@ {Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday};

Now, we can use it anywhere (should be fixed):

{2021, 11, Thursday[4]}
(* Out: {2021, 11, 25} *)

{2021, 5, Monday[Last]}
(* Out: {2021, 5, 31} *)

AbsoluteTime[{2021, 5, Monday[Last]}]
(* Out: 3831408000 *)

(* Second to last friday *)
{2021, 12, Friday[-2]}
(* Out: {2021, 12, 24} *)

Limiting UpValues to date functions only and supporting month names, would greatly improve functionality and readability.

For Advanced Users

This journey of rewriting date functions in wolfram as joyful and teachable as it was, was ending and after looking at my code, I notice in many places, DateList and AbsoluteTime were used many times, I thought, if I could make them faster, the whole package could benefit this speed up. In many parts, I reached my goal but I didn't test the performance gained from switching.

Since these functions are Kernel-Level functions (probably written in C language), Wolfram Language is not the right tool to compete, so I choose Rust. You could read about its benefits and guarantees over C on other sites. Here you'll see the comparison. C language is still used as the interface for Rust functions. (If you have a better solution, I'm more than happy to hear about Rust integration.)

This journey just like the previous one doesn't end with DateList and AbsoluteTime, but it also came with writing 3 calendars ("Greogiran", "ArithmeticPersian" with 5 method, AstronomicalPersian, "Islamic" which is a civil arithmetic calendar, with 4 method). For additional options they support, visit the Github Page.

Installation

For testing this section, you need to follow the Github instructions.

AbsoluteTime / FromAbsoluteTime

cAbsoluteTime is on average 0.9 times slower than AbsoluteTime over 1,000,000 random DateList.

cAbsoluteTime is on average the 2 times faster as AbsoluteTime over 100,000 random DateObject.

If we call AbsoluteTime to get the current value without any argument, cAbsoluteTime is on average 42 times faster than AbsoluteTime while having 6 times lower memory usage.

cFromAbsoluteTime on the other hand is 1.5 times faster than FromAbsoluteTime over 100,000 random real, which could be faster considering creating a DateObject overhead (see Comparing Internals section).

Also unlike Mathematica, cAbsoluteTime does support other calendars:

cDateList@cAbsoluteTime[{1400, 7, 20}, CalendarType -> "ArithmeticPersian"]
(* Out: {2021, 10, 12, 0, 0, 0} *)

cDateList@cAbsoluteTime[{1443, 3, 5}, CalendarType -> "Islamic"]
(* Out: {2021, 10, 12, 0, 0, 0} *)

DateList

cDateList is on average ~1.7 times faster than DateList over 1,000,000 random numbers (Real/Integer).

cDateList is on average ~1.9 times faster than DateList over 1,000,000 random un-normal DateList.

CalendarConvert

All the numbers come from testing 10,000 random samples.

In "Gregorian" to "ArithmeticPersian"\ cCalendarConvert is on average 16 times faster than CalendarConvert.

In "Gregorian" to "AstronomicalPersian"\ cCalendarConvert is on average 140 times faster than CalendarConvert.

In "Gregorian" to "Islamic"\ cCalendarConvert is on average 9 times faster than CalendarConvert.

In "Islamic" to "ArithmeticPersian"\ cCalendarConvert is on average 19 times faster than CalendarConvert.

"AstronomicalPersian" uses an algorithm introduced by Edward M. Reingold, Nachum Dershowitz in their book, "Calendrical Calculations: The Ultimate Edition". It was implemented to perform the test but because of its license, it's not included in the source code until I get approval from the author (the source is commented in the interface code).

Since both "Islamic" and "ArithmeticPersian" have methods to use, converting one to another while using Method option, input Method will be used for the output format and input will be converted using the default method, if you want more control, first convert it to "Gregorian", then convert it to your specified calendar.

LeapYearQ

Because of different methods of calculations, this LeapYearQ uses kernel-level functions (instead of pure wolfram language shown earlier, which will be overridden) to handle 3 calendars. Tests were done on over 100,000 random samples.

"Gregorian"\ cLeapYearQ is on average 8 times faster than LeapYearQ.

"ArithmeticPersian"\ cLeapYearQ is on average 9 times faster than LeapYearQ (as discussed earlier, LeapYearQ gives the wrong result for this type of calendar).

"AstronomicalPersian"\ cLeapYearQ is on average 102 times faster than LeapYearQ.

"Islamic"\ cLeapYearQ is on average 6 times faster than LeapYearQ.

And remember in cLeapYearQ with "ArithmeticPersian" and "Islamic", you can choose different methods of leap-year calculation, visit Github for more information.

JulianDate / FromJulianDate

cJulianDate is on average 3 times faster than JulianDate over 10,000 random DateList.

cFromJulianDate is on average 1.8 times faster than FromJulianDate over 100,000 real, actually, it's faster than stated, creating a DateObject takes most of the time.

UnixTime / FromUnixTime

cUnixTime is on average 7 times faster than UnixTime over 100,000 random DateList samples.

cFromUnixTime is on average 2 times faster than FromUnixTime but it shares the same problem as cFromJulianDate.

cMonthView / cPartialView

With access to these calendars and different methods of calculation, it was the best opportunity to create a function that prints month/year view.

See 2021 calendar at a glance:

YEAR_IMAGE

or other calendars (use CalendarType)

OTHER<em>YEARIMAEG

Unlike earlier cMonthView it does not support highlighting yet. For printing a range/month continuously use cPartialView which accepts a start and end date.

Wolfram technology conference at a glance (with manually highlighting):

MapAt[Highlighted, 
 cPartialView[{2021, 11, 7}, {2021, 11, 20}], {{1, 1, 1, 6 ;; 7}, {1, 
   1, 2, 1 ;; 2}}]

cPartialView_IMAGE

Limitations

Since all these functions are written in low-level and should support C types, not arbitrary precision that Wolfram-Language has (although Wolfram has its own limitation, like rounding error).

cAbsoluteTime:

cAbsoluteTime has a limit, going further will not raise any error but cause overflow (wrap around).

(* max date - second is not important *)
cAbsoluteTime[{292277026526, 12, 5, 15, 30}]
(* Out: 9223372036854720000 *)

(* going further lead to overflow *)
cAbsoluteTime[{292277026526, 12, 5, 15, 31}]
(* Out: -9223372036854775756 *)

(* min date*)
cAbsoluteTime[{-292277022728, 1, 27, 0, 0, 0}]
(* Out: -9223372036854720000 *)

(* going further lead to overflow *)
cAbsoluteTime[{-292277022728, 1, 26, 0, 0, 0}]
(* Out: 9223372036854745216 *)

cDateList:

Unlike cAbsoluteTime which crossing the limit will not raise any error, here Mathematica will raise an error because it can not convert numbers greater than 2^63-1 to machine size 64-bit integers. But because of some internal design decisions, the safe range is shifted towards the negative side. So there is also a range before 2^63-1, which doesn't raise any error but leads to overflow. (We're discussing around year 292,277,024,627, which if you're interested, I'll add more details.)

This pattern also applies to other kernel-level functions.

Comparing Internals

In the built-in Internal context, there are three functions that do the DateList functionalities separately:

  • DateListToDateList

  • DateListToSeconds

  • SecondsToDateList

Like Mathematica, c* functions also have kernel-level functions to call but there is a caveat. Since passing arrays to/from C language (interface), should be a single type, I couldn't find an easy way to handle DateList mix-type arrays beside WSTP (second could be Real, LibraryLink developers, if you're reading, I'm waiting for integration of TypeProduct to use structs), so these kernel functions only support maximum 5 elements as input/output ({Y,M,D,H,M}, second will be calculated in the interface, not in the kernel part).

Comparing Mathematica Internal with cDateFunctions > LibraryLink over random 1,000,000 samples without considering second (Internal support second but not cDatefunctions Kernel):

DateListToSecond (AbsoluteTime): LibraryLink is on average 0.7 times slower than Internal.

SecondToDateList (DateList): LibraryLink is on average 0.8 times faster than Internal.

DateListToDateList: LibraryLink is on average 1.12 times faster than Internal.

Calling raw LibraryLink outperforms the internals but input/output doesn't include second, the result shown above came from calling the simplest interface. (For those curious people, calling raw interface is on average 8, 7, 9 times faster respectively).

As discussed, cDateFunctions`LibraryLink is fast but the reason that their interface is not as fast as their kernel, is mainly because of the pattern matching (to support CalendarType and Method) and second calculation.

Final note

For those who are interested in calendars and their implementations, I recommend "Calendrical Calculation" by Edward M. Reingold, Nachum Dershowitz which I think the Wolfram/Microsoft team has seen this book because of some similarities I noticed in their sources. Both book's authors and developer teams did a great job which is appreciated (Microsoft: .Net). The book also comes with a source file in Lisp language. The implementation of cCalendarConvert "AstronomicalPersian" is ported from this book.

Because of my limited time, I did cover 4 calendars and introduced different methods of calculation to cover different needs and you should know both "Persian" and "Islamic" in some countries use observation not algorithms to determine leap year/month length.

The Rust code has a framework within, to support other calendars with similar characteristics as Gregorian and ... , with providing months length and two functions for checking and counting leap years and some little details, you'll have AbsoluteTime and DateList engine + Month/YearView (currently for 12 months calendars) for your calendar for free.

"ArithmeticPersian" and "Islamic" also use frameworks, so for any future changes in their leap year calculations that follow the old structure, you can add them as a new method under 20 lines of code.

Improving is a never-ending process. For me, I reached beyond the main goal I set for myself, of course, there were a lot of moments which I was so close to abandoning the project, especially when I was figuring out how LibraryLink works and build the project without compiling errors but looking back, it's worth it and I feel more powerful than before. I hope you learn something and if you had any questions/comments/suggestions, feel free to add them.

6 Replies

Thanks for the suggestion, I think because of having multiple functions + LibraryLink files, it's a perfect candidate to use the upcoming paclet repository to activate the "Time-Related Computation" section.

Thanks for putting this analysis together Benjamin, I've gone through the details you put together as well as the github repository you linked.

As you noted in the description, some of the differences are down to extra features like time zone/time system/calendar handling (Calendrical Calculations is actually the basis of a number of the in-built algorithms for some of the supported calendars...which you may have already guessed). Things like DayCountConvention and HolidayCalendar also add compexity to arithmetic operations.

..but having said that the most common cases for these are still substantially slower than they need to be, which is a particular problem for arrays of dates. Optimization is an on-going task, but one we hope to have greatly improved in the coming release, as well support in future features like dedicated date arrays for storing and operating on large collections of dates at once.

If you have any additional questions or feedback please feel free to note it here, or contact me directly, and I'll be happy to talk in more detail.

-Nick Lariviere Kernel Developer at Wolfram Research Inc

POSTED BY: Nick Lariviere

Thanks for the recognition.

Since you're more familiar with the wolfram communications system, should I report the "Possible Issues/Bugs" section separately, or has it been reported? What is your suggestion?

Posted 3 years ago

Hi Benjamin,

Very useful, thanks for sharing! You should consider submitting these to the Wolfram Function Repository.

POSTED BY: Rohit Namjoshi

enter image description here -- you have earned Featured Contributor Badge enter image description here Your exceptional post has been selected for our editorial column Staff Picks http://wolfr.am/StaffPicks and Your Profile is now distinguished by a Featured Contributor Badge and is displayed on the Featured Contributor Board. Thank you!

POSTED BY: EDITORIAL BOARD

Thanks for sharing! I wonder why Wolfram did not improve it themselves. I once logged a case about slowness and I don't think I'm the only one. Great work!

POSTED BY: l van Veen
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract