https://community.wolfram.com RSS Feed for Wolfram Community showing ideas tagged with Packages sorted by most likes. https://community.wolfram.com/groups/-/m/t/1931315 [![enter image description here][2]][1] &[Wolfram Notebook][3] [1]: https://youtu.be/rOa5IntICFA [2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=ScreenShot2020-04-09at12.54.33PM.png&userId=11733 [3]: https://www.wolframcloud.com/obj/afe2a2fb-ee55-4df5-a6fb-9bc16dd08af7 Brenton Bostick 2020-04-09T15:04:38Z https://community.wolfram.com/groups/-/m/t/560469 *WOLFRAM MATERIALS for the ARTICLE:* > Szabolcs Horvát, Jakub Podkalicki, Gábor Csárdi, Tamás Nepusz, Vincent Traag, Fabio Zanini, Daniel Noom, (2022). > *IGraph/M: graph theory and network analysis for Mathematica*. > arXiv:2209.09145 **[physics.soc-ph]**. > https://doi.org/10.48550/arXiv.2209.09145 [](https://igraph.discourse.group) [](https://github.com/szhorvat/IGraphM/releases) [](https://github.com/szhorvat/IGraphM#contributions) [](https://doi.org/10.5281/zenodo.1134932) ---- ##Article abstract IGraph/M is an efficient general purpose graph theory and network analysis package for Mathematica. IGraph/M serves as the Wolfram Language interfaces to the igraph C library, and also provides several unique pieces of functionality not yet present in igraph, but made possible by combining its capabilities with Mathematica's. The package is designed to support both graph theoretical research as well as the analysis of large-scale empirical networks. ---- ##Introduction The post below was written for the original release of IGraph/M. The package has come a long way since then and now contains ~300 functions. See http://szhorvat.net/mathematica/IGraphM for more details on the current release. Compatibility: 64-it Windows/macOS/Linux or Raspberry Pi; Mathematica <del>10.0</del> 11.0 or later. <a href="http://szhorvat.net/mathematica/IGraphM"><img src="https://community.wolfram.com//c/portal/getImageAttachment?filename=IGraphM-ad-3.png&userId=38370" width="300"></a> ---- I would like to announce IGraph/M, a new igraph interface for Mathematica: http://szhorvat.net/mathematica/IGraphM [igraph](http://igraph.org/) is a graph manipulation and analysis package. IGraph/M makes its functionality available from Mathematica. This initial release, version 0.1, covers only some igraph functions, as I focused on the things that I need personally. However the main framework is complete, and new functions can be added quickly. If anyone would like to contribute, please contact me. Binary packages for OS X (10.9 or later) and Linux can be downloaded [from GitHub](https://github.com/szhorvat/IGraphM/releases). Unfortunately, I was unable to compile the development version of igraph for Windows, so I cannot provide a Windows version. If you can help with compiling igraph itself (not IGraph/M) on Windows, please let me know! Functionality in this release that is not built into Mathematica: * Vertex betweenness centrality for weighted graphs * Estimates of vertex betweenness, edge betweenness and closeness centrality; for large graphs * Minimum feedback arc set for weighted and unweighted graphs * Find all cliques (not just maximal ones) * Count 3- and 4-motifs * Rewire edges, keeping either the density or the degree sequence * Alternative algorithms for isomorphism testing: Bliss, VF2 * Subgraph isomorphism * Test if a degree sequence is graphical * Alternative algorithms for generating random graphs with given degree sequence * Layout algorithms that take weights into account Note that IGraph/M is *not a replacement* for Mathematica's graphs and networks functionality. It is meant to complement what is already available in Mathematica, thus it primarily focuses on adding functionality that is not already present. Why did I release the package before covering most of the igraph functionality? I do not have time to work on things I do not personally need or use, so I am unlikely to extend it further unless the need comes up. I do think that the functions that are included in v0.1 can already be useful to others too. I would also like to give the opportunity for people to contribute to the project if they wish to. The groundwork has been laid, so further extensions should be quick and relatively easy. Also check out a related project, [IGraphR](https://github.com/szhorvat/IGraphR), which makes igraph available for Mathematica users through RLink. I wrote IGraph/M because I needed higher performance and greater reliability (especially for parallel computing) than what RLink could provide. ---- **A request:** If any of you have used IGraphR in the past to access igraph from Mathematica, please post a response to this thread and let me know which specific functions you were using. &[Wolfram Notebook][1] [1]: https://www.wolframcloud.com/obj/94639221-60b4-47e3-8862-d996caa40388 Szabolcs Horvát 2015-09-06T12:55:14Z https://community.wolfram.com/groups/-/m/t/2445356 ![enter image description here][1] The popular game Wordle can take up a lot of your time. The author designed it so that it you can only play it once a day, thus saving us from ourselves :-). [Wordle][2] [NYTimes article on Wordle][3] But I couldn't resist the challenge to create a version of it in Mathematica, just for fun and because I was bored this past weekend. See the attached notebook and enjoy. Alas, since you can run it any number of times you are only to blame for yourself it you spend too much time on it. After executing the notebook just execute MWordle[Deploy] to bring up the game. A few additional comments. The notebook MWordle.nb has the option AutoGeneratedPackage -> Automatic which causes it, when saved, to create an MWordle.m package file in its same directory. The code in MWordle.nb is set up as a package with the context MWordle`Mwordle` If you want to set things up so that the package gets loaded and the MWordle game is automatically launched, do the following. Create a directory MWordleGame on your disk (The name MWordleGame can actually be whatever you wish.) And in the MWordleGame directory create a new directory called MWordle. (This name must be exactly that so that the MWordle`Mwordle` Context is property respected.) Put the MWordle.nb notebook in the MWordle dierectory, open it in Mathematica and save it so that the MWordle.m file is created in the MWordle directory. Then you can close the MWordle.nb notebook. Now in your MWordleGame directory save a new notebook -- you can call it whatever you wish, but something like LaunchMwordle.nb is a sensible choice. In that notebook create a button with the following command: CellPrint[TextCell[Button["Launch MWordle", Monitor[ If[! MemberQ[$Path, NotebookDirectory[]], AppendTo[$Path, NotebookDirectory[]]]; Needs["MWordle`MWordle`"]; MWordle`MWordle`MWordle[Deploy], Row[{ProgressIndicator[Appearance -> "Necklace", ImageSize -> Small], Spacer[5], Style["Launching MWordle...", 12, Blue, FontFamily -> "Arial"]}]], Method -> "Queued"], "Text", GeneratedCell -> False, CellAutoOverwrite -> False]] You now have a button in your LaunchMwordle.nb notebook which you can use any time you want to launch MWordle without having to execute the cells in the MWordle.nb notebook. Download the actual notebook from the link at the end of this post. The following is a version here to read. &[Wolfram Notebook][4] [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Wordle.gif&userId=20103 [2]: https://www.powerlanguage.co.uk/wordle/ [3]: https://www.nytimes.com/2022/01/03/technology/wordle-word-game-creator.html [4]: https://www.wolframcloud.com/obj/08c015e2-0d65-4634-bf54-4b73e518f6d5 David Reiss 2022-01-13T21:47:44Z https://community.wolfram.com/groups/-/m/t/3062832 ![enter image description here][1] &[Wolfram Notebook][2] [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=w5qgsdf.jpg&userId=11733 [2]: https://www.wolframcloud.com/obj/04458d24-aacf-4cd7-8a5d-46efde37e927 Marco Thiel 2023-11-09T22:51:38Z https://community.wolfram.com/groups/-/m/t/1170226 One of the great things about Mathematica is that it makes it so easy to do so much in so little code. It is partly because of this that I am so frustrated with the amount of time I need to spend developing the *interface* of package functions compared to their *functionality*. Suppose you wrote a Wolfram Language function that does something useful and interesting. Now you want to wrap it up into a package, and make it usable by everyone. A polished package is expected to have functions that: - Will check their input for errors - Will report errors in an informative way - Will use messages appropriately (i.e. associate them with the correct symbol name) - Will adhere to the de-facto Mathematica interface conventions: proper use of optional arguments, options, option inheritance (as in Graphics -> Plot), default option value handling, use of `Automatic`, etc. - Have SyntaxInformation I find that not infrequently I spend more time on making the function user-friendly than developing its functionality. **How do people generally deal with this task? How do you implement error checking and reporting in your packages?** To give an example, take a function as simple a moving average calculator. It is really easy to implement: movingAverage[vec_, n_] := Mean /@ Partition[vec, n, 1] But to bring it to the quality of the built-in `MovingAverage`, it should at least: - check the number of arguments (precisely 2) - check the types of arguments (a list and an integer) - check the values of arguments for correctness (non-empty list and positive integer) - make sure that all these checks don't introduce severe performance degradation (such as array unpacking, which can even be triggered by an inefficient argument pattern) This also involves the introduction of multiple messages (for each type of error) associated to `movingAverage`. If we now want to add a `movingMedian`, we will find that we will mostly need to carry out the same checks and report the same messages. There will be small differences though, e.g. average calculations are feasible for symbolic lists like `{1,x}`, but not median calculations. So the checks won't quite be identical. The messages will be mostly identical, but each function must associate messages to its own symbol, which means a lot of duplication. So if we care about a high-quality interface and high-quality error reporting, we will end up writing considerably more code for this than for the function's core task. We will also end up with a lot of code duplication, which is frustrating and a maintenance burden. The whole thing ends up being a lot of work and not a lot of fun (which is not very Mathematica-like :-) ) Are there good ways to simplify these tasks? Option handling also used to be error-prone and frustrating, but the introduction of `OptionsPattern[]` and `OptionValue[]` made it much easier. ---- I can see that there are some built-in tool to ease these tasks, though they are mostly undocumented. One example is `ArgumentCountQ`, another is ``Developer`CheckArgumentCount``, or some tools in ``GeneralUtilities` ``. I would love to hear from others about how they deal with the tasks I described, which such internal functions they make use of, etc. Szabolcs Horvát 2017-08-25T11:15:29Z https://community.wolfram.com/groups/-/m/t/963158 Recently John Fultz [announced][1] GitLink as open source on GitHub. He also mentioned the desire to offer more Wolfram related open source to the community. Draw was not mentioned by John but readers of his post followed the GitHub [link][2] and discovered the Draw repository. Naturally questions followed so here we are with an introduction of Draw, the latest open source contribution to the community. Draw is a vector drawing application. Many of the technical illustrations found in the Wolfram Language documentation were created using it. The release of Draw is to promote the advantages of using the Wolfram Language for software development. Draw was a personal project that grew from an idea to develop a simple tool to automate the drawing of flowcharts. This simple tool expanded in response to the need of functionality for solving specific challenges required by various illustrations. The Wolfram Language was very instrumental in helping Draw evolve during this process. Some of the more prominent features of Draw include: - Orthographic 3D drawing - Circuit, Mechanical and 3D library presets - Adjustable shapes such as gears or spirals - Shape arrays - Image autotrace The gallery of examples below demonstrate the capabilities of Draw: ![gallery][3] A look inside the application reveals an interface divided into four distinct sections: ![interface sections][4] An example of Draw being used to work on a flowchart: ![interface][5] The Circuit, Mechanical and 3D libraries: ![libraries][6] The adjustable gear shape: ![gear][7] The shape array: ![shape array][8] Hope this brief introduction is enough to inspire you to download the Draw notebook from [this GitHub link][9]. Start creating illustrations or explore the Wolfram Language code to discover how the application works. Support is available by clicking the blue help button for topics such as the basics, features, interaction, tools, examples and tips. Tooltips appear for all buttons detailing their behavior. Remember this is open source software and comes with no guarantee. That being said, feel free to kick the tires and take it for a spin. Draw works best when used in Mathematica 11 but most functionality is supported by Mathematica 10.4.1. [1]: http://community.wolfram.com/groups/-/m/t/960333 [2]: https://github.com/WolframResearch [3]: http://community.wolfram.com//c/portal/getImageAttachment?filename=examples.png&userId=28355 [4]: http://community.wolfram.com//c/portal/getImageAttachment?filename=interfaceSections.png&userId=28355 [5]: http://community.wolfram.com//c/portal/getImageAttachment?filename=interface.png&userId=28355 [6]: http://community.wolfram.com//c/portal/getImageAttachment?filename=libraries.png&userId=28355 [7]: http://community.wolfram.com//c/portal/getImageAttachment?filename=gear.png&userId=28355 [8]: http://community.wolfram.com//c/portal/getImageAttachment?filename=shapearray.png&userId=28355 [9]: https://github.com/shdlbwr/draw Tim Shedelbower 2016-11-14T20:15:12Z https://community.wolfram.com/groups/-/m/t/3062403 ![enter image description here][1] &[Wolfram Notebook][2] [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=fg45qw.jpg&userId=11733 [2]: https://www.wolframcloud.com/obj/34ac42c4-10de-4201-a158-6af47eda2bd8 Marco Thiel 2023-11-08T21:48:25Z https://community.wolfram.com/groups/-/m/t/849709 This is in the spirit of the "New Functions I would like to see in future..." post. I think that one area where Mathematica could improve considerably is supporting and encouraging third-party package development. Mathematica is great for end-users, but it feels like WRI isn't really thinking about making it truly extensible. A few concrete problems I noticed: - Recent versions add new symbols to the ``System` `` namespace indiscriminately, and prioritize always-present built-in functions over user-loadable standard packages. At 6000 built-in symbols this is just not sustainable and it's hostile to third-party packages: at this rate of adding new symbols each new release is virtually guaranteed to break some package through symbol name conflict. Previous versions used to ship with a few new standard packages bundled with Mathematica. Packages felt like a first-class citizen of the Mathematica ecosystem as WRI was using them too. Recent versions never add anything that is clearly a separate package from the user's perspective and needs to be loaded. Instead everything is crammed into the default namespace. Technically, new packages are added in each release, but these are now always auto-loaded and appear as built-in from the user's perspective. *With the amount of functionality Mathematica has, better namespace management is sorely needed!* Even MATLAB, long without namespace support, is now adopting namespaces. Mathematica had them from the beginning but doesn't use them anymore, which seems like a step backwards. A particularly awful thing about cramming everything into the ``System` `` context is that naming conflicts actually *break* packages, not just cause shadowing. If two packages use the same symbol names, one will simply shadow the other one, but won't break it. If there's a conflict with a ``System` `` symbol, the package will need fixes to work again. * Features that would typically be used by package developers are not well documented or completely undocumented. There are many symbols in the ``Internal` `` context that are very useful, such as `InheritedBlock`, `WithLocalSettings`, `PositiveMachineIntegerQ`, etc. There are also many in the semi-documented ``Developer` `` context. * Formerly many features of Mathematica were designed to be extensible and came with some documentation on how to extend them. `NDSolve` and `NIntegrate` are good examples. Recent additions tend to be less extensible and more opaque in their working. Is there a way to extend machine learning functions with new algorithms? Or can I add new graph layouts? Can I re-use the built in packing methods for graph layouts in my own layout computation? These don't seem to be possible. A good example of how strongly the focus has shifted to the highest level use case while ignoring more sophisticated use cases is the recently added `Dendrogram`: it goes directly from data to plot, and supports only the most superficial use. The old HierarchicalClustering package has a special data structure for dendrograms which can be manipulated separately from their plot. We had separate functions for *computing* or *plotting* a dendrogram, thus we could add our own, separate computation or plotting methods (and I did in the IGraph/M package which does produce dendrograms in this format). * There are some things that would be really useful or even necessary for package development, but they are missing. For example, how can we clean up temporary files created by packages, on kernel exit? There's no robust solution for this, even though the need must have come up internally (`Compile`-generated shared libraries do get cleaned up on exit). * Wolfram Workbench, *the only tool for creating proper documentation*, is stagnating and the released version (2.0) is not compatible with the latest Mathematica. I am aware that we can request 3.0 beta but it seems to take forever until it gets released. Also, Workbench is only for Premiere Support members, thus difficult to get. * The licensing that Wolfram offers does not make it easy to create multiplatform packages. One license is for one computer. If a package needs to special case different operating systems (which is always the case for LibraryLink/MathLink stuff) then the only realistic way for most of us to develop a package is to have access to an academic site license, so we can install three copies on OS X/Windows/Linux. A company that makes and sells commercial packages can afford multiple licenses. But for a healthy package ecosystem we need lots of free and open source packages, which are not easy with Mathematica at the moment. To be fair, recent versions have also added very useful developer-oriented tools. LibraryLink seems to be evolving nicely: it got the very necessary "managed library expressions" feature. Associations are not purely developer-oriented, but they are extremely useful in this scenario. 10.4 added support for RawArrays in LibraryLink. I appreciate all these features very much. `MUnit`, a unit testing package was updated finally added to Mathematica in v10 (and doesn't require the separate Workbench), but unfortunately this update also brought quality and usability problems so in the end I don't use this tool ... the old Workbench version seemed more practically useful. The point I am trying to make with this post is that I have the impression that *overall* Mathematica encourages third-party package development less than it used to. This is a bit surprising because there has been a big push to market Mathematica outside its traditional academic niche, to programmers (all the Wolfram Language rebranding, Wolfram Development Platform, etc.). In 2016 a healthy package ecosystem is critical for a software like Mathematica to survive. I am hoping that in the future there will be more emphasis on supporting and encouraging third-party package development. Szabolcs Horvát 2016-05-02T16:04:03Z https://community.wolfram.com/groups/-/m/t/1311342 Around 22:15 in [this talk](https://www.wolfram.com/broadcast/video.php?v=2057) a paclet repository is mentioned. To my mind, this is far-and-away the best repository Wolfram could make, seeing as Mathematica's got no effective way to share packages like [python](https://pypi.python.org/pypi), [R](https://cran.r-project.org/), [JavaScript](https://www.npmjs.com/), or other modern languages. I think its introduction would make working with Mathematica exponentially better, and so I'm really hoping WRI puts some effort into getting it off the ground sooner rather than later (especially because I would like to contribute my paclets before I drift away from using Mathematica seriously on a daily basis). For this to work effectively, though, I think it really has to be a free system. That's not to say that people shouldn't be able to charge money for their packages--it's their prerogative if they don't want anyone to use their code. What I mean is that if it costs money to distribute packages/contribute them to the repository, people will be significantly less likely to share their code, which will keep the Mathematica dev community small and in turn decrease the amount to which Mathematica can penetrate into the broader programming community. b3m2a1 ​  2018-03-29T18:09:21Z https://community.wolfram.com/groups/-/m/t/2864162 &[Wolfram Notebook][1] [1]: https://www.wolframcloud.com/obj/04e935d4-d928-4868-adb8-ae6f84fae1f8 Anton Antonov 2023-04-02T19:01:27Z https://community.wolfram.com/groups/-/m/t/2330900 Daily Study Groups are back, now providing expertise on practical programming! After a short break, we're starting up with a series that picks up where Wolfram Language Basics sessions ended. This 3-week series will take you from basic programming concepts to package development. Need tips on improving your code’s speed or functionality? Programming Tutorials offer a great opportunity to level-up your skills while interacting with software development professionals. See hands-on examples and get questions answered by Wolfram Language experts. Plus: participants who pass weekly quizzes are awarded a program completion certificate. Level 1 certification is available for those who pass manually graded exercises. Join us any time between August 2 and August 20! Check out our [registration page][1] for more details. [1]: https://wolfr.am/Study_Group15 Wolfram U 2021-07-30T15:24:02Z https://community.wolfram.com/groups/-/m/t/787142 ## General information and download links ## If you're interested in crystal structures, you can now download the Crystallica application from the Wolfram Library Archive, and then you can do things like this: Needs["Crystallica`"]; CrystalPlot[ {{5.4,0,0},{0,5.4,0},{0,0,5.4}}, {{0,0,0},{0,0,.5},{0,.5,0},{.5,0,0},{.24,.24,.24},{.24,.76,.76},{.76,.24,.76},{.76,.76,.24}}, {1,2,2,2,3,3,3,3}, AtomCol->{"Firebrick","YellowGreen",White},AtomRad->.4, BondStyle->2,BondDist->3, CellLineStyle->False,AddQ->True,Lighting->{{"Directional",White,ImageScaled[{0,0,1}]}},Background->Black] ![enter image description here][1] Here are the download links for Crystallica and two other packages you may need: [Crystallica][2] - contains the functions `CrystalPlot` and `CrystalChange` [CifImport][3] - contains an import function for CIF files [VaspImport][4] - contains an import function for files related to [VASP][5] Once you've installed Crystallica (by saving the entire Crystallica folder - not the zip archive - to `$USerBaseDirectory/Applications` and re-starting the Kernel), you can enter Crystallica into the Documentation Center and you'll find lots of useful examples. Most of the examples in this post are taken from the Documentation. For the other two packages, just install them and evaluate this: ?CifImport ?VaspImport I'll first show you a few things the `CrystalPlot` function can do when you already have crystal structure data inside Mathematica, wherever it may have come from. Then we'll take a look at how to get the data into Mathematica in the first place, which is where `CifImport` and `VaspImport` will come into play - but we'll get data from other sources as well. I'll cover the different import solutions in separate replies to this thread, because I have a feeling that I'll be rambling on and on and on... ## Simple plot ## Traditional ball-and-stick plots are usually just fine, so the simplest thing you can do is this: CrystalPlot[ {{4.5,0,0},{0,4.5,0},{0,0,3}}, {{0,0,0},{.5,.5,.5},{.2,.8,.5},{.3,.3,0},{.7,.7,0},{.8,.2,.5}}, {1,1,2,2,2,2}] ![enter image description here][6] As you can see, `CrystalPlot` expects three arguments. The first one contains the lattice vectors, which are simply the three vectors that create the parallelepiped that constitutes the cell. The second argument contains the atomic coordinates, but they're given in the basis of the lattice vectors (which is quite useful in crystallography). The third argument is a list of integers that gives the atom types, with one entry for each atom. If you want to plot a molecule instead, you can call `CrystalPlot` with just two arguments: A list of atom coordinates in cartesian space, and a list of atom types. Everything else you see in the plot - the atoms, bonds, colours, arrows etc. - represents the default settings of various layout options. ## Advanced atoms and bonds ## Let's take a look at some more advanced options just for fun. For instance, atoms and bonds can look any way you need them to, because you can specify your own functions for them. You can also fine-tune where to put bonds and what to do with their thickness and colour in a physically (or chemically) meaningful way, but I won't show that here. So here are some customized atoms and bonds: Row[Table[ CrystalPlot[{{4,0,0},{0,4,0},{0,0,4}},{{0,0,0},{.4,.4,.4},{.8,.8,.8}},{1,2,3}, AtomRad->{.4,1.2,.7},AtomFunction->style,ImageSize->400], {style,{ (Ball[#1,#2]&), (Scale[Sphere[#1,#2],{1,1,.5}]&), ({EdgeForm[Thick],Opacity[.7],Cuboid[#1-.5*#2,#1+.5*#2]}&) }}]] ![enter image description here][7] Row[Table[ CrystalPlot[{{0,0,0},{5,0,0},{2.5,4,0}},{1,2,3},BondDist->6,BondStyle->style,ImageSize->400], {style,{ 1, Function[{bonds,partcol},Table[{If[ii<.5,partcol[#,1],partcol[#,2]],Sphere[bonds[[#,1]]+ii*(bonds[[#,2]]-bonds[[#,1]]),.15]},{ii,0,1,1/9}]&/@Range[Length[bonds]]], Function[{bonds,partcol},Module[{spiral,points,rad=.05}, spiral[atoms_]:=Module[{scale=.5,dist=atoms[[2]]-atoms[[1]],curls=60,normal,rot,scaled}, normal=Table[{scale*Cos[ii],scale*Sin[ii],.1*ii},{ii,0,curls,\[Pi]/10}]; scaled={#[[1]],#[[2]],10*Norm[dist]/curls*#[[3]]}&/@normal; rot=scaled.Quiet[RotationMatrix[{dist,{0,0,1}}]]; Join[{atoms[[1]]},#+atoms[[1]]&/@(rot[[25;;-25]]),{atoms[[2]]}]]; points=spiral/@bonds; {partcol[#,1],Tube[BSplineCurve[points[[#,;;Round[Length[points[[#]]]/2]]],rad]],partcol[#,2],Tube[BSplineCurve[points[[#,Round[Length[points[[#]]]/2];;]],rad]]}&/@Range[Length[bonds]] ]] }}]] ![enter image description here][8] ## Lattice planes ## Crystallica can also add lattice planes to the plot. You can specify them using [h,k,l] Miller indices and distance to the origin. CrystalPlot[{{3,0,0},{0,3,0},{0,0,3}},{{0,0,0}},{1}, AddQ->True,AtomRad->.3,AtomCol->"CadmiumYellow",Sysdim->2,CellLineStyle->2, LatticePlanes->Table[{{1,1,1},dist},{dist,1,5}],ContourStyle->{"TerreVerte",Opacity[.7]},BoundaryStyle->Thick] ![enter image description here][9] ## Coordination polyhedra ## You can automatically search for and plot coordination polyhedra. This is not limited to the commonly occurring tetrahedra and octahedra - you can actually look for polyhedra with arbitrary numbers of corners. There are also options to fine-tune both the searching and the rendering. plot[corners_,mixed_]:=CrystalPlot[{{0,0,0},{0,0,1.8},{-.9,-1.5,-.6},{-.9,1.5,-.6},{1.7,0,-.6},{.8,.8,.8}},{1,2,2,2,2,3}, BondStyle->False,ImageSize->250, PolyMode[corners]->{"Show"->All,"AllowMixed"->mixed},PolyStyle[corners]->Directive[Opacity[.5],EdgeForm[Thick]]]; Grid[{{ "", "Search for polyhedra with \n4 corners", "Search for polyhedra with \n5 corners" },{ "Allow \nmixed corners", plot[4,True], plot[5,True] },{ "Don't allow \nmixed corners", plot[4,False], plot[5,False] }},Dividers->All] ![enter image description here][10] CrystalPlot[{{2.5,-4.3,0},{2.5,4.3,0},{0,0,5.5}}, {{.5,0,0},{0,.5,.7},{.5,.5,.3},{.2,.4,.5},{.6,.8,.2},{.2,.8,.8},{.8,.6,.5},{.4,.2,.2},{.8,.2,.8}},{1,1,1,2,2,2,2,2,2}, PolyMode[4]->True,PolyStyle[4]->EdgeForm[None],AddQ->True, Sysdim->2,AtomRad->0,CellLineStyle->False,AtomCol->{"SlateGray","Firebrick"}, ViewAngle->.4,ViewPoint->{3.2,0,1.1},ViewVertical->{.5,0,1.2}] ![enter image description here][11] ## Other things ## Visualization aside, you can also build supercells, change cell shapes, or add, remove and sort atoms... but that's a bit boring to read, so I'll refer you to the Documentation page of the `CrystalChange` function instead. If you're interested, we can use this thread to talk about any questions you may have, or you can share your use of the package (if you decide to use it). I'm not offering full support here, but I'll be floating around, and I'd like to hear your feedback. We don't have any intentions to be involved in further development. But if you have a good idea and some time, then by all means, work on it for yourself, or host it on your favourite code collaboration site. Bianca Eifert and Christian Heiliger [1]: http://community.wolfram.com//c/portal/getImageAttachment?filename=teaser.png&userId=69107 [2]: http://library.wolfram.com/infocenter/MathSource/9372/ [3]: http://library.wolfram.com/infocenter/MathSource/9373/ [4]: http://library.wolfram.com/infocenter/MathSource/9375/ [5]: http://vasp.at/ [6]: http://community.wolfram.com//c/portal/getImageAttachment?filename=9692simple.png&userId=69107 [7]: http://community.wolfram.com//c/portal/getImageAttachment?filename=atoms.png&userId=69107 [8]: http://community.wolfram.com//c/portal/getImageAttachment?filename=bonds.png&userId=69107 [9]: http://community.wolfram.com//c/portal/getImageAttachment?filename=planes.png&userId=69107 [10]: http://community.wolfram.com//c/portal/getImageAttachment?filename=polys.png&userId=69107 [11]: http://community.wolfram.com//c/portal/getImageAttachment?filename=polys2.png&userId=69107 [12]: http://rruff.geo.arizona.edu/AMS/CIF_text_files/13532_cif.txt [13]: http://cms.mpi.univie.ac.at/vasp/vasp/POSCAR_file.html [14]: http://wiki.jmol.org/index.php/File:Caffeine.mol Bianca Eifert 2016-02-05T18:43:18Z https://community.wolfram.com/groups/-/m/t/2328597 > **GitHub Repository:** https://github.com/alecGraves/Howl &[Wolfram Notebook][1] [1]: https://www.wolframcloud.com/obj/da6d3b76-7bbf-4a55-9fcd-058e7f1e17bf Alec Graves 2021-07-27T14:09:12Z https://community.wolfram.com/groups/-/m/t/762950 **RadarChart Mathematica Package** The *RadarChart* mathematica package incorporates Radar Charts (also known as Spider Charts, Web Charts and Star Plots) as a new set of visualization tools into the mathematica environment. A detailed explanation on the use of radar charts can be found in the following [wikipedia page](https://en.wikipedia.org/wiki/Radar_chart). The project is listed at the mathematica package repository ([http://packagedata.net][1]), or can be accessed directly at [github][2]. Thanks to [halirutan](https://github.com/halirutan) for his guidance on setting up this project. Installation Instructions are heavily based on his existing projects. --- **Navigation** - [Detailed Usage](#-detailed-usage) - [Examples](#examples) - [Neat Examples](#neat-examples) - [Installation](#installation) - [Automatic Installation for *Mathematica* 9 and above](#automatic-installation-for-mathematica-9-and-above) - [Manual Installation](#manual-installation) - [Contact](#-contact) --- ## Detailed Usage RadarChart[{y1,y2,y3,...}] generates a radar plot (also known as web, star, spider, cobweb or kiviat diagram) corresponding to a list of values. This type of chart is suitable for showing commonality and outliers across different variables. RadarChart[{list1,list2,...}] generates a radar plot to compare several series. RadarChart[association] generates a radar plot to view the variable values in an association. RadarChart[{association1,association2,...}] generates a radar plot to compare several series. RadarChart[dataset] generates a radar plot to compare several series. Options are listed in the help file, and also in the readme document @ the github site. ##Examples Needs["RadarChart`"] RadarChart[{1, 2, 4, 5, 3}, ChartLegends -> {"Private Label Strawberry Juice"}, AxesLabel -> {"Ripe", "Green", "Candy", "Juicy", "Sulphur"}, PlotLabel -> Style["Sensory Map Strawberry Juice", Bold, Large], ImageSize -> Medium]  Multiple Series can be charted. Needs["RadarChart`"] RadarChart[{{1, 4, 3, 5, 2}, {2, 4, 3, 2, 1}}, Filling -> Axis, AxesLabel -> {"Sweet", "Sour", "Salty", "Bitter", "Umami"}, PlotStyle -> {Red, Blue}, ChartLegends -> {"Fernet-Cola", "Jugo"}]  Example of a start plot. Needs["RadarChart`"]; RadarChart[{{1, 4, 3, 5, 2}, {2, 4, 3, 2, 1}}, AxesType -> "Star"]  Compare survey results. Needs["RadarChart`"]; RadarChart[{{3, 4, 3, 4, 2}, {4, 5, 4, 5, 3}}, AxesLabel -> {"Assets", "Reliability", "Cost Control", "Abstenteeism", "Revenue"}, ChartLegends -> {"Past Year", "Current Year"}, Filling -> Axis, PlotStyle -> {{Gray}, {Black}}, ImageSize -> Medium]  ##Neat Examples Analyze and compare crime statistics across states. Needs["RadarChart`"]; states = EntityClass["AdministrativeDivision", "AllUSStatesPlusDC"]; stateNames = First@StringSplit[#, ","] & /@ states["Name"]; crimeProps = {"AggravatedAssaultRate", "BurglaryRate", "ForcibleRapeRate", "LarcenyTheftRate", "MurderNonnegligentManslaughterRate", "PropertyCrimeRate", "RobberyRate", "ViolentCrimeRate"}; crimeData = EntityValue[states, EntityProperty["AdministrativeDivision", #] & /@ crimeProps]; crimeRanking = Transpose[ Ordering[crimeData[[All, #]]] & /@ Range@Length@crimeProps]; ds = Dataset[ AssociationThread[stateNames, AssociationThread[crimeProps, #] & /@ crimeRanking]]; GraphicsRow[{RadarChart[ds["Georgia"], AxesType -> "Star", PlotLabel -> Style["Georgia", Bold, Large], Epilog -> {Dashed, Circle[{0, 0}, 25.5]}, ImageSize -> Large], GraphicsGrid[ Partition[ RadarChart[ds[#], PlotLabel -> #, AxesLabel -> None, PlotRange -> {0, 50}, AxesType -> "Star", PlotRangePadding -> Full, FrameTicks -> None, Epilog -> {Dashed, Circle[{0, 0}, 25.5]}] & /@ stateNames, UpTo[6]], ImageSize -> 600]}]]  #Installation This package should work with Mathematica >8 if not using associations or datasets. The package was developed using mathematica 10. The installation is simple: Copy the `RadarChart` package directory into a location where *Mathematica* can find it. Usually this is the `Applications` directory in your `$UserBaseDirectory`. Just evaluate FileNameJoin[{$UserBaseDirectory, "Applications"}] to see it. If there is an old installation of the `RadarChart`, remove it. Please find detailed steps below. ###Automatic Installation for *Mathematica* 9 and above We have set up [an installation script](https://raw.githubusercontent.com/catrasca/RadarChart/master/RadarChart/installer.m) that does all the steps, except deleting old installations, for you. If it finds an old installation, it will prompt you with the location and quit, so that you can remove the old installation. After removing the old files, just start it again and it will proceed through all the steps pointed out in the manual installation section. To start the installation script, simply call Import["http://tinyurl.com/ntmhkca"] After this, the package should be available in your mathematica instance. ###Manual Installation ####Removing old Installations Old installation packages can be found by simply searching directories in your `$Path`. FileNames["RadarChart", $Path] Please remove old installation directories that appear after evaluating the commands above. You can use DeleteDirectory[dir, DeleteContents -> True] for that, but note that on Windows this might fail, because there, some files are locked when *Mathematica* is running. In this case, close *Mathematica* and do it manually using an explorer. ###Downloading, Extracting and Copying the New Version The easiest way is, to download the whole repository as zip file. Use [this master.zip](https://github.com/catrasca/RadarChart/archive/master.zip) or click the *Download ZIP* on the right side on this page. After you have downloaded the file extract it. If you have no tool for this on Windows, you could use the [free 7-Zip](http://7-zip.org/). Under Mac OSX and Linux this should work out of the box. Inside the extracted directory, you will find a subdirectory `RadarChart` which has the following structure RadarChart/ ??? Documentation ? ??? English ? ??? ReferencePages ? ??? Symbols ? ??? RadarChart.nb ??? Kernel ? ??? init.m ??? Installer.m ??? pacletInfo.m ??? RadarChart.m Copy the whole `RadarChart` directory with all its content to your `Applications` folder under your `$UserBaseDiretory`. If everything is in place proceed to the next step. ###Finishing the Installation To make the package works, you can simply restart *Mathematica*. ## Contact If you find bugs or have any other questions, please [create a new issue](https://github.com/catrasca/RadarChart/issues) in the bug-tracker. [1]: http://packagedata.net/ [2]: http://github.com/catrasca/RadarChart Diego Zviovich 2015-12-22T20:02:41Z https://community.wolfram.com/groups/-/m/t/3467978 ![Collatz conjecture visualizations][1] &[Wolfram Notebook][2] [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Main227052025.png&userId=20103 [2]: https://www.wolframcloud.com/obj/85fd1874-61c4-4798-8e48-d8ba6d037984 Anton Antonov 2025-05-27T12:23:54Z https://community.wolfram.com/groups/-/m/t/1646303 # UNET [](https://zenodo.org/badge/latestdoi/137186334) [](https://github.com/dwyl/esta/issues) ---------- ![Automated 3D muscle segmentation using UNET / RESNET using DIXON MRI data][17] A package to generate and train a UNET deep convolutional network for 2D and 3D image segmentation. Some code was based on [work][1] by [@Ali Hashmi][at0], which was also dicussed in [this post][2] The full version of the toolbox can be found on my [github page][3]. * [Information](#information) * [Install toolbox](#install-toolbox) * [Using the toolbox](#using-the-toolbox) * [Functionality](#functionality) * [Visualization](#visualization) * [Example](#example) ## Information UNET is developed for [Mathematica](https://www.wolfram.com/mathematica/). It contains the following toolboxes: - UnetCore - UnetSupport Documentation of all functions and their options is fully integrated in the Mathematica documentation. The toolbox always works within the latest version of Mathematica and does not support any backward compatibility. All code and documentation is maintained and uploaded to github using [Workbench](https://www.wolfram.com/workbench/). ## Install toolbox Install the toolbox in the Mathematica UserBaseDirectory > Applications. FileNameJoin[{$UserBaseDirectory, "Applications"}] ## Using the toolbox The toolbox can be loaded by using <<UNET` The notbook ``UNET.nb`` shows examples of how to use the toolbox on artificially generated 2D data. There are also examples how to visualize the layer of your trained network and how to visualize the training itself. ## Functionality The network supports multi channel inputs and multi class segmentation. * UNET generates a UNET convolutional network. * 2D UNET ![UNET 2D][4] * 3D UNET ![UNET 3D][5] * Loss Layers: Training the data is done using three loss layers: a SoftDiceLossLayer, BrierLossLayer and a CrossEntropyLossLayer. ![SoftDiceLossLayer, BrierLossLayer and a CrossEntropyLossLayer][6] * Convolution Blocks: The toobox contains five different convolution blocks that build up the network: [UNET][7], UResNet, [RestNet][8], UDenseNet, [DensNet][9]. ![Convolution blocks][10] * SplitTrainData splits the data and labels into training, validation and test data. ![split train Data][11] * TrainUNET trains the network. ![Train UNET][12] ## Visualization * Visualize the network and results. * Visualize the features of the layers. ![Visualize layer features][13] * Visualize the results. ![Visualize the results][14] * Animate the training process. ![UNET 2D animation][15] ![UNET 3D animation][16] ## Example * Example: 3D segmentation of lower legg muscles using MRI data. ![Automated 3D muscle segmentation using UNET / RESNET using DIXON MRI data][17] [1]: https://github.com/alihashmiii/UNet-Segmentation-Wolfram [2]: https://community.wolfram.com/groups/-/m/t/1341081?p_p_auth=w8PIeeiA [3]: https://github.com/mfroeling [4]: https://community.wolfram.com//c/portal/getImageAttachment?filename=UNET2D.PNG&userId=1332602 [5]: https://community.wolfram.com//c/portal/getImageAttachment?filename=UNET3D.PNG&userId=1332602 [6]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Loss.PNG&userId=1332602 [7]: https://arxiv.org/abs/1505.04597 [8]: https://arxiv.org/abs/1512.03385 [9]: https://arxiv.org/abs/1608.06993 [10]: https://community.wolfram.com//c/portal/getImageAttachment?filename=convblocks.PNG&userId=1332602 [11]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Split.PNG&userId=1332602 [12]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Train.PNG&userId=1332602 [13]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Visualize1.PNG&userId=1332602 [14]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Visualize2.PNG&userId=1332602 [15]: https://community.wolfram.com//c/portal/getImageAttachment?filename=amin0-v2.gif&userId=1332602 [16]: https://community.wolfram.com//c/portal/getImageAttachment?filename=amin4-v2.gif&userId=1332602 [17]: https://community.wolfram.com//c/portal/getImageAttachment?filename=Muscle_Segmentation.jpg&userId=1332602 [at0]: https://community.wolfram.com/web/alihashmi87 Martijn Froeling 2019-04-03T20:01:26Z https://community.wolfram.com/groups/-/m/t/1273720 ## Background I enjoy coding in the FrontEnd (except it crashes and lookup across files does not exist), but I often miss 'hands on keyboard', customizable code templates. E.g. I often forget to wrap an option name with quotes "_" or I'm starting a new function and would like to avoid retyping `Attributes/Options` `Catch/Check` etc. I don't like palettes for something that I need to do quickly and frequently. The are not 'hands on keyboard' either. So I created a little package/stylesheet, should work on Win/MacOs with MMA 10.4+ https://github.com/kubaPod/DevTools In case you are interested and/or have any ideas about this / similar features, let me know here or create an Issue in GitHub. Topic cross posted on Mathematica.stackexchange: https://mathematica.stackexchange.com/q/164653/5478 [![enter image description here][2]][2] ## Index Most up to date examples, setup and other details can be found in [**project's Readme.MD**](https://github.com/kubaPod/DevTools) - [v0.10.0 (01-12-2018) NotebookActions](https://community.wolfram.com/groups/-/m/t/1273720#lukr_reply) - [v0.8.0 (06-07-2018) VerificationTest template](https://community.wolfram.com/groups/-/m/t/1273720#kxdi_reply) - [v0.7.0 (21-02-2018) support for V10.4](https://community.wolfram.com/groups/-/m/t/1273720#gkow_reply) [2]: https://i.stack.imgur.com/g96TY.gif Kuba Podkalicki 2018-01-28T18:00:23Z https://community.wolfram.com/groups/-/m/t/1421180 Two months ago, Albert Rich posted ["What's the hardest integral Mathematica running Rubi can find?"](http://community.wolfram.com/groups/-/m/t/1343015) here on the Wolfram Community. You might have also seen that I responded in detail, and pointed out a few things that could help improve Rubi (Rule-based integrator). While it appears nothing really happened afterward, this is far from reality. Since then, Albert and I have worked closely together to make Rubi more accessible and user-friendly. If you would like to learn how our productive collaboration evolved, let me invite you to read [my latest blog-post](http://halirutan.de/programming/Rubi/). However, here, we want to share an update that should serve as an overview of what we have done to improve Rubi. First of all, Rubi has got a new home under [rulebasedintegration.org](https://rulebasedintegration.org/), and its old website will no longer be updated. On the new website, you will find information, installation instructions, and links to the source-code and test-suites. Secondly, we created a [Rubi Organization](https://github.com/RuleBasedIntegration) on GitHub that serves as the headquarters for all things Rubi. It contains all Rubi's code, notebooks, and test-suites nicely structured into several repositories. At the moment, we provide repositories for the * loadable package files and notebook source files defining over 6700 integration rules, * PDF files displaying the rules in human-readable mathematical notation alongside the Mathematica code, and * test-suite files containing over 71000 integration problems and their solutions. The integration test files are available in the syntax used by 4 popular computer algebra systems (Mathematica, Maple, Maxima, and Axiom). The test-suite can be used to compare Rubi's results with other symbolic integrators, including Mathematica's `Integrate` function. In addition to the transition to GitHub, we recently released version 4.16.0.3 of Rubi which significantly expands the class of expressions the system can integrate. But the most noticeable change for users is the completely reworked display of the rules and intermediate steps Rubi uses to integrate expressions. Although installation, usage, and examples are given on [Rubi's website](https://rulebasedintegration.org/), let me show you how easy it is to install and run Rubi 4.16.0.3 using Mathematica 11.3: The command PacletInstall["https://github.com/RuleBasedIntegration/Rubi/releases/download/4.16.0.3/Rubi-4.16.0.3.paclet"]; installs the Rubi-4.16.0.3 paclet on your computer. After that, to load Rubi into Mathematica all you have to do is issue the `Get` command << Rubi` Then to integrate an expression with respect to a variable, use Rubi's `Int` command similar to Mathematica's `Integrate` command. For example, evaluating Int[(Sec[x]^2 + Sec[x]^2*Tan[x])/((2 - Tan[x])*Sqrt[1 + Tan[x]^3]), x] returns the antiderivative (a.k.a. the indefinite integral) ![enter image description here][1] Rubi's unique ability to display the steps it uses to integrate expressions is a great feature of the system. For example, the `Steps` command Steps@Int[(Sec[x]^2 + Sec[x]^2*Tan[x])/((2 - Tan[x])*Sqrt[1 + Tan[x]^3]), x] displays ![enter image description here][2] Here, in red are the rules used to integrate the expression, and in blue are the intermediate results. Each rule can be expanded to show the rule number, which directly corresponds to the index of the rule in `Int`'s list of DownValues. More importantly, you can see the conditions that have to be satisfied so the rule can be applied. ![enter image description here][3] Rubi's `Stats` command provides statistics about the integration. For example, Stats[Int[(Sec[x]^2 + Sec[x]^2*Tan[x])/((2 - Tan[x])*Sqrt[1 + Tan[x]^3]), x]] displays ![enter image description here][4] The leaf-count size of an antiderivative is a crude measure of its complexity. As you can see, Rubi's antiderivative for this integral has a leaf-count of 25. Now compare Rubi's antiderivative with that produced by Mathematica 11.3 for the same integral: Integrate[(Sec[x]^2 + Sec[x]^2*Tan[x])/((2 - Tan[x])*Sqrt[1 + Tan[x]^3]), x] ![enter image description here][5] LeafCount[%] (* 290 *) Note that not only is Mathematica's result more than 11 times the size of Rubi's, it unnecessarily involves elliptic integral functions *and* the imaginary unit. Skeptics might be inclined to ask if Rubi's dramatically simpler result is actually a valid antiderivative. Since symbolic differentiation is much easier than integration, antiderivatives can be verified correct by seeing if its derivative equals the original integrand as follows: expr = (Sec[x]^2 + Sec[x]^2*Tan[x])/((2 - Tan[x])*Sqrt[1 + Tan[x]^3]); FullSimplify[D[Int[expr, x], x] == expr] FullSimplify[D[Integrate[expr, x], x] == expr] ![enter image description here][6] As you can see Mathematica easily verifies Rubi's antiderivative correct, but has a hard time verifying its own antiderivative correct... Albert and I are working on publishing the program used to thoroughly test each new version of Rubi before being released. The test program ensures Rubi's result equals the optimal antiderivative for the over 71000 problems in the test-suite. And yes, the optimal antiderivatives have all been verified correct by differentiation. Of course, the optimal antiderivatives stored in the test-suite are actually just the simplest ones found so far. If you should find a substantially simpler antiderivative than the one in the test-suite, please report it so the test-suite can be made even harder on Rubi! If all that has got you interested in joining Rubi's community of users, check out its website or talk to us in our [Gitter chatroom](https://gitter.im/Rule-Based-Integration/Lobby). [1]: https://community.wolfram.com//c/portal/getImageAttachment?filename=10564sSs5M.png&userId=11733 [2]: https://community.wolfram.com//c/portal/getImageAttachment?filename=qDtIH.png&userId=11733 [3]: https://community.wolfram.com//c/portal/getImageAttachment?filename=vQ4GR.png&userId=11733 [4]: https://community.wolfram.com//c/portal/getImageAttachment?filename=43t8e.png&userId=11733 [5]: https://community.wolfram.com//c/portal/getImageAttachment?filename=xjKTJ.png&userId=11733 [6]: https://community.wolfram.com//c/portal/getImageAttachment?filename=eCoWB.png&userId=11733 Patrick Scheibe 2018-08-24T02:50:37Z https://community.wolfram.com/groups/-/m/t/866748 ## Introduction This post describes the use of face-like diagrams to visualize multidimensional data introduced by Herman Chernoff in 1973, see [1]. The idea to use human faces in order to understand, evaluate, or easily discern (the records of) multidimensional data is very creative and inspirational. As Chernoff says in [1], the object of the idea is to "represent multivariate data, subject to strong but possibly complex relationships, in such a way that an investigator can quickly comprehend relevant information and then apply appropriate statistical analysis." It is an interesting question how useful this approach is and it seems that there at least several articles discussing that; see for example [2]. I personally find the use of Chernoff faces useful in a small number of cases, but that is probably true for many "creative" data visualization methods. Below are given both simple and more advanced examples of constructing Chernoff faces for data records using the *Mathematica* package [3]. The considered data is categorized as: 1. a small number of records, each with small number of elements; 2. a large number of records, each with less elements than Chernoff face parts; 3. a list of long records, each record with much more elements than Chernoff face parts; 4. a list of nearest neighbors or recommendations. For several of the visualizing scenarios the records of two "real life" data sets are used: Fisher Iris flower dataset [7], and "Vinho Verde" wine quality dataset [8]. For the rest of the scenarios the data is generated. A fundamental restriction of using Chernoff faces is the necessity to properly transform the data variables into the ranges of the Chernoff face diagram parameters. Therefore, proper data transformation (standadizing and rescaling) is an inherent part of the application of Chernoff faces, and this document describes such data transformation procedures (also using [3]). ### Package load The packages [3,4] are used to produce the diagrams in this post. The following two commands load them. Import["https://raw.githubusercontent.com/antononcube/\ MathematicaForPrediction/master/ChernoffFaces.m"] Import["https://raw.githubusercontent.com/antononcube/\ MathematicaForPrediction/master/MathematicaForPredictionUtilities.m"] ### Mirror This Community post mirrors the blog post ["Making Chernoff faces for data visualization"](https://mathematicaforprediction.wordpress.com/2016/06/03/making-chernoff-faces-for-data-visualization/). ## Making faces ### Just a face Here is a face produced by the function `ChernoffFace` of [3] with its simplest signature: SeedRandom[152092] ChernoffFace[] [](http://i.imgur.com/PIcrIcK.gif) Here is a list of faces: SeedRandom[152092] Table[ChernoffFace[ImageSize -> Tiny], {7}] [](http://i.imgur.com/SPWgX0r.gif) ### Proper face making The "proper" way to call ChernoffFace is to use an association for the facial parts placement, size, rotation, and color. The options are passed to Graphics. SeedRandom[2331]; ChernoffFace[ AssociationThread[ Keys[ChernoffFace["FacePartsProperties"]] -> RandomReal[1, Length[ChernoffFace["FacePartsProperties"]]]], ImageSize -> Small , Background -> GrayLevel[0.85]] [](http://i.imgur.com/wY3bgKC.gif) The Chernoff face drawn with the function `ChernoffFace` can be parameterized to be asymmetric. The parameters argument mixes (1) face parts placement, sizes, and rotation, with (2) face parts colors and (3) a parameter should it be attempted to make the face symmetric. All facial parts parameters have the range [0,1] Here is the facial parameter list: Keys[ChernoffFace["FacePartsProperties"]] (* {"FaceLength", "ForheadShape", "EyesVerticalPosition", "EyeSize", \ "EyeSlant", "LeftEyebrowSlant", "LeftIris", "NoseLength", \ "MouthSmile", "LeftEyebrowTrim", "LeftEyebrowRaising", "MouthTwist", \ "MouthWidth", "RightEyebrowTrim", "RightEyebrowRaising", \ "RightEyebrowSlant", "RightIris"} *) The order of the parameters is chosen to favor making symmetric faces when a list of random numbers is given as an argument, and to make it easier to discern the faces when multiple records are visualized. For experiments and discussion about which facial features bring better discern-ability see [2]. One of the conclusions of [2] is that eye size and eye brow slant are most decisive, followed by face size and shape. Here are the rest of the parameters (colors and symmetricity): Complement[Keys[ChernoffFace["Properties"]], Keys[ChernoffFace["FacePartsProperties"]]] (* {"EyeBallColor", "FaceColor", "IrisColor", "MakeSymmetric", \ "MouthColor", "NoseColor"} *) ### Face coloring The following code make a row of faces by generating seven sequences of random numbers in $[0,1]$, each sequence with length the number of facial parameters. The face color is assigned randomly and the face color or a darker version of it is used as a nose color. If the nose color is the same as the face color the nose is going to be shown "in profile", otherwise as a filled polygon. The colors of the irises are random blend between light brown and light blue. The color of the mouth is randomly selected to be black or red. SeedRandom[201894]; Block[{pars = Keys[ChernoffFace["FacePartsProperties"]]}, Grid[{#}] &@ Table[ChernoffFace[Join[ AssociationThread[pars -> RandomReal[1, Length[pars]]], <|"FaceColor" -> (rc = ColorData["BeachColors"][RandomReal[1]]), "NoseColor" -> RandomChoice[{Identity, Darker}][rc], "IrisColor" -> Lighter[Blend[{Brown, Blue}, RandomReal[1]]], "MouthColor" -> RandomChoice[{Black, Red}]|>], ImageSize -> 100], {7}] ] [](http://i.imgur.com/QN5Rhvf.png) ### Symmetric faces The parameter "MakeSymmetric" is by default `True`. Setting "MakeSymmetric" to true turns an incomplete face specification into a complete specification with the missing paired parameters filled in. In other words, the symmetricity is not enforced on the specified paired parameters, only on the ones for which specifications are missing. The following faces are made symmetric by removing the facial parts parameters that start with "R" (for "Right") and the parameter "MouthTwist". SeedRandom[201894]; Block[{pars = Keys[ChernoffFace["FacePartsProperties"]]}, Grid[{#}] &@Table[( asc = Join[AssociationThread[ pars -> RandomReal[1, Length[pars]]], <|"FaceColor" -> (rc = ColorData["BeachColors"][RandomReal[1]]), "NoseColor" -> RandomChoice[{Identity, Darker}][rc], "IrisColor" -> Lighter[Blend[{Brown, Blue}, RandomReal[1]]], "MouthColor" -> RandomChoice[{Black, Red}]|>]; asc = Pick[asc, StringMatchQ[Keys[asc], x : (StartOfString ~~ Except["R"] ~~ __) /; x != "MouthTwist"]]; ChernoffFace[asc, ImageSize -> 100]), {7}]] [](http://i.imgur.com/7J7vl12.png) Note that for the irises we have two possibilities of synchronization: pars = <|"LeftIris" -> 0.8, "IrisColor" -> Green|>; {ChernoffFace[Join[pars, <|"RightIris" -> pars["LeftIris"]|>], ImageSize -> 100], ChernoffFace[ Join[pars, <|"RightIris" -> 1 - pars["LeftIris"]|>], ImageSize -> 100]} [](http://i.imgur.com/8i16NUg.gif) ## Visualizing records (first round) The conceptually straightforward application of Chernoff faces is to visualize ("give a face" to) each record in a dataset. Because the parameters of the faces have the same ranges for the different records, proper rescaling of the records have to be done first. Of course standardizing the data can be done before rescaling. First let us generate some random data using different distributions: SeedRandom[3424] {dists, data} = Transpose@Table[( rdist = RandomChoice[{NormalDistribution[RandomReal[10], RandomReal[10]], PoissonDistribution[RandomReal[4]], GammaDistribution[RandomReal[{2, 6}], 2]}]; {rdist, RandomVariate[rdist, 12]}), {10}]; data = Transpose[data]; The data is generated in such a way that each column comes from a certain probability distribution. Hence, each record can be seen as an observation of the variables corresponding to the columns. This is how the **columns** of the generated data look like using `DistributionChart`: DistributionChart[Transpose[data], ChartLabels -> Placed[MapIndexed[ Grid[List /@ {Style[#2[[1]], Bold, Red, Larger]}] &, dists], Above], ChartElementFunction -> "PointDensity", ChartStyle -> "SandyTerrain", ChartLegends -> dists, BarOrigin -> Bottom, GridLines -> Automatic, ImageSize -> 900] [](http://i.imgur.com/iXQfzZG.gif) At this point we can make a face for each **record** of the rescaled data: faces = Map[ChernoffFace, Transpose[Rescale /@ Transpose[data]]]; and visualize the obtained faces in a grid. Row[{Grid[ Partition[#, 4] &@Map[Append[#, ImageSize -> 100] &, faces]], " ", Magnify[#, 0.85] &@ GridTableForm[ List /@ Take[Keys[ChernoffFace["FacePartsProperties"]], Dimensions[data][[2]]], TableHeadings -> {"Face part"}] }] [](http://i.imgur.com/LMTsbCf.gif) (The table on the right shows which facial parts are used for which data columns.) ## Some questions to consider Several questions and observations arise from the example in the previous section. #### 1. What should we do if the data records have more elements than facial parts parameters of the Chernoff face diagram? This is another fundamental restriction of Chernoff faces -- the number of data columns is limiter by the number of facial features. One way to resolve this is to select important variables (columns) of the data; another is to represent the records with a vector of statistics. The latter is shown in the section "Chernoff faces for lists of long lists". #### 2. Are there Chernoff face parts that are easier to perceive or judge than others and provide better discern-ability for large collections of records? Research of the [pre-attentiveness](https://en.wikipedia.org/wiki/Pre-attentive_processing) and effectiveness with Chernoff faces, [2], shows that eye size and eyebrow slant are the features that provide best discern-ability. Below this is used to select some of the variable-to-face-part correspondences. #### 3. How should we deal with outliers? Since we cannot just remove the outliers from a record -- we have to have complete records -- we can simply replace the outliers with the minimum or maximum values allowed for the corresponding Chernoff face feature. (All facial features of `ChernoffFace` have the range $[0,1]$.) See the next section for an example. ## Data standardizing and rescaling Given a full array of records, we most likely have to standardize and rescale the columns in order to use the function `ChernoffFace`. To help with that the package [3] provides the function `VariablesRescale` which has the options "StandardizingFunction" and "RescaleRangeFunction". Consider the following example of `VariableRescale` invocation in which: 1. each column is centered around its median and then divided by the inter-quartile half-distance (quartile deviation), 2. followed by clipping of the outliers that are outside of the disk with radius 3 times the quartile deviation, and 3. rescaling to the unit interval. rdata = VariablesRescale[N@data, "StandardizingFunction" -> (Standardize[#, Median, QuartileDeviation] &), "RescaleRangeFunction" -> ({-3, 3} QuartileDeviation[#] &)]; TableForm[rdata /. {0 -> Style[0, Bold, Red], 1 -> Style[1, Bold, Red]}] [](http://i.imgur.com/LdhvQJM.png) **Remark:** The bottom outliers are replaced with 0 and the top outliers with 1 using `Clip`. ## Chernoff faces for a small number of short records In this section we are going use the Fisher Iris flower data set [7]. By "small number of records" we mean few hundred or less. ### Getting the data These commands get the Fisher Iris flower data set shipped with *Mathematica*: irisDataSet = Map[Flatten, List @@@ ExampleData[{"MachineLearning", "FisherIris"}, "Data"]]; irisColumnNames = Most@Flatten[ List @@ ExampleData[{"MachineLearning", "FisherIris"}, "VariableDescriptions"]]; Dimensions[irisDataSet] (* {150, 5} *) Here is a summary of the data: Grid[{RecordsSummary[irisDataSet, irisColumnNames]}, Dividers -> All, Alignment -> Top] [](http://i.imgur.com/upxeRHv.png) ### Simple variable dependency analysis Using the function `VariableDependenceGrid` of the package [4] we can plot a grid of variable cross-dependencies. We can see from the last row and column that "Petal length" and "Petal width" separate setosa from versicolor and virginica with a pretty large gap. Magnify[#, 1] &@ VariableDependenceGrid[irisDataSet, irisColumnNames, "IgnoreCategoricalVariables" -> False] [](http://i.imgur.com/vJA6kth.png) ### Chernoff faces for Iris flower records Since we want to evaluate the usefulness of Chernoff faces for discerning data records groups or clusters, we are going to do the following steps. 1. Data transformation. This includes standardizing and rescaling and selection of colors. 2. Make a Chernoff face for each record **without** the label class "Species of iris". 3. Plot shuffled Chernoff faces and attempt to visually cluster them or find patterns. 4. Make a Chernoff face for each record using the label class "Specie of iris" to color the faces. (Records of the same class get faces of the same color.) 5. Compare the plots and conclusions of step 2 and 4. #### 1. Data transformation First we standardize and rescale the data: chernoffData = VariablesRescale[irisDataSet[[All, 1 ;; 4]]]; These are the colors used for the different species of iris: faceColorRules = Thread[Union[ irisDataSet[[All, -1]]] \ -> Map[Lighter[#, 0.5] &, {Purple, Blue, Green}]] (* {"setosa" -> RGBColor[0.75, 0.5, 0.75], "versicolor" -> RGBColor[0.5, 0.5, 1.], "virginica" -> RGBColor[0.5, 1., 0.5]} *) Add the colors to the data for the faces: chernoffData = MapThread[ Append, {chernoffData, irisDataSet[[All, -1]] /. faceColorRules}]; Plot the distributions of the rescaled variables: DistributionChart[ Transpose@chernoffData[[All, 1 ;; 4]], GridLines -> Automatic, ChartElementFunction -> "PointDensity", ChartStyle -> "SandyTerrain", ChartLegends -> irisColumnNames, ImageSize -> Large] [](http://i.imgur.com/rwm71tn.gif) #### 2. Black-and-white Chernoff faces Make a black-and-white Chernoff face for each record without using the species class: chfacesBW = ChernoffFace[ AssociationThread[{"NoseLength", "LeftEyebrowTrim", "EyeSize", "LeftEyebrowSlant"} -> Most[#]], ImageSize -> 100] & /@ chernoffData; Since "Petal length" and "Petal width" separate the classes well for those columns we have selected the parameters "EyeSize" and "LeftEyebrowSlant" based on [2]. #### 3. Finding patterns in a collection of faces Combine the faces into a image collage: ImageCollage[RandomSample[chfacesBW], Background -> White] [](http://i.imgur.com/zmBSQZR.gif) We can see that faces with small eyes tend have middle-lowered eyebrows, and that faces with large eyes tend to have middle raised eyebrows and large noses. #### 4. Chernoff faces colored by the species Make a Chernoff face for each record using the colors added to the rescaled data: chfaces = ChernoffFace[ AssociationThread[{"NoseLength", "LeftEyebrowTrim", "EyeSize", "LeftEyebrowSlant", "FaceColor"} -> #], ImageSize -> 100] & /@ chernoffData; Make an image collage with the obtained faces: ImageCollage[chfaces, Background -> White] [](http://i.imgur.com/uPBZJuf.gif) #### 5. Comparison We can see that the collage with colored faces completely explains the patterns found in the black-and-white faces: setosa have smaller petals (both length and width), and virginica have larger petals. ## Browsing a large number of records with Chernoff faces If we have a large number of records each comprised of a relative small number of numerical values we can use Chernoff faces to browse the data by taking small subsets of records. Here is an example using "Vinho Verde" wine quality dataset [8]. [](http://i.imgur.com/jDwXgfT.png) [](http://i.imgur.com/ZOqwuFT.png) ## Chernoff faces for lists of long lists In this section we consider data that is a list of lists. Each of the lists (or rows) is fairly long and represents values of the same variable or process. If the data is a full array, then we can say that in this section we deal with transposed versions of the data in the previous sections. Since each row is a list of many elements visualizing the rows directly with Chernoff faces would mean using a small fraction of the data. A natural approach in those situations is to summarize each row with a set of descriptive statistics and use Chernoff faces for the row summaries. The process is fairly straightforward; the rest of the section gives concrete code steps of executing it. ### Data generation Here we create 12 rows of 200 elements by selecting a probability distribution for each row. SeedRandom[1425] {dists, data} = Transpose@Table[( rdist = RandomChoice[{NormalDistribution[RandomReal[10], RandomReal[10]], PoissonDistribution[RandomReal[4]], GammaDistribution[RandomReal[{2, 6}], 2]}]; {rdist, RandomVariate[rdist, 200]}), {12}]; Dimensions[data] (* {12, 200} *) We have the following 12 "records" each with 200 "fields": DistributionChart[data, ChartLabels -> MapIndexed[ Row[{Style[#2[[1]], Red, Larger], " ", Style[#1, Larger]}] &, dists], ChartElementFunction -> ChartElementData["PointDensity", "ColorScheme" -> "SouthwestColors"], BarOrigin -> Left, GridLines -> Automatic, ImageSize -> 1000] [](http://i.imgur.com/HJWZo6T.gif) Here is the summary of the records: Grid[ArrayReshape[RecordsSummary[Transpose@N@data], {3, 4}], Dividers -> All, Alignment -> {Left}] [](http://i.imgur.com/HmIUX0d.png) ### Data transformation Here we "transform" each row into a vector of descriptive statistics: statFuncs = {Mean, StandardDeviation, Kurtosis, Median, QuartileDeviation, PearsonChiSquareTest}; sdata = Map[Through[statFuncs[#]] &, data]; Dimensions[sdata] (* {12, 6} *) Next we rescale the descriptive statistics data: sdata = VariablesRescale[sdata, "StandardizingFunction" -> (Standardize[#, Median, QuartileDeviation] &)]; For kurtosis we have to do special rescaling if we want to utilize the property that Gaussian processes have kurtosis 3: sdata[[All, 3]] = Rescale[#, {3, Max[#]}, {0.5, 1}] &@Map[Kurtosis, N[data]]; Here is the summary of the columns of the rescaled descriptive statistics array: Grid[{RecordsSummary[sdata, ToString /@ statFuncs]}, Dividers -> All] [](http://i.imgur.com/SHbLThq.png) ### Visualization First we define a function that computes and tabulates (descriptive) statistics over a record. Clear[TipTable] TipTable[vec_, statFuncs_, faceParts_] := Block[{}, GridTableForm[ Transpose@{faceParts, statFuncs, NumberForm[Chop[#], 2] & /@ Through[statFuncs[vec]]}, TableHeadings -> {"FacePart", "Statistic", "Value"}]] /; Length[statFuncs] == Length[faceParts]; To visualize the descriptive statistics of the records using Chernoff faces we have to select appropriate facial features. faceParts = {"NoseLength", "EyeSize", "EyeSlant", "EyesVerticalPosition", "FaceLength", "MouthSmile"}; TipTable[First@sdata, statFuncs, faceParts] [](http://i.imgur.com/bEOFgXH.png) One possible visualization of all records is with the following commands. Note the addition of the parameter "FaceColor" to also represent how close a standardized row is to a sample from Normal Distribution. {odFaceColor, ndFaceColor} = {White, ColorData[7, "ColorList"][[8]]}; Grid[ArrayReshape[Flatten@#, {4, 3}, ""], Dividers -> All, Alignment -> {Left, Top}] &@ MapThread[ (asc = AssociationThread[faceParts -> #2]; chFace = ChernoffFace[ Join[asc, <| "FaceColor" -> Blend[{odFaceColor, ndFaceColor}, #2[[-1]]], "IrisColor" -> GrayLevel[0.8], "NoseColor" -> ndFaceColor|>], ImageSize -> 120, AspectRatio -> Automatic]; tt = TipTable[N@#3, Join[statFuncs, {Last@statFuncs}], Join[faceParts, {"FaceColor"}]]; Column[{Style[#1, Red], Grid[{{Magnify[#4, 0.8], Tooltip[chFace, tt]}, {Magnify[tt, 0.7], SpanFromAbove}}, Alignment -> {Left, Top}]}]) & , {Range[Length[sdata]], sdata, data, dists}] [](http://i.imgur.com/CGVbZck.png) ## Visualizing similarity with nearest neighbors or recommendations ### General idea Assume the following scenario: 1. we have a set of items (movies, flowers, etc.), 2. we have picked one item, 3. we have computed the Nearest Neighbors (NNs) of that item, and 4. we want to visualize how much of a good fit the NNs are to the picked item. Conceptually we can translate the phrase "how good the found NNs (or recommendations) are" to: * "how similar the NNs are to the selected item", or * "how different the NNs are to the selected item." If we consider the picked item as the prototype of the most normal or central item then we can use Chernoff faces to visualize item's NNs deviations. **Remark:** Note that Chernoff faces provide similarity visualization most linked to Euclidean distance that to other distances. ### Concrete example The code in this section demonstrates how to visualize nearest neighbors by Chernoff faces variations. First we create a nearest neighbors finding function over the Fisher Iris data set (without the species class label): irisNNFunc = Nearest[irisDataSet[[All, 1 ;; -2]] -> Automatic, DistanceFunction -> EuclideanDistance] Here are nearest neighbors of some random row from the data. itemInd = 67; nnInds = irisNNFunc[irisDataSet[[itemInd, 1 ;; -2]], 20]; We can visualize the distances with of the obtained NNs with the prototype: ListPlot[Map[ EuclideanDistance[#, irisDataSet[[itemInd, 1 ;; -2]]] &, irisDataSet[[nnInds, 1 ;; -2]]]] [](http://i.imgur.com/dnUlWSD.gif) Next we subtract the prototype row from the NNs data rows, we standardize, and we rescale the interval $[ 0, 3 \sigma ]$ to $[ 0.5, 1 ]$: snns = Transpose@Map[ Clip[Rescale[ Standardize[#, 0 &, StandardDeviation], {0, 3}, {0.5, 1}], {0, 1}] &, Transpose@ Map[# - irisDataSet[[itemInd, 1 ;; -2]] &, irisDataSet[[nnInds, 1 ;; -2]]]]; Here is how the original NNs data row look like: GridTableForm[ Take[irisDataSet[[nnInds]], 12], TableHeadings -> irisColumnNames] [](http://i.imgur.com/7r6habk.png) And here is how the rescaled NNs data rows look like: GridTableForm[Take[snns, 12], TableHeadings -> Most[irisColumnNames]] [](http://i.imgur.com/P7xnwR5.png) Next we make Chernoff faces for the rescaled rows and present them in a easier to grasp way. We use the face parts: Take[Keys[ChernoffFace["FacePartsProperties"]], 4] (* {"FaceLength", "ForheadShape", "EyesVerticalPosition", "EyeSize"} *) To make the face comparison easier, the first face is the one of the prototype, each Chernoff face is drawn within the same rectangular frame, and the NNs indices are added on top of the faces. chfaces = ChernoffFace[#, Frame -> True, PlotRange -> {{-1, 1}, {-2, 1.5}}, FrameTicks -> False, ImageSize -> 100] & /@ snns; chfaces = MapThread[ ReplacePart[#1, 1 -> Append[#1[[1]], Text[Style[#2, Bold, Red], {0, 1.4}]]] &, {chfaces, nnInds}]; ImageCollage[chfaces, Background -> GrayLevel[0.95]] [](http://i.imgur.com/EKpJ7mj.gif) We can see that the first few - i.e. closest -- NNs have fairly normal looking faces. Note that using a large number of NNs would change the rescaled values and in that way the first NNs would appear more similar. ## References \[1\] Herman Chernoff (1973). "The Use of Faces to Represent Points in K-Dimensional Space Graphically" (PDF). Journal of the American Statistical Association (American Statistical Association) 68 (342): 361-368. doi:10.2307/2284077. JSTOR 2284077. URL: [http://lya.fciencias.unam.mx/rfuentes/faces-chernoff.pdf](http://lya.fciencias.unam.mx/rfuentes/faces-chernoff.pdf) . \[2\] Christopher J. Morris; David S. Ebert; Penny L. Rheingans, "Experimental analysis of the effectiveness of features in Chernoff faces", Proc. SPIE 3905, 28th AIPR Workshop: 3D Visualization for Data Exploration and Decision Making, (5 May 2000); doi: 10.1117/12.384865. URL: [http://www.research.ibm.com/people/c/cjmorris/publications/Chernoff_990402.pdf](http://www.research.ibm.com/people/c/cjmorris/publications/Chernoff_990402.pdf) . \[3\] Anton Antonov, [Chernoff Faces implementation in Mathematica](https://github.com/antononcube/MathematicaForPrediction/blob/master/ChernoffFaces.m), (2016), source code at [MathematicaForPrediction at GitHub](https://github.com/antononcube/MathematicaForPrediction), package [ChernofFacess.m](https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/ChernoffFaces.m) . \[4\] Anton Antonov, [MathematicaForPrediction utilities](https://github.com/antononcube/MathematicaForPrediction/blob/master/MathematicaForPredictionUtilities.m), (2014), source code [MathematicaForPrediction at GitHub](https://github.com/antononcube/MathematicaForPrediction), package [MathematicaForPredictionUtilities.m](https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/MathematicaForPredictionUtilities.m). \[5\] Anton Antonov, [Variable importance determination by classifiers implementation in Mathematica](https://github.com/antononcube/MathematicaForPrediction/blob/master/VariableImportanceByClassifiers.m)[, ](https://github.com/antononcube/MathematicaForPrediction/blob/master/IndependentComponentAnalysis.m)(2015), source code at [MathematicaForPrediction at GitHub](https://github.com/antononcube/MathematicaForPrediction), package [VariableImportanceByClassifiers.m](https://raw.githubusercontent.com/antononcube/MathematicaForPrediction/master/VariableImportanceByClassifiers.m). \[6\] Anton Antonov, ["Importance of variables investigation guide"](https://github.com/antononcube/MathematicaForPrediction/blob/master/Documentation/Importance-of-variables-investigation-guide.pdf), (2016), [MathematicaForPrediction at GitHub](https://github.com/antononcube/MathematicaForPrediction), [https://github.com/antononcube/MathematicaForPrediction](https://github.com/antononcube/MathematicaForPrediction), folder [Documentation](https://github.com/antononcube/MathematicaForPrediction/tree/master/Documentation). \[7\] Wikipedia entry, Iris flower data set, [https://en.wikipedia.org/wiki/Iris_flower_data _set](https://en.wikipedia.org/wiki/Iris_flower_data_set) . \[8\] P. Cortez, A. Cerdeira, F. Almeida, T. Matos and J. Reis. Modeling wine preferences by data mining from physicochemical properties. In Decision Support Systems, Elsevier, 47(4):547-553, 2009. URL [https://archive.ics.uci.edu/ml/datasets/Wine+Quality](https://archive.ics.uci.edu/ml/datasets/Wine+Quality) . Anton Antonov 2016-06-03T02:17:45Z https://community.wolfram.com/groups/-/m/t/2434669 > *GitHub repository:* https://github.com/ben-izd/cDateFunctions Recently, I was working on an [infographic](https://mathematica.meta.stackexchange.com/a/2608/77079) which heavily involves working with dates and times. After using `DateObject` and some date-related functions, it surprises me how slow they are (in the context of tens of thousands of samples). So I decided to rewrite the main one (`DateDifference`) in Wolfram language. But after seeing the performance boost gained by rewriting, I decided to write some other functions too. After a couple of months, I end up rewriting 19 functions in wolfram language and 9 in kernel-level (with Rust), in this post you'll see the result and interesting points I discovered in this journey. Before we start, remember these notes: - Most of the implementations were done with minimum options, many of them do not support options that their built-in ones support (like `TimeZone`, `TimeSystem`, ...) and functions in the first section written in pure Wolfram language without using `Compile` or related functions - Default calendar is `"Gregorian"`, which doesn't have year `0` (is also true for `"ArithmeticPersian"` calendar) unless it specified - In the 'Possible Issues/Bugs + Suggestions' section order is not important, some of them relate to the design decisions that developers decided (it would be awesome to shed some light on them) - All the implementations have the same name with 'c' added to start (`DayName` is `cDayName`, which has nothing to do with `C` language) - Some built-in functions were not `Listable`, which is understandable because of date formats like `DateList`, but some of my implementations are `Listable` - All the functions were tested with a wide range of random dates and numbers - Except some cases which will be discussed. The result of the functions with their built-in ones are equal. If you find any case with a wrong result, please comment it - Platform is Mathematica 13.0.0 for Microsoft Windows 10 20H2 (64-bit) on AMD Ryzen 1700 with 16 GB RAM with time-zone offset: +3.5 - You can access the code in [GitHub](https://github.com/ben-izd/cDateFunctions) which also include the `LibraryLink` section (`DLL` files and their source code) # Performance Comparison Here are the result of comparing c* functions with their built-in ones in terms of timing (`RepeatedTiming` were used). Tested on different input formats and the number shown here is the floored average of the result. ## LeapYearQ  ## DayName  ## BusinessDayQ  ## DayMatchQ  ## DayRound  ## DateBounds  ## DateOverlapQ  ## DateWithinQ  ## DayCount  ## CurrentDate  ## NextDate  ## PreviousDate  ## DateDifference  ## DayPlus  ## DatePlus   ## DayRangee  ## DateRange  # Possible Issues/Bugs + Suggestions During testing and developing the code, I notice some strange/unexpected cases + some ideas which I'll discuss for each function. ## LeapYearQ ### 1. `"ArithmeticPersian"` uses `"Gregorian"` formula With `CalendarType -> "ArithmeticPersian"` input will be converted to `"Gregorian"` calendar and the result will be calculated like `CalendarType -> "Gregorian"` (note that they have different formulas): ``` LeapYearQ[{1403, 1, 1}, CalendarType -> "ArithmeticPersian"] (* Out: True *) LeapYearQ[{1403, 12, 1}, CalendarType -> "ArithmeticPersian"] (* Out: False *) ``` The reason for above results is that date `{1403, 1, 1}` in `"ArithmeticPersian"` is `{2024, 3, 20}` in `"Gregorian"` which is a leap year in `"Gregorian"` but `{1403, 12, 1}` fall into `{2025, 2, 19}` which is not a leap year, according to `"Gregorian"` calendar. More comprehensive test: ``` And @@ ( LeapYearQ[{#}, CalendarType -> "ArithmeticPersian"] === LeapYearQ[ CalendarConvert[ DateObject[{#}, CalendarType -> "ArithmeticPersian"], "Gregorian"], CalendarType -> "Gregorian"] & /@ Range[1, 5000]) (* Out: True *) ``` Note that in `"ArithmeticPersian"` leap year gap (which has different methods to calculate, Mathematica uses 2820 period) is 4 and sometimes 5 years which is different than `"Gregorian"`. ## BusinessDayQ ### 1. Handling `TimeObject` without raising any message (related to AbsoluteTime-1 problem) When using `TimeObject` in `BusinessDayQ`, it will be converted to a date and return a result! instead of raising an error: ``` DayName[] (* Out: Thursday *) BusinessDayQ[TimeObject[]] (* Out: True *) ``` ## DayMatchQ ### 1. Missing/displace Veterans Day Veterans Day which starts in 1938, on YYYY-11-11 except 1971-1978, which was on Oct 4th Monday [\[Source\]](https://www.timeanddate.com/holidays/us/veterans-day). Considering observed holidays (holiday fall on Saturday/Sunday the day before/after will be observed day), a holiday could be either the specified day or a day before or after. ``` Select[Range[1938, 2050], Not@Or[DayMatchQ[{#, 11, 10}, "Holiday"], DayMatchQ[{#, 11, 11}, "Holiday"], DayMatchQ[{#, 11, 12}, "Holiday"]] &] ``` Result: ``` {1939, 1940, 1944, 1950, 1961, 1967, 1972, 1978, 1989, 1995, 2000, 2006, 2017, 2023, 2028, 2034, 2045} ``` which if you apply `Differences`, we'll get a pattern: ``` {1, 4, 6, 11, 6, 5, 6, 11, 6, 5, 6, 11, 6, 5, 6, 11} ``` Some years are missing the holiday and in the range of 1971-1978 it should not be on YYYY-11-11, but for some years it is. Correct result (use `cDayMatchQ` instead of `DayMatchQ`): ``` {1971, 1972, 1973, 1974, 1975, 1976, 1977} ``` Also, it misses some dates in the future [\[Source\]](https://www.timeanddate.com/calendar/?year=2045&country=1): ``` DayMatchQ[{2045, 11, 10}, "Holiday"] (* Out: False *) (* 1 day before/after also is not a holiday*) DayMatchQ[{2045, 11, 9}, "Holiday"] (* Out: False *) DayMatchQ[{2045, 11, 11}, "Holiday"] (* Out: False *) ``` ### 2. Handling `TimeObject` (related to AbsoluteTime-1 problem) When using `TimeObject` in `DayMatchQ`, it will convert it to a date and return the result ! instead of raising an error: ``` DayName[] (* Out: Thursday *) DayMatchQ[TimeObject[], Thursday] (* Out: True*) ``` ### 3. Documentation typo Based on [`DayMatchQ`](http://reference.wolfram.com/language/ref/DayMatchQ.html) documentation, default value for `DayMatchQ` is `All` while giving error to a single argument: ``` DayMatchQ[{2021, 11, 12}] ```  ## DateWithinQ ### 1. Does not support `DateList` format for input: The documentation notes that input should be `DateObject` of any calendar, it's a good feature to support different calendars but what about `DateList` format? What's the reason behind it? Is it much different from `DateDifference` which supports this format? ``` DateWithinQ[{2021}, {2021, 1}] ``` Also, it should be noted that a *Q function in this example returned an `Unevaluated` expression instead of a Boolean, is this a normal behavior? ### 2. Returning `Unevaluated` expression without raising any message: ``` DateWithinQ[DateObject@{2020, 1, 1, 1, 1, 1}, DateObject@{2020, 1, 1}] ```  ## DayCount ### 1. Round results If it's a day and more than half, it adds one day, otherwise, it doesn't (opposite of what documentation in `Properties & Relations` says, equal to the length of `DayRange` with some options). ``` DayCount[{2020, 1, 1}, {2020, 1, 1, 11}, All] (* Out: 0 *) DayCount[{2020, 1, 1}, {2020, 1, 1, 13}, All] (* Out: 1 *) ``` `cDayCount` returns 0 for both of the above cases. ### 2. Wrong result in year 1 ``` DayCount[{1, 12, 23}, {2, 3, 25}, "BeginningOfMonth"] (* Out: 13 *) ``` ## NextDate ### 1. Inconsistent behavior When using a granularity with an input that doesn't have enough precision, on most of the types, the upper-bound will be used as a start date to find the next occurrences but not for weekdays (not exactly the upper-bound is used). ``` NextDate[{2020, 1}, "Day"] (* Out: DateObject[{2020,2,1}, "Day", "Gregorian", 3.5`] *) NextDate[{2020, 1}, Sunday] (* Out: DateObject[{2020,1,5}, "Day", "Gregorian", 3.5`] *) ``` Also, note that calendar type and time-zone offset was added to the output (input argument does not have those). ### 2. Handling `TimeObject` `NextDate` support `TimeObject`. It can even give us the next `Day` of a `TimeObject`: ``` First@TimeObject[] (* Out: {15, 38, 31.} *) First@NextDate[TimeObject[], "Hour"] (* Out: {2021, 11, 14, 16} *) First@NextDate[TimeObject[], "Day"] (* Out: {2021, 11, 15} *) ``` `cNextDate` result: ``` {16} ``` ### 3. Wrong result in year 1 or -1 ``` DateList@NextDate[{1, 12, 1}, "BeginningOfMonth"] (* Out: {1, 2, 1, 0, 0, 0.} *) DateList@NextDate[{-1, 12, 1, 0, 0, 0}, "Quarter"] (* Out: {-1, 1, 1, 0, 0, 0.} *) ``` Correct result (`cNextDate`): ``` {2, 1, 1, 0, 0, 0.} {1, 1, 1, 0, 0, 0.} ``` ### 4. Inconsistent keyword in documentation [`NextDate`](http://reference.wolfram.com/language/ref/NextDate.html) documentation uses `"MonthFirstDay"` and `"MonthLastDay"` instead of `"BeginningOfMonth"` and `"EndOfMonth"`. ## DateDifference ### 1. Type "quarter"/"Year" includes year 0 As noted in the beginning, the `"Gregorian"` calendar in Mathematica doesn't have year 0 but in these cases it does! ``` DateDifference[{-1, 9, 1}, {1, 9, 1}, "Quarter"] (* Out: Quantity[9.96739, "QuarterYears"] *) ``` Correct Result (using `cDateDifference`): ``` Quantity[4., "QuarterYears"] ``` How many quarters do you see in this picture?  Another example: ``` DateDifference[{-1, 10, 1, 0, 0, 0}, {1, 1, 1, 0, 0, 0}, "Year"] (* Out: Quantity[1.74795, "Years"] *) ``` ### 2. Slightly different result ``` DateDifference[{2020, 3, 5, 15}, {2020, 7, 15, 5}, "Week"] DateDifference[AbsoluteTime@{2020, 3, 5, 15}, AbsoluteTime@{2020, 7, 15, 5}, "Week"] ``` Result (see the last digits): ``` Quantity[18.797619047619047, "Weeks"] Quantity[18.797619047619044, "Weeks"] ``` `cDateDifference` returns the first result. The real decimal can be calculated manually, which is `67/84` or: ``` 0.7976190476190476 ``` ### 3. Raise error for multiple units including `Decade` /`Century` / `Millennium` ``` DateDifference[{2020, 1, 5}, {2120, 1, 4}, "Decade"] DateDifference[{2020, 1, 5}, {2120, 1, 4}, {"Decade", "Year"}] cDateDifference[{2020, 1, 5}, {2120, 1, 4}, {"Decade", "Year"}] ```  ## DayPlus ### 1. Does not support `Week` It supports `"BeginningOfMonth"` and `"EndOfMonth"` but not `"Week"`: ``` DayPlus[{2020, 1, 1}, 1, "Week"] ```  Possible with `cDayPlus`: ``` {2020, 1, 8} ``` ## DatePlus ### 1. Different results with Quantity and List ``` DatePlus[{2020, 1, 1}, {1, "Quarter"}] (* Out: {2020, 4, 1} *) DatePlus[{2020, 1, 1}, Quantity[1, "QuarterYears"]] (* Out: {2020, 4, 1, 12} *) ``` ### 2. Odd Behavior with year 1 ``` DatePlus[{1, 8, 15}, {3, "EndOfMonth"}] (* Out: {1, 3, 31} *) ``` Correct result (with `cDatePlus`): ``` {1, 10, 31} ``` Another example: ``` DatePlus[{1, 10, 5}, {2, "BeginningOfMonth"}] (* Out: {1, 3, 1} *) ``` Correct result: ``` {1, 12, 1} ``` ### 3. Strange result with an empty list When we use an empty list (`{}`) as offset, it uses the first argument as day offset from now: ``` First@DatePlus[0, {}] (* Out: {2021, 11, 13, 15, 7, 0.} *) First@DatePlus[1, {}] (* Out: {2021, 11, 14, 15, 7, 0.} *) DatePlus[0, {1, "Day"}] (* Out: 86400 *) ``` ## DayRange ### 1. Odd behavior with year 1 or -1 It starts from `1-1-31`, just to avoid clutter, you'll see the length: ``` Length@DayRange[{1, 7, 9}, {2, 2, 15}, "EndOfMonth"] (* Out: 13 *) Length@DayRange[{-1, 7, 9}, {1, 2, 15}, "EndOfMonth"] (* Out: 12 *) ``` Proper result (with `cDayRange`): ``` 7 7 ``` ## DateRange ### 1. The backward calculation is done using the forward formula When the end date is larger than the start date, we add the granularity (should be positive) to the start until we reach the end, but if the start is bigger than the end and granularity is negative, we should subtract it from the start until we reach the end. It's true for all the types except `"BeginningOfMonth"` and `"EndOfMonth"`. ``` DateRange[{2019, 12, 5}, {2019, 7, 1}, {-3, "BeginningOfMonth"}] (* Out: {{2019, 10, 1, 0, 0, 0.}, {2019, 7, 1, 0, 0, 0.}} *) ``` Which should be (using `cDateRange`): ``` {{2019, 12, 1, 0, 0, 0.}, {2019, 9, 1, 0, 0, 0.}} ``` Pay attention to hour element: ``` (* Normal *) DateRange[{2019, 12, 5, 12}, {2019, 7, 1}, {-3, "Month"}] (* Out: {{2019, 12, 5, 12, 0, 0.}, {2019, 9, 5, 12, 0, 0.}} *) (* Odd *) DateRange[{2019, 12, 5, 12}, {2019, 7, 1}, {-3, "EndOfMonth"}] (* Out: {{2019, 10, 31, 0, 0, 0.}, {2019, 7, 31, 0, 0, 0.}} *) (* Odd *) DateRange[{2019, 12, 5, 12}, {2019, 7, 1}, {-3, "BeginningOfMonth"}] (* Out: {{2019, 10, 1, 0, 0, 0.}, {2019, 7, 1, 0, 0, 0.}} *) ``` In other terms, the result of the examples below are equal except in different ordering: ``` DateRange[{2019, 12, 5}, {2019, 7, 1}, {-3, "BeginningOfMonth"}] (* Out: {{2019, 10, 1, 0, 0, 0.}, {2019, 7, 1, 0, 0, 0.}} *) DateRange[{2019, 7, 1}, {2019, 12, 5}, {3, "BeginningOfMonth"}] (* Out: {{2019, 7, 1, 0, 0, 0.}, {2019, 10, 1, 0, 0, 0.}} *) ``` ## AbsoluteTime ### 1. Handling `TimeObject` `AbsoluteTime` convert `TimeObject` by adding `Today` value, which I think could be counterintuitive when somebody wants to compare `AbsoluteTime` of two `TimeObject` which was applied in different days, should use `Mod[,86400]` first. ``` DateList@AbsoluteTime@TimeObject[] (* Out: {2021, 11, 13, 15, 42, 56.} *) ``` Why not return this result ( `cAbsoluteTime`): ``` {1, 1, 1, 15, 42, 56.} ``` ## UnixTime ### 1. Truncating sub-second precision: Why `UnixTime` does not have floating numbers to handle more accurate dates? ``` DecimalForm[UnixTime[{2021, 11, 12, 12, 30, 30.35}], 30] (* Out: 1636700430 *) ``` What it could be: `1636707630.35`. ## DateObject ### 1. Raise overlapping message for non-overlap inputs For years, before `812` and after `2989` equality of two consecutive days raise `Message` while `DateOverlapsQ` returning `False`: ``` DateObject[{-1, 1, 1}] == DateObject[{-1, 1, 2}] DateOverlapsQ[DateObject[{-1, 1, 1}], DateObject[{-1, 1, 2}]] (* Out: False *) DateOverlapsQ[DateObject[{-1, 1, 2}], DateObject[{-1, 1, 1}]] (* Out: False *) ```  Far future example: ``` DateObject[{2988, 10, 12}] == DateObject[{2988, 10, 13}] (* No Message, Out: False*) (* But *) DateObject[{2989, 10, 12}] == DateObject[{2989, 10, 13}] ```  ### 2. Suggestion - Round 'Second' in `Second` granularity Is this considered `Second` granularity or `Sub-Second`/`Instant` granularity ? ``` DateObject[{2020, 1, 1, 30, 45, 12.25}, "Second"] (* Out: DateObject[List[2020,1,2,6,45,12.25`],"Second","Gregorian",3.5`] *) ``` ### 3. Incorrect result with AbsoluteTime values with custom granularity Consider date `{2020, 12, 10}` which is `Thursday` and every week starts on Monday (default option): ``` DateList@DateObject[3816547200, "Week"] (* Out: {2020, 12, 10, 0, 0, 0.} *) DateList@DateObject[DateList@3816547200, "Week"] (* Out: {2020, 12, 7, 0, 0, 0.} *) DateList@DateObject[FromAbsoluteTime@3816547200, "Week"] (* Out: {2020, 12, 7, 0, 0, 0.} *) ``` This also applies to `WeekBeginningSunday`, `Century`, `CenturyBeginning01`, `Millennium`, `MillenniumBeginning01`. ## TimeObject ### 1. Suggestion - Embed TimeZone offset to TimeObject like DateObject Since `DateObject` uses `$TimeZone` and can be manipulated in a session while `TimeObject` uses system clock, why not use the time-zone offset from the system ? (My system time-zone offset is +3.5) ``` First@TimeObject[] (* Out: {14, 4, 3.} *) First@TimeZoneConvert[TimeObject[], "GMT"] (* Out: {14, 4, 3.} *) First@TimeZoneConvert[TimeObject[TimeZone -> 3.5], "GMT"] (* Out: {10, 34, 3.} *) (* Real answer *) TimeZoneConvert[DateObject[], "GMT"][[1, -3 ;;]] (* Out: {10, 34, 3.} *) ``` ## CalendarConvert ### 1. Negative/Zero in the month! For far future years, conversion to `"AstronomicalPersian"` will give negative/zero months ``` First@CalendarConvert[DateObject@{9999997, 9, 24}, "AstronomicalPersian"] (* Out: {9999386, -15, 5} *) ``` Another example (wrong output probably starts from this year): ``` First@CalendarConvert[DateObject@{82380, 7, 4}, "AstronomicalPersian"] (* Out: {81758, 12, 29} *) First@CalendarConvert[DateObject@{82380, 7, 5}, "AstronomicalPersian"] (* Out: {81759, 0, 26} *) ``` If we don't apply `First` to the `CalendarConvert` you'll see:  ### 2. Two consecutive leap years in the Persian calendar Persian Calendar has a little complex leap year system, that's why it has an `"ArithmeticPersian"` and `"AstronomicalPersian"` option in calendar conversion. `"AstronomicalPersian"` type is the more accurate system. I don't think two consecutive leap years is correct. Also, I should mention, leap month in the `Persian` calendar is the 12th month which in normal years has 29 days, and leap years have 30 days. ``` First@CalendarConvert[DateObject@{622, 3, 21}, "AstronomicalPersian"] First@CalendarConvert[DateObject@{621, 3, 21}, "AstronomicalPersian"] ``` If we calculate with `cCalendarConvert`: ``` {-1, 12, 30, 0, 0, 0.} {-1, 1, 1, 0, 0, 0.} ``` ## TimeZoneConvert ### 1. Wrong result For version 13 live stream, I want to schedule to watch it live, but I live in a different time-zone, so I notice this strange case: First to show the real offset from my place to `New_York`: ``` TimeZoneOffset["America/New_York", "Asia/Tehran", DateObject[{2021, 12, 13, 15, 30}, TimeZone -> "America/New_York"]] (* Out: -8.5 *) ``` Now, with `8.5` hour offset, convert the date to the Persian calendar, then convert time zones: ``` First@TimeZoneConvert[ CalendarConvert[ DateObject[{2021, 12, 13, 15, 30}, TimeZone -> "America/New_York"], "AstronomicalPersian"], "Asia/Tehran"] (* Out: {1400, 9, 22, 23, 51} *) ``` We get `8h 21m` offset! Further investigation, shows that the offset used in the calculation may differ from the real one: ``` First@TimeZoneConvert[ DateObject[{1400, 9, 22, 3, 31, 15}, CalendarType -> "AstronomicalPersian"], "Europe/London"] (* Out: {1400, 9, 22, 0, 0, 0.} *) ``` If we change the destination to `GMT` the correct result shows up: ``` First@TimeZoneConvert[ DateObject[{1400, 9, 22, 3, 30, 0}, CalendarType -> "AstronomicalPersian"], "GMT"] (* Out: {1400, 9, 22, 0, 0, 0.} *) ``` # Bonus If you have read this far, congratulations. These functions were absent in wolfram but are useful to have, so here you are: ## cHolidayName For getting the name of a holiday: ``` cHolidayName[{2021, 11, 25}] (* Out: "Thanksgiving Day" *) ``` Observed Holidays, the original day is not considered a holiday: ``` cHolidayName[{2021, 7, 5}] (* Out: "Independence Day' day off" *) cHolidayName[{2021, 7, 4}] (* Out: Missing["Holiday"] *) ``` Also if you're more interested in holidays, there are two related internal functions for getting next/previous holiday: ``` DateList@cDateFunctions`Private`cNextHoliday[{2021, 11, 12}] (* Out: {2021, 11, 25, 0, 0, 0.} *) DateList@cDateFunctions`Private`cPreviousHoliday[{2021, 11, 12}] (* Out: {2021, 11, 11, 0, 0, 0.} *) ``` Also for getting the previous holiday, there is a lower limit: ``` cDateFunctions`Private`cPreviousHoliday[{1500, 11, 11}] (* Message: "No official holiday exists before 1863-11-26." *) ``` ## cCalendarView For getting a month/multiple month/custom range calendar views with `Grid` which supports highlighting and is only for `"Gregorian"` calendar (See Advance section for other calendars): ### View a month at a glance: ``` cCalendarView[{2021, 11}] ```  ``` cCalendarView[{2021, 11}, {2021, 11, 12} -> Red] ```  ``` cCalendarView[{2021, 11}, 12 ;; 15] ```  ``` cCalendarView[{2021, 11}, {6, 7} ;; ;; 7] ```  ``` cCalendarView[{2021, 11}, "Holiday"] ```  Use `cCalendarMultipleView` to view multiple months in a row. Could be positive or negative: ``` (* show 2021-11 and 1 month after that with highlighting *) cCalendarMultipleView[{2021, 11}, 1, 6 ;; ;; 6] ```  Use `cCalendarRangeView` to view a range: ``` cCalendarRangeView[{2021, 11}, {2021,12}, {{{2021, 11, 14}, {2021, 11, 15}, {2021, 11, 16}} -> Gray, {2021, 11, 13} -> Green}] ```  Note that `cCalendarMultipleView` and `cCalendarRangeView` automatically highlight the first day of each month. ## Suggestion - N-th Weekday How should we program to express 4th Thursday (for Thanksgiving ) / last Monday (for Memorial Day) ? Here is my proposal inspired by the `C++ Chrono` library, implemented via `UpValues` (minimum viable product): ``` (Unprotect[#]; UpValues[#] = {}; # /: {year_Integer, month_Integer, #[n_Integer?Positive]} := DatePlus[{year, month, 0}, {n, #}]; # /: {year_Integer, month_Integer, #[n_Integer?Negative]} := DatePlus[{year, month + 1, 1}, {n, #}]; # /: {year_Integer, month_Integer, #[First]} := DatePlus[{year, month, 0}, {1, #}]; # /: {year_Integer, month_Integer, #[Last]} := DatePlus[{year, month + 1, 1}, {-1, #}]; Protected[#];) & /@ {Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday}; ``` Now, we can use it anywhere (should be fixed): ``` {2021, 11, Thursday[4]} (* Out: {2021, 11, 25} *) {2021, 5, Monday[Last]} (* Out: {2021, 5, 31} *) AbsoluteTime[{2021, 5, Monday[Last]}] (* Out: 3831408000 *) (* Second to last friday *) {2021, 12, Friday[-2]} (* Out: {2021, 12, 24} *) ``` Limiting UpValues to date functions only and supporting month names, would greatly improve functionality and readability. # For Advanced Users This journey of rewriting date functions in wolfram as joyful and teachable as it was, was ending and after looking at my code, I notice in many places, `DateList` and `AbsoluteTime` were used many times, I thought, if I could make them faster, the whole package could benefit this speed up. In many parts, I reached my goal but I didn't test the performance gained from switching. Since these functions are Kernel-Level functions (probably written in `C` language), Wolfram Language is not the right tool to compete, so I choose Rust. You could read about its benefits and guarantees over `C` on other sites. Here you'll see the comparison. `C` language is still used as the interface for Rust functions. (If you have a better solution, I'm more than happy to hear about Rust integration.) This journey just like the previous one doesn't end with `DateList` and `AbsoluteTime`, but it also came with writing 3 calendars (`"Greogiran"`, `"ArithmeticPersian"` with 5 method, `AstronomicalPersian`, `"Islamic"` which is a civil arithmetic calendar, with 4 method). For additional options they support, visit the [Github Page](https://github.com/ben-izd/cDateFunctions). ## Installation For testing this section, you need to follow the [Github](https://github.com/ben-izd/cDateFunctions) instructions. ## `AbsoluteTime` / `FromAbsoluteTime` `cAbsoluteTime` is on average **0.9 times slower** than `AbsoluteTime` over 1,000,000 random `DateList`. `cAbsoluteTime` is on average the **2 times faster** as `AbsoluteTime` over 100,000 random `DateObject`. If we call `AbsoluteTime` to get the current value without any argument, `cAbsoluteTime` is on average **42 times faster** than `AbsoluteTime` while having **6 times lower** memory usage. `cFromAbsoluteTime` on the other hand is **1.5 times faster** than `FromAbsoluteTime` over 100,000 random real, which could be faster considering creating a `DateObject` overhead (see Comparing Internals section). Also unlike Mathematica, `cAbsoluteTime` does support other calendars: ``` cDateList@cAbsoluteTime[{1400, 7, 20}, CalendarType -> "ArithmeticPersian"] (* Out: {2021, 10, 12, 0, 0, 0} *) cDateList@cAbsoluteTime[{1443, 3, 5}, CalendarType -> "Islamic"] (* Out: {2021, 10, 12, 0, 0, 0} *) ``` ## `DateList` `cDateList` is on average **~1.7 times faster** than `DateList` over 1,000,000 random numbers (`Real`/`Integer`). `cDateList` is on average **~1.9 times faster** than `DateList` over 1,000,000 random un-normal `DateList`. ## `CalendarConvert` All the numbers come from testing 10,000 random samples. In `"Gregorian"` to `"ArithmeticPersian"`\ `cCalendarConvert` is on average **16 times faster** than `CalendarConvert`. In `"Gregorian"` to `"AstronomicalPersian"`\ `cCalendarConvert` is on average **140 times faster** than `CalendarConvert`. In `"Gregorian"` to `"Islamic"`\ `cCalendarConvert` is on average **9 times faster** than `CalendarConvert`. In `"Islamic"` to `"ArithmeticPersian"`\ `cCalendarConvert` is on average **19 times faster** than `CalendarConvert`. `"AstronomicalPersian"` uses an algorithm introduced by Edward M. Reingold, Nachum Dershowitz in their book, "Calendrical Calculations: The Ultimate Edition". It was implemented to perform the test but because of its license, it's not included in the source code until I get approval from the author (the source is commented in the interface code). Since both `"Islamic"` and `"ArithmeticPersian"` have methods to use, converting one to another while using `Method` option, input `Method` will be used for the output format and input will be converted using the default method, if you want more control, first convert it to `"Gregorian"`, then convert it to your specified calendar. ## `LeapYearQ` Because of different methods of calculations, this `LeapYearQ` uses kernel-level functions (instead of pure wolfram language shown earlier, which will be overridden) to handle 3 calendars. Tests were done on over 100,000 random samples. `"Gregorian"`\ `cLeapYearQ` is on average **8 times faster** than `LeapYearQ`. `"ArithmeticPersian"`\ `cLeapYearQ` is on average **9 times faster** than `LeapYearQ` (as discussed earlier, `LeapYearQ` gives the wrong result for this type of calendar). `"AstronomicalPersian"`\ `cLeapYearQ` is on average **102 times faster** than `LeapYearQ`. `"Islamic"`\ `cLeapYearQ` is on average **6 times faster** than `LeapYearQ`. And remember in `cLeapYearQ` with `"ArithmeticPersian"` and `"Islamic"`, you can choose different methods of leap-year calculation, visit [Github](https://github.com/ben-izd/cDateFunctions) for more information. ## `JulianDate` / `FromJulianDate` `cJulianDate` is on average **3 times faster** than `JulianDate` over 10,000 random `DateList`. `cFromJulianDate` is on average **1.8 times faster** than `FromJulianDate` over 100,000 real, actually, it's faster than stated, creating a `DateObject` takes most of the time. ## `UnixTime` / `FromUnixTime` `cUnixTime` is on average **7 times faster** than `UnixTime` over 100,000 random `DateList` samples. `cFromUnixTime` is on average **2 times faster** than `FromUnixTime` but it shares the same problem as `cFromJulianDate`. ## `cMonthView` / `cPartialView` With access to these calendars and different methods of calculation, it was the best opportunity to create a function that prints month/year view. See 2021 calendar at a glance:  or other calendars (use `CalendarType`)  Unlike earlier `cMonthView` it does not support highlighting yet. For printing a range/month continuously use `cPartialView` which accepts a start and end date. Wolfram technology conference at a glance (with manually highlighting): ``` MapAt[Highlighted, cPartialView[{2021, 11, 7}, {2021, 11, 20}], {{1, 1, 1, 6 ;; 7}, {1, 1, 2, 1 ;; 2}}] ```  # Limitations Since all these functions are written in low-level and should support `C` types, not arbitrary precision that Wolfram-Language has (although Wolfram has its own limitation, like rounding error). ## `cAbsoluteTime`: `cAbsoluteTime` has a limit, going further will not raise any error but cause overflow (wrap around). ``` (* max date - second is not important *) cAbsoluteTime[{292277026526, 12, 5, 15, 30}] (* Out: 9223372036854720000 *) (* going further lead to overflow *) cAbsoluteTime[{292277026526, 12, 5, 15, 31}] (* Out: -9223372036854775756 *) (* min date*) cAbsoluteTime[{-292277022728, 1, 27, 0, 0, 0}] (* Out: -9223372036854720000 *) (* going further lead to overflow *) cAbsoluteTime[{-292277022728, 1, 26, 0, 0, 0}] (* Out: 9223372036854745216 *) ``` ## `cDateList`: Unlike `cAbsoluteTime` which crossing the limit will not raise any error, here Mathematica will raise an error because it can not convert numbers greater than `2^63-1` to machine size 64-bit integers. But because of some internal design decisions, the safe range is shifted towards the negative side. So there is also a range before `2^63-1`, which doesn't raise any error but leads to overflow. (We're discussing around year `292,277,024,627`, which if you're interested, I'll add more details.) This pattern also applies to other kernel-level functions. # Comparing `Internal`s In the built-in `Internal` context, there are three functions that do the `DateList` functionalities separately: - `DateListToDateList` - `DateListToSeconds` - `SecondsToDateList` Like Mathematica, c\* functions also have kernel-level functions to call but there is a caveat. Since passing arrays to/from `C` language (interface), should be a single type, I couldn't find an easy way to handle `DateList` mix-type arrays beside `WSTP` (second could be Real, `LibraryLink` developers, if you're reading, I'm waiting for integration of `TypeProduct` to use structs), so these kernel functions only support maximum 5 elements as input/output (`{Y,M,D,H,M}`, second will be calculated in the interface, not in the kernel part). Comparing Mathematica `Internal` with `cDateFunctions` > `LibraryLink` over random 1,000,000 samples without considering second (`Internal` support second but not `cDatefunctions Kernel`): `DateListToSecond` (`AbsoluteTime`): `LibraryLink` is on average **0.7 times slower** than `Internal`. `SecondToDateList` (`DateList`): `LibraryLink` is on average **0.8 times faster** than `Internal`. `DateListToDateList`: `LibraryLink` is on average **1.12 times faster** than `Internal`. Calling raw `LibraryLink` outperforms the internals but input/output doesn't include second, the result shown above came from calling the simplest interface. (For those curious people, calling raw interface is on average **8, 7, 9 times faster** respectively). As discussed, `cDateFunctions`\``LibraryLink` is fast but the reason that their interface is not as fast as their kernel, is mainly because of the pattern matching (to support `CalendarType` and `Method`) and second calculation. # Final note For those who are interested in calendars and their implementations, I recommend "Calendrical Calculation" by Edward M. Reingold, Nachum Dershowitz which I think the Wolfram/Microsoft team has seen this book because of some similarities I noticed in their sources. Both book's authors and developer teams did a great job which is appreciated (Microsoft: .Net). The book also comes with a source file in `Lisp` language. The implementation of `cCalendarConvert` `"AstronomicalPersian"` is ported from this book. Because of my limited time, I did cover 4 calendars and introduced different methods of calculation to cover different needs and you should know both `"Persian"` and `"Islamic"` in some countries use observation not algorithms to determine leap year/month length. The Rust code has a framework within, to support other calendars with similar characteristics as Gregorian and ... , with providing months length and two functions for checking and counting leap years and some little details, you'll have `AbsoluteTime` and `DateList` engine + `Month/YearView` (currently for 12 months calendars) for your calendar for free. `"ArithmeticPersian"` and `"Islamic"` also use frameworks, so for any future changes in their leap year calculations that follow the old structure, you can add them as a new method under 20 lines of code. Improving is a never-ending process. For me, I reached beyond the main goal I set for myself, of course, there were a lot of moments which I was so close to abandoning the project, especially when I was figuring out how `LibraryLink` works and build the project without compiling errors but looking back, it's worth it and I feel more powerful than before. I hope you learn something and if you had any questions/comments/suggestions, feel free to add them. Benjamin Izadpanah 2022-01-01T06:44:51Z