Version control of Mathematica notebooks to include it in git repositories

Questions
1

In order to clean the Mathematica notebooks to meet the requirement of git version control, I noticed the following code snippets here:

CleanNotebook[file_] := Module[{nb, contents, newcontents},
   nb = NotebookOpen[file];
   SetOptions[nb, "TrackCellChangeTimes" -> False, 
    PrivateNotebookOptions -> {"FileOutlineCache" -> False}];
   contents = NotebookGet[nb];
   newcontents = 
    contents /. {(CellChangeTimes -> _) -> 
       Sequence[], (CellTags -> _) -> Sequence[]};
   NotebookPut[newcontents, nb];
   NotebookSave[nb];
   NotebookClose[nb];
   ];

But based on tries by re-evaluating, the notebook still changes a lot even after using the above code snippet.

So, I wonder if there is any good way to clean up Mathematica notebooks and track (by continuous commits) them in the git repository easily.

Regards,
HZ

2

The expression for a notebook is quite verbose so a small change in the frontend appearance can result in many changes to the expression. It is not easy to understand what has changed by looking at the textual differences. Trying to resolve merge conflicts is very painful. If you want version control use packages or save notebooks as .m / .wl and version control them.

3

If you want version control use packages or save notebooks as .m / .wl and version control them.

But this doesn’t have the capability of tracking image or other multimedia related content.

4

Hello Hongyi Zhao,

I think the following answer of mine can help you:

5

@Alexey Popkov

Thank you for your wonderful comments and sharing your valuable experience. But your method seems quite complicated. I need to learn it. If there is any problem, I will give follow-up feedback.

6

Here is an improved version of the original code snippet which should fit your purposes much better:

CleanNotebookForGit[inputFile_, outputFile_] := 
  Module[{contents = Get[inputFile], newcontents, 
    uniqName = "$$$MyUniqueCellHead$$$" <> ToString[RandomInteger[{10^8, 10^10}]]},
   newcontents = 
    contents /. {HoldPattern[
        CellLabel | CellChangeTimes | ExpressionUUID | WindowSize | WindowMargins -> _] :>
        Sequence[]};
   Export[outputFile, 
    StringReplace[
     ExportString[newcontents /. Cell -> Symbol[uniqName], "Package", 
      PageWidth -> Infinity, "Comments" -> None], uniqName -> "\nCell"], "Text"]
   ];

An example of use:

CleanNotebookForGit["ExampleData/document.nb", "document (for Git).nb"] // SystemOpen
7

I want to execute the code snippet in the original nb file and write the processing results back to the same file. Is that possible?

8

I want to execute the code snippet in the original nb file and write the processing results back to the same file. Is that possible?

No, it isn’t possible since the NB file will be blocked until you close it.

9

This implementation doesn’t require FrontEnd, so it can be used in a script.

10

I try to use the code snippet above from another new notebook to deal with the notebook in the attachment. But what is generated is a corrupt file with syntax errors.

Debugging-and-semantic-logic-analysis.nb (16.1 KB)

11

Something was changed in the exporter to the "Package" format since developing the original implementation. With Mathematica 8.0.4 it still works correctly, but not with the latest versions. I’ll inverstigate and try to fix it this week.

12

The immediate reason for the syntax error is that in recent Mathematica versions Export as "Package" writes Pattern[sym,obj] as sym:obj instead of Pattern[sym,obj] as it was in earlier versions. The syntax sym:obj is recognized by the FrontEnd as a syntax error when opening the file as Notebook. Exactly the same problem is with Put. Hence in the recent versions they broken the compatibility between Get and Put. I think that it is a bug.

13

Here is an improved version of CleanNotebookForGit with a fix for above-described Pattern export bug. Another improvement is that it doesn’t clean up options inside of BoxData, where this can cause problems:

CleanNotebookForGit[inputFile_, outputFile_] := 
  Module[{contents = Get[inputFile], newcontents,
    uniqCellHeadName = "$$$MyUniqueCellHead$$$" <> ToString[RandomInteger[{10^8, 10^10}]],
    uniqPatternHeadName = 
     "$$$MyUniquePatternHead$$$" <> ToString[RandomInteger[{10^8, 10^10}]]},
   newcontents = contents /. {bd_BoxData :> bd,
      HoldPattern[
        CellLabel | CellChangeTimes | ExpressionUUID | WindowSize | WindowMargins -> _] :>
        Sequence[]};
   Export[outputFile,
    StringReplace[
     ExportString[
      newcontents /. {Cell -> Symbol[uniqCellHeadName], 
        Pattern -> Symbol[uniqPatternHeadName]},
      "Package", PageWidth -> Infinity, "Comments" -> None],
     {uniqCellHeadName -> "\nCell", uniqPatternHeadName -> "Pattern"}], "Text"]];
14

Hi Alexey Popkov,

This works. Some additional considerations:

  1. How can I script the above code snippet instead of running it from inside Mathematica?
  2. Why not make it a customized package, to facilitate loading and using it?
  3. Some advice about the default arguments: Set the inputFile’s default value to the notebook name where this function is called, and set outputFile’s default value to something meaningful ending with git or like.
15
  1. Please read the Tutorial.
  2. and 3. The function is quite simple, so everyone can easily adapt it to suit their goals.

P.S. I just discovered that in the WFR is available ResourceFunction["SaveReadableNotebook"] which at first glance looks like a more generic and advanced version of CleanNotebookForGit.

16

Thank you very much for telling me this function. The biggest problem is that the first execution of this function is time-consuming because of the network query. But downloading the notebook and deploying it locally on the computer can solve this problem, as shown below:

Selection_174.png
Selection_174.png1013×357 93.6 KB

I tried some example codes shown in the document of ResourceFunction[“SaveReadableNotebook”], but encountered the following error:

FilePrint::badfile: The specified argument, FunctionRepository`$31760fa6b86145928a98537accf1a6b8`SaveReadableNotebook[nb,FileNameJoin[{/home/werner,readable.nb}]], should be a valid string or File.

See the attachment for more detailed information.

SaveReadableNotebook.nb (5.73 KB)

17

A correct call to this function:

ResourceFunction["SaveReadableNotebook"][EvaluationNotebook[], "readable.nb"]
18

Yes. This does the trick. But the diff gives the following results for continuous evaluation of the same NB file:

$ diff readable.nb readable1.nb 
50c50
<       CellLabel -> "In[18]:="
---
>       CellLabel -> "In[16]:="
55c55
<       CellLabel -> "Out[18]="
---
>       CellLabel -> "Out[16]="
84c84
<       CellLabel -> "In[19]:="
---
>       CellLabel -> "In[17]:="
89c89
<       CellLabel -> "Out[17]="
---
>       CellLabel -> "Out[15]="

As you can see, the In/Out cell numbering has been growing. This information is not conducive to git version control. It would be better if the numbers of these cells don’t be tracked in the result file.

SaveReadableNotebook.nb (3.39 KB)

19

Please read carefully the Documentation page of this function! It has an option "ExcludedCellOptions"… Try it!

20

Thank you for your hint. Based on the insightful and straightforward examples there, the following code does the trick:

ResourceFunction["SaveReadableNotebook"][
 EvaluationNotebook[], "readable.nb" , 
 "ExcludedCellOptions" -> {CellChangeTimes, ExpressionUUID, 
   CellLabel} ]

SaveReadableNotebook.nb (4.6 KB)