I suppose, although I've always reserved ZIP for delivery or archiving. I've never really considered repeatedly compressing and decompressing thousands of big files that are part of an ongoing workflow or project.

Greg

I agree this isn't related to encoding, it's best to stick to UTF-8. There is a hidden option to CSV and TSV Export "ForceDelimieters" that controls whether the setting for "TextDelimiters" is wrapped around text fields always, or only when needed. Since this isn't documented yet, this might change in future versions. The reason we always wrap non-numeric values in the delimiter is export would be much slower if it has to test each string for the presence of a comma, newline, or delimiter. Compare these outputs:

In[2382]:= ExportString[{{1, "B", "test,comma"}}, "CSV"]

Out[2382]= "1,\"B\",\"test,comma\"
"

In[2383]:= ExportString[{{1, "B", "test,comma"}}, "CSV", 
 "ForceDelimiters" -> False]

Out[2383]= "1,B,\"test,comma\"
"

In[2385]:= ExportString[{{1, "B", "test,comma"}}, "CSV", 
 "ForceDelimiters" -> False, "TextDelimiters" -> "'"]

Out[2385]= "1,B,'test,comma'
"

If your data does not contain any commas or newlines, then the best approach is to just disable text delimiters all together so that the string matching doesn't kick in (notice this mangles the field with a comma):

In[2387]:= ExportString[{{1, "B", "test,comma"}}, "CSV", 
 "TextDelimiters" -> None]

Out[2387]= "1,B,test,comma
"

P.S. I stumbled on this post by accident, and probably would have seen it sooner if CSV was in the title :)

Good points, Hans. I agree that wrapping all fields in quotes is the safest bet for CSV, despite the increase in file size. My students who set their computer's OS to a language other than English often encounter problems opening data files in Excel because their language uses the comma to represent the decimal point in real numbers.

Greg:

I tried a few things:

outs = Table[{RandomReal[{-10,10}],RandomReal[{-10,10}],RandomReal[{-10,10}],RandomReal[{-10,10}],RandomReal[{-10,10}],RandomReal[{-10,10}],RandomReal[{-10,10}],RandomReal[{-10,10}],RandomReal[{-10,10}],RandomReal[{-10,10}],RandomReal[{-10,10}],RandomReal[{-10,10}]},{721314}];
Dimensions[outs]
{721314, 12}
Export["couts.csv", outs]

The file size without any options on my system is 155 MB (163,184,848 bytes) for this default Export. This is Windows 10 M 12.1. The file size is the same even with some additional options which M does not bark at:

CharacterEncoding -> "PrintableASCII", RecordSeparators -> "\r\n"

CharacterEncoding -> "ASCII", RecordSeparators -> "\r\n"

CharacterEncoding -> "ASCII", RecordSeparators -> "\n"

CharacterEncoding -> "UTF8"

CharacterEncoding -> "UTF-8"

"Export encodes line separator characters using the convention of the computer system on which the Wolfram Language is being run." From the Help files

The file size changed to 309 MB (324,927,070 bytes) when CharacterEncoding -> "Unicode" was used.

It could be the RecordSeparators or even quoted vs unquoted CSV format that may account for size dfference.

Import and Export options for CSV are documented here. UTF-8 encoding is the default.

Hi Rohit,

Thanks for the suggestion. I will compare the files in terminal, but see my response to Hans.

Regards,

Greg