Thanks everyone,

Helpful comments. My needs are simple, the main one being that I want to be able to import the Mathematica expressions (lists, associations, etc.) exactly as I have exported them, so that I can continue analyzing the data. Speed is not a big deal because, while I am analyzing over 100,000 files, the stuff that I will be exporting from each is small. Compatibility with future releases of Mathematica is important, of course; so if Wolfram is thinking of dropping WDX, then maybe I should learn more about MX.

Gregory

I do not like WDX. It tends to be very slow for even moderately large data. Fortunately there are alternatives.

I am aware of four ways to save arbitrary Mathematica expressions:

• Save as InputForm: Export[..., "Package"]. Disadvantages: it takes a lot of space and it is slow. Advantages: cross-platform, cross-version.

• Save as WDX: Export[..., "WDX"]. It doesn't take as much space, but it is still slow.

• Save as MX: Export[..., "MX"]. This is very fast to import/export. Space efficiency is not excellent, but can be improved by gzip-ping the result: Export["file.mz.gz", data]. Before version 9, this was neither cross platform, nor cross version, so usually we could only re-import on the same machine where the data was exported, and only with the same Mathematica version. Since version 10, compatibility has been improved: the files are not compatible between different operating systems (OS X, Windows, Linux), but not between 32-bit and 64-bit versions of Mathematica. There is also backward-compatibility: files written by 10.0 are still readable by 10.3 (but not the reverse).

• My preferred choice: use Compress. I use

      zimport[filename_] := Uncompress@Import[filename, "String"]
      zexport[filename_, data_] := Export[filename, Compress[data], "String"]
  

  This method can be much faster than WDX, and it is both cross-platform and cross-version compatible in practice. But Wolfram did not make any guarantees about compatibility (it is not mentioned at all in the documentation), so I would not consider it safe for archival. (Interestingly, there are no comments in the documentation about WDX compatibility either!)

To summarize:

• If you need to archive data, then the best way is to use a format that is not Mathematica-specific. If it needs to be Mathematica expressions, the best way is to archive them is textual InputForm, as it is the most compatible.

• If you need to save data for immediate re-use on the same machine, for example for continuing your work tomorrow, use MX.

• If you need to move data between multiple potentially incompatible platforms (e.g. you are using 10.3 on a laptop and 10.2 on a server), and the data is used in the short term only (no archival), then use Compress.

I don't see a situation where WDX is really the best format.

Further reading:

• http://mathematica.stackexchange.com/a/1960/12 (how to export with Compress)
• http://mathematica.stackexchange.com/a/1417/12 (on Compress compatibility)
• http://mathematica.stackexchange.com/a/87355/12 (comments on WDX, already linked by Alexey)

Thanks, Sander. Much obliged. I messed around with a couple hundred files and it seems to work well.

Gregory

Yes WDX or MX format, MX being only compatible with the machine you run it on. WDX cross-compatible.