My group uses Mathematica for three-dimensional image processing
. Our light sheet-based fluorescence microscopes (LSFM, SPIM, DSLM) generate Terabytes of data from live specimens that are usually processed as stacks of images, which are 1-4 GBytes large. We segment the data and store the results as Excel workbooks, which in turn are further processed by other programs.
Switching from Version 8 to 9 we suddenly realized that TIFF images larger than 2GByte cannot be imported anymore -
see Mathematica SE question: Maximum size of TIF files in Mathematica
We also realized that we have a problem
moderately large (more than ~20k rows) Excel workbooks
From the various comments we have received so far I understand that Excel Import/Export as well as TIFF Import/Export are handled by Java code and that a new TIFF library is used since version 9. The Excel Export issue can be addressed by increasing the memroy available to the Java Engine but I still do not understand that no error message and no warning is generated. In any case we can only assume that the error will occur again once our Excel workbooks become even larger. The TIFF Import issue is still unresolved.
To make matters really bad, these errors occur silently without creating any error messages. We only realize that the Jave Scripts fail because the data we intend to process does not exist.
I find this extremely worrying since this could hint at a fundamental problem. We have already reported our findings as errors but we have not heard that they will be resolved with 9.0.1. All our machines operate with two Xeons (8 or 12 cores), 96GByte of RAM, nVidia's D5000 and Windows 7-64bit.