Group Abstract Group Abstract

Message Boards Message Boards

[?] Avoid XLSX import failure on first row in small file?

Posted 9 years ago

Hello,

I have an Excel workbook with one sheet containing 86370 rows and 28 columns of numeric cells. File size is ~ 27MB.

I cannot successfully import this file, even if I ask for just the first row. The Java heap always explodes, regardless of JVM heap parameters -Xmx and -Xms (or where they're specified).

For example,

Import["file.xlsx",{Data,1,1}]

fails.

It seems that WRI does not take the Excel file format seriously.

My workaround saves to and imports from binary. But since I'm often interacting with internal/external customers via Excel, I'd strongly prefer not to introduce additional data products, to make the workflow simpler, more transparent, and less error-prone.

Here's hoping that XLSX import/export gets some TLC from WRI in the near future, with full/efficient support for its maximum specs (~ 1M rows, 65K cols, unlimited sheets as of ~ Excel 2016).

Am I missing something?

Vince

POSTED BY: Vincent Virgilio
5 Replies
POSTED BY: Vincent Virgilio
POSTED BY: Rolf Mertig

My machine is Windows 10 with 32 GB RAM and the same version of Mathematica. I'm not sure what is causing the problem for you.

POSTED BY: Chad Knutson
POSTED BY: Vincent Virgilio

Using my machine, I had to set -Xmx to 5 GB to make it work. I agree that is excessive. What was the max that you tried?

In[138]:= data = RandomReal[1,{86370,28}];
In[142]:= Needs["JLink`"]
In[149]:= ReinstallJava[JVMArguments->"-Xmx5g"]
In[146]:= Export["bigd.xlsx",data]
Out[146]= bigd.xlsx
In[151]:= d2=Import["bigd.xlsx"];
In[152]:= Dimensions[d2]
Out[152]= {1,86370,28}
POSTED BY: Chad Knutson
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard