Group Abstract Group Abstract

Message Boards Message Boards

Ignore portions that are part of a CSV file when importing?

Posted 9 years ago
POSTED BY: Erick van Rijk
6 Replies
Posted 9 years ago

Hi David, many thanks for the example. It will take me a while to study the method you made, since there are things that are new to me in there.

Thanks for the help! Erick

POSTED BY: Erick van Rijk
Posted 9 years ago

Hi David,

thank you very much for the detailed examples you provided. This allowed me to continue playing with processing the raw data. Do you have a suggestion for the processing of the # statements as variables? Using the Cases statement?

thanks Erick

POSTED BY: Updating Name
Posted 9 years ago

Hi Erick,

Attached is a notebook which defines a function for importing data from such a file. It accepts a filename and returns a list structure containing the imported elements. Following are examples for using this to assign the data to arbitrary variables, or to convert the control tags a form suitable for use as a Mathematica symbol and assign the data elements to them. It assumes a form similar to "test.txt" and does no error checking. It assumes the data file is in the same directory as the notebook, although a full path could be used.

Kind regards, David

Attachments:
POSTED BY: David Keith
Posted 9 years ago

I generally do this by importing all of the data, so that I have the numbers imported as numbers, and then select the data from the raw import. In the code below two methods are shown. The first assumes that the data is a single block of lines, each with 4 reals. The second method finds the positions of the start and end delimiters and takes the elements in between.

Best,

David

In[1]:= SetDi[NotebookDirectory[]]

Out[1]= "C:\\Users\\David\\Desktop\\Erick"

In[2]:= raw = 
 Import["test.txt", "Table", "FieldSeparators" -> { " ", ","}]

Out[2]= {{"#A", "Albert"}, {"#B", 0.5, "%"}, {"#C", 
  0.01726}, {"$V"}, {-25., -25., 5., 0.053609}, {-24., -25., 5., 
  0.065964}, {-23., -25., 5., 0.051466}, {-22., -25., 5., 
  0.053896}, {"$E"}}

In[3]:= (* By pattern match *)

In[4]:= data1 = Cases[raw, {_Real, _Real, _Real, _Real}]

Out[4]= {{-25., -25., 5., 0.053609}, {-24., -25., 5., 
  0.065964}, {-23., -25., 5., 0.051466}, {-22., -25., 5., 0.053896}}

In[5]:= (* by delimiters *)

In[6]:= start = Position[raw, {"$V"}][[1, 1]] + 1

Out[6]= 5

In[7]:= end = Position[raw, {"$E"}][[1, 1]] - 1

Out[7]= 8

In[8]:= data2 = raw[[start ;; end]]

Out[8]= {{-25., -25., 5., 0.053609}, {-24., -25., 5., 
  0.065964}, {-23., -25., 5., 0.051466}, {-22., -25., 5., 0.053896}}
Attachments:
POSTED BY: David Keith
Posted 9 years ago

Is it always the first row of the CSV to be removed? Then read the file and delete the first line.

If it is not always the first row then is there some other way a program can determine what are comments and what are data?

If you could include a small example file, describe what parts the data are then that might help.

If a program were able to correctly handle that file would imply that it would almost certainly do any other file you might have then that would be even better.

POSTED BY: Bill Simpson
Posted 9 years ago
Attachments:
POSTED BY: Erick van Rijk
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard