Help with importing a CSV file as Dataset

Questions
, ,
1

Hi,

I’m trying to import a CSV file as a Dataset object. All the entries in the CSV file are real numbers, except for the first row, which contains the variable names. Checking the documentation the function SemanticImport is supposed to do that, but I can’t get it to work. This is what I’m using:

dataset = 
  SemanticImport["cleandata.csv", "Real", "Dataset", 
    HeaderLines -> 1, Delimiters -> ",", MissingDataRules -> {"." -> Missing[], "" -> Missing[]}];

I can import the data without problem using Import but I want to try the new Dataset object in Mathematica. Any suggestions?

2

Hi Miguel,

this seems to work fine on my system (OS X, MMA10.0.2).

I generate some test data and export that:

data = RandomReal[1, 100];
Export["~/Desktop/cleandata.csv", Join[{"Test Data"}, data]]

Running your command:

dataset = SemanticImport["~/Desktop/cleandata.csv", "Real", "Dataset", HeaderLines -> 1, Delimiters -> ",", 
   MissingDataRules -> {"." -> Missing[], "" -> Missing[]}];

gives:

ScreenShot2015-02-19at22.34.42.png
ScreenShot2015-02-19at22.34.42.png260×465 9.63 KB

What happens when you execute your command?

Cheers,

Marco

3

I get this:

ScreenShot2015-02-19at3.44.22PM.png
ScreenShot2015-02-19at3.44.22PM.png883×314 15.7 KB

The first row contains the variable names in the CSV file. I think the problem arises when there is more than 1 column in the CSV. If I run your example with RandomReal[1,{100,2}], it imports the first column only.

4

Oh, I see where the problem is. You might want to try the two column data set and Import it like so:

dataset = SemanticImport["~/Desktop/cleandata.csv", {"Real", "Real"}, "Dataset", HeaderLines -> 1, Delimiters -> ",", MissingDataRules -> {"." -> Missing[], "" -> Missing[]}];

or alternatively like so:

dataset = SemanticImport["~/Desktop/cleandata.csv", Automatic, "Dataset", HeaderLines -> 1, Delimiters -> ",", MissingDataRules -> {"." -> Missing[], "" -> Missing[]}];

Cheers,

Marco

5

I see. I guess that does solve the issue. However, it seems to be pretty slow. My machine has been importing the file for the las 30 minutes, if I use Import it takes less than a minute. I guess I’ll have to play with that when I buy a faster machine. Thanks anyways.

6

Hey Marco, I saw you profile and it seems that you also work with data. I have used Mathematica since version 4 for symbolics and for simple data analysis using matrices but now I feel outdated with respect of these new objects that Mathematica has introduced in the newer versions. I wanted to ask you if you know of any tutorial on Mathematica data analysis using these new objects.

7

Dear Miguel,

I don’t really know any written tutorial/book which discusses this in sufficient detail. There is, however, an online course

http://www.wolfram.com/training/courses/dat101.html

which might be useful.

Cheers,

Marco