Message Boards Message Boards

0
|
11518 Views
|
11 Replies
|
6 Total Likes
View groups...
Share
Share this post:

Problem with big input not accepted

Posted 12 years ago
Hi everyone!
I'm new of this forum, congratulations for it!
I have a little problem: I need to process a lot of data (hundreds of thousands of lines of .txt, each line is about 150 char). I am using the trial version (30 days then expires) and Mathematica does not allow me to paste (not process, just paste) more than about a thousand lines. If I paste just one thousand and then the second thousand again it processes them, I just have a problem in pasting more than one thousand a time.
If I buy the 6 months licence will it accept bigger data as input? If yes, how big? Because I red that the trial version does not allow you to input from excel and so on, but I am actually just trying to paste them from the .txt file... do you think that it is a limitation of the trial version or will I have the same problem also with the licence?
Thank you!
l'AlbiBotto
11 Replies
Since the content of data.txt is a Mathematica expression, you can also import the file using "Package":
Import["data.txt", "Package"]
This returns the expression you need, without having to call ToExpression on the string.

http://reference.wolfram.com/mathematica/ref/format/Package.html
http://reference.wolfram.com/mathematica/guide/BasicFormats.html
POSTED BY: Arnoud Buzing
now it works thanks!!
Posted 12 years ago
Ah, that is very helpful. Thank you.

I create a sample text file containing a single line
{{1,2,3},{4,5,6},{"ciao",e,f}}
I import using
data=Import["data.txt"];
Mathematica will often think it is being helpful by hiding many important details so I use
FullForm[data]
That shows me]
"{{1,2,3},{4,5,6},{\"ciao\",e,f}}"
Using Fullform on an expression when something does not work is a very powerful trick and it will help you often if you remember it.
This shows that Mathematica read the file not as a matrix of numbers, but as a long string in quotes.
I try to turn the string into a matrix of numbers
FullForm[ToExpression[data]]
which gives me
List[List[1,2,3],List[4,5,6],List["ciao"]]
and
MatrixForm[ToExpression[data]]
shows you your matrix.

I caution you, MatrixForm and FullForm and other "Form"s in Mathematica are good to look at the data,
but each of those has placed the information inside a "container" and new users are often confused
when they try to then do math operation on the output from FullForm or MatrixForm. If you want to do
operations on your matrix I would suggest you use
newdata=ToExpression[data]
which will give you your data in a two dimensional list, but not contained inside FullForm formatting.
POSTED BY: Bill Simpson
My .txt file is something like
{{1,2,3},{4,5,6},{"ciao",e,f}}
just bigger.
I would like mathematica to read it as a matrix such that, if I write
data=Import["data.txt"];   (or also   data=Import["data.txt","Table"]; )
MatrixForm
the output should be the matrix 
   1     2  3
   4     5  6
ciao   e  f
the brackets and commas are in the right positions but mathematica does not reat it as a matrix 3x3 but as a matrix 3x1 as follows
   {1,2,3}
   {4,5,6}
{"ciao",e,f}
Anybody knows why?!
Posted 12 years ago
If you can give a clear simple explanation of what you want to accomplish and
put a copy of enough of your data in a place that anyone can get that without needing to sign up with some service then
perhaps someone can take a few minutes and write a few lines of code to help you get started.
POSTED BY: Bill Simpson
this is what happens:
That is not a big file at all. I would recommend browsing our extensive documetation system. For example, this tutorial can give you a few hints on how to get this working: Importing and Exporting Data
POSTED BY: Vitaliy Kaurov
Thank you for the answers!
it is 4MB, 26k lines of 150 char each, but it will be bigger...
It makes the import now but it does not recognise it as a Table. If I copy paste Mathematica sees it as a Table, with the Import function no...
Here is a sample of my file:
{{ciao,7,XXX},{ciao2,9,Y},{pippo,3,zz}}
Watch this video: How to | Insert a File Path

Here is the summary.

I suppose you know where file is located in the directory. So, when using Import for exact path do the following.
  • Type Import[]
  • Place cursor inside square parenthesis Import[|]
  • Go: Top menu >> Insert >> File path...
  • Browse to the file, select it and click button "Choose"
You will see complete path appearing inside Import. By the way, how big is the file size?
POSTED BY: Vitaliy Kaurov
I bought the 6 months version and I can import the file from Insert->file->open but if I try with the Import[] it does not work, it says that it can't find the file. Is it probably because that the file isn't in the right directory? I've looked for an answer on the internet but found nothing... thanks!
Posted 12 years ago
With the full version you can use Import so you wont even need to copy paste:
data = Import["data.txt", "Table"];
POSTED BY: Simon Schmidt
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract