Message Boards Message Boards

0
|
9802 Views
|
9 Replies
|
0 Total Likes
View groups...
Share
Share this post:

Merging Large Text Files Without Reading Entire File

Posted 9 years ago

I have some large data files that each contain 2 columns of data as shown below (strings in row are separated by spaces). I need to merge these files into one large file that contains 4 columns each separated by tabs. Is there an efficient built-in function that will let me do this using file streams without the need to read these large files in completely?

thanks

input1.txt 613 -112 1737 1685 1443 891 -39 273 1761 356 2007 -8 -198 -882 -1634

input2.txt 1044 249 -716 207 -841 989 1076 1446 899 969 -1574 1003 -1833 1192 241 -406 804 -1836

Desired output file 613 -112 1044 249 1737 392 -716 207 1685 -176 -841 989 1443 -813 1076 1446 891 -39 899 969 273 1761 -1574 1003 356 2007 -1833 1192 -8 -198 241 -406 -882 -1634 804 -1836

POSTED BY: Bob Stephens
9 Replies
Posted 9 years ago

sorry, missed the export to a file........thanks for your help

POSTED BY: Bob Stephens
Posted 9 years ago
POSTED BY: Bob Stephens
Posted 9 years ago

Looks like the issue is that each row is currently a string and so the Join function is not getting the number of expected elements

POSTED BY: Bob Stephens

A direct (but not very elegant) way could be:

ClearAll["Global`*"]
SetDirectory[NotebookDirectory[]];
input1 = Import["first.txt", "Data"];
input2 = Import["second.txt", "Data"];
result = Join[input1, input2, 2];
Export["result.txt", 
  StringReplace[
   ToString[result], {"}, {" -> "\n", ", " -> "\t", "{{" -> "", 
    "}}" -> ""}]];

But I have to confess that this is not solving your primary question: This way the entire input files have to be red in first.

Henrik

POSTED BY: Henrik Schachner
Posted 9 years ago

Adding sample files to discussion.

Attachments:
POSTED BY: Bob Stephens
Posted 9 years ago
POSTED BY: Bob Stephens

Well, yes, according to you second posting I was assuming that your data are already in form of a matrix like so:

{{613, -112}, {1737, 
   392}, {1685, -176}, {1443, -813}, {891, -39}, {273, 1761}, {356, 
   2007}, {-8, -198}, {-882, -1634}} // TableForm

(*output: 613   -112
1737    392
1685    -176
1443    -813
891 -39
273 1761
356 2007
-8  -198
-882    -1634*)

Obviously this is not the case in your example ...

POSTED BY: Henrik Schachner
POSTED BY: Henrik Schachner
Posted 9 years ago

Sorry, text formatting (ironically) in the web app does not clearly illustrate the problem. See image below

thanks

enter image description here

POSTED BY: Bob Stephens
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract