I have a long list (about 12000 entries) where each entry is of the form { Namei, Timei } (I call this list1 below). Basically it is a list of dimensions {~12000, 2}. Here the names are strings and the times are real numbers.
I need to look up these entries in a longer list (40000 entries) that has the form { Namei, Timei, Temp_i } (I call this list2 below). I basically want to get the value of temperature corresponding to each case in the first list. I have managed to do this using the function:
myfind = Function[x, Part[Select[list2, Drop[#, -1] == x &, 1], 1, 3]];
AbsoluteTiming[mytemps= Map[myfind, list1];]
A small detail that makes this a bit harder, is that the entries in list1 are not all unique, the same name and time can repeat a few times in the list, but this needs to be kept in the same position it appears. The longer list (list2), is unique in that each entry of list1 has a corresponding entry (and only one corresponding entry) in list2.
This works, but it is very slow. I have improved performance a bit by first throwing away everything from the second list that I do not need:
AbsoluteTiming[list2short = Select[list2, MemberQ[list1, Drop[#, -1]] &];]
This takes about 20 seconds, but helps making the previous operation faster. Even after this 'clean-up', the first operation takes about 45 sec. Does any one know of a way to speed up this kind of operations? Perhaps I should be using some database approach, rather than a purely list-based approach. I tried Sow and Reap constructs, and got it to work after a few tries, but it seemed even slower.
Thanks,
OL.