Message Boards Message Boards

How to perform function to elements of Dataset?

Posted 9 years ago

Good day,

I need to process several series of geophysical measurements. Data was observed in two geographic locations (loc1 and loc2 see attachment ). For each location I have different methods measurements (method 1, method 2, method 3). I combine these data to one dataset. And now I need to perform function (e.g Standardize or Rescale )for each method and for each location. What is the right way to make it?

Thank a lot!

POSTED BY: Anton Ekimenko
3 Replies


It's not very clear what you are trying to do. If you are trying to do a regression analysis and explore whether the locations or methods influence the results, then Emerson's reply makes sense. You can do regressions (linear or not) and set some variables as nominal. I can add that you can get much information out of the functions that result from the fit. In Emerson's example the object f contains not only the best fit function, but all sort of information about the results of the regression. For example:


will print an analysis of variance for the regression. You can type:


to see the entire list of properties available.

If, on the other hand, you want to apply a function to your dataset that takes different forms depending of the method or location, then I have found function definitions like the one below to be convenient:

g["loc 1","meth 1",x_]:= some expression

g["loc 1","meth 2",x_]:= some other expression

and so on. Then you can just Map this function on your dataset.

But I may not have understood what you want, so all this reply may not be as useful.



POSTED BY: Otto Linsuain

I think this will give you a more useful arrangement of your data.

data1 = Table[{"loc1", i, i^2, i^3}, {i, 1, 10}];
data2 = Table[{"loc2", i, RandomReal[i], RandomReal[i]^3}, {i, 1, 10}];
data = Join[data1, data2];
header = {"Location", "methodname1", "methodname2", "methodname3"};
data = Dataset@Map[AssociationThread[header -> #] &, data];

You can then starting playing with your data. If you want to use Location as a key use GroupBy

plots = data[GroupBy[#Location &],Query[Transpose, Values@Rest@# &] /* ListPlot];

enter image description here

Alternatively, you may want to fit models to your data. If methodname3 was a response variable then you could do the following:

data2 = Values@Normal@data[Select[#Location == "loc1" &]];
f = LinearModelFit[data2, {x, y^3}, {x, y, z}, NominalVariables -> x];
f["loc1", 6, 36]

With real data you could use both locations to find a model you like and then estimate a location effect.

data3 = Values@Normal@data;
f = LinearModelFit[data3, {x, y, z}, {x, y, z}, NominalVariables -> x]
POSTED BY: Emerson Willard
Posted 9 years ago


Sorry for the late reply. Thank you for your decision. On another forum I found the following response - you can see attachment. As far as I understand in any case I need to convert Dataset to the list (In your answer I see it too -using Normal).


POSTED BY: Anton Ekimenko
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
or Discard

Group Abstract Group Abstract