Modify dataset elements based on criteria

Questions
1

I have the following dataset:

dataset=Dataset[{
  <|"x" -> -1, "y" -> 163|>,
  <|"x" -> 4, "y" -> 183|>,
  <|"x" -> -7, "y" -> 104|>,
  <|"x" -> -5, "y" -> 191|>,
  <|"x" -> 2, "y" -> 161|>
  }]

I want to replace every "y" element for which the corresponding "x" values is larger than 0, into Null, and thus to get the new dataset

Dataset[{
  <|"x" -> -1, "y" -> 163|>,
  <|"x" -> 4, "y" -> Null|>,
  <|"x" -> -7, "y" -> 104|>,
  <|"x" -> -5, "y" -> 191|>,
  <|"x" -> 2, "y" -> Null|>
  }]

I have a few loose ends to tie up:

  • dataset[Select[#x > 0 &]] givse the rows with "x" value > 0
  • dataset[Select[#x > 0 &], "y"] gives the corresponding "y" values
  • Position[dataset[All, "x"] // Normal, _?(# > 0 &)] gives the row numbers for which "x" > 0
  • RepacePart can be used to modify the values; I don’t know why but ReplacePart[dataset[All, "y"] // Normal, {2, 5} -> Null] doesn’t work

So I need some help with getting it to work.

Any help would be very welcome!

2

Something along these lines, maybe:

dataset[All, <|"x" -> #["x"], "y" -> If[Positive[#["x"]], Null, #["y"]]|> &]

There might be a more elegant way, but essentially you’re just applying the transformation you want to each association in the dataset.

3

I see. Thanks, I think I can take it from here.

Can you tell me why ReplacePart[dataset[All, "y"] // Normal, {2, 5} -> Null] doesn’t return a list with modified elements?

4

Well, there are a couple of things going on. First,

dataset[All, "y"] // Normal

returns a new expression:

{163, 183, 104, 191, 161}

So, that is what ReplacePart will work on. You won’t be updating dataset at all. So, if you were expecting to update dataset, the explanation can stop there. If you want to know why that new expression wasn’t modified, it’s because your position specification didn’t correspond to an existing place in this new structure.

ReplacePart[dataset[All, "y"] // Normal, {2, 5} -> Null]

will look for position {2,5} (which corresponds to something like x[[2,5]]). The main argument is just a flat list, however, so there is no “fifth element in the second element” to be replaced. Presumably, what you wanted to replace was two things, the second and the fifth element. To do that, you need a list of positions: {{2},{5}}. Like this:

ReplacePart[dataset[All, "y"] // Normal, {{2}, {5}} -> Null]
(* {163, Null, 104, 191, Null} *)