Group Abstract Group Abstract

Message Boards Message Boards

0
|
4.1K Views
|
7 Replies
|
3 Total Likes
View groups...
Share
Share this post:

[Solved] Split a matrix of data into new rows based on a condition

POSTED BY: Kalum Thurgood-Parkes
7 Replies
Sort By:
Updating Name
Updating Name
Posted 6 years ago

Hi Kalum,

A handy way to visualize the result of Map, BlockMap, MapAt and other WL functions that evaluate a given function over a list is to use Framed. e.g.

Map[Framed, Range@5]

enter image description here

MapAt[Framed, Range@5, 3]

enter image description here

BlockMap[Framed, Range@5, 3]

enter image description here

BlockMap[Framed, Range@5, 3, 1]

enter image description here

So in this code the pure function is passed a sliding window across the data and it evaluates to True if a peak is detected. The positions of the True values is extracted. It is not the index it is offset by -1.

BlockMap[#[[1, 1]] < #[[2, 1]] > #[[3, 1]] &, data, 3, 1] // Position[True];

TakeDrop is simple, check the documentation. To understand FoldPairList take a look at the documentation, but first look at the documentation for FoldList.

This gives the lengths of the segments into which the data should be partitioned. 

Flatten@{0, peakOffsets, Length@data} // Differences

FoldPairList takes the data, the lengths of the segments, and repeatedly calls TakeDrop to do the partitioning.

Hope that helps
POSTED BY: Updating Name

Hey Rohit,

Thank you soo much!! It worked beautifully on my own data, as did Bills. I really appreciate the effort showing how you can plot it to show it works too.

Honestly, you are quite literally my hero. It was driving me insane.

I hope you don't mind, but could you explain how the BlockMap, FoldPairList and TakeDrop functions work and how that these:

BlockMap[#[[1, 1]] < #[[2, 1]] > #[[3, 1]] &, data, 3, 1] // Position[True];

FoldPairList[TakeDrop, data, Flatten@{0, peakOffsets, Length@data} // Differences]

actually work? I just want to understand it from a learning point of view.

Honesty, I could not be more grateful.
POSTED BY: Kalum Thurgood-Parkes

Hey Bill,

Thanks so much for the help!
POSTED BY: Kalum Thurgood-Parkes
Rohit Namjoshi
Rohit Namjoshi
Posted 6 years ago

Thanks for the explanation. The following works on the sample you provided but it may not correctly deal with edge cases.

peakOffsets = BlockMap[#[[1, 1]] < #[[2, 1]] > #[[3, 1]] &, data, 3, 1] // Position[True];
peakPositions = peakOffsets + 1;

Plot to verify

values = data[[All, 1]];
peakValues = Extract[values, peakPositions];

ListLinePlot[{values, Transpose[{peakPositions // Flatten, peakValues}]}, 
 Joined -> {True, False}]

enter image description here

Split the list according to the offsets

FoldPairList[TakeDrop, data, Flatten@{0, peakOffsets, Length@data} // Differences]
POSTED BY: Rohit Namjoshi
Bill Nelson
Bill Nelson
Posted 6 years ago

Try

data = {{4, 1.2}, {3, 15.4}, {2, 6.6}, {1, 19.5}, {0, 5.5}, {-1, 1.8}, {-2, 11.1}, {-1, 6.3}, {0, 3.5},
  {1, 15.3}, {2, 11.1}, {1, 4.1}, {0, 14.9}, {-1, 2.4}, {-2, 10.1}, {-3, 7.3}, {-4, 10.8}, {-3, 7.2},
  {-2, 10.9}, {-1, 9.6}, {0, 17.3}, {1, 4.4}, {2, 10.0}, {3, 9.8}, {4, 0.4}, {3, 15.2}, {2, 14.5}, {1, 11.8},
  {0, 3.1}, {-1, 19.6}, {0, 1.9}, {1, 14.1}, {2, 17.8}, {3, 8.60}, {4, 0.559}, {5, 13.0}, {6, 10.6}};
positions=Flatten[Position[Map[If[#[[1,1]]<#[[2,1]]&&#[[2,1]]>#[[3,1]],1,0]&,Partition[data,3,1]],1]];
temp=data;
deltas=Flatten[{positions[[1]],Rest[positions]-Most[positions]}];
solution=Append[Map[(head=Take[temp,#];temp=Drop[temp,#];head)&,deltas],temp];
solution=={{{4, 1.2}, {3, 15.4}, {2, 6.6}, {1, 19.5}, {0, 5.5}, {-1, 1.8}, {-2, 11.1}, {-1, 6.3}, {0, 3.5},
  {1, 15.3}}, {{2, 11.1}, {1, 4.1}, {0, 14.9}, {-1, 2.4}, {-2, 10.1}, {-3, 7.3}, {-4, 10.8}, {-3, 7.2},
  {-2, 10.9}, {-1, 9.6}, {0, 17.3}, {1, 4.4}, {2, 10.0}, {3, 9.8}}, {{4, 0.4}, {3, 15.2}, {2, 14.5}, {1, 11.8},
  {0, 3.1}, {-1, 19.6}, {0, 1.9}, {1, 14.1}, {2, 17.8}, {3, 8.60}, {4, 0.559}, {5, 13.0}, {6, 10.6}}}

which returns

True

There should be a simpler cleaner way than this.
POSTED BY: Bill Nelson

Hey Rohit,

Thanks so much for the reply.

So that point you referenced, {-2, 11.1}, {-1, 6.3}, is as you correctly pointed out the first decrease to increase, but I want to split at the increase to decrease point, so that would be {2, 11.1}, {1, 4.1}.

So on that first split, you can see I wanted it to split at prior to that increase to decrease {2, 11.1}, {1, 4.1}, is why {1, 15.3} is the last element of the sublist.

Does that make it a little clearer what I'm trying to do Rohit?

Honestly, any help would be incredible.
POSTED BY: Kalum Thurgood-Parkes
Rohit Namjoshi
Rohit Namjoshi
Posted 6 years ago

Hi Kalum,

Can you explain the split criteria in more detail. Why is the first split 

{{4, 1.2}, {3, 15.4}, {2, 6.6}, {1, 19.5}, {0, 5.5}, {-1, 1.8}, {-2, 11.1}, {-1, 6.3}, {0, 3.5}, {1, 15.3}}

The first decrease to increase of the first element of the sublists happens at {-2, 11.1}, {-1, 6.3} so why was the list not split at that point?
POSTED BY: Rohit Namjoshi
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard