Group Abstract Group Abstract

Message Boards Message Boards

0
|
3.4K Views
|
7 Replies
|
3 Total Likes
View groups...
Share
Share this post:

[Solved] Split a matrix of data into new rows based on a condition

Hey, I'm pretty new to Mathematica and am still learning, but I've struck upon a problem on how to do something. No amount of reading forums or language documentation has seemed to have helped, so please could someone help me out.

So I have data from an experiment in which the data follows this general trend of the simulated data (for simplicity) below. 

data = {{4, 1.2}, {3, 15.4}, {2, 6.6}, {1, 19.5}, {0, 5.5}, {-1, 
    1.8}, {-2, 11.1}, {-1, 6.3}, {0, 3.5}, {1, 15.3}, {2, 11.1}, {1, 
    4.1}, {0, 14.9}, {-1, 2.4}, {-2, 10.1}, {-3, 7.3}, {-4, 
    10.8}, {-3, 7.2}, {-2, 10.9}, {-1, 9.6}, {0, 17.3}, {1, 4.4}, {2, 
    10.0}, {3, 9.8}, {4, 0.4}, {3, 15.2}, {2, 14.5}, {1, 11.8}, {0, 
    3.1}, {-1, 19.6}, {0, 1.9}, {1, 14.1}, {2, 17.8}, {3, 8.60}, {4, 
    0.559}, {5, 13.0}, {6, 10.6}};

What I want to do with this data is yo be able to split the data into a new row at the point within the row value data[[All,1] ]. The point I want to do that at is when data[[All,1] ] goes from increasing to decreasing, so when x<y>z, if that makes sense?

To elaborate, if you used the sample data set above, the desired output of the script I want would be:

{{{4, 1.2}, {3, 15.4}, {2, 6.6}, {1, 19.5}, {0, 5.5}, {-1, 1.8}, {-2, 
    11.1}, {-1, 6.3}, {0, 3.5}, {1, 15.3}}, {{2, 11.1}, {1, 4.1}, {0, 
    14.9}, {-1, 2.4}, {-2, 10.1}, {-3, 7.3}, {-4, 10.8}, {-3, 
    7.2}, {-2, 10.9}, {-1, 9.6}, {0, 17.3}, {1, 4.4}, {2, 10.0}, {3, 
    9.8}}, {{4, 0.4}, {3, 15.2}, {2, 14.5}, {1, 11.8}, {0, 3.1}, {-1, 
    19.6}, {0, 1.9}, {1, 14.1}, {2, 17.8}, {3, 8.60}, {4, 0.559}, {5, 
    13.0}, {6, 10.6}}};

So that actual data I'm using is a much larger data set, and I want to be able to spit the data automatically into separate rows in this way so I can plot the new split data individually, to allow me to just plot specific sets of the data.

I've been trying to solve how to write this script to work for a couple of days now and it's honestly driving me insane. Any help on how you would go about doing this would be SO appreciated.
POSTED BY: Kalum Thurgood-Parkes
7 Replies
Sort By:
Updating Name
Updating Name
Posted 5 years ago

Hi Kalum,

A handy way to visualize the result of Map, BlockMap, MapAt and other WL functions that evaluate a given function over a list is to use Framed. e.g.

Map[Framed, Range@5]

enter image description here

MapAt[Framed, Range@5, 3]

enter image description here

BlockMap[Framed, Range@5, 3]

enter image description here

BlockMap[Framed, Range@5, 3, 1]

enter image description here

So in this code the pure function is passed a sliding window across the data and it evaluates to True if a peak is detected. The positions of the True values is extracted. It is not the index it is offset by -1.

BlockMap[#[[1, 1]] < #[[2, 1]] > #[[3, 1]] &, data, 3, 1] // Position[True];

TakeDrop is simple, check the documentation. To understand FoldPairList take a look at the documentation, but first look at the documentation for FoldList.

This gives the lengths of the segments into which the data should be partitioned. 

Flatten@{0, peakOffsets, Length@data} // Differences

FoldPairList takes the data, the lengths of the segments, and repeatedly calls TakeDrop to do the partitioning.

Hope that helps
POSTED BY: Updating Name

Hey Rohit,

Thank you soo much!! It worked beautifully on my own data, as did Bills. I really appreciate the effort showing how you can plot it to show it works too.

Honestly, you are quite literally my hero. It was driving me insane.

I hope you don't mind, but could you explain how the BlockMap, FoldPairList and TakeDrop functions work and how that these:

BlockMap[#[[1, 1]] < #[[2, 1]] > #[[3, 1]] &, data, 3, 1] // Position[True];

FoldPairList[TakeDrop, data, Flatten@{0, peakOffsets, Length@data} // Differences]

actually work? I just want to understand it from a learning point of view.

Honesty, I could not be more grateful.
POSTED BY: Kalum Thurgood-Parkes

Hey Bill,

Thanks so much for the help!
POSTED BY: Kalum Thurgood-Parkes
Rohit Namjoshi
Rohit Namjoshi
Posted 5 years ago

Thanks for the explanation. The following works on the sample you provided but it may not correctly deal with edge cases.

peakOffsets = BlockMap[#[[1, 1]] < #[[2, 1]] > #[[3, 1]] &, data, 3, 1] // Position[True];
peakPositions = peakOffsets + 1;

Plot to verify

values = data[[All, 1]];
peakValues = Extract[values, peakPositions];

ListLinePlot[{values, Transpose[{peakPositions // Flatten, peakValues}]}, 
 Joined -> {True, False}]

enter image description here

Split the list according to the offsets

FoldPairList[TakeDrop, data, Flatten@{0, peakOffsets, Length@data} // Differences]
POSTED BY: Rohit Namjoshi
Bill Nelson
Bill Nelson
Posted 5 years ago

Try

data = {{4, 1.2}, {3, 15.4}, {2, 6.6}, {1, 19.5}, {0, 5.5}, {-1, 1.8}, {-2, 11.1}, {-1, 6.3}, {0, 3.5},
  {1, 15.3}, {2, 11.1}, {1, 4.1}, {0, 14.9}, {-1, 2.4}, {-2, 10.1}, {-3, 7.3}, {-4, 10.8}, {-3, 7.2},
  {-2, 10.9}, {-1, 9.6}, {0, 17.3}, {1, 4.4}, {2, 10.0}, {3, 9.8}, {4, 0.4}, {3, 15.2}, {2, 14.5}, {1, 11.8},
  {0, 3.1}, {-1, 19.6}, {0, 1.9}, {1, 14.1}, {2, 17.8}, {3, 8.60}, {4, 0.559}, {5, 13.0}, {6, 10.6}};
positions=Flatten[Position[Map[If[#[[1,1]]<#[[2,1]]&&#[[2,1]]>#[[3,1]],1,0]&,Partition[data,3,1]],1]];
temp=data;
deltas=Flatten[{positions[[1]],Rest[positions]-Most[positions]}];
solution=Append[Map[(head=Take[temp,#];temp=Drop[temp,#];head)&,deltas],temp];
solution=={{{4, 1.2}, {3, 15.4}, {2, 6.6}, {1, 19.5}, {0, 5.5}, {-1, 1.8}, {-2, 11.1}, {-1, 6.3}, {0, 3.5},
  {1, 15.3}}, {{2, 11.1}, {1, 4.1}, {0, 14.9}, {-1, 2.4}, {-2, 10.1}, {-3, 7.3}, {-4, 10.8}, {-3, 7.2},
  {-2, 10.9}, {-1, 9.6}, {0, 17.3}, {1, 4.4}, {2, 10.0}, {3, 9.8}}, {{4, 0.4}, {3, 15.2}, {2, 14.5}, {1, 11.8},
  {0, 3.1}, {-1, 19.6}, {0, 1.9}, {1, 14.1}, {2, 17.8}, {3, 8.60}, {4, 0.559}, {5, 13.0}, {6, 10.6}}}

which returns

True

There should be a simpler cleaner way than this.
POSTED BY: Bill Nelson

Hey Rohit,

Thanks so much for the reply.

So that point you referenced, {-2, 11.1}, {-1, 6.3}, is as you correctly pointed out the first decrease to increase, but I want to split at the increase to decrease point, so that would be {2, 11.1}, {1, 4.1}.

So on that first split, you can see I wanted it to split at prior to that increase to decrease {2, 11.1}, {1, 4.1}, is why {1, 15.3} is the last element of the sublist.

Does that make it a little clearer what I'm trying to do Rohit?

Honestly, any help would be incredible.
POSTED BY: Kalum Thurgood-Parkes
Rohit Namjoshi
Rohit Namjoshi
Posted 5 years ago

Hi Kalum,

Can you explain the split criteria in more detail. Why is the first split 

{{4, 1.2}, {3, 15.4}, {2, 6.6}, {1, 19.5}, {0, 5.5}, {-1, 1.8}, {-2, 11.1}, {-1, 6.3}, {0, 3.5}, {1, 15.3}}

The first decrease to increase of the first element of the sublists happens at {-2, 11.1}, {-1, 6.3} so why was the list not split at that point?
POSTED BY: Rohit Namjoshi
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Tag limit exceeded
Note: Only the first five people you tag will receive an email notification; the other tagged names will appear as links to their profiles.
Reply Preview
Attachments
Remove
or Discard