Message Boards Message Boards

1
|
3814 Views
|
1 Reply
|
2 Total Likes
View groups...
Share
Share this post:

How does the Linear method for DimensionReduction works?

Posted 5 years ago

Hi everybody.

I'm currently working at a problem where it comes very useful to have a linear dimensionality reduction, just as the fuction DimensionReduction[Method-> "Linear"] does.

I would like to use the found reduction matrix in an other context and I was looking for extracting it.

So I used the following code:

dummy = RandomReal[{-1, 1}, {10, 4}];

drf = DimensionReduction[dummy, 2, Method -> "Linear"]

drfM = drf[[1]]["Model"]["Matrix"]

But when then this check tells me that I'm missing something:

In[10]:= dummy.drfM == drf[dummy]

Out[10]= False

In[11]:= dummy.drfM == drf[Standardize@dummy]

Out[11]= False

I also used Standardize because I saw it as a step in:

drf[[1]]["Model"]["Processor"]

So how can I find the linear algebra transformation that the reducerFunction found? And to what method "Linear" correspond to? It is not clear from the documentation...

Thanks in advance :)

PS: I empirically found that there is a systematic -5% error between the two tranformations:

In[106]:= (Standardize[dummy].drfM - drf[dummy])/drf[dummy]

Out[106]= {{-0.0513167, -0.0513167}, {-0.0513167, -0.0513167}, \
{-0.0513167, -0.0513167}, {-0.0513167, -0.0513167}, {-0.0513167, \
-0.0513167}, {-0.0513167, -0.0513167}, {-0.0513167, -0.0513167}, \
{-0.0513167, -0.0513167}, {-0.0513167, -0.0513167}, {-0.0513167, \
-0.0513167}}
POSTED BY: Ettore Mariotti
Posted 3 months ago

I suspect that different scaling is applied to standadizae variables in MMA, which confuses me much. The usual standardization indicates that variables have average values of zero and standard deviations of one for variables. However, MMA may apply "biased" standardization as indicated in https://community.wolfram.com/groups/-/m/t/3079946.

Assuming the linear method in dimension reduction function is similar to PCA or singular value decomposition; X=USV'=TV'

dummy = N@{{1, 4, 1}, {2, 3, 2}, {3, 2, 1}, {4, 1, 2}};
dummyS = Standardize[dummy];

Biased & standardized input (not a usual standardization):

dummySBiased = Sqrt[Length[dummy]/(Length[dummy] - 1)]*Standardize[dummy]

The dimension reduction function is applied

drf = DimensionReduction[dummy, 2, Method -> "PrincipalComponentsAnalysis"]

drf // Normal;

enter image description here

The eigenvector ("drfM") is extracted from the dimension reduction function, which is similar to the "V" in SVD (X=TV'):

drfM = drf[[1]]["Model"]["Matrix"]

The reduced vectors (drf[dummy]="T" fromSVD) by applying the dimension reduction function is equal to the biased&standardized input times the reduced vector.

dummySBiased.drfM == drf[dummy]
POSTED BY: Sangdon Lee
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract