How does the Linear method for DimensionReduction works?

Questions
,
1

Hi everybody.

I’m currently working at a problem where it comes very useful to have a linear dimensionality reduction, just as the fuction DimensionReduction[Method-> "Linear"] does.

I would like to use the found reduction matrix in an other context and I was looking for extracting it.

So I used the following code:

dummy = RandomReal[{-1, 1}, {10, 4}];

drf = DimensionReduction[dummy, 2, Method -> "Linear"]

drfM = drf[[1]]["Model"]["Matrix"]

But when then this check tells me that I’m missing something:

In[10]:= dummy.drfM == drf[dummy]

Out[10]= False

In[11]:= dummy.drfM == drf[Standardize@dummy]

Out[11]= False

I also used Standardize because I saw it as a step in:

drf[[1]]["Model"]["Processor"]

So how can I find the linear algebra transformation that the reducerFunction found?
And to what method “Linear” correspond to? It is not clear from the documentation…

Thanks in advance :slight_smile:

PS: I empirically found that there is a systematic -5% error between the two tranformations:

In[106]:= (Standardize[dummy].drfM - drf[dummy])/drf[dummy]

Out[106]= {{-0.0513167, -0.0513167}, {-0.0513167, -0.0513167}, \
{-0.0513167, -0.0513167}, {-0.0513167, -0.0513167}, {-0.0513167, \
-0.0513167}, {-0.0513167, -0.0513167}, {-0.0513167, -0.0513167}, \
{-0.0513167, -0.0513167}, {-0.0513167, -0.0513167}, {-0.0513167, \
-0.0513167}}
2

I suspect that different scaling is applied to standadizae variables in MMA, which confuses me much. The usual standardization indicates that variables have average values of zero and standard deviations of one for variables. However, MMA may apply “biased” standardization as indicated in https://community.wolfram.com/groups/-/m/t/3079946 .

Assuming the linear method in dimension reduction function is similar to PCA or singular value decomposition; X=USV’=TV’

dummy = N@{{1, 4, 1}, {2, 3, 2}, {3, 2, 1}, {4, 1, 2}};
dummyS = Standardize[dummy];

Biased & standardized input (not a usual standardization):

dummySBiased = Sqrt[Length[dummy]/(Length[dummy] - 1)]*Standardize[dummy]

The dimension reduction function is applied

drf = DimensionReduction[dummy, 2, Method -> "PrincipalComponentsAnalysis"]

drf // Normal;

dimensionReduction1.png
dimensionReduction1.png1003×436 59.7 KB

The eigenvector (“drfM”) is extracted from the dimension reduction function, which is similar to the “V” in SVD (X=TV’):

drfM = drf[[1]]["Model"]["Matrix"]

The reduced vectors (drf[dummy]=“T” fromSVD) by applying the dimension reduction function is equal to the biased&standardized input times the reduced vector.

dummySBiased.drfM == drf[dummy]