6.1 General Analysis

Notation:

\(\mathbf{X}\): matrix of \(n\) rows (corresponding to the individuals), and \(p\) columns (corresponding to the variables). We assume mean-centered variables.

\(\mathbf{N}\): diagonal matrix of dimension \(n \times n\), with \(n_{ii} \geq 0\)

\(\mathbf{M}\): matrix of dimension \(p \times p\), positive definite (\(\mathbf{a^\mathsf{T} M a} > \mathbf{0}\) for all \(\mathbf{a}\)).

Cloud of Points

In \(\mathbb{R}^p\) we assume \(n\) points weighted by \(\mathbf{N}\), and with metric \(\mathbf{M}.\)

In \(\mathbb{R}^n\) we assume \(p\) points weighted by \(\mathbf{M}\), and with metric \(\mathbf{N}.\)

Fit in p-dimensions

Let \(\dot{\mathbf{u}}\) be a unit vector (e.g. \(\dot{\mathbf{u}}^\mathsf{T} \mathbf{Mu} = \mathbf{1}\)), defining a direction in \(\mathbb{R}^p\). In this direction we want to maximize the inertia of the projection of the individuals.

Projection of individuals: \(\mathbf{\Psi} = \mathbf{XM} \dot{\mathbf{u}}\)

Inertia of projection: \(\mathbf{\Psi^\mathsf{T} N \Psi} = (\mathbf{XM}\dot{\mathbf{u}})^\mathsf{T} \mathbf{N} (\mathbf{XM}\dot{\mathbf{u}})\)

The maximum projected inertia can be obtained by maximizing: \(\dot{\mathbf{u}}^\mathsf{T} (\mathbf{MX^\mathsf{T}NXM}) \dot{\mathbf{u}}\), with the condition \(\dot{\mathbf{u}}^\mathsf{T} \mathbf{M} \dot{\mathbf{u}} = 1\).

The vector \(\dot{\mathbf{u}}\) that maximizes the previous expression is the eigenvector associated to the eigenvalue of:

\[ \mathbf{M}^{-1} (\mathbf{M X^\mathsf{T} NXM}) \dot{\mathbf{u}} = \lambda \dot{\mathbf{u}} \]

Simplifying we get:

\[ \mathbf{X^\mathsf{T} NXM} \dot{\mathbf{u}} = \lambda \dot{\mathbf{u}} \]

The maximum is obtained by:

\[ (\dot{\mathbf{u}} \mathbf{M})(\mathbf{X^\mathsf{T}NXM}) \dot{\mathbf{u}} = \lambda (\dot{\mathbf{u}}^\mathsf{T} \mathbf{M} \dot{\mathbf{u}}) = \lambda \]

The image of \(\dot{\mathbf{u}}\) by the metric \(\mathbf{M}\) is called factor: \(\hat{\mathbf{u}} = \mathbf{M} \dot{\mathbf{u}} = \mathbf{M}^{1/2} \mathbf{u}\)

Then:

\[ \mathbf{M}(\mathbf{X^\mathsf{T}NX}) \hat{\mathbf{u}} = \lambda \hat{\mathbf{u}} \]

where \(\hat{\mathbf{u}}\) is normalized by \(\mathbf{M}^{-1}\):

\[ \dot{\mathbf{u}}^\mathsf{T} \mathbf{M} \dot{\mathbf{u}} = \hat{\mathbf{u}}^\mathsf{T} \mathbf{M}^{-1} \hat{\mathbf{u}} = 1 \]

Diagonalizable Matrix

In practice, the matrix that is diagonalized is:

\[ \mathbf{M}^{1/2} \mathbf{X^\mathsf{T}NXM}^{1/2} \mathbf{u} = \lambda \mathbf{u} \]

being \(\mathbf{u} = \mathbf{M}^{1/2} \dot{\mathbf{u}}\), and consequently, \(\mathbf{u^\mathsf{T}u} = 1\)

Let \(\mathbf{Y} = \mathbf{N}^{1/2} \mathbf{XM}^{1/2}\), we can express the previous relation as:

\[ \mathbf{Y^\mathsf{T}Yu} = \lambda \mathbf{u} \]

Fit in n-dimensions

Let \(\dot{\mathbf{v}}\) a unit vector in \(\mathbb{R}^n\), with \(\dot{\mathbf{v}}^\mathsf{T} \mathbf{N} \dot{\mathbf{v}} = \mathbf{1}\).

The maximum projected inertia is obtained by maximizing:

\[ \dot{\mathbf{v}}^\mathsf{T} (\mathbf{NXMX^\mathsf{T}N}) \dot{\mathbf{v}} \]

We find the maximum by diagonalizing the matrix:

\[ (\mathbf{XMX^\mathsf{T}N}) \dot{\mathbf{v}} = \lambda \dot{\mathbf{v}} \]

The eigenvector \(\dot{\mathbf{v}}\) associated to the largest eigenvalue defines the direction of \(\mathbb{R}^n\) with maximum inertia.

We call factor to the vector in \(\mathbb{R}^n\): \(\hat{\mathbf{v}} = \mathbf{N} \dot{\mathbf{v}}\)

This factor verifies the relationship: \((\mathbf{NXMX^\mathsf{T}}) \hat{\mathbf{v}} = \lambda \hat{\mathbf{v}}\), with \(\hat{\mathbf{v}}^\mathsf{T} \mathbf{N}^{-1} \hat{\mathbf{v}} = 1\)

Symmetric Matrix

Introducing the metric in the coordinates: \(\mathbf{v} = \mathbf{N}^{1/2} \dot{\mathbf{v}}\)

\[ \mathbf{N}^{1/2} \mathbf{XMX^\mathsf{T}N}^{1/2}\mathbf{v} = \lambda \mathbf{v} \quad \text{with} \quad \mathbf{v^\mathsf{T}v} = 1 \]

Utilizing the matrix \(\mathbf{Y}\) we have:

\[ \mathbf{Y^\mathsf{T}Yv} = \lambda \mathbf{v} \]

Transition Relations

Fit in \(\mathbb{R}^p\): \((\mathbf{X^\mathsf{T}NXM}) \dot{\mathbf{u}} = \lambda \dot{\mathbf{u}}\)

Fit in \(\mathbb{R}^n\): \((\mathbf{XMX^\mathsf{T}N}) \dot{\mathbf{v}} = \lambda \dot{\mathbf{v}}\)

\[ \mathbf{XMX^\mathsf{T}N} (\mathbf{XM} \dot{\mathbf{u}}) = \lambda (\mathbf{XM} \dot{\mathbf{u}}) \]

\[ \mathbf{X^\mathsf{T}NXM} (\mathbf{X^\mathsf{T}N} \dot{\mathbf{v}}) = \lambda (\mathbf{X^\mathsf{T}N} \dot{\mathbf{v}}) \]

Comparing the previous two relations, and imposing a normalizing restriction on the eigenvectors we have:

\[ (\mathbf{XM}\dot{\mathbf{u}})^\mathsf{T} \mathbf{N} (\mathbf{XM}\dot{\mathbf{u}}) = \lambda \]

and

\[ (\mathbf{X^\mathsf{T}N}\dot{\mathbf{v}})^\mathsf{T} \mathbf{M} (\mathbf{X^\mathsf{T}N}\dot{\mathbf{v}}) = \lambda \]

We deduct the so-called transition relations:

\[ \dot{\mathbf{v}} = \frac{1}{\sqrt{\lambda}} \mathbf{XM} \dot{\mathbf{u}} \]

and

\[ \dot{\mathbf{u}} = \frac{1}{\sqrt{\lambda}} \mathbf{X^\mathsf{T}N} \dot{\mathbf{v}} \]

In practice, we operate with symmetric matrices and thus the transition relations become:

\[ \mathbf{v} = \frac{1}{\sqrt{\lambda}} \mathbf{N}^{1/2} \mathbf{XM}^{1/2}\mathbf{u} \]

and

\[ \mathbf{u} = \frac{1}{\sqrt{\lambda}} \mathbf{M}^{1/2} \mathbf{X^\mathsf{T}N}^{1/2}\mathbf{v} \]