6.1 General Analysis

Notation:

$$\mathbf{X}$$: matrix of $$n$$ rows (corresponding to the individuals), and $$p$$ columns (corresponding to the variables). We assume mean-centered variables.

$$\mathbf{N}$$: diagonal matrix of dimension $$n \times n$$, with $$n_{ii} \geq 0$$

$$\mathbf{M}$$: matrix of dimension $$p \times p$$, positive definite ($$\mathbf{a^\mathsf{T} M a} > \mathbf{0}$$ for all $$\mathbf{a}$$).

Cloud of Points

In $$\mathbb{R}^p$$ we assume $$n$$ points weighted by $$\mathbf{N}$$, and with metric $$\mathbf{M}.$$

In $$\mathbb{R}^n$$ we assume $$p$$ points weighted by $$\mathbf{M}$$, and with metric $$\mathbf{N}.$$

Fit in p-dimensions

Let $$\dot{\mathbf{u}}$$ be a unit vector (e.g. $$\dot{\mathbf{u}}^\mathsf{T} \mathbf{Mu} = \mathbf{1}$$), defining a direction in $$\mathbb{R}^p$$. In this direction we want to maximize the inertia of the projection of the individuals.

Projection of individuals: $$\mathbf{\Psi} = \mathbf{XM} \dot{\mathbf{u}}$$

Inertia of projection: $$\mathbf{\Psi^\mathsf{T} N \Psi} = (\mathbf{XM}\dot{\mathbf{u}})^\mathsf{T} \mathbf{N} (\mathbf{XM}\dot{\mathbf{u}})$$

The maximum projected inertia can be obtained by maximizing: $$\dot{\mathbf{u}}^\mathsf{T} (\mathbf{MX^\mathsf{T}NXM}) \dot{\mathbf{u}}$$, with the condition $$\dot{\mathbf{u}}^\mathsf{T} \mathbf{M} \dot{\mathbf{u}} = 1$$.

The vector $$\dot{\mathbf{u}}$$ that maximizes the previous expression is the eigenvector associated to the eigenvalue of:

$\mathbf{M}^{-1} (\mathbf{M X^\mathsf{T} NXM}) \dot{\mathbf{u}} = \lambda \dot{\mathbf{u}}$

Simplifying we get:

$\mathbf{X^\mathsf{T} NXM} \dot{\mathbf{u}} = \lambda \dot{\mathbf{u}}$

The maximum is obtained by:

$(\dot{\mathbf{u}} \mathbf{M})(\mathbf{X^\mathsf{T}NXM}) \dot{\mathbf{u}} = \lambda (\dot{\mathbf{u}}^\mathsf{T} \mathbf{M} \dot{\mathbf{u}}) = \lambda$

The image of $$\dot{\mathbf{u}}$$ by the metric $$\mathbf{M}$$ is called factor: $$\hat{\mathbf{u}} = \mathbf{M} \dot{\mathbf{u}} = \mathbf{M}^{1/2} \mathbf{u}$$

Then:

$\mathbf{M}(\mathbf{X^\mathsf{T}NX}) \hat{\mathbf{u}} = \lambda \hat{\mathbf{u}}$

where $$\hat{\mathbf{u}}$$ is normalized by $$\mathbf{M}^{-1}$$:

$\dot{\mathbf{u}}^\mathsf{T} \mathbf{M} \dot{\mathbf{u}} = \hat{\mathbf{u}}^\mathsf{T} \mathbf{M}^{-1} \hat{\mathbf{u}} = 1$

Diagonalizable Matrix

In practice, the matrix that is diagonalized is:

$\mathbf{M}^{1/2} \mathbf{X^\mathsf{T}NXM}^{1/2} \mathbf{u} = \lambda \mathbf{u}$

being $$\mathbf{u} = \mathbf{M}^{1/2} \dot{\mathbf{u}}$$, and consequently, $$\mathbf{u^\mathsf{T}u} = 1$$

Let $$\mathbf{Y} = \mathbf{N}^{1/2} \mathbf{XM}^{1/2}$$, we can express the previous relation as:

$\mathbf{Y^\mathsf{T}Yu} = \lambda \mathbf{u}$

Fit in n-dimensions

Let $$\dot{\mathbf{v}}$$ a unit vector in $$\mathbb{R}^n$$, with $$\dot{\mathbf{v}}^\mathsf{T} \mathbf{N} \dot{\mathbf{v}} = \mathbf{1}$$.

The maximum projected inertia is obtained by maximizing:

$\dot{\mathbf{v}}^\mathsf{T} (\mathbf{NXMX^\mathsf{T}N}) \dot{\mathbf{v}}$

We find the maximum by diagonalizing the matrix:

$(\mathbf{XMX^\mathsf{T}N}) \dot{\mathbf{v}} = \lambda \dot{\mathbf{v}}$

The eigenvector $$\dot{\mathbf{v}}$$ associated to the largest eigenvalue defines the direction of $$\mathbb{R}^n$$ with maximum inertia.

We call factor to the vector in $$\mathbb{R}^n$$: $$\hat{\mathbf{v}} = \mathbf{N} \dot{\mathbf{v}}$$

This factor verifies the relationship: $$(\mathbf{NXMX^\mathsf{T}}) \hat{\mathbf{v}} = \lambda \hat{\mathbf{v}}$$, with $$\hat{\mathbf{v}}^\mathsf{T} \mathbf{N}^{-1} \hat{\mathbf{v}} = 1$$

Symmetric Matrix

Introducing the metric in the coordinates: $$\mathbf{v} = \mathbf{N}^{1/2} \dot{\mathbf{v}}$$

$\mathbf{N}^{1/2} \mathbf{XMX^\mathsf{T}N}^{1/2}\mathbf{v} = \lambda \mathbf{v} \quad \text{with} \quad \mathbf{v^\mathsf{T}v} = 1$

Utilizing the matrix $$\mathbf{Y}$$ we have:

$\mathbf{Y^\mathsf{T}Yv} = \lambda \mathbf{v}$

Transition Relations

Fit in $$\mathbb{R}^p$$: $$(\mathbf{X^\mathsf{T}NXM}) \dot{\mathbf{u}} = \lambda \dot{\mathbf{u}}$$

Fit in $$\mathbb{R}^n$$: $$(\mathbf{XMX^\mathsf{T}N}) \dot{\mathbf{v}} = \lambda \dot{\mathbf{v}}$$

$\mathbf{XMX^\mathsf{T}N} (\mathbf{XM} \dot{\mathbf{u}}) = \lambda (\mathbf{XM} \dot{\mathbf{u}})$

$\mathbf{X^\mathsf{T}NXM} (\mathbf{X^\mathsf{T}N} \dot{\mathbf{v}}) = \lambda (\mathbf{X^\mathsf{T}N} \dot{\mathbf{v}})$

Comparing the previous two relations, and imposing a normalizing restriction on the eigenvectors we have:

$(\mathbf{XM}\dot{\mathbf{u}})^\mathsf{T} \mathbf{N} (\mathbf{XM}\dot{\mathbf{u}}) = \lambda$

and

$(\mathbf{X^\mathsf{T}N}\dot{\mathbf{v}})^\mathsf{T} \mathbf{M} (\mathbf{X^\mathsf{T}N}\dot{\mathbf{v}}) = \lambda$

We deduct the so-called transition relations:

$\dot{\mathbf{v}} = \frac{1}{\sqrt{\lambda}} \mathbf{XM} \dot{\mathbf{u}}$

and

$\dot{\mathbf{u}} = \frac{1}{\sqrt{\lambda}} \mathbf{X^\mathsf{T}N} \dot{\mathbf{v}}$

In practice, we operate with symmetric matrices and thus the transition relations become:

$\mathbf{v} = \frac{1}{\sqrt{\lambda}} \mathbf{N}^{1/2} \mathbf{XM}^{1/2}\mathbf{u}$

and

$\mathbf{u} = \frac{1}{\sqrt{\lambda}} \mathbf{M}^{1/2} \mathbf{X^\mathsf{T}N}^{1/2}\mathbf{v}$