Singular Value Decomposition — Section 17: Linear Algebra

Every matrix $A \in \mathbb{R}^{m \times n}$ has a singular value decomposition

A = U \Sigma V^\top

where $U$ and $V$ have orthonormal columns and $\Sigma$ is diagonal with non-negative entries (the singular values $\sigma_1 \ge \sigma_2 \ge \dots \ge 0$ ).

SVD is the Swiss Army knife of linear algebra:

Rank of $A$ is the number of non-zero singular values.
$\|A\|_2 = \sigma_1$ , the largest singular value.
The best rank- $k$ approximation to $A$ in Frobenius norm is the truncation $\sum_{i=1}^{k} \sigma_i u_i v_i^\top$ — the basis of low-rank compression and noise reduction.
The pseudo-inverse $A^+ = V \Sigma^+ U^\top$ solves the least-squares problem even when $A$ is singular.

In finance, SVD powers PCA-on-data-matrices, robust factor extraction, and the dimensionality reduction behind risk-factor decomposition of large covariance matrices.

Every matrix $A \in \mathbb{R}^{m \times n}$ has a singular value decomposition

A = U \Sigma V^\top

where $U$ and $V$ have orthonormal columns and $\Sigma$ is diagonal with non-negative entries (the singular values $\sigma_1 \ge \sigma_2 \ge \dots \ge 0$ ).

SVD is the Swiss Army knife of linear algebra:

Rank of $A$ is the number of non-zero singular values.
$\|A\|_2 = \sigma_1$ , the largest singular value.
The best rank- $k$ approximation to $A$ in Frobenius norm is the truncation $\sum_{i=1}^{k} \sigma_i u_i v_i^\top$ — the basis of low-rank compression and noise reduction.
The pseudo-inverse $A^+ = V \Sigma^+ U^\top$ solves the least-squares problem even when $A$ is singular.

In finance, SVD powers PCA-on-data-matrices, robust factor extraction, and the dimensionality reduction behind risk-factor decomposition of large covariance matrices.