Singular Value Decomposition

Every matrix $A \in \mathbb{R}^{m \times n}$ has a singular value decomposition

where $U$ and $V$ have orthonormal columns and $\Sigma$ is diagonal with non-negative entries (the singular values $\sigma_1 \ge \sigma_2 \ge \dots \ge 0$).

SVD is the Swiss Army knife of linear algebra:

Rank of $A$ is the number of non-zero singular values.$\|A\|_2 = \sigma_1$, the largest singular value.The best rank-$k$ approximation to $A$ in Frobenius norm is the truncation $\sum_{i=1}^{k} \sigma_i u_i v_i^\top$ — the basis of low-rank compression and noise reduction.The pseudo-inverse $A^+ = V \Sigma^+ U^\top$ solves the least-squares problem even when $A$ is singular.

In finance, SVD powers PCA-on-data-matrices, robust factor extraction, and the dimensionality reduction behind risk-factor decomposition of large covariance matrices.