Positive Definite Matrices — Section 17: Linear Algebra

A symmetric matrix $A$ is positive semi-definite (PSD) if $x^\top A x \ge 0$ for all $x$ , and positive definite (PD) if the inequality is strict for $x \ne 0$ .

Equivalent characterizations:

All eigenvalues are non-negative (PSD) or strictly positive (PD).
Cholesky decomposition $A = L L^\top$ exists with $L$ lower-triangular (and unique with positive diagonal for PD).
Every principal minor's determinant is non-negative (PSD) or positive (PD).

Covariance matrices are always PSD because $a^\top \Sigma a = \mathrm{Var}(a^\top X) \ge 0$ . They're PD if no linear combination of the variables is constant — usually the case in practice.

PD matrices are essential for solving linear systems efficiently (Cholesky is $\sim 2\times$ faster than LU), for guaranteeing convex quadratic objective functions, and for ensuring that mean-variance portfolio optimization has a unique solution.

A symmetric matrix $A$ is positive semi-definite (PSD) if $x^\top A x \ge 0$ for all $x$ , and positive definite (PD) if the inequality is strict for $x \ne 0$ .

Equivalent characterizations:

All eigenvalues are non-negative (PSD) or strictly positive (PD).
Cholesky decomposition $A = L L^\top$ exists with $L$ lower-triangular (and unique with positive diagonal for PD).
Every principal minor's determinant is non-negative (PSD) or positive (PD).

Covariance matrices are always PSD because $a^\top \Sigma a = \mathrm{Var}(a^\top X) \ge 0$ . They're PD if no linear combination of the variables is constant — usually the case in practice.