Maximum Entropy — Section 16: Information Theory

The maximum entropy principle says: given constraints (e.g. specified mean, variance, or moments), pick the distribution with maximum entropy that satisfies them. This gives the most non-committal distribution consistent with what you know.

Some classic results:

Maximum entropy on a finite support with no other constraints → uniform.
Maximum entropy on $[0, \infty)$ with fixed mean → Exponential.
Maximum entropy on $\mathbb{R}$ with fixed mean and variance → Gaussian.

In quant work, maximum entropy underlies the principle of indifference (uniform priors when nothing else is known) and a particular flavor of model calibration: pick model parameters that maximize entropy subject to matching observed prices or moments.

Some classic results:

Maximum entropy on a finite support with no other constraints → uniform.
Maximum entropy on $[0, \infty)$ with fixed mean → Exponential.
Maximum entropy on $\mathbb{R}$ with fixed mean and variance → Gaussian.