Bayesian Philosophy — Section 10: Bayesian Inference

Bayesian inference treats probability as a quantitative measure of belief. Parameters aren't fixed unknowns — they have probability distributions reflecting uncertainty.

A Bayesian analysis has three ingredients:

The prior $P(\theta)$ encodes what we believe about the parameter before seeing data.
The likelihood $P(\text{data} \mid \theta)$ describes how data are generated given the parameter.
The posterior $P(\theta \mid \text{data})$ is the updated belief, computed via Bayes' Rule.

The frequentist asks "what is the value of $\theta$ ?" and answers with point estimates and intervals about the procedure. The Bayesian asks "what is my distribution over $\theta$ ?" and answers with the full posterior. Bayesian methods naturally handle nested models, regularization (priors as soft constraints), and propagating uncertainty into predictions — at the cost of needing to specify a prior, which can be controversial.

Bayesian inference treats probability as a quantitative measure of belief. Parameters aren't fixed unknowns — they have probability distributions reflecting uncertainty.

A Bayesian analysis has three ingredients:

The prior $P(\theta)$ encodes what we believe about the parameter before seeing data.
The likelihood $P(\text{data} \mid \theta)$ describes how data are generated given the parameter.
The posterior $P(\theta \mid \text{data})$ is the updated belief, computed via Bayes' Rule.