Ordinary Least Squares — Section 11: Regression Analysis

Given data $(x_i, y_i)$ , OLS picks coefficients $(\alpha, \beta)$ to minimize the sum of squared residuals $\sum_i (y_i - \alpha - \beta x_i)^2$ . The closed-form solutions are

\hat{\beta} = \frac{\mathrm{Cov}(x, y)}{\mathrm{Var}(x)}, \qquad \hat{\alpha} = \bar{y} - \hat{\beta}\, \bar{x}

In matrix form, with response vector $y$ and design matrix $X$ :

\hat{\beta} = (X^\top X)^{-1} X^\top y

OLS rests on five assumptions: linearity, independent errors, homoscedasticity (constant error variance), no perfect multicollinearity, and (for classical inference) Normally distributed errors. Under the Gauss-Markov conditions (the first four), OLS is BLUE — the best linear unbiased estimator — even without Normality. Normality just buys you exact $t$ - and $F$ -distributions for inference.

In quant finance, OLS is the warhorse of factor models — regressing returns on market, size, value, and other factors gives you a portfolio's exposures.

Given data $(x_i, y_i)$ , OLS picks coefficients $(\alpha, \beta)$ to minimize the sum of squared residuals $\sum_i (y_i - \alpha - \beta x_i)^2$ . The closed-form solutions are

\hat{\beta} = \frac{\mathrm{Cov}(x, y)}{\mathrm{Var}(x)}, \qquad \hat{\alpha} = \bar{y} - \hat{\beta}\, \bar{x}

In matrix form, with response vector $y$ and design matrix $X$ :

\hat{\beta} = (X^\top X)^{-1} X^\top y

In quant finance, OLS is the warhorse of factor models — regressing returns on market, size, value, and other factors gives you a portfolio's exposures.