Hypothesis Testing — Section 8: Frequentist Inference

A hypothesis test is a structured way to evaluate claims about a population using sample data. The framework pits a null hypothesis $H_0$ against an alternative $H_1$ , computes a test statistic from the data, and compares it to a reference distribution under $H_0$ .

The standard recipe:

1. Specify $H_0$ and $H_1$ . 2. Choose a significance level $\alpha$ — usually $0.05$ . 3. Compute the test statistic and its $p$ -value. 4. Reject $H_0$ if $p < \alpha$ .

A canonical example: the two-sample $t$ -test for a difference in means uses

t = \frac{\bar{X}_1 - \bar{X}_2}{\mathrm{SE}}

compared to a $t$ -distribution with the appropriate degrees of freedom.

In trading, hypothesis tests are everywhere — A/B comparisons of execution algorithms, signal strength tests, alpha decay studies. Note that financial data is heavy-tailed and serially dependent, so vanilla $t$ -tests can be misleading. Robust standard errors, block bootstrap, and stationary corrections become important in practice.

The standard recipe:

1. Specify $H_0$ and $H_1$ . 2. Choose a significance level $\alpha$ — usually $0.05$ . 3. Compute the test statistic and its $p$ -value. 4. Reject $H_0$ if $p < \alpha$ .

A canonical example: the two-sample $t$ -test for a difference in means uses

t = \frac{\bar{X}_1 - \bar{X}_2}{\mathrm{SE}}

compared to a $t$ -distribution with the appropriate degrees of freedom.