Forecasting future values from past observations has a long history. ARIMA dominated until the 2010s; today's toolkit is broader.
ARIMA
Combines autoregression (AR: dependence on past values), differencing (I: handle trends), and moving average (MA: dependence on past errors). Fit via maximum likelihood. The Box-Jenkins methodology: check stationarity → identify (p, d, q) → fit → diagnose residuals → forecast.
Strengths: principled, gives prediction intervals, well-understood. Weaknesses: assumes linearity, needs careful hand-tuning, doesn't handle multiple seasonalities, struggles with breaks/changepoints.
Exponential smoothing (ETS)
Forecasts are weighted averages of past observations, with exponentially decaying weights. Holt-Winters extends this with trend and seasonal components. Often comparable to ARIMA in accuracy, simpler to tune, robust default for short horizon.
Prophet
Facebook's open-source forecaster. Models trend + multiple seasonalities + holidays as an additive regression with changepoints. Works well out-of-the-box on business time series with seasonality and irregular events.
State-space models / Kalman filters
Model time series as a latent state evolving via known dynamics plus measurement noise. Extremely general — ARIMA, ETS, structural time series are all special cases. Implementation: pykalman, statsmodels' UnobservedComponents.
Machine learning approaches
Tree-based (LightGBM, XGBoost) on lagged features often wins on M4-style competitions. Pros: handles many series jointly, easy to incorporate external regressors. Cons: doesn't naturally extrapolate trends beyond seen values; needs careful feature engineering for lags.
Neural networks
RNNs (LSTM, GRU) and Transformers can model long dependencies in long, complex series. Modern approaches (N-BEATS, N-HiTS, Temporal Fusion Transformer, TimeGPT) achieve state-of-the-art on benchmarks. Need a LOT of data to beat simpler methods; for moderate data, ARIMA / Prophet / ETS still hold their own.
Picking a method
- Short series (< 100 points), seasonal: ETS or seasonal ARIMA
- Business metrics with holidays, changepoints: Prophet
- Many parallel series (sales for thousands of products): tree-based with cross-series features
- Long, complex, large datasets: neural networks
- Need uncertainty quantification: state-space or Bayesian methods