Linear regression — fitting a line through returns

The setup

Given pairs of observations $(x_t, y_t)$ for $t = 1, \ldots, N$, linear regression fits the best straight line:

$y_t = \alpha + \beta x_t + \varepsilon_t$

Where:

• $\alpha$ is the intercept
• $\beta$ is the slope
• $\varepsilon_t$ is the residual at observation $t$

"Best" usually means ordinary least squares (OLS): minimize the sum of squared residuals $\sum \varepsilon_t^2$.

The OLS estimates have closed forms:

$\hat\beta = \frac{\text{Cov}(x, y)}{\text{Var}(x)}, \qquad \hat\alpha = \bar{y} - \hat\beta \bar{x}$

So beta is "how much $y$ moves with $x$, in $y$-units per $x$-unit, normalized by the variance of $x$."

In finance: regressing on market returns

The single most-used regression in finance:

$r_{i,t} = \alpha_i + \beta_i \cdot r_{m,t} + \varepsilon_{i,t}$

• $r_{i,t}$ = return on asset $i$ at time $t$
• $r_{m,t}$ = market return (usually SPY or a broader index)
• $\alpha_i$ = asset's idiosyncratic mean return (what you'd earn if market were flat)
• $\beta_i$ = asset's sensitivity to market

For AAPL on SPY since 2010, $\beta \approx 1.1$. For utilities, $\beta \approx 0.5$ (less market-sensitive). For high-beta tech and biotech, $\beta$ can be 1.5–2.5. For long/short hedged portfolios, $\beta \approx 0$ by construction.

This is the Capital Asset Pricing Model regression — we'll cover its economic interpretation in the next lesson (D2-01).

Correlation vs. slope

A frequent confusion: correlation $\rho$ and slope $\beta$ are related but different:

$\rho_{xy} = \frac{\text{Cov}(x, y)}{\sigma_x \sigma_y}, \qquad \beta = \frac{\text{Cov}(x, y)}{\sigma_x^2} = \rho_{xy} \cdot \frac{\sigma_y}{\sigma_x}$

Correlation is unitless and bounded in [−1, 1]. Slope has units of "y per x" and can be any real number.

If returns of AAPL and SPY have $\rho = 0.7$ and $\sigma_\text{AAPL} = 1.5\%$, $\sigma_\text{SPY} = 1.0\%$:

$\beta = 0.7 \cdot \frac{1.5}{1.0} = 1.05$

You can have high correlation but low beta (correlated but $y$ moves less than $x$) or low correlation but high beta (rarely co-moves but when it does, $y$ moves a lot).

$R^2$ — explained variance

$R^2$ measures the fraction of $y$'s variance explained by the regression:

$R^2 = 1 - \frac{\text{Var}(\varepsilon)}{\text{Var}(y)}$

For a one-factor model, $R^2 = \rho^2$ (the squared correlation).

• $R^2 = 0$: regression explains nothing.
• $R^2 = 1$: regression is a perfect fit (all residuals zero — usually a sign of overfitting or an identity).
• $R^2 = 0.5$: regression explains half the variance.

For AAPL on SPY: $R^2 \approx 0.5$. About half the variance of AAPL is explained by the market; the rest is idiosyncratic (Apple-specific news).

Multiple regression — more factors

If you have multiple predictors $x_1, x_2, \ldots, x_k$:

$y_t = \alpha + \beta_1 x_{1,t} + \beta_2 x_{2,t} + \cdots + \beta_k x_{k,t} + \varepsilon_t$

OLS still has a closed form (the normal equations), but the math is easier in matrix notation. We'll meet this in Track A3 (linear algebra) and use it in Track D2 (factor models — Fama-French 3 and 5 factors, etc.).

A common mistake: regress AAPL on (SPY, momentum, value) and get coefficient estimates that "look reasonable." But if the factors are correlated with each other (multicollinearity), the individual coefficient estimates become unstable. We'll cover diagnostics in A2-07.

Code snippet (for laptop)

import statsmodels.api as sm
import yfinance as yf
import numpy as np

data = yf.download(["AAPL", "SPY"], start="2010-01-01")["Adj Close"]
returns = np.log(data / data.shift(1)).dropna()

# Need to add a constant for the intercept
X = sm.add_constant(returns["SPY"])
y = returns["AAPL"]

model = sm.OLS(y, X).fit()
print(model.summary())
# Look for: const (alpha), SPY (beta), R-squared, t-statistics, p-values

statsmodels gives you a full diagnostic printout — standard errors, t-stats, p-values, R². For Phase 1 we'll use statsmodels for almost all regressions; for production models, numpy.linalg.lstsq or scikit-learn are also options.

Assumptions OLS makes (and where they break)

OLS is "BLUE" (Best Linear Unbiased Estimator) under these assumptions:

1. Linearity — the true relationship is linear.
2. No autocorrelation — residuals are independent over time.
3. Homoscedasticity — residuals have constant variance.
4. Normality of residuals — only needed for hypothesis tests, not the point estimate.
5. No multicollinearity among predictors.

Financial data breaks 2, 3, and often 5. The point estimates are still consistent — the standard errors are wrong, which means your t-statistics and p-values are wrong.

Fix: use Newey-West standard errors (corrects for autocorrelation), White / robust standard errors (corrects for heteroscedasticity), or both (HAC). statsmodels provides these via model.fit(cov_type='HAC', cov_kwds={'maxlags': 5}).

We'll cover diagnostics and corrections in A2-07.

Try it

Compute beta from the closed form — no statsmodels needed, just the covariance-over-variance formula:

Predict before the next lesson

Tomorrow: CAPM (D2-01), where we interpret the alpha and beta of the market regression economically. Predict:

• If a stock has positive alpha vs. the market, what does this imply about its risk-adjusted return?
• Why might a stock with β = 1.5 reasonably be expected to have higher returns than the market?