Linear regression — fitting a line through returns

The setup

You have pairs of observations $(x_t, y_t)$ for $t = 1, \ldots, N$ — read: "x and y at time t, for t running from 1 up to N." Think of a scatter plot: each day gives you one dot, with that day's SPY return on the x-axis and that day's AAPL return on the y-axis. Linear regression fits the best straight line through the dots:

$y_t = \alpha + \beta x_t + \varepsilon_t$

In words: each day's y equals a fixed baseline, plus a slope times that day's x, plus whatever is left over.

Symbol by symbol:

• $\alpha$ — the Greek letter alpha — is the intercept: the value of $y$ when $x$ is zero.
• $\beta$ — beta — is the slope: how much $y$ moves for each 1-unit move in $x$.
• $\varepsilon_t$ — the Greek letter epsilon, read "epsilon sub t" — is the residual at observation $t$: the leftover gap between the actual $y_t$ and what the line predicted. Every dot misses the line by some amount; that miss is the residual.

What counts as the "best" line? The usual answer is ordinary least squares (OLS): choose the line that minimizes the sum of squared residuals, $\sum \varepsilon_t^2$ — for each dot, square its miss, add them all up, and make that total as small as possible. Squaring means big misses hurt much more than small ones.

The OLS answers have closed forms — direct formulas, no trial-and-error search needed. One more piece of notation first: a hat, as in $\hat\beta$ (read "beta hat"), marks an estimate computed from data; a bar, as in $\bar{x}$ (read "x bar"), means the average of $x$:

$\hat\beta = \frac{\text{Cov}(x, y)}{\text{Var}(x)}, \qquad \hat\alpha = \bar{y} - \hat\beta \bar{x}$

In words: the estimated slope is the covariance of x and y (how much they move together — see A2-08) divided by the variance of x (how much x moves on its own). The estimated intercept is the average of y minus the slope times the average of x — which just forces the line to pass through the point of averages.

So beta answers: "how much does $y$ move with $x$, in $y$-units per $x$-unit?" The division by $\text{Var}(x)$ rescales the co-movement into that per-unit form.

In finance: regressing on market returns

The single most-used regression in finance is the same line, with returns plugged in:

$r_{i,t} = \alpha_i + \beta_i \cdot r_{m,t} + \varepsilon_{i,t}$

In words: the return of stock i on day t equals the stock's own baseline return, plus its market sensitivity times the market's return that day, plus stock-specific noise.

Reading the subscripts (each one is just a label):

• $r_{i,t}$ — "r sub i, t" — the return on asset $i$ at time $t$.
• $r_{m,t}$ — "r sub m, t" — the market return at time $t$ (usually SPY or a broader index).
• $\alpha_i$ — the asset's idiosyncratic mean return — "idiosyncratic" means specific to this asset, unrelated to the market. It's what you'd earn if the market were flat.
• $\beta_i$ — the asset's sensitivity to market moves.

Real-world beta values: for AAPL regressed on SPY since 2010, $\beta \approx 1.1$. Utilities sit around $\beta \approx 0.5$ — less market-sensitive. High-beta tech and biotech can run 1.5–2.5. Long/short hedged portfolios — funds that own some stocks and bet against others — target $\beta \approx 0$ by construction.

This is the Capital Asset Pricing Model regression — we'll cover its economic interpretation in the next lesson (D2-01).

Correlation vs. slope

A frequent confusion: correlation $\rho$ (the Greek letter rho, from A2-08) and slope $\beta$ are related but different:

$\rho_{xy} = \frac{\text{Cov}(x, y)}{\sigma_x \sigma_y}, \qquad \beta = \frac{\text{Cov}(x, y)}{\sigma_x^2} = \rho_{xy} \cdot \frac{\sigma_y}{\sigma_x}$

In words: both start from the covariance of x and y, but divide by different things. Correlation divides by both standard deviations, giving a pure, unitless number. Slope divides by the variance of x only — so the slope equals the correlation times the ratio of the two standard deviations.

Correlation is unitless and always lands between −1 and +1. Slope has units of "y per x" and can be any number.

Concrete example. Returns of AAPL and SPY have $\rho = 0.7$, with $\sigma_\text{AAPL} = 1.5\%$ and $\sigma_\text{SPY} = 1.0\%$ (AAPL swings 1.5× as much as SPY). Then:

$\beta = 0.7 \cdot \frac{1.5}{1.0} = 1.05$

You can have high correlation but low beta (correlated but $y$ moves less than $x$) or low correlation but high beta (rarely co-moves but when it does, $y$ moves a lot).

$R^2$ — explained variance

$R^2$ — read "R squared" — measures what fraction of $y$'s variance the regression explains:

$R^2 = 1 - \frac{\text{Var}(\varepsilon)}{\text{Var}(y)}$

In words: take the variance of the leftovers (the residuals), divide by the total variance of y, and subtract that fraction from 1. What remains is the share of y's movement the line accounts for.

For a regression with one predictor, $R^2 = \rho^2$ — just the correlation, squared. Correlation of 0.7 means $R^2$ of about 0.49.

• $R^2 = 0$: regression explains nothing.
• $R^2 = 1$: regression is a perfect fit (all residuals zero — usually a sign of overfitting, or that you accidentally regressed a variable on itself).
• $R^2 = 0.5$: regression explains half the variance.

For AAPL on SPY: $R^2 \approx 0.5$. About half the variance of AAPL is explained by the market; the rest is idiosyncratic (Apple-specific news).

Multiple regression — more factors

If you have multiple predictors $x_1, x_2, \ldots, x_k$ — that's $k$ different input series, each with its own subscript:

$y_t = \alpha + \beta_1 x_{1,t} + \beta_2 x_{2,t} + \cdots + \beta_k x_{k,t} + \varepsilon_t$

In words: y is a baseline, plus a separate slope times each predictor, plus a residual. Each slope $\beta_j$ measures y's sensitivity to predictor $j$ holding the other predictors fixed.

OLS still has a closed-form solution (the normal equations — a system of equations you solve directly), but the math is cleaner in matrix notation. We'll meet that in Track A3 (linear algebra) and use it in Track D2 (factor models — Fama-French 3 and 5 factors, etc.).

A common mistake: regress AAPL on (SPY, momentum, value) and get coefficient estimates that "look reasonable." But if the predictors are correlated with each other — a condition called multicollinearity — the individual coefficient estimates become unstable. We'll cover diagnostics in A2-07.

Code snippet (for laptop)

import statsmodels.api as sm
import yfinance as yf
import numpy as np

data = yf.download(["AAPL", "SPY"], start="2010-01-01")["Adj Close"]
returns = np.log(data / data.shift(1)).dropna()

# Need to add a constant for the intercept
X = sm.add_constant(returns["SPY"])
y = returns["AAPL"]

model = sm.OLS(y, X).fit()
print(model.summary())
# Look for: const (alpha), SPY (beta), R-squared, t-statistics, p-values

statsmodels gives you a full diagnostic printout — standard errors, t-stats, p-values, R². For Phase 1 we'll use statsmodels for almost all regressions; for production models, numpy.linalg.lstsq or scikit-learn are also options.

Assumptions OLS makes (and where they break)

Under five assumptions, OLS is "BLUE" — the Best Linear Unbiased Estimator, meaning no other line-fitting method of its kind gets closer to the truth on average:

1. Linearity — the true relationship really is a straight line.
2. No autocorrelation — residuals are independent over time — today's miss tells you nothing about tomorrow's.
3. Homoscedasticity — read "ho-mo-skuh-das-TIH-city" — residuals have constant variance: the misses are equally noisy everywhere, not calm in some periods and wild in others.
4. Normality of residuals — the misses follow a bell curve. Only needed for hypothesis tests, not for the fitted line itself.
5. No multicollinearity among predictors.

Financial data breaks 2, 3, and often 5. The point estimates — the fitted $\hat\alpha$ and $\hat\beta$ themselves — are still consistent (they converge to the truth as data grows). But the standard errors are wrong, which means your t-statistics and p-values are wrong. You can trust the line more than you can trust the claim that the line is "significant."

Fix: use corrected standard errors. Newey-West corrects for autocorrelation; White (also called "robust") corrects for heteroscedasticity; HAC — heteroscedasticity and autocorrelation consistent — corrects for both. statsmodels provides these via model.fit(cov_type='HAC', cov_kwds={'maxlags': 5}).

We'll cover diagnostics and corrections in A2-07.

Try it

Compute beta from the closed form — no statsmodels needed, just the covariance-over-variance formula:

Predict before the next lesson

Tomorrow: CAPM (D2-01), where we interpret the alpha and beta of the market regression economically. Predict:

• If a stock has positive alpha vs. the market, what does this imply about its risk-adjusted return?
• Why might a stock with β = 1.5 reasonably be expected to have higher returns than the market?