The Law of Large Numbers and the Central Limit Theorem

The Law of Large Numbers

Take $N$ independent draws from any distribution with true mean $\mu$. The Law of Large Numbers (LLN) says the sample mean converges to the truth:

$\bar{X}_N = \frac{1}{N}\sum_{i=1}^{N} X_i \;\longrightarrow\; \mu \quad \text{as } N \to \infty$

This is why casinos are not gambling. On American roulette, a $1 bet on red wins with probability 18/38, so the house's expected take is$2/38 \approx 5.3$ cents per dollar — but any single spin is nearly a coin flip, and any single gambler can walk away rich. Over a million spins, though, the average take per spin locks onto 5.3 cents with almost no wiggle. The casino doesn't need to win each bet; it needs volume.

Trading translation: a strategy's true edge only reveals itself over many independent trades. Ten winning trades tell you almost nothing (the gambler walking away rich); a thousand trades start to pin down whether $\mu > 0$. This is also the honest reading of A2-01's warning that mean returns are hard to estimate — the LLN guarantees convergence, but says nothing about it being fast.

The Central Limit Theorem, stated properly

The LLN says where the sample mean goes. The Central Limit Theorem (CLT) says how it fluctuates on the way. In plain terms:

Take $N$ independent draws from any distribution with mean $\mu$ and finite standard deviation $\sigma$. For large $N$, the sample mean $\bar{X}_N$ is approximately normally distributed, centered at $\mu$, with standard deviation $\sigma / \sqrt{N}$.

Every clause earns its keep:

• Any distribution. Uniform, coin flips, skewed payoffs — the average comes out bell-shaped regardless. This is why the normal distribution is everywhere: it's the universal shape of aggregated noise.
• Centered at $\mu$ — that's the LLN inside the CLT.
• Spread $\sigma/\sqrt{N}$ — quadruple the data, halve the noise. This $\sqrt{N}$ is the same one behind A2-01's standard error and A2-02's annualization. It is arguably the most important constant-shape fact in all of statistics: precision grows only as the square root of effort.
• Finite $\sigma$, independent draws — the fine print. Both clauses get violated in finance, which is where the next section lives.

Paying off two IOUs

Two earlier lessons name-dropped the CLT and promised an explanation. Debts now settled:

• A1-03 claimed sample averages and derived statistics (Sharpe estimates, regression coefficients) are approximately normal even when individual returns aren't. That's the CLT: those statistics are all averages in disguise, and averaging manufactures normality.
• A2-03 claimed monthly returns look more normal than daily returns. A monthly log-return is the sum of ~21 daily ones — a partial CLT aggregation. More aggregation (yearly), more normal.

What the CLT does not do

The CLT is about averages of many draws. Tomorrow's return is one draw. No averaging happens inside a single day, so the CLT does nothing to tame the fat tails from A1-03 — a 5-sigma daily move remains vastly more likely than the normal distribution says, and no theorem rescues you from it.

Two further honest caveats:

• Convergence is slowest in the tails. Even for monthly returns, the center of the distribution normalizes long before the extremes do. The CLT approximation is best exactly where you need it least.
• It assumes independence and finite variance. Volatility clustering weakens the independence assumption, and very heavy tails (where variance is barely finite) slow convergence dramatically.

Rule of thumb: trust CLT-based reasoning for estimates and averages (standard errors, confidence intervals — coming in A2-09); never use it to reason about single-period risk.

See it, don't take my word

The CLT is one of the few theorems you can watch happen. A uniform distribution on $[0, 1]$ is flat — nothing bell-shaped about it. Its mean is $1/2$ and its variance is $1/12$, so the CLT predicts that means of $N$ draws should pile up around $0.5$ with standard deviation $\sqrt{1/(12N)}$: about $0.0289$ for $N=100$ and $0.0144$ for $N=400$.

While the code runs, also plot a histogram of the batch means in your head: flat input, bell-shaped output. That transformation is the whole theorem.

Draw it

From memory, sketch three histograms side by side: (1) single draws from uniform(0,1), (2) means of 10 draws, (3) means of 100 draws. Label the center of each and annotate the widths so they shrink in the right proportion. Then add one sentence under the sketch: why doesn't panel (3)'s bell shape apply to a single day's stock return?