Plotting market data honestly

Plot before you compute

The first thing a quant does with new data is not statistics — it's a plot. Your eye is a superb anomaly detector, and price data arrives broken more often than you'd expect:

• Unadjusted splits. A raw Close series for AAPL shows a fake −75% "crash" on its 4-for-1 split day in August 2020. Feed that into a backtest and your strategy just "learned" a phantom event. On a plot it's an unmissable cliff.
• Gaps and stale data. Missing months, a price that flatlines for 60 days (a halted or delisted ticker still being carried), timestamps that jump backward.
• Fat-finger vendor errors. A single tick at 10× the neighboring prices will dominate your volatility estimate and be blindingly obvious on a chart.

Every one of these will silently poison a mean, a vol estimate, or a backtest. Thirty seconds of looking is the cheapest data-quality check that exists. Make "plot it first" a reflex.

The core plot: prices over time

import matplotlib.pyplot as plt

fig, ax = plt.subplots(figsize=(10, 5))
ax.plot(spy.index, spy["Adj Close"], lw=1)
ax.set_title("SPY, adjusted close")
ax.set_ylabel("Price ($)")
fig.autofmt_xdate()   # readable, angled date labels

With a pandas DatetimeIndex, matplotlib handles dates natively. For anything longer than a few years, add one line:

ax.set_yscale("log")

Why log scale matters: compounding is multiplicative, so the honest question is always "what percentage move was that?" On a log axis, equal vertical distances are equal percentage moves — a double from 100 to 200 takes exactly as much ink as a double from 1,000 to 2,000. A steady 8%-a-year asset plots as a straight line, and deviations from trend become visible across the whole history, not just the expensive recent part. Linear scale for short windows (months) is fine; log scale for long horizons is non-negotiable.

Histograms of returns

Prices trend; returns are where the statistics live. Plot their distribution:

fig, ax = plt.subplots()
ax.hist(spy["Adj Close"].pct_change().dropna(), bins=100)
ax.set_xlabel("Daily return")

Two practical rules. First, use enough bins — with thousands of daily observations, 100 bins is reasonable; the default of 10 smears the sharp peak and the tails into mush, hiding exactly the features you're looking for. Second, know what you're looking for: real daily equity returns show a tall narrow peak and long tails compared to a normal curve with the same standard deviation. Those few extreme bars far from the center are the fat tails from A2 — a histogram makes them physical.

For a sharper tail diagnostic, use the Q-Q plot you met in A2-03: sample quantiles against normal quantiles, straight line if normal. Market returns bend away at both ends — the pattern to memorize is "S-shaped tails = fatter than normal." The histogram gives intuition; the Q-Q plot gives evidence.

Comparing two series: rebase to 100

Plotting SPY (~$500) against a$40 stock on one axis tells you nothing — the cheap stock is a flat line pinned to the bottom. Normalize both to start at 100:

for col in ["SPY", "AAPL"]:
    rebased = 100 * prices[col] / prices[col].iloc[0]
    ax.plot(prices.index, rebased, label=col)
ax.legend()

Now the y-axis reads directly as "growth of 100 invested at the start," and the two lines are comparable at a glance. This is the standard chart of fund factsheets and backtest reports — you'll draw it hundreds of times. One caveat: the picture depends on the start date. Rebasing at a peak vs. a trough can flip which asset "looks better," which brings us to —

The honesty rules

Charts persuade — including their own author. Backtest fraud (and self-deception) is usually committed with axes, not numbers:

1. Never truncate the y-axis to exaggerate. A price chart windowed to [495, 505] makes a 0.5% wiggle look like a crash. Zero-based or clearly-labeled full-range axes for anything you'll show others.
2. No cherry-picked windows. A strategy chart that starts March 2009 (the exact post-crisis bottom) is an argument, not evidence. Show the longest history you have, drawdowns included.
3. Label everything. Title, y-axis units, whether returns are cumulative or per-period, whether the scale is log. An unlabeled chart is unfalsifiable.
4. Same scale for compared things. Two panels with different y-ranges invite false conclusions; rebase to 100 on one panel instead.

Tufte's phrase for the goal is graphical integrity: the visual impression should be proportional to the numbers. Your future self, reviewing an old backtest chart, is the person you're protecting.

Practice locally (Jupyter)

This lesson has no in-browser exercise — the embedded runtime has NumPy only, no matplotlib. Run this in a local Jupyter notebook.

import matplotlib.pyplot as plt
import yfinance as yf

spy = yf.download("SPY", start="1995-01-01", auto_adjust=False)
# Note: with auto_adjust=False you get both Close and Adj Close.
# Use Adj Close — raw Close has fake jumps on dividend/split days.
px = spy["Adj Close"].squeeze()

fig, axes = plt.subplots(2, 2, figsize=(12, 8))
axes[0, 0].plot(px);                axes[0, 0].set_title("SPY linear")
axes[0, 1].plot(px);                axes[0, 1].set_yscale("log")
axes[0, 1].set_title("SPY log")
rets = px.pct_change().dropna()
axes[1, 0].hist(rets, bins=100);    axes[1, 0].set_title("Daily returns")
axes[1, 1].plot(100 * px / px.iloc[0])
axes[1, 1].set_title("Growth of 100")
fig.tight_layout()

What to look for: the linear panel makes pre-2010 history look flat while the log panel shows 2000–02 and 2008 as the huge drawdowns they were (both roughly −50% peak to trough — on the log panel they finally look comparable to recent moves). The histogram should have a tall center spike and visible outlier bars beyond ±4 standard deviations. Then plot raw Close instead of Adj Close on the log panel and note how the dividend drag makes the two series diverge steadily over 30 years.

Draw it from memory

Your generative activity: on paper, sketch the same 30-year rising price series twice — once on a linear axis, once on a log axis. Mark where an early-history 20% crash appears on each, and label which chart makes it visible. Then sketch the shape of a daily-returns histogram versus a normal curve, exaggerating the features that differ.