Plotting market data honestly

Plot before you compute

The first thing a quant does with new data is not statistics — it's a plot. Your eye is a superb anomaly detector. And price data arrives broken more often than you'd expect:

• Unadjusted splits. A stock split — a company dividing each share into several cheaper ones — drops the price mechanically without changing anyone's wealth. A raw Close series for AAPL shows a fake −75% "crash" on its 4-for-1 split day in August 2020. Feed that into a backtest — a simulation of your strategy on historical data — and the strategy just "learned" a phantom event. On a plot it's an unmissable cliff.
• Gaps and stale data. Missing months. A price that flatlines for 60 days (a halted or delisted ticker still being carried). Timestamps that jump backward.
• Fat-finger vendor errors. A fat-finger error is a typo-like glitch in the data feed. A single tick — one recorded trade price — at 10× the neighboring prices will dominate your volatility estimate and be blindingly obvious on a chart.

Every one of these will silently poison a mean, a volatility ("vol") estimate, or a backtest. Thirty seconds of looking is the cheapest data-quality check that exists. Make "plot it first" a reflex.

The core plot: prices over time

import matplotlib.pyplot as plt

fig, ax = plt.subplots(figsize=(10, 5))
ax.plot(spy.index, spy["Adj Close"], lw=1)
ax.set_title("SPY, adjusted close")
ax.set_ylabel("Price ($)")
fig.autofmt_xdate()   # readable, angled date labels

With a pandas DatetimeIndex, matplotlib handles dates natively. For anything longer than a few years, add one line:

ax.set_yscale("log")

Why log scale matters: compounding is multiplicative — growth multiplies your money rather than adding to it — so the honest question is always "what percentage move was that?" On a log axis, equal vertical distances are equal percentage moves. A double from 100 to 200 takes exactly as much ink as a double from 1,000 to 2,000. A steady 8%-a-year asset plots as a straight line, so deviations from trend become visible across the whole history — not just the expensive recent part. Linear scale for short windows (months) is fine; log scale for long horizons is non-negotiable.

Histograms of returns

Prices trend; returns are where the statistics live. Plot their distribution:

fig, ax = plt.subplots()
ax.hist(spy["Adj Close"].pct_change().dropna(), bins=100)
ax.set_xlabel("Daily return")

Two practical rules. First, use enough bins — bins are the buckets the x-axis is chopped into, one bar per bucket. With thousands of daily observations, 100 bins is reasonable. The default of 10 smears the sharp peak and the tails into mush, hiding exactly the features you're looking for. Second, know what you're looking for: real daily equity returns show a tall narrow peak and long tails compared to a normal curve with the same standard deviation. Those few extreme bars far from the center are the fat tails from A2 — a histogram makes them physical.

For a sharper tail diagnostic, use the Q-Q plot you met in A2-03: your sample's quantiles plotted against a normal distribution's quantiles (a quantile is a cut-point — the 5% quantile is the value below which 5% of the data falls). If the data really were normal, the points would form a straight line. Market returns bend away at both ends — the pattern to memorize is "S-shaped tails = fatter than normal." The histogram gives intuition; the Q-Q plot gives evidence.

Comparing two series: rebase to 100

Plotting SPY (~$500) against a$40 stock on one axis tells you nothing — the cheap stock is a flat line pinned to the bottom. The fix is to rebase — rescale each series so both start at the same value, conventionally 100:

for col in ["SPY", "AAPL"]:
    rebased = 100 * prices[col] / prices[col].iloc[0]
    ax.plot(prices.index, rebased, label=col)
ax.legend()

Concrete: a stock that goes from $40 to$60 rebases to 100 → 150, and SPY going from $500 to$600 rebases to 100 → 120 — now you can see at a glance that the stock grew more. The y-axis reads directly as "growth of 100 invested at the start," and the two lines are comparable. This is the standard chart of fund factsheets and backtest reports — you'll draw it hundreds of times. One caveat: the picture depends on the start date. Rebasing at a peak vs. a trough can flip which asset "looks better," which brings us to —

The honesty rules

Charts persuade — including their own author. Backtest fraud (and self-deception) is usually committed with axes, not numbers:

1. Never truncate the y-axis to exaggerate. A price chart windowed to [495, 505] makes a 0.5% wiggle look like a crash. Use zero-based or clearly-labeled full-range axes for anything you'll show others.
2. No cherry-picked windows. A strategy chart that starts March 2009 (the exact post-crisis bottom) is an argument, not evidence. Show the longest history you have, drawdowns included — a drawdown is the fall from a previous peak.
3. Label everything. Title, y-axis units, whether returns are cumulative or per-period, whether the scale is log. An unlabeled chart can't be checked or challenged.
4. Same scale for compared things. Two panels with different y-ranges invite false conclusions; rebase to 100 on one panel instead.

Tufte's phrase for the goal is graphical integrity: the visual impression should be proportional to the numbers. Your future self, reviewing an old backtest chart, is the person you're protecting.

Practice locally (Jupyter)

This lesson has no in-browser exercise — the embedded runtime has NumPy only, no matplotlib. Run this in a local Jupyter notebook.

import matplotlib.pyplot as plt
import yfinance as yf

spy = yf.download("SPY", start="1995-01-01", auto_adjust=False)
# Note: with auto_adjust=False you get both Close and Adj Close.
# Use Adj Close — raw Close has fake jumps on dividend/split days.
px = spy["Adj Close"].squeeze()

fig, axes = plt.subplots(2, 2, figsize=(12, 8))
axes[0, 0].plot(px);                axes[0, 0].set_title("SPY linear")
axes[0, 1].plot(px);                axes[0, 1].set_yscale("log")
axes[0, 1].set_title("SPY log")
rets = px.pct_change().dropna()
axes[1, 0].hist(rets, bins=100);    axes[1, 0].set_title("Daily returns")
axes[1, 1].plot(100 * px / px.iloc[0])
axes[1, 1].set_title("Growth of 100")
fig.tight_layout()

What to look for: the linear panel makes pre-2010 history look flat. The log panel shows 2000–02 and 2008 as the huge drawdowns they were — both roughly −50% peak to trough — and on the log panel they finally look comparable to recent moves. The histogram should have a tall center spike and visible outlier bars beyond ±4 standard deviations. Then plot raw Close instead of Adj Close on the log panel. Raw Close ignores dividends, so it grows more slowly — note how that dividend drag makes the two series diverge steadily over 30 years.

Draw it from memory

Your generative activity: on paper, sketch the same 30-year rising price series twice — once on a linear axis, once on a log axis. Mark where an early-history 20% crash appears on each, and label which chart makes it visible. Then sketch the shape of a daily-returns histogram versus a normal curve, exaggerating the features that differ.