Pairs trading and cointegration

Two companies in the same business often move together: two oil producers, two banks, a stock and its index. Pairs trading bets that when they drift apart, they will come back together: buy the one that fell behind, short the one that ran ahead. The idea is sound only when the link between them is real, and telling a real link from a coincidence is what this lesson is about.

Correlated is not enough

Two prices can be highly correlated in their daily moves and still drift apart for ever. What a pairs trade needs is stronger: some combination of them that doesn't wander. Each price on its own is a random walk; the pair is cointegrated if the spread

is mean-reverting. A common model is the Ornstein–Uhlenbeck process, whose discrete version is $s_t = \phi\, s_{t-1} + \varepsilon_t$ with $0 < \phi < 1$: each day a fraction $1 - \phi$ of the deviation disappears, so a deviation halves in $\ln 2 / (-\ln\phi)$ days, the half-life.

If instead $\phi = 1$, the spread is itself a random walk. It can look stable for a while by chance, but nothing pulls it back.

The trade

1. Form. On a past period, estimate $\beta$ and $\alpha$ by regressing $y$ on $x$, and the spread's mean and standard deviation.
2. Trade. In the following period, compute the spread's $z$-score each day. Above $+2$, short the spread (short $y$, buy $\beta$ of $x$); below $-2$, buy it; close the position when the spread crosses its mean.
3. Lag. Decide at the close, trade from the next day, as in every honest backtest (the look-ahead lesson).

Below, the left half of the top panel is the formation period; the trade runs on the right half. Switch between a cointegrated pair and a look-alike, two independent random walks that the formation period happens to make look related:

For the cointegrated pair the spread crosses $\pm 2\sigma$ and snaps back over and over, and each round trip adds to the P&L. For the look-alike, the spread drifts away from its formation-period mean and may never come back: positions sit open for long stretches, and the P&L goes nowhere or down.

Key idea. A pairs trade is a bet on mean reversion of the spread, not on correlation of the prices. Without a real economic link that keeps the spread stable, the strategy is a bet that a random walk will return to where it started.

Testing for cointegration

The obvious test is to regress the daily change of the spread on its previous level, $\Delta s_t = a + b\, s_{t-1} + \varepsilon_t$, and ask whether $b$ is negative (a pull back to the mean). The half-life follows from $b$ as $-\ln 2 / \ln(1 + b)$.

There is a catch. Even for a pure random walk, the estimated $b$ is biased negative, and the usual $t$-statistic doesn't have the usual distribution. A random walk observed for 500 days will often look mean-reverting. The Dickey–Fuller test (and the Engle–Granger procedure for pairs) uses critical values that allow for this: roughly $-2.9$ at the 5% level rather than $-1.65$, and more negative still when $\beta$ was estimated from the same data.

The simulation below shows the bias. The look-alike pairs have no mean reversion at all, but the median estimated half-life is 46 days, a finite number that could easily be mistaken for a slow real effect.

In code

import numpy as np

def simulate(rng, cointegrated, days=1000, beta=1.2, vol=0.015, spread_vol=0.004, half_life=10):
    keep = 2 ** (-1 / half_life)  # daily fraction of a spread deviation that survives
    x = np.log(50) + np.cumsum(vol * rng.standard_normal(days))
    s = np.zeros(days)
    for t in range(1, days):
        s[t] = (keep if cointegrated else 1.0) * s[t - 1] + spread_vol * rng.standard_normal()
    return x, beta * x + 0.5 + s

def backtest(x, y, formation=500, entry=2.0):
    beta, alpha = np.polyfit(x[:formation], y[:formation], 1)  # hedge ratio from the past only
    spread = y - beta * x - alpha
    mean, sd = spread[:formation].mean(), spread[:formation].std(ddof=1)
    lagged, change = spread[: formation - 1], np.diff(spread[:formation])
    b = np.polyfit(lagged, change, 1)[0]
    half_life = -np.log(2) / np.log(1 + b) if -1 < b < 0 else np.inf
    position, pnl = 0, 0.0
    for t in range(formation, len(x)):
        pnl += position * (spread[t] - spread[t - 1])  # yesterday's decision earns today's move
        z = (spread[t] - mean) / sd
        if position == 0 and abs(z) > entry:
            position = -np.sign(z)  # bet on the spread returning to its mean
        elif position * z >= 0 and position != 0:
            position = 0  # crossed back through the mean: take profit
    return pnl, half_life

rng = np.random.default_rng(1)
for cointegrated in (True, False):
    results = [backtest(*simulate(rng, cointegrated)) for _ in range(200)]
    pnl = np.array([r[0] for r in results])
    half_lives = np.array([r[1] for r in results])
    label = "cointegrated" if cointegrated else "look-alike  "
    print(f"{label}: mean P&L {pnl.mean():+.3f}, profitable {np.mean(pnl > 0):.0%}, "
          f"median estimated half-life {np.median(half_lives):.0f} days")

Over 200 simulated pairs of each kind, the cointegrated pairs average a P&L of $+0.125$ (in log points of the spread, roughly 12.5% of the position) and every one of them is profitable, with a median estimated half-life of 9 days against a true 10. The look-alikes average $-0.006$, are profitable exactly half the time, and show the misleading 46-day half-life.

Where this shows up in quant work

• Statistical arbitrage. Pairs trading generalises to baskets: many stocks hedged against factors or each other, trading the residuals' mean reversion. It is one of the classic quant equity strategies.
• Relative value. Futures across expiries, a bond against its futures contract, an ETF against its holdings, or the same asset on two exchanges: when an arbitrage or economic link holds the spread, mean reversion is real.
• Risk. Relationships break. Mergers, regulation or a change in one company's business can end cointegration, and a spread that "always came back" stops doing so. Stop-losses and limits on holding time protect against that.

Exercises

• In the code, use a 250-day formation period instead of 500. How do the cointegrated results and the look-alike half-lives change?
• Add a transaction cost of 0.001 (in log points) each time a position is opened or closed. Which entry threshold, 1.5, 2 or 2.5, now performs best for the cointegrated pairs?
• Compute the Dickey–Fuller $t$-statistic for $b$ in the formation period of each simulated pair. What fraction of look-alikes have $t < -2.9$? What fraction of cointegrated pairs?

Key takeaways

• Pairs trading needs a mean-reverting spread (cointegration), not merely correlated prices.
• Fit the hedge ratio and thresholds on past data, trade later data, and lag every decision.
• The half-life measures how fast the spread reverts; random walks often look mean-reverting in sample, so use Dickey–Fuller critical values, not normal ones.
• Economic links can break; protect a pairs book against spreads that stop coming back.