What 21,000 Games Taught Me About Thin Margins

I've been playing chess on chess.com since 2017. Over 21,084 games and nine years I noticed a pattern in myself: stretches where I win consistently and climb well past a 1,500 rating, followed by stretches where I fall back toward 1,100, sometimes beating players 300 points above me in one phase and losing to players 300 points below me in another. I wanted to know whether that swing is a real, detectable shift in how I actually play, or just noise I was telling myself a story about.

I had the data to check it properly: every one of those 21,084 games, with every move by both sides scored by a chess engine. This piece walks through the pipeline I built to answer the question rigorously, because defining “hot” and “cold” precisely turned out to be most of the work, and what I found once I had a definition that survived my own attempts to break it.

The pipeline

I built this in four stages, each producing a distinct data layer.

The dataset I ended up with carries these dimensions per game: rating (mine and my opponent's, at time of game), colour, time control, outcome, termination type, ply count, opening, average centipawn loss (ACPL), blunder/mistake/inaccuracy counts, and the minimum and maximum win probability I reached during the game. The per-move layer carries, for every ply: which side moved, the centipawn loss on that specific move, and the evaluation immediately after it. That per-move layer is what made the phase-level and timing-level findings possible; the per-game layer alone would not have supported them.

Average centipawn loss, or ACPL, is the metric that does most of the work in this piece. Before a move, the engine has an opinion on the position's value. After the move, it has another. The gap between those two values, when negative for me, is that move's cost, measured in hundredths of a pawn. Average that cost across a game and you get ACPL for the game: a single number for how far my actual moves diverged from the moves the engine would have played.

Defining the regime: a sensitivity analysis, not an assertion

The entire analysis hinges on one definition: which games count as “hot” and which count as “cold.” Get this wrong and every downstream number is an artefact of the definition, not a finding. I tried three approaches before settling on one, and the failure of the first two is worth showing, because a definition that survives my own attempts to break it is more trustworthy than one I just present without the failed attempts behind it.

Attempt one: a trailing rating delta. Label game i hot if my rating rose over the trailing 20 games, cold if it fell. This was my obvious first move, and it contains a subtle but serious flaw: the trailing window, naively computed, included the game's own outcome in its own label. A win pulled its own trailing delta upward, which then labelled that same win as “hot,” which inflated the measured hot/cold gap by roughly double versus the corrected version. The fix (shifting the window to only include prior games) is a one-line change, but it is exactly the kind of one-line change that would have silently doubled my headline number if I'd missed it. It's a leakage bug, the same structural mistake as using a loan's own default status as an input to a model predicting that default.

Attempt two: a local-trend classifier. Instead of a fixed window, classify each point by the short-run slope of a smoothed rating series. This fixes the leakage problem but introduces a new one: it fragments a real, multi-week regime into on-again-off-again slivers every time I had a single bad week inside an otherwise strong stretch. A hot period, correctly understood, should tolerate minor pullbacks without ending. A slope classifier doesn't know the difference between a real reversal and a pause.

Attempt three, and the one used throughout this piece: a ZigZag swing with a retracement tolerance. On the 10-game trailing average of rating, find local peaks and troughs using a prominence threshold: a swing only ends when the series gives back at least N points from the extreme reached so far. This is the same construct used in technical trend detection generally, and it matches what I actually experience: a hot period can contain small dips, as long as none of them are large enough to count as a genuine reversal. Swings smaller than a floor threshold are treated as noise and excluded from the hot/cold comparison entirely, rather than forced into one bucket or the other.

The result isn't sensitive to the exact tolerance I chose. The table below repeats the entire analysis at four different prominence values, from tight (15 points) to loose (60 points):

Tighter tolerances find more, shorter regimes and a slightly larger gap (some noise leaks into the hot/cold buckets); looser tolerances find fewer, longer regimes and a slightly smaller gap (some real signal gets averaged into the excluded “flat” middle). The direction and rough size of the effect is unchanged across a 4x range in the threshold. I used 25 points as the primary definition throughout the rest of this piece; it's the middle of the range I tested, not the value that produced the largest number.

One further correction is worth recording, because I'd rather show it than quietly fix it and move on. An early version of my period-tiling logic used an inclusive index range, which double-counted the exact boundary game between two adjacent regimes into both. I later built an audit (a regression test pinning every function to its expected output) that caught this: the fix moved roughly 0.5% of games between buckets and shifted my headline win-rate gap from 19.9 to 19.6 points. Immaterial to the conclusion, but exactly the kind of thing I'd rather a test caught than a reader.

The chart

Every game sits inside exactly one regime, or in the unshaded remainder if no swing there cleared the noise floor. 106 hot regimes and 95 cold regimes cleared the floor across my 12,493 Blitz games, covering 89% of the total.

Findings

Restricting to the 11,119 Blitz games that fall inside a defined regime of mine:

The ACPL gap (6.2 points) and the blunder-rate gap (0.77 percentage points) both look modest in isolation. The win-rate gap they compound into is not: 19.6 percentage points, roughly evenly split between me playing worse and being punished harder for it. The collapse rate (blowing a near-certain win) is the single most sensitive metric to my regime state, running roughly twice as high cold as hot.

I checked whether this holds by rating band, by colour (White/Black), and by opponent strength, and it does, at broadly the same magnitude in every cut:

The finding doesn't concentrate in one lucky rating band. It also doesn't disappear once I control for opponent strength directly, rather than relying on the rating system to hold it roughly constant. Splitting my games by the rating gap to the specific opponent I faced, and comparing my realised win rate to the win rate that gap alone would predict:

In every band, my cold win rate sits below what the rating gap alone predicts and my hot win rate sits above it, by a wide and fairly stable margin. The gap is largest, in relative terms, exactly where it's most diagnostic: against opponents 100 or more points weaker than me, I lose an upset roughly twice as often cold as hot (7.9% of those games lost cold, versus 4.5% hot).

Where in the game the gap lives

Because I had per-move data, not just per-game averages, I could ask one further question the per-game layer alone couldn't answer: which phase of the game carries the cost. I classified each move as opening (first 20 ply), middlegame, or endgame (combined non-pawn material on the board at or below a fixed threshold), using the actual board state rather than a fixed move count.

Three-quarters of every point I've ever lost was lost in the middlegame, both because it's the largest share of my moves and because my per-move accuracy there is worst by a wide margin. The hot/cold gap concentrates in the same place: my opening ACPL barely moves between regimes (1.6-point gap, largely memorised or forcing lines), while my middlegame gap is nearly five times larger (7.6 points). The same phase carries my largest baseline cost and my largest regime sensitivity, and I don't think that's a coincidence: it's the phase that most depends on live calculation rather than preparation, which is exactly what a shift in decision quality would hit hardest.

What does not explain it

Before I accepted “my decision quality shifted” as the explanation, I tested and ruled out nine specific behavioural mechanisms, using move-timestamp data extracted directly from my game records:

None of the mechanisms I could think of that would normally explain a performance gap through habit or circumstance survived contact with the data. What's left is a broad, uniform shift in my decision quality that shows up identically across pace, time of day, session structure, opening choice, and pressure. That's consistent with a real underlying state of mine, rather than a habit I could just correct, though a dataset like this can only observe the downstream fingerprint on my move quality; it can't see the state itself.

Thin margins

I spend my working hours thinking about lending portfolios, and the parallel here is hard to miss. A portfolio's profitability sits in a comparably thin margin: the lending rate, minus the cost of capital, minus the operating cost of running the book. None of those three components move by much month to month under normal conditions. And yet realised portfolio outcomes are not stable month to month, in the same way my rating is not a straight line despite no single game of mine looking dramatically different from the last.

My chess finding maps onto this directly. A 6-point gap in my average move quality and a 1-point gap in my blunder rate look like rounding error next to a 20-point swing in win rate, but they turned out to be the entire explanation for it, compounded over hundreds of decisions per game and thousands of games. The equivalent question for a credit book isn't whether the average month is good; averages are carried by outliers there just as often as anywhere else. It's whether discipline holds during the bad stretch: whether loss recognition stays prompt when the portfolio turns, and whether operating cost stays controlled regardless of how the book is performing. The stretch where a portfolio is under pressure is where the real variance in realised margin gets decided, the same way my own middlegame, not my opening or my endgame, carried three-quarters of every point I've ever lost. Consistency under pressure is the actual skill being measured, in both domains; average performance in calm conditions tells you comparatively little about it.