We measured every substitution, every shape change and every resulting point across three seasons. Coaches turn out to be strikingly consistent in what they do, and indistinguishable in what it is worth.
The same question, asked two ways, gives two very different answers.
Ask what a coach does and the data is emphatic. Split any manager's matches at random into two halves and his habits reproduce almost perfectly: how many change windows he uses, whether he reshapes the side mid-match, how quickly he reacts to conceding. Nuno Espírito Santo changed formation in 63% of his matches; Sébastien Pocognoli did it in 4%. That gap is real, stable and would survive another season.
Ask what it is worth in points and the signal vanishes. Every outcome measure we built: points gained after the first change, the half-time swing, the value of each individual intervention, all fail to reproduce itself across two halves of the same manager's own matches. Not weakly. At zero.
A manager's in-game record over one season tells you essentially nothing about his in-game record over the next. His habits tell you almost everything about his next set of habits.
Split-half reliability: cut each manager's matches in two at random, measure both halves, and see whether they agree. A measure that carries real signal correlates with itself. One that is noise does not.
Does the measure reproduce itself?
Correlation between two random halves of the same manager's matches · 66 managers with 20+ matches
The raw picture you asked for: the score state at the first change, and how those matches finished.
Score state at the first change → final result
4,134 team-matches · first change made on average in the 57th minute
This is the part that holds up. Sort by any column; click a row for the full profile. 54 managers with 30 or more matches.
Points gained after the first change, against what the score, the clock, the venue and the two teams' strengths already predicted.
In-game residual per match, 95% interval
Sorted best to worst · a bar crossing the zero line is indistinguishable from average
Individual coaches carry too few matches to rank. The decisions do not: there are 11,675 change windows. Each comparison below is made between teams in the same score state, at the same stage, with the same strength gap, so the only difference is what the coach did next.
Effect on expected points
Matched comparison · 95% interval · positive means the decision gained points
One result did look like a manager effect, and it is worth showing why it isn't.
Rating teams against squad market value rather than form, managers appear to separate (correlation 0.25 across halves). But grouping the identical data by club separates far more strongly (0.52). Some clubs simply beat their valuation year after year, and because most managers only ever work at one club in this window, the club's edge is wearing the manager's name.
The clean test is a change of manager at the same club. For every pair of managers who shared a club, we measured the gap between them twice, from independent halves of their matches. If one really is better, both measurements should agree.
Does the manager-to-manager gap reproduce?
48 manager pairs across 25 clubs · 95% interval, resampling clubs
A null result bounds an effect, it does not erase it. The data rules out any persistent in-game edge larger than 0.13 points per match, about five points across a 38-game season. A real edge of two or three points a season could be sitting underneath this and we would never see it in three seasons. That is worth a great deal in a relegation fight and is invisible to this method.
Rating teams by Elo makes in-game skill disappear by construction: Elo is built from results, so a coach who is genuinely good raises his own bar until his residual returns to zero. We rebuilt the whole measure on squad market value, which carries no such feedback. That is what exposed the club effect in Figure 5, and it is the single most important methodological point here.
The second trap is the sub premium. Substitutes out-produce starters by roughly 18% per minute on the ball simply because they arrive fresh into a stretched game. Comparing an incoming player with the man he replaced without that correction makes every substitution look like a masterstroke. Baseline-adjusted, the median manager's incoming players are worth 0.000 more per minute than the ones they replace.
Every played match in the Premier League and Belgian Pro League from 2023/24 to 2025/26. Substitutions were reconstructed from per-player position timelines and clustered into change windows; movements within 60 seconds count as one decision, which the data separates cleanly (43% of gaps fall under a minute, the rest average over six). Manager spells came from Transfermarkt staff histories and matched to 4,145 of 4,156 team-matches.
The backbone is an expected-points model over score difference, time remaining, man advantage, venue and team strength, blending a logistic model on engineered features with a gradient-boosted model on binned strength. It is trained with grouped cross-validation so no match is ever scored by a model that has seen it: out-of-fold log loss 0.742 against 1.085 for the base rates, and it is calibrated to within 0.07 points across every score-state and time cell, and within 0.002 across home and away.
Known limits. Shot quality and phase-of-play tags only exist from 2025/26, so goals and on-ball value carry the impact measure across all three seasons. Eleven matches whose goal events did not reconstruct the final score were dropped, along with one carrying a corrupt match clock. Managerial change is not random (clubs sack after bad runs), which if anything should exaggerate the differences between managers at one club, and we still find none.