Opponent-Adjusted Evaluation of NFL Pass Blocking and Pass Rushing Performance

This paper introduces an opponent-adjusted framework using ridge-regularized Bradley-Terry models on NFL tracking data to evaluate pass blocking and pass rushing performance, demonstrating that these interaction-level models improve predictive accuracy over baselines and align well with expert All-Pro selections.

The Gist

Imagine the NFL offensive and defensive lines as a massive, high-stakes game of Rock, Paper, Scissors played inside a tiny, chaotic box.

For years, trying to figure out who is actually the best "Rock" (the blocker) or the best "Paper" (the rusher) has been incredibly hard. Why? Because the scoreboard doesn't tell the whole story.

• If a quarterback gets sacked, was it because the rusher was a superstar, or because the quarterback held the ball too long?
• If a blocker holds his ground for 3 seconds, was it because he's a genius, or because the rusher was having a bad day?

This paper by researchers from the University of Pennsylvania tries to solve that puzzle. They built a new way to grade these players that adjusts for who they played against, using a massive amount of data from the 2021 season.

Here is the breakdown of their method, using simple analogies:

1. The Data: A "Play-by-Play" Video Game

The researchers didn't just look at the final score (sacks or no sacks). They used Hudl tracking data, which is like a high-speed video game recording the exact position of every player 10 times a second.

They broke the game down into tiny 1-on-1 duels.

• The Setup: Every time a pass rusher (Defender) tries to get past a blocker (Offense), that's a "matchup."
• The Twist: Sometimes, two defenders gang up on one blocker (a "double team"). The researchers made sure to note this, just like a referee noting that a player had help.

2. The Two "Scorecards"

The team created two different ways to grade these duels, like having two different video game modes:

Mode A: The "2.5-Second Timer" (Win/Loss)

• The Rule: If the rusher gets within a certain distance of the quarterback in 2.5 seconds, the rusher "wins." If the blocker holds them off, the blocker "wins."
• The Analogy: It's like a sprint. Did the runner cross the finish line before the timer beeped? Simple, binary, and easy to understand.

Mode B: The "Severity Scale" (Loss, Win, Hit, Sack)

• The Rule: This is more nuanced. Not all wins are created equal.
  • Loss: The rusher got pushed back.
  • Win: The rusher got close, but didn't touch the QB.
  • Hit: The rusher slapped the QB (but didn't knock him down).
  • Sack: The rusher tackled the QB.
• The Analogy: Think of this like a video game health bar. A "Win" chips off a little health. A "Hit" chips off more. A "Sack" is a critical hit that knocks the player out. This model values the impact of the play, not just whether the rusher got close.

3. The Secret Sauce: The "Opponent-Adjusted" Rating

This is the most important part. The researchers used a statistical tool called Bradley-Terry models (think of it as a super-smart Elo rating system, like in chess or video games).

• How it works: If a rusher beats a "weak" blocker, they get a small boost. If they beat a "strong" blocker, they get a huge boost. Conversely, if a blocker holds off a "superstar" rusher, they get a massive credit, even if they lost the play.
• The Result: The system separates the player's skill from the quality of their opponent. It answers the question: "How good is this player, regardless of who they are fighting today?"

4. Did It Work? (The Proof)

The researchers tested their new system against the old ways of judging players (like just counting sacks or win rates).

• The Prediction Test: They tried to predict the outcome of future plays. Their new system was slightly better at predicting who would win than the old methods. It wasn't a magic jump, but it was a consistent, reliable improvement.
• The "Expert" Test: They compared their rankings to the AP All-Pro team (the list of players voted as the best by sports journalists).
  • The Result: Their "Severity" model (Mode B) lined up almost perfectly with the experts' choices. It correctly identified the elite players better than the simple "Win/Loss" model.

5. Why This Matters

Imagine you are a coach trying to build a team.

• Old Way: "Player A has 10 sacks, so he's great." (But maybe he only played against bad offensive lines).
• New Way: "Player B has 5 sacks, but he beat the three best offensive lines in the league, and our model says he's the most efficient rusher."

The Bottom Line

This paper didn't just count sacks; it built a fair referee for the trenches. By looking at every single 1-on-1 battle and adjusting for who was playing whom, they created a clearer picture of who the true stars are.

• The "Win/Loss" model is like a stopwatch: Did you beat the clock?
• The "Severity" model is like a damage meter: How much trouble did you cause?

The study concludes that while no model is perfect, this "opponent-adjusted" approach gives us the most honest, transparent, and fair way to evaluate the unsung heroes and villains of the NFL line of scrimmage.

Technical Summary

1. Problem Statement

Evaluating individual performance for NFL offensive linemen (blockers) and defensive linemen (pass rushers) is notoriously difficult due to three primary factors:

• Sparse Outcomes: Direct box-score statistics (sacks, hits) are rare events.
• Opponent Dependence: Performance is heavily influenced by the quality of the opposing player.
• Contextual Noise: Outcomes are mediated by surrounding factors such as quarterback time-to-throw, coverage schemes, and game situation.

Existing metrics, such as ESPN's Pass Block Win Rate (PBWR) or tracking-based measures like STRAIN, capture more signal than raw sacks but fail to jointly estimate the latent ability of both the blocker and the rusher while adjusting for opponent strength. This paper aims to construct an opponent-adjusted framework that separates individual ability from opponent quality and team context.

2. Methodology

Data and Outcome Construction

• Data Source: 2021 NFL regular-season tracking data (Hudl) at 10 Hz, covering 266 games, 33,283 pass plays, 620 rushers, and 348 blockers.
• Unit of Analysis: A blocker–rusher interaction. A single play can generate multiple interactions if multiple engagements occur simultaneously.
• Covariates: A binary double-team indicator is included to account for overlapping help (multiple blockers on one rusher).
• Outcome Definitions (from the rusher's perspective):
  1. Binary Win/Loss: Defined by the 2.5-second rule. A "win" occurs if the rusher gets closer to the quarterback than the blocker within 2.5 seconds of the snap.
  2. Four-Class Severity: A hierarchical multinomial outcome:
     • Sack (Most severe)
     • Hit (QB contact, not a sack)
     • Win (Pressure without contact)
     • Loss (No pressure)
  • Severity Mapping: To create scalar summaries, the four classes are mapped to a 0–1 scale based on Expected Points Added (EPA) benchmarks: Loss (0), Win (0.10), Hit (0.20), Sack (1.00).

Modeling Framework

The authors employ Ridge-Regularized Bradley–Terry (BT) models, a paired-comparison framework where every interaction is treated as a head-to-head contest. Two separate models are fitted:

1. Binary Win/Loss Model:

   • Uses a logistic regression form: $\text{logit}(P(\text{Win})) = \alpha + r_i - b_j + \delta D_t$
   • $r_i$: Rusher ability; $b_j$: Blocker ability (negative sign); $D_t$: Double-team indicator.
   • Regularization: Ridge penalty ($\lambda \|\theta\|_2^2$) is applied to shrink player effects toward zero, stabilizing estimates for players with sparse exposure.

2. Severity Model:

   • Uses a multinomial BT model to predict the probability of each of the four outcome classes.
   • After fitting, predicted probabilities are converted to an Expected Severity Score using the EPA-based weights.

Validation Strategy

• Split: An ordered 80/20 holdout split based on game_id, play_id, and event_game_index (Train: 122,510; Test: 30,628).
• Baselines:
  • Global Baseline: Uses the overall training-set frequency (ignores player identity).
  • Matchup Baseline: Uses smoothed empirical win rates/severity frequencies for specific players (rusher and blocker) without a shared latent rating model. This serves as a strong, competitive baseline.
• Uncertainty: Estimated via game-level bootstrap resampling (1,000 iterations) and weekly path bootstraps (100 iterations) to assess stability over the season.

3. Key Contributions

1. Joint Opponent-Adjusted Framework: A system that evaluates blockers and rushers simultaneously, preserving role-specific interpretation while accounting for opponent strength.
2. Dual-Model Approach: Separate ridge-regularized BT models for binary outcomes (win/loss) and multiclass severity, allowing tailored objectives and interpretations.
3. Rigorous Validation:
   • Internal validation against strong matchup baselines using log-loss.
   • External validation against 2021 AP All-Pro selections using AUC and Enrichment@K.
   • Longitudinal uncertainty summaries via weekly bootstrap paths.
4. Interpretability: The framework provides transparent ratings that can be used for scouting and personnel evaluation, distinct from "black box" machine learning approaches.

4. Results

Internal Predictive Performance

• Both models outperformed global and matchup baselines on the holdout set.
• Log-Loss Improvements:
  • Win Model: Improved over global baseline by ~0.68% and over matchup baseline by ~0.14%.
  • Severity Model: Improved over global baseline by ~0.77% and over matchup baseline by ~0.15% (directionally positive but less statistically certain against the matchup baseline).
• Stability: Bootstrap intervals confirmed that gains over the global baseline are robust. Gains over the matchup baseline are positive but narrower, indicating the difficulty of beating player-specific historical frequencies.

External Validation (All-Pro Alignment)

The models were tested against 2021 AP All-Pro selections (First Team and First+Second Team).

• AUC (Area Under Curve): The Severity Model showed superior alignment with expert selections in 3 out of 4 role/accolade slices.
  • Example: For Blockers (First Team), the Severity model achieved an AUC of 0.854 compared to 0.783 for the raw baseline.
• Enrichment@K: The Severity model demonstrated the highest enrichment, particularly for rushers, suggesting it better identifies elite players who generate high-impact outcomes (hits/sacks) rather than just binary wins.

Leaderboards

• Win/Loss Model: Top rushers included Adam Gotsis and Josh Allen; top blockers included Erik McCoy.
• Severity Model: Top rushers included Robert Quinn and T.J. Watt; top blockers included Joe Thuney.
• The leaderboards were directionally similar for elite rushers but diverged for blockers, reflecting the Severity model's heavier weighting on high-impact outcomes (sacks/hits).

5. Significance and Conclusion

This paper demonstrates that Ridge-Regularized Bradley–Terry models provide a stable, interpretable, and opponent-adjusted framework for evaluating NFL line play.

• Practical Value: The models successfully separate player ability from opponent quality, offering a statistical foundation for scouting that improves upon simple win-rate metrics.
• Nuance: The Severity Model is highlighted as particularly valuable because it aligns more closely with expert recognition (All-Pro selections) by capturing the magnitude of disruption (hits/sacks) rather than just the binary occurrence of pressure.
• Limitations & Future Work: The authors note that current labels rely on distance-to-QB rules which may miss functional pressure quality. Future iterations could incorporate pocket geometry, quarterback decision constraints, and hierarchical shrinkage to model teammate effects and position families.

In summary, the study validates that modeling richer outcome structures (severity) alongside opponent adjustment yields more accurate and actionable evaluations of NFL linemen than traditional aggregate statistics.