ABC News
It’s Hard To Measure NFL Matchups, So We Built A Better Metric

The NFL is a league of matchups, and no matchup is more crucial to wins and losses than the battle between a quarterback and the pass defense. That’s important for more than just the coaches calling X’s and O’s: Each week, the media, betting markets and fantasy players all make predictions about which teams (and which players) will best take advantage of their opponent’s weaknesses.

Because football is such a complex sport, quantitative analysts haven’t found a great way to capture all the relevant information for each offensive and defensive passing unit in a way that lets us make reasonable predictions about future outcomes. Perhaps the biggest flaw of most approaches to strength-of-schedule and matchup analyses, for example, is that they assume that because a defense has or has not allowed yards, completions and points generally, we should expect that performance to continue against a future opponent. The problem: That isn’t really true. Defenses have weaknesses, but to exploit them, an offense first needs to know they exist. Then it must possess the personnel needed to actually take advantage. In the NFL, neither of those circumstances is a given.

[How Do College Football Dynasties Fall?]

One approach to quantifying these matchups is to look at where on the field a quarterback tends to complete more passes than average and where a defense tends to allow more completions than average. We’ve published heat maps like these before on FiveThirtyEight — both for defenses and for quarterbacks. (We think they’re pretty cool.) If we use some fancy math,1 we can roll these heat maps into a single, convenient number that describes the similarity between where a QB’s passes are completed and where a defense allows passes in a given matchup, with higher numbers indicating a better matchup for the quarterback. These similarity scores then give us an idea of which QBs have faced a more difficult slate and who might have a challenge ahead in the upcoming week.

Like most quantitative measures, similarity scores are worthless without a decent sample size. To ensure we have enough pass attempts to make good comparisons, and to eliminate early-season variance, we chose to calculate our scores only for matchups that occurred from Week 5 onward. We then averaged the scores for each QB who played in at least three games over the seven-week period from Week 5 through Week 11 to see who’s had the toughest schedule, ranking them from the most challenging schedule to the most favorable.

##### Tagovailoa started his career with tough matchups

Average similarity score for current starting quarterbacks with at least three qualifying appearances* in Weeks 5 through 11 of the 2020 season, ranked from toughest to easiest matchups

Quarterback Team appearances Avg. sim score
Tua Tagovailoa Miami Dolphins 3 63.1
Russell Wilson Seattle Seahawks 6 64.7
Matthew Stafford Detroit Lions 6 67.5
Lamar Jackson Baltimore Ravens 6 69.4
Kirk Cousins Minnesota Vikings 5 71.9
Nick Mullens San Francisco 49ers 3 75.1
Kyler Murray Arizona Cardinals 6 76.3
Ryan Tannehill Tennessee Titans 7 76.5
Justin Herbert Los Angeles Chargers 6 77.1
Joe Flacco New York Jets 4 78.2
Aaron Rodgers Green Bay Packers 6 79.8
Alex Smith Washington Football Team 3 80.3
Drew Lock Denver Broncos 6 80.3
Tom Brady Tampa Bay Buccaneers 6 80.5
Cam Newton New England Patriots 5 81.2
Matt Ryan Atlanta Falcons 6 81.4
Philip Rivers Indianapolis Colts 6 82.1
Josh Allen Buffalo Bills 6 83.3
Daniel Jones New York Giants 6 83.4
Deshaun Watson Houston Texans 6 83.7
Carson Wentz Philadelphia Eagles 6 83.8
Jared Goff Los Angeles Rams 5 84.2
Ben Roethlisberger Pittsburgh Steelers 7 85.5
Baker Mayfield Cleveland Browns 6 87.4
Patrick Mahomes Kansas City Chiefs 6 92.0
Nick Foles Chicago Bears 6 92.0
Derek Carr Las Vegas Raiders 6 92.9

A higher score indicates that a QB tends to complete more passes in areas where the defense he was facing also tends to allow more passes. Data includes games through Nov. 22.

*Qualifying appearances are those for which a similarity score can be calculated; quarterbacks must have at least 15 previous pass attempts to calculate a score.

Source: ESPN Stats & Information Group

Dolphins rookie quarterback Tua Tagovailoa — who was benched for poor performance on Sunday by Miami head coach Brian Flores — is at the top of our list, indicating that he’s had a tough set of matchups to begin his NFL career. Meanwhile, Derek Carr — who is having the best season of his career — has had the easiest slate of matchups since Week 5.

Tagovailoa’s situation, though, makes him difficult to score. With only two passing attempts in his pro career going into his first start, which came against the Los Angeles Rams in Week 8, his sample size was too small for us to calculate a score for that first start. In his second start, Tagovailoa faced Arizona in Week 9, leading the Dolphins to a 34-31 win while throwing for 8.9 yards per attempt. The similarity score we calculated for that matchup was low — 35.4 — but it was still based on a small sample: 24 career attempts. By Week 10, which he entered with 52 career attempts, Tagovailoa’s sim score rose to 67.2 against the Chargers, a game in which he threw for 169 yards and two touchdowns. This past week against a Denver team with only three wins, his score was 86.7, meaning the places where the defense tends to allow completions lined up well with the places where he tends to make successful throws. That might explain his coaches’ frustration with his performance — this was a game in which he probably should have played well.

Seahawks quarterback Russell Wilson has had the second-toughest midseason schedule among current starting QBs. In Week 11, Wilson’s similarity to Arizona’s pass defense was just 67.9. Wilson led the Seahawks to a win, but he ended up throwing for only 197 yards against a Cardinals team that had been allowing around 250 yards per game through the air. If Wilson’s upcoming opponents turn out to be more accommodating, his numbers — and his case for league MVP — could improve.

Lions quarterback Matthew Stafford, who has the third-lowest average sim score this season, had his toughest game in Week 9 against the Minnesota Vikings — a matchup with a sim score of 42.5. The heat map for Stafford heading into the game showed that while the Viking pass defense was weak over the middle and up the hashes, so was Stafford, making him ill-equipped to take advantage of their vulnerabilities. Meanwhile the Vikings were better than average at taking away deeper sideline passes, spots where Stafford is usually strong.

Stafford finished the Vikings game 23-of-32 for 211 yards with one touchdown and two interceptions. His adjusted yards per attempt (a stat that gives extra credit for touchdowns and penalizes for interceptions) for the game was 4.41, his worst performance of the year so far.

[The Indianapolis Colts Finally Built A Defensive Monster]

We’ve used our similarity scores to look back at a player’s performance, but let’s shift ahead to try to spot matchup problems in this weekend’s games.

##### Who might have a tough matchup in Week 12?

Similarity scores for Week 12 quarterbacks* and the defenses they will face, sorted from toughest to easiest matchups

quarterback vs. opponent opponent record Sim score
Sam Darnold Miami Dolphins 6-4 49.0
P.J. Walker Minnesota Vikings 4-6 54.7
Aaron Rodgers Chicago Bears 5-5 59.8
Taysom Hill Denver Broncos 4-6 60.9
Nick Foles Green Bay Packers 7-3 63.9
Andy Dalton Washington Football Team 3-7 72.5
Cam Newton Arizona Cardinals 6-4 76.3
Nick Mullens Los Angeles Rams 7-3 77.7
Alex Smith Dallas Cowboys 3-7 78.4
Teddy Bridgewater Minnesota Vikings 4-6 80.0
Russell Wilson Philadelphia Eagles 3-6 81.6
Lamar Jackson Pittsburgh Steelers 10-0 81.6
Ryan Tannehill Indianapolis Colts 7-3 83.7
Josh Allen Los Angeles Chargers 3-7 85.7
Matthew Stafford Houston Texans 3-7 85.7
Justin Herbert Buffalo Bills 7-3 86.2
Derek Carr Atlanta Falcons 3-7 87.0
Ben Roethlisberger Baltimore Ravens 6-4 87.7
Kyler Murray New England Patriots 4-6 88.3
Jared Goff San Francisco 49ers 4-6 88.5
Joe Flacco Miami Dolphins 6-4 89.2
Baker Mayfield Jacksonville Jaguars 1-9 90.2
Deshaun Watson Detroit Lions 4-6 90.3
Tua Tagovailoa New York Jets 0-10 90.6
Kirk Cousins Carolina Panthers 4-7 92.6
Matt Ryan Las Vegas Raiders 6-4 92.9
Philip Rivers Tennessee Titans 7-3 93.9
Tom Brady Kansas City Chiefs 9-1 94.0
Patrick Mahomes Tampa Bay Buccaneers 7-4 94.6
Drew Lock New Orleans Saints 8-2 94.9
Daniel Jones Cincinnati Bengals 2-7 95.2
Carson Wentz Seattle Seahawks 7-3 95.8
Jake Luton Cleveland Browns 7-3 98.9

A higher score indicates that a QB tends to complete more passes in areas where the defense he’s facing also tends to allow more passes. Similarity scores based on games through Nov. 22. Quarterbacks need a minimum of 15 previous pass attempts to qualify for a similarity score.

*Including backups if the regular starters are questionable for this week’s games.

Source: ESPN Stats & Information Group

We might have cause for concern about Aaron Rodgers against Chicago: A sim score of 59.8 will be Rodgers’s toughest matchup in the period we’re measuring. Taysom Hill and P.J. Walker also feature low sim scores, but both played decently in their first NFL starts, and sample size issues similar to Tagovailoa’s definitely apply here, so maybe we should be more bullish about their matchups. But in New York, even if Sam Darnold is healthy, the Jets should probably start Joe Flacco against Miami. Flacco’s matchup against the Dolphins is over 40 points higher than Darnold’s, which should make coach Adam Gase’s choice an easy one. Unless, of course, the Jets’ goal is to remain winless for the rest of the year, in which case starting Darnold is absolutely optimal.

##### FiveThirtyEight’s NFL Elo ratings

How each team ranks through Week 11 of the 2020 season, according to our quarterback-adjusted predictions

Chance To …
Rk Team Starting QB QB Rk* Elo Rating Proj. Wins Make Playoffs Win Division Win SB
1 Chiefs Mahomes 1 1744 13.6 >99% 98% 26%
2 Steelers Roethlisberger 6 1694 14.4 >99 96 18
3 Packers Rodgers 4 1619 11.3 96 88 7
4 Colts RiversÐ²ÑÑ 15 1605 10.7 77 56 3
5 Rams Goff 12 1603 10.8 94 45 6
6 Ravens Jackson 8 1600 10.2 72 2 2
7 Seahawks Wilson 5 1599 11.2 96 43 5
8 Bills Allen 7 1595 10.6 89 84 3
9 Buccaneers Brady 13 1592 10.1 85 14 4
10 Titans Tannehill 11 1569 10.6 82 43 2
11 Raiders Carr 9 1561 9.8 67 2 1
12 Cardinals Murray 2 1540 9.2 63 12 1
13 Browns Mayfield 24 1526 10.4 69 2 1
14 Vikings Cousins 18 1522 7.4 20 2
15 Patriots Newton 16 1511 7.1 8 3
16 49ers Mullens 29 1507 7.0 11
17 Bears FolesÐ²ÑÑ 23 1487 8.0 34 10
18 Dolphins Tagovailoa 25 1481 8.9 33 13
19 Saints Hill 32 1478 11.5 99 86 19
20 Broncos Lock 28 1475 6.4 2
21 Texans Watson 3 1473 5.9 1
22 Falcons Ryan 17 1451 5.1 1
23 Chargers Herbert 10 1451 5.2
24 Eagles Wentz 26 1429 5.8 35 35
25 Washington Smith 21 1414 5.2 18 18
26 Giants Jones 20 1408 5.4 25 25
27 Panthers BridgewaterÐ²ÑÑ 14 1407 5.9 1
28 Lions Stafford 19 1365 5.7 2
29 Cowboys Dalton 27 1365 5.4 23 23
30 Jets DarnoldÐ²ÑÑ 30 1293 1.2
31 Bengals Finley 31 1287 4.0
32 Jaguars MinshewÐ²ÑÑ 22 1214 2.1

*Ranking among Week 12 starters, according to our QB Elo ratings.

Ð²ÑÑ Starter is currently injured and may not play.

†Starter could be replaced for performance reasons.

Simulations as of Nov. 24, 2020.

Source: ESPN

Looking ahead: AFC South showdowns haven’t always been synonymous with “epic action,” but the best game of Week 12 is probably Sunday’s Indianapolis-Tennessee rematch, with the division lead on the line. Philip Rivers and the Colts are fresh off a comeback victory over the Packers and have won four of their past five contests — including one against the Titans — elevating themselves into the top five of this week’s Elo rankings. Their strong defense will look to slow down Ryan Tannehill, Derrick Henry and the rest of the Titans, who broke out of their recent skid with an impressive comeback of their own on the road against the Ravens. The more important matchup may be on the other side of the ball, however, as Tennessee struggled to stop Indy during the Colts’ 34-17 win two weeks ago. Both teams are 7-3, and the winner’s division-title odds would soar from roughly 50-50 to about 85 percent, with the loser dropping to about 15 percent. Our model gives the Colts the slight edge at home, with a 60 percent probability of moving into the AFC South driver’s seat. Elo’s spread: Indianapolis -3

Neil Paine contributed research.

Check out our latest NFL predictions.

## Footnotes

1. We compute the kernel density estimates for completions on each piece of the field for the offense and defense, split into chunks that are 1 yard vertically by roughly 3.5 yards horizontally. We find the differences between the two units for each chunk, take the absolute value, and scale the result from 0 to 100. Values closer to 100 indicate that a QB’s completion rate in that area is similar to that of previous quarterbacks against the defense in question. Data includes games through Nov. 22.

Josh Hermsmeyer was a football writer and analyst.