Skip to main content
Menu
Are Defensive Play-Callers Consistent Enough To Be Exploitable?

If NFL analytics is a set of festive Russian nesting dolls, the final pea-sized matryoshka is defense. We’re actually pretty good at understanding offensive skill position players at this point. When a quarterback change is made on a team, our models and the betting markets move in response. No Patrick Mahomes? “That’s bad for Kansas City’s chances! Let’s move the line down and predict fewer points scored.” But when a defender is out, everyone shrugs. No Xavien Howard?1 “Um, well, I’m not sure what that means for Miami.” We know so little about defense that some unhinged analysts have even pounded out “defense doesn’t matter” on various social media platforms in fits of rage and frustration.


Who’s the most chaotic fictional football coach? | FiveThirtyEight

It’s likely that the lack of predictive power in defensive analytics has to do with the reactive nature of defense. NFL defenses are highly interconnected, weak-link systems. What does weak link mean? It means that if just one person in the defensive chain fails, the entire unit is assigned that failure.2 We can measure that failure, but the fact that we can’t predict future defensive performance implies that the precise types of failures we’re measuring are unlikely to occur exactly that way again. Most NFL defenses are good enough to not repeat the same mistakes over and over — they just find new and exciting ways to fail. All of this makes apportioning blame and properly accounting for success pretty difficult.

[How To Build A Placekicker]

But a recent tweet from Anthony Reinhard made me wonder if we can assign some of that blame reliably on coaches. Could we successfully predict how coaches will respond to game states like obvious passing or obvious running situations? After all, defensive coordinators typically call the defensive plays, and they’re just one person, not an interconnected unit. If we can predict what a coach will do, maybe we can gain a deeper understanding of how a defense is likely to react in certain situations.

Reinhard looked at how teams respond to probable passing and running situations by measuring the number of defenders that line up “in the box” — or near the line of scrimmage — and comparing it to the probability that a team will drop back to pass. In passing situations, it’s usually advantageous to drop a defender into coverage. Likewise, if it’s a more obvious running situation, the defense will want to make sure they have enough players near the line of scrimmage to be “gap sound” and properly defend the run.

I took Reinhard’s idea and calculated a new metric based on it: Defenders in the Box Over Expected (DBOE). Then I looked at how stable DBOE was year-to-year by subsetting the data to just teams with the same defensive coordinator from one year to the next.

[Do NFL Teams Really Get A Bounce After Firing Their Coaches?]

It turns out that DBOE is remarkably stable,3 a rare feature for an advanced defensive metric, indicating that coaches tend to react to obvious passing and rushing situations in reliable ways over time.4

Interestingly, in the period for which we have data, the league has moved toward favoring lighter boxes over what we might expect. The downward trend was pretty reliable, right up until around 2019. And in 2020, the trend reversed itself entirely.

There’s more work that needs to be done to apply DBOE to football in useful ways, but the implications are interesting. If a defensive coordinator will reliably overcommit defenders to the box in obvious passing situations, opposing offenses could potentially take advantage. Similarly, defenses that back off and play the pass on more obvious rushing downs could be exploited by an offense spreading out the defense and running. Even better, DBOE is based on the probability of a play being a dropback, a metric that’s fairly opaque to a defensive play-caller since they often simply match big bodies to big bodies when the offense trots out their personnel. That means having a high or low DBOE isn’t a tendency the defensive coordinator can easily obscure without some effort. It’s likely an integral part of how that coach approaches calling a game.

Here is the full list of NFL defensive coordinators and their DBOE, through Week 15:

The Rams stack the box less, the Jaguars more

NFL teams by Defenders in the Box Over Expected (DBOE) on all regular-season plays for which the probability of a dropback could be calculated

Team Coordinator dboe
Los Angeles Rams Brandon Staley -0.44
Denver Broncos Ed Donatell -0.26
New York Giants Patrick Graham -0.23
Green Bay Packers Mike Pettine -0.22
Carolina Panthers Phil Snow -0.22
New England Patriots Bill Belichick* -0.22
Pittsburgh Steelers Keith Butler -0.12
Indianapolis Colts Matt Eberflus -0.05
Buffalo Bills Leslie Frazier -0.02
Chicago Bears Chuck Pagano +0.01
Philadelphia Eagles Jim Schwartz +0.01
Los Angeles Chargers Gus Bradley +0.01
Las Vegas Raiders Paul Guenther +0.05
Tennessee Titans Mike Vrabel* +0.05
Detroit Lions Cory Undlin +0.05
Cleveland Browns Joe Woods +0.08
Dallas Cowboys Mike Nolan +0.10
Seattle Seahawks Ken Norton Jr. +0.11
New Orleans Saints Dennis Allen +0.12
Houston Texans Anthony Weaver +0.13
New York Jets Gregg Williams/Frank Bush +0.14
Arizona Cardinals Vance Joseph +0.16
Atlanta Falcons Raheem Morris/Jeff Ulbrich +0.16
Miami Dolphins Josh Boyer +0.18
Cincinnati Bengals Lou Anarumo +0.20
San Francisco 49ers Robert Saleh +0.23
Minnesota Vikings Andre Patterson & Adam Zimmer +0.24
Baltimore Ravens Don Martindale +0.24
Kansas City Chiefs Steve Spagnuolo +0.24
Tampa Bay Buccaneers Todd Bowles +0.29
Washington Football Team Jack Del Rio +0.33
Jacksonville Jaguars Todd Wash +0.38

Through Week 15.

*The Patriots and Titans don’t have official defensive coordinators separate from the head coaches. Other head coaches may be involved in defensive play calling.

Source: ESPN Stats & Information Group

Through Week 15 this year, the coordinators who seem to be most concerned about getting beat through the air include Brandon Staley of the Los Angeles Rams and Ed Donatell of the Denver Broncos. The Rams have one of the best defenses in the league this year, allowing a league-low 4.6 yards per offensive play. (Of course, they also have Aaron Donald and Jalen Ramsey, which probably has something to do with that.) Meanwhile, the Broncos have been less successful, ranking 13th in the league at 5.5 yards allowed per play.

[Who’s The Most Chaotic Movie Football Coach?]

On the other end of the spectrum, Todd Wash of the Jacksonville Jaguars and Jack Del Rio of the Washington Football Team would prefer to stack the box to prevent the easy yards on the ground. Jacksonville has the league’s worst defense, allowing 6.3 yards per play, but Washington is fourth best at 5.0 yards per play. All of this suggests that while DBOE might help us understand coaches and their preferences, football is still a game predicated on having good players on the field — especially on defense.

FiveThirtyEight’s NFL Elo ratings

How each team ranks through Week 15 of the 2020 season, according to our quarterback-adjusted predictions

Chance To …
Rk Team Starting QB QB Rk* Elo Rating Proj. Wins Make Playoffs Win Div. Win SB
1 Chiefs Mahomes 1 1771 14.8 100% 100% 35%
2 Saints Brees 7 1715 11.6 100 97 18
3 Bills Allen 3 1689 12.5 100 100 10
4 Packers Rodgers 4 1656 12.3 100 100 13
5 Ravens Jackson 5 1644 10.7 86 0 3
6 Colts Rivers 14 1622 11.4 92 44 3
7 Bucs Brady 10 1610 10.5 99 3 4
8 Titans Tannehill 8 1609 11.1 98 56 3
9 Browns Mayfield 16 1591 11.4 93 27 2
10 Seahawks Wilson 9 1587 11.2 100 65 3
11 Rams Goff 17 1584 10.1 98 35 2
12 Steelers R’lisberger 12 1568 11.9 100 73 2
13 Cardinals Murray 2 1538 9.0 70 0 1
14 Dolphins Tagovailoa 23 1536 9.9 32 0 <1
15 Bears Trubisky 21 1516 8.1 31 0 <1
16 Patriots Newton 25 1494 7.1 0 0 0
17 Vikings Cousins 18 1493 6.8 2 0 <1
18 Broncos Lock 28 1465 6.1 0 0 0
19 49ers Beathard 30 1459 5.7 0 0 0
20 Falcons Ryan 20 1453 4.4 0 0 0
21 Wash. Smith✚ 26 1447 7.1 74 74 <1
22 Eagles Hurts 24 1434 5.5 10 10 <1
23 Panthers Bridgewater 11 1431 4.6 0 0 0
24 Chargers Herbert 13 1429 5.6 0 0 0
25 Texans Watson 6 1428 5.0 0 0 0
26 Cowboys Dalton 27 1407 6.0 5 5 <1
27 Raiders Carr✚ 15 1397 7.8 <1 0 <1
28 Giants Jones✚ 29 1391 5.7 11 11 <1
29 Lions Stafford 19 1360 5.6 0 0 0
30 Bengals Allen✚ 32 1313 3.9 0 0 0
31 Jaguars Minshew 22 1294 1.4 0 0 0
32 Jets Darnold 31 1292 1.4 0 0 0

*Ranking among Week 16 starters, according to our QB Elo ratings.

✚ Starter is currently injured and may not play.

†Starter could be replaced for performance reasons.

Simulations as of Dec. 23, 2020.

Source: ESPN

Looking ahead: The best and most significant game of Week 16 — from a Super Bowl odds perspective, at least — features the Tennessee Titans paying a visit to the Green Bay Packers at Lambeau Field. Neither team is in danger of missing the playoffs, but there’s still plenty to play for: The Pack have not yet locked up the NFC’s lone first-round bye, but they could with a win and losses by the Saints and Seahawks. And as AFC South leaders by way of a tiebreaker with Indianapolis, the Titans are currently the No. 4 seed in the conference, though they need to keep pace with Indy in order to host a playoff game instead of going on the road. That sets the stakes in this one, where each team’s weak defensive link will get a stress test against the NFL’s best before the playoffs: Tennessee’s suspect pass defense (No. 28 in schedule-adjusted EPA) going against Aaron Rodgers, and Green Bay’s awful run D (No. 27) staring down Derrick Henry. Both teams are in pretty great form right now as well, with Tennessee winning four of its past five contests while Green Bay has won four straight (and six of seven). We think the Packers — playing at home — have the edge, with a 62 percent chance of grabbing the W and moving closer to the No. 1 seed in the NFC. Elo’s spread: Green Bay -3½

Neil Paine contributed research.

Check out our latest NFL predictions.

Footnotes

  1. Xavien Howard is the highest graded cornerback by Pro Football Focus at the time of this writing.

  2. Offenses share this feature too, but it’s less pronounced. Sometimes a play can be made by one or two individuals — one is often the QB — that salvages an otherwise doomed play.

  3. Year-to-year correlation of 0.55, 2008-20, n = 231 team-season pairs.

  4. Coordinators in our sample may have been credited for plays they didn’t call, based on in-season role changes.

Josh Hermsmeyer is a football writer and analyst.

Comments