Skip to main content
Menu
Our New Metric Shows How Good NFL Receivers Are At Creating Separation

Is Michael Thomas elite? On its face, that seems like an outrageous question. Michael Thomas, the New Orleans wide receiver who led the NFL in receiving yards and set an NFL record for receptions in 20191 on the way to winning Offensive Player of the Year — that Mike Thomas might not be an elite receiver? Yet throughout the offseason, NFL analysts have debated whether Thomas’s production is best explained by his skill and talent, or if instead he’s merely a good receiver who runs a lot of slants and benefits from being in an elite offense.

Playing with Drew Brees — a former Super Bowl MVP who holds the NFL record for highest career completion percentage, most career passing yards and most career passing touchdowns — certainly doesn’t hurt. And running routes in schemes designed by Sean Payton, a coach with a .630 career win percentage (208-131, third among active NFL coaches), probably also has a strong positive effect on his production. But Thomas’s numbers are still eye-popping, and his peers in the NFL recently ranked him first among all wide receivers (and fifth overall) in the NFL 100. So which is it: elite talent, or elite situation?

One way to try to answer the question is to look at how well a receiver creates and maintains separation from a defender. Being quick and fast isn’t enough for NFL success; just ask Yamon Figurs or Darrius Heyward-Bey, both of whom ran a 4.3 40-yard dash at the NFL combine. The elements of savvy route running — footwork, head and body fakes, disguising the intent of the route, changing direction sharply without losing speed — all appear to be more reliable indicators of NFL skill and talent than speed. And the ultimate goal of every route is to create enough separation from a defender to earn a target and make a catch.

To try to capture the results of this game of cat and mouse between receiver and defender, we used NFL Next Gen Stats data that measures the distance between a receiver and the nearest defender at key moments in each play. Perhaps it’s no surprise that in the ultimate team sport, assigning credit for something even as seemingly straightforward as separation is complicated. For instance, short passes are, on average, associated with more separation than deeper passes because a defender’s top priority is to avoid getting beat deep. So we need to account for depth of target and how far the QB had to throw the ball when we apportion credit for the separation a receiver got on a given play. The type of play call matters, too. Play-action passes create more separation than non-play-action passes at nearly every depth of target on average, so we need to contextualize players who are targeted more often on play-action as well.

We also needed to decide when to measure receiver separation. Since the most interesting routes to analyze are those that earned a target, there are two obvious points in a play to focus on: the moment the ball leaves the QB’s hand and the moment the ball arrives at the receiver’s location.

But there again we’re confronted with confounders that make apportioning credit and blame difficult.2 For example, some QBs throw with more anticipation than others, releasing the ball before a receiver has made his break and created the separation necessary for a successful completion. Measuring separation at the time of the throw punishes teammates with great chemistry.

So we decided to focus on separation at the moment the ball arrives, on the theory that scheme and QB play have the least influence at this crucial moment in a pass play. Ultimately, our separation model ending up including features that account for quarterback arm strength, the receiver’s separation at the time the QB targeted them, the horizontal and vertical position of the receiver on the field at the time of the throw, where the receiver lined up pre-snap, the distance to the goal line, the amount of break in the receiver’s route during the football’s journey through the air after it was released, the depth of the QB’s drop, the number of other routes that were being run on the play, if the play was a play-action pass or a screen, and the number of deep safeties.3

The final model does a decent job of predicting separation at the catch point on a given play.4 We used its predictions as a baseline for performance and compared each receiver’s actual separation on a given play to what the model expected an average player’s separation would have been, given similar circumstances. This result is a context-adjusted “separation over expected” (SOE) metric that we can calculate for each NFL receiver.

Since depth of target is so important in determining separation, we grouped plays into buckets — depending on whether a pass attempt was short, intermediate or deep — and created an SOE leaderboard for each. The resulting lists have reasonably strong face validity — players at the top of the leaderboards tend to be widely regarded as good route runners — suggesting SOE could be useful as a descriptive metric.

Separation is highest on short passes, but their value is low

Best and worst receiver seasons on short passes as measured by separation over expected (SOE) per play, 2017-19

Top 15 Receivers season targets aDOT epa/play soe/play
Evan Engram 2018 40 1.7 0.07 0.98
Zach Ertz 2019 45 2.3 0.43 0.59
Cole Beasley 2019 43 2.8 0.04 0.57
Diontae Johnson 2019 43 0.7 0.15 0.40
Nelson Agholor 2018 51 0.7 0.14 0.38
James Conner 2018 67 -0.9 0.19 0.36
Chris Carson 2019 43 -1.7 0.21 0.35
T.J. Yeldon 2018 71 0.2 -0.09 0.35
Duke Johnson 2018 48 -0.6 0.16 0.33
Jarvis Landry 2018 46 1.4 -0.10 0.33
Davante Adams 2017 48 2.3 0.31 0.30
Dede Westbrook 2019 52 1.1 -0.21 0.29
DeMarco Murray 2017 41 -0.6 0.00 0.29
Jaylen Samuels 2019 53 -1.5 -0.02 0.28
Keenan Allen 2018 64 2.9 0.14 0.28
Bottom 15 Receivers season targets aDOT epa/play soe/play
Latavius Murray 2019 41 -1.4 0.17 -0.28
Chris Thompson 2017 43 0.2 0.49 -0.30
Chris Thompson 2018 49 0.8 0.03 -0.31
Larry Fitzgerald 2018 42 2.4 -0.08 -0.32
Adam Humphries 2018 54 1.9 0.23 -0.33
Royce Freeman 2019 45 -0.3 -0.11 -0.34
Phillip Lindsay 2019 44 -1.8 -0.01 -0.34
Larry Fitzgerald 2019 59 1.3 -0.02 -0.35
Nick Chubb 2019 45 -1.0 0.02 -0.36
Le’Veon Bell 2019 71 -0.5 0.00 -0.39
Javorius Allen 2017 56 0.4 -0.12 -0.40
Giovani Bernard 2019 41 -0.8 -0.34 -0.42
David Johnson 2018 66 -1.2 -0.26 -0.44
Robert Woods 2019 48 -0.1 -0.08 -0.45
Todd Gurley 2019 45 0.5 -0.07 -0.49

Regular-season passes of 5 air yards or shorter, minimum of 40 targets.

Sources: NFL Next Gen Stats, ESPN Stats & Information Group

Two tight ends — Evan Engram of the New York Giants and Zach Ertz of the Philadelphia Eagles — top the list of receiver seasons with the highest separation over expected on short throws (5 air yards or less).5 Ertz’s targets especially were high value. While averaging more than a half-yard over expected in separation, his per-play expected points added (EPA) was worth more than 0.4 points in 2019. As a general rule, however, separation and value are decoupled on short passes. The defense is typically willing to allow an offense to throw to wide-open players short, then rally to make a tackle for a short gain.

Another interesting feature of short-target SOE is that a player’s ability to consistently get open on short throws appears to be mostly nonexistent. Having a high SOE in one season says very little about whether you will have a high SOE in the next.6 Short targets are also the only leaderboard where running backs make an appearance, owing to the dearth of targets they get deeper downfield.

These short passes, however, are where Michael Thomas frequently shines. Forty-eight percent of Thomas’s 185 targets came on passes 5 yards or less downfield in 2019. In the three seasons for which we have Next Gen data, Thomas has regularly posted high EPA per play values on these short targets and has been above average in creating separation at the catch point in two of the past three seasons.

Short, open targets to Mike Thomas are high value …

Receiving stats on short passes for Michael Thomas, including separation over expected (SOE) and expected points added (EPA) per play

season targets Avg. Depth of Target SOE/PLAY epa/PLAY
2019 88 3.2 0.05 0.25
2018 73 2.9 0.06 0.31
2017 57 2.5 -0.05 0.43

Regular-season passes of 5 air yards or less.

Sources: NFL Next Gen Stats, ESPN Stats & Information Group

Other players appear able to consistently create separation on short targets — Jarvis Landry is one example — but the value of Landry’s targets as measured by EPA are much lower than those directed at Thomas. In fact, on an EPA-per-play basis, throwing short to Landry has a worse point expectation than running the ball. That’s somewhat shocking — and perhaps it explains why the number of Landry’s short targets dropped by nearly half after he moved to the analytics-friendly Cleveland Browns.

… while short, open targets to Jarvis Landry are not

Receiving stats on short passes for Jarvis Landry, including separation over expected (SOE) and expected points added (EPA) per play

season targets Avg. Depth of Target soe/play epa/play
2019 49 1.2 0.23 -0.19
2018 46 1.4 0.33 -0.10
2017 89 1.3 0.23 -0.11

Regular-season passes of 5 air yards or less.

Sources: NFL Next Gen Stats, ESPN Stats & Information Group

Wide receivers make the bulk of their high-value receptions on passes at intermediate depths (between 5 and 15 air yards). The intermediate SOE leaderboard includes seasons from receivers like Davante Adams, Keenan Allen, Danny Amendola, Cooper Kupp and Adam Humphries, all of whom are generally regarded as skilled route runners. Interestingly, among the three target depths, SOE on intermediate passes appears to be the most stable year to year. So while we’d like more data, we should probably expect players who show the ability to separate above expectation on routes at these depths to continue to do so.7

Davante Adams and Keenan Allen are technicians

Best and worst receiver seasons on intermediate passes as measured by separation over expected (SOE) per play, 2017-19

top 15 Receivers season targets aDOT epa/play soe/play
Davante Adams 2018 67 9.5 0.53 0.65
Keenan Allen 2017 69 10.3 0.58 0.58
Davante Adams 2019 44 9.1 0.29 0.52
John Brown 2018 42 10.6 0.32 0.46
Danny Amendola 2019 47 8.9 0.25 0.43
Cooper Kupp 2019 50 10.7 1.12 0.40
Adam Humphries 2018 43 9.4 0.53 0.40
Courtland Sutton 2019 49 10.2 0.39 0.39
John Brown 2019 54 10.2 0.18 0.32
Jared Cook 2017 41 9.7 0.54 0.30
Keenan Allen 2019 56 10.1 0.65 0.30
Robert Woods 2019 50 10.5 0.40 0.30
Odell Beckham Jr. 2019 44 9.7 0.29 0.29
Adam Thielen 2018 56 10.1 0.44 0.28
Zach Ertz 2018 79 9.7 0.20 0.25
Michael Thomas 2018 54 10.1 0.59 0.25
BOTTOM 15 RECEIVERS SEASON TARGETS ADOT EPA/PLAY SOE/PLAY
DeAndre Hopkins 2019 49 9.7 0.12 -0.23
D.J. Chark 2019 47 10.0 0.27 -0.26
DeAndre Hopkins 2018 80 9.9 0.50 -0.27
Alshon Jeffery 2017 52 10.4 0.33 -0.27
Eric Decker 2017 40 9.5 0.54 -0.28
T.Y. Hilton 2018 52 9.0 0.29 -0.30
Julio Jones 2019 58 10.8 0.42 -0.31
Allen Robinson 2018 47 9.2 0.21 -0.31
Kenny Golladay 2018 51 10.9 0.16 -0.31
Allen Robinson 2019 68 9.9 0.10 -0.32
Terry McLaurin 2019 43 9.9 0.57 -0.36
Devin Funchess 2017 47 10.7 0.35 -0.36
Jared Cook 2018 41 8.7 0.67 -0.38
Cameron Brate 2017 41 11.1 0.23 -0.53
Kenny Golladay 2019 42 9.8 0.31 -0.54

Regular-season passes of between 5 and 15 air yards, minimum of 40 targets.

Sources: NFL Next Gen Stats, ESPN Stats & Information Group

Thomas isn’t in the same class as Adams and Allen when it comes to creating separation — Adams has averaged over a half-yard of SOE the past two years on intermediates routes — but despite the tighter windows, the expected value Thomas created on these targets ranks him among the best in the league on a per-play basis. Perhaps because of his success, Thomas has seen a steady increase in the number of targets at those depths. And compared to Atlanta’s Julio Jones — a receiver whose natural talent and skill set are rarely questioned — Thomas comes out ahead on both our separation and value metrics.

Thomas did more on midrange targets than Jones

Receiving stats on intermediate passes for Michael Thomas and Julio Jones, including separation over expected (SOE) and expected points added (EPA) per play

PLAYER season targets Avg. Depth of Target soe/play epa/play
Michael Thomas 2019 71 10.6 0.08 0.72
Michael Thomas 2018 54 10.1 0.25 0.59
Michael Thomas 2017 48 9.4 0.01 0.39
Julio Jones 2019 58 10.8 -0.31 0.42
Julio Jones 2018 65 9.8 -0.07 0.36
Julio Jones 2017 44 10.3 0.03 0.29

Regular-season passes of between 5 and 15 air yards.

Source: NFL Next Gen Stats, ESPN Stats & Information Group

Timo Riske of Pro Football Focus has shown that the best receivers in the league earn their targets all over the field, so it’s no surprise to see familiar names in the deep-target SOE ranks. But we also see deep ball specialists like D.J. Chark and Kenny Stills at the top of the list — the “stretch X” receivers whose job is to take the top off a defense. And at the bottom of the list we find names like Robby Anderson, a free agent this offseason who reportedly drew few offers from teams; an aging Larry Fitzgerald; and the unsigned Kelvin Benjamin. Finally, we see the continued effect of depth of target on separation. Deep targets (at least 15 air yards) are the most valuable in football on a per-play basis, but they’re also the throws with the tightest windows.

D.J. Chark was open deep a lot in 2019

Best and worst receiver seasons on deep passes as measured by separation over expected (SOE) per play, 2017-19

TOP 15 Receivers season targets aDOT epa/play soe/play
D.J. Chark 2019 30 27.1 0.96 0.41
Kenny Stills 2017 36 25.4 0.46 0.40
Amari Cooper 2019 37 23.3 0.90 0.39
Odell Beckham Jr. 2018 33 24.7 0.72 0.36
Adam Thielen 2017 38 22.0 0.78 0.34
Travis Kelce 2017 31 22.2 1.08 0.33
Jaron Brown 2017 30 24.0 0.52 0.29
Robert Woods 2018 32 23.4 0.29 0.25
Tyreek Hill 2018 48 31.4 0.78 0.25
Curtis Samuel 2019 41 25.7 -0.26 0.24
Brandin Cooks 2017 41 29.2 0.65 0.23
Calvin Ridley 2019 30 25.2 0.68 0.22
Jarvis Landry 2018 39 23.2 0.05 0.19
JuJu Smith-Schuster 2018 33 23.8 0.68 0.17
Brandin Cooks 2018 37 26.9 0.65 0.17
Chris Godwin 2019 30 21.9 1.29 0.13
BOTTOM 15 RECEIVERS SEASON TARGETS ADOT EPA/PLAY SOE/PLAY
Odell Beckham Jr. 2019 42 26.6 0.09 -0.18
Dez Bryant 2017 37 23.7 -0.26 -0.20
Marquise Goodwin 2017 35 28.1 0.41 -0.20
Doug Baldwin 2017 36 27.2 0.85 -0.24
DeVante Parker 2019 38 25.2 1.10 -0.25
Alshon Jeffery 2017 41 24.3 0.06 -0.26
Mike Evans 2017 44 23.7 0.20 -0.29
Kelvin Benjamin 2018 30 24.7 -0.47 -0.31
Robby Anderson 2019 42 24.9 0.18 -0.32
Tyler Lockett 2019 31 27.1 0.69 -0.34
Larry Fitzgerald 2017 32 20.1 0.41 -0.34
Marvin Jones 2019 30 23.7 0.48 -0.35
Jarvis Landry 2019 33 22.1 0.89 -0.36
Allen Robinson 2019 37 23.1 0.63 -0.36
Robby Anderson 2018 31 30.7 0.04 -0.42

Regular-season passes greater than 15 air yards, minimum of 30 targets.

Sources: NFL Next Gen Stats, ESPN Stats & Information Group

Again it seems instructive to compare Thomas to Jones. Jones was targeted deep more frequently, suggesting that his skillset is better suited to the demands of beating fast humans in a footrace, but he’s also not as successful at creating separation from defenders as Thomas is. And Julio’s targets have, on average, been worth less than Thomas’s in the previous three years. (Again, no one doubts Jones’s talent or skills, and both he and Thomas are consistently in the conversation for best receiver in the league.)

When we account for the most impactful context that affects a receiver’s most important job — getting open — Thomas is routinely above average in creating that separation. And targets to him are among the most valuable plays in football across all depths. There isn’t much evidence to support the idea that Mike Thomas is anything but an elite football talent.

Footnotes

  1. The second season in a row that he led the NFL in catches.

  2. It also means the entire analysis is conditional on a player actually being targeted. We can’t say anything about the skill of receivers who fail to earn targets.

  3. The data set includes all regular-season pass attempts from the last three seasons, excluding spikes and passes from punts and field goal formations.

  4. The model is an xgboost trained with fivefold cross validation and tested on out-of-sample data. Root mean squared error 1.52, r-squared 0.61, mean absolute error 1.09.

  5. As we’re using air yards — the vertical yards on a pass attempt relative to the line of scrimmage — this bucket includes passes behind the line of scrimmage.

  6. Year-over-year r-squared of 0.02.

  7. Sample size caveats here. Perhaps even Simpson’s paradox caveats. Year-over-year r-squared of 0.13, n = 53 player season pairs from 2017-2019.

Josh Hermsmeyer is a football writer and analyst.

Comments