Skip to main content
Menu
What Our NBA Projections Got Right (And Wrong) Last Season

CARMELO is back! As my colleague Nate Silver detailed Thursday, we’re issuing our second set of NBA player career forecasts; you can find the latest batch of projections here. For those unfamiliar with CARMELO,1 it’s an algorithm that uses the career arcs of similar historical players to predict what’s in store for today’s stars, journeymen and scrubs.

I want to dig into what CARMELO predicts for 2016-17, but first let’s look back at the best — and worst — moments of the projection’s rookie season. To help isolate its biggest hits and misses, I gathered wins above replacement2 data for the 435 players who both played in the NBA in 2015-16 and were issued a CARMELO forecast last fall. Here’s a simple histogram of the differences in WAR between what was predicted and what transpired on the court:

paine-carmelo-recap-1

About 55 percent of the players finished with a WAR total within a win (plus or minus) of what CARMELO predicted; that grows to 80 percent if we look for players who fell within two wins of their WAR forecast. It’s tough to say how that compares to other projection systems, since there aren’t many alternatives available in the public domain, but in a vacuum that doesn’t seem like an awful rookie showing, particularly since CARMELO’s errors appear to be roughly symmetrical along the shape of a bell curve — meaning it isn’t systematically biased toward over- or undervaluing players.

CARMELO wasn’t perfect, though. Here were its biggest misses, high and low, of 2015-16:

FORECAST ACTUAL
PLAYER AGE TM MINUTES +/- WAR MINUTES +/- WAR DIFF
Stephen Curry 27 GS 2608 +8.2 14.7 2700 +12.5 21.6 +6.8
Kyle Lowry 29 TOR 2307 +3.5 7.0 2851 +6.8 13.9 +6.8
Anthony Davis 22 NO 2568 +6.3 11.8 2164 +2.2 5.1 -6.7
Ty Lawson 28 2442 +1.3 4.4 1411 -4.6 -2.0 -6.4
R. Westbrook 27 OKC 2449 +7.3 12.7 2750 +10.0 18.3 +5.6
Kemba Walker 25 CHA 2452 +1.3 4.4 2885 +4.0 9.7 +5.2
K. Towns 20 MIN 1859 -0.2 1.8 2627 +2.8 6.8 +5.0
Paul George 25 IND 2074 +2.4 5.1 2819 +4.5 10.1 +5.0
Blake Griffin 26 LAC 2581 +3.9 8.5 1170 +3.3 3.5 -5.0
Elfrid Payton 21 ORL 2404 +2.3 5.7 2145 -1.3 0.9 -4.8
Kyrie Irving 23 CLE 2743 +3.3 8.0 1667 +1.6 3.3 -4.7
Joakim Noah 30 CHI 2160 +3.0 6.0 635 +1.9 1.3 -4.7
Paul Millsap 30 ATL 2149 +3.2 6.2 2647 +5.3 10.8 +4.6
Tyson Chandler 33 PHX 2045 +3.2 5.9 1618 -0.5 1.3 -4.6
M. Kidd-Gilchrist 22 CHA 2260 +1.5 4.4 205 -1.4 0.0 -4.4
Jae Crowder 25 BOS 1364 +0.5 1.9 2308 +2.8 6.2 +4.3
Markieff Morris 26 2073 +1.1 3.6 1629 -2.8 -0.7 -4.3
Marvin Williams 29 CHA 1568 +0.0 1.8 2338 +2.7 5.9 +4.2
Draymond Green 25 GS 2189 +4.6 8.0 2808 +5.8 12.1 +4.1
Al Horford 29 ATL 2063 +2.3 4.9 2631 +4.1 9.0 +4.1
What CARMELO got wrong in 2015-16

Among players who were issued a forecast and played in the NBA in 2015-16.

Source: Basketball-Reference.com

By WAR, the biggest miss on CARMELO’s résumé was also the game’s biggest star: Stephen Curry. It wasn’t that the algorithm thought Curry would be bad — CARMELO predicted that he’d be the game’s most valuable player in 2015-16 — but the projection didn’t foresee the quantum leap his game would take the season after he’d already established himself as league MVP. Outlier performances are outliers for a reason; most players would regress to the mean after posting one of the top 50 seasons in modern NBA history, not one-up themselves with a campaign ranking in the top 10. Obviously, Curry isn’t “most players.”

Similarly, CARMELO knew Kyle Lowry and Russell Westbrook were good, but it didn’t bank on them being quite so good. The numbers also didn’t see Kemba Walker’s breakout performance coming, or that Karl-Anthony Towns would be one of the best rookies in modern history. And it was taken completely by surprise when Anthony Davis — CARMELO’s pick as the game’s most valuable franchise player — turned in a historically disappointing season.

Davis, who was less than 100 percent for much of the season, brings us to the bumps and bruises — or worse — that players have to deal with. Injuries are notoriously difficult to predict, and since playing time and performance are so fundamentally intertwined, many of the players on the list above saw various ailments rob them of both minutes and per-minute production. Joakim Noah and Michael Kidd-Gilchrist, for instance, missed most of the season with injuries, and they weren’t themselves when they did suit up.

And it goes without saying that CARMELO knows little about the personal-life problems of mere humans. That’s why it — like me — was so utterly, woefully wrong about Ty Lawson’s disastrous 2015-16 season.

Things weren’t all bad for CARMELO’s inaugural season, though. Here are the players — among those who played at least 1,500 minutes — for whom the projected WAR totals most closely matched the player’s output in 2015-16:

FORECAST ACTUAL
PLAYER AGE TEAM MINUTES +/- WAR MINUTES +/- WAR DIFF
Dwight Howard 30 HOU 1920 +1.1 3.3 2280 +0.6 3.3 +0.0
Jeff Teague 27 ATL 2072 +0.5 2.8 2255 +0.3 2.9 +0.0
Arron Afflalo 30 NY 1952 -2.5 -0.5 2371 -2.4 -0.4 +0.1
Brandon Knight 24 PHX 2337 -0.6 1.8 1870 -0.3 1.8 -0.1
Nicolas Batum 27 CHA 2435 +2.0 5.4 2448 +2.0 5.5 +0.1
Marcin Gortat 31 WSH 1889 +1.6 3.8 2256 +1.2 4.0 +0.2
Jimmy Butler 26 CHI 2688 +3.6 8.3 2474 +4.0 8.1 -0.2
Goran Dragic 29 MIA 2169 +1.1 3.7 2363 +0.7 3.5 -0.2
Shane Larkin 23 BKN 1654 -2.1 -0.1 1751 -2.2 -0.2 -0.2
T.J. McConnell 23 PHI 387 -2.8 -0.2 1606 -2.1 0.0 +0.2
Andre Iguodala 32 GS 1735 +1.9 3.7 1732 +1.6 3.5 -0.2
Lavoy Allen 26 IND 1201 +0.3 1.5 1599 -0.6 1.3 -0.2
Tim Duncan 39 SA 1652 +3.5 5.1 1536 +4.1 5.3 +0.2
Derrick Favors 24 UTA 2195 +2.3 5.3 1983 +2.7 5.1 -0.2
Enes Kanter 23 OKC 1824 -2.0 0.0 1721 -1.7 0.2 +0.2
Frank Kaminsky 22 CHA 1272 -1.4 0.4 1708 -1.2 0.7 +0.2
D. Cunningham 28 NO 1360 -1.2 0.6 1971 -1.2 0.9 +0.2
Patrick Patterson 26 TOR 1841 +1.5 3.6 2020 +1.0 3.3 -0.3
Luis Scola 35 TOR 1052 -1.3 0.4 1636 -1.2 0.7 +0.3
Jerami Grant 21 PHI 1656 -0.7 1.2 2066 -1.3 0.9 -0.3
What CARMELO got less wrong in 2015-16

Among players who were issued a forecast and played 1,500 NBA minutes in 2015-16.

Source: Basketball-Reference.com

That’s a pretty good list! Nic Batum, for instance, was coming off of a down year by the conventional metrics, but CARMELO predicted he’d bounce back to something more like his old form. It also predicted that Tim Duncan, at age 39, would play at a high level, and that lottery pick Frank Kaminsky would underwhelm.

So, now that we’ve assessed CARMELO’s debut season, what can it tell us going forward? Here are the players our system thinks will see the biggest improvements (or declines) by WAR in 2016-17:

BIGGEST IMPROVEMENTS BIGGEST DECLINES
PLAYER 2016 WAR 2017 WAR CHANGE PLAYER 2016 WAR 2017 WAR CHANGE
A. Davis 5.0 8.8 +3.7 S. Curry 21.7 16.1 -5.6
K. Porzingis 2.4 5.5 +3.1 L. James 16.7 11.5 -5.2
K. Irving 3.3 6.2 +2.8 K. Lowry 13.9 9.4 -4.5
A. Wiggins -0.2 2.6 +2.8 P. Millsap 10.7 7.3 -3.4
M. Smart 2.5 5.1 +2.6 K. Durant 14.1 10.9 -3.2
E. Payton 0.8 3.4 +2.6 A. Horford 8.9 5.7 -3.2
D. Russell 0.3 2.7 +2.5 R. Westbrook 18.3 15.2 -3.1
B. Griffin 3.4 5.9 +2.4 K. Leonard 13.6 10.5 -3.1
E. Mudiay -2.6 -0.3 +2.4 P. George 10.1 7.2 -2.9
T. Evans 1.6 3.8 +2.2 C. Paul 13.1 10.3 -2.9
CARMELO’s most (and least) improved players for 2016-17

Among players who will not be rookies in 2016-17.

Since CARMELO uses previous seasons to inform its projections, along with a heavy dose of regression to the mean, there’s some crossover between the lists of its 2015-16 misses and its 2016-17 improvements or declines. Curry, Lowry and company can’t possibly be that dominant two years in a row, right? We’ll see; projection systems are conservative by nature, always abiding by the law of averages, and an explosive individual performance represents a rebellion against that law. Maybe some of the names on the right-hand list will buck the odds and make history again; maybe they won’t. The left-hand list, however, is the one to keep an eye on — these are largely young players the projection expects to make big improvements, as well as a few veterans (Davis, Blake Griffin, Tyreke Evans) that it expects to bounce back.

On that note, here’s a list based on pure volatility — the players for whom CARMELO projects the biggest range between what could reasonably be termed their best-case (90th percentile) and worst-case (10th percentile) outcomes next season:

WAR
PLAYER POSITION AGE BEST CASE MEAN WORST CASE RANGE (+/-)
James Harden SG 27 19.5 14.2 8.0 11.6
Chris Paul PG 31 14.9 10.3 3.4 11.5
Damian Lillard PG 26 13.4 8.2 2.3 11.1
DeAndre Jordan C 28 13.9 8.0 2.9 11.0
Kawhi Leonard SF 25 15.8 10.5 5.0 10.8
Kyle Lowry PG 30 14.5 9.4 3.8 10.8
John Wall PG 26 13.2 8.0 2.6 10.7
Karl-Anthony Towns C 21 14.3 8.8 3.9 10.4
Draymond Green PF 26 14.6 9.6 4.4 10.2
LeBron James SF 32 16.3 11.5 6.2 10.1
Russell Westbrook PG 28 19.2 15.2 9.4 9.8
Anthony Davis PF 23 13.5 8.8 3.7 9.8
Kemba Walker PG 26 11.6 7.2 2.1 9.6
Kristaps Porzingis PF 21 10.2 5.5 0.7 9.5
Marcus Smart PG 22 9.9 5.1 0.5 9.3
Jimmy Butler SG 27 11.8 7.6 2.6 9.2
Gordon Hayward SF 26 11.2 6.8 2.0 9.2
Stephen Curry PG 28 20.5 16.1 11.3 9.2
Ben Simmons PF 20 9.1 4.1 -0.1 9.2
Paul Millsap PF 31 12.2 7.3 3.0 9.1
CARMELO’s most volatile players of 2016-17

Naturally, young players such as Towns, Davis, Kristaps Porzingis and No. 1 draft pick Ben Simmons will have wider variation in potential outcomes because we have less of a sample from which to draw their projections. But some veterans are also highly volatile because their comparable-player lists contain both stars and duds. From here out, James Harden could have the career arc of a Kobe Bryant (who stuck around in the league forever) or a Steve Francis (who was great early in his career but was out of the league by age 31).

That’s the beauty of the NBA — we never truly know what will happen. But with CARMELO’s help, we have a slightly better idea than we would otherwise.

Check out FiveThirtyEight’s CARMELO NBA player projections.

Footnotes

  1. Which, as a complete coincidence, stands for Career-Arc Regression Model Estimator with Local Optimization.

  2. WAR can be calculated by multiplying a player’s Box Plus/Minus by the percentage of his team’s minutes he played and then multiplying that by 2.2.

Neil Paine is a senior sportswriter for FiveThirtyEight.

Filed under

Comments