What Our NBA Projections Got Right (And Wrong) Last Season

CARMELO is back! As my colleague Nate Silver detailed Thursday, we’re issuing our second set of NBA player career forecasts; you can find the latest batch of projections here. For those unfamiliar with CARMELO,¹ it’s an algorithm that uses the career arcs of similar historical players to predict what’s in store for today’s stars, journeymen and scrubs.

I want to dig into what CARMELO predicts for 2016-17, but first let’s look back at the best — and worst — moments of the projection’s rookie season. To help isolate its biggest hits and misses, I gathered wins above replacement² data for the 435 players who both played in the NBA in 2015-16 and were issued a CARMELO forecast last fall. Here’s a simple histogram of the differences in WAR between what was predicted and what transpired on the court:

About 55 percent of the players finished with a WAR total within a win (plus or minus) of what CARMELO predicted; that grows to 80 percent if we look for players who fell within two wins of their WAR forecast. It’s tough to say how that compares to other projection systems, since there aren’t many alternatives available in the public domain, but in a vacuum that doesn’t seem like an awful rookie showing, particularly since CARMELO’s errors appear to be roughly symmetrical along the shape of a bell curve — meaning it isn’t systematically biased toward over- or undervaluing players.

CARMELO wasn’t perfect, though. Here were its biggest misses, high and low, of 2015-16:

By WAR, the biggest miss on CARMELO’s résumé was also the game’s biggest star: Stephen Curry. It wasn’t that the algorithm thought Curry would be bad — CARMELO predicted that he’d be the game’s most valuable player in 2015-16 — but the projection didn’t foresee the quantum leap his game would take the season after he’d already established himself as league MVP. Outlier performances are outliers for a reason; most players would regress to the mean after posting one of the top 50 seasons in modern NBA history, not one-up themselves with a campaign ranking in the top 10. Obviously, Curry isn’t “most players.”

Similarly, CARMELO knew Kyle Lowry and Russell Westbrook were good, but it didn’t bank on them being quite so good. The numbers also didn’t see Kemba Walker’s breakout performance coming, or that Karl-Anthony Towns would be one of the best rookies in modern history. And it was taken completely by surprise when Anthony Davis — CARMELO’s pick as the game’s most valuable franchise player — turned in a historically disappointing season.

Davis, who was less than 100 percent for much of the season, brings us to the bumps and bruises — or worse — that players have to deal with. Injuries are notoriously difficult to predict, and since playing time and performance are so fundamentally intertwined, many of the players on the list above saw various ailments rob them of both minutes and per-minute production. Joakim Noah and Michael Kidd-Gilchrist, for instance, missed most of the season with injuries, and they weren’t themselves when they did suit up.

And it goes without saying that CARMELO knows little about the personal-life problems of mere humans. That’s why it — like me — was so utterly, woefully wrong about Ty Lawson’s disastrous 2015-16 season.

Things weren’t all bad for CARMELO’s inaugural season, though. Here are the players — among those who played at least 1,500 minutes — for whom the projected WAR totals most closely matched the player’s output in 2015-16:

That’s a pretty good list! Nic Batum, for instance, was coming off of a down year by the conventional metrics, but CARMELO predicted he’d bounce back to something more like his old form. It also predicted that Tim Duncan, at age 39, would play at a high level, and that lottery pick Frank Kaminsky would underwhelm.

So, now that we’ve assessed CARMELO’s debut season, what can it tell us going forward? Here are the players our system thinks will see the biggest improvements (or declines) by WAR in 2016-17:

Since CARMELO uses previous seasons to inform its projections, along with a heavy dose of regression to the mean, there’s some crossover between the lists of its 2015-16 misses and its 2016-17 improvements or declines. Curry, Lowry and company can’t possibly be that dominant two years in a row, right? We’ll see; projection systems are conservative by nature, always abiding by the law of averages, and an explosive individual performance represents a rebellion against that law. Maybe some of the names on the right-hand list will buck the odds and make history again; maybe they won’t. The left-hand list, however, is the one to keep an eye on — these are largely young players the projection expects to make big improvements, as well as a few veterans (Davis, Blake Griffin, Tyreke Evans) that it expects to bounce back.

On that note, here’s a list based on pure volatility — the players for whom CARMELO projects the biggest range between what could reasonably be termed their best-case (90th percentile) and worst-case (10th percentile) outcomes next season:

Naturally, young players such as Towns, Davis, Kristaps Porzingis and No. 1 draft pick Ben Simmons will have wider variation in potential outcomes because we have less of a sample from which to draw their projections. But some veterans are also highly volatile because their comparable-player lists contain both stars and duds. From here out, James Harden could have the career arc of a Kobe Bryant (who stuck around in the league forever) or a Steve Francis (who was great early in his career but was out of the league by age 31).

That’s the beauty of the NBA — we never truly know what will happen. But with CARMELO’s help, we have a slightly better idea than we would otherwise.

Check out FiveThirtyEight’s CARMELO NBA player projections.