Skip to main content
Menu
How Our 2017 MLB Predictions Work

Editor’s note: This article is an adapted version of one we published last year about how our MLB predictions work.

FiveThirtyEight is again publishing a season’s worth of MLB win predictions, using our baseball Elo. The predictions preview upcoming games and show the chances that each team will make the playoffs and that it’ll win the World Series. We’ll also be updating our Complete History Of MLB feature throughout the season, which charts the successes and failures of every franchise throughout history. Here’s how both work.

Thanks to Retrosheet, we’ve collected game results and box scores back to 1871 and used them to create an Elo-based rating system and predictive model for baseball that incorporates home-field advantage, margin of victory, travel, rest and — most importantly — starting pitchers. The ratings are also adjusted for park and era effects and account for the fact that favorites are more likely to win in the postseason than in the regular season.

Elo is a simple but elegant system that can be tuned and customized endlessly to incorporate available data. In our baseball Elo system, each team has a rating (the average is about 1500), and after every game, the winning team gains some Elo points while the losing team loses the same number of points. The number of points exchanged is based on the chances our model gave each team to win the game and the margin of victory; a win by a big underdog results in a bigger exchange of Elo points than a win by a favorite, and the larger the margin of victory, the larger the exchange.

Before every game, we also adjust each team’s rating based on whether it has home-field advantage, how far it’s traveled to the game, how many days of rest it’s had and which pitcher is slated to start.

Home-field advantage is worth 24 Elo points in our model, and travel and rest adjustments are worth up to about 5 points each1; these three components are combined into the “Adjust for home field, travel and rest” section in the image above.2

Starting pitcher adjustments can have a substantial impact on pre-game team ratings and win probabilities. For example, in June of 2000, Pedro Martinez was worth about 109 Elo points to the Red Sox each time he started, which is the equivalent of a 15 percent boost to their chances of winning the game.3 This means Martinez was worth 109 points more than the average starting pitcher on his team (or even a bit more, since his starts were already incorporated into the team’s overall rating).

To generate these pitcher adjustments, we’re using a version of Bill James’s game scores proposed by Tangotiger (and slightly modified by us) to isolate pitching performances. After each game, the starting pitcher’s game score is calculated as:

47.4 + 1.5*outs + strikeouts – 2*walks – 2*hits – 3*runs – 4*homeruns

We maintain a running average of these game scores for each pitcher to produce his overall pitcher score.4 Here’s a list of the pitchers with the highest peak scores in history, the dates when they peaked and their corresponding Elo bonus:5

DATE NAME TEAM PITCHER SCORE TEAM RATING ADJ.
6/8/2000 Pedro Martinez Red Sox 78.0 +108.6
5/16/2000 Randy Johnson Diamondbacks 71.8 +83.3
7/19/1995 Greg Maddux Braves 71.8 +67.3
7/28/1997 Roger Clemens Blue Jays 70.6 +75.3
5/5/1929 Dazzy Vance Robins 69.7 +76.3
6/21/2016 Clayton Kershaw Dodgers 69.6 +77.5
4/7/2002 Curt Schilling Diamondbacks 69.1 +64.3
5/25/1969 Bob Gibson Cardinals 68.6 +76.6
5/6/1986 Dwight Gooden Mets 68.5 +74.4
6/24/1977 Frank Tanana Angels 68.4 +72.2
8/4/1940 Bob Feller Indians 68.4 +64.7
10/7/2015 Jake Arrieta Cubs 68.3 +63.4
10/14/1965 Sandy Koufax Dodgers 68.1 +64.4
4/30/1980 J.R. Richard Astros 67.7 +67.1
7/13/1915 Pete Alexander Phillies 67.7 +64.7
10/5/2004 Johan Santana Twins 67.6 +66.9
10/8/1998 Kevin Brown Padres 67.4 +65.4
4/26/1972 Tom Seaver Mets 67.3 +72.8
6/13/1932 Lefty Grove Athletics 66.9 +64.1
5/18/1987 Mike Scott Astros 66.9 +61.4
9/15/1978 Ron Guidry Yankees 66.9 +62.4
All-time pitcher peaks, based on pitcher score

The pitcher score is a running average of a pitcher’s individual game scores.

Sources: Retrosheet and ESPN

Pitchers in the table above are listed only once, at their absolute peak. The only new entrant from last year is Clayton Kershaw.

In addition to scores for each pitcher, we maintain a pitching score for every team — these are based on the game scores of every starting pitcher on that team. Each pitcher’s Elo adjustment is relative to his team’s pitching score; pitchers above the team average give the team a bonus when they start, and pitchers below the team average give the team a penalty. Note that in the table above, one pitcher may have a higher overall score than another pitcher but a smaller Elo adjustment; this generally means that his team had a better rotation or that he started more games and his game scores contributed more to the team’s overall average.

These are the top 2017 pitchers, according to our pitcher scores:

NAME TEAM PITCHER SCORE TEAM RATING ADJ.
Clayton Kershaw Dodgers 65.1 +50
Justin Verlander Tigers 62.3 +47
Jon Lester Cubs 61.4 +29
Corey Kluber Indians 61.0 +36
Max Scherzer Nationals 60.6 +33
Kyle Hendricks Cubs 60.2 +24
Madison Bumgarner Giants 58.9 +23
Chris Sale Red Sox 58.9 +22
Johnny Cueto Giants 58.8 +23
Top 2017 starters by pitcher score

Sources: Retrosheet and ESPN

Since starting pitcher bonuses from the two teams are additive, they cancel each other out when two top pitchers face each other and can be quite large when a strong pitcher faces a weak pitcher. The biggest mismatch of all time according to our pitcher scores was a 137-point Elo swing back in 1997, when Randy Johnson (+87) faced Ricky Bones (-50).

Our 2017 preseason team ratings are a blend of our 2016 end-of-season ratings (reverted to the mean by one-third) and three projection systems (PECOTA, FanGraphs and Davenport). Our preseason team pitcher scores use our 2016 end-of-season pitcher scores combined with projected starts from the same three projection systems.

We use Monte Carlo simulations to play out the season thousands of times to see how often each team makes the playoffs or wins the World Series. As with our other forecasts, we run simulations “hot,” meaning that a team’s rating changes within each simulation based on simulated results, including the bonus for playoff wins. For games where a starting pitcher is not yet known, we assume a pitcher of average strength will play.

Our Complete History Of MLB uses a slightly simplified Elo system that doesn’t take pitchers, travel or rest into account.6 Like the forecast, it will update after each game.

It’s early, but the Cubs are looking pretty good again. We’re looking forward to the rest of the season and hope you’ll follow along with us.

Footnotes

  1. The travel adjustment is calculated as (-MILES_TRAVELED^(1/3))*0.31, and the rest adjustment is DAYS_REST*2.3.
  2. Here are some more of our baseball Elo parameters:

    • The general K factor is 4, though it’s 6 for postseason games and also is adjusted based on margin of victory.
    • The difference in rating between two teams is multiplied by 4/3 for postseason games.
    • Expected margins of victory are calculated with elo_diff^3*5.46554876e-08 + elo_diff^2*8.96073139e-06 + elo_diff*2.44895265e-03 + BASE (BASE is dependent on the year and stadium the game is played in but has an average of about 3.4).
    • The actual margin of victory in each game is also adjusted for era/stadium effects and then flattened a bit with ((abs(margin_adj) + 1)^0.7)*1.41.
    • Thus, the whole Elo shift is (outcome – winprob) * (adjusted_margin / expected_margin) * (6 if postseason else 4).

  3. Starting pitcher adjustments give our model about a 1 percentage point improvement in the percentage of games correctly “called” and a corresponding improvement in mean squared error.
  4. Just like the margin of victory modifier, game scores are normalized for eras and stadiums so pitchers are directly comparable. They’re also adjusted to take the opposing team’s offensive strength into account.
  5. A pitcher’s Elo bonus is calculated with 4.7*(pitcher_score – team_pitching_score).
  6. This means that the ratings in our Complete History interactive won’t always match the ratings in our 2017 MLB Predictions, but using separate rating systems gives us the flexibility to alter our forecast methodology from year to year but keep our Elo history interactives unchanged.

Jay Boice is a computational journalist for FiveThirtyEight.

Comments