Skip to main content
Menu
The College Football Playoff Committee Is Answering The Wrong Question

Before losing to Alabama in the College Football Playoff semifinals, Notre Dame was widely considered one of the four best teams in the country. However, you’d have been hard-pressed to find anyone who thought Notre Dame could have been the best, given its blowout loss to Clemson in the ACC championship. So if Notre Dame clearly wasn’t the best, what was the point of including the Irish in the playoff at all?

[Everything About Sports In 2020 Was Weird … Except Who Won]

Under its own guidelines, the playoff selection committee picks what it deems to be the four best teams, which seems pretty straightforward. Outside of expanding the playoff, there’s another way it could make its selection — a way that is arguably more fair and that could increase the diversity among selected teams.1 Instead of picking the teams it deems most likely to be among the top four, which might include a team that has lost to a slightly stronger team and is therefore unlikely to end up as the best, the committee could instead pick the four teams that still have the best chance to be the best. While this sounds like a trivial change, it might have affected this year’s playoffs — and could very well bring new programs into the playoff.


Who’s the most chaotic fictional football coach? | FiveThirtyEight

To see how this is possible, we need a way to assess the probability that a team is indeed the nation’s best. Here I introduce one such way — a way that makes several simplifying assumptions, but it’s a start.

First, let’s make use of ESPN’s Football Power Index, or FPI, which takes into account both strength of schedule and margin of victory, and which has been shown to be predictive of game scores. To predict how two teams will fare against each other at a neutral site, the team with the higher FPI is expected to win, and the most probable margin of victory is the difference between their FPIs.

Now, for any given team, let’s assume we have no prior information about its FPI — nada, zip. Its probability distribution is uniform, equally likely to be -100, 0 or 100. Next, let’s look at how this team performed one game at a time. For example, if this team beats a team with an FPI of 10 by a margin of 20 points, it becomes more likely that this team’s true FPI was close to 30 and less likely that it was much lower than 30 or much greater than 30. If we play this out over the course of a season, we get a probability distribution for this team’s FPI.2 Once we have the FPI probability distributions for the different teams, we can calculate the probability of each being the best team.

[Losing Money Betting On College Football This Year? You’re Not Alone.]

Before we see how this plays out, it’s also worth noting that teams playing fewer games (like Ohio State, which went 6-0 on the season and played about half as many games as most other title contenders) had a slight advantage. Playing fewer games meant they had a wider FPI probability distribution, which increased their chances of being the best team overall.

With these FPI probability distributions calculated, it’s time for the big question: Prior to the playoffs, what was the probability that each school was the best? And would this have changed the playoff picture?

To figure this out, I ran 1 million simulations with the top 10 teams from the final College Football Playoff rankings (before the playoff semifinals) and recorded the share of simulations in which each team ranked first overall.

How likely was Notre Dame to be the best team?

Top 10 football teams in the final College Football Playoff ranking by their share of outcomes as the best team in 1 million simulations run using ESPN’s Football Power Index as of Dec. 22

Rk School Record Share AS BEST TEAM
1 Alabama Crimson Tide 11-0 85.02%
2 Clemson Tigers 10-1 8.29
3 Ohio State Buckeyes 6-0 5.13
4 Notre Dame Fighting Irish 10-1 0.04
5 Texas A&M Aggies 8-1 0.03
6 Oklahoma Sooners 8-2 0.77
7 Florida Gators 8-3 0.03
8 Cincinnati Bearcats 9-0 0.39
9 Georgia Bulldogs 7-2 0.31
10 Iowa State Cyclones 8-3 0.00

Percentages may not sum to 100 due to rounding.

Source: ESPN Stats & Information Group

According to this model, there was about an 85 percent chance that Alabama was the best team in the country before the playoff, which sounds about right. The next tier down included Clemson and Ohio State, each of which claimed between a 5 and 10 percent chance of being the nation’s best team.

The third tier of schools was where this model significantly diverged from the CFP rankings. Sure enough, Notre Dame was not fourth most likely to be the best team — that distinction fell to Big 12 champion Oklahoma (No. 6 in the rankings), followed by unbeaten Cincinnati (No. 8). Meanwhile, Notre Dame failed to distinguish itself from Texas A&M and Florida, which similarly had solid seasons but were not in strong contention for being considered the best team. Ninth-ranked Georgia was also more likely to be the best than Notre Dame.

[Ohio State Made The Committee Look Good — And Bad]

So according to these results, how well did the playoff committee do in picking its four teams? By selecting Alabama, Clemson, Ohio State and Notre Dame, there was a 98.5 percent chance that one of them was the best team in the country. That may sound pretty good, but the committee could have done better. If it had chosen Oklahoma over Notre Dame, its chances of having selected the best team would have been 99.2 percent.

Let’s return to the present. Now that the two semifinal matches (and all of the other nonplayoff bowl games) have been played, how are these 10 teams faring? With the championship game just days away, here are the updated probabilities that each team is the best in the nation:

Ohio State looks even better now

Top 10 football teams in the final College Football Playoff ranking by their share of outcomes as the best team in 1 million simulations run using ESPN’s Football Power Index as of Jan. 4

Rk School Record Share AS BEST TEAM
1 Alabama Crimson Tide 12-0 80.82%
2 Clemson Tigers 10-2 3.61
3 Ohio State Buckeyes 7-0 11.96
4 Notre Dame Fighting Irish 10-2 0.02
5 Texas A&M Aggies 9-1 0.02
6 Oklahoma Sooners 9-2 3.28
7 Florida Gators 8-4 0.01
8 Cincinnati Bearcats 9-1 0.16
9 Georgia Bulldogs 8-2 0.12
10 Iowa State Cyclones 9-3 0.02

Percentages may not sum to 100 due to rounding.

Source: ESPN Stats & Information Group

In all likelihood, Alabama is still the best team in the country (duh), while Ohio State’s chances of being the best now stand at close to 12 percent. Clemson and Oklahoma are next, at a bit more than 3 percent, followed by a more distant Cincinnati and Georgia, and then everyone else.

While the very best teams will always rise to the top — whether it’s a committee picking them or an algorithm — every year brings some new controversy over who deserves the final spot or two in the playoffs. And every year, it seems that the committee plays it safe, picking a legacy team that had a strong prior season instead of an undefeated up-and-comer who may be deserving of a shot at the playoffs.

Cincinnati may have narrowly lost its bowl game this year. But had the Bearcats won, we could have had another UCF situation on our hands, as in 2017 when the NCAA was forced to recognize two national champions. With the slightest shift in its decision-making process — picking the four teams most likely to be the best, rather than the four teams it perceives to be the best — the committee could make the playoffs more diverse while also increasing the chances that the title winner is the true premier team in college football. And isn’t that the point?


What makes LeBron James so great | FiveThirtyEight



Dr. Fauci on life post-vaccine and Biden’s approach to the pandemic | FiveThirtyEight

CORRECTION (Feb. 4, 2021, 6:02 p.m.): An error in the original simulations led to incorrect probabilities for the top 10 teams. Iowa State no longer had a higher pre-playoff chance than Notre Dame of being the best team, and Cincinnati fell behind Oklahoma in those chances. The playoff committee could have achieved a higher chance of selecting the best team by choosing Oklahoma, not Cincinnati, over Notre Dame. The text and both tables have been updated to reflect these changes.

Footnotes

  1. Just two programs (Alabama and Clemson) have accounted for 12 of the 28 playoff berths, while there have been only two appearances among Pac-12 teams (Oregon in 2014, Washington in 2016) and zero appearances by teams outside of the Power 5 conferences.

  2. One problem with this approach is that we’re effectively recalculating each team’s FPI in a slightly different way but using the original FPIs of all the other teams. We’re also dampening the effect of prior years’ results for each team. That’s intentional — if we want to depart from a system in which the same teams make the playoffs every year, let’s stop rewarding them for prior seasons.

Zach Wissner-Gross leads development of math curriculum at Amplify Education and is FiveThirtyEight’s Riddler editor.

Comments