How Madden Ratings Are Made
The Secret Process That Turns NFL Players Into Digital Gods
graphics by Reuben Fischer-Baum
illustration by Mike McQuade
Hunched over a keyboard, surrounded by computer monitors, Donny Moore, 37, controls the fate of the National Football League. Its players throw as hard as Moore wants. They run as fast as he says they should. And the stars of America’s most popular sport aren’t always delighted by his judgments.
Carolina Panthers quarterback Cam Newton, for instance, was upset. “I want to talk about my speed,” Moore remembers Newton saying as he clambered into Moore’s cubicle last April.
Despite leading all NFL quarterbacks in rushing yards in 2013, Newton ranked as only the ninth-fastest QB in the league, according to Moore — hence Newton’s unhappiness. But as Moore wheeled around from his den of screens, he was confronted by not only Newton, but also an enormous boot on Newton’s foot, the result of recent ankle surgery.
“Yeah,” Moore said as Newton hobbled toward him, “let’s talk about your speed.”
Eventually, Newton was pleading with Moore to not make him slower.
In that role, Moore is tasked with assigning more than 40 numerical grades to each of the NFL’s roughly 2,600 players,2 evaluating them in categories ranging from passing accuracy to tackling ability. Moore’s process has largely been a black box, and yet it shapes how more than 5 million gamers simulate pro football — particularly because there’s no official alternative to his numbers. A decade after signing a controversial exclusivity deal with the league and the players union, Madden3 is still the only licensed NFL game in town.
In fact, an entire culture has grown up around Madden and its attempts to distill human athleticism into numbers. It is all good marketing for EA Sports but also speaks to the sway Madden holds. The ratings are a de facto time capsule from the year they were produced, a digital archive that offers players some measure of immortality in a sport where the average career lasts only a shade over three years.
The allure of the Madden rating might also speak to the relative lack of meaningful statistics in football itself. It would be strange for a baseball player to complain about his ratings in MLB: The Show, for instance, because a realistic baseball simulator (by necessity) has ratings grounded in actual statistics. But in a sport where objective measurements are often inadequate, subjective numbers — like those generated by Moore — take on greater currency.
All these factors put more pressure on Moore to produce ratings of ever-increasing accuracy even as they highlight the fundamental paradox limiting Madden’s realism: It’s nearly impossible to accurately simulate some players as long as a gamer must assume control of the athlete’s brain.
Madden has generated more than $4 billion in revenue over its 26-year lifetime. Yet for all EA’s
resources, Moore performs his czar duties
in surprising solitude.
On the early December day I visited his cramped cubicle, tucked away in a corner of an office building on the EA Sports campus near Orlando, Moore was putting the last touches on a roster update that would be downloaded later in the week by PlayStation and Xbox owners around the globe. When he was finished, 577 players saw some aspect of their skills re-evaluated on the basis of the previous week’s action. It’s an activity that Moore repeats every week of the season in an attempt to ensure that Madden imitates NFL reality.
Moore’s job has morphed from a behind-the-scenes technician to a sort of celebrity — and villain — in gaming circles. And in an increasingly data-heavy sports landscape, it’s a role that sits squarely in two intersections of growing importance — between scouting and analytics, simulation and reality. Because in both Madden and the post-“Moneyball” sports world, humans become a collection of data points and then are turned back into a digital approximation of themselves.
Translating the athletic skills of flesh-and-blood humans into digital form has been a necessary part of sports gaming as long as real-life players have been incorporated into the software. According to Good, that practice dates at least5 as far back as the 1984 release of “Micro League Baseball” for the Commodore 64.
“Micro League” was among the first sports simulations with a license to use Major League Baseball players on team rosters, an innovation that helped usher in the modern era of sports gaming. Developers were suddenly presented with a novel problem: having to represent actual players’ likenesses within the technical constraints of the day. One solution? Assigning numerical skill ratings to help differentiate good players from bad.
“They were transitioning from primitive video games that were just trying to approximate the basic acts of a sport,” Good told me, pointing to the Atari 2600’s sports titles as examples of early games without rated players.
“Basically, all players perform equally,” Good said. “They just wanted [to know]: ‘Do we have three outs and three strikes? You can throw the ball from third to first to put a guy out?’ ”
The method of “Micro League Baseball” was to algorithmically translate a player’s real-world statistical benchmarks — his batting average or home-run total, for instance — into skill ratings that would presumably6 spit similar numbers back out on the other side of the simulation. It was effectively the computerized version of older board games like All-Star Baseball whose colorful spinners reduced the essence of a ballplayer to a series of probabilities, locked in an eternal marriage with chance.7
But player performance is notoriously more difficult to quantify in football than in baseball. On the gridiron, detailed individual statistics are kept for only a handful of positions, and those numbers frequently miss the whole story because of interactions between 11 players on each side of the ball. Game developers quickly realized that football players, in contrast with their cousins in baseball, would need to be graded on a wider variety of skills — and that ratings-makers would have to temper the science with a whole lot of art.
This did not lend itself to a great deal of differentiation between players.
The assortment of skills that needed to be collected for each player grew as football games progressed through the 1990s. By the end of the decade, EA Sports’ NCAA Football series9 assigned 14 attributes to every player in the game,10 while Madden began handing out 17. With the advent of fifth-generation consoles, the degree of complexity in simulating football increased dramatically, requiring a wider range of player attributes to feed into the new game engines.
It was against this backdrop that Moore began his professional affiliation with EA. In 1998, he was still a student at the University of Central Florida, pursuing a degree in political science, when in November when he noticed advertisements for an “NCAA Football 99” tournament at the student union. Moore and his roommates were devotees of the game — he recalls elaborate house rules requiring that a witness be present every time a game was played in the friends’ shared Dynasty Mode savefile. On a whim, he skipped class, entered the contest using Florida State as his team, and won first prize.
Moore’s detailed knowledge of the game’s minutiae impressed NCAA Football developers on hand for the tournament. On the spot, they offered him a part-time job testing the coming edition of the game, a break he would later parlay into a full-time gig as an EA Sports football tester.
Playing and testing the games, Moore got a firsthand look at the give-and-take between gameplay and player ratings. He recalls the time a long-forgotten defensive tackle for the University of Kentucky inadvertently became a superstar after a typo assigned him a speed rating11 of 85 — blazing for a lineman — and when the Oakland Raiders, long known for their real-life fixation on speed, became unstoppable because the Madden game engine translated the overall velocity of the Raiders’ roster into far better virtual performances than the team was capable of in reality.
Moore would eventually be promoted to a designer and associate producer along his path to Ratings Czar, but the relationship between Moore’s testing background and his current job is clear. “The [ratings] are probably the single biggest factor in gameplay,” Moore said.
And the ratings have far more moving parts now than when Moore began working at EA. Each player in the game is graded in 43 categories — many of which were added when Madden transitioned from the sixth generation of consoles to the seventh. There are also nearly 20 new player-tendency tags, known as “traits,” that control specific player behaviors.
“Some designers and producers complain that there are too many ratings,” Moore said of his colleagues. “They ask, ‘How can we limit the number of ratings?’ But I would argue you make players more vanilla with fewer numbers.”
That’s why Moore wants to continue adding categories even if it makes his job progressively more daunting. “The developers and programmers hate this,” he said, “but it would be great to do more ratings.” He then launched into an only-half-joking suggestion that the game might add a long-snapping rating someday.
When Moore sits down to build a player’s ratings for the newest version of Madden, he goes through one of two separate processes, depending on whether the player is a veteran or a rookie. Each type of player offers its own challenges. Veterans have existing ratings from previous games, but the degree to which each of their categories must be changed is uncertain. Rookies, meanwhile, must be created completely from scratch.12
For the past few versions of the game, users can download Moore’s latest roster update every week of the season via Xbox Live or the PlayStation Network. That means a player’s ratings fluctuate based on how he plays each time he takes the field. To figure out whose ratings to tweak and by how much, Moore said he combines his observations and notes taken during games13 with subsequent film study, conventional statistics, and — increasingly — advanced metrics from outlets such as Football Outsiders (particularly for schedule strength) and Pro Football Focus.
“The big factor during the season is stat-based,” Moore said. “The advanced metric sites like Pro Football Focus, like the Football Outsiders, this guy named Ryan Riddle — [a] former NFL player who actually does a blog that has a lot of great information — those types of things bring out the snap-by-snap look on a player. And that, to me, gives a truer look in terms of a player's value, rather than the conventional ‘how many receiving yards’ or ‘what was the completion percentage for the quarterback.’ ”
Pulling all that data together, however, is when Moore’s instincts take over. Listening to him describe the process of rating a Madden player across all 43 categories, I began to realize that he has unwittingly adopted a sort of ad hoc Bayesian updating process.
He said there’s a bit of “What have you done for me lately?” in the ratings but that certain categories are fundamentally more or less prone to short-term adjustments — another Bayesian-sounding notion.
“Speed largely stays the same,” Moore said, “although when guys get hurt, I will make a change to [their] speed and agility” — a fact Cam Newton came to know firsthand. “Carry rating can be in flux if a guy fumbles a few times,” Moore continued. “Every position has certain ratings more impacted in-season than others.”
Meanwhile, a category like “spectacular catch” can change on the basis of a single play. Moore defended the policy, saying that this particular rating has little to no effect on actual gameplay — but it underscores the ad hoc nature of some of Moore’s changes.
Such modifications will take a veteran player through the end of the season. Moore makes additional adjustments over the summer based on age-related factors and other reports coming out of OTAs and training camps. “In the offseason, I look for guys trending up — young guys — or down — older players,” Moore said.
As for the rookies, their creation process dominates Moore’s spring schedule.
Starting around March, Moore begins sketching out the ratings framework for projected draftees using a series of templates for each position. Working with measurables from the scouting combine and pro days, he establishes a player’s ratings in categories such as speed and strength, which are almost directly tied to the drills performed by prospects in advance of the NFL draft. For instance, a 4.50-second time in the 40-yard dash will usually earn a prospect a speed rating of 89, barring particularly strong scouting comments about his fleetness of foot (or lack thereof).
At the same time, Moore dives into the various public pre-draft scouting reports for each player, marking down pluses and minuses when consistent themes emerge and applying those adjustments to the templates in areas where the raw combine measurables are less useful, such as coverage skills or play recognition.
Finally, after all the component categories are set for both rookies and veterans, the numbers are used to generate the fabled Overall Rating.
When I asked Moore about this, I expected him to decline to comment, citing a proprietary formula shrouded in secrecy. Instead, he cheerfully called up a spreadsheet containing the values that go into computing a player’s overall grade (and later e-mailed it to us). And somewhat anticlimactically, the overall rating is simply a weighted average of a player’s ratings across the 43 skill categories tracked by Madden, with different categories taking on more weight depending on the position.14
For example, the most heavily weighted categories for quarterbacks in “Madden NFL 15” are awareness and throwing power (each contributes about one-fifth of a quarterback’s overall rating), accuracy by zone (short and medium accuracy are both slightly more valuable than deep accuracy), and play-action passing. Together, those categories make up 89 percent of the overall grade for QBs, with the rest of the weight given to throwing on the run, agility, speed and acceleration.
The effects of the ratings on actual gameplay can be arcane for even the most seasoned Madden veterans. Perhaps no one outside of EA Sports has spent more time contemplating Madden’s inner workings than SB Nation’s Jon Bois, the creator of Breaking Madden, a football column that doubles as an absurdist meditation on the game itself. But Bois confessed that he still can’t quite figure out what some of the rating categories actually do for players on the virtual gridiron.
“There are definitely more obscure settings,” Bois said. “There’s a slider called ‘press,’ which I have no idea what it means. I set it to zero every time, and I still have no idea what it does, or what ‘press’ even is.”15
Bailey offered an illustration from franchise mode, in which a gamer uses a finite supply of experience points to boost players’ skills. “It can be unclear which ratings you want to increase,” she said. Spending experience points on awareness, for instance, will almost always increase a player’s overall rating — but for defensive players and even quarterbacks, that boost won’t cause them to play better when they’re being controlled in-game. Meanwhile, wide receivers can actually see tangible in-game benefits from a higher awareness score.
“I’ve heard it speculated that even the Madden team themselves don’t really know what’s going on,” Bailey said. “Because they’ve got years upon years of systems and code just layered on top of one another, where it’s not always entirely clear how they’re interacting.
“It’s kind of a crazy system.”
Whatever the virtues and drawbacks of Madden’s player ratings, though, they’re just a starting point. The game must still be played.
If you hang around the Madden studio enough, you’ll hear a curious phrase tossed around, a relic from games of a decidedly different genre and medium:
Although Madden takes pains to immerse gamers in the fantasy that they are controlling real NFL players, every interaction between two players in the game is also dictated by underlying probabilities, weighted by the various ratings of the respective players.
As Rex Dickson, Madden’s creative director, said: “The way the game works on a high level is that when two players run into each other and are about to do an interaction — let's say it's a ball-carrier and a linebacker — it's going to take several ratings and throw them into a formula and do a dice roll.
“The dice roll is how we get the randomness. So it's basically his break-tackle chance against your tackle chance and then a dice roll on top of that. The ratings mismatch is your modifier on the dice roll. If he's 10 points higher than you for his tackle rating, the dice roll is going to be favored in his favor to get the tackle versus the flip where it'd favor the break-tackle.”
After hours of hearing this kind of talk during our visit to EA’s studios, my colleague Walt Hickey had a realization: This is basically Dungeons & Dragons for sports fans.
If an obsessive following is a requirement for that comparison, consider Madden certified. The annual unveiling of Madden’s player ratings is an event unto itself. Beyond just Newton, many NFL players grouse to Moore about their numbers. (Kerry Rhodes, while playing safety for the New York Jets, made a YouTube video titled “WTF Madden!” demonstrating pretty conclusively that his throwing power rating was too low.) There are also sites where amateurs, apparently dissatisfied with Moore’s handiwork, aim to create their own competing sets of ratings. And the ratings have even been deemed to constitute “likeness” in lawsuits such as Parrish, et. al v. NFLPA.16
Bois told me that he thought the game does about as well as it can when it comes to converting real players’ abilities into a series of numbers. “If we were going to be entirely realistic, you’d probably have to have 1,500 different skill sliders” per player, he joked. “But this is a decent way to distill it into a thing that the layman can play around with and use.”
Bailey agreed. “It’s an inexact science,” she said, “but for the most part [Moore] gets it right.”
Yet for all of Madden’s quest for realism, there are certain aspects of football that the game still, and may always, have trouble simulating.
On one level, sports video games give regular Joes and Jills the opportunity to perform athletic feats that most could only dream about in real life — blazing runs, high-flying dunks, cannon-armed throws — and games have gotten pretty good at simulating these aspects of what it means to be a professional athlete.17 But sports video games will always struggle to accurately replicate the cerebral side of sports; that’s why it’s more difficult to simulate being Ricky Rubio than LeBron James.
There are certain player archetypes that have always been difficult for Madden to accurately simulate. The classic example is Michael Vick from “Madden NFL 2004“; Vick’s speed was overvalued in the game. But less egregious examples occur whenever a gamer takes control of a player whose real-life counterpart possesses immense physical gifts but lacks awareness. The converse is also true; players whose defining skill is football IQ — Peyton Manning and Reggie Wayne, for example — are inept in the hands of an inept gamer.
When I asked Moore what types of players Madden had the most difficulty simulating, two archetypes sprung to his mind. “There’s the Peyton Manning guy — not [Tom] Brady or [Aaron] Rodgers because they have strong arms18 — but the quarterback that’s pure decision-making, accuracy and touch,” Moore said. “And the crafty veteran wide receiver with the ability to find holes in zone [coverage], like Reggie Wayne or Roddy White.”19
As he is wont to do, Moore suggested the latter could be better simulated with even more ratings or traits. “We could rate if a receiver was on the same wavelength with his QB,” he offered. After all, Moore loves to add to Madden’s stable of ratings.
His pet example is passing accuracy, which was originally represented by a single rating but is now broken down into three separate classifications by distance. In previous versions of the game, the single rating was applied to the sureness of every throw, which failed to capture a big part of what makes each quarterback unique. In reality, some QBs have better accuracy on short tosses than deep bombs (think Brady), while others are better at judging long throws than short passes (Joe Flacco comes to mind). By splitting pass types into subcategories, Moore thinks he is better able to represent the characteristics that define different quarterbacks.
Greater rating detail has also recently enabled Moore to address one of the criticisms that has dogged sports games — and Madden in particular — from time immemorial: that fast players are fundamentally overvalued. (Regarding the tendency for digital speedsters to far surpass the skills of their real-life counterparts, ESPN’s Patrick Hruby once mused that “Oakland Raiders boss Al Davis should have been a game programmer.”)
Moore doesn’t shy away from that critique. “When you make a guy fast in a video game, he’s just a little too effective,” he said.
Take receiver Darrius Heyward-Bey, a former track standout who hasn’t exactly had a great NFL career despite ranking among the league’s fastest players. Heyward-Bey is just the kind of one-dimensional speedster whose avatar traditionally wreaks undue havoc in Madden, but Moore insists the revamped ratings have made it more difficult for those players to trick the game’s engine.
Bailey confirmed this development. “It was true for a long time [that] the little speedy guys could just get open, and that was that,” she said. “I don’t think that’s as much of the case anymore.”
But while speed can be mitigated, Moore granted that there’s no good way to overcome the problem of simulating a quarterback like Manning, whose most important skills — reading defenses, calling audibles, seeing things on the field that no one else can, and making sound decisions — are instantly negated when a gamer picks up the controller.
For example, one of the most important categories when computing a quarterback’s overall rating is awareness, which is based on experience and attempts to quantify factors like decision-making and depth of playbook knowledge. Awareness clearly affects how computer-controlled quarterbacks play the game; the lower the rating, the more likely a CPU opponent is to throw into coverage or miss open receivers. But despite its heavy weighting in the overall rating formula, a quarterback’s awareness score makes no difference when a human is controlling him. When you, the gamer, are in charge, it’s up to you to be “aware” for the avatar you’re controlling.
Bois described the same conundrum using a basketball analogy. “I’ve found that the video-game version of J.R. Smith is incredible,” he said. “He’s just way better than he is in real life, and I think the reason is because you get to make the decisions and not him. He’s an amazing player, but once you distill him to his physical self, he’s just unstoppable half the time.”
So at what point does an unaware Peyton Manning — or an aware Smith — cease to be a true representation of the original? It’s an interesting philosophical issue and one connected to the delicate balancing act of favoring skilled gamers while having virtual players mirror the strengths and weaknesses of their real-life counterparts.
“It's a fine line,” Dickson said. “You want to make sure that somebody with great stick skills can still win and be successful but at the same time not transcend a really crappy team and all of a sudden they can beat anyone.”
When that equilibrium is disturbed, the game can sometimes even be criticized as too realistic. In 2013, testers from Houston complained to Dickson after playing as the Texans’ undrafted rookie quarterback Case Keenum. “They all said, ‘[With] these computer physics, I can't have fun with this guy: Every pass is inaccurate,’ ” Dickson said. “We're watching film on [Keenum], and it's like, ‘Well, he's a third-stringer. Every pass is inaccurate. This is real life.’ ”
“They said to us, ‘Well, if I'm good and making good decisions, I should be able to make good reads and get the pass there,’ ” Dickson continued. “But all of a sudden then you're making guys like Geno Smith look like Peyton Manning, and your stick skills are now superseding the simulation. We've actually had that in the game before, and it didn't go over well. People reject that.
“You shouldn't be able, just because you're really good at Madden, to make Geno Smith all of a sudden awesome. Geno Smith, even if he makes a good read, is still an inaccurate passer. That's just the way it works in the NFL, and that's modeled in our simulation.”
Polygon’s Good used a baseball example to illustrate this tug of war between enjoyment and realism: “How do you make failing seven out of 10 times fun?”
It’s a challenge that doesn’t seem to be completely solvable using player ratings, no matter how much faith Moore places in them.
Then again, I doubt it will stop him from trying.
“In a perfect world, in ‘Madden 2037,’ we might have [ratings] split into 10 or 15 different awareness-type categories,” Moore said, still rattling off items from his wish list. “Like awareness around the end zone, awareness in the fourth quarter …
UPDATE (July 24, 7:15 p.m.): In July 2015, Moore announced his retirement as Madden’s Ratings Czar, leaving EA Sports to join the daily fantasy sports company FanDuel. Madden NFL 16, which will be released in August, will be the final game in the series to contain ratings curated by Moore.
CORRECTION (Feb. 25, 1:23 p.m.): A previous version of this article incorrectly characterized Tecmo Super Bowl’s rating system. The game kept 14 rating categories in total, not per player.
- It says so right on his business card. ^
- Including practice-squad members, free agents and other fringe players. ^
- The namesake of retired coach and commentator John Madden. ^
- Their equipment styles, for instance. ^
- Good rightly cautioned me about the imprudence of declaring anything in gaming a definitive “first,” given the variety — and, just as often, the obscurity — of games and platforms in the medium’s history. ^
- It’s impossible to say because “Micro League” didn’t actually have the capacity to track statistics from simulated games. ^
- Perhaps it’s no coincidence that in another region of the baseball universe, the field of sabermetrics was tracing a parallel path at roughly the same time. Both endeavors were seeking to distill a player’s on-field performance to its numerical core, albeit for somewhat different purposes. ^
- Acceleration, agility, hands, strength, speed, endurance, intelligence and discipline. ^
- Also developed by EA’s Tiburon studio outside Orlando. ^
- EA Sports’ ratings are on a 1 to 99 scale. ^
- Not counting the overall rating, which just serves as a composite of a player’s component attributes. ^
- There aren’t even college ratings to fall back on anymore; EA Sports put its NCAA Football series on hold in 2013. ^
- Typically watched on the NFL’s RedZone channel. ^
- In fact, the overall rating generally doesn't use all — or even most — of the 43 component attributes at its disposal. Many categories effectively receive a weight of zero. ^
- After a little digging, I found that the “press” rating measures a very specific skill: the defensive back’s ability to jam a receiver at the line of scrimmage while lining up in tight coverage. ^
- In that class-action lawsuit, a group of retired NFL players sued the NFL Players Association — and won — over royalties that the union received from sales of Madden games containing classic teams with unnamed players whose ratings bore a striking resemblance to the talents of those teams’ real-life members. The unnamed players had “the same height, same ethnicity, same uniform number, same position,” Good said. “Although those things are very substantial in identifying a person, it was also that they would perform as you would expect that player to have performed in real life.” ^
- As NBA Jam illustrates, there’s no limit to how much a video game can crank up the dial on pure athleticism. ^
- Brady and Rodgers boast throwing power ratings of 93 and 95, respectively; Manning sits at 85. ^
- Both Wayne and White boast speed ratings beneath the average of 90 for receivers in “Madden NFL 15”. ^