ESPN tries to build a better quarterback rating 
Posted by Michael David Smith on August 1, 2011, 5:11 PM EDT
New England Patriots v Detroit Lions Getty Images

Quarterback rating is a statistic every football fan knows but few really get. Why is it scaled so that 158.3 is a perfect score? Why does it only include passing, and not other aspects of quarterback play like avoiding sacks and running? Why does it penalize interceptions but not fumbles? Why does it treat a six-yard completion on second-and-5 the same as a six-yard completion on third-and-11?

The NFL adopted passer rating as an official statistic in 1973, and the quest to improve passer rating began almost immediately. Now ESPN is poised to take the biggest step yet toward an improved rating that will be both a better assessment of how good a quarterback is and an easier stat for average NFL fans to understand.

The new stat, which ESPN is calling the Total Quarterback Rating (Total QBR), will be unveiled during a TV special on Friday night. Developing Total QBR was a joint venture between some of the statheads at ESPN and NFL analysts like Trent Dilfer, Jon Gruden and Ron Jaworski. The bottom line, as one of ESPN’s statisticians put it in a press release, is, “If you want one stat that measures the totality of a quarterback’s performance, it’s QBR.”

Unlike passer rating, Total QBR considers everything a quarterback does except handoffs. Quarterback runs, sacks, fumbles and penalties, all of which are ignored by passer rating, are included in Total QBR.

Total QBR also includes the situation of the play, such as down and distance, field position and the time left in the game. So a quarterback who runs for two yards on third-and-1 will be rewarded in the system, while a quarterback who runs for two yards on third-and-10 will be penalized. And a quarterback who racks up a bunch of passing yards while his team is way behind in the fourth quarter, playing against a prevent defense, won’t be treated the same as a quarterback who racks up a bunch of passing yards in the process of helping his team build a lead in the first half.

One of the aspects of Total QBR that could be both a strength and a drawback is that it considers data that the average fan doesn’t have access to, like how far a pass travels in the air, and whether the quarterback was under pressure when he threw it. That could be a great benefit of Total QBR because it incorporates detailed information that only comes from film study, not from the box score. The drawback, however, is that it means fans can’t see for themselves exactly where Total QBR comes from — fans just have to trust that ESPN correctly measured how far the ball traveled, and correctly assessed the amount of pressure the quarterback felt on the play.

Despite all those extra elements that go into Total QBR, the stat is designed to be easier to understand than traditional passer rating. Toward that end, Total QBR is based on a scale where 100 is perfect and 50 is average. No more perfect scores of 158.3.

If you’re curious how Total QBR works in practice, Mike Sando of ESPN.com writes that the top quarterbacks of 2010, according to Total QBR, were Tom Brady, Peyton Manning, Matt Ryan, Michael Vick, Aaron Rodgers and Drew Brees. The worst quarterbacks, according to Total QBR, were Derek Anderson, Brett Favre and Jimmy Clausen.

That seems roughly correct, but then again you didn’t need an advanced stat to tell you that. And right now we don’t know all the details of exactly how this stat is calculated.

If ESPN is committed to this stat and is able to clearly and concisely explain it on the Worldwide Leader’s NFL broadcasts, then fans will quickly become familiar with it and it will soon become a staple of how we talk about quarterbacks. On the other hand, if the stat comes across as too convoluted — or if it doesn’t really seem like much of an improvement on the current passer rating — then this will all feel like a rather pointless exercise. We’ll be interested to see how it’s presented on Friday night. And we’ll be interested to hear what PFT Planet thinks.

How to identify NFL's best quarterbacks 
August, 1, 2011
By Mike Sando

The late Don Smith never claimed his passer-rating formula was perfect.

Quite the opposite, in fact.

"Some people call it a quarterback rating system, but that really is not what it is," Smith told me during a 2002 interview. "It’s simply a passing statistic."

I've actually defended Smith's rating system because the quarterbacks with the highest ratings -- Tom Brady, Philip Rivers and Aaron Rodgers led the way last season -- usually are the best quarterbacks. But there's so much more to quarterbacking than passing stats for touchdowns, interceptions, attempts, completions and yardage.

Game situations should count for something, and now they do.

With input from football people, including ESPN analyst Trent Dilfer, our statistical analysts have developed a 100-point ratings scale for quarterbacks taking into account advanced stats, game situations and relevant non-passing stats, including fumbles and sacks, to evaluate quarterbacks far more thoroughly. The methodology is complex -- one of the formula's key algorithms spans some 10,000 lines -- but the resulting "Total Quarterback Rating" (QBR for short) beats the old passer rating in every conceivable fashion. The ratings scale will debut this season.

I've been bugging the Stats & Information team for a sneak peak ever since learning former NBA statistical analyst Dean Oliver had joined our production analytics unit and was playing a prominent role in QBR development. Oliver, a Caltech grad with a Ph.D. in statistical applications, revolutionized how NBA teams use advanced statistics. Menlo College professor Ben Alamar, who has consulted with the San Francisco 49ers, is also part of the team.

Our stats team has been using game video to track stats relating to pressure, personnel, formation, game situation and more since 2008. The QBR stat represents a significant leap in harnessing those statistics for something more.

The old formula Smith created treated stats the same regardless of circumstance. A touchdown pass thrown against a prevent defense during a blowout defeat equals one thrown against pressure to win the game. A 5-yard completion on third-and-4 counts the same as a 5-yarder on third-and-15. A critical quarterback scramble, sack or fumble doesn't even factor.

"There is no way to statistically say how effective a guy is under fire," Smith lamented during our 2002 conversation. "None of that can be put into something like this."

Now it can, along with a whole lot more.

The QBR formula takes into account down, distance, field position, time remaining, rushing, passing sacks, fumbles, interceptions, how far each pass travels in the air, from where on the field the ball was thrown, yards after the catch, dropped balls, defensed balls, whether the quarterback was hit, whether he threw away the ball to avoid a sack, whether the pass was thrown accurately, etc. Each play carries "clutch weight" based on its importance to game outcome, as determined by analyzing those 60,000 plays since 2008. The stats adjust for quarterbacks facing an unusually high number of these situations.

"If it is a running clock late in the game, maybe you only get a few yards here or there, that is the right football play to make," Jeff Bennett, senior director of ESPN's production analytics team, said Sunday. "We spent a month learning about ratings to make sure quarterbacks couldn’t game the system, so they're not afraid to throw that deep pass at the end of the first half and risk an interception."

I've seen an outline for the rating system breaking down 2010 quarterbacks into six general categories, from top tier to poor. Precise rating numbers were not yet available. The quarterbacks under consideration broke down as follows:

• Top tier: Brady, Peyton Manning, Matt Ryan, Michael Vick, Rodgers and Drew Brees.


• Well above average: Josh Freeman, Eli Manning and Philip Rivers.


• Above average: Ben Roethlisberger, Tony Romo, Joe Flacco, Matt Schaub, David Garrard and Kerry Collins.


• Around average: Matt Cassel, Ryan Fitzpatrick, Mark Sanchez, Carson Palmer, Colt McCoy, Kyle Orton and Jon Kitna.


• Below average: Shaun Hill, Jason Campbell, Jay Cutler, Matt Hasselbeck, Chad Henne, Donovan McNabb, Sam Bradford and Alex Smith.


• Poor: Derek Anderson, Brett Favre and Jimmy Clausen.

Could be very cool. I like that they are trying to take into account all aspects of quarterbacking. The main problem I see is that no matter what your formula is for, people will probably argue about the way some aspects are weighed versus others. I've tried to make some formulas for quaterbacks and defenses before, and one of the toughest things is that you have to value every significant aspect. People will probably disagree about which statistics are most important. Still I'm very intrigued.

Originally Posted by: mcaldie 

Could be very cool. I like that they are trying to take into account all aspects of quarterbacking. The main problem I see is that no matter what your formula is for, people will probably argue about the way some aspects are weighed versus others. I've tried to make some formulas for quaterbacks and defenses before, and one of the toughest things is that you have to value every significant aspect. People will probably disagree about which statistics are most important. Still I'm very intrigued.

Originally Posted by: mcaldie