Alex Smith QBR: Comparing Week 5 and Week 9

DISCLAIMER: This is a stat geek thread. It's late. I'm tired. I'm browsing stats. Please don't respond to this thread with "QBR sucks!!!" or "LOL didn't read!!!" I'd definitely appreciate some input from other open-minded stat geeks and people who have read the QBR guide and the FAQ.

I'm sure the QBR developers are well aware that 49er fans are their toughest constituency at the moment. The 49ers are 7-1 and Alex Smith's bust-to-resurgence story and very visible relationship with the obvious COY are both great stories. That stat geeks at ESPN caught flak for Eli Manning's impressive 2010 QBR despite 25 interceptions, but they wrote a pretty good justification here. With Alex I'm sure they'd point out primarily that we are winning almost exclusively because of our defense and running game, and they'd have a point. Still, they might not be thrilled that their system spits him out behind the likes of McNabb, Orton, and Moore. Just like they're probably not thrilled that it spits out Hasselbeck at #4 overall. Anyway, moving on.

Week 5: 48-3 win vs. Tampa Bay -- 11-19 (57.9%), 170 yards (8.95 avg.), 3 TD, 0 INT, 127.2 rating

Week 9: 19-11 win @ Washington -- 17-24 (70.8%), 200 yards (8.33 avg.), 1 TD, 0 INT, 109.7 rating

Week 5 QBR = 98.2

Week 9 QBR = 44.5

Now, I understand the slight difference in Passer Rating. It's been a while since I analyzed the formula, but I know it values TDs quite a bit, so having three versus having one is big. Still, both 127.2 and 109.7 are awesome ratings, so it doesn't bother me. The difference in QBR is astonishing for at least two reasons.

The first reason is the sheer magnitude of the difference. 98.2 is, I believe, the highest single-game QBR of any QB in the past three years. 44.5 is below average and put Alex at 18th in the league for Week 9. I am shocked at how two similar-looking stat lines could produce such wildly different QBRs.

The second reason I'm confused by this is because the QBR is supposed to do a better job than Passer Rating of accounting for situational factors and clutch. Against Tampa Bay, Alex's first TD made it 7-0. He didn't throw another TD until he made it 31-3 midway through the 3rd quarter; Rodgers and Gore did the damage in the 2nd. In a sense, Alex boosted his stats in garbage time. Against Washington, his only TD was again the game's first TD, and made it 13-0. Also, the throw to Miller was almost as good (and long) as the throw to Walker. All this is to say that I would have thought QBR would do a better job of reconciling the 3-to-1 TD disparity in these games than Passer Rating did, but the opposite is true. In fact, I'm pretty sure QBR is even supposed to account for being on the road.

Just for fun, let's bring in Week 8.

Week 8: 20-10 win vs. Cleveland -- 15-24 (62.5%), 177 yards (7.38 avg.), 1 TD, 0 INT, 98.8 rating

Week 8 QBR = 87.2, good for #3 in the league

If you compared Week 8 to Week 9, you have fewer yards and fewer completions on the same number of attempts and the same 1/0 TD ratio. It should be a no-brainer. Yet to QBR, Alex in Week 8 was almost as good as Week 5 and "twice" as good as Week 9.

A final word about sacks, another area in which QBR purports to be better than Passer Rating. The FAQ explains that their analysis over time indicated that QBs are "responsible" for about 53% of sacks as compared to the offensive line. Fair enough, I say. To QBR's credit, Week 5 had zero sacks compared to two in Week 9. But Week 8 had one sack, so I can't really accept that the extra sack contributed a bulk of the QBR difference. Also note that Alex didn't lose a fumble in either game, and that our 3rd down conversion rates of 2-7 (Week 5) and 3-12 (Week 9) were almost equally bad.

QBR is supposed to take a lot more into account than Passer Rating. So my question is this, for those who have watched the Bucs game more recently than I have or perhaps have watched the Redskins game two or three times: Is there anything legitimate you feel QBR may be capturing in this enormous difference? Did he have more egregious overthrows this in Week 9? Any boneheaded plays I'm not thinking of?

As a bit of a stat geek, I tend to be less hostile and more forgiving to QBR than most Niner fans. Though I think Smith is clearly better than the #24 QB in the league this season, I'm prepared to acknowledge that although he is getting there, Smith is not quite elite yet. If the wins and ~100 Passer Rating keep piling up for a year and a half and he's still behind Matt Moore, I may jump ship. I also recognize that less is asked of a QB in our offense than in probably any other offense in the NFL. That's more a compliment to our defense and running game than an indictment on our passing game. I'm probably far more confident in Alex's ability to take a game over if needed than QBR is, since it can't take it into account at all.

Finally, the reason I'm so determined to make sense of this Week 5/Week 9 discrepancy is because QBR has a whole lot going for it. It agrees that Rodgers is having an other-worldly season at 88.0. Brady was first last year at around 76, and this year's #2 and #3 are Brees and Brady at around 74 and 73. Those numbers all look right on track. Also, only one name below Alex on the list that makes me scratch my head. Check out this cast of characters: Gabbert, Tebow, Beck, Bradford, Painter, Kolb, Sanchez, Tavaris, McCoy, Grossman.

Bradford might surprise casual NFL fans who bought into his hype last year, but given the Rams' record and Bradford's efficiency down compared to last year, it makes sense. I think only Jets fans might have as big of a complaint with QBR as we do. I'm no Sanchez fan personally, but he has a 13/7 TD:INT and a respectable 84.0 Passer Rating good for 17th in the league. He also throws for 222 yards per game compared to Alex's 183 and has his team in a good position.

7 - 1

Originally posted by SanDiego49er:
7 - 1

Beautiful thing, isn't it?

Now I don't know a lot about the QBR rating but doesn't it take into account 3rd down conversions? I beleive the Niners were up to their old tricks on 3rd down converting will below 20%

Originally posted by miked1978:
Now I don't know a lot about the QBR rating but doesn't it take into account 3rd down conversions? I beleive the Niners were up to their old tricks on 3rd down converting will below 20%

We were 3-12 against the Redskins and 2-7 against the Bucs. By percentage they are almost equally bad, but I couldn't find out more just by looking at box scores. It's possible more of our 3rd downs this week "mattered" since the game was more competitive longer. It's also possible Alex failed on more 3rd downs he should have had. For example I'm sure missing on a 3rd and 3 pass is worse for QBR than missing on 3rd and 11.

Originally posted by miked1978:
Now I don't know a lot about the QBR rating but doesn't it take into account 3rd down conversions? I beleive the Niners were up to their old tricks on 3rd down converting will below 20%

but alex doesn't throw on every third down

I thought QBR statistics include the QB running yards which the Passer Rating stat does not. Alex Smith is a pretty good runner and uses his legs to pick up first downs and occasionally TDs. Alex's QBR should be higher

Fellow stat geek here too.

I think the answer the OP's question (what is QBR measuring that made the #s so different in the TB and Wash games) is that QBR focuses more on "win probability added", not TD themselves. Basically, every point in a football game can be shown as an expected point value. If SF begins a drive at the 20, then that drive might be expected to produce 1 point for SF (on average). If Alex throws a 20 yard pass, now it's 1st and 10 on the 40, and the drive can now be expected to produce 2 points on average, and so on. So that throw by Alex was "worth" 1 point.

Without getting too deep into the numbers then I think the QBR difference in the Wash and TB games comes down to this: in the TB game (especially early, when the game was in doubt) we didn't have great field position to start. Alex had to drive a long-ish field to score. And we did it mostly by passing – as I remember, we surprised everyone by throwing a lot to open the game. So Alex got a lot credit not just for the Delanie TD but also for the long drives that preceded it. Basically, in QBR's eyes we started a drive with a low expected point value, and Alex turned it into 7 points [almost] all on his own.

In the Wash game, we consistently had excellent field position (i.e. a higher initial point value for the drive) but didn't convert it into lots of points, so Alex got penalized for it. On the Miller TD, we started practically in the red zone after the fumble, so the drive would be "expected" to produce almost a TD. So Alex's throw was just an "average" play in QBR's eyes.

Many things wrong with QBR in my eyes. But this might be at least an explanation.

Love stats as well but know that if you give me some time I can show that Sopoaga is the best receiver in the NFL--statistically.

My problem with QBR is the built in subjectivity of trying to "Divide Credit" to assess how much responsibility each player deserves for a play. How close to the QB does the DL have to be before the OLine are partly responsible for the overthrow, dump off, Int? Do they have to touch the QB? If so, how hard? Should the rating be skewed when the QB is a huge guy like Big Ben? Does he get extra points for being hard to tackle? Same with Cam Newton?

I'm not on board this train yet but will keep on watching as it corrects and modifies and hope it can become more useful. Don't mind an emphasis on scoring though as that's the point of the game. Ints are more difficult to interpolate.

Originally posted by SanDiego49er:
7 - 1

this

Considering they rate players worth on fantasy impact as much as anything else now nothing surprises me. You would think listening to the talking heads that Cam Newton is 7-1 and Alex Smith is 2-6.

Originally posted by dtg_9er:
Love stats as well but know that if you give me some time I can show that Sopoaga is the best receiver in the NFL--statistically.

My problem with QBR is the built in subjectivity of trying to "Divide Credit" to assess how much responsibility each player deserves for a play. How close to the QB does the DL have to be before the OLine are partly responsible for the overthrow, dump off, Int? Do they have to touch the QB? If so, how hard? Should the rating be skewed when the QB is a huge guy like Big Ben? Does he get extra points for being hard to tackle? Same with Cam Newton?

I'm not on board this train yet but will keep on watching as it corrects and modifies and hope it can become more useful. Don't mind an emphasis on scoring though as that's the point of the game. Ints are more difficult to interpolate.

Good post. You echo some of my reasons for not being at all impressed with the current version of this method of rating QBs.

I applaud the attempt to delve more deeply into things a QB MIGHT do to increase his value to the team, however, in the effort, they have included several things that the QB does not have more than partial control. OTOH, other things that the QB DOES have full control over cannot be rated, such as, the audible adjustments.

Originally posted by walker807:
Considering they rate players worth on fantasy impact as much as anything else now nothing surprises me. You would think listening to the talking heads that Cam Newton is 7-1 and Alex Smith is 2-6.

Ironic that pro sports try so hard to control gambling by players/coaches but the entire injury reporting system is set up to PROMOTE gambling interests. Reporting injuries has ZERO impact on how the game is actually played. The only logical reason to report on Wednesday is to give the bookies more time to set the lines and promote their business.

Originally posted by dj43:
Ironic that pro sports try so hard to control gambling by players/coaches but the entire injury reporting system is set up to PROMOTE gambling interests. Reporting injuries has ZERO impact on how the game is actually played. The only logical reason to report on Wednesday is to give the bookies more time to set the lines and promote their business.

Hey, you just watched Damon Wayons say that in The Last Boy Scout !!!

I love this thread! Nice OP and great points throughout.