Super Bowl 45 Analysis

I'm growing a little tired of all the Roman numerals.

Players of the Game

Aaron Rodgers was truly the game's MVP, leading all players with a 0.19 Win Probability Added (WPA). His 304 yards, 6.9 net YPA, and 3 TDs were good enough to produce 10.5 Expected Points Added (EPA) and ultimately a 6-point margin. Strangely, despite Ben Roethlisberger's two interceptions and failed final drive, he actually produced 11.6 EPA, slightly more than Rodgers did. This means, in terms of moving the ball and getting first downs, Roethlisberger played just as well as Rodgers. It's just that the way the game unfolded, BR was holding the bag at the end of the game. Rodgers' Success Rate was very low at 38.1%, while BR's was 47.8%, right where his season average was.

Had it not been for the Mendenhall fumble, costing -0.18 WPA, things may have unfolded very differently. As expected, the running game (aside from the fumble) for both teams was inconsequential. James Starks totaled -0.4 EPA but cost a net of -0.10 WPA. Mendenhall carried the Steelers for a TD drive, but his lost fumble more than washed it out, netting him -0.11 WPA.

Despite his drops, Jordy Nelson's 9 catches for 140 yards topped all receivers with 0.16 WPA and 6.4 EPA, making him a legitimate contender for MVP. Over 60% of Rodgers' EPA and 80% of his WPA came in passes to Nelson. It's very hard for a WR to do much better.

The tight ends were not a factor.

Linebacker Desmond Bishop was the defensive MVP of the game. Much of his game-leading 0.35 +WPA and 8.8 +EPA came from his fumble recovery, but nearly as much came from other less notable plays. Bishop led all front-seven defenders with 6 tackles plus 2 assists, including 3 tackles for losses, good for a 1.40 Tackle Factor (TF). Nick Collins was next, by virtue of his pick-six in the first quarter. Clay Matthews was third in +WPA mostly thanks to forcing the Mendenhall fumble.

Plays of the Game

The Matthews-forcedBishop-recovered Mendenhall fumble mentioned above was worth 0.18 WPA. Just as big was the 3rd and 10 31-yd pass to Greg Jennings with 5:59 to play, also worth 0.18 WPA. The leverage was huge there. A punt gives PIT all kinds of time and options to either play for the tie or the win. The successful conversion helped put GB up by 6, putting immense pressure on PIT on their final possession.

Next was the Collins pick-six, worth 0.17 WPA. Roethlisberger's 25-yd TD pass to Mike Wallace on 3rd and 3 was also big, worth 0.14 WPA. Jordy Nelson had the next two biggest plays. His 38-yd reception to the PIT-2 was worth 0.13 WPA, and his 29-yd TD catch in the first quarter was worth 0.11 WP.

Should the Packers have gone for it on 4th and goal from the 5 on their final drive of the game?

They opted for the easy FG to go up by 6. The Steelers would now need a touchdown, but often, forcing a team to go for the win rather than the tie can be counter-productive. This might be a little abstract, but channeling your opponent into a more aggressive, and likely more optimal risk/reward posture, might not be smart. In other words, even if GB fails on the 4th down TD attempt, the Steelers are left at or inside their 5 yd-line and are "thinking FG."

From the 5, conversions are successful 37% of the time. A successful conversion puts GB up by 10 points, sealing the win with a 1.00 WP. A failed conversion gives PIT the ball at their own 5 with 2:10 to play, worth 0.87 WP to GB. On net, the go-for-it option is worth a 0.92 WP.

FGs from the 5 are good 97% of the time. Going up by 6 and kicking off is with worth 0.75 WP. A missed FG puts the ball on the 20, worth 0.83 WP. On net the FG option is worth 0.75 WP.

WAIT! Did I just say that missing the FG would be better than making it? Yes, that's exactly what I said, and historically, that's exactly the case. The reason is likely because teams down by 3 play for the FG in that situation, while teams down by 6 are forced to play for the win. Once inside FG range, they pull up and stop taking risks, accepting a long FG attempt that, even if successful, only buys them a tie--0.50 WP. I suspect Tomlin would be thinking differently, so the answer to whether the Packers should have gone for it isn't so clear. But based on league-baseline numbers, and some counter-intuitive thinking, going for it would have been the better decision by large margin, about 0.15 WP.

The only thing crazier than going for the TD in that situation would have been attempting an onside kick after the FG.

If the Steelers were unprepared for it (and that's a big if), the onside attempt would have been worth +0.09 WP more than the deep kick. 

Up by 6 points, an successful onside kick seals the game. A failed attempt still requires PIT to score a TD and leaves GB with a 0.67 WP. When onside kicks are surprises, they are recovered 60% of the time. This makes the net value of the onside kick 0.86 WP. Kicking deep typically gives PIT the ball at their own 27, worth 0.75 for GB.

I doubt it would have been a surprise, given that Tomlin did the virtually same thing to McCarthy in last year's match-up between the two teams, making the deep kick almost certainly the right call.

Congratulations to the Packers and their fans. Again, we were treated to another great Super Bowl that came down to the final drive.

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

39 Responses to “Super Bowl 45 Analysis”

  1. Anonymous says:

    Can you analyze the Suisham miss?

  2. Brian Burke says:

    The Suisham miss was cost -0.07 WPA. With 15 yds to-go, attempting the FG was the right call there.

  3. Jeff Clarke says:

    I would have loved to hear Aikman's reaction to going for it on 4th and 5 there. I don't think there is a coach in the league that would have done it. I agree it should have been done but if McCarthy tried it and failed, its all anyone would have talked about for the next nine months.

    An onside kick would also been extremely cool to see in that situation. It almost certainly would have been a surprise. I think you might be reading a little too much into the surprise/not surprise question. Its not really a "could I ever picture someone doing this" question. Its really just a personnel question. Do I sacrifice my strong blockers for a bunch of tiny guys with good hands? As long as the receiving coach isn't playing onside defense, any onside kick would qualify as a surprise. Tomlin almost certainly had the normal guys out there. I'd love to hear the post game press conference if Tomlin had decided to sacrifice a return in that situation on the off chance that McCarthy would try an onside.

  4. Unknown says:

    Honestly, I think the way GB was playing the entire game, their defense would likely have held anyway. Although I understand that it put the Steelers in Aggressive Mode, taking away the field goal meant the Packers' only needed to stop them from reaching the endzone. There has to be a certain confidence level there, because (assuming the Packers failed to convert) playing to stop the FG puts you on your heels, as the field is shorter, and giving it up suddenly puts you at an even game.

    Additionally, having to go for a TD means more plays to run... and against a defense that's forced 3 turnovers, more opportunities for them to make a play.

    (Also known as J.B.)

  5. Dr I Don't Know says:

    What about the Packers' 4th and 8 or so from the Steelers' 38, right after the Suisham miss? I know it's probably asking a lot for McCarthy to defy NFL conventional wisdom and go for it there, but the data is pretty clear that a conversion attempt is better than a punt, right? Even though they did "pin" the Steelers at the 13 yard-line and force a three-and-out.

  6. Dr I Don't Know says:

    A related question might be: in that kind of 4th and medium/4th and long situation, near the enemy 40-yard line, how good does the punt have to be for it to be the winning play? My instinct is the cut-off is somewhere around the 10 yard line, depending on the exact situation, but it would be interesting to have a chart or so that related conversion confidence to punter-pinning confidence.

  7. Subhasis says:

    The other factors playing into this decision were PIT would get the ball with under two minutes to go and one time out. For PIT ont heir last drive, I think that the 4 yard completion to Hines Ward that took about 25 seconds off the clock would have been better as an incompletion.

  8. Anonymous says:

    Hi Brian. Two comments. When BR was backed up near his GL down by seven, perhaps he should have thought safety rather than airing out and risking the interception.

    Secondly, am I alone in believing that the PIT FG was the wrong play trailing 14-0 ? As now team has ever overcome more than 10, let alone the 18 it later extended to, I would have been very tempted to attempt the conversion. Not only that, but once PIT trailed 21-3, a 2-PtC was almost inevitable sometime in Q4, as events proved.

  9. Adam D says:

    Firdous, getting into certain field goal range from your own 5 yard line is not much shorter than receiving a kickoff and needing a touchdown. Consider that teams down by 3 kicking a field goal at the 35 yard line as time expires only have about a 20% chance of winning the game (I may have misused the WP calculator).

  10. Bryan says:

    Some glaring numbers from the WP calculator:

    All situations are with 2:00 left in the 4th quarter from the team's own 29 (team losing has possession, and WP is for that team)

    Tie game: .68
    Down 1: .42
    Down 2: .36
    Down 3: .21
    Down 4: .25
    Down 5: .25
    Down 6: .25
    Down 7: .11
    Down 8: .15

    Really interesting that teams are playing so poorly when down by 3 points. And it's not just at the 29 yard-line - it seems anywhere in their own territory, teams are generally giving up WP by simply being down 3 points rather than 4, 5, or 6. As Brian stated, this is probably due to being so risk-averse, and simply playing for the tie rather than the win.

    I'm not sure how to explain the increased WP for being down 8...it seems to be fairly persistent for different field positions as well. Maybe small sample size? Maybe the psychological effect on coaches and players being down 8 rather than 7 is enough to make them so much less risk-averse that they play more optimally?

  11. Brian Burke says:

    Bryan-
    To be perfectly honest, I'm not as confident on the 8-point number as the others. In the last off-season, I redid the model for the most common end-game scenarios to a much higher level of granularity. The 8-points down scenarios did not get that treatment.

    It could be true, but I'm not nearly as confident in it as the 1-7 points down numbers.

  12. Bigmouth says:

    In fairness, wouldn't Rodgers's Success Rate have been higher but for all those drops? I counted five or six more completions he should have had, one of which would likely have gone for a touchdown.

  13. Anonymous says:

    Hi,
    Any thoughts on Pittsburgh's decision not to go no huddle, shotgun until the final drive? Before that, down by 11 or 18, they were
    *huddling up
    *letting the play clock run down on every play
    *hiking the ball under center
    *running the ball
    *settling for short routes in the middle of the field
    How much did not going into the hurry up offense earlier cost them?

    In addition to the clock management issue, Woodson was out and two others in the GB secondary were hurt. With that kind of weakness, why wouldn't you try to exploit it on every play, especially with a no huddle offense that gives GB fewer opportunities to sub out players?

  14. Anonymous says:

    How much of a difference if the Steelers recover the Packer muffed punt at the start of the game? An early score by the Steelers would have changed the game significantly in my opinion.

    Great site by the way.

  15. Anonymous says:

    Recovering the muffed punt would have given the Steelers a 1st and 10 from the GB 21 with 13:13 to go in the 1st, which is good for a .64 WP

    The play as it stood (GB recovery) left the Steelers with a .48 WP

  16. Anonymous says:

    Can you analyze the Steelers' 2-point attempt in the 4th quarter, please? With 7 minutes to go, it seemed like the right call. But after GB tacked another FG on, Pit needed a TD anyway. Is the decision similar to the "down by 14, so go for 2 after the first TD" decision?

  17. Ian Simcox says:

    Wow, I thought going for it was the best choice but didn't realise it was by that much.

    I wonder if coaches of teams that are ahead undervalue overtime as much as trailing teams overvalue it (by that I mean that trailing team coaches seem to think that getting into overtime is a success when it's nothing of the sort, perhaps leading team coaches think of overtime as a failure). Hence why the FG, to make it a 6 pt lead, is so overvalued.

  18. Anonymous says:

    One other fairly big play came in the 3rd quarter when Tramon Williams a) failed to field the punt at the 40 or so, allowing it to roll 15 more yards, and b) got an unnecessary roughness penalty as the ball was being downed by the Steelers. It cost the Packers about 27 yards and dropped their WP from .73 to .68.

    That's assuming the yardage where Williams could have caught the punt is right, and that he really could have caught the punt. It seemed like he had a chance to fair catch it, though.

  19. Erik Agard says:

    From the way the announcers were talking, I would have expected Clay Matthews to be defensive MVP. Gut feeling is he played a slightly larger role than his +WPA indicates

  20. Anonymous says:

    This is why it's difficult to use WP as a decision-making tool. We have to assume that the opposing coach is incapable of making optimal decisions in order to believe that the Packers should have intentionally missed the FG.

    Is there really a large enough sample size to conclude that every coach is better off down by 6 than down by 3? Maybe Tomlin is one of the smart ones who knows how to play for the win.

  21. Bryan says:

    Maybe Tomlin isn't smart and will play for the tie. Maybe he's average and will play for the win, but only to an extent.

    The analysis is better interpreted as a baseline probability calcualted by aggregating all past events that have happened in equal or similar situations, rather than a series of assumptions about the coaching ability of Tomlin, offensive abilities of the Steelers, and defensive abilities of the Packers. We know that every situation is different, and that a WP or a conversion rate will almost never exactly apply, but it works as a baseline from which to make a decision.

    With a large enough sample size, I'd argue that the WP calculator can serve as an explicit example of coaching ineptitude in certain situations. It's not based on theory. It's based on actual, past events - mostly, the decisions of the coaches, and probably partly becaue of risk-aversion by players as well. If coaches had really been making the correct decisions (or if players really had been playing as optimally), the WP calculator would show it.

  22. Anonymous says:

    Actually, something like this is probably a step back for a statistic like WP reaching the mainstream...imagine trying to explain how Steelers fans should have cheered as Crosby's field goal sailed through the uprights - as long as Tomlin isn't too stats savvy.

  23. Joseph says:

    I thought that the mistake GB made was THROWING A PASS to try to get the TD. They got 2 completions in bounds to keep the clock running, but they could have achieved the same results with running plays (I think they gained 3 yds on those passes). I know they were going for the TD to try and win the game, and I applaud them for doing so. But had the clock run down to the 2 min. warning, then they kick the field goal, then they kick off, there is less time on the clock. As it stands, the kickoff caused the clock to reach the 2 min. warning, depriving PIT of a free time out.
    Brian, I know your WP calculator takes time into account, but does it take TIMEOUTS into account? IMO, even if PIT converts on 4th down, they do not have enough time left to make a TD to win the game--simply because they only had 1 TO left. For example, had they had 2, after the completion to Ward, they would have called timeout to save themselves ~30 sec. Wanting to save it for later in case they needed it, they burned more time than they could afford--and boxed themselves in.

  24. Anonymous says:

    If the difference between the Steelers recovering the muffed punt and the Packers recovering is 0.16 (0.64-0.48=0.16) then that play was the fourth most important of the game, right? Or am I misunderstanding WPA?

    It's only behind Mendenhall's fumble (0.18), Jennings 31 yard catch (0.18) and the Collins pick six (0.17).

  25. Anonymous says:

    When looking back to see the most important plays of the game, you can't compare what did happen with what could have happened. It really only makes sense to compare the WP prior to the play compared with the WP following the play. In the case of the muffed punt, the GB recovery probably barely changed the WP, if even at all. There's always the possibility of a huge game-changing play - had the muffed punt been recovered by the Steelers, then you can compare the .64 WP with their WP on the 4th down to see just how much of a swing the recovery caused. Needless to say, turnovers are usually the biggest WP changers, and the later in the game they happen (in a close game, anyway) the more sensitive WP is.

  26. Anonymous says:

    Thanks.

    If I've plugged this into the calculator correctly, the Steelers had a 0.47 WP thanks to a 4th and 10 punt from their own 36 at 13:19 of the first quarter.

    If they had recovered the fumbled punt on the Packer 21 at 13:13 their WP goes up to 0.64.

    That's a 0.17 improvement...the same as Collins pick six.

    A pretty big play that could have changed the game dramatically. Right?

  27. Anonymous says:

    A WP question: when I run the calculator for down 6 with 2:05 to go, it gives a FG percentage of .11. What does that mean? Field goals aren't actually kicked that often in that scenario, right?

    Also, I too would like to see the data with timeouts.

    Thanks as always for the interesting discussion.

  28. Anonymous says:

    To elaborate on my previous comment, here is what the WP calculator spits out for down 6 with 2:05 left and ball at your own 27:

    Win Probability: 0.24
    Expected Points: +0.72
    First Down Prob: 0.67
    TD Prob: 0.18
    FG Prob: 0.11

    Even if they somehow are kicking FGs 11 percent of the time, there is no way the TD percentage can be 18, and the win percentage can be 24, right?

  29. Anonymous says:

    Also, one final comment. Why are you using 1.00 WP for when the Packers score a TD? Based on your WP calculator, it should be 0.96, no?

  30. Anonymous says:

    I believe the TD and FG probabilities are strictly dependent on field position, and ignore all other constraints. Basically, the average team that starts with the ball on their 27 has an 18% chance of getting a touchdown and an 11% chance of getting a field goal. Adjusting the other other parameters do not affect these percentages...only adjusting field position will. I could be wrong though.

  31. Anonymous says:

    And to address the muffed punt...correct, had it been recovered by the Steelers it would have been +.17 WPA. As the play stood, it was only +.01 WPA.

  32. Anonymous says:

    Does anyone know what WP effect the PIT personal foul on the final kickoff had? It was a good return, but the 15 yd penalty really set them back.

  33. Anonymous says:

    It took them from .24 WP to .16

  34. Unknown says:

    Adam D, good point.

    (Also known as J.B.)

  35. Ian Simcox says:

    On the point about how being 3 points up is better than being 6 late, Easterbrook makes a valid point that teams up late in the game tend to switch to a prevent style defense, which essentially allows offenses to march down the field.

    Maybe it's not just that being 3 points down distracts the offense, it could equally be that being 6 points up gives the defense an unwarranted feeling of security.

  36. Brian Burke says:

    That's a really good point, Ian. It never dawned on me.

  37. Ian Simcox says:

    Just as a quick check, I've got that, with less than 3 minutes to go and the ball between their opponents 30 and half way (so edge of field goal range), offenses down by 3 points are sacked on 6.7% of pass plays. Those down by 6 points are sacked on 4.5% of pass plays.

    Of course, that's just one stat, and you could probably make a case for a multiple end point argument, but I do think it shows at least a little bit of evidence that agrees with the suspicion that defenses up by six points ease off.

    There's also a curious result that when the offense needing a TD is in it's own half, the sack rate is 7.1%, but once they move into the other half that drops to 4.1%. More of that easing off I suspect, as coaches stop calling blitzes because they don't want to be the ones who gave up a 40 yard TD pass on a blitz.

  38. Anonymous says:

    It appears that the Steelers offense was in fact slightly more effective than the Packers offense -- after all Pittsburgh's offense scored one more point than GB, gained more yards, more first downs, better 3d down efficiency, etc. So,I'm curious about any confidence that GB's defense would have stopped the Steelers in some of the comments above. It seemed to me that the last series by Pittsburgh was mishandled, more than anything else.

  39. Bruce Stram says:

    I'm a bit late to the ball because I didn't know about your great site.
    But anyway, one thing I think you have missed is the dropped balls by GB receivers. Obviously some dropped balls are to be expected, but I think its obvious that for whatever reason Packer receivers exceeded that number substantially. So the dropped passes over expected should be credited to AR's performance for the purpose of assessing his quality of play, even though they don't contribute to the win.

Leave a Reply

Note: Only a member of this blog may post a comment.