Lovie's Blunder

I'm hoping not to turn this site into 4thDownAnalysis.com, but I did get several requests to look at another controversial 4th down decision. (And I'll always be happy to take requests.) Down 14-13 to DET, CHI faced a 4th and goal from the 1 with about 9 minutes left in the 4th quarter. A FG is nearly automatic, giveing CHI a 2-point lead and a 0.59 WP.

Conversion attempts from the 1 are successful about 68% of the time. A successful TD gives CHI a 6-point lead and a 0.79 WP. A failed attempt leaves DET at its own 1, but with a 1-point lead, worth 0.44 WP for CHI. On net, the conversion attempt is worth 0.68*0.79 + (1-0.68)*0.44 = 0.68 WP. Going for it was the right call by a large margin--0.68 to 0.59.

CHI went for it, but Matt Forte was stopped for no gain. But that's only half the story.

The bigger blunder was the next 4th down. With 4:24 left in the game, still down by 1, CHI punted on 4th and 7 from the...DET 38! DET took possession at their own 14, for a net punt distance of a whopping 24 yards.

The punt left CHI with a 0.30 WP. A 4th and 7 conversion in that region of the field is successful about 42% of the time. A success is worth 0.65 WP, while a failed attempt is worth 0.26 WP to CHI. On net, we have 0.42*0.65 + (1-0.42)*0.31 = 0.45 WP. This was a big blunder, costing CHI 0.12 WPA.

Many coaches these days will go for it on 4th down from the opponent's 38 regularly, simply because kicking is pretty worthless in that spot. But to kick in that situation while losing by a single point, with time dwindling in the 4th quarter, is pretty amazing. Think of all the things that had to go CHI's way for them to win: they keep DET from running out the clock or scoring, then drive most of the field, just to get back to where they were on the 38. Although 42% is far from automatic, converting just a single 1st and 10 is just a 67% proposition. Besides, even if the attempt fails, CHI still has all the same options if they had punted, minus 24 yards of field position.

You have to think that the failure to score on the previous 4th down was probably in the back of Lovie's head when he punted there.

As luck would have it, CHI pulled out the win. But that was in-part thanks to a freakish (although technically correct) DET incompletion in the end zone as the final seconds of the game ticked away.

[Edit: Thanks to Dave and 'tgt' for pointing out an error in the orignial analysis that showed the error as a larger one (0.19 vs 0.12). The post has been updated with the correct numbers.]

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

31 Responses to “Lovie's Blunder”

  1. slushhead says:

    What do you think about "taking points off the board?" Specifically, should Washington have declined the penalty on the third quarter field goal?

  2. Alex says:

    My admittedly poor memory tells me that Chicago punted 4 times while in Lions territory. With those all put together I imagine it had to cost around half a win.

  3. Brian Burke says:

    You absolutely take the penalty and the 1st down. You don't need to go to the WP model at that point of the game--It's still a contest of net point maximization early in the 3rd qtr.

    A 1st down at the DAL is worth +4.6 Expected Points. A made FG is only worth +2.4 EP. (It's not 3 EP because you need to account for the subsequent kickoff.)

  4. Brian Burke says:

    Slush-Great question. Made it into a post. Might make into the Wash Post this week.

  5. Patrick Nance says:

    I know this is going to come off as being exceptionalist but the WPA everything's-in-a-vacuum model really isn't working for me when we're talking about the Lions on the road with a backup quarterback at the helm. Maybe the decision is closer than I thought but I simply have a hard time believing it's better than 50/50 in favor of going for it.

  6. Jeff Clarke says:

    A couple of points on the Redskins game...

    I was really dreading the "they took points off the board" theme that would have inevitably occured if the Skins lost.

    I find it extremely unlikely that the media would have given Shanahan credit if they got the TD and scored there.

    b. They would have been enormously idiotic if they punted instead of taking the fg with 2:00 left. Ultimately, the fact that Shanahan made the "gutsy" call was the reason that the Skins won. Its actually fairly easy for teams to drive the length of the field with 2:00 and all timeouts. I'm nearly certain the Cowboys would have gotten a fg if they punted.

    ps On the Bears move, I think you're assuming a TD would have given them a 6 point lead. At that point, the Bears would have certainly gone for two. That was probably part of Lovie's calculus in going for it.

  7. Brian Burke says:

    Good point about going for 2. Hadn't considered that.

  8. zlionsfan says:

    Coaching against a bad team with a backup quarterback wouldn't make you more aggressive? With the other team's offense expected to be even worse than with their starter in the game, you should expect them to have more trouble moving the ball against your defense, which should encourage you to go for it in more situations.

  9. Patrick Nance says:
    This comment has been removed by the author.
  10. Patrick Nance says:

    zlionsfan: I agree with you if we're talking about the first or second quarter but this situation was about halfway through the fourth. Clearly, Lovie Smith made the correct decision in terms of maximizing his point expectation, I just feel that, this late in the game, he was sacrificing too much in terms of win expectation. Certainly if there had been two minutes left (instead of nine), he should kick the field goal, right?

  11. John says:

    I also noticed this strange play when watching the game today. What is really strange is that last year, during Week 2, Lovie called a similarly insane play, when he ran a fake on a punt formation on 4th and 11 with a hand off to Garrett Wolfe at their own 74 yard line. It must be an annual tradition for him.

  12. zlionsfan says:

    Well, I think pretty much any coach would kick with two minutes left ... but even then, the problem is that you give the Lions the ball needing only a field goal to win. (And if the Lions still had Shaun Rogers, kicking a FG against them wouldn't have been a sure thing, either.) Without a full set of timeouts, you have few options, but with a full set, you'd still have a chance to force yet another three-and-out and get the ball back for another shot at the end zone.

    In the first situation, I think it was absolutely the correct call, even for a team that struggles in short yardage and the red zone. If they didn't get it, there was all kinds of time to get the ball back and try again (as they ended up doing). After all, the Bears needed just one score of any kind to retake the lead; even a safety would have done it, and of course in that case, they'd have both the lead and the ball.

    Sure, at some point, you do actually have to score to win, but I still think it's okay to try for more points with that much time left. I'm not sure how much the skill of the other team's offense would change my thinking ... after all, beyond a certain point, that becomes a threat you might feel you have to hold off. If they were playing the Colts, Saints, or Patriots, I don't know that a two-point lead would be safe at all.

  13. Patrick Nance says:

    I think we're pretty much in agreement about our general analysis of the situation (unsurprising, considering we both read a site called advancednflstats.com). Where we differ seems to be in how "good" of a scoring opportunity 4th and goal on the 1 is relative to the other scoring opportunities the Bears would go on to have. In my opinion, the chance they had right then was too good to pass up, justified, I think, by the fairly low score (14-13). In your opinion, though, the Bears would probably get at least one more solid scoring chance in the game and, at that point, it was more important to try to build an insurmountable lead.

    You are a gentleman and a scholar.

  14. Patrick Nance says:

    P.S. - Yeah, obviously my "kick the field goal" position is heavily dependent on the assumption that the Lions have a weak offense. Against a good offense that is always a threat to score, I'm trying for the touchdown very often.

  15. Bob Weber says:

    I'm curious if you've had any thoughts on analyzing end of half plays like the broken pitch play Dallas ran at the end of the 1st have of the Washington/Dallas game. Collinsworth and Dungy both made the statement that Dallas had 'nothing to gain' by not running a play at that point. I hate to argue with Coach (don't care about Collinsworth), but it seems to me that Dallas had 7 points and the lead to gain.

    I guess, my question is, is there any statistical evidence that the last play of a half is any more likely to have a larger chance of epic failure than any other play?

  16. Patrick Nance says:

    Bob: I'm a Cowboys fan and I'm actually in agreement with you. It's very easy to say, in hindsight, that the ball should've been thrown away or they should've just kneeled, but, really, how likely is a fumble-six in that situation? Is it more likely than a weird touchdown where Tashard Choice weaves his way through the defense (which, although I'd probably need to see one more replay, looked to have given him quite a hole when the pitch was made)?

    Clearly, it's unfortunate for the Cowboys that things happened the way they did but I think I probably describe that situation as purely "unfortunate" and little else.

  17. zlionsfan says:

    Thanks, Patrick, this has been a good discussion, and I appreciate your approach as well ... I think you're right, we're not really that far apart. It is an interesting puzzle, isn't it? A team with a terrible red-zone offense against a team with a terrible offense ... at what point do you have to swallow your pride and kick? (It feels weird to be "supporting" a Bears coaching decision, by the way ...)

    It would have been interesting to watch the game play out differently (and not just because of the stupid rule thing). If the Lions took the kickoff and went threw and out, giving the Bears a two-point lead and the ball, that might have worked out just fine for Chicago as well.

    I guess this is where we'd like a better idea of Chicago's conversion rate. Even if they only score 44% of the time, it's roughly a break-even decision ... but if they approach atrocious levels, like 25%, suddenly it seems like a much better idea to kick. It'd be interesting to see what their conversion rate from the one was last season!

  18. Jeff Clarke says:

    I think the conversion rate from the one would be a borderline ridiculous sample size. It seems to me like using NFL average stats was more appropriate here. The difference in single play expectations for good and bad teams isn't really that much.

    Nobody commented on the Buffalo intentional safety decision. That was a really odd one for me. They could have at least followed it with an onside kick. That might have made it justifiable. It was obvious they were highly likely to lose anyway, but it seemed like that took them from about 4% WP to maybe 1% or 2% at best.

  19. Dave says:

    I'm confused. How did the punt leave them with a 0.26 WP, while the failed 4th down would have given them a 0.31 WP, even though the failed fourth down would give the Lions better field position.

  20. tgt says:

    I'm confused by some WP numbers. How does a failed 4th and 7 conversion from the 38 leave chicago with a better WP (.31) then the punt to the 14 yard line (.26)?

    18 yards (gain of 6) to 34 yards (big sack) of worse field position nets .05 WP?

    Either there's a typo, or I have even less faith in WP numbers.

  21. tgt says:

    Apparently Dave & I are equally confused.

  22. Brian Burke says:

    Thanks. There was a typo in my WP table for the 4:30 minute entry for being 1 point up. It should have been .715 rather than .615.

    In case anyone is curious, the WP is based off a large series of tables. The tables come from regressed/smoothed data from winning % of actual games. The actual WP are generated by interpolating between tables.

  23. Brian Burke says:

    Ok. It should be .26 for the failed conv att and 0.30 for the punt.

    That makes it a net 0.42 for the conv att, and a 0.30 for the punt, changing the error from a -0.19 WPA monumental blunder to a -0.12 big blunder.

    I will update the post.

  24. zlionsfan says:

    Jeff, I think we agree that the NFL average is best for discussing the decision in general, but I was thinking about experience perhaps modifying the factors that would influence a coach's decision.

    Of course Chicago's offense last season isn't supposed to be that close to this season's offense, as anyone who's suffered - er, anyone who's seen Martz coach an offense can tell you. But there is a reasonable amount of data there, and it's a starting point.

    The Bears ran 62 plays last year in one-yard-to-go situations, converting 38 of them for a rate just over 61%. Sixteen of those plays were run from the opponent's one, and the Bears converted only five of those, 31%. (They were 2 for 3 against Detroit.) With Forte running, they were 2 for 7. None of those numbers are good, and if Smith had been thinking about last year, that might have suggested that he kick, but then again, that was last year.

    Prior to that drive, Chicago had a third-and-goal from the 1 that they failed to convert, plus three downs with 1 yard to go (one second, two third) in the middle of the field on which they were 2 for 3. So when fourth down on that drive came around, Chicago was 0 for 4 in attempts from the Detroit 1 in that very game, 2 for 7 in 1-to-go attempts overall.

    I still agree with the general decision to go for it from the one, but there was a growing body of evidence to suggest that perhaps the Chicago offense might be an exception rather than the rule, and of course after trying and failing, now that evidence has grown a little more. (Evidence to be used only for future Bears decisions, that is.)

  25. Jeff Clarke says:

    If the true NFL average is 68% from the 1, 5-16 represents about 3 standard deviations below the mean. Thats considerably higher than I thought it would be so perhaps the Bears are really substandard in goal line offense.

    Still, I tend to think that we are really overextrapolating small sample sizes. I suspect the number is an outlier and extreme regression towards the mean should be expected. I guess this would be an interesting topic for future study. Go back and look at which teams did significantly better or worse than the NFL average from the goal line in a year and compare their numbers to the NFL average for the next year. Exactly how much regression should you expect. I admit haven't done the study so I have no idea but my hypothesis would be that the numbers would say teams perform much closer to NFL average than their own previous years.

  26. Dan says:

    As for the goal-line decision, I wonder what the conversion rate from the 1 yd line is after three failed prior attempts from the same place.

    In other words, whatever the average conversion rate from the 1 yard line is, given that it was 1st and goal from the 1 and three plays later, it is 4th and goal from the 1... I would not use a broad average to guess the probability of converting on the next play. I would assume the conversion rate to be substantially smaller than the broad average because we now know that the offense was unable to score from the 1 on three consecutive plays. Why not use this information?

    OK, it would be hard to compile and use that information, but sometimes using common sense is better than using bad data. In this case, I believe that common sense would tell someone that their probability of scoring on the 4th and 1 play was less than 68%. Much less.

    If the probability of their scoring on any play from the 1 were 68% against Detroit, that would imply that their probability of not scoring on four consecutive plays would be 1%. I'm going to go out on a limb and say that what we witnessed on that series was not a 1/100 event :) In other words, after not scoring on three consecutive plays, Lovie should have realized that they did not have a 68% chance of scoring on the next play. I'm probably giving him too much credit, though--I doubt that he put as much rational thought into this as your analysis did.

  27. Anonymous says:

    Wait, you're telling us that your super-advanced WP model is hand-transcribed? That doesn't exactly instill confidence in its accuracy. Do you compute WPA with a slide rule and protractor, too?

  28. Brian Burke says:

    It's a circular slide rule, sorta like a decoder wheel.

    Actually, it's that parts of each table were compiled from different runs of regressions, and I copied some cells into the wrong column.


    Dan, that's an excellent point. I guess it's sort of Bayesian approach you're suggestion. The league-wide average is your prior distribution, then you revise your estimate with each subsequent trial.

    The biggest variable is where the ball is on the one--one inch line, or the 1.4 yard-line?

  29. Dan says:

    That's true, Brian (about the ball's placement around the 1 yard line). In this case, I think the ball was spotted at about the middle of the range you described.

    Another factor to consider is how dominant the Bears' defense had been since the Lions' backup quarterback had entered the game (or how inept the Lions' offense had been). Given that situation, a premium should have been placed on simply taking the lead rather than risking that lead by trying to take a larger lead.

    [Side comment: in the first overtime game of the season, the team that received the ball first lost]

  30. zlionsfan says:

    It could even be just inside the 2, couldn't it? My impression is that the official LOS is the 1 if the ball is just past the 2, but I could be wrong.

    Jeff, FO has done some research on backs (as opposed to teams), and they found that RBs significantly better or worse than the mean do regress toward the mean in the following season, so it would be reasonable from that standpoint as well to believe that Forte's goal-line runs this year would be more successful.

  31. xtimmygx says:

    I have a question about average punt distance from the 38 yd line. If it had been downed at the 1-yd what would the Bears WP have been? Much better than 0.30? We are analyzing the decision based on the result, something we all seem to be against, but perhaps it doesn't matter because there isn't too much of a difference.

Leave a Reply