WP Forfeited

It's 4th and 2 from the opponent's 42. The score is tied at 17. There's 1 minute and 17 seconds left on the clock. What would you do? If you're Ken Wisenhunt, you send in Dave Zastudil to punt.

One thing I've learned about human nature is that to help convince someone of something, I should frame the issue in terms of fearing a potential loss. That's usually a stronger motivation than expecting a potential gain. For example, I wouldn't suggest to a coach that he could improve his chances of winning by going for it on 4th down more often. Instead, I'd tell him he's forfeiting a significant chance of winning by not going for it. Nobody likes forfeiting stuff. My wife suggested using the phrase "leaving points on the table." Coach Wisenhunt forfeited 13% chance of winning that game.

By far, the most common question I get from reporters is whether teams are going for it more often. My answer is almost always "it's complicated, but I think so." The difficulty in measuring 4th down aggressiveness is that it's so dependent on the situational variables. You can't just count how often teams go for it rather than kicking. To-go distance, score, and time all weigh heavily on the decisions, and there are just too many possible combinations to compare rates from year to year or even decade to decade.

If we constrain the analysis to certain parameters--inside opponent territory and when the score difference is within reason, for example--we'd get an incomplete picture. We've learned over the last few years that there can be many situations outside traditional 'go-for-it' limits in which it can be beneficial to go for the conversion rather than kick or punt. And each situation can have a drastically different magnitude of effect on a team's chances of winning. Also, why would we reward a coach if he goes for it on 4th and whatever when he's down by 5 with a minute left to play? Coaches always do that.

Here's my stab at the problem. With every 4th down situation in which it would usually make sense to go for it but a coach decides to kick, he forfeits some amount of win probability. We can total all the WP forfeited to measure the degree to which teams are erring on the side of conservatism.

But first, let's look at a more simple measure. The table below lists each team in order of "go for it rate" for the 2011 regular season, which measures the proportion of times in which the offense attempts to convert when the numbers say go. For example, there were 44 situations in which the Vikings should have gone for the first down (at least according to my analysis). They went for it 16 times and kicked 28 times, for a 36% go for it rate.

 Team Went Kicked Go For It Rate MIN 16 28 36% TB 21 37 36% BUF 19 38 33% PHI 13 26 33% DET 12 24 33% SL 23 51 31% KC 17 38 31% CLV 18 42 30% JAX 15 37 29% ARZ 16 41 28% MIA 14 37 27% NYG 14 38 27% TEN 13 37 26% WAS 11 33 25% DAL 9 27 25% SD 8 24 25% GB 7 21 25% SEA 12 38 24% NYJ 12 39 24% SF 11 37 23% IND 10 34 23% ATL 11 38 22% OAK 11 41 21% CAR 9 34 21% CIN 10 39 20% PIT 7 28 20% NE 5 23 18% CHI 7 33 18% DEN 11 58 16% NO 4 23 15% BLT 5 32 14% HST 6 39 13%

But because every situation is unique in terms of distance, time, and score, we can't make any judgments about the aggressiveness or timidity of any coaches yet. The next table totals all the WP forfeited by each team on 4th downs when the numbers said go but the coach said kick. Also listed are the total number of 4th down go opportunities as well as the WP forfeited per opportunity. The higher the WP Forfeited number, the greater the sum of the 4th down errors. Click on the table headers to sort.

 Team WP Forfeited Opp. WP F. per Opp. ARZ 1.27 57 0.022 DEN 1.19 69 0.017 SEA 0.93 50 0.019 SL 0.91 74 0.012 CIN 0.84 49 0.017 CLV 0.81 60 0.014 DAL 0.74 36 0.021 OAK 0.74 52 0.014 MIA 0.74 51 0.014 HST 0.73 45 0.016 ATL 0.70 49 0.014 SF 0.69 48 0.014 TEN 0.67 50 0.013 BUF 0.67 57 0.012 NYG 0.66 52 0.013 CHI 0.66 40 0.016 KC 0.65 55 0.012 JAX 0.63 52 0.012 PIT 0.61 35 0.018 NYJ 0.61 51 0.012 WAS 0.52 44 0.012 CAR 0.51 43 0.012 PHI 0.51 39 0.013 BLT 0.50 37 0.014 NO 0.48 27 0.018 TB 0.48 58 0.008 IND 0.48 44 0.011 GB 0.43 28 0.015 NE 0.39 28 0.014 MIN 0.37 44 0.008 DET 0.32 36 0.009 SD 0.31 32 0.010

I should note that the WP numbers do not represent the actual outcome of the decision. In other words, the analysis is based on the probabilities at hand before the execution of the play, and not on whether the field goal was good or the punt was pinned inside the 5 or the conversion attempt was successful.

The Cardinals were the most timid this season. They cost themselves 1.27 WP by not going for it when the numbers suggest they should have. In other words, had they gone for it, the stats suggest they would have won 1.3 more games than they did in 2011. The Chargers were the least timid, costing themselves only 0.3 wins.

Overall, there were 1,492 squandered 4th down opportunities in 2011, totaling 20.8 WP lost. On average, teams squandered 46.6 opportunities at an average cost of 0.016 WP per opportunity--i.e. each instance cost a 1.6% chance of winning. The average team cost itself 0.65 wins this season due to its 4th down decision errors.

Last week Jim Armstrong published a guest column at Football Outsiders that used a methodology called "Aggressiveness Index" to measure how aggressive coaches are in 4th down situations. Jim found that, alas, based on his definition of when to go for it, coaches have not become more likely to go for it in situations they generally should be. Jim's Aggressiveness Index looks at plays in opponent territory and when the score difference is greater than 14 points in the 3rd quarter or 8 points in the 4th quarter. Although I'm not certain of the formula used for the index, the qualifying limits seem within reason.

And it turns out that WP Forfeited says Jim's findings were correct. Offenses are not going for it more often this past season. The last two seasons in particular indicate that coaches have forfeit an increasing amount of WP. Despite the long term trend that hints that 2010 and 2011 might be aberrations, it appears that there is too much year-to-year variance to say anything for sure.

The chart below shows the average WP Forfeited per go for it 4th down.

On one hand this is disappointing because coaches are not heeding the advice of analysts on 4th downs. On the other hand, it means there is still a competitive advantage available to coaches with the nerve to follow the numbers.

7 Responses to “WP Forfeited”

1. bmcosas says:

most bad coaching decisions, especially 4th down decisions late in game come down to the simple fact that coaches want to feel like they are still "in the game" -coaches will do anything to continue to feel like they have a chance to win even if it clearly reduces probability of victory

2. Jonpaul W. says:

If I recall, Armstrong suggested the infamous 4th-and-2 call by Belichick against Indianapolis in 2009 might have led to the less-aggressive ripple we've seen the last two years.

Unsure if I agree, considering your chart here shows respectable variation annually, but it is worth noting that on the whole the forfeited WP had been trending downward through 2009, after which it has strongly increased. Will be interesting to follow up the next few years. Perhaps there exists some natural range of WP forfeited that we won't see broken by the league wide grouping of coaches' tendencies.

3. Lab says:

Brian, I know you've posted before about how a team's strength or weakness on offense and defense can affect their willingness to go-for-it versus likelihood of punting. Have you ever formally cross-referenced those items? In other words, for example, measure somehow each team's strength of offense (perhaps average yards gained in short-yardage situations), and compare those numbers against each team's willingness to go for it in short yardage situations. I'd imagine that teams with strong short-yardage offenses are more likely to go for it than are teams with weak offenses. I suppose you also could measure the defense's strength in short-yardage situations too, but since the offense is deciding, I'd imagine its own strength carries more weight in the decision process.

Just a thought. Probably something you've done somewhere previously.

4. Anonymous says:

Are you going to post the win probabilities for the Super Bowl?

5. S says:

Did you look at teams going for it when they ought to kick, or does that literally never happen?

6. Tom says:

Superbowl analysis is where?

7. Boston Chris says:

Here's the 5th down writeup, looks like Brian forgot to post the link this week.

http://fifthdown.blogs.nytimes.com/2012/02/02/super-bowl-probability-giants-just-barely/