The Wall Street Journal's Michael Saffino wrote an article last week about the Falcons' aggressive tendencies on fourth down this season. In opponent territory, with 3 yards or less to go and more than 4 minutes remaining, the Falcons went for the conversion on 72% of their opportunities, the most in the league. They converted on 85% of those attempts. The review of the game's play-by-play indicated that Atlanta would have scored 21 points according to league-average field goal rates, but following their successful conversions the Falcons actually went on to score 51 points--a net of 30 additional points. And in 3 games, the points scored off of 4th down conversions made the difference in the win. Could this be the explanation for the Falcons' dramatic over-performance this season?
In this post, I'll attempt a few things. First, I'll discuss how we should properly measure the impact of 4th down decisions and measure the wisdom of the decisions themselves. Then I'll look specifically at the Falcons' 4th down situations and measure their impact on Atlanta's fortunes in terms of EPA and WPA. Lastly, I'll look at how all the teams did in terms of 4th down decisions.
Measuring the value of 4th downs
The wisdom of 4th down decisions must be judged based on the situation and information available at the time. The outcome itself should not be part of the equation when we decide whether a particular decision was correct or unwise. Additionally, it's not quite correct to count up the points scored on the drive subsequent to a fourth down conversion and then say that they made the ultimate difference in the score of the game. Had the Falcons kicked a FG instead of going for it in any of these situations, the entire game from that point forward changes. Each teams' risk/reward equation is affected and their strategies would change. We can't inject or remove points from a game and pronounce the outcome would have been just so. Things have to be measured probabilistically, and this is where Win Probability (WP) comes in especially handy. In fairness to the author, Mr. Saffino only suggests that the points may have made the difference in the end, which is the best one can do without the benefit of a WP model.
Measuring the actual impact of a 4th down conversion attempt is straightforward, at least in retrospect. We can measure the Win Probability Added (WPA) of the play or the Expected Points Added (EPA) of the play. In other words, the impact of any 4th down attempt can be measured by subtracting the WP of the game state prior to the 4th down play from the WP of the game state following the conversion attempt. The same concept applies with EPA.
Measuring the wisdom of the decision itself, before we know the outcome of the attempt, is more difficult. The best way to measure 4th down decisions is to compare the expected WP of a punt or field goal attempt with the total expected WP of the go-for-it gamble. To the degree that the total go-for-it WP exceeds the punt/kick WP, a coach forfeits that amount of WP when he punts or kicks. This how I have typically analyzed 4th down decisions, including this one in which the Falcons made a big mistake.
To summarize, there are two separate measures here. One is a retrospective measure of the impact of the 4th down plays actually attempted. The other measure is the value of the decision itself, analyzed prior to the outcome of the play.
The 2010 Falcons on 4th down
Looking specifically at the 2010 Falcons, it's true that they led the league in attempts under the specifications in the WSJ article--on opponent territory, 3 or fewer yards to go, and more than 4 minutes left in the game. (I think these are fair restrictions because they identify situations in which the conventional wisdom says it is somewhat reasonable to go for it. The 4-minute restriction is intended to filter out situations in which it's typical for trailing teams to go for it.)
The results of these plays added 0.90 WPA. In other words, in total, Atlanta's 4th down attempts created about one additional win in 2010. That may sound fairly impressive, especially for a league in which the difference between 10-6 and 9-7 can be enormous. However, all teams go for it on 4th downs sometimes, at least in some obvious situations, and the league-average WPA on 4th down under the conditions specified in the WSJ article is 0.20, making the Falcon's net advantage 0.70 WPA. That's impressive and should be cheered loudly, but it's not the 3 wins implied in the article.
In terms of EPA, the Falcons' 4th downs translated into 30.7 expected points (over and above FG attempts or punts), which is very close to the 30 net points estimated in the article.
What if we remove the restrictions of opponent territory, 3 or less yards to go, and more than 4 minutes to play? The Falcons 4th down conversion attempts brought them 0.93 WPA, while the league average was 0.52. That's a net of 0.49 WPA for the Falcons attributed to the outcomes of all 4th downs conversion attempts.
Impressive, but this is based on the outcomes, and Atlanta was very fortunate on their conversion attempts. What if we evaluate their 4th downs decisions, based on expected conversion success rates, prior to knowledge of the outcomes? Using the specifications of the article, when the Falcons punted or kicked in situations where the WP for a conversion attempt exceeded that of the punt or kick, they passed up only 0.09 WPA. This ties them for 2nd best in the league. But when we expand the data to consider all 4th downs, the Falcons forfeit 0.75 WPA when unwisely kicking or punting, which puts them square in the middle of the pack at 17th in the league.
All 32 teams
Now let's take a look at how all the teams fared in 2010 on 4th down. I'll give you two tables. The first limits the data to the specifications from the article, and the second is for all 4th downs.
Each table has several columns. The first data column lists the percentage of opportunities that each team attempted to convert 4th downs in situations in which the go-for-it WP exceeded both the kick and punt WP, an indication of how frequently each team went for the first down when it would have been to their advantage. The second column lists the actual net WP as a result of 4th down conversion attempts, and indication of how much each team actually benefited from its 4th down attempts. The third column lists the total WP forfeited when each team passed up a conversion attempt opportunity that would have been beneficial. The smaller the amount of WP forfeited, the more astute the team's coaching staff.
|Team||Go4It Rate (%)||Net WP Gained||WP Forfeited|
|Team||Go4It Rate (%)||Net WP Gained||WP Forfeited|
For the most part, teams with small amounts of forfeited WP are the teams with the fewer number of opportunities for conversion attempts. I could present the average amount of WP forfeited for each team, but I don't think that tells us much. The total WP forfeited numbers are still useful because they tells us which teams, and which kinds of teams, would have benefited the most from a more aggressive strategy.
It's interesting that less than a third of the WP forfeited was on plays inside the constraint of opponent territory, 3 or fewer yards to go, and more than 4 minutes left. Part of the reason is that it very often makes sense to teams ahead to go for it on 4th down within 4 minutes. Just like in the Patriots-Colts game from last season, a relatively easy 4th down conversion that seals the game is a better gamble than punting to a team that has all 4 downs to use and time to score the go-ahead points..
Another consideration is that it very often makes sense to go for it when there is far more than 3 yards to go. In no-man's-land, where a FG is an iffy proposition and a punt buys you very little field position, it can make sense to go for it on 4th and long depending on the score and time.
As always, the usual caveats apply. These numbers are based on league-average baselines. One other caveat is that these WP numbers are minimums. They assume a conversion success would get the exact number of yards required and no more. Realistically, they would often gain more yards than needed, making the conversion attempt even more lucrative.