Comments on Advanced NFL Stats: The 4th Down Study - Part 4

Hi Brian, LOVE the post. Just found your blog for...

2012-02-06T17:14:47.371-05:00

Hi Brian,

LOVE the post. Just found your blog for the first time today and I already know where the next several days of my life are going.

I wanted your thoughts on how you would adjust the final graph for a team that doesn't have much that's average -- the NE Pats.

The offense is almost always in the top 5 for the league, and the defense in the bottom 5. If I understand your article right:

1) The top-tier offense has a better than average EP for all of their first downs. That makes 4th down attempts more valuable than average.

2) A defense that is 31st in yards means that the EP on a punt are reduced. That makes punts worse than average.

How drastic an impact do you think that has on the graph?

Would the line on the last graph shift up a couple...

2011-11-06T13:12:22.612-05:00

Would the line on the last graph shift up a couple yards for NCAA football? I know that NCAA football is more of an offensive game, and the kickers/punters aren't as good as their NFL counterparts. I think the breakeven point for going for it on fourth down would be much lower.

This is a very interesting post. But it seems to ...

2011-01-17T15:52:49.273-05:00

This is a very interesting post. But it seems to me that this analysis does not take into account the decreasing marginal utility of points in many situations. The three points of a touchdown are more valuable to a team than the additional four a touchdown would yield, right? This effect is probably stronger later in games rather than earlier. At what point in the game would coaches need to consider the nonlinear value of points?

thehurt-You should look up Kevin Kelly, a high sch...

2010-05-24T16:08:21.439-04:00

thehurt-You should look up Kevin Kelly, a high school coach who *never* punts. I've got some random links about him around here if you do a search.

As a high school coach, I'd be curious to see ...

2010-05-24T16:04:55.790-04:00

As a high school coach, I'd be curious to see how the numbers change for the high school game. Obviously our kickers don't bring the same leg to the game, so we don't average 40 yard punts or kick field goals from the 35, but our offenses and defenses tend to vary quite a bit more.
I suspect that the "Go for it" range would just be much wider, maybe ranging from about the -30 to the +15 or so, but I'd love to see if the math would back that up.

Many comments here about morale. According to w...

2009-12-21T11:00:03.177-05:00

Many comments here about morale.

According to what I've read, that high school coach that never kicks or punts does something else really important. He also spends considerable time convincing his players why not kicking is the best approach. This removes any morale aspect. His players all understand that, even when they don't make a first down, the decision is still the best one.

Dave Mc, this study is based on actual data, which...

2009-11-17T14:54:10.072-05:00

Dave Mc, this study is based on actual data, which in turn is based on the sum of all of the variables you claim he isn't accounting for. I agree further studies should be down against each variable seperately, especially time remaining in half as this is one variable that can cause number to change greatly from the mean.

Would it be possible for you to look at how indivi...

2009-11-16T16:56:02.871-05:00

Would it be possible for you to look at how individual coaches (or just teams, if that's easier) fare against this curve (i.e. what percentage of their fourth downs do they make the "right" decision on, holding the data to the 1st and 3rd quarters again)? It would be interesting to get a number for "lost points per season" for the more conservative teams.

I would think the underdog should be even more agg...

2009-11-16T15:22:16.159-05:00

I would think the underdog should be even more aggressive, while the favorite might be more inclined to do some kicking on close calls as that should lower the variance and increase the likelihood of victory.

I would argue the use of the term "Probabilit...

2009-11-16T13:45:16.661-05:00

I would argue the use of the term "Probability" when you're actually presenting mathematical "Possibility".
Even a fantasia rate of 70% leaves a failure risk of 30%.
That risk would be considered monumentally dangerous in any endeavor where loss is importantly avoided.
(for instance a vaccine given to a child that has a 1 in 3 chance of being fatal is an unacceptable risk)

The question as to why coaches don't make the attempts also ignores the local application.
A 4th Down attempt by the New England Patriots is not equivalent to an attempt made by the Kansas City Chiefs.

Left out of your demonstration, also, is the effect of failure on Time Management, Field Position, Point Differentials, and the intangibles of Morale and Rhythm.

I can only applaud your definition for one variable in the full of equation of the decision process to make the attempt.
They can't rely on what is possible.
They must rely on what they deem to be probable.

Very interesting analysis. The 37yard line repres...

2009-10-30T13:33:58.353-04:00

Very interesting analysis. The 37yard line representing the boundary between punts and field goals is interesting and equates to a 55yard FGA. Accepting these stats are purely from Q1 and Q3, a kicker's effective range must also come into play. KER however is not consistent as late in a half, the decision is going to be slewed by individual records at longer distances, side of pitch, wind/weather and altitude

We need to look at the graphs slightly differently therefore to obtain base criteria. It is pointless punting on or inside the 37 yet the FGA option other than late in a half is equally unattractive until we get down to an EP of around 1.0; let's call it the 31 or a 49yarder for sake of argument

Now we know that rushing on 3rd or 4th is no gimmie though there is a fair chance with less than two to go, so facing 4th and 4 or more, which in theory lies wholly within the go-for-it area of the graph in defensive territory, for sake of argument, depends purely on the confidence of offense to complete a pass long enough to pick up a FD. I imagine Def 2-6 is a no-brainer FGA in most situations due to the lack of room for receivers

On the face of it, therefore, 38 to 50 yards out is a punting situation unless c1.5yards-to-go (rush) or c7yards-to-go (pass) for it must be nett yards per pass play to allow for sacks, not average per completion. That said, average field position off a punt also needs to be factored in...

For 32-37 yards it is a no-brainer attempt-the-pass or possibly a short rush rather than the 50-55yard FGA, though this is influenced by the KER and other factors

For 7-31 yards/25 to 49yard FGAs, all other things being equal, IT IS BETTER TO ATTEMPT THE FOURTH DOWN CONVERSION with just under four yards to go, or roughly half the nett pass average

For 1-6 yards/19 to 24 yards, it all gets very blurred for a multiplicity of factors. Defense know that more than a yard out, a pass would be almost certain yet there is little room for receivers to work. In fact if a TD is need in this territory, a QB sneak or designed play is probably the best option

In conclusion, all things being equal, once range and other factors combine to produce a measurable probability of a FG being missed, rather than what could be construed a random event, such as a missed EP, it is better to attempt the fourth down conversion with less than four yards to go. This would not occur with a line of scrimmage on or inside the 6 nor unlikely and if our decision is based on an EP of 2.0 then we're looking tops around the 17 or a 35yarder. Being positive and unless the points situation dictates - and in Q1 or Q3 at least - with the LofS 16-31 (34-49 FGA equivalent), it is better to attempt the conversion when four or fewer yards are required.

Like the influence on overtime when the KO was moved back from the 35 to the 30, taking over at the spot of kick with a failure has also had an influence

Good catch. I did that for the opponent's goal...

2009-10-29T16:46:19.122-04:00

Good catch. I did that for the opponent's goal line (e.g. no 4th and 5s from the 2), but completely missed it for a team's own goal line.

Great work, discussion and graphs. Minor quibble ...

2009-10-29T15:56:03.377-04:00

Great work, discussion and graphs. Minor quibble with your final chart. I believe it is impossible to have a, say, 4th and 2 from your own five yard line (think about it). So rather than "go for it," there are some portions of your chart that should be shaded out as N/A territory.

Same goes for the other side of the chart, although it looks like you've half-accounted for it there.

I'm so glad to read this. For years, it has d...

2009-10-28T05:46:13.284-04:00

I'm so glad to read this. For years, it has driven me crazy to see teams (well, mainly my Vikings teams) settle for FGs when they have fourth-and-goal from the 1 or 2 yard line. I certainly never put all this mathematical or statistical analysis into it, but it just seemed intuitively obvious that:

(a) TDs are (assuming a successful PAT) worth more than twice as much as FGs;

(b) Even if you make that fourth-and-goal-short significantly less than half the time (which is kind of weak), you are pinning the other team way way back in their own end zone. Even if you don't get a safety out of it, on average given the ebb and flow of field position, this should be worth three points all by itself compared to kicking off;

(c) So in poker terms (and I sense there are a lot of fellow poker players about here), the attempt to get into the end zone from the one or two yard line is, at least, a freeroll!

-Alan

Hmm. I'd think I agree with Anon above, althou...

2009-10-16T12:28:43.517-04:00

Hmm. I'd think I agree with Anon above, although the language is confusing (defense scoring? You mean the team currently on defense getting the ball back and subsequently scoring I think).

I also think there's another factor in play, and that's "getting & playing with a lead". This is somewhat akin to the excellent "momentum hypothesis" commented here:
http://www.advancednflstats.com/2009/07/worst-4th-down-decision-of-2008.html?showComment=1252657875439#c8286330708448674093
in that teams perhaps psychologically, or even tactically, can play better when they have a lead, even if it's a relatively small one.

So let's call it the "taking a lead" hypothesis. I'd be interested to see that teams that decide to punt more often (to help avoid being behind) or kick field goals more often than they 'should' on 4th down in terms of pure EP do better in the long run.

eg. teams that take a 3, 6 or 10 point lead.

The trouble is that teams may well take a lead because they were simply better and going to win anyway, so maybe you're probably better to look at other performance indicators (defensive pass defense, 3rd down conversion on offense etc) before and after "playing with a lead" rather than just win rates. ie: do teams really play better with a lead, or is it just a perception/myth?

I know that if the strengths of a team are defense and running game, taking the "3" or punting do seem to become much better decisions than the 4th down gamble.

Sorry to be so late, and thanks for the correction...

2009-10-14T00:17:19.695-04:00

Sorry to be so late, and thanks for the corrections to my earlier postings.

I still think there's a flaw in the analysis. It doesn't look at comparable situations. For simplicity, assume the positive EP for a team means they score now, rather than first.
In two cases (punting and failed fourth down attempt), the analysis ends by looking at how many points the defense is expected to score. In the other case (going for it and making) the analysis stops with the offense scoring, and the defense doesn't get a try with the football.
FOr parity, it seems that you should subtract from the EP of the offense when 4th down is successful the EP of the defense when it receives the ball in average kickoff-return position.
If you remove the assumption about EP meaning scoring immediately, then life is more complicated. Perhaps one solution is to prorate the EP for the defense from the kickoff-return position (EP-D) by the probability that they didn't score first. For example, if the EP for the offense being successful on 4th down is based on the offense scoring first 60% of the time, then use 60% of the EP-D.
Another problem is time constraints. Since a positive EP may mean scoring on the 5th possession, the defense may never actually get a chance on a kickoff after the offense scores. One solution is to someone discount the EP-D, based on how likely they are to have enough time. Another solution is to use something more limited than EP, say a probable result of the current drive (either -- some number of points scored OR turning the ball over to the defense at a particular point). Then, under the assumption of the posting that time isn't a constraint, there will generally be enough time for two drives.

Someone posted earlier about how it's not good...

2009-10-07T11:43:34.064-04:00

Someone posted earlier about how it's not good strategy to go for it on 4th and 1 deep in your own territory late in the game with a 1 or 2 pt lead. I would think your WP would be higher going for it rather than punting. If you punt from your own 15 an average punt would give your opponent the ball around midfield. His chance of scoring a winning FG or TD would be pretty high(especially if that QB's last name is Brady or Manning). It sounds nuts, but a coach should always go for it in that situation.

Thanks, Brian. I agree that the consistency of 3r...

2009-09-30T09:46:29.698-04:00

Thanks, Brian. I agree that the consistency of 3rd and 4th down conversion rates does suggest that defensive teams don't change strategy to reduce the chance of "just enough" conversion gains. Or if they do, the offense is likely adjusting as well to maximize its chance of making that gain (sacrificing some probability of a long gain) and the two changes offset each other.

Still, I'm curious: is there any data on how often teams score from various distances within 10 yards of the end zone? It wouldn't only be 3rd and 4th down attempts that would be instructive, as teams are presumably trying to gain exactly that number of yards on each attempt (with no benefit of a longer gain). That's not precisely what happens on 4th down attempts, of course -- a long gain still has additional value. And the offense has other constraints at the end of the field. But it might give you another read on worst-case 4th down conversion rates.

Guy-Great point. I do have a great deal of 4th dow...

2009-09-30T00:18:48.650-04:00

Guy-Great point. I do have a great deal of 4th down data, unfortunately it's subject to considerable bias due to desperation situations and other factors.

I don't completely assume 3rd down = 4th down. I graphed both and compared them. For the most part, I can confidently say 3rd and 4th down conversion rates are equal. Where the 4th down data was extremely thin, such as on 4th and longs, I used 3rd down rates and adjusted them conservatively (lower) if there was a general discrepancy between 3rd and 4th. But those adjustments were extremely slight.

To answer your point fully, if a defense does pursue a different strategy on 3rd and 2, it is likely committing an error away from the Nash equilibrium, leaving an even greater opportunity for conversion to the offense. But in practice, we don't see an increase. Conversion rates are virtually the same. So either defenses are not adjusting strategies, or offenses are not taking advantage of the error.

Brian: I'm not sure it's fair to assume t...

2009-09-30T00:10:22.363-04:00

Brian: I'm not sure it's fair to assume that 4th down conversion rates would be the same as current 3rd down conversion rates, which are the basis of many of your conversion estimates. The incentives for the offensive team change at 4th down: the cost of failing to convert rises. At 3rd down a team is not only seeking a conversion, they may also take a higher-risk attempt at a longer yardage gain. But at 4th down (same field position), the value of getting just enough yards to convert rises relative to the value of a larger gain. And -- here's the trick -- the defense knows all this as well. So the defense should change strategy at 4th down, devoting resources to minimizing the probability of a conversion, even at risk of larger gain.

For example, this analysis says a team should go for it at 4th and 2 on their own 12. In that case, where preventing conversion offers a huge reward, wouldn't the defense pursue a different strategy than it might on 3rd and 2? Isn't it likely they would succeed in reducing the conversion frequency a bit?

I realize the 4th down attempt data is limited. But simply projecting the 3rd down data seems problematic. Maybe a look at success rates near the goal line -- how often do teams score on 3rd and goal from the 3? the 4? -- would provide some insight into likely outcomes in similar all-or-nothing situations on 4th down.

Ben-You are correct. Using the complete distributi...

2009-09-24T08:35:19.047-04:00

Ben-You are correct. Using the complete distribution would be more accurate. The EP curve bends slightly toward each goal line.

I chose not to use the full distributions for a couple reasons. First, it multiplies the complexity of the study several-fold. And two, the difference to the final results would be very minor. I wanted to make the study as straightforward as possible, so that even someone without a strong math background could grasp it.

Hey, I posted this yesterday in the part 2 comment...

2009-09-24T00:35:53.072-04:00

Hey, I posted this yesterday in the part 2 comments but realized that this seems to be the live commenting page for the whole topic.

In part 2, when you say "a punt from a team's own 40 (60 yds from the end zone) nets around 37 yards, giving the opponent a 1st down at their own 23. This corresponds to 0.5 EP for the opponent, which is -0.5 EP for the punting team," are you just using the straight average net punt yardage from a team's own 40? I don't know how drastically this would affect your values, but for a more rigorous calculation you would need to use the probability distribution of net punt yardage to calculate the EP of a punt from, say the team's own 40.

For a simplisitic example, when punting from your own 40, if there were a 30% probability of a net of 45 yards and a 70% chance of a net of 30 yards, then your EP would be -(.3 * (EP from your own 15) + .7 * (EP from your own 30)) which would not equal the EP of the average (EP is not linear).

Again, I don't know if you did this calculation behind the scenes, or if there simply isn't enough data to generate comprehensive distributions from every yard line (I would think there probably is, given the number of coaches who punt from inside opponents' territory).

LNG-Good question. Fist, keep in mind this study i...

2009-09-23T13:50:11.427-04:00

LNG-Good question. Fist, keep in mind this study is intended to outline a general baseline for "normal" football drives--when teams are not very far ahead or behind and when the clock is not yet a factor.

2nd and 4th quarter data is excluded because that's when the clock comes into play. The 4th quarter is also when teams start to become over-aggressive and over-conservative depending on the score.

Because I have such an abundance of data, I can afford to narrow the data to the 1st and 3rd quarters, and still have very reliable results.

For situations outside of "normal" situations, we have to use a different kind of analysis. In the Zorn article, for example, I don't use an (Expected Point) EP analysis like this one, but a win probability (WP) analysis. WP can account for the particulars of time and score.

Can you explain the effect of only using first and...

2009-09-23T13:02:09.984-04:00

Can you explain the effect of only using first and third quarter data? Especially when, for example, you use this study to defend Jim Zorn for going for it late in the game, precisely in a situation where you have clock issues or teams getting desperate. How do the numbers change? They just get a little less certain, with the same approximate results? Or is there a noticeable shift in the results?

Hey Brian, Great work here, and you really explai...

2009-09-22T14:40:33.565-04:00

Hey Brian,

Great work here, and you really explain what you're doing well. Transparency is very important when doing statistical analysis. I had a few comments about your analysis, and what some of the commenters have mentioned.

1. I think you should augment your EP when making decisions to take into account the emotional effects of going for it on fourth down. When a team goes for it on 4th and doesn't get it, there is a huge emotional gain for the other team. There probably is enough data to say EP(yard line X | previous play was a turnover on downs) I think this will give a more accurate measure of when to go for it.

2. Point 1 may be counterbalanced by something a few commenter have mentioned, which is that if a coach actually implements this and goes more aggressive, it changes his play calling on second and third downs such that his EP in different situations may go up, making it more worth it to go on 4th.

3. Lastly, to commenter who have mentioned that you have to take into account your own teams skills when making these decisions, of course you do. This analysis is only there to show what to do when an average team is playing an average team in all respects, and to show that coaches on average may be too conserative. If you have a punter that can consistantly blast 80 yard punts, or has the ability to drop a punt perfectly at the 5 yard line consistantly, your decisions may deviate from these conclusions. Similarly if you're facing a team where you can get 2-3 yards at will while rushing, your decisions may deviate in the other direction.