In the Buccaneers-Redskins game this past Sunday, the Redskins were able to score a potentially game-tying touchdown at the end of regulation, only to fail to hit the extra point due to a mishandled snap. Gregg Easterbrook suggested the Redskins should have gone for the two-point conversion, which is a plausible strategy in many circumstances. But Easterbrook went on to add this little tidbit: "Rushing deuce attempts are about 65 percent successful in the NFL -- a better proposition than the 50/50 of advancing to overtime."It's well established that 2-point conversion attempts are successful slightly less than 50% of the time, so could the 65% number for runs possibly be true? If so, what would that mean for NFL strategy?
There have been 718 2-point conversion attempts from 2000-2009, including playoff games. Overall, they've been successful 46.3% of the time. But this is slightly misleading because it includes aborted kick attempts. If we weed those out, along with some other mysterious plays, such as Josh McCown's kneel-down while trailing by 5 points in the final few seconds of the Cardinals-Vikings 2003 game, we get a different answer. For all normal 2-point conversions, the success rate is 47.9%.
Now look at the success rate broken out by play type:
| Play Type | Success Rate | Attempts |
| Passes | 43.4% | 525 |
| Runs | 61.7% | 183 |
Running plays have been successful nearly 62% of the time. So Easterbrook's number is very close, the difference likely due to the span of years he looked at.
This is a classic football game theory problem. Despite being significantly more successful than passing, running plays are much less frequent. It's clear that offenses should be running more often and defenses should be more biased against the pass. We can't tell exactly what run/pass ratio is optimum from these numbers, but the equilibrium will occur when the success rates for running and passing equalize.
Unfortunately, it's not that simple. Many of those runs are QB scrambles or bootleg options. If we remove all the plays in which the QB is credited for the run and just look at conventional runs we get the following:
| Play Type | Success Rate | Attempts |
| Passes | 43.4% | 525 |
| QB Runs | 74.5% | 47 |
| RB Runs | 57.4% | 136 |
QB runs are successful a whopping 75% of the time, but the picture is still muddled. Often QBs will sprint for the end zone only when they sense they have a good chance, so there's likely a considerable amount of bias in that number. And only when they happen to be tackled between the 2-yard line and the end zone would the attempt be recorded as an unsuccessful run. If they're tackled behind the line of scrimmage it probably gets recorded as a sack, which is considered a pass play.
Still, even after removing all QB runs, conventional RB runs are successful 57% of the time. This suggests that if teams ran more often, the overall success rate would increase. Defenses would likely respond, and eventually the success rates for both running and passing would equalize at a success rate somewhat over 50%.
If true, this would mean that going for the 2-point conversion is a net positive expected-value play. In 2009, the success rate for extra point kicks was 98.3%, and so far in 2010 it's 98.8%. So for 2-point conversions to be the higher expected-value play, it would only need to be successful about 49.5% of the time. A strategy mix that's heavy on running would almost certainly exceed that rate.
These results tend to confirm the previous finding that teams should generally be running more often inside the 10-yard line.
So should coaches go for 2 more often than not? Perhaps. The score and time remaining would ultimately dictate the strategy in each situation, but as long as the game is a point-maximization contest, which is usually until the end of the 3rd quarter, I'd say it's good idea. And in the end-game, when an extra point ties, but a 2-point conversion takes the lead, it would almost certainly be a good idea, all other things being equal.
Think how much more exciting every touchdown would be if there was almost always a 2-point conversion attempt. Unfortunately, I doubt coaches would do it. They're so risk-averse, it would take a success rate considerably higher than 50% to convince them to adopt a more aggressive strategy.

How often are there sacks on 2-point conversions? There's absolutely no reason to take one since there's no risk to fumbling or throwing an interception.
ReplyDeleteIronically, re Easterbrook, it was the Bucs who employed this exact strategy in 2005 agains the Redskins, running Mike Alstott on a 2-point conversion for the win very late in regulatino.
Something to consider as part of an underdog strategy? Given that the expected payoff isn't massively more than just kicking the PAT (0.16 pts per TD), the only difference is in the variance of the two options. Given your work saying that underdogs need to be on a higher variance strategy, it seems they should be going for it and it's the favourites that should be just kicking the PAT.
ReplyDeleteI agree with Ian, here. It's more variance than payoff at its core. Underdogs should go for 2 more often, big favorites probably not.
ReplyDeleteAnd in a game involving teams with good offenses but bad defenses, the teams should go for 2 more often...
ReplyDelete"Defenses would likely respond, and eventually the success rates for both running and passing would equalize at a success rate somewhat over 50%."
ReplyDeleteExcept you don't really know this. The game is far too complex to just assume they'll split the difference. It's far more reasonable to assume that the current success rate for 2PT conversions is the equilibrium outcome and that a change in offensive strategy--and a response in defensive strategy, would result in little if any departure from the current rate.
Isn't there a sample size issue with so many fewer runs than passes?
ReplyDeleteExcept you don't really know this. The game is far too complex to just assume they'll split the difference. It's far more reasonable to assume that the current success rate for 2PT conversions is the equilibrium outcome and that a change in offensive strategy--and a response in defensive strategy, would result in little if any departure from the current rate."
ReplyDeleteIt's not an equilibrium. It can be exploited. The fact is, it's not being exploited. Once it is exploited, we would EXPECT defenses to respond. When Team A starts going for two-point conversions every time, and chooses to run every time, the defense SHOULD respond by expecting the run, therefore decreasing the success rate of the two point conversion by running. If both offense and defense play optimally, there will necessarily be an equilibrium. Success rate for pass and run will be equal. This doesn't necessarily mean the number of pass attempts vs. number of rush attempts will be equal, but the conversion rate should be.
If the success rate for TPC is not equal, teams are failing to exploit opportunities.
J-Doug-I don't agree. The payoff values for running and passing are equal (a success = a success). So shouldn't success rates be equal at the minimax?
ReplyDeleteTo clarify, I'm not assuming they'd split the difference. However, we do know the equilibrium must be somewhere between the 2 success rates, and very likely be higher than the 49.25% required to be break-even for going for 2.
Steve-You're correct. Only 12 sacks in the sample.
Bigmouth-Sample size may be an issue with relatively few runs. The SE for n=136, p=.57 would be +/-.04%. So depending on how you look at it, things are 'significant.' Passing and running are >2 SEs apart, but running is not quite 2 SEs from the overall rate of 48%.
Shouldn't the midpoint be in proportion to the types of play calls? Assume the QB runs are pass plays:
ReplyDeletePass: 572 plays, 46%
Run: 136 plays, 57%
Therefore, I'd assume something closer to 48% as the equilibrium point as opposed to 50-51%.
"It's not an equilibrium. It can be exploited. The fact is, it's not being exploited. Once it is exploited, we would EXPECT defenses to respond. "
ReplyDelete"and very likely be higher than the 49.25% required to be break-even for going for 2."
Well, it is by definition an equilibrium--if there were no equilibrium there wouldn't be anything to exploit. It may not be a stable one, but I see the analysis in this blog making a very large leap to assume that, after both offense and defense adjust, that the success rate would exceed break even.
"The payoff values for running and passing are equal (a success = a success). So shouldn't success rates be equal at the minimax?"
The payoff value isn't the issue here. The issue is that the success ratios that you're measuring are dependent in part on the choices and expectations of the defense.
Based on the data you're working with in this post, there's no reason at all that the real number should be in between the current success rate and the current success rate for 2PC RB runs, because that number is dependent on the fact that the defense has a certain degree of expectation that that the offense will make that play
On the contrary, if the offense in any way signals an increase in the probability that they will hand off to the RB on a 2 pt attempt, one should expect the success rate to drop significantly irrespective of the current values for success rates.
Put it another way:
ReplyDeleteThe real success rate of an RB run on a 2PT conversion should be equal to P(A)*P(C)+P(B)*(1-P(C))
Where P(A) = Probability of reaching the end zone on a 2 yd run at the 2 yd line when the defense knows the offense will hand it off to the RB
Where P(B) = Probability of reaching the end zone on a 2 yd run at the 2 yd line when the defense guesses incorrectly
And P(C) = Probability of the defense being right about what the play is going to be.
The first problem here is that the value you observe is equal to that equation, but you're assuming to know P(A) and P(B) without knowing the probability of P(C). The second problem here is that P(C) increases as the offense increases its frequency of choosing this play.
Without knowing P(C) and without acknowledging the correlation between P(C) and the act of the offense changing its strategy, it's perfectly plausible that the success rate would not fall within the two numbers you specify.
Even if the expected number of points is greater when going for two, there are quite often good reasons not to do so, especially when the game was tied prior to a late game touchdown.
ReplyDeleteExample: Suppose the score is tied 20-20 before team A scores a touchdown with 3 minutes left in the game.
Assumptions:
1) All teams have a 60% success rate on 2 point conversions and a 99% success rate on extra points.
2) Team B has a 40% chance of scoring a touchdown in the remaining 3 minutes, and if they do, that will be the final score of regulation.
3) If team B is down by 7 when they score a touchdown, they will attempt an extra point to send the game to overtime, and in overtime each team would have a 50% chance of winning.
Should team A go for 1 or 2? Let's look at the numbers.
(note: In either scenario, team A has at least a 60% of winning because there is only a 40% chance that team B responds with a touchdown. So we'll ignore that for the moment and consider only situations where team B answers with a touchdown.)
If team A goes for 1, here are their winning scenarios:
a) 99%(team A makes) * 99%(team B makes) * 50% (chance in OT) = 49.005%
b) 99% * 1% (team B misses) = 0.99%
c) 1% * 1% (both miss) * 50% (OT) = 0.005%
Total = 50%
If team A goes for 2, here are their winning scenarios:
a) 60% (team A succeeds) * 60% (team B succeeds) * 50% (OT) = 18%
b) 60% * 40% (team B fails) = 24%
c) 40% * 1% (team B misses extra point) * 50% (OT) = 0.2%
Total = 42.2%
Adding back in the 60% chance of winning by keeping team B from scoring a touchdown, and we have:
Going for 1 = 60% + 40%*50% = 80% chance of winning.
Going for 2 = 60% + 40%*42.2% = 76.8% chance of winning.
Clearly in this situation, going for 2 is a bad move because if you fail, the other team can exploit that by adjusting their strategy and kicking the extra point.
My hunch is that this logic could be extended to say that in all situations where the score was tied prior to the touchdown, going for 2 is a bad move. If a team is down 7 prior to scoring a touchdown, going for 2 is a great move.
"On the contrary, if the offense in any way signals an increase in the probability that they will hand off to the RB on a 2 pt attempt, one should expect the success rate to drop significantly irrespective of the current values for success rates"
ReplyDeleteyes yes yes
"On the contrary, if the offense in any way signals an increase in the probability that they will hand off to the RB on a 2 pt attempt, one should expect the success rate to drop significantly irrespective of the current values for success rates"
ReplyDeleteBut remember that the success rate of PASSING will increase... and at some point it's fair to expect that the success rate of passing will equal the success rate of running...
"But remember that the success rate of PASSING will increase... and at some point it's fair to expect that the success rate of passing will equal the success rate of running..."
ReplyDeleteBoth of which, again, depend on the ability of the defense to adapt and the amount of information the offense creates by adopting any sort of strategy at all.
"My hunch is that this logic could be extended to say that in all situations where the score was tied prior to the touchdown, going for 2 is a bad move. If a team is down 7 prior to scoring a touchdown, going for 2 is a great move."
ReplyDeleteYour analysis only holds true for late game situations - not necessarily for situations where the score was previously tied after a touchdown. If we assume Brian's analysis is correct and that two-point conversions are +EV, they should be attempted up to the point where the benefit of minimizing variance by going for 1 outweighs the benefit of the added EV by going for two. I imagine this would only present itself in the 4th quarter, but perhaps earlier.
J-Doug-
ReplyDeleteI think you may be thinking too specifically. Sure, certain teams will struggle to adapt to other specific teams' specific strategies.
If all teams aggregated make a two-point conversion on average 50 percent of the time, it'd be difficult to find a matchup that's exactly 50/50. most off/def matchups will swing slightly one way or another. it doesn't change the fact that game theory is at play and that if the success rates aren't equal then the offense is missing opportunities
"it doesn't change the fact that game theory is at play"
ReplyDeleteThis is exactly my point, game theory IS at play. But the conclusion of this post is--in my opinion--ignoring a very important aspect of the strategic interaction between offense and defense.
I'm not talking about the matchup level, I'm talking about the league-wide level. The only way you can assume that a change in offensive strategy will increase the success rate towards the RB run success rate is if you assume that defense--and I mean the aggregate of all defenses at the league level--will not incorporate new information into their own strategy.
To be clear, I'm not saying Brian Burke is absolutely wrong. It's entirely possible that offenses aren't running 2PT conversions enough. What I'm saying is that based on the data Brian has provided here, you cannot conclude that this will be the case, and you most certainly can't conclude anything about whether or not that value will exceed break even.
I think j doug could be right as this is not a perfect game theory situation as while the offence has two choices the defence has an infinite choice between extreme pass defence and extreme run defence. If standard defence allows 57% success on runs and 47% success on passes but moving towards pass defence increases run success by more than it lowers pass success and vice versa for a run defence then standard defence is the equilibrium. Of course this is purely theoretical and can never be proved because we cannot determine what the defence is doing but it shows that passing and running do not have to be equal to be on equilibrium. I really need to draw a graph to show this idea clearly
ReplyDeleteJames
by definition, an offense should choose the higher success rate every time. in this case, they would choose run. the defense should respond over time, decreasing the success rate of the run, and (theoretically) increasing the success rate of the pass). the offense should always choose the higher success rate until they are equal.
ReplyDelete"It's far more reasonable to assume that the current success rate for 2PT conversions is the equilibrium outcome and that a change in offensive strategy--and a response in defensive strategy, would result in little if any departure from the current rate. "
ReplyDeleteIt can't be the equilibrium outcome. If you consider some representative offense taking as given the mixed strategy of the defense they can strictly improve by running at a higher frequency.
What is this McCown kneel down? http://sports.yahoo.com/nfl/recap?gid=20031228022&prov=ap says it was a pass to Emmitt Smith that was too short
ReplyDelete"the offense should always choose the higher success rate until they are equal."
ReplyDeleteThis is a fantastic strategy if the success rates aren't in any way related to the expectations of your opponent (the defense). Unfortunately, in football (and pretty much everything) they are.
"It can't be the equilibrium outcome. If you consider some representative offense taking as given the mixed strategy of the defense they can strictly improve by running at a higher frequency."
Again, there's no way of knowing if running more often will improve the offense's chances unless we ignore the fact that the offense's chances are dependent on defensive expectations, which is a ridiculous thing to ignore. Second, I don't actually think the current outcome is an equilibrium—I was making a point about what we can and cannot know about the problem at hand. Third, the issue remains that if you don't know P(C) from my third comment or the difference between P(A) and P(B), there's absolutely no way of knowing what the real success rate would be in the event of a change in strategy, and you absolutely cannot conclude the numbers that would bound the eventual outcome.
Why wouldn't the proper thing to do be to add the QB runs to the pass plays? I presume 2 pointers are too far out that anybody is calling the QB sneak. Those QB runs are part of the options provided by a pass play.
ReplyDeleteIt's shocking that teams don't even attempt the 2pt conversion after a defensive penalty on the PAT, even though the expected point value of a 2pt conversion is well over 1 after you've moved half the distance to the goal.
ReplyDeleteAdding QB runs to pass plays would give pass plays around a 46% success rate
ReplyDelete@J-Doug
ReplyDeleteTeam A and Team B both attempt two-point conversions. Team A runs the ball. Team B passes the ball. Who is more likely to score?
Hopefully, your answer is Team A, which is the correct answer.
"Again, there's no way of knowing if running more often will improve the offense's chances unless we ignore the fact that the offense's chances are dependent on defensive expectations, which is a ridiculous thing to ignore."
While we can't explicitly PROVE that running the ball increases a team's chances of scoring on a 2-point conversion on any given attempt, it seems asinine to say that attempting a play with SR of 57% does not improve your chances of scoring over attempting a play with 46% SR.
Willkoky, QB runs could also be designed QB draw plays, which should be considered runs.
ReplyDelete@Anonymous: Everything you just said is--again--dependent on defensive expectations remaining constant. This is not something you can assume in a strategic interaction or any sort of game-theoretical analysis. If you want to go ahead assuming that then fine, but it's a completely false assumption that has major implications for the conclusion of this post.
ReplyDeleteIn no way, shape, or form were defensive expectations held constant over time in analysis. That is the crux of game theory. The defense will adapt. Nothing I said is dependent on defensive expectations remaining constant.
ReplyDeleteGiven ONLY the information that Team A will run, and Team B will pass, we would expect 57% chance of success for team A, and 46% for team B (whatever the original numbers were). I don't understand why you think this aggregated average is dependent on defensive expectation. For this exact instance, of course team A and team B's ACTUAL percent chance of success is dependent on defensive expectations (as well as many, many other factors). The averages of 57% and 46% essentially "account" for defensive expectation.
It's like empirically knowing that 57% of women suffer some side effect of a drug, compared with only 46% of men. If person A suffered the side effect, would you guess they were female or male (given equal base rates)? Of course there are other contextual factors, like medical history, age, etc. but, like particular and specific matchups in the nfl, can be taken on a case by case basis.
Anonymous: About McCown's kneeldown. I assume Brian meant the final touchdown of the game. He said "kneel-down while trailing by 5 points", but I think he meant the final touchdown came when trailing by 5 points. I assume the kneel-down came after they were up by 1?
ReplyDeleteExample: Suppose the score is tied 20-20 before team A scores a touchdown with 3 minutes left in the game.
ReplyDelete3) If team B is down by 7 when they score a touchdown, they will attempt an extra point to send the game to overtime, and in overtime each team would have a 50% chance of winning.
Assumption 3 does not hold true, because, if obeying ideal strategy, a team should go for 2 if down by seven.
If they do that, the chances that team A wins if they kick an extra point become
(.99)(.4)=39.6%
+(.01)(.01)(.5)=.005%
=39.605%
Which is clearly a lower chance of winning than if team A had gone for 2.
If the expectation of converting was always 50%, I'd go for it every time in order to not risk my players getting injured in overtime and to be more rested for the next game.
ReplyDeleteRe: the 2003 Vikings/Cardinals game:
ReplyDeleteCards went for two twice that game.
1) Cards were down 17-6. Scored a TD to make it 17-12. Went for 2 with a pass to Smith, who was stopped short of goal line.
2) Cards were down 17-12. Scored a TD with 4 seconds left. Inexplicably kneeled for the conversion. Makes no sense. TD was last play of the game and opponents can't return a blocked PAT for a point. So Cards were in no danger of losing the game.
Only two possible reasons
a) didn't want to rub it in. Cards had just eliminated the Vikes from the playoffs by scoring at the end.
b) Wanted to keep the score on the under. I have no idea what the over/under was, but it's possible (though highly unlikely, way too obvious) that the O/U was 35.5 and by kneeling, coach kept the game at the Under.
TMQ wasn't talking about going for two in every situation. Only about going for two when the choice is a PAT to send the game to OT or going for two for the win. Probably every ready here realizes that going for two when down by one late in the 4th is the better proposition. But TMQ is writing for "the masses" who may not understand that concept.
ReplyDeleteInteresting discussion. My view is that the data is skewed by the QB option to pass/run. By running out of the pocket, the QB can ascertain the success of running into the endzone fairly easily; if chances of running seem low, QB can lob pass into endzone for a "jump ball". The catagorization of "run/pass" depends on the choice of the QB. Ergo, this skews the successful run 2 point conversion on the high side.
ReplyDeleteWhat this boils down to is if a given team can maintain the norm (which is a 1pt. field goal per touchdown at the end of the game given that a 2pt. play wasn't necessary due to a comeback scenario)would playing the odds of using a 2pt. conversion net a gain in score? If the odds are better than 50% it would be no different than kicking a field goal per touchdown with a net of 1 extra pt per touchdown on each success and no loss of pt. on one failure for every 2 touchdowns. Why not go for it if you can maintain pace with the other teams scoring? If push comes to shove and you are out a second TD opportunity (because you will need an even number of touchdowns to justify the odds of going for a 2pt. play (reality check) at the end of the game) then kick it and go into OT and take your chances there.
ReplyDeleteThe numbers used in the analysis are based off the current decision making logic in the NFL -- namely, how many 2pt conversions occurred in the 1st and 2nd quarters? It might be that the success rates shown should be higher in that the good teams don't "need" to go for it and are therefore left out of the sample. Furthermore, if most or all of the 2pt conversions are done in close games, this implies a close matchup between the teams (admittedly not necessarily between offense/defense as it could be a shootout). If we want to compare a strategy of going for two through the first three quarters versus kicking the extra point, we might be understating the EV.
ReplyDeleteIt's not just 2 point conversions - many offenses seem to think that 3rd and 3+ yards is a pass-only situation and they'll line up with 4 or 5 WRs and not even consider a run, even though even poor running teams average more than 3 yards a rush.
ReplyDeleteHere's the problem with all of your theorizing...running the ball may just be easier to convert than passing. There may never be an equilibrium because one play is just generally easier to complete. Passing against a defense with 12 vertical yards is tough, but running may be easier even when they are expecting because the blockers and back just need to push 2 yards or run to the outside or qb sneak up the middle..there are still many options at the run which is very hard to defend in short yardage. Don't forget that the defense will virtually never go into an 11 man run stop defense, let alone 9 maybe 8 because of the threat of a pass to a 3 tight end set, so running may just be more effective like shooting a 3 versus a 16 footer in basketball even if the 16 footer is contested equally to the 3 it should be more successful.
ReplyDeleteBut what if, as I imagine, many of the runs are checks at the line of scrimmage based upon certain looks? i.e. if there are 6 in the box or less it's an automatic run play, if not the pass play remains? There may be several "unprofitable" decisions when running the ball is very negative. You can't always look at stats and say "teams should run more". Perhaps every time they pass on a 2 point conversion they have a less than 50% chance of a successful run. Of course in some situations they are better off taking the delay of game and kicking the extra point... But there usually is a reason they are going for two, and they don't gain anything from "losing by 1 rather than 2". But much like a chart on a variation of blackjack and "when to surrender", there are certain situations when "only" being successful around 45% of the time (I estimate because QB could run or throw the ball laterally which counts as a run as well) is perhaps much greater than running.
ReplyDeleteIf the defense stacks the box and shows blitz, and calls a "pinch" defense I can't imagine it would be worth sticking with the current run play. Teams prepare for this, and if certain defensive looks are given they go with their passing play. It is not as if you call a run play and stick with it regardless of the look, or a pass play and go regardless of the look. Runs are successful because teams do them when the defense is lined up to defend pass, and passes aren't successful because it's the "surrender option" when you can't get the look in which running the ball is effective.
I think you can also advance the stats by considering the average number of possessions (and scores) per game. The higher varience strategy of going for two may not always be best, particularly if team is superior and already has a sufficient enough lead where the only thing they can do to let team back into it is a string of badluck. I don't think there is enough time in an NFL game for the highest EPA decision to always be correct. the wider the varience, the less consistent the score and the greater the chance of a team coming back on you. Additionally, the 2 point conversion probably won't be nearly as high if a team regularly goes for it.
With that being said, this type of analysis is still a great start to ground coaches decision making and to get them to start to consider going for two, doing their own research as it applies to their own team where more information is known, and their opponents which more information is known, and make a more informed decision
ReplyDelete