Comments on Advanced NFL Stats: 2009 Team-Specific Run-Pass Balance

Dan Rosenberry... But the data set excludes hail ...

2009-12-06T13:46:04.331-05:00

Dan Rosenberry...

But the data set excludes hail marys, plunges etc (it's all first down plays in the first and third quarter, between the 20s with the game within 10 pts). They won't skew the points. This is the 'normal' football situation he speaks of.

I suppose you've a point about the model assuming play-action, screens and standard drop-back passes all being the same (it would also class a QB scramble as a run rather than a called pass).

I think Brian has it correct with the wording above e.g. so far this season Arizona has done better passing on first down than it has running, and would likely benefit by increasing the number of times they pass on first down.

A point worth mentioning is that the underlying fa...

2009-12-05T12:56:21.134-05:00

A point worth mentioning is that the underlying factors that make running or passing more successful are in flux. Pittsburgh is a better running team with Mendenhall and Dixon than they were with Parker and Roethlisberger. NYJ look like they should run a lot more, but some of those numbers came from a healthy Leon Washington, and a QB who threw a ton of picks. As Sanchez gets better at ball security, the expected value of passing should rise.

I think coaches probably look at things more in this way. They have a philosophy about their ideal offense and its run-pass mix at different points in the game, and they seek to construct and coach a team that can produce those outcomes.

It isn't an argument with von Neumann, it'...

2009-12-05T11:15:34.426-05:00

It isn't an argument with von Neumann, it's a suggestion that the model for football might include information about what the other team is doing (personnel choices). Sorry about not being clear that I was picking at the the model, rather than the result.

Specific example of how marginal play EPA could matters
--------------------
The game has an extensive form where the defense lines up, the offense makes a read as to the type of defense and then makes a play call. The offense three different reads on the defense, run stack, unknown, and pass stack. In that order, the payoffs for passing are 0.36, -0.3, -1, and the payoffs for running are -1, -0.18, 0.09.

When the defense can be decoded the payoffs indicate calling the other type of play. When it's unknown, it's worse to try passing than to try running.

Assuming the defense plays 50% run stack, 25% unknown, 25% pass stack, this payoff structure is consistent with the observed datapoints for the Colts. The Colts pass 55% of the time, all the run stack defenses and a bit of the unknown defense, average EPA is 0.30. The 45% runs are called on the pass stacks and most of the unknown defenses, average EPA is -0.03.

However the marginal play is against the unknown defense, where the payoffs suggest that the Colts should run more, not less, despite the observed average passing EPA being higher than the observed average running EPA.
------------

Just because the value for one strategy is higher than for another doesn't mean it should be called more often, if the choices for the participants are interrelated.

Yup. Once any team falls behind by a lot or time g...

2009-12-04T21:32:48.380-05:00

Yup. Once any team falls behind by a lot or time gets tight, their plays are almost all going to be passes, including first downs. The Lions unfortunately find themselves far behind early in games and become very predictable.

Re Detroit: perhaps part of the problem is Detroit...

2009-12-04T21:22:24.132-05:00

Re Detroit: perhaps part of the problem is Detroit's predictability on first down.

According to Football Outsiders, the Lions' DVOA on 1st down run plays is -15.0%, 30th in the NFL, and their DVOA on 1st down pass plays is -22.6%, 31st in the NFL. Note that these numbers include all first-down plays, not just those that you looked at ... so it's possible that in "normal" first-down situations, the Lions throw the ball better than they run, and in situations where they trail by a lot, the opponent is expecting pass and the Lions' lower success might drop passing below running. (For the record, the only down/play combination in which the Lions succeed is in 3rd/4th-down running plays, 42.2%, 5th in the NFL, and I wonder if that's also a function of the games in which the Lions were so far behind that it didn't matter if they converted a third-and-short.

Dude-I have not changed the article at all. No equ...

2009-12-04T21:03:50.213-05:00

Dude-I have not changed the article at all. No equivocation, intentional or otherwise.

The guys who are still debating the marginal vs average semantics in a 2-player zero-sum game aren't in an argument with me. They can take up the topic with John von Neumann.

...if still accidental equivocation.

2009-12-04T20:45:12.908-05:00

...if still accidental equivocation.

Avg. v. Marginal I notice you've taken the wo...

2009-12-04T20:44:37.667-05:00

Avg. v. Marginal

I notice you've taken the word "average" out of your article, but you're still talking about averages. But you've only made the language ambiguous and did nothing to the data. Intentional equivocation.

And true, as I always say 'the usual caveats a...

2009-12-04T19:55:26.963-05:00

And true, as I always say 'the usual caveats apply,' which means not all passes are the same animal. We don't know which kinds of passes in what kind of mix might be optimal, but we do know one thing: they're not runs!

John-You're looking at 3/4 of a single season ...

2009-12-04T19:52:34.725-05:00

John-You're looking at 3/4 of a single season here. For all teams 2000-2009, passing more often on first down correlates positively with greater overall EPA (0.14), conversion % (0.18), and average gain (0.27).

To me this means that pass-heavy teams are doing better, and they would do even better if they passed even more.

Does it matter that there is almost no correlation...

2009-12-04T19:34:26.394-05:00

Does it matter that there is almost no correlation between optimal play calling and team success (a coefficient of -.036 using the absolute values for EPA diff and each team's winning percentage)? Obviously, other factors contribute to wins and losses, but if play calling is a crucial component to success, wouldn't you expect the teams with the best records to have a close to optimal mix? 4 of the 6 teams with the largest positive EPA difference are almost locks for the playoffs, whereas most of the teams with the lowest absolute EPA difference are pretty much hovering right around .500 records. It is only the teams with the largest negative EPA differences that appear to be struggling on average.

Which brings me to the next point. The assertion that "offenses should pass more often than they currently do" seems to be refuted by the fact that teams with positive EPA differences have a combined record of 113-85, while teams with negative values are 57-88. If anything, doesn't this suggest, in general, teams should be running more, as run-heavy teams are more successful?

I am sure there are plenty of underlying factors which explain play calling mix (for instance, the point about teams like Tampa Bay being frequent underdogs), and I agree with the general principle behind your argument, but it just seems like there should be a relationship between team success and play calling. Any ideas why there is not?

Ian- What I meant was that a pass is not the same...

2009-12-04T19:07:08.511-05:00

Ian-

What I meant was that a pass is not the same as any other pass. Hail-mary's are utilized between 0 and 10% passing, and line plunges on ""and inches"" are part of 0 and 10% running. Brian's talking about first down calls, so that's less of an impact here (I missed it completely)

------
Commenting on marginal vs average as it applies to first downs.

The passing play between 50% passing and 60% passing might not be the same mix as between 0% and 10% (fewer screens in 0 to 10% for instance, long bombs are thrown first, etc). If it is the same, then the average EPA is the marginal EPA, if it's not then it's not. I can't think of any way to quantify this, but it is an assumption of applying the game theory model.

Another part of this is ""taking what the defense gives you"" with a flexible quarterback that audibles (it's not just paper/rock/scissors). The marginal run call might be against a fairly stacked/obvious pass defense, and the passing EPA in that situation would be significantly less than the average, even less than the low running EPA, causing run to the be correct choice.

If the defense plays pass defense 45% of the time and Indy is smart enough to recognize all those instances, 55% passing (on every down that the defense plays run) might be their best response.

My point was that the game theory assumption about a pass being a pass being a pass might not be an accurate depiction. The root question is what the marginal pass and marginal run looks like and if they're the same as the average pass/run, the result (pass more) holds, if they're not, it isn't clear. I can't think of any way to figure out which passes the team almost called as runs versus which runs they almost called as passes without inside information.

Yes, turnovers, penalties, and everything else are...

2009-12-04T16:34:40.262-05:00

Yes, turnovers, penalties, and everything else are factored in.

Jonathan - that is only 2009 for the Eagles and no...

2009-12-04T15:56:21.461-05:00

Jonathan - that is only 2009 for the Eagles and not indicative of previous seasons. Additionally, the Eagles HAVE run the ball more this year as evidenced by the last two games. We are talking about ridiculous situations where the Eagles keep passing in 80-20 ratios in games that are within 10 points. The Eagles are 7-4...the best record they've had since 2004. Moreover, this also doesn't account for advantages in controlling the clock when running.

Is the possiblity of a turnover considered in the ...

2009-12-04T12:48:55.352-05:00

Is the possiblity of a turnover considered in the EP calculations?

"Funny that the "optimal" play-call...

2009-12-04T11:48:40.239-05:00

"Funny that the "optimal" play-calling mix is from Philly. Reid and/or Mornhinweg have probably taken as much criticism as anyone for neglecting the "run to set up the pass" myth. Turns out they are the model of efficiency!"

Wow, good call.

Anon 2 up - Yeah, right. That's where my erro...

2009-12-04T11:14:42.540-05:00

Anon 2 up - Yeah, right. That's where my error is. Thanks.

THINK ON THE MARGIN If, for example, passing yiel...

2009-12-04T11:06:26.583-05:00

THINK ON THE MARGIN

If, for example, passing yields more yards or estimated points per (first down) play, you don't keep passing until your running and passing averages are the same.

You keep passing until your expected value ON THE NEXT PLAY is equal. This will be different from your average of the last X first downs.

Dr. Obvious - I think you're failing to accoun...

2009-12-04T11:00:46.401-05:00

Dr. Obvious - I think you're failing to account for their current mix of Run & Pass. E.g., Philly's play-calling is optimal based on passing 64% of the time.

Funny that the "optimal" play-calling mix is from Philly. Reid and/or Mornhinweg have probably taken as much criticism as anyone for neglecting the "run to set up the pass" myth. Turns out they are the model of efficiency!

As for Reid's game management .... that's another story.

Please correct me if I'm wrong, but won't ...

2009-12-04T10:48:52.813-05:00

Please correct me if I'm wrong, but won't we be able to find the optimal run/pass mix with the equation (assuming both run and pass EPA are possitive)
Pass EPA(1-Optimal Pass %) = Run EPA(Optimal Pass %)
Using SD, that comes out to 95% pass on first down. A few more:
Pit = 78%
Philly = 50% (note they have the same EPA's)
Dal = 11%

Or I guess that would only work as a starting point, and as defenses adjust, the EPAs will change and you should track this and adjust.

If this is right though, I'm having trouble wrapping my head around why it is that equation, and not:
Pass EPA(Optimal Pass %) = Run EPA(1-Optimal Pass %).

The marginal vs. average debate is mostly semantic...

2009-12-04T08:49:37.445-05:00

The marginal vs. average debate is mostly semantic here.

What 2-player zero-sum game theory says is that the more a strategy is selected, the more often an opponent will counter it with his own strategy choices. Therefore, every additional choice of a given strategy will reduce the expected payoff of that strategy while enhancing the payoff of alternative strategies. That's all part of game theory.

Note that I'm not saying that, for example, the Cardinals would add 0.7 points for every future pass they throw on first down. I've said that every past run they've called has cost 0.7 points. I've also qualitatively stated that passing more often would benefit the Cardinals. They should theoretically pass more often until the payoffs equalize, but exactly how often that is and how much it will benefit them remains to be seen.

this kind of analysis is really really awesome, lo...

2009-12-04T08:29:42.029-05:00

this kind of analysis is really really awesome, look forward to more.

Dan Just so I can get the idea of marginal versus...

2009-12-04T08:26:25.040-05:00

Dan

Just so I can get the idea of marginal versus average correct in my head, can you confirm that what follows is what you're saying?

Let's assume I'm the head coach at Detroit (presumably I sinned in a past life) and I've run 100 first down plays this season. My 39 passes scored 0.26 EPA and my 61 rushes scored -0.20 EPA, therefore Brian's game theory suggests that I ought to pass the ball more often. But what you are saying is that if, after my 39 passes and 61 rushes, the expected EPA (that is, the expected 'expected points added') of my next play is the same regardless of whether I rush or pass, then my strategy would be balanced at the margin.

If that's the case then I would expect that past performance is a reasonable predictor of future performances (although I have no stats to back this up) and that the average EPA is the same as the marginal (I don't know how one would measure/prove that though).

There's an embedded assumption that may not be...

2009-12-04T07:40:05.331-05:00

There's an embedded assumption that may not be true in this analysis. Is the marginal EPA always constant? If it's not then average EPA doesn't equal marginal EPA.

Teams should work to make the marginal value of the strategy choices equal, not the average value, so that at the margin they're indifferent.

It's possible to construct marginal EPA by play call % curves that justify any conclusion by modifying the shap of the curve. Without knowing which calls are a teams marginal runs and marginal passes (they almost called the other type of play) it's hard to compare to see if the value of the strategies is balanced at the margin or not.