By Brian Burke
Last week a WSJ article about the Seahawks' defensive backs claimed that they "obstruct and foul opposing receivers on practically every play." I took a deeper look in to the numbers and found that as long as referees are reluctant to throw flags on the defense in pass coverage (as claimed in the article), holding the receiver is a very efficient defensive strategy despite the risk of being penalized.
The following is an analysis using the concepts of expected utility, expected cost, and bayesian statistics.
The reason defensive holding is an optimal strategy comes down to one word. Economics. The referee's reluctance to call penalties on the defensive secondary is analogous to a market inefficiency. The variance in talent on NFL rosters, coaching staffs, and front offices between the best and worst teams in the league is probably very small. Successful teams win within a small margin. Seattle has found a way to exploit a relaxation in marginal constraints within the way the game is called that their competitors have not, and turned it into a competitive advantage.
If you think about committing a penalty in the same way as committing a crime, the expected utility is essentially the same. The expected utility (EU) for defensive holding is (opponent loss of down due to incomplete pass - probability of being penalized x cost of penalty). In other words, EU is the benefit of an incomplete pass minus the cost of the penalty times the probability of getting caught.
Denver and Seattle are facing off in the super bowl, Imagine that the game is happening right now, it's in the third quarter, the game is close and Denver's offense has the ball on third and 7 and are not in field goal range. Manning averages about 12 yards per completion. Denver's utility from a completed pass is 12 yards and a first down, an incomplete pass results in a punt. The difference between success and failure on this play for Denver is huge.
Manning completes 68% of his passes. Denver's EU for this play is 0.68 x 12 or 8.3, (this is the same thing as yards/attempt). On average a pass play in this situation will move the chains for Denver.
*For now we will make an assumption that if the Seattle DB covering the receiver that Manning is throwing to holds, the result will be an incomplete pass.
If Seattle is flagged for defensive holding, Denver gains 5 yards as a result of the infraction and a fresh set of downs. However, that's 3.3 yards less than the expected outcome of a Manning pass. Even when Seattle is flagged for holding they still come out ahead by committing the penalty. Denver get's the first down but only 5 yards rather than the expected 8.3.
But, as Mike Pereira points out in the article, Seattle's DBs doesn't always get called for holding. We have no way of knowing exactly how many times Seattle got away with holding but we do know that the secondary was flagged 10 times for defensive holding during the regular season. 541 passes were thrown against Seattle during the regular season which gives us a rough estimate of the probability of Seattle being flagged p(F) for holding on an any individual pass play, 10 flags / 541 attempts is a little under 2%. (It is important to note that this is without regard to whether holding actually occurred or not, it's just the overall rate of Seattle being penalized for defensive holding)
What that means is that 2% is not the conditional probability of being flagged given that a penalty was actually committed.. It's the unconditional probability of a flag being thrown for defensive holding on any play. The conditional probability (the probability of being flagged GIVEN a penalty was actually committed), p(F|P) , could be derived from Bayes theorem, however we would need to know the actual defensive holding penalty rate including holding that isn't flagged which we don't know. (we'll get around that)
Seattle's expected cost (EC) for committing a defensive holding penalty can be approximated as (cost of penalty x probability of being flagged), or in the case of defensive holding (-5 yds x p(F|P)). Here p(F|P) is the conditional probability of a flag being thrown GIVEN that the player committed a foul (in a perfect world this would be 100%. A penalty would be accessed every time a player commits a foul).
p(F|P) = p(P|F) x p(F) / p(P) = 0.9 x 0.02 / 0.6 = 0.03
Somewhere in the neighborhood of 3%. That's the probability that a defensive back will actually get flagged for holding the receiver when they foul the receiver 60% of the time.
To estimate the Seahawks expected utility of holding, we subtract the EC of the penalty from Denver's EU of a pass play.
Seattle's EC = 0.03 x -5 yds for an average loss of 0.15 yards.if DBs hold
Denver's EU = 8.3 (Manning yards/attempt) if the DB's don't hold
Seattle's EU = -(8.3 - 0.15) = -8.15 yards (negative value to distinguish offensive yards gained from defensive yards prevented)
Denver's EC = 0 yards (incomplete pass) + 0.15 (Seattle's EC) = +0.15 yards
Denver Passes, DB's don't hold: Denver gains 8.3 expected yards
Denver Passes, DB's do hold: Denver gain 0.15 expected yards
Seattle may get a boost in the EU of defensive Holding during the super bowl. The article also states that there was a 40% drop in penalties during the wildcard playoff games with respect to the regular season games, which if true would lower Seattle 's expected costs for defensive holding even further. But because that sample size is so small, I would elect not to adjust for any post-season penalty calling trends and keep the numbers from the regular season.
What if Kevin Gilbride's estimate that Seattle's DBs do it on nearly every play?
If Seattle's holding rate were 95%:
p(F|P) = 0.9 x 0.2 / 0.95 = 0.2
Seattle's EC = (-5 yard) x 0.2 = - 1 yard
Seattle's EU = -(8.3 -1) = -7.3 yards
Denver's EC = 0 yards + 1 yard = + 1 yards
It's still a pretty sweet deal for Seattle.
There is a caveat. Defensive holding also gives the opponent a automatic first down. So the utility (and cost) of holding on first and ten is a different than holding on third and two.
An optimal pass protection strategy would have the defensive backs commit holding penalties on every third down and on plays when Denver needs less than 8.3 yards for a first down. Or as Kevin Gilbride states it, on nearly every play.
It may be at best morally ambiguous, but from a tactical standpoint, it's an extremely effective pass defense. As long as ref's are reluctant to call penalties on every play, the gains far outweigh the penalty. That probably wont't happen though, because the constant interruption in play would outrage most fans. How long would it take to play the Super Bowl if the game had to be delayed 95% of the time Manning threw the football?
Forty-Niners' (and Broncos') fans would call this cheating, and they certainly could make a strong case as technically it is, but through the filter of statistics and economics, it's extremely efficient and actually quite brilliant.