Comments on Advanced NFL Stats: Sunday Night 4th Down

In reading these posts and ALL of those relating t...

2009-12-22T23:40:09.044-05:00

In reading these posts and ALL of those relating to Patriots/Colts, I am struck by the number of people who seem to think that, once the relative probabilities have been determined, the outcome is certain. In fact, the uncertainties (individual plays, players, referee decisions, luck...) will, in all cases, determine the outcome after the decision. Brian has demonstrated that the decision was rational (that the chances of a Patriots win after Belicheck's decision were comparable to those of a loss, and possibly better). Probabilities are a powerful decision-making tool. But the outcome is still determined on the field. Coach B did not lose the game - the team did that. Had he punted, the game might still have been lost. Almost every negative post, in one way or another, declaims that the proof of the error is the outcome.
Horsefeathers.

Since you touched on it in the article (and I have...

2009-12-11T01:16:44.601-05:00

Since you touched on it in the article (and I haven't seen it mentioned on your site anywhere), I'm curious about the relative value of a timeout. I've always wondered if the conventional play of an offense taking a timeout to avoid a delay of game penalty is the right play. I do realize the intent of taking a timeout might not be clear, so the research might be difficult, but I think any reasonable analysis would still be interesting.

I did a crude analysis for my site (it's an op...

2009-12-01T20:02:09.259-05:00

I did a crude analysis for my site (it's an options trading site, but using some of these football "problems" as metaphors for analyzing risk/rewards on trading). If Steelers fail to get a 1st after a punt, Ravens get ball back on roughly their 35, vs. their 15 if they go on 4th and fail and then Steelers fail. So seems likely the field position diff. is only those 20 yards, which explains why the WP doesn't vary much. Abd Only real big diff. I see is Steelers MAYBE in situation to go on 4th down and really short in Ravens territory whereas no shot they go Belichek in their own territory with Dixon at QB.

Punting with 3 timeouts would result in a touch hi...

2009-12-01T00:10:14.850-05:00

Punting with 3 timeouts would result in a touch higher WP, I would guess, but even then the WP would represent two small chances: forcing a turnover on the ensuing series of downs or managing to score with three fewer timeouts and possibly worse field position ...

The thing about punting is that you're basically offering all your timeouts plus maybe 20-40 seconds (two punts plus three rushing plays with timeouts after them) for three extra downs. Sure, 1st and 10 looks better than 4th and X, but you're giving up a lot to get it, and sometimes you'll never get the ball back.

Another point re: weather: playing the game indoo...

2009-11-30T18:00:08.131-05:00

Another point re: weather: playing the game indoors presumably aids both offenses as well, which is a plus for going for it.

A theoretical question, if I may? Does your data ...

2009-11-30T17:08:53.657-05:00

A theoretical question, if I may? Does your data include timeouts remaining and if not, why? It would probably limit the real world examples, but I would tend to think that punting with 3 timeouts results in a higher WP than punting with 0 timeouts.

The game has changed in the past 40 year. Unfortun...

2009-11-30T15:53:37.721-05:00

The game has changed in the past 40 year. Unfortunetly the rules of thumb that coaches use have not evolved as much during that time.

Yeah, nobody should be surprised that indoor and o...

2009-11-30T15:26:46.210-05:00

Yeah, nobody should be surprised that indoor and outdoor punting numbers are pretty even. Of the top of my head,

Factors in favor of punting indoors:
1) Reduced chance of botched snap or hold.
2) Warmer ball = livelier bounce off the kicking foot.
3) No swirling winds = greater control = an easier time doing directional punts.
4) Possibly a more consistent bounce on punts that aren't caught, making it slightly easier to down a punt inside the 10. (I'm not sure the data supports this, though.)

Factors in favor of punting outdoors:
1) Upwind/downwind effects should actually increase average distance, not decrease it. A punter can use a high arc to take advantage of a tailwind, and a low arc to minimize the impact of a headwind.
2) Winds and weather make it much harder for the returner to field the ball cleanly, decreasing the number of returns.

Put all that together and I think it's about a wash, all told.

"Do you think Harbaugh ran the WP equation be...

2009-11-30T13:32:53.255-05:00

"Do you think Harbaugh ran the WP equation before he made his decision? "

I would hope so. Yes. You don't have to do the exact equation every time. You can estimate the probabilities on a the fly. I would hope that at some point during a year of 80 hour work weeks, coaches would sit down and run through a number of sample situations. They can then compare the results with the results that the complicated model provides. They should already have an outline of the probabilities in their head. After a number of samples, the right decision starts to become obvious. If its so close a call that there is disagreement, you can just pick either. Its nowhere near as difficult as I think you're implying.

"Why does Jeff's evaluation of Pats SB 4th and 13 disagree with Zeus' statistical analysis of the same situation?"

There are a number of estimates involved here. With shorter yardage situations, you typically use 3rd down situations as a reasonable proxy. That is difficult with 4th-13. One reason is that on 3rd and 13, a lot of teams aren't even trying to get the first down. They run draws that are designed with minimax in mind. The point isn't to get the exact probability down to the third decimal place. The point is to find the egregiously wrong decisions and highlight them. The 4th and 13 was a closer call. The 4th and 2 wasn't.

"I can play that game too: following options with my numbers plugged in:

A: .69
B: .84"

Did you just make those numbers up off the top of your head? Don't you feel some obligation to explain how you arrived at them?

How'd you get to those numbers? I think Brian did a very good job of explaining exactly how he got to 70 and 79. How did you get to yours?

In essence, what you are arguing is that the Colts would be far more likely to score from 28 than avg and far less likely to score from 70. You are also arguing that the Patriots would be far less likely to convert the fourth than league average. How did you get to any of these conclusions?

Why don't you try to build your own model defending conventional wisdom and publish it? I think you'll find that its basically impossible. I've tried a couple of times. You need to start with league averages and you can make adjustments for whatever you want to make adjustments for, but those adjustments need to be consistent with the existing data. The situational stats rarely vary from the average by anywhere near as much as you instinctually think they will. Adjusting for quality has an impact, but its somewhat closer to a wash than you would think Remember, if we increase Manning's ability to score from the 28 over league average, we need to increase his likelihood of scoring from 65 over league average. In this case, that just made it more of a case to go.

"Also, I suppose you combine indoor punt data with outdoors. "

Theres been a lot of talk about this. It seems like some people are under the impression that being indoors adds 10-15 yards to a kick. I downloaded the stats. I created a macro to go to http://sports.espn.go.com/nfl/players/splits?playerId=2273 and loop through all the different playerids. I found that punters do have higher nets inside. Its a grand total of 0.57 yards.

The other thing regarding these stats - it's l...

2009-11-30T12:31:23.649-05:00

The other thing regarding these stats - it's like poker - once you get the basic feel for a few situations, you can "interpolate" from there. For example, if next week Harbaugh was 4th and 4 from the 45 instead of 4th and 5 form the 48, the decision to go for it is still correct.

Wouldn't it make more sense to use the Steeler...

2009-11-30T12:26:58.794-05:00

Wouldn't it make more sense to use the Steelers allowed conversion rate on 4th and 5? Probably too small a sample size, but some estimate relative to 48 would at least be more accurate - they are a great defense. I plugged in 38% chance at the 4th down conversion, and got .22WP - so still a good decision by Harbaugh.

My theory on why Field Position doesn't matter...

2009-11-30T09:54:12.023-05:00

My theory on why Field Position doesn't matter in that situation is that teams are more conservative closer to the end zone. They are more likely to pass from their own 20 than they are at the opponents 40.

Alchemist-True, possession is all that really matt...

2009-11-30T09:22:55.642-05:00

Alchemist-True, possession is all that really matters. But I'm still surprised that field position for the leading team is almost irrelevant. You'd think that somewhat often they'd have to punt, and field position is critical for the trailing team. I guess the difference is in the constraint of the punt. The closer a team gets to the end zone, the shorter the punt. Plus, a FG might not help at all. Being up by 6 just forces the opponent go for the TD, which wins the game outright.

Anon-You are free to make up your own numbers and ...

2009-11-30T09:18:28.961-05:00

Anon-You are free to make up your own numbers and come to your own conclusions. I'd like to think the math I'm using here is something a 6th grader could grasp, so I'd like to get away from all the advanced-degree stuff. The data are from all non-preseason NFL games 2000-2008. Yes, indoor/outdoor punts are lumped together. Keep in mind outdoor punts can be longer or shorter depending on wind direction, and a few yds of field position do not change the results here.

I happen to think the SB 4th and 13 was a mistake, but I haven't run the numbers. ZEUS may disagree because it's a different kind of model. It's a simulation that's run over and over, and it takes the % of times each team won. My approach is far more empirical, meaning it is based on real outcomes in real games.

The ZEUS approach has its advantages, but I think my approach is more transparent and ultimately more convincing. With a computer simulation, you have to simply take the creators' word for it that he got everything right, and the model applies to reality. With my approach, you can see clearly how the conclusion was made, and you can plug in your own numbers and come to your own conclusion.

(I'm not bashing ZEUS. It is superior in some ways to a purely empirical method like mine.)

Brian, I think you've hit on something here an...

2009-11-30T08:06:27.789-05:00

Brian, I think you've hit on something here and I'm hopeful that maybe this will be understood by the leathernecks: As a game wanes, the leading team often must do nothing more than POSSESS the ball to ensure victory. Or, as you put it above, field position really matters very little at that point. If you are the Steelers, it doesn't matter too much if you have the ball on your own 20 or at midfield. The important point is that you HAVE the ball.

Punting is purely a field-position tactic. You are trading possession (because you are assuming that you would have lost possession anyway) in return for better field position. But as your numbers show, sometimes field position is not very valuable.

I don't have any faith that the average NFL wonk will ever dive into the (basic) math enough to understand this. But I do have at least some small hope that even a hardcore football traditionalist might eventually see the logic in not willingly giving up possession - especially late in a game.

Jeff Clarke said... "I know Americans don&#...

2009-11-30T04:11:46.318-05:00

Jeff Clarke said...

"I know Americans don't like math but a little bit of it would go a long way."

Do you think Harbaugh ran the WP equation before he made his decision? How do you think he made his decision w/out the math?

Personally, with only 2 or 1 TO left, down by 3, w/ that field position, I'd say he made the right decision. If he even gets the ball back after a punt, he's probaly going to have worse field position with almost no time left. Seems like a pretty sensible decision to go for it. I'm surprised that Collinsworth and Michaels both thought he should punt in that situation. I don't see it. And I missed their commentary about the decision.

But this condescending attitude about people not knowing math is what I detect when I read many posts here. It's annoying and not germane to the accuracy and relevance of NFL statistics.

Like the post in the other thread about "blah, blah, blah people w/ grad degrees say Belichick's call was good, people w/ HS diplomas say it was bad." Please. Anyone with a grad degree from a decent school would question the dataset a lot more closely than many here do.

I did notice that, in the punt graph, you state that the stats are taken from "recent" games. How "recent"? I was under the impression that your dataset is from 5 to 9 years. Also, I suppose you combine indoor punt data with outdoors.

Brian Burke in the Belichick thread said, "gee, following options:

A: .79
B: .70

which do you pick?"

I can play that game too: following options with my numbers plugged in:

A: .69
B: .84

which do you pick?

Brian & Jeff, please try to focus and answer this:

Why does Jeff's evaluation of Pats SB 4th and 13 disagree with Zeus' statistical analysis of the same situation?

The above question is the only thing I'd like for you both to think about honestly and then provide a reason. I mean, Bower is an Astrophysicist and all. He knows math. How is it that 2 statistcians can reach different conclusions?

I mean, if you were analyzing coin flips or dice rolls, you'd agree on those stats with the Astrophysicist 100% of the time, right? Are your stats just better than Bowers? Or are his better than yours?

If the announcers could at least make some attempt...

2009-11-30T00:35:44.615-05:00

If the announcers could at least make some attempt at doing the calculations, I think it would help a lot. I mean they don't have to get it perfect, but it seems pretty obvious to me that field position is a secondary concern there. If you punt, your only hope is a 3 and out followed by a desperation drive. A few extra yards on the desperation drive (which very well might not happen at all) don't matter nearly as much as the opportunity to maintain the ball in decent position right now.

If any announcers are reading this:

Please, please, at least consider these numbers and these equations. If you want to try to edit them for "situational statistics", whatever. I'm not sure you could come up with a set of numbers at all realistic that say "you have to punt" here.

You don't have to do anything except try to win the game. I know Americans don't like math but a little bit of it would go a long way.

I think those numbers are supposed to be 83 and 87...

2009-11-30T00:28:33.506-05:00

I think those numbers are supposed to be 83 and 87. I noticed that too. He used the correct numbers later in the article.

Oops. That should say 87% and 83%. Fixed. I also a...

2009-11-30T00:28:01.638-05:00

Oops. That should say 87% and 83%. Fixed. I also added the graph. The interesting thing to me is that, according to raw numbers, field position doesn't really matter in this situation.

How do teams win 87% from their own 20 and 87% at ...

2009-11-30T00:04:40.492-05:00

How do teams win 87% from their own 20 and 87% at the opponents 46. That doesn't make sense.