Response to Luck and Belichick Article Comments

The traffic to this site has increased quite a bit lately. A lot of it is likely due to the interest in the playoffs, but much of it is from direct links from other sites. Most direct links are to my articles about Rating 'Gameday' Coaches and about Belichick Cheating Evidence. They have appeared on many message boards across the football world, and many of the comments and criticisms are outstanding. Allow me to address some of the comments here. The foundation of both articles deals with luck. Because there are so many new readers here, I'd like to clarify what I mean by luck, and how I use this definition when I apply statistics to make observations about coaching.

Luck

To me, a good example of luck is the "bunching" of successful events. In football, first downs are nice, but consecutive first downs are what allow touchdown drives. The number of first downs and the yardage gained in each should be ascribed to skill. Those are things the teams on the field control. But whether those first downs come in bunches or are interspersed is something different.

Perhaps a baseball analogy illustrates my point best. One single per inning gets a team zero runs after nine innings. But nine singles in one inning, followed by zero hits in eight innings would usually yield about six runs. In football, think of a drive as an inning--a team usually needs consecutive successes to score. If players could control when successful plays occurred, sports would be very different. Batters would save their hits for when runners are on base, or when the game is on the line. Receivers would save their dropped passes for the 4th quarter of blowouts.

Here is a football example I've used before: Let's say both PIT and CLE each get 12 1st downs in a game against each other. PIT's 1st downs come as 6 separate bunches of 2 consecutive 1st downs followed by a punt. CLE's 1st downs come as 2 bunches of 6 consecutive 1st downs resulting in 2 TDs. CLE's remaining drives are all 3-and-outs followed by a solid punt. Each team performed equally well-same yards, first downs, turnovers, kicking etc. But the random "bunching" of successful events gave CLE a 14-0 shutout.

That's what I mean by luck. There are several other factors that could be considered random, but my theory is that the bunching effect could explain the bulk of the observed differences in in-game performance and ultimate outcomes.

Luck and the Model of Team Wins

So when I rank teams by luck, I am using my prediction model to estimate how many games a team would be expected to win given their on-field performance. If they win more than expected, I say they're lucky. If they win less, they're unlucky.

Admittedly, the model can't possibly account for every possible consideration on the field of play. There are too many moving parts and inter-dependencies in football to just say, "whatever I don't account for must be luck." There are other factors, such as weather or coaching tactics, some of which are unmeasurable.

But we do know how accurate the model is. We know it accounts for 80% of the variance in team win totals. And there are sound techniques showing that luck, or randomness, accounts for a very large part of the 20% that's left over. That's why I call the model's residual (the difference between estimated and actual wins) luck. Or least the bulk of it is.

Coaching Tactics

Coaching tactics on gameday, such as clock management and whether to kick or go for a first down, are one of the things the model does not account for. It is a small part of the residual. When I ranked coaches on their gameday tactics I used the residual of the model. But a critic would rightfully point out the obvious contradiction--How can the residual be considered luck in one case and 'coaching tactics' in the other?

The answer is that luck is random by definition. It does not correlate with anything. So if you average out enough years of performance, the luck part of the residual tends to cancel itself out, and what's left over is non-random considerations, including coaching tactics. The more years you have in the data, the more likely it is that the luck cancels out. When there is only one year of data, the residual will still contain the luck. This isn't my own personal theory, but one of the central tenets of inferential statistics.

Further, when you have enough years of data and divide up and analyze the data by coaches, and not by something else, you get a good estimate of that coach's gameday contribution to his teams' win totals. Essentially, I'm saying other coaches, given the same on-field capability of their players, would win X many games. Coach so-and-so won on average X+1.2 games per year, so he is credited with a +1.2 "wins added wins per year" score.

A coach that takes a fantastically talented football team to a 10-6 record would not score high. But a coach that can consistently take an average team to a 10-6 record would be ranked at the top.

By the way, I call it 'gameday,' because the preparation and practice part of the coaching job would be reflected in the on-field performance stats and not in the residual. It's the 4th down decisions and such that aren't captured in the efficiency data I use.

Belichick and Cheating

After ranking all the coaches, I had expected to see Belichick at or near the top of the list. He was actually near the middle. So I split his ratings for his tenures at Cleveland and New England. His 'wins added' score was literally off the charts. It was 3 standard deviations beyond any other coach, and he never had any single year that wasn't off-the chart itself. That would make him not only a once-in-a-lifetime type of tactician, but a once-in-a-millennium super-genius.

At first I thought, wow, he really is something special. But then the cheating revelations hit, and I thought this could be due to more than just genius. In fact, it makes a lot of sense given that we already know he is willing to break rules for a competitive edge. There were many other reports of cheating by the Patriots, beyond taping signals, such as exploiting QB helmet radio communications in various ways.

I'm not saying the Patriots aren't a great team or even that Belichick isn't a great coach. They obviously are. But both things can be true. They can be both great and cheating.

A solid criticism of my approach would be that I can't just chalk up the one team that breaks my model to cheating. I'm not. I was scratching my head wondering why this one team defies the statistical tendencies of the 31 other teams. Then several weeks later, it was revealed that that same team had been cheating.

By no means do I claim that my analysis is iron-tight. I think it's useful and interesting. Feel free to disagree.