Comments on Advanced NFL Stats: Luck and NFL Outcomes 1

Injuries, refs, and turnovers. Three things you c...

2010-11-15T15:15:24.525-05:00

Injuries, refs, and turnovers. Three things you can't predict. To me it usually determines success and failure.

Hi Brian; I used your approach to check for the le...

2010-06-05T17:52:52.309-04:00

Hi Brian;
I used your approach to check for the level of skill/luck in the nhl (my fav. sport)
Unfortunately i don't know how to compare
graphs via chi test? Can you match distributions by checking the SD .

for example actual sdw% nhl last 5 seasons
(150 team seasons) is .083

my all luck (nhl)league has a sd w% of
0.053
my skill league sdw% is (.29)
my 40% skill league sdw% is (.145)
my 30% skill league sdw% is ( .10)
my 20% skill league ....(.073)
my 24% skill league ....is .083

therefore nhl is 24% skill 76% luck
and best predicition model can achieve
62%correct (24%+38%...half of luck)

is this reasoning sound? did you have another way of calculating luck of a league
thanks Dan

I've always felt like luck played a big role. ...

2009-11-02T16:56:19.707-05:00

I've always felt like luck played a big role. For instance, take fumbles and tipped balls. The players themselves can cause these actions, but whether they actually result in a turnover usually involves a great deal of chance.

A couple of years ago I looked at fumbles lost and one team (TEN I think), fumbled 20+ times, but only lost 2 or 3. Conversely there was one team (Ravens I think) that recovered more fumbles than they actually forced. Typically these numbers were around 50%.

Likewise penalties are a bit of a wild card. For example, on 3rd and goal a team defends a pass in the endzone, but is called for an iffy PI. If that play was run a hundred times, maybe 50% of the time it's a penalty. So half the time the opponent will line up for a FG, and half the time they will line up for 1st & G at the 1...with a very high chance of scoring a TD.

So that penalty probably has a 3.5+ point swing (assuming not every 1st & G gets a TD). In a perfect world there would be a penalty based on exactly how bad the PI was...maybe in a case like this it would be half the distance to the goal and repeat 3rd down.

I'd also guess that penalties against the offense probably have the most effect given that increased down and distance probably greatly increases the chance of punting.

Conversely, a foul on the defense often gives a first down, but it also resets the down and distance. So the offense does benefit, but the benefit varies based on what the prior offensive situation was. If the defense goes offside on 2nd & 1, the actual penalty isn't really that great.

Brian; This is your seminal work. Brilliant! I sti...

2009-10-24T14:18:57.555-04:00

Brian;
This is your seminal work. Brilliant!
I still don't feel even us hardcore stats
gurus have digested the consequences of your findings.
1) If my logic is sound does it follow from a 48% skill NFL league, that each observation (1 game in a 16 game league) contributes only 3% towards the 'skill bank' so to speak.Or rather our certainity in the skill stats we observe.
In other words, if you were to only watch ONE of a team's game one would only have 3% confidence in the results that they were skill-based because that team was better/worse than other.
(Even NE blowout of TENN)
Maybe one can't simply divide the skill per game
is there a proper formula?

What would happen if the nFL played 32games?
what do you think the graph would look like? how much would the team wins look more like skill league?
keep up the great work!
Joe

Yes, it assumes teams are trying to win games.

2008-11-22T21:58:00.000-05:00

Yes, it assumes teams are trying to win games.

Another potential problem with this study is the a...

2008-11-22T21:00:00.000-05:00

Another potential problem with this study is the assumption that all teams are trying to win every game. Teams that have locked up home field advantage in the playoffs sometimes tend to play any remaining regular season games like a pre-season games, resting some key players and pulling out most starters in the second half. I don't think this is a good strategy because of the effect it can have on the mentality of the team but some teams do it and it is definitely another factor besides luck that explains why the best team doesn't always win every game.

I also think it would be interesting to see a study on how expected points from a given point on the field change between regulation and overtime and a study on what correlation there is between winning or losing streaks and the probability of winning the next game. If there are already articles on this please direct me to them or if those are just things that would not be worthwhile to study I'm sorry, I'm new to this site and not a big statistics buff but I'm learning a lot from the site, I love it.

Fred-You're right. Thanks for the correction.

2008-11-21T15:12:00.000-05:00

Fred-You're right. Thanks for the correction.

I know this is an old post, but I've just stumbled...

2008-11-21T15:04:00.000-05:00

I know this is an old post, but I've just stumbled upon your site. It has been a pleasure reading it, btw.

Just a question - it appears to me from your graphs that the most common record is 9-7 followed by 10-6, and not 10-6 followed by 9-7 as you stated. Am I mis-reading your graphs? The 9-7/10-6 scenario feels more intuitively obvious to me.

Keep up the great work on the site! (and I hope your week 12 probabilities for the Chargers vs. Colts proves accurate. GO BOLTS!)

First, I think it's important to point out that th...

2007-08-01T14:15:00.000-04:00

First, I think it's important to point out that the "best" computer models had a test set of only 2 weeks. From what I've seen, it's not terribly difficult to find 2-week periods of very high predictive success.

By DVOA standards, the overall better team wins about 63% of the time, but that total varies significantly from year to year.

<Bee in bonnet>
To me, intangibles fall partially into the same category as luck and partially into the same category as skill. We're using averages to predict future performance, but players perform above and below averages due to foreseeable factors (pre-game injury, opponent quality) and nonforeseeable factors (in-game injury, random bounces of the ball, death in the family). When Brett Favre's father died, if he had choked in the Oakland game, people would've said he was distracted and shouldn't have played. But he played so well that it seemed as if he were especially motivated. I'm somewhat in line with the Le Batard school of thought that results dictate perceived intangibles.

That's not to say intangibles do not exist, however. If a player is especially motivated throughout the season (e.g. contract year), it should show up in his performance metrics as an abnormal spike. If a team chokes in big games, it should show up as an abnormal drop in *opponent-adjusted* statistics. If a team is out of shape and not motivated, it will show up in their performance (see the 2006-7 Miami Heat).
</Bee in bonnet>

Okay, I see where you're going with this...I'll wa...

2007-08-01T14:13:00.000-04:00

Okay, I see where you're going with this...I'll wait to see part II before I say anything else.

I'm not doing a regression of 40 yd sprint times o...

2007-08-01T14:04:00.000-04:00

I'm not doing a regression of 40 yd sprint times or bench press reps. I'm not basing it on regression models or correlations at all.

You'll see more when I'm able to post the rest of the study. Basically I just need to know what the win distribution of a theoretical skill-only league looks like. By a skill-only league, I just mean no luck is involved in determining the winner. The better team always wins, and there would never be an upset.

See my original post regarding an example of what I consider luck.

The rest of my comment is not relevent to this post, but I'll respond to your points.

About intangibles. I don't disagree they exist, but their effect would show up in the performance stats, would they not? Additionally, I believe the things commonly referred to as intangibles tend to be over-emphasized. People see an underdog upset a favorite and naturally need to create a reason.

I'd also consider injuries as luck. People commonly say "injuries are part of the game." That is completely untrue, of course.

Injuries are a reality of the game. They are a consequence of the game, but they are not part of the sport. Injuring an opponent is not an objective. It is not rewarded and players frown on it. But when injuries are intential, or even appear intentional, the culprits are penalized, fined, and suspended.

Hard hits are part of the game. Shaking someone's confidence and their will to fight is part of the game, but intentionally endangering their career is certainly not. Causing injury is not a planned, repeatable, practicable skill or talent within the rules of the sport.

Neat post. I am curious though - what exactly is i...

2007-08-01T08:46:00.000-04:00

Neat post. I am curious though - what exactly is included in skill? Agility, quickness, 40-yard dash times, strength...those kinds of things that are quantifiable? What then is included in luck? Bounces of the ball, stuff like that.

I think there is a third category that falls outside skill and luck - the "intangibles", the things that are mostly mental and can't be assigned a value. Mental toughness, motivation, health, etc. They have to be accounted for, otherwise the analysis is incomplete. They're skills, but they can't be quantified. They're definitely not luck, as they are skills that have to be developed.