Comments on Advanced NFL Stats: Pre-Season Predictions Are Worthless

Do you think it makes more sense to regress teams ...

2009-11-27T02:02:21.997-05:00

Do you think it makes more sense to regress teams that have established winning continuity less than teams that seem to have more volatile records?

Lets say you and I are going to bet one another a dollar per game in which we're off on our predictions ie. if I predict the Panthers to win 11 games and they win 9, I owe you $2. We make our picks in separate rooms at the same time to

yeah -- sorry muddled projections wasn't very clea...

2007-09-07T19:56:01.225-04:00

yeah -- sorry muddled projections wasn't very clear. FO projects the probability of wins by bucket. Some of the teams are very clearly projected to be very good or bad -- Philly ~90% in the 9-10 plus 11+ buckets; KC ~70% in the 0-4 plus 5-6 buckets. Some team's projections are ~equally spread through all buckets, like Minn -- 10% in the 0-4 and 11+ and ~25% in the others. If you throw out

Thanks for running that analysis on the 8-win pred...

2007-09-05T22:10:34.238-04:00

Thanks for running that analysis on the 8-win predictions. I didn't really feel like it to be honest. I'm surprised how little value the FO predictions add. But I think this is why it's important to publish your error rates and accuracy metrics. If you're trumpeting what you got right, you need to be forthright about what you got wrong. It's not good business (and clearly FO has built a

Miles-I saw that post, but I don't understand what...

2007-09-05T18:29:36.758-04:00

Miles-I saw that post, but I don't understand what you mean by muddled predictions.

Here's a similar discussion on the FO site: http:/...

2007-09-05T12:57:48.049-04:00

Here's a similar discussion on the FO site:
http://footballoutsiders.com/discussion/viewtopic.php?t=146
I did find the FO predictions to be better (in 2006) when ignoring the teams that had muddled predictions, but only marginally better in 2005. Not a whole lot of data to use.

Derek, I agree MAE is only one way to look at it. ...

2007-09-04T23:33:20.802-04:00

Derek, I agree MAE is only one way to look at it. But I disagree that 17/32 within 2 games, and 10/32 within 1 game, is any good.

I ran the same comparison for the "mindless" 8-win prediction method (which I guess should be called the "null prediction").

Over the past 2 years, it was within 2 games an average of 16.5 times, and within 1 game an average of 10 times.

For

I think there are other ways to look at it besides...

2007-09-04T23:03:41.971-04:00

I think there are other ways to look at it besides mean absolute error. By MAE, yeah, they're no good. Correlation coefficient is another way to look at it. I looked at it another way (as I know you've read) and found that 17 out of the 32 predictions were within 2 games

Chip-By "last year's record" I mean a simple regre...

2007-09-04T15:32:00.069-04:00

Chip-

By "last year's record" I mean a simple regression based on last year's record, not just using a team's previous wins as the projection for next year. See the end note for the formula.

What this does is simply apply a regression to the mean to last year's wins. So a prediction for a 13-win team would be about 11 wins, and a prediction for a 2 win team would be about 5 wins.

FO's projections are not great, but not worthless ...

2007-09-04T15:26:33.559-04:00

FO's projections are not great, but not worthless either.

I looked at the Pearson rank correlation for the FO's and Vegas' projections as well as last year's record:

2006 2005
FO 43% 33%
Vegas 45% 20%
Last Year's 30% 20%

The only pattern I recognize is that expert picks ...

2007-09-04T14:38:25.688-04:00

The only pattern I recognize is that expert picks tend to go out on a limb. They need to validate their existence, so if they pick teams to win within a range of 6-10 games, no one would bother paying attention. But the smartest prediction strategy is to go as short out on a limb as possible (like the 8-win strategy).

I'd put money on teams that Vegas has as outliers: 5 or less wins and

Is there any discernable tendency if we look at th...

2007-09-04T12:00:06.266-04:00

Is there any discernable tendency if we look at the predictions where Vegas/FO/8-8/regression to the mean disagree strongly with one another?

Let me give an example. I have no intention of wagering on team wins. But if I were to place such a wager, I would look for teams where:
- The Vegas over/under and the FO prediction differed by more than 2 games.
- The Vegas over/under