Comments on Advanced NFL Stats: Elo Ratings

JTapp "somewhere I saw a statistics paper ci...

2010-04-21T22:43:53.856-04:00

JTapp

"somewhere I saw a statistics paper cited that in the NFL, teams are sufficiently "sorted" after 10 games"

is that 10 games or 10 weeks of games?

Interesting. I have been using a modified ELO rank...

2008-06-11T15:15:00.000-04:00

Interesting. I have been using a modified ELO ranking for the last 3 years in my office pool with great success (~70% accuracy), although I am considering using a different approach this year.

I think it was on this site, or maybe Sabermetrics...

2008-05-20T12:31:00.000-04:00

I think it was on this site, or maybe Sabermetrics, but somewhere I saw a statistics paper cited that in the NFL, teams are sufficiently "sorted" after 10 games. There are fewer NFL teams than there are chess players, therefore fewer games would be needed.

I would think that Elo would still be a decent measure of ranking after 10 games. The % chances it puts on winning may not be accurate until many more games, however. The world may never know.

Sandbagging chess players impugn themselves. As to...

2008-05-16T14:14:00.000-04:00

Sandbagging chess players impugn themselves. As to Halo and Xbox, you'll know you've reached a certain age when after spending your entire life explaining the latest electronic stuff to others they start explaining it to you.

My point about the number of games needed for a reliable Elo rating is for predictive purposes. Looking backward, if one wants a strength-of-opposition weighted ranking of what a team *actually did*, Elo does that just fine. Nobody can argue with it.

Predictively, however, for what a team is expected to do in the future, I believe there are too few games in the NFL season for the Elo system to show its stuff (at least until the very end of the season). That's from my memory of how the system works, which is old, maybe Sagarin or somebody has since then pushed the system's bounds, I dunno. But if that's not so I'd certainly prefer a Pythagorean or some other method for making predictions in the mid-NFL season.

If I'm right about the predictive limits with this number of games it may relate to the Pats 99% win rating. That's a good measure of how they actually performed last year objectively, 18 wins and a loss to the #3 team, but if they could magically replay last season I'd really hesitate before predicting a 99% win performance for them again, as good as they were.

Footnote: I have a copy of the college ranking paper, it doesn't have anything to do with sports but if you actually are interested the price would be right. Also, Elo himself wrote a book about his system and its ranking system predecessors and their problems. It doesn't have anything to do with sports either, but history-wise it's interesting -- today's "post-Bill James everybody has a PC and their own rating system on it" world is very different from back when people were building systems from first principles using pen, paper and mechanical calculators. The book is long out of print but probably would be in a good library. FWIW.

JG brings up a very interesting point, since the N...

2008-05-16T13:51:00.000-04:00

JG brings up a very interesting point, since the NFL ratings can hardly be said to have converged after a mere 16 games per team. There is likely still some sensitivity to the initial ranking (which I presume was all 1500?). The Pats, for example, are getting big credit for their early wins over their division rivals before we found out how terrible that division was from 2-4. What happens if you re-run the season using the given rankings as the initial? Or last year's rank?

JG-Very interesting about the college rankings, bu...

2008-05-16T10:59:00.000-04:00

JG-Very interesting about the college rankings, but I'm too cheap to lay out the 5 bucks for the paper.

By the way, I didn't mean to impugn chess players when I mentioned inflation. I was referring to the kids who play Halo on their XBox all day.

"Often players will set up phony opponents to beat...

2008-05-16T10:35:00.000-04:00

"Often players will set up phony opponents to beat in order to inflate their own scores."

As a former tournament chess player I can report that in the chess world at least the problem practice is the opposite. Players "sandbag" to deflate their ratings to be artificially low, so they can enter the lower-rated sections of tournaments and have a better chance of winning the prize fund.

The Elo system can be used to measure the outcome of virtually any kind of ongoing, repeated competition.

E.g.: It's been used to rank univerisities on the basis of which one a student chooses to attend when accpeted by two of them. http://www.nber.org/papers/w10803

This was actually pretty interesting, because many of the rankings were very different from those published by US News and such, meaning students value things rather differently than the "rankers" do.

Using the 4-digit Elo scale of the chess world, with a 200-pt gap meaning the higher-rated wins about 75% of the time, the top and bottom schools of the 105 rated were, if anybody wonders ...

1. Harvard 2800
2. Yale 2738
3. Stanford 2694
4. Cal Tech 2632
5. MIT 2624
.............
101. Purdue 1525
102. Colorado State 1513
103. Syracuse 1506
104. Scripps 1479
105. Loyola 1221

But there's a problem with applying the Elo system to the NFL, I think. To have a reliable, accurate, stable rating for a team you need the outcomes from more games than there are in a season. When I was a chess player (which admittedly was a while back) IIRC one needed the results from 24 to 40 games to get a real rating (depending on the flavor of the system and the purpose it was used for), until then a rating got an asterick* as being provisional = unreliable, dubious.

I haven't looked at applying the system to the NFL per se, but I don't see why this problem would go away.

Phil-Probably not, but strictly in terms of win/lo...

2008-05-15T20:19:00.000-04:00

Phil-Probably not, but strictly in terms of win/loss they were 18-1. Their only loss was to the #2 ranked team.

Justin-I tried to figure it out. It's not an average of the two, nor a weighted average. It's slightly and non-uniformly biased toward the margin of victory system, especially at the extremes of very good and very bad teams.

"Belief" doesn't play into it, it's what the formu...

2008-05-15T11:42:00.000-04:00

"Belief" doesn't play into it, it's what the formula says.

Have you figured out his secret formula for his "overall" rating, the synthesis between Elo and Pure Points?

Interesting ... the probabilities in the last tabl...

2008-05-15T11:02:00.000-04:00

Interesting ... the probabilities in the last table look a bit extreme. Does Sagarin really believe that the Patriots are 99-1 favorites against an average team?