Comments on Advanced NFL Stats: Bill Polian Doesn't Get It

Without commenting on whether or not Bill Polian &...

2010-06-09T11:48:13.965-04:00

Without commenting on whether or not Bill Polian "gets it", I was in the audience for this panel and my interpretation of his comment was that statistics *predicting the outcome of individual plays* are useless because the sample sizes are too small. How is that possible? Because who's on the field, situation (time/score/down/yards to go), etc. all matter and those combinations (plus a given play) aren't repeated often enough over the course of a season to have a reliable dataset to model off of.

One can quibble with the viewpoint that a play outcome model built at a less granular level is useless, but I think that's what Bill meant.

Actually, I think Polian gets it. What he says ou...

2010-04-19T22:13:31.603-04:00

Actually, I think Polian gets it. What he says out in public at a conference and what he actually does or uses in the front office are not necessarily the same thing. Most of the media would love this quote. Its basically just good PR for him. No more, no less.

Jeff, This is true. If I were the GM of an NFL t...

2010-03-09T23:34:50.404-05:00

Jeff,

This is true. If I were the GM of an NFL team, I would immediately hire Brian Burke (i.e, someone very good with numbers) and a Tony Dungy or Bill Cowher (someone with a lot of experience with football). I don't understand why so many front-office types put all their eggs in one basket, instead of doing this sort of thing.

It's funny that you should mention Simmons; he once suggested a "closer" role for NFL coaches (specifically for Andy Reid), to make important (read: late) decisions for them, since guys like Reid seem to have no idea what they're doing with the game on the line. Simmons is, of course, the kind of guy who writes from a fan's point of view, and this is a particularly silly argument, but he does have a point: NFL clubs put way too much emphasis on "football knowledge." Just like MLB clubs that put all their stock in scouting "5-tool players," while Billy Beane's laptop is telling him that some fat schlub with a .460 OBP is the best pickup he can make.

Simmons, by the way, is not completely anti-stats: he's a big Daryl Morey supporter (the Rockets GM who's revolutionizing sabermetrics in basketball). I know he's not on the forefront of statistical analysis, but he does occasionally recognize what works. And in the case of the 4th-and-2 play, he does defer to his Pats fandom, but also to his experience watching tons and tons of football. So really, I wouldn't blame him for pointing out the particulars of the drive (timeouts wasted, bad clock/down management) and why they shouldn't have gone for it in that case, because he's probably right.

In fact, I'd probably hire him as my 3rd in command, since I think having a "common sense" guy, like he talks about, could do a lot of good in the long run. I think too many people think the answer is either "stats" or "experience," and not enough people realize it's a delicately balanced combination of both. Polian might have been wrong by saying the statistics "mean nothing," just like Brian Billick was wrong by saying every 4th down is 50/50, but I actually think Polian's siding with the statisticians on this one, for the most part, since he's commending Belichick for sticking to his guns even though the traditional "experts" (like... Dungy!) said the numbers were against him.

Like Alchemist said, I really don't blame Polian for not trusting the stats alone, but nobody ever trusts what they don't understand: I really wish they would at least pay more attention to them.

I'm not sure I'd go so far as to say he...

2010-03-09T20:07:30.615-05:00

I'm not sure I'd go so far as to say he's a narcissistic know it all. That wasn't exactly my point. My point was that almost everybody in positions of power needs to play the political game at least to a certain extent in order to keep their job. I think thats what Polian is doing.

I also think he realizes that Belichick made the right call. That puts him a couple steps ahead of most of the football establishment.

Jeff, you are absolutely correct when you say that...

2010-03-09T13:38:39.592-05:00

Jeff, you are absolutely correct when you say that Polian does't get it. Polian is such a narcissistic know it all, that he refuses to give credit or legitimacy to any statistic that could possibly be contrary to the nonsense he spews.

As a Jaguars season ticket holder, I am definitely...

2010-03-09T08:57:10.370-05:00

As a Jaguars season ticket holder, I am definitely in no mood to defend Bill Polian. But in all fairness, I have to admit that, until quite recently, sports statistics were often the tool of the most egregious wannabe sports "expert".

When I was growing up, the only consistent use of statistics was as a pseudo-mathematical tool pulled out by sports writers and announcers whenever it supported the point they were trying to make. (And of course, statistics were conveniently ignored when they did not support the chosen point.)

People often cherry pick completely out-of-context stats to "support" some absurd point. I don't even think that most of those people realize (or care about) the errors in their logic. This is why we hear things like, "There are lies, there are damn lies, and there are statistics." I think that saying really speaks volumes because it arises from the frequent (blatant) manipulation of statistics to "prove" misguided points.

For this reason, I find it hard to have too much contempt for the old guard like Bill Polian. They have experienced great success by trusting their instincts. And for most of their lives they were absolutely correct to dismiss the rantings of those who preached the gospel of statistical analysis because until fairly recently almost all statistical analysis in the sports arena was crap.

Ridiculous that Polian would say that the historic...

2010-03-08T17:29:07.226-05:00

Ridiculous that Polian would say that the historical stats are irrelevant. He presented some good reasons why the numbers should be adjusted based on the game situation (increasing the Colts chance of scoring because of the Pats' gimpy defense and Manning being hot; increasing the Pats' chance of picking up the first down because of Brady, Welker, and Faulk), but to totally throw out the historical numbers is even more ridiculous than considering the specifics of the game.

Football may never be quantifiable in quite the same way as baseball, but to say that statistics add nothing is wrong.

We at HSAC are very appreciative of your work, Brian.

I think the biggest problem comes from people who ...

2010-03-08T15:58:34.227-05:00

I think the biggest problem comes from people who see statistical analysis as a direct threat; they fear that they will be "replaced by a computer" or some such nonsense and so they try to discredit any analytical work they can in an attempt to avoid what they think is coming. (Of course, if their jobs could be done solely by computers, it would happen eventually, as it does in other industries.)

I don't know anyone who intends to replace "conventional wisdom" with analytical tools ... at least I don't think I do. Everyone I see presents their tools as providing additional ways of looking at things or giving different views of conventional data. The coaches that get it (like Payton and Schwartz) have more weapons at their disposal. The coaches that don't, well ...

I don't know if you can tell exactly what Polian really does think unless he does an interview with an analytical site and compliments their work. People can say pretty much anything to the press to present a certain position on things.

Yeah. You're right. As I said, I'm pretty...

2010-03-08T12:19:08.643-05:00

Yeah. You're right. As I said, I'm pretty sure all of the football insiders would prefer if the internet didn't exist. The way they see it, a couple of kids with spreadsheets are arguing that they know more than these old men that spent 30 years of working 80 hours a week to get to where they are now. To make matters worse, the kids with spreadsheets are actually winning the argument. We have the exact facts and figures. All they have is cliches and empty talking points. Its fairly easy to see that a lot of the cliches contradict each other.

The article you linked to referred to MIT as "enemy territory". I think they meant that as a reference to the Patriots. It was enemy territory, but not for that reason. I'm not sure there is a 50+ year old sports executive that doesn't quake with fear at the mention of the name Theo Epstein. Think about what happened with the Red Sox. A new owner came in and decided that he was tired of the old way of business. The old way of business had succeeded in turning his team into the sporting world's single most frustrating franchise. When every other owner would just hire another veteran with 30 years of experience reciting the same old cliches, the Red Sox hired a 28 year old kid willing to shake up the whole way of thinking. As a result, the lovable losers won 2 World Series in the next 5 years.

I can guarantee that when Polian and the other GMs go out for a beer after the league meetings, Epstein's name comes up and not in a good way. They know that the owners know this story. They know that is only a matter of time before a smart NFL owner decides to hire his own 28 year old with a spreadsheet. If stats are all that matters, then why would you hire a 60 year old with little computer knowledge on how to extract the right stats. You wouldn't. Thats what prompted him to fly to MIT (the heart of enemy territory in almost every conceivable way) and talk in a forum entitled "What the Geeks don't know"

As I said before, I've looked at his resume and experience and I think that Polian does get it. Its obvious his job is in no immediate danger. He's the best of the old school GMs. That doesn't mean he doesn't still see the threat on the horizon. It doesn't mean he doesn't feel sympathy for men he's worked with for 20 years getting canned in favor of young turks. These guys also depend on each other for trades. If both the Colts and Redskins are offering the 49ers roughly the same thing, Polian wants to be on the 49ers good side.

Really long response...Bottom line: Don't take any of this at face value. Be aware of the internal politics going on here. That is what is really guiding this debate.

Jeff-I think it was more backhanded than that. He ...

2010-03-08T09:16:44.149-05:00

Jeff-I think it was more backhanded than that. He was being dismissive. It was more like, "Your resume of skills and experience mean nothing..."

I think our words sometimes betray our true (and often hidden) thoughts and feelings.

Side note: The Globe article that article linked ...

2010-03-08T03:40:03.552-05:00

Side note:

The Globe article that article linked to had Bill Simmons saying:

"Yeah, but the Pats were a mess on that drive."

I find it hilarious how the layperson "analysis" that follows complaints about serious statistical analysis always seems to rely on absurdly small sample sizes.

I agree totally with Phil. The football insider...

2010-03-08T03:35:26.232-05:00

I agree totally with Phil.

The football insiders (announcers, coaches, GMs, etc) really want to be the experts. They see sites like this one as second guessing them and that makes them feel extremely defensive.

Think about the resentment a lot of doctors feel towards the internet. A generation ago when you went to the doctor, you told him your symptoms and then he told you what the disease could be. You then thanked him as an all-knowing genius. Today, you can get a list of possible diseases very quickly by doing a little research on the internet. For the diagnosis portion, you're really only going to the doctor because he can order tests that you can't do at home. Since you already know the possible diseases, its almost like a pop quiz when you ask the doctor "what do you think it could be". Good docs don't feel threatened by this, but bad doctors really resent it when a patient says "don't you think you might be overlooking..."

I actually think that you might be overreacting a little bit to the Polian comments. I think Polian is for the most part on our side. Polian knows that saying that Belichick was right means saying that every other football insider for the last 20+ years has been wrong. He is probably very well aware of the backlash that Billy Beane took from the rest of the baseball insider community for Moneyball. I think he also knew that the headline would be "Polian says Belichick was right". I think he just wanted to temper that headline a little bit by throwing the other side a bone. To me that quote just read like the "Although your resume and experience is impressive" line that comes before they tell you that won't be hiring you.

The unfortunately unveiled fact of all this is how...

2010-03-08T03:33:42.535-05:00

The unfortunately unveiled fact of all this is how sports reporters are equally thick-headed.

How in the world did anyone pass up the comic opportunity to ask Mr. Polian to explain what exactly he meant by "most successful"?

Well said! It's funny ... they say they don&#...

2010-03-07T22:23:27.436-05:00

Well said!

It's funny ... they say they don't like statistical analysis, but what they really don't like is statistical analysis that they can't contribute their expertise to. They LOVE statistical analysis when they get to use those vague terms like "pretty strong likelihood" ... but let someone say "87%," which is basically the SAME ANSWER for the SAME QUESTION (because, after all, 87% IS a pretty strong likelihood), and, suddenly, that needs to be dismissed, because it's beyond their understanding.