Team Efficiency Rankings - Week 13

GB takes the biggest hit this week after their loss to the Giants, falling from #7 in efficiency to #12. It made me realize that focusing on the rankings exaggerates the movement of teams up and down the rankings. GB actually only dropped 2 percentage points in terms of generic winning probability, from 0.56 to 0.54. That's a fairly modest revision to an estimate of team strength. A lot of movement up and down the rankings can be due to other teams leapfrogging. For example the GWP of NE, WAS, DET, and CIN and NYG all improved slightly, magnifying GB's apparent slide down the rankings.

I discussed CAR last week. The other glaring difference between these rankings and other rankings is BAL. They're 19th in terms of opponent-adjusted efficiency. It's easy to see why in the second table below. Their 9-2 record is largely due to some very good fortune in some high leverage situations, as we witnessed last Sunday. My system here is basically saying they're an average team that's faced a relatively weak schedule so far. If you flip a coin 11 times, it will come up at least 9 heads rarely enough that I'm convinced they are probably significantly better than their efficiency numbers indicate.

The biggest difference between this system and others is that it heavily regresses turnovers. Turnovers of all kinds are extremely random, even for the best and worst quarterbacks and teams. The one category BAL exceeds in is offensive turnovers. While it may be true that the overwhelming majority of teams do not maintain consistent turnover rates, there can be exceptions. Because I follow BAL closely, I see how this might be the case for them. The bulk of offensive turnovers are by the quarterback--interceptions obviously, but they are the prime fumblers as well. Joe Flacco seems happy to accept an overthrow to the sideline on 3rd down or give in to the pass rush while protecting the ball. It can be frustrating to fans, but it may be working for the Ravens. Throughout his career, Flacco's WPA has significantly exceeded his EPA, and that's the case again this season. He's had a few game winning drives over the years, but a lot of that extra WPA comes from playing situational football--taking the smart sack or throwing the ball away when a turnover would be exceptionally damaging. As with most things, the truth is probably in the middle. It may be that part of their consistently low turnover rate is good luck, and part of it is by design.

Here are the efficiency rankings for week 13. Click on the table headers to sort. Raw efficiency data is in the second table below.


RANKTEAMLAST WKGWPOpp GWPO RANKD RANK
1 DEN10.740.5032
2 SF20.710.5023
3 HOU30.630.4869
4 CAR50.620.53105
5 SEA40.600.52126
6 NYG80.570.52820
7 ATL60.570.481319
8 NE90.570.49131
9 WAS110.550.51423
10 CIN120.550.471115
11 DET100.550.50716
12 GB70.540.511411
13 STL150.520.521513
14 CHI130.510.51311
15 TB160.500.50527
16 DAL140.490.531621
17 MIA240.480.482017
18 PIT170.480.48254
19 BAL190.480.471718
20 NYJ180.470.542314
21 BUF220.460.491922
22 SD200.460.492612
23 MIN210.440.502810
24 NO230.440.53932
25 IND270.430.451829
26 CLE260.420.48298
27 OAK290.410.492225
28 PHI250.400.502424
29 ARI280.390.53327
30 TEN300.390.482128
31 KC310.330.503026
32 JAC320.320.502730



TEAMOPASSORUNSR%OINT%OFUM%DPASSDRUNSR%DINT%PENRATE
ARI4.8352.81.36.0584.40.45
ATL7.3353.00.56.5543.10.23
BAL6.4391.80.46.2562.70.50
BUF6.0483.41.96.4542.40.44
CAR7.1403.01.56.0582.10.44
CHI5.4354.01.25.2594.90.40
CIN6.7422.91.15.9532.10.39
CLE5.6383.30.86.0583.10.51
DAL6.6393.21.46.7591.50.46
DEN7.4432.02.25.3593.30.37
DET6.5432.01.56.0581.80.48
GB6.3411.80.75.9542.90.44
HOU6.9422.60.35.7612.70.35
IND6.4432.91.46.6581.40.39
JAC5.5362.11.67.1542.20.45
KC5.4434.42.77.4612.30.38
MIA6.2403.41.76.2621.90.36
MIN5.3412.51.85.8581.50.37
NE7.3480.70.87.2543.30.37
NO6.9392.50.67.7552.00.44
NYG6.8422.80.97.0544.90.27
NYJ5.8392.82.26.3532.30.39
OAK6.3352.71.57.0621.60.41
PHI5.6472.92.66.8602.00.45
PIT6.1332.02.15.1541.80.53
SD6.1383.71.46.3602.20.37
SF6.9502.10.95.1642.70.48
SEA6.5432.81.25.4532.60.44
STL6.1442.81.16.1573.10.49
TB7.3402.00.87.8633.70.42
TEN5.9412.31.87.1572.60.39
WAS7.1471.90.76.9583.00.55
Avg6.3412.71.36.3572.60.42

  • Spread The Love
  • Digg This Post
  • Tweet This Post
  • Stumble This Post
  • Submit This Post To Delicious
  • Submit This Post To Reddit
  • Submit This Post To Mixx

37 Responses to “Team Efficiency Rankings - Week 13”

  1. Anonymous says:

    Is the running Bears/Patriots Best/(almost) Worst and the SF/NE 2nd/3rd pairings unique this season? It seems very odd to have two pairs of teams that mirror each other like that, especially the combination of 1st/31st for the Bears and Patriot.

  2. Dave Archibald says:

    Flacco fumbles kind of a lot - 11 times last year (only Gabbert, Matt Moore, and Tebow fumbled more). Since his rookie year of 2008, no player has fumbled more than his 44 times. This year he's done better (5), but that's still basically just average.

    He is good at avoiding INTs though; since 2008, among QBs with 2000+ passing attempts, only Brady (121) and Rodgers (116) have better INT%+ figures than Flacco's 110.

  3. Anonymous says:

    Brian; Wouldn't it be the case that Baltimore is benefiting from the favorite/conservative underdog/risk taking optimal strategy. Since they are playing a weaker schedule their conservative approach is maximizing results. However in the playoffs this could or has worked against them. Why the patriots/Giants are so successful playign 'aggressive' football.?
    Dan

  4. Anonymous says:

    In response to your claim that SRS is backward-looking, Doug Drinen himself describes SRS as predictive and not retrodictive.
    http://www.pro-football-reference.com/blog/?p=37
    Serious question: why do you disagree?

  5. Boston Chris says:

    Anonymous, what comment of Brian's are you referring to? I don't see anything in this post discussing SRS?

  6. Brian Burke says:

    SRS is awesome. I love it. But it is very certainly backward-looking. It is an opponent-adjusted net point difference. That's all, which is why it's so great.

    However...Every turnover, kickoff return of a TD, tipped pass, fumble recovery, blocked FG, 4th and 29 conversion, etc. is treated as equally predictive as every other event in a game. I think we'd all disagree with that.

    Some events in football, like pass efficiency, are consistent and predictive. Some are very random and inconsistent, like turnovers and special teams events. The GWP system accounts for the differences in predictive consistency.

    That said, simple net point differential is going to be somewhat predictive, but it will be over-fit to past events. That's the distinction I make between backward and forward looking models. The distinction is between "How well has a teamed played?" and "How well is a team likely to play?" Subtle, but starkly different.

    Backward looking systems will always conform with our conventional notions of good and bad teams and their records to-date. And they are deceptively reassuring.

  7. David says:

    Is there anything on the site showing analysis that offensive interceptions are "extremely random"? It feels like to me (and I admit it's a feeling) that you could predict with a reasonable degree of accuracy where quarterbacks would rank in interception rate at the start of the season. It seems reasonable that fumbles and defensive interceptions are largely random, but offensive interceptions seem like a different story, as you allow for a bit in the article. Surely Joe Flacco can't be the exception to the rule... I'm happy to be proven wrong if there's an analysis I've missed though.

  8. Chase Stuart says:

    There are certainly backwards-looking elements of the SRS. Brian is correct in that it treats a blocked punt TD the same as a passing TD, and it ignores lot of useful, predictive elements.

    But I wouldn't classify the SRS as backwards looking, unless your bar for what makes a system forwards looking is really, really high. The SRS, in both college and pro, largely mimics the Vegas lines (except for things like injuries, which all computer systems fail to account for), which is as forward-looking as it gets.

    Brian's right that his system is more forward-looking, but that wouldn't make me think the SRS isn't forward looking. These systems are on a continuum; they don't exist in a binary world. In college football, the BCS is a good example of a backwards-looking system. In the NFL, the standings (and most power rankings) are backwards-looking. But the most forward-looking system out there is what the guys in Vegas use.

  9. Nate says:

    The discussion of large rank swings being misleading suggests that
    it might make more sense to display this data as a ranking/time graph. Of course, with 32 lines, it might just look like spaghetti.

    > But the most forward-looking system out there is what the guys in
    > Vegas use.

    The guys in Vegas are looking to maximize their long term returns, which means they should optimize for high volume balanced betting action, rather than predictive accuracy. That is to say, they're looking forward to the bank balance, rather than the game score.

  10. Anonymous says:

    @Brian - love ya' as always, but "Baltimore is better than the rankings indicate" comes off as homer-ish to me considering you insisted that couldn't be true of Tebow's Broncos last year. I believe the relevant quote is "to the extend that the team protects the football, that's reflected in the rankings."

    @Nate - I want to agree with you, especially since I think Brian's ratings are amazing, Chase's fun and useful within limits, and FO's as useful as putting teams in a RNG and outputting the results (for a low low fee!), but are you sure that's correct? I had read in the past that Vegas preferred handicapping the game rather than the action, trusting that in the long run they'd have the same outcome.

    -sunrise089

  11. Anonymous says:

    Its stunning how consistently bad NE's defense has been throughout the past few years. You would think you'd see some kind of mean regression going on, but that defense just stays stubbornly in the 30's. I wonder if there is any significance to it. I'm thinking its just Belichick throwing money at the offense year after year and completely ignoring the D

  12. TOP says:

    What's good for the goose (http://www.advancednflstats.com/2010/11/whats-deal-with-falcons.html) is good for the gander. Baltimore hasn't had staggering luck in the TO department on defense as Atlanta did, but is near the top in not turning it over.


    But you can't get away with (dis)believing the numbers b/c it does(n't) fit with your (un?)conventional wisdom.

  13. Ken R. says:

    Is NE's defense as bad as it seems or is part of it a product of the offense's efficiency? I.e. offensive efficiency results in more opponent possessions and so on...

    Also, is it Flacco or Harbaugh who really deserves credit for his ball security?

  14. Nate says:

    I meat a graph of rating, not rank, but I think the context covered it.

    >I had read in the past that Vegas preferred handicapping
    >the game rather than the action, trusting that in the
    >long run they'd have the same outcome.

    Let's assume - for the sake of discussion - that the bookie can calculate accurate odds for any handicap, and has good information about what the action would be like on them.

    If the action is (much) bigger than the bookie's reserves (not sure that's true in Vegas) then the optimum long term scenario for the bookie is to have maximum balanced action. That leaves the bookie with no risk and substantial return.

    Even if the bookie has infinite money, the most profitable line is going to split the difference between the action and the game. This is especially true because it may be impossible to handicap to exactly 50/50.

  15. Chase Stuart says:

    Nate, you're essentially saying that bookies would rather have bonds than stocks. Vegas is concerned with profits, not short-term risk. The only game Vegas sets with the goal of splitting the action is the Super Bowl.

  16. James says:

    David, Re: interceptions. I would start here: http://www.advancednflstats.com/2010/11/how-random-are-interceptions.html

    "This is something similar to a study Chase Stuart did last year at Footballguys. He created 25 notional QBs, each with an identical interception rate. He demonstrated that after 500 attempts each, there could be a wide range (from 9 to 22) in total interceptions due to sample error alone."

  17. Anonymous says:

    It's easy to define backwards- vs. forwards-looking. Backwards looking systems use MLE (the model which makes the data most likely) and forwards-looking systems use MAP (the model which the data makes most likely). This is a well-understood, well-defined, and binary difference.

    Of course, this means that all major public sports prediction systems are backwards-looking, but (given that they were all created by sports-fans with minimal statistical backgrounds) didn't we already know that? Sometimes backwards-looking systems produce numbers very similar to forwards-looking ones...

  18. Anonymous says:

    Are there any examples of a MAP model for football in action other than this site

  19. Alan says:

    [Sorry, forgot to add quote marks; would have edited if that were an option.]

    "I'm convinced they [the Ravens] are probably significantly better than their efficiency numbers indicate."

    I really think they are. As I understand it, this model does not account for punting and punt coverage, something I consider one of the most underrated, overlooked elements (particularly in combination with strong defense and ball control offense) that can make a team very strong. And the Ravens seem for years to have focussed on this aspect of the game more than most teams.

    On turnovers, it is disappointing that nothing non-random can be teased out of the noise. Even if it can not be discerned in a statistical way, I don't believe they are truly random overall (even if a majority of them are). Some defensive backs have good instincts for where a QB will throw, combined with good hands to catch it when it gets there; others do not (though those others may "stay home" and make more solid tackles and give up fewer big plays). Some linebackers and safeties have a knack for stripping the ball when making a tackle (again with a tradeoff, as this at the possible expense of missing a tackle entirely), some do not.

  20. Anonymous says:

    Vegas...is most concerned with public perception.There's is not the most 'forward looking system. THey use power rankings that are formed in large part to historical performance.i.e. what they did last year and before. This is done because the bettors remember successful teams & are slow to jump on perennial bad teams.(who is the leader in ATS this year surprise? if you are a successful gamble ryou will know its Cleveland. Could Brian improve his model to use portion of last years stats?Perhaps I have tested correlations and efficiency stats do have moderate correlation to year before.
    The reason why Vegas make money is not because they have the best prediction ability but they can predict what the public will be on. They are experts on the psychology of gambling. With so many few games in one season. It is impossible to predict accurately what 'will' happen.
    Vegas margin of error is really not very good. Brian has showed that ~75% is the upper end of a prediction system
    Vegas falls way short of that.Also many can beat Vegas on a regular basis. Pick the necessary number of winners. But
    get messed up with the psychology part.
    mike

  21. Anonymous says:

    Where would Jeff Sagarin's ratings fall on the forward-looking scale?

  22. Anonymous says:

    "many can beat Vegas on a regular basis."

    Mike, do you have any evidence to support this claim?

  23. Anonymous says:

    hi brian,
    in the article you just wrote for the washington post (http://www.washingtonpost.com/blogs/football-insider/wp/2012/11/28/playoffs-bye-weeks-and-penalties-everything-you-need-to-know/ ), you said that if the skins win all their remaining games that they are guaranteed a playoff berth. i don't think that's the case. the giants could easily win enough to take the division. there are 4 teams ahead of the skins competing for the 2 wild card spots. the way i see it, at least two of them could win out too and stay ahead of the skins.

    it would be awesome if you could come over to the redskins "insider" blog and explain it.

  24. Alan says:

    Anonymous, I too would be curious to see documented support for the claim about beating Vegas consistently. Anecdotally, I did very well for a couple years before the site I used stopped taking American money. If I were in a state or country where I could bet fully above board, I'd be back in a heartbeat.

    As the poster said, it strikes me that the "masses" end up moving lines too far in the direction of favourites with big names and pedigrees, and are too slow to recognise potent, small market up and comers. And most people want to bet "for" a strong team, even when they must cover a spread, so whether the bookmakers shift a line to keep the money even on both sides, or whether they just want to take people for suckers, the incentive is the same: the point spread the strongly favoured team must cover is usually higher than it should be.

    For someone sharp with stats software, it should be easy enough to put in all the cases where a team is favoured by more than a field goal, and see whether (as I'd predict) they actually covered that spread less than half the time. I would be shocked if it weren't so!

  25. Nate says:

    > For someone sharp with stats software, it should be easy enough to > put in all the cases where a team is favoured by more than a field > goal, and see whether (as I'd predict) they actually covered that
    > spread less than half the time.

    Through 2000-2011 the team covered 1004 times, and failed to cover 1028 times, so I guess you're right. (Up to noise, the spread is correct.) This holds if I raise the threshold to 7, 10, or 14 points, though noise becomes more significant at those levels.



  26. Michael Beuoy says:

    Alan - Favorites of more than three points have gone 49.5% against the spread since 2007. Not sure if that is significant. Definitely not a market inefficiency you could make any money off of.

    Source: http://sportsdatabase.com/nfl/query?sdql=line%3C+-3+and+season%3E%3D2007&submit=S+D+Q+L+%21

  27. Nate says:

    > Vegas is concerned with profits, not short-term risk.

    The Vegas line should still be influenced by the action.

    Consider an idealized model where the possible point differential in a game is continuous and has a cumulative probability distribution P: Reals->[0,1] and there's a betting pool of size 1, where the fraction that takes the under when the line is set at a particular value is U:Reals->[0,1].
    Then the bookie's EV for setting the line at L is:
    (2*P(L)-1) * (2*U(L)-1)

  28. Alan says:

    Interesting--thanks, guys. Wasn't expecting a result so quickly. So it's true, but only barely so.

    Can you filter for heavy favourites playing on the road? I have a hunch that the average bettor does not adjust for that as they should.

    When I was doing this for real and making money, I also adjusted for a fuzzier and somewhat subjective sense of teams that had a big rep but were past their prime, aging and on a downward trajectory that hadn't yet trickled through to popular opinion. Helps too if they have an especially large and rabid fan base, like the Cowboys for instance. I'm not sure how you could quantify this...and of course with my tiny sample size I do understand it's entirely possible I just got lucky.

  29. Anonymous says:

    Nate There is a famous article/study written on how Books set lines to maximize profit. (sorry can't remember the name but I will find it) It explains that a "combination" of reducing outstanding liability & taking position on some games does this more than splitting action. To do this as I said they take advantage of betters psychology how they react to big wins big losses/ popular team etc.


    As for proof on beating Vegas. There are some documented online Dr. Bob over huge sample a volume approach. Byeweekpicks using a selective smaller number of games approach I make a healthy second income over last 10 years my self.
    I can give you other evidence. Underdogs on a whole cover more than favorites.This +ev increases when you look at teams with lower public betting on them. (ie fading the public) yes large underdogs are another +ev double digits are best. And, teams off large losses or off of powerful wins also has +ev in very large samples. Like I said its easy to find +ev opportunities over large samples the hard part is actually 'pulling the trigger on putting your hard earned cash on god awful teams (I did on Jacksonville last week And KC vs Pit a few weeks ago on Monday night. even though I knew analytically I had value and on right team. Vegas knows in a psychological sense it is very hard to back a loser.As they say everyone loves a winner and to be on a dominant team. Also betting certain teasers blind are proven to be +Ev.

    Also there are a large selection of people who lose well be one their expected return. This is because they dont flat bet they cut into edge by using star 1*2*3*5* systems (crazy) Or chase after losses by increasing bets.or play sucker bets.

    Mike



  30. Anonymous says:

    I'm wondering if you could tweak the model to have different regression rates for different teams based on how consistent they are

  31. Kulko says:

    Is there any Proof, that this model is a MAP model?

    is there any Proof, that it is statistically better than FOs model when forward predicting?

    But from the way I see it, both models makes significant unproven assumptions what is relevant and what is not, and just play it from there. Both models include assumptions that i deem downright unlikely.

    Both models have year in year out teams, where there data not only looks wrong, but stays wrong during the whole season.

    Both models are significantly better than typical power ratings or won/loss.

    Any proof that one of them is statistically better than the other are notably absent.

  32. Anonymous says:

    @Kulko

    It doesn't matter. When you start out with the absurd assumption, as FO does, that past turnover rates are solidly predictive of future turnover rates, then the rest of your model is worthless.

    The biggest flaw with Brian's model is that people don't understand that a team having WP of 60% over another team doesn't mean that they will win every time.

  33. Anonymous says:

    The biggest flaw with Brian's model is that it is actually not very predictive.

  34. Anonymous says:

    First, there are no (including Brian's) active MAP-oriented models for ANY professional sport I am aware of. The whole argument about which model is "more forwards-looking" is silly.

    Second, it is relatively easy to look at prediction quality when win probabilities are involved (I mentioned this before and actually computed them for the ANS model for a week). This can be done using the normalized average negative log likelihood over a set of predictions. I have privately computed these numbers for the ANS models for a few more weeks, and compared them to some simple alternatives. I don't have the numbers handy, but ANS beat not only the 50/50 on all games predictor (i.e. it's better than random noise), but also a home-field based predictor with parameters optimized on the test set (in other words, as over-fitted as it could possibly be).

    Sadly, I don't know anywhere that that has historical numbers for straight-up win/loss bet payoffs from Vegas, and this is the "gold standard" everyone wants to compare against. It might be possible to convert lines into implied win probabilities, but I don't know how. If someone has these numbers, I'd be grateful for them, and happy to use them to compare models' accuracy.

  35. Mike M says:

    The big "regression to the mean" team this week is the Bengals.

    The model has them favored, they will likely lose and maybe big.

    Another "regression to the mean" team the model has right is the Patriots who are due a regression to the mean but will likely pull-out a very close win but they will not cover the spread, however, a loss would not surprise me at all.

    A 'regression to the mean" team which should play much better is the Raiders.

    Something to think about, in regards to turnovers being random, Super Bowl winners almost always win the TO battle in the regular season and continue to win the TO battle in the post season.

    If TO's are random then why don't we find teams losing the TO battle in the regular season completely reversing coarse and winning the TO in the postseason and winning the SB on a regular basis ?

    Last season this was the Saints who were very highly rated on this site but won the TO battle in only 5 games in the regular season and then were taken out behind the woodshed for a TO beating by the 49ers in the playoffs when the 49ers had the most games winning the TO battle in the league but were only ranked like 12th or some such thing last season by this site ?

  36. Anonymous says:

    Regression happens over a long period expecting it to happen in one or two games is foolish Mike...Season to season shows consistent inverse correlations with TO's providing excellent proof of To's randomness...
    time to take a basic stats course?

  37. Boston Chris says:

    @ Mike M. lol

Leave a Reply