Jan 27, 2010

Win Probability Added (WPA) Explained

I’ve begun using some new stats here lately, and I want to take the opportunity to explain them clearly. Readership has been increasing rapidly too, and some readers aren't as familiar with the terms we use around here. This post will serve as a reference each time I use the new stats in the future.

Although it’s been used in baseball sabermetrics for several years now, Win Probability Added (WPA) is a new, or at least rediscovered, concept for football stats. It measures each play in terms of how much it increased or decreased a team’s chances of winning the game.

WPA starts with a Win Probability (WP) model of the game of football. Every situation in a game gives each opponent a particular chance of winning, and a WP model estimates those chances. The model created here at Advanced NFL Stats uses score, time, down, distance, and field position to estimate how likely each team will go on to win the game. For example, at the start of the 2nd quarter, a team down by 7 points with a 2nd down and 5 from their own 25 will win about 36% of the time--in other words a 0.36 WP.

On that 2nd down and 5, let’s say there is a 30-yard pass, setting up a 1st down and 10 on the opponent’s 45. Now that team has gone from a 0.36 to a 0.39 WP. The WPA for that play would be +0.03.

If instead the quarterback throws an interception returned back to the line of scrimmage, the opponent now has the ball at the 25, giving the trailing team a 0.28 WP. The WPA for the interception would be -0.08.

WPA is very sensitive to the context of the game. That same interception that cost -0.08 when a team was down by 7 points in the 2nd quarter would cost much more if it the offense was leading by a point late in the 4th quarter. Putting your opponent in immediate field goal range would be nearly fatal.

Stats are tools, and each tool has its own purpose. WPA is what I call a narrative stat. Its purpose is not to be predictive of future play or to measure the true ability of a player or team. It simply measures the impact of each play toward winning and losing.

WPA has a number of applications. For starters, we can tell which plays were truly critical in each game. From a fan’s perspective, we can call a play the ‘play of the week’ or the ‘play of the year.’ And although we still can't separate an individual player's performance from that of his teammates', we add up the total WPA for plays in which individual players took part. This can help us see who really made the difference when it matters most. It can help tell us who is, or at least appears to be, “clutch.” It can also help inform us who really deserves the player of the week award, the selection to the Pro Bowl, or even induction into the Hall of Fame.

As interesting as that might be to us as fans, WPA might be even more useful to coaches and strategists. We can measure whether a kicker’s high field goal accuracy was worth a trade-off in short kickoffs. And for the first time, we can measure the effectiveness of clock management strategies—is it better to run three times and punt, or pass and go for the first down to end the game?

Even better, WPA can measure the risk-reward balance of a given situation. For example, when is it best to take a sack, and when is it best to take your chances throwing into traffic? Or for defenses, when is it best to roll the dice on a big blitz and risk a long completion due to a big hole in the secondary? We can measure which interceptions were the most costly, and which quarterback tends to throw the costliest ones. WPA may actually be able to measure decision-making ability on the field and on the sideline.

WPA is not ‘the one stat to rule them all,’ and it does have its disadvantages. It can certainly be improved in several respects, and I’m already working on several ideas. As time goes on, we’ll find even more new and interesting applications for it, and this is only the beginning. More than any other sport, football is about strategy, risk, and reward, and WPA is well suited to capturing much of what makes football so compelling.

20 comments:

  1. nitpicking but second to last paragraph wHOLE
    ReplyDelete
  2. Hi Brian - I've often wondered why you don't include time outs remaining in the WP model, since they seem like they would be very important to 4th quarter scenarios. Is it a correlation issue because teams almost always have all 3 in the first and third quarters, or is it a sparseness issue where there aren't enough 4th quarter data points?
    ReplyDelete
  3. It's sparseness. I'm working on some methods now. One idea would be to equate a timeout to a certain amount of time added or subtracted if on defense or on offense.
    ReplyDelete
  4. I'm not so sure about it being able to tell us about the individual player. Chris Johnson appears to be a great running back but if he were on the Seahawks there's no way his WPA numbers would be anywhere close to as high. I suppose you could compare players on the same team but even this runs into the problem of game theory (ie. Johnson is hurt so the Titans opponent defends more against the pass which in turn makes the backup RB look better than he actually is.)
    ReplyDelete
  5. First off I would like to say I am a big fan of this website. Keep up the good work Brian.
    This may be a little off topic but, I have noticed in many of your posts you use WP to calculate how crucial a play was for team X to win or whether a coach Y made the right call. On these posts you often have to qualify your assessment by saying WP is based on league average stats. This is also a point many of your critics often use to discredit your work. I have very little education in statistics, but I have always wondered if it is possible for you to develop a "matchup coefficient" that I guess would be some kind of ratio of the strength of one team's offense vs the other team's defense on any given play type. In other words the next time there is a controversial 4th down between a high ranked rushing team vs a poor rush defense you could factor that in to your equation to make going for the 1st more preferable.
    If such a variable were possible you could make your predictions more specific to each situation and I hope it will quiet some of your detractors who feel your calculations are not in line with their gut feeling. That kind of commentary was especially bad on the Belichek 4th and 2 topic last November.
    ReplyDelete
  6. The problem with the 4th and 2 detractors is that nearly all of the situational adjustments (i.e. Manning is awesome, the Pats defense was gassed, Brady had been pretty efficient) point MORE toward going for it.

    Anyway,

    Brian, I agree with you entirely when you say "WPA is what I call a narrative stat. Its purpose is not to be predictive of future play or to measure the true ability of a player or team. It simply measures the impact of each play toward winning and losing." As long as it is used and treated as such, it's a meaningful stat.

    I suppose in the most literal reading, this makes using WPA for MVP a pretty meaningful thing to do. However, this is why I think that, for example, the EPA measurements of punters were far more meaningful than the WPA measurements. The punters don't really have much control over the game context, so the relatively context-free EPA tells us more about their value.
    ReplyDelete
  7. Derek

    Or alternatively show that no matter what offense/defense combination is on the field, the chances of converting e.g. 4th and 2 are pretty steady across the league.

    One shouldn't change the facts to match the theory (see Climategate). Just because people may have a gut instinct about these things, the stats may well show something completely different (e.g. the WP model shows that statistically teams should go for 4th and 1 pretty much anywhere on the field, in marked contrast to commentators who 'know' that you never go for 4th down in your own half and are even dubious about 4th and 1 inside the 'maroon zone').

    That said, it would be interesting if team-specific adjustment could be made, if not for WP then perhaps just for 3rd and X conversion rates.
    ReplyDelete
  8. Very interesting stuff. I was looking through the wpa and epa for players last year, and there's a piece of data missing that I'd very much be interested in seeing (and hopefully you'll be interested enough to do the work for me!).

    I figured that for running backs epa is probably more important than wpa because running backs tend to run more at times when wpa won't be affected (eg at the end of a blowout).

    For running backs especially (but for the other skill positions as well), I'm interested in knowing what their epa/play are depending on the particular win probability of the game. I was thinking if you made a graph of all running backs and had epa/play on one axis and win probability of the game on the other axis, what that graph would look like. Since running backs are often given the ball to let the clock run down when the team's win probability is extremely high, I wonder if that will change the stats of a running back on a good team vs a bad team. But in any case, I was hoping you could not only put up epa stats, but epa/play stats within ranges of win probability (for instance, epa/play when win probability is between 30 and 70 percent...ie when it's still a game...the specific limits could be determined by looking at the curve of the graph mentioned above).

    I think that would provide novel information for the stat.

    Also, an adjustment for the quality of opponents would be interesting.

    Also, for quarterback stats, I was curious to know what counted as a "play." I divided epa by epa per play and couldn't figure out what that number of plays referred to. I would assume for quarterback it's all plays that aren't handoffs? Does it include penalites (which I think would actually be a good idea)?

    Thanks for a fantastic site, hope you find my comments worthwhile.
    ReplyDelete
  9. Sorry, one last thing. For receivers, I think it would be a good idea to have another stat that's epa/play where the "play" isn't a pass thrown to them, but rather any passing down in which they were on the field. Since getting open so that you're passed to is just as important as making a catch.

    PS: I also can't figure out what counts as a play for a running back. I added up carries and number of times thrown at and it's still short by a few dozen plays. Penalties?
    ReplyDelete
  10. Sorry, one last thing. I think for receivers it would be worthwhile to have another stat that's epa/play where play is any passing down in which the receiver participated. Since getting open is just as important as making the catch for a receiver.

    PS: how did you calculate plays for running backs. I added carries and targets for receptions and it doesn't add up. Penalties?
    ReplyDelete
  11. Anonymous said...

    nitpicking but second to last paragraph wHOLE
    Either it was fixed or you are one of the dumnest people of all time
    ReplyDelete
  12. "Dumnest...?"

    Yeah, you tell 'em, Einstein :P
    ReplyDelete
  13. Two Jet WinstonOct 28, 2010 10:19 PM
    There's been a lot of speculation about the statistical benefits of always going for it on 4th down. Where does WPA come out on that?
    ReplyDelete
  14. Greetings, Brian

    I wish to congratulate you for all the great work you have done in developing WPA. Undoubtedly, WPA is an immensely valuable tool in analyzing the game’s situational strategies. Its value in this respect should not be underestimated. Again, I congratulate you for your great success in this regard.

    When WPA is applied to evaluating a player’s value, however, I believe WPA offers very little utility unless the time-sensitive feature of the model is omitted. I submit that the nature of the game is such that an individual player’s true value cannot be truly discerned by any quantitative method that seeks to add the parts of a game into a quantifiable whole. While I concede that the value of a player’s contribution may loom especially large AT EXACTLY THE MOMENT IT OCCURS in a game, the story is altogether different when the game is over. When the game is over, it is no longer possible to break it into parts to determine the relative significance of each play. Instead, the game can only be taken as a whole. Plays that happen to occur early in the game are just as determinant to the outcome as plays occurring at the finish. That is the unfortunate reality.
    ReplyDelete
  15. Hey Brian,

    I'm a new reader and I really love the site. I recently read Moneyball (the book about sabermetrics in baseball) and I'm a huge fan of the application to football. I have a quick question that sort of relates to what Scott said above about putting Chris Johnson on the Seahawks. How does WPA interact with offensive lines? Under statistics on the website you have WPA and EPA for offensive lines, but how do you know when to attribute WPA to offensive lines as opposed to RBs (or QBs or WRs on pass plays, for that matter)?

    Please respond, I'm very interested in how it works. Thanks so much!
    ReplyDelete
  16. I too am appreciative of the effort and thought that goes into this site, and look forward to advances in these techniques. Like some others i too am a bit skeptical about the ability to translate WPA from team results to an individual. The example given to explain WPA is the impact of a 30 yard pass under certain conditions. Certainly there is an impact, but can you use that to "rate" the play of the qb? Some teams virtually never throw a 30 yard pass. And that is not always due to the quarterback's abilities. Sometimes it is a matter of offensive philosophy. Alternatively the team might want to throw long passes but know that their tackle tandem can rarely hold off a rush long enough to set up a long pass. So the ability to complete a long pass reflects the whole team's capabilities as well as even those of the coaches. In the end differences in WPA for different qb's may say nothing about their relative abilities but more about the relative abilities of the teams on which they play.

    Other articles on this site indicate that more passing equates to more wins. It would seem then that teams that pass more would increase the WPA of their qbs. Comparing the WPA of such a qb to a qb on a team that emphasizes running more would tend to make the qb on the "running team" look less good, even though the difference might reflect differences in the extent to which those teams coaches understand the relative value of running vs passing. No? I think this is the similar to the comment made by the third post in this thread.
    ReplyDelete
  17. I agree. But there is a distinction in the purpose of various statistics. A 30-yd pass has many components, influences, some of which are random. However, no matter how you look at it, QB Steely McGunner actually threw that pass, and WR Speedy O'Grabby actually caught it and ran it for 30 yds. In the immortal words of Ricky Bobby, "That just happened."

    In other words, WPA captures the narrative of what happened and puts it into a number. It does not divide credit or blame among players. It does not pretend to predict future performance.

    We say 'so-and-so made a big play' all the time. WPA just puts a number on 'big'.

    But it's more useful than that. Were Barry Sanders' frequent tackles for losses worth his explosive gains? WPA would tell us. Were Kurt Warner's turnovers in STL worth his big arm? WPA tells us. Are Adrian Peterson's fumbles worth his big runs? WPA tells us. We no longer have to conjecture about these things.
    ReplyDelete
  18. Do these points get distributed evenly - i.e. on a pass play that increased WP by .03, does the QB and WR get .03 points?

    How about a DB, if he knocks away a pass that would have increased the offense's WP by .05, does he get the .05? I'm guessing no because that doesn't show up as a stat in game logs.

    How about something like a hurry on a QB by a DE or LB that results in an incomplete pass - do the defensive players get those "avoided" WPs?
    ReplyDelete
  19. There's no perfect way to do it. In my implementation, I give full credit for the play to all those directly involved. For example, say there is a play in which a QB passes to a WR, and that play takes the team from a 0.40 to a 0.50 win probability. The play as a whole is worth 0.10 WPA. Both the QB and the WR get credited for the full 0.10 WPA (just like both get credited for the yards gained in conventional stats).
    ReplyDelete