## Significance

If you're going to be doing sports statistical research, this is required reading. Click to expand. (From xkcd.com)

### 9 Responses to “Significance”

1. DSMok1 says:

For more in-depth discussion on similar matters, check out Andrew Gelman's slides on "Statistical Challenges in Estimating Small Effects": http://www.stat.columbia.edu/~cook/movabletype/archives/2011/04/my_talk_at_nort.html

2. Andy says:

The problem described in XKCD is a multiple comparisons problem (http://en.wikipedia.org/wiki/Multiple_comparisons). It has basically nothing to do with problems of estimating small effect sizes, except that they're both statistical pitfalls.

3. David says:

Don't forget the alt-text:

So, uh, we did the green study again and got no link. It was probably a-- RESEARCH CONFLICTED ON GREEN JELLY BEAN ACNE LINK; MORE STUDY RECOMMENDED!

4. SportsGuy says:

Ups to DSMok1 for posting that; Taleb's stuff has really hammered home this for me too.

5. tally says:

More or less, he didn't use a Bonferroni's correction for multiple comparisons.

6. underpoint05 says:

I would guess that Gelman would say that effect sizes are rarely precisely zero, and that almost all testing problems can be analysed at least as well by treating them as estimation problems. According to this philosophy, the problem is to simultaneously estimate the effects of all the different jellybean colours, and in any sensible way of doing this it will be plausible that all the effect sizes are basically zero.

7. Tim says:

lol i love it.

EXACTLY why stats are worthless without common sense.

8. Tarr says:

Tim, I think it's more of a case of "stats can be misleading if you don't understand how stats work". Although that's less sexy, I guess.

9. Anonymous says:

The real joke is that they're frequentists... they should join the 21st century and switch to Bayesian.