UPDATE: Just found this: Nature has finally retracted an infamous 2005 paper on body symmetry, dancing ability, and sexual selection. The lead author, William M. Brown, duped his co-authors (including leading evolutionary biologist Robert Trivers) by faking the data, and has been unaccountably defended by one of the co-authors, Lee Cronk. Trivers has been trying for years to get the paper retracted, going to far as to self-publish an entire book documenting the fakery in exhaustive detail. A Rutgers University investigation backed Trivers’s account. It’s not an entirely happy ending–the retraction notice contains no details (why the heck not?!), and Brown apparently still has his job. And count me among those who think Nature should’ve retracted this thing years ago. But better late than never. We’ve discussed this case before on Dynamic Ecology (see this post and the comments on this one), so I wanted to note that it’s now concluded.
Terry McGlynn on living a purposeful life, in and outside science. Notes that purpose starts from asking “What do I want to do?”, not “What am I trained to do?” A lovely and very personal post, one of Terry’s best. Which is saying something.
I know I sound like a broken record on this, and I bet at least some of you will dismiss this as irrelevant because it’s about biomedical research rather than ecology and evolution. But you really should read this news piece from Science on why studies of mice so often produce unreliable results. Presents hard evidence that bad practices in study design and statistical analysis (no blinding, no random assignment, throwing out data post hoc, etc.) are a widespread and serious problem. And hits on the right explanation for why such practices persist: people are doing things the way their mentors did, who are doing things the way their mentors did. Which I emphasize is a totally natural and reasonable thing to do. But the fact that it’s totally natural and reasonable does not make it optimal, or even close to optimal, or even adequate. So maybe instead of just attacking zombie ideas, we should also be attacking zombie statistics: study designs and analytical practices that should be dead, but aren’t. And while
I do think ecologists and evolutionary biologists as a group have better analytical practices than biomedical researchers(UPDATE: on reflection, that’s a generalization I’m in no position to make, not knowing nearly enough about the massive range of biomedical research), I’m not so confident in that as to be completely blasé about the issue. I highly doubt that zombie statistics is something that only happens to people in other fields. I worry about this in part because I worry about the interaction with the trend towards statistical machismo. I freely admit this is a vague impression, but it seems like a small but increasing number of people feel that they can just specify and estimate a really detailed hierarchical model of the processes that generated the data (including the sampling process). Thereby relaxing the need to design the study in such a way that the process that generate the data are forced to be ones that we know how to model using classical statistical tools. The trend towards using fitting complex hierarchical models may not solve the problem of zombie statistics because it’s treating the symptoms while simultaneously encouraging (or at least not discouraging) the causes. If that makes any sense. Anyway, I’m doing my bit to fight the zombies: I’m using this Science piece, and many others I’ve linked to in the past, as teaching tools in my introductory biostats class.
Following on from the previous link, Andrew Gelman (who like me has apparently decided that he’s ok with sounding like a broken record on this issue) has some comments on the news piece as well. His thoughts are always worth reading, even if I don’t agree with them entirely. In particular, I don’t agree with him on the bit at the end where he suggests that the root problem is viewing statistics as a way to identify real effects and remove uncertainty, as opposed to viewing statistics as a way describe variation. I’d say statistics is, and should be, both. There are a lot of contexts–much of biomedical research is one, the search for the Higgs boson is another–in which our scientific goal quite rightly is to separate signal from noise, identify real effects, and make yes-or-no decisions (should this drug be used in humans? does the Higgs boson exist?). Again, this is something I talk to my introductory biostats students about. The whole apparatus of frequentist hypothesis testing is designed for situations in which it is both reasonable and desirable to assume that there is some fixed, true-but-unknowable state of the world, about which we want to make inferences based on incomplete information (sample data). There really are many such situations in science that work approximately that way, though their frequency probably varies among fields. But if that’s not the situation you’re faced with (at least approximately), well, then you should be using different statistical tools, designed for a different purpose.
Speaking of philosophy of statistics, this paper is from 2011 and I’ve known about it for a while but can’t recall ever linking to it. So better late than never. It’s from statistics professor Robert Kass. It’s a thoughtful, non-technical attempt to articulate a principled yet pragmatic philosophy of statistics. As opposed to ad hoc pragmatism, which I don’t like. It’s fine to do “whatever works”–so long as you understand why it works. Anyway, Kass’ philosophy addresses the limitations of both textbook frequentism and subjective Bayesianism while retaining their strengths, and resonates with and justifies how many practicing, pragmatic statisticians go about their work. I found it especially interesting because Kass suggests that his philosophy is something that could easily be taught to beginning undergraduates. I’m currently teaching intro biostats, and I believe even beginners ought to understand why they’re doing what they’re doing. I’m planning to mull over whether and how to modify my own teaching in light of Kass’ suggestions.
+1 to Carl Zimmer’s compilation of silly science acronyms. Includes some from ecology and evolution. Can you guess what RAP-MUSIC stands for? 🙂 I’ve never been involved in anything with an acronym as silly as that. Best I can offer from my own experience is InterACT (Interactions Advancing Community Theory), a working group I was involved in as a postdoc. But some of the folks from InterACT went on to start SIZEMIC, which is pretty good (I’m told that European Science Foundation projects pretty much have to have an acronym). And of course, there’s NutNet, which despite the name is not a network about nuts, or involving people who are nuts. 🙂 And ecological stoichiometry types have meetings known as Woodstoich, but I don’t think that’s actually an acronym.