Also this week: Olympian statistical machismo, successful scientist vs. failure, grant success rate vs. length, Zen Faulkes vs. Reddit, and more
From Jeremy:
Deborah Mayo shreds Burnham and Anderson. They’re seriously confused about “falsification” as a scientific method and how frequentist statistical tests and model selection approaches might contribute to it. See the comments there for some blunt remarks from statistical ecologist Brian Cade. A sample:
So, yes, be scandalized. There is an entire generation of ecologists and wildlife biologists that have been taught the Burnham and Anderson approach and assume that they could not have erred.
Deborah Mayo writes for a pretty advanced audience, though this particular post has some nice background material. If you’re new to her work, you might want to first consult this little primer I did on her ideas. Related: our own Brian McGill on why AIC appeals to ecologists’ lowest instincts.
Dan Bolnick’s Game of Thrones-based taxonomy of scientific failure. Includes a long list of Dan’s own failures, in case you were under the mistaken impression that you’re the only one who ever has experiments eat themselves. Related: Meg on system envy and failed experiments and on how sticking to a system you know helps you avoid failed experiments.
Why success rates only seem like they‘ve dropped at Canada’s Social Science and Humanities Research Council. Or rather, they have dropped, but as a mathematically-inevitable consequence of changing the length of funded grants while holding all else constant. I’m now wondering: how much would NSF DEB success rates rise if all grants were limited to 3 years and nothing else changed? Ok, that change would have other consequences, some of them undesirable. But if you want success rates to go up, your choice is which eggs to break in order to make the higher success rate omelette. And yes, I recognize that “and nothing else changed” is a very strong assumption given that limiting the grant length to 3 years and changing the success rate might feed back and affect the number and quality of applications.
This is old but I missed it at the time: Zen Faulkes on his reluctance to take his science outreach to where the people are–specifically, to a Reddit “Ask Me Anything”. Your mileage may vary, of course–Margaret Kosmala did an AMA and had a very positive experience.
As a prospective grad student, should you worry about the possibility that your adviser might move to another university or be denied tenure? If so, is there anything you can do to mitigate or hedge the risk to your own studies? And what are your options if your adviser does move? Useful discussion.
And finally (ht @dandrezner):
Dynamic Ecology 
Thanks for the links, Jeremy. Yes, Mayo’s audience is pretty advanced, but I have no mathematical statistics or philosophy training and I still get a lot out of Mayo’s blog and other writing. It took me a while before I could get my mind around some of the complicated language. For me, my main takeaway is that evidence doesn’t come merely from data fitting a hypothesis but from how good a test that data is for that hypothesis. This seems like a really fundamental idea to me but it has made me look askance at papers that only present model comparisons a la Brian’s AIC post. It’s personally a little ironic to see Mayo get to B&A; I became aware of Mayo’s work through your old blog post (why and how to do statistics), but it was the heavy rhetoric in B&A’s “Model Based Inference in the Life Sciences” (I don’t have it here but I remember a crack about null hypothesis testing being of historical interest only, and so many references to Hard Thinking (implying anyone who didn’t agree with wasn’t thinking hard)) that made me want to consider a different take. Thanks again, I always enjoy it when DE covers statistics and philosophy!