Also this week: are universities a partisan political issue in Canada, advice on giving talks, “turn that shit into a blog post”, and more.
From Jeremy:
Poll data (unreviewed as yet, but it looks solid to me) on how often ecologists engage in various questionable research practices. I’m most alarmed that half the respondents have at some point reported an unexpected finding as if they had predicted it beforehand. Ok, it’s possible for that not to compromise the severity of your test–but in most cases it squashes the severity of your test like a grape at a winemaking festival. Had me thinking back to this study from a few years ago of the “chrysalis effect”–how ugly results get turned into beautiful hypothesis tests. I’m also thinking back to Brian’s post on how ecologists should stop treating exploratory statistics like “the crazy uncle nobody wants to talk about”.
Sociologist Kieran Healy’s advice on giving talks and making slides. Some of it is specific to sociology, but most of it generalizes, such as this:
[N]o one wants to watch someone bomb. This means that presenters start out with the audience on their side to a much greater degree than people often realize.
A bit (and I do mean a bit) of data on how Canadian attitudes about the purpose of higher education vary with political affiliation. Some similarities to the US, but also some differences.
Erving Goffman’s 1982 Presidential address to the American Sociology Association is funny and scathing about his own field–and pretty much every other field. 🙂
And finally, I endorse this. Obviously. 🙂
Dynamic Ecology 
I don’t think “reporting an unexpected finding as if they had predicted it beforehand” is malicious but simply the naive but natural human way of doing things — our brains are built for instantaneously creating post-hoc explanatory stories for patterns (the split-brain confabulation literature makes fun reading). I think what has gone wrong is that we don’t start training against this intuition in intro bio lab and continue this throughout undergraduate sequencing. Then really hammer it in a first year “methods” graduate course. Don’t rely on PIs to train this.
Oh, I agree. I don’t worry about it because I think anyone’s malicious!
No doubt! The title of the manuscript “Questionable research practices” could imply ‘unethical’. I think a better way to describe reporting an unexpected finding as if they had predicted it beforehand is ‘naive’. I scanned the manuscript last week and need to read it in more detail.
I think the intent in referring to “questionable” research practices is that it sounds neutral. Imagine calling them, say, “p-hacking”, which to many people has a connotation of nefarious intent. I’ve used the term “p-hacking” myself in the past but I’ve decided that was wrong because I didn’t intend any negative connotation about investigator intent. So I’m trying to remember to stick to more neutral terms like Gelman’s “garden of forking paths”.