Also this week: the pluses and minuses of preregistering your research, does more pressure to publish really make publication bias worse, the first “man on the street”, and more.
Andrew Gelman on what makes for good exploratory research. I think this is really important to think about. On the one hand, there are strong arguments that hypothesis development is at least as important as hypothesis testing, and that both are badly degraded if you try to treat the former as the latter (intentionally or not). On the other hand, exploratory studies can be done badly even when they’re not disguised as confirmatory studies. The fact that a study is exploratory doesn’t get it off the hook for being badly thought out or poorly designed or boring or whatever. And there’s the somewhat orthogonal question of whether we need to publish more exploratory studies, and if so in what form. Brian and I chatted about this a long time ago in the comments here.
Margaret Kosmala preregistered one of her research projects. Her thoughts on the pluses and minuses.
“The senior guy in my field acts like a total jerk sometimes. What should I do?” From economics, and somewhat specific to that field, but much of the advice generalizes.
“While centralized staff often create more work for faculty members, localized staff reduce it.” Yup. (ht Frances Woolley)
A long while ago I linked to a PNAS paper by Fanelli and Ioannidis, finding that publication biases in the “soft” behaviorial sciences are stronger in the US than in other countries. I was far from the only one who paid attention; the finding went viral and has become a key piece of evidence that higher pressure to publish leads to publication of shoddier research. I recently learned that the finding has been questioned: it doesn’t hold up under an alternative, at least equally-reasonable method of analysis. In reply, the authors argue that the original result is robust. I’m not comfortable adjudicating without digging into the original data myself (though I confess I don’t really see the signal I’m supposed to be seeing in the graphs in Fanelli & Ioannidis’ reply…). Just wanted to pass it on for interested readers. (ht Andrew Gelman)
The Dutch Royal Academy is holding two special elections to which only women can be nominated.
The first reference by a scientist or mathematician to the importance of being able to explain one’s work to “the man on the street” has been traced. (UPDATE: link fixed)