Also this week: sociology vs. good news, how analyzing data is like composing music, perverse effects of (optional) double-blind review, best references section ever, and more.
What would a theory of practical data analysis look like? Very interesting question, with some tentative stabs at an answer based on an analogy to musical theory:
Similarly in music, theory provides an efficient way to summarize the useful aspects of what has come before in a manner that can be fit into something like a semester-long course. It also provides a language for describing characteristics of music and for communicating similarities and differences across musical pieces. But when it comes to the creation of new music, theory can only provide the foundation; the composer must ultimately build the house.
I really like that way of putting things. It’s what I’m trying to do in the book I’m supposed to be writing.
Am Nat EiC Dan Bolnick examines whether Am Nat’s switch to double-blind review (with an option for authors to opt out) is having any effect. Unfortunately, Dan later discovered there was a mistake in his data–they’re data on the gender of corresponding authors of Am Nat submissions, not first authors as mistakenly stated in Dan’s post. That affects the interpretation of some of the results in the post, because women are less likely to be corresponding authors than they are to be first authors, and women first authors are less likely to be corresponding authors than men first authors are. Still, Am Nat’s data to seem to show a small but unfortunately perverse effect of their switch to optional double-blind review. The good news is that Am Nat publication decisions are gender-blind (with respect to corresponding author gender) for papers reviewed double-blind, and for papers reviewed single blind. But male corresponding authors are slightly more likely than women to opt out of double-blind review, and it looks like papers subject to double blind review get reviewed a bit more negatively on average than papers subject to single blind review (aside: that last result doesn’t surprise me; as Dan notes, it has been shown before in studies of peer review). The obvious solution here is to make double-blind review mandatory. As an Am Nat editor, I’d support a switch to mandatory double-blind review. We’d just have to hope (or ideally, check) that reviewers don’t see through the blinding, or think they’ve seen through it, non-randomly with respect to author gender or other attributes, and that reviewers who think they’ve seen through the blinding aren’t any less critical on average than those who don’t think they have. I don’t think that would happen, but it’s not far-fetched, and so I think it would be best to check.
“If something, like, totally great happened, would sociology cover it?” (ht Andrew Gelman) One could ask the same question about ecology: if some aspect of conservation, or human impacts on the environment, got better (as some aspects have!), would ecologists cover it? Maybe the better (and trickier) question is, would they cover it the right amount? That’s a trickier question because what is the “right amount”, exactly? Related old posts from Brian and me, and me again. One could also ask the same question about the practice of ecology. Is the practice of ecology getting better, and if so, do ecologists talk about that improvement the “right amount”? Brian’s post earlier this week addressed some of this, and here’s an old post from me approaching the topic from a different angle.
Economists these days increasingly use rigorous experimental and statistical methods to address narrow, tractable questions about causality in “local” contexts. Rather than bigger, more important questions that aren’t directly addressable with those rigorous methods. So, with that background, discuss: could the same be said of ecology today? Or maybe the same could be said of ecology back in, say, the 1980s-90s when field experiments in 1 m² quadrats were all the rage, but not any more thanks to remotely-sensed data, globally distributed experiments, our increasingly macroecological focus, etc.? More broadly, when it is a waste of everyone’s collective time to argue about big questions on which the evidence is inconclusive? Does rigorously answering narrow, tractable “small scale” questions contribute to answering big, broad questions? If so, how exactly? Looking forward to your comments.
“The fact that I cannot remember the last time the internet made me feel, on balance, less anxious and better about other people tells you something about how much has changed online since 1999, 2001, and even 2007.” I’m curious if this resonates with anyone much younger than me. I’m in my mid-40s, only a few years older than the author. And I’m not even on Facebook.
A few simple tweaks to make your Word document look just as sharp as a LaTeX document. Angry replies from LaTeX users in 3…2… 🙂 (ht @dendrezner)
They don’t make lit cited sections like they used to. 🙂 Although my favorite is still Price (1970 Nature).