Also this week: causal inference vs. Facebook, why read old papers, and more.
Recently in the comments Mark Vellend suggested that we poll readers on controversial ideas in evolution, as a complement to our poll on controversial ideas in ecology. I passed the suggestion on to Andrew Hendry, who’s a better person to do it–and he’s doing it! He’s soliciting suggestions as to what questions to poll on.
Various methods of causal inference from observational data fail to recover the results of randomized controlled trials of online advertising, even when you have as much observational data as Facebook has (note: link goes to an unreviewed preprint). The details are specific to online advertising, but I wonder how much the broader story generalizes. Is there any field in which observational methods like matching, differences-in-differences, regression discontinuity, instrumental variables, structural equation modeling, etc., have been shown in practice (not just in theory) to reliably recover causal effects estimated by randomized controlled experiments? And if you say that that’s the wrong question to ask, on the grounds that experimental estimates of causality are also problematic, what general advice would you have for investigators who are faced with a serious discrepancy between observational and experimental estimates of some causal effect?
Why read old philosophy? (ht Marginal Revolution) The same question could be asked about old ecology–would it have a different answer? I agree with Meghan that there are old ecology papers that you can usefully reread, to get new insights and to remind yourself of old insights you’d forgotten. And I think there are some old papers that it’s useful to read so that you’re better able to recognize when “new” research is truly new vs. just old wine in new bottles (though obviously there are other ways to learn that, such as by reading histories of your field). And I think that there’s an element of different strokes for different folks here. Reading old papers might work for some people but not others. Somewhat like how reading blogs outside my field helps me think about ecology, but wouldn’t necessarily help others. But I also think that old papers that are worth rereading, by anyone, for any purpose besides historical interest, are fairly rare. Possibly, they’re outnumbered by old papers that ecologists continue to read and cite out of habit rather than for some good reason.
How a shoddy, ad hoc statistical method entered the sports science mainstream. I may use this as a statistical vignette in intro biostats next term.
Theory vs. (statistically-sophisticated) empiricism in one meme. 🙂 (ht @noahpinion)