I have been thinking a lot about crises in fields of science. I don’t mean “grants are shrinking” crises or “we continue to treat subgroups abominably” crises. Nor am I talking about the fact we are documenting an ecological crisis on our planet. Those are real and important. But I mean here “the science we are producing and communicating is wrong” kinds of crises. I think these crises probably say a lot about science. Both in how they managed to go wrong. And in how the crises got recognized and fixed.
I am going to list five different crises in five different fields of science (four are recent, one is old), and then I am going to ask what kind of crisis ecology is most likely to have (or is having?).
- We got one big thing wrong (dietetics) – In the 1950s there was an alarming increase in the rates of coronary heart disease in America. A prominent scientist argued that this was due to the increase of fats in our diet. This view won out. By the 1980s nobody would have come up with any other explanation. The US government, in their dietary guidelines painted fat as unhealthy and produced guidelines that would limit its intake. The American Heart Association agreed. Thus probably billions of research dollars were channeled into researching this. In the 1960s a number of researchers also suggested that carbohydrates/sugars were the cause, but they lost. And only in the last decade has this become an acceptable hypothesis again and it seems increasingly clear that the increase in carbohydrates in our diet (a trend present in the 1950s but actually accelerated by the avoidance of fats) is also highly problematic. At least as problematic as fat. Evidence on the relative balance is not yet clear, but it is at least credible that sugar is much worse for us than fat. The definition of an acceptable research agenda, research dollars, and public policy were all off target for half a century. Oops.
- We got a lot of little things wrong (psychology, medicine) – Insiders know this as the reproducibility crisis. In several fields researchers decided to devote some effort to actually reproducing previous findings rather than always trying to do something novel. And the end result has been that very many results, even results that ended up in textbooks were not reproducible (rates of over 50% not being reproducible). Fivethirtyeight has a nice discussion of this crisis. But unlike the dietetics crisis, these results were largely one-off types of findings and not necessarily particularly interconnected or driven by a an across-the-board agenda. And they haven’t particularly driven policy. It might be that the word “little” in my subheadings an exaggeration (some of these results got a lot of press coverage like the idea of power poses), but the things that failed to repeat are certainly not “big” either in the same sense of did fat or sugar cause the surge in heart disease deaths or of driving an entire research agenda for decades. I think what disconcerts most people following this is not the size of the findings being overturned but just the sheer volume of them. We rationally expect some reversals in science. But we don’t expect more than 50% reversals. This has led to a great deal of focus on poor research practices such as post hoc hypotheses, p-value hacking, etc.
- We paid too much attention to the small stuff (medicine) – Medicine like every field focused on p<0.05 as a criterion for publication. But eventually people noticed that this was leading to changes in medical treatment that, yes p<0.05, but mortality or morbidity was reduced by only a few percent. This was basically an effect size problem. This is also why you would see coffee is good for you news one week and coffee causes cancer the next week – anything with a big sample size (hence p<0.05) could get reported no matter how small the effect size. Medicine reformed itself (primarily by changes to the requirements to publish in journals) and now instead of focusing on p-values focuses on effect sizes, usually expressed as odds-ratios, and confidence intervals around those odds-ratios. This of course still ends up letting one assess statistical significance (is 1.0 in the 95% confidence interval of the odds ratio?) but it places the focus where it should be on the effect size. Are 50% of patients better off or 2%?
- We totally missed the big one (economics) – Jeremy has mentioned several time that the failure to foresee the 2008-2009 Great Recession threw the field of economics into turmoil. As Jeremy has pointed out, the creation of a blog culture in economics was one outcome – paper publication was too slow for the real-time processing needed to track this. The use of preprint servers in economics also upticked for reasons of speed. There has also been an increasing focus on empirical reality (via econometrics – i.e. analysis of data) and away from neoclassical equilibrial theories (the Great Recession was anything but a stable equilibrium). Its too early to tell if economics will be more nimble (and more able to predict black swans), but at least they’re trying.
- It took us deaths and a genius to move past a silly idea (physics) – throughout the late 1800s physicists watched a beautiful theory, the wave theory of light occuring in vibration of an undetectable aether that was the fixed firmanent of the universe become increasingly untenable. The Michelson-Morley experiments didn’t work. The response was to post hoc hypothesize the earth having a boundary layer of aether, which kind of defeated the whole point of aether. Or to invent Lorentz contractions (which turned out to be right but nobody had any idea why they worked). Scientists also found evidence that light behaved like a particle (e.g. the photoelectric effect) rather than a wave. Despite this growing body of evidence that the aether theory of light was wrong, it remained firmly believed and taught. It took 50 years and the retirement of many senior scientists, plus a genius named Einstein to offer an alternative explanation to get back on track. This example of clinging to a wrong idea was so egregious (in hindsight) that several historians of science have held it up as an important case study. But it is hardly alone. It took 70 years for geologists to get on board with Wegener’s continental drift hypothesis.
So there you have it. Five different types of crisis for a scientific field to have. All involve sloppy thinking. Several involve misuse of statistics and p-values. But all are real. All have caused entire fields of science to get off track for years or decades. But they differ enormously in the consequences of the crises.
But let me add one more flavor of crisis that could apply to any of the five crisis types:
- We were tools of a special interest (dietetics) – there is a special twist to the dietetics crisis (focusing exclusively on fat as an explanation for heart disease for 50 years). It happened because one big name pushed it really hard, and we are now increasingly finding out that the one big name was bought and paid for by the sugar industry (always a strong lobbying group in the US).The Sugar Research Foundation not only bought a leading name, but they continued for decades to tilt the scientific consensus in a variety of more subtle ways through their creative use of research funding. This is shades of the tobacco industry.
So. It is easy (and perhaps fun) to look at other fields and feel a sense of amusement at their failures. As that delicious German word, schadenfreude, suggests, it is human to enjoy others being discomfited. But that is not the point of this post. The point of this post is to force ecologists to look at ourselves. Is ecology in the midst of a scientific crisis that we’re just not talking about (but will be in 5 years)? If so which kind of crisis? Do we have that “tools of special interests” flavor as well? If so, which special interest are we tools of? How would you fix ecology to fix/preempt a crisis? I’ve developed a quick poll on this to solicit your opinions, but of course I hope you’ll share your comments below as well. And I will share my own opinions on this topic either in a later post or in the comments once there is a robust discussion going on.