Been keeping an eye on the molecular ecology live online chat. Got to the office just as my contrarian question about the futility/pointlessness of genotype-phenotype mapping was posed to the group. Not that I actually believe it’s futile/pointless–I don’t know nearly enough to judge one way or the other–but it’s an issue that some famous people recently raised in a top journal, so I was curious what the panelists would say. It’s a nice change to be able to watch a serious debate as a curious bystander, rather than as a participant! Basically, none of the panelists really agreed with the critique. More importantly, they had a variety of cogent-sounding reasons why not, touching on both things we already know, and things we could learn in future (e.g., by combining studies of selection in natural populations with whole-genome sequencing).
But I was struck by a passing remark by one of the panelists (Loren Rieseberg) that I wanted to comment on, because I think it articulates a widely-held view with which I completely disagree. Loren said in passing that he always has a problem with people criticizing the approaches of others without proposing an alternative. (His remark wasn’t directed at me, by the way, but at the paper in Evolution that inspired my contrarian question) As I said, I think Loren’s articulating a widely-held view here. So don’t think I’m picking on him specifically when I say that I totally disagree.
If a way of addressing some question is ineffective for whatever reason, then it’s ineffective, full stop. Not only is there nothing wrong with pointing this out, it’s absolutely essential to point this out! I say this for the following reasons:
- Whether or not approach X is flawed is logically independent of whether or not there are any alternative approaches, and of any flaws in whatever alternative approaches might exist. If I point out flaws in approach X, I’m no more obliged to propose or discuss alternative approaches than I am to discuss other logically-unrelated matters. And if approach X is flawed, you can’t defend it by saying “But no alternative exists!” That does not make the flaws in approach X go away, any more than if you’d said “But chocolate is delicious!”
- Surely it behooves us to have an honest understanding of the flaws of any approach, whether or not any alternative exists! I mean, would you argue that the flaws in some approach should be hushed up and swept under the rug just because no alternative approach is available? Even if for some reason we have no choice but to study some question, and to do so using some flawed approach (and such cases are rare, I think), surely it’s best if we’re aware of the flaws!
- Alternative approaches may also be seriously flawed. In such situations, the best idea often is not to settle on the best of a bad bunch, but to stop pursuing the question entirely. It’s very rarely the case in science that we have no choice but to pursue a particular question (see also the following point)
- Opportunity costs are ever present. The time, effort, and money being spent pursuing an ineffective approach to addressing some question could be spent addressing some other question effectively. In other words, there’s always an “alternative approach” available in science–ask some other question!
- Ineffective approaches often are worse than nothing. What’s worse than not knowing anything about the answer to a question? Having a wrong, misleading, or biased answer that is not recognized as wrong, misleading, or biased. See also: zombie ideas.
- One very strong motivation for developing new approaches is recognition of problems with existing approaches.
I suspect there’s an implicit assumption behind Loren’s point of view–that nobody’s approach is ever totally ineffective. So if you criticize someone’s approach without proposing an alternative, you’re basically arguing that we stop doing imperfect-but-still-valuable science. To which I’d respond in three ways. First, my point above about the need for an honest understanding of the effectiveness of any approach still applies. Second, my point above about opportunity costs still applies. Time, money, and effort spent on a somewhat-effective approach could be spent addressing some other question more effectively. I don’t claim that such judgments are easy to make–they’re not–but they’re the sorts of judgments that scientists (and our funding agencies) make all the time. They’re unavoidable. Third, the implicit assumption that every approach is at least somewhat effective needs justifying on a case-by-case basis. Scientists, even very good ones, and even lots of them at a time, sometimes do make really serious mistakes and pursue completely ineffective approaches. That’s simply an empirical fact. And when that happens, I don’t think it’s polite or respectful or professional to remain silent. Science works best when we all push each other, not when we all pat each other on the back.
p.s. The panelists have now gone into an interesting discussion of how metagenomic studies (e.g., “Here are the whole-genome sequences of all the microbes at site X and time Y”) need to move beyond description into experiments and hypothesis testing. Woohoo! That’d be expensive, but still…woohoo!