In finance, you often buy a stock or bond if you think it’s going to go up in value in future, and sell if you think it’s going to go down in value. There’s an obvious (loose) analogy here to bandwagon-jumping in science. Someone publishes some new method or approach that promises a shortcut to insight, and others “buy”: they jump on the bandwagon by rushing out to apply the approach. And then once it starts becoming apparent that the method doesn’t live up to its initial promise, or has been pushed as far as it can go, people “sell”: they stop using the approach and move on to something else.
You can view this behavior, in both finance and science, as a form of betting. Much as with buying a stock, applying some new scientific approach amounts to a bet that the approach will work, and that others will “buy” the results (believe the results and view them as interesting and important). And ceasing to use some approach amounts to a bet that others will stop “buying” the results.
But how do you bet against a scientific idea you never bought into in the first place? How do you jump off a bandwagon that you never got on?
In finance, you can bet against a stock or bond that you don’t own by selling it short. To do this, you sell stocks or bonds that you don’t currently own, and buy them back later. If the price has fallen in the interim, you make money because you spend less on the buyback than you made on the initial sale. Mathematically, it’s equivalent to buying a negative amount of the stock or bond. In practice, this is done by borrowing stocks or bonds from someone who owns them, selling them immediately, buying them back later, and then returning the repurchased stocks or bonds to the lender while pocketing the profit (if there is a profit; the short seller loses money if the stock or bond rises in price).
But doing the same thing in science is more difficult. Thomas Basbøll notes that, in science, there isn’t a big “market” for criticism of ideas, especially popular ideas. I think he’s right about this (and his remarks inspired this post). Purely critical papers, or even critical technical comments, often are difficult to publish, and when they are published they’re often ignored. A lot of scientists take a dim view of criticism of the work of others, especially if it’s not accompanied by proposal of an alternative approach. On this view, if you don’t want to “buy” stock A, you can go buy stock B instead (i.e. use some other approach, or work on some other topic entirely)–but shorting stock A is illegitimate.
Similarly, short selling in finance is controversial. It’s been accused of increasing market volatility and giving rise to self-perpetuating market crashes, though the evidence for this is not clear-cut (according to Wikipedia, anyway).* Conversely, short selling is defended as an important price discovery mechanism that improves market efficiency. Short sellers provide a counterweight against irrationally bullish investors, may prevent financial bubbles from developing, and have played a role in uncovering fraudulent business practices (the financial equivalent of uncovering serious technical flaws in scientific papers).
As I’ve said before (but with different phrasing), I think we need more short selling in science. Short selling in science has the same benefits as short selling in finance. Even better, some of the purported downsides of short selling in finance don’t exist in science. For instance, there’s no way criticism of existing scientific ideas is ever going to lead to a self-fulfilling panic in which everyone tries to abandon all scientific ideas, analogous to a financial market crash in which everyone simultaneously tries to sell all their assets. And while in finance an individual short seller can lose massive amounts of money if the borrowed asset rises in price, in science if you criticize an idea and your criticism turns out to be wrong or gets refuted, you don’t suffer any major loss. You don’t get fired, or lose all credibility so that you can never publish again, or anything like that. For instance, back when the biodiversity-ecosystem function bandwagon was first getting rolling in 1997, Huston and Aarssen pushed back by arguing that “sampling effects” were driving the then-new experimental results. Their criticisms were refuted by additional data and new analytical techniques, and biodiversity-ecosystem function research rolled on. Had they been short sellers in finance, they’d have lost their shirts. But because they’re scientists, they didn’t suffer any “losses” at all as far as I’m aware (nor should they have).
One way in which blogs have improved scientific communication is by providing a mechanism for short selling. If journals won’t publish your criticisms of some scientific idea, you can still make the case on your blog, and you’ll still have an impact insofar as people read your criticisms and alter their scientific behavior accordingly. Rosie Redfield’s criticism of the arsenic life paper is the most prominent example, but here at Dynamic Ecology we’re no slouches at short selling either (e.g., our most-commented post is Brian’s “short sell” of the use of detection probabilities in wildlife research). This is really important, I think. Some financial bubbles (for instance, in subprime mortgages) are thought to have developed in part because, for technical reasons, there was no way to short sell them.
Scientific “bubbles” distort the allocation of scientific effort. They should be popped before they get that far. And the way to pop them is to short them.
UPDATE: On Twitter, David Wescott takes my financial analogy even further:
@boraz fascinating read.I wonder if we could securitize scientific ideas and market them a la Fannie Mae. would you buy discovery futures?
— David Wescott (@dwescott1) May 8, 2013
That snapping sound you just heard was the sound of my analogy being stretched beyond the breaking point. 😉 Discovery futures?! The thought is somehow hilarious and vaguely terrifying at the same time…
*You get the background research you pay for on this blog.
I play a game called “guess who wrote this post” based on the title alone. Somehow I guessed right on this one…
“there’s no way criticism of existing scientific ideas is ever going to lead to a self-fulfilling panic in which everyone tries to abandon all scientific ideas”
Dark ages? How about the general public loses all faith in any scientific credibility, because the criticisms become much louder? I’m exaggerating a bit, but there may well be unintended consequences based on interpretation of ‘science’ by the general public.
Re: a general loss of faith in scientific credibility, yes, it’s possible in principle. But in practice, if the critics are honest scientists, I don’t think it’s likely. Organized, politically-motivated efforts to undermine scientific credibility are a whole ‘nother ballgame (a hugely important ballgame, but not the one I was thinking of when I wrote the post).
“I play a game called “guess who wrote this post” based on the title alone. Somehow I guessed right on this one…”
Now I’m tempted to mess with you by giving my posts wonky titles. But that probably wouldn’t work, as something like “The title of this post is meant to mess with Margaret Kosmala” would be a dead giveaway. And I doubt I could give my posts informative titles while nevertheless making them sound like posts Brian or Meg might’ve written. Unless I actually went ahead and tried to *write* the sort of post Brian or Meg might write, so that the subject matter of the post (as reflected in the title) wouldn’t be a giveaway. Hmm, that might actually be a fun exercise if Meg, Brian, and I all had a few hours to spare sometime. We each write a post pretending to be one of the others… 😉
Organized political efforts should be a whole other ball game. But they absolutely skim the true scientific literature and pull out and amplify all the doubt and critique that is in there. Check out the critiques of climate change – it often pulls heavily from scientific literature. It just pulls things totally out of context.
Not saying we shouldn’t criticize each other. And I generally agree with your post here. But we shouldn’t kid ourselves that fields with lots of internal criticism don’t lead to public perception of clueless.
I’m not actually sure if the amount of internal debate in a field correlates with public perception of that field, independent of organized political efforts to manipulate public perception. Precisely because organized political efforts pull things out of context, as you rightly note–and anything can be pulled out of context. Including lack of debate within the field, because that can be spun as indicating closemindedness and bias (e.g., that’s how creationists explain the universal refusal of evolution journals to publish creation “science”).
I do agree with you and Margaret that, if a field is entirely occupied with vociferous, never-ending debates over fundamentals, that’s probably not a productive field and probably not even a scientific field. Such a field might well be perceived differently by the educated public than a more productive scientific field, and rightly so. In the post, I was implicitly assuming (and probably should’ve made explicit) that ecology will never reach that point.
Hi Jeremy, one of the things I love about Dynamic Ecology is that it’s often prescriptive – it seems that many scientists want to avoid the conflict that comes with being prescriptive,,,but that’s not a problem here. And this advice seems to fit with what I sense is an underlying philosophy… that science can be a little too genteel, a little too congenial and that a little sand and grit might not hurt. Hey, it’s only science. And I can’t decide whether the shortage of papers that take the form of “XXXX et al. have it completely wrong” is because we’re unwilling to have these kind of pointed discussions (but which needn’t be painful if we don’t take them too seriously) or just too busy on our own things to pay much attention to what others are doing. Neither explanation is very satisfactory. When was the last time we had an in print debate that matched the heat and light of Diamond versus Connor & Simberloff. or Andrewartha and Birch versus Nicholson? I’m sure there are more recent examples but they don’t come to my mind. Best, Jeff Houlahan
Thanks Jeff. You’ve given me a good post idea: ask people to list examples of vigorous-but-productive debates in ecology (“heat and light”), and vigorous-but-unproductive ones (“heat but not light”). Your list is a good opening bid.
Good point that lack of debate in part stems from people understandably focusing on their own stuff, rather than lack of disagreement per se. I think I mentioned a while back that there’s a paper in a leading journal which seriously misuses an approach I helped develop. Had I been asked to review it, I’d have strongly recommended rejection. But I decided I couldn’t be arsed to write a comment on it (though I did correspond a bit with the authors), as I didn’t think it likely that anyone else would follow the paper’s lead. And so far, no one has as far as I know. Conversely, I eventually decided that it was worth my time to write an opinion piece for TREE based on my blog posts on the zombie IDH. And in between are ideas that I’ve been willing to spend time critiquing on the blog, but no more time than that.
As for more recent examples, the debate over sampling effects in biodiversity-ecosystem function was quite narrowly focused, but was productive in that the issue was quickly clarified and resolved (well, resolved to the satisfaction of everyone but a few odd holdouts).
“it seems that many scientists want to avoid the conflict that comes with being prescriptive…but that’s not a problem here.”
Understatement of the year. 😉
“…what I sense is an underlying philosophy… that science can be a little too genteel, a little too congenial and that a little sand and grit might not hurt.”
I should probably hire you to take my posts and compress them into single sentences with no loss of information. 😉
And that post is now up:
p.s. Thanks for commenting Jeff–until you showed up all I had was a comment from Margaret rightly making fun of my predictability, and a pity comment from Brian. 😉
I think you could have this if reviews (or excerpts from them) could be published along with the paper, or as “comments” on the paper. Reviews often short methods, but are never made available to the public :(.
You could always publish the reviews your own papers receive. Terry McGlynn of Small Pond Science does.
Pingback: Hoisted from the comments: name the most productive, and unproductive, ecology debates ever | Dynamic Ecology
Pingback: When, if ever, is it ok for a paper to gloss over or ignore criticisms of the authors’ approach? | Dynamic Ecology
Pingback: John Stuart Mill on the value of contrarianism | Dynamic Ecology
Pingback: Why ecologists might want to read more philosophy of science | Dynamic Ecology
Pingback: Ask us anything: how do you critique the published literature without looking like a jerk? | Dynamic Ecology
Pingback: Post-publication review is here to stay–for the scientific 1% | Dynamic Ecology
Pingback: Ask us anything: investing in your scientific beliefs, and applied papers as corporate prospectuses | Dynamic Ecology