Citation concentration, filtering, incentives, and green beards

Scientific citations are highly concentrated (Ioannidis 2006). In any field, a small fraction of papers garner a large fraction of the citations, and a small fraction of journals publish a large fraction of those highly-cited papers.

Why is that? It could be because the distribution of paper quality is highly skewed. Only a small fraction of papers are really good and really important, and those papers garner many citations. But although I do think the distribution of paper quality is skewed, I doubt that’s mainly what drives citation concentration. I think we’d see high citation concentration even if there weren’t much variation in paper quality. It’s all to do with filters and incentives.

There are far more papers published than anyone can read, even if you’re just reading the abstracts (heck, even if you’re just reading the titles!) So everyone needs “filters”–some way of deciding what small fraction of the literature to pay attention to. In practice, one very common filter is “read what’s in the leading journals”* Now, you can argue about whether or not that’s a good filter, in the sense of a good way to identify high-quality papers. But frankly, I don’t know that it matters if that filter is good or not. Because here’s the thing: the fact that it’s a common filter means that everybody has a strong incentive to use it. Even if you think Nature and Science (and in ecology, Ecology Letters) are just “tabloids” and that the ecology papers in those journals aren’t any better than those in any decent ecology journal, you have to pay attention to Nature or Science, because everybody else does. And you also have strong incentives to try to publish in those journals, because they’re leading journals. And what’s a “leading journal” but a journal that everyone reads and tries to publish in? And papers in those journals get cited more, because everyone reads them.

I often hear people bemoan this state of affairs. They complain that it makes too much of our science a crapshoot. Too much about what research topics become “hot”, who gets hired for faculty positions, who gets grants, who becomes famous, etc., depends on who gets lucky enough to publish in a very short list of journals. And while I completely agree that a lot does depend on publishing in a very short list of journals, and even agree (to an extent) that what gets published in those journals is a crapshoot, I don’t complain about it (well, at least not much). Because I don’t see how it could be any different. So long as much more stuff is being published than any one person can read, everyone is going to need to use filters. And further, everyone will always have a strong incentive to use the same filters as everyone else. You say you only want to read the “best” stuff? Well, in science, the “best” stuff is in part defined as “the stuff everybody else is reading”. As a scientist, you can’t just ignore what all your colleagues are reading about, thinking about, talking about, working on, and citing, not unless you’re ok with everyone ignoring what you read about, think about, talk about, and work on. And insofar as there are other attributes that define what’s “best” besides “what everybody else is reading”, how do you plan on picking out those other attributes except by using “what everybody else is reading” as a surrogate?

Even if we did away with journals, and just published all our work in PLoS ONE or on ArXiv or on our own blogs, we’d still have high citation concentration (or its equivalent–high incoming link concentration or high article-level metric concentration or whatever). Because we’d all still have filters of some sort, we’d all still have strong incentive to use the same filters, and we still wouldn’t know any other way to filter out the “best” stuff except to use “what everybody else is paying attention to” as a surrogate.

Duncan Watts’ Everything is Obvious (Once You Know the Answer) is a fine book which includes lengthy discussions of the same phenomenon in all kinds of contexts. This isn’t just a scientific publishing thing. So you think getting rid of “citation concentration” or its equivalent could be accomplished just changing the way in which we all publish, think again.

My original draft of this post stopped there. But then last night I thought of what might be a half-baked evolutionary analogy for this. I haven’t thought it through, so I could be way off. But it’s too fun not to share:

I think citation concentration is a green beard effect.

If you don’t know, the “green beard effect” was first proposed by Bill Hamilton, but Richard Dawkins gave it its name. A green beard gene, or cluster of linked genes, has three effects:

1. produces a perceptible trait, such as a green beard

2. causes the bearer to recognize that trait in others

3. causes the bearer to direct preferential treatment towards others exhibiting the perceptible trait

The neat thing about a green beard is that it’s not a signal of intrinsic “quality” or “fitness”. A green beard doesn’t make you more fecund or long-lived or etc., nor is it a signal that you have other traits making you more fecund or long-lived or etc. A green beard is an arbitrary signal, and is only favored because everybody else with green beards favors it.

Filtering the literature by only reading and seeking to publish in the same few places as everyone else is a sort of green beard, I think. It produces a perceptible trait, namely papers in Nature and Science. It also causes recognition of, and preferential treatment of, those bearing that perceptible trait. I admit I’m still fuzzy on whether the “traits” here are attributes of individuals or journals or both, so maybe the analogy can’t actually be made all that precise. But don’t let that stop you from showing up to the Ecology Letters reception at the ESA meeting dressed like this. 😉

*Another fairly common filter is “read stuff written by famous people”. Much the same argument applies to this filter. Everyone has a strong incentive to use this filter, because it’s a common filter. You have an incentive to read what everyone else is reading–and everyone else is probably reading Dr. Famous’ latest paper.

p.s. Before anyone points it out, yes, I am aware of Vince Jansen’s paper on beard chromodynamics (coexistence of beards of various colors, with bearers directing preferential treatment towards others with the same color beard). That paper could be interpreted to mean either that my analogy to green beards isn’t a good one, or else that the analogy is good but that the green beard effect need not lead inevitably to high citation concentration because different filtering rules could coexist.