What sort of ecology papers get highly cited these days?

Everybody knows that the most-cited papers these days are mostly review and methods papers (see here and here). But exactly how predominant are such papers among the most cited papers in ecology? And what, if anything, does that tells us about the state of the field?

I searched Web of Science for papers published in the last 10 years in a long list of ecology journals, plus papers on the topic “ecology” published in the last 10 years in leading general science and general biology journals. Then I looked at the 150 most-cited papers from the ecology journals and the 100 most-cited ecology papers from the general science/general biology journals to see what sort of papers they were. Not an exhaustive or infallible approach, obviously, but good enough for government work.

As you’d expect, the bulk of these papers were review papers, methods papers, and syntheses/perspectives/conceptual frameworks. Continental- or global-scale data compilations and many-species data compilations were pretty common too.

In terms of topics, I didn’t do a formal tally. But climate change papers were really common. Particularly papers on range shifts and phenology shifts under climate change. Microbiome papers were very common among recently-published, highly-cited ecology papers from the general science and general biology journals, thus proving Meg right.* And not surprisingly, most of the methods papers were about statistical methods. Brian will be happy to hear that several of the most-cited ones concern statistical methods he likes–boosted regression trees, random forests, exploratory data analysis, etc.

What was rare or absent were (i) proper theory papers (as distinct from verbal “syntheses”,  “perspectives”, and “conceptual frameworks”), and (ii) empirical studies on macroscopic organisms, based on data the authors collected themselves, or else on data from only a modest number of sites/species. I’ll call (ii) “small scale non-microbiome” empirical studies for short, although I’m obviously using “small scale” very vaguely.

One thing I found interesting was just how rare such papers were among the 250 recent, highly-cited ecology papers I looked at. So before we go any further, take a guess as to how rare they were:

The answer is…



About 15. Actually 16 by my count**, but I was skimming so might’ve missed a couple, plus a couple were borderline. So, well under 10% of the most-cited ecology papers published in the last 10 years are theory papers or “small scale non-microbiome” empirical studies. Which is a smaller fraction than I would’ve guessed.

A few comments, and some questions for you to discuss:

  • This post is descriptive, not prescriptive. I’m not bemoaning or applauding the fact that highly cited ecology papers these days are almost all reviews/syntheses, methods papers, large-scale data compilations, and microbiome papers.
  • Note that just because certain sorts of papers dominate the list of highly-cited ecology papers doesn’t mean they dominate the ecological literature as a whole. As I’ve noted in previous linkfests, other data show that ecological journals are not being overrun by reviews, syntheses/perspectives, and big data compilations. Those sorts of papers are a small fraction of all papers, and they’re not increasing in frequency very fast, if at all. No, not even if you restrict attention to leading journals.
  • All of the 16 papers on my list are “small-scale non-microbiome” empirical papers. I suspect I’d have had to expand my search considerably to find any proper theory papers.
  • To what extent does the lack of proper theory among our recently-published, highly-cited papers reflect our increasing collective focus on climate change, vs. maturation of the field (we have the right ideas, we just need to apply those ideas now)? That is, if anthropogenic climate change weren’t happening, and so weren’t a focus of so much research attention, would a lot of the people working on climate change be developing and citing new fundamental ideas instead? I’m not sure.
  • Are verbal “perspectives”, “syntheses”, and “conceptual frameworks” increasing in frequency and prominence at the expense of proper theory? If so, why? I confess I don’t often find such papers very useful myself, and I’m not alone in that. Historically, most of the successful new theories and concepts in ecology have been simple mathematical models that did not not develop from data synthesis or verbal conceptual frameworks (e.g., island biogeography theory, optimal foraging theory, resource competition theory, metapopulation theory; some would add neutral theory, r/K selection, and MaxEnt to this list). And verbal syntheses, perspectives, and conceptual frameworks have a stubborn habit of not developing into proper theories. Now, you could reply that these days, ecologists want system-specific models rather than general theories, and you might be right. But verbal perspectives, syntheses, and conceptual frameworks aren’t notable for spawning system-specific models either.
  • There’s good mathematical theory on range shifts, phenology shifts, and their consequences. How come none of the theory papers on those topics are massively cited?
  • Or alternatively, is looking at the most highly-cited ecology papers a bad way to diagnose and summarize what ecologists as a whole are thinking and doing? For instance, perhaps the fact that we all cite reviews of the effects of climate change just shows that we’ve all decided to use the same spin to “sell” whatever it is that we’re really working on.
  • I’m now curious if/how the picture differs in other fields, like evolutionary biology. The challenge with that analysis would be deciding where evolutionary biology stops and genetics/genomics/bioinformatics begins.

Having post some deliberately-provocative but unanswerable questions, I look forward to hearing you argue in the comments! [grabs popcorn]🙂

*Has Meg ever been wrong?

**My count, in no particular order: Sitch et al. 2008 GEB, Elith et al. 2010 MEE, Broennimann et al. 2007 Eco Letts, Heaton et al. 2008 GEB, Araujo & Luoto 2007 GEB, Inouye 2008 Ecology, Cornwell & Ackerly 2009 Ecol Monogr, Bell & Sih 2007 Eco Letts, Dingemanse et al. 2007 JAE, Lips et al. 2006 PNAS, Schindler et al. 2008 PNAS, Edwards et al. 2007 Nature, Franks et al. 2007 PNAS, Schindler et al. 2010 Nature, Maherali & Klironomos 2007 Science, Hector et al. 2007 Nature, Wittebolle et al. 2009 Nature.

17 thoughts on “What sort of ecology papers get highly cited these days?

  1. I believe highly cited theory papers often accrue their citations slowly (though same could be said of methods, which were common in your dataset).

    It could be that “highly cited” theory papers have been published in the last ten year, they just haven’t been cited much yet. If you did this test in the early 80s, papers like these would not come out as highly cited.

    • Hmm, interesting. On that theory, it’s not that theory used to be highly cited, and nowadays it’s all climate change and statistical methods. Rather, it’s that recent theory hasn’t had enough time to accumulate the many citations it will accumulate (while highly-cited recent papers on climate change and statistical methods will vanish down the memory hole, presumably?)

      Will have to try to figure a way to check this. Not sure I totally buy it, but there might be something to it.

      • The analogue might be citation half-life – how long it takes a journal to accummulate 50% of its citations. TR reports it along with a journal’s IF, but it’s usually ignored. Could be calculated for pubs as well to address question you posed?

        (As an aside, some high IF jrnls have half-lives of 3-5 yrs, while some muddy-boots jrnals gave half-lives of >10 yrs. Hmmm…trendyness vs solid biology w/ staying power?)

      • “Hmmm…trendyness vs solid biology w/ staying power?”

        Mostly just differences among fields, in all sorts of respects, most of which have nothing to do with “trendiness” or “solidity”.

  2. I would count Maherali and Klironomos 2007 as a microbiome paper. Or do you mean by “microbiome” only descriptive inventories?
    One reason for the prevalence of microbiome papers may be (other than being just a hot topic), that work on it was initially underrepresented, and so that many papers would cite them in the intro or conclusion (“oh yeah, microbes may also matter in system x but we don’t work on them”) and so a few papers draw a lot of citations in many fields.

    • Most of the microbiome papers were descriptive inventories based on genomics, and they often had “microbiome” in the title, but I didn’t keep a formal tally.

      I suspect they’re highly cited mostly because the topic is hot, not because lots of non-microbiome papers make passing nods to the microbiome by citing the same few recent microbiome papers. But I didn’t check (though it wouldn’t be hard–just glance at the list of papers that have cited one of those highly-cited microbiome papers, it should be immediately obvious if they’re getting cited by lots of non-microbiome papers)

      • I don’t know how likely this is, but it seems plausible that one factor driving up the number of highly cited microbiome papers is that they’re at the interface of, and cited by, two distinct literatures. Some (certainly not all) microbiome papers might get cited both by the microbial ecologists and the people who are interested in the microbiome for separate reasons (e.g. human health), perhaps because they use common methods or conceptual tools. I don’t know if this is true, since I don’t know which microbiome papers fall onto your list.

        Brian Leiter’s list of the most-cited living philosophers might help illustrate the principle. Seven or eight of the top 10 are people with an explicitly cognitive bent to their philosophy. It seems likely that this is because they get cited by both the philosophers and the cognitive scientists. (And the latter probably publish more papers, and cite more previous papers per paper they write.) I think most philosophers would see Tim Williamson’s work (epistemology) more central to philosophy than Paul Thagard’s (philosophy of computation/cognition), but Thagard gets cited by computer scientists, psychologists, etc. while Williamson is mainly just cited by other philosophers.

      • “one factor driving up the number of highly cited microbiome papers is that they’re at the interface of, and cited by, two distinct literatures.”

        I’m sure that’s part of it. This came up in my old post on the most-cited ecology papers published in the last 10 years. #1 was a paper on “networks” of various sorts, including food webs, that was cited both by lots of ecologists and by lots of people in other fields concerned with “networks” of various sorts.

  3. “Or alternatively, is looking at the most highly-cited ecology papers a bad way to diagnose and summarize what ecologists as a whole are thinking and doing?”

    In a word, yes. To summarize what ecologists are “doing” it’s probably better to count and categorize the publications they are producing, no? Grant proposals, funded grants, and conference presentations are other data that can address this question. Especially since citations counts are biased by lots of stuff other than if the paper is a methods or review paper.

    • I agree that highly cited papers are probably a bad way to summarize what most ecologists are doing? By definition the majority of ecologists cannot all be doing top 1% work in citations (unless we live in Prairie Home Companion’s Lake Wobegone where every student is above average).

      But high citation rates ought to be a sign of what ecologists think is important, no?

      • Sure – except some kinds of research papers are rarely cited, such as the papers describing/naming one’s focal species. If we did I wager some if those papers would make the list. Does that suddenly make them important? Plus, for example, papers with more authors, that have more international author lists, that are written by authors from particular combinations of countries, or that are by authors in the global north get cited more than other papers published *in the same journal*.. We are biased when we identify what is “important”.

  4. I think “does highly cited = important” is the real question. As timcdlucas suggested above, I think the most important papers aren’t necessarily recognized as important until later on and so won’t necessarily have a lot of early-on citations. (Didn’t you write a post about this once?)

    I personally really like reviews and syntheses, as I’ve had to pick up new systems and domains fairly regularly and these sorts of papers — especially if they’re from the past couple years — to be a great shortcut. But really, we cite them because we’re expected to. If I write a paper on topic X, I’ll invariably cite the recent review paper on the topic, because *how can I not?* Reviewers will be all like, “wow, authors didn’t even cite the recent review paper. They’re either sloppy or totally out of touch.” So I wouldn’t read *anything* into reviews being highly cited other than that they’re being used as citation network hubs.

    As for theory, I think ecology has a certain boots-on-the-ground (and/or butt-on-the-bench?) mentality. Being strictly a theorist is a pretty niche job. Mostly these days you gotta do both theory and empirical work to get a job. And if you do both, you probably spend a lot of time and energy on the empirical stuff, because it demands attention. Theory is also not sexy in our field. My read is that many people go into biology because they don’t feel secure in math. Add that to various rules-of-thumb I’ve heard about trying to minimize explicitly writing equations in ecology papers that aren’t strictly in journals that have a mathematically-bent audience, and I’d say that the culture as a whole has a certain math love/hate relationship. Is it a problem? I think that depends on your perspective on what the field ought to be trying to do as a whole and that perspective varies widely.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s