Poll results: how ecologists find papers to read

Recently we polled y’all on how you filter the literature and find papers to read. This was a follow-up to a similar poll we did five years ago. People’s filtering methods surely are changing–but how, and how fast? Is it only old fogeys faculty who still look at journal TOCs these days, or what?

tl;dr: No major changes from last time, although one filtering method in particular seems to be growing in popularity…

Demographics

We got 304 respondents (thanks everybody!), split about evenly between grad students (30%), postdocs (30%), and faculty (26%), plus 10% government/NGO scientists and 4% others. As with all our polls, it’s not a random sample from any well-defined population. And as I’ll discuss, I suspect that it’s a fairly biased sample for purposes of the question asked. But whatever; it’s still a much bigger and more diverse sample than “people you happen to know”, so let’s talk about it.

Results

Respondents were first asked to indicate every method they use to find papers and/or preprints to read. Here they are in descending order of popularity:

1. Following up citations in other papers (used by 87% of respondents)
2. Keyword searches/keyword alerts (71%)
3. Journal TOCs (67%)
4. Twitter (60%)
5. Algorithmic recommendations (55%)
6. Email and/or face-to-face conversations (53%)
7. Author name searches/author name alerts (47%)
8. Blogs not associated with journals (26%)
9. Journal blogs (11%)
10. Other (8%)
11. Facebook (7%)
12. Human recommendation services such as Faculty of 1000 and Peer Community In Ecology (3%)

The vast majority of respondents use at least four different methods to filter the literature. Only 3% use just one or two methods.

Respondents were then asked to indicate their primary filtering method. Here are their responses:

1. Keyword searches/keyword alerts (23%)
2. Journal TOCs (23%)
3. “I use multiple methods about equally” (20%)
4. Following up citations in other papers (12%)
5. Twitter (8%)
6. Algorithmic recommendation services (8%)
7. (no other method is primary for more than 2% of respondents)

Finally, respondents were asked to identify any filtering methods they object to and wish no one would ever use. I asked this because years ago, I got into an argument with somebody who objected to anyone using journal TOCs as a way of filtering the literature, because that helped entrench journals and postponed the Journal-Free Publishing Revolution. That made me curious: just how common is it for scientists to have blanket objections to how other scientists decide what papers to read?

Turns out it’s more common than I thought, but not all that common in an absolute sense: 73% of respondents had no objection to anyone using any of the listed filtering methods. The remaining 27% disagreed with one another as to which methods were objectionable. Every single filtering method listed had at least one person object to its use. The most commonly disapproved-of filtering methods were Twitter (disapproved of by 9% of respondents), Facebook (7%), author name searches/author name alerts (4%), and algorithmic recommendations (4%).

Cross tabs

Use of journal TOCs increases with seniority: 59% of grad student respondents use journal TOCs, vs. 72% of postdocs and 77% of faculty. However, use of journal TOCs as one’s primary filtering method doesn’t vary much with seniority (22% of grad student respondents, 26% of postdocs, and 25% of faculty).

Use of Twitter as a filtering method doesn’t vary much with seniority: 64% of grad student respondents and 66% of postdocs use it vs. 59% of faculty. Nor does use of Twitter as a primary filtering method vary with seniority.

Grad students seem to be a bit more likely than more senior people to follow up citations in papers they read as their primary filtering method. 21% of grad student respondents say that’s their primary method, vs. 12% of postdocs and 13% of faculty. Presumably that’s because students who are first getting up to speed on the literature, and students studying for candidacy exams, are especially likely to follow up citations in papers they read.

There’s a hint that more senior people may be less judgmental of other people’s choices of filtering method. 30% of graduate student respondents object to at least one of the listed filtering methods, vs. 24% of postdocs and 21% of faculty.

Objection to Twitter (the most objected-to filtering method) doesn’t vary appreciably with seniority. If anything, the trend is for decreasing objections to Twitter with increasing seniority (8% of grad student respondents object to Twitter as a filtering method, vs. 7% of postdocs and 4% of faculty).

Discussion

• Many of these results are the same as five years ago. Most people still use several filtering methods. Traditional methods like journal TOCs and face-to-face conversations are still among the most widely-used methods. Certain new and new-ish methods like Facebook, journal blogs, and human recommendation services remain little used. And senior and junior people use pretty similar mixes of methods. (aside: the previous poll specifically asked about methods other than searches, which in retrospect was dumb of me. So you can’t compare use of keyword and author searches in this poll to their use in the previous poll.)
• The big difference with five years ago is that far more people now report using Twitter as a way of filtering the literature. Five years ago, Twitter was one of the less-popular filtering methods; in this poll it was one of the most popular. I’m sure that’s a real change. But I’m not at all confident in its magnitude, because of likely sampling bias. I bet our readers are far more likely to use Twitter than are randomly-chosen scientists (only about 20% of US scientists are on Twitter, if memory serves, which it may not).
• I would not assume that use of journal TOCs will decline over time because their use increases with seniority. Those grad students who go on to become postdocs, and those postdocs who go on to become faculty, might be more likely than others to use journal TOCs. And students who don’t currently use journal TOCs might start doing so as they progress in their careers. Might be interesting to poll people on how their own filtering methods have changed over the years.
• I am very curious to hear comments from people who object to anyone using one or more of the listed filtering methods. In particular, I hope to hear from people who object to methods other than social media or algorithms (because I’m pretty sure I know why small minorities object to those filtering methods). In particular, I’m shocked that anyone would object to other people reading papers suggested to them by their colleagues, or following up citations in papers they read, or doing keyword searches or author name searches. I’m sure people must have their reasons for these objections–but for the life of me I can’t imagine what they are! Which of course just shows that I am unimaginative. So if you object to any of these filtering methods, I hope you’ll explain why in the comments. I’m not looking to rip anyone, I’m just genuinely curious. It’s like if someone told me that they object to anyone riding buses, or eating ice cream, or having a pet dog. My response would be “Really? Why?”