Survey results: how do you find papers to read when you can’t do a search?

The results of the survey are in! Or at least, enough results for me to write a blog post! (responses have slowed to a trickle, so waiting longer wouldn’t change the results) So without further ado, here’s how y’all find papers to read when you can’t do a search.

We’ve had 149 responses so far as of this writing (almost 2 days after the post went up), breaking down as 46% students, 26% postdocs, 22% faculty, 6% other, and 1 person who didn’t answer. That’s roughly the breakdown I expected based on the reader survey we did last year.

Here’s a graph of the number of respondents in each occupational/career stage category who use each of the main filtering methods (I omitted some rare methods from the graph):

Based on a sophisticated statistical approach known as “eyeballing the data”*, here are the key takeaways:

• Journals rule. The single most popular way that respondents identify interesting papers to read, used by 82% of respondents, is by looking at journal tables of contents. I leave it to you to decide what, if anything, this means for radical arguments that traditional journals (selective ones in particular) should be killed off.
• Following up citations rules too. The second most popular way that respondents identify interesting papers to read, used by 73% of respondents, is by following up citations they happen across in other papers.
• People still talk to people, apparently. The third most popular approach, used by 60% of respondents, was recommendations from colleagues not coming via social media. You know, actual face-to-face conversations, and presumably email as well. 🙂 Indeed, this category was a more common response than all forms of social media combined (Twitter, Facebook, other non-blog social media).
• Blogs are big. I was surprised that 52% of respondents use blogs as a way of finding interesting papers to read. That’s more respondents than use Google Scholar, and “blogs” was almost as common an answer as all forms of social media combined. Maybe I shouldn’t be surprised, since after all I did survey blog readers! I guess I’m surprised because back when I was at Oikos Blog, my posts highlighting new Oikos papers were among my least-popular. Or maybe I’m surprised because I don’t use blogs this way myself. I’m curious: how much of this is due to people reading journal-associated blogs, where most or all of the posts are about the journal’s papers?
• Twitter and Facebook work well for some people and poorly for others. A substantial minority of respondents use Twitter and/or Facebook as a way to find interesting papers. I suspect that how useful you find social media as a way of filtering the literature depends on how many friends and colleagues you have who use it that way, and the extent to which you’re happy to read the same stuff as them.
• Twitter and Facebook are the only social media that matter. Only 4% of respondents use “other social media” (i.e. not Twitter or Facebook, and not blogs if you count blogs as social media) as a way to find interesting papers to read. Presumably that reflects user base. Using social media as a literature filter depends on having a critical mass of friends and colleagues who are also doing so.
• Diversity of methods. The average respondent uses 3.8 different methods. Unless I missed something, every survey respondent uses at least two different methods.
• Choice of methods is (mostly) independent of career stage. I was struck that students, faculty, and postdocs are so similar in how they filter the literature. The only exception is that postdocs and faculty are more likely to use Google Scholar recommendations than are students. Presumably because Google Scholar recommendations are only useful once you’ve published a few papers. It’s definitely not the case that students are relying on newfangled social media tools while the oldsters still rely on journal tables of contents.
• Hardly anybody uses Faculty of 1000. At least, that’s what I infer from the fact that only 10% of respondents use “other” methods–a category which includes, but isn’t limited to, Faculty of 1000.
• Hardly anybody just searches. Less than 3% of respondents said that they only read papers on topics that they’ve searched on or could’ve searched on. I take this as a sign that the vast majority of respondents still believe in keeping up with the broader literature and in the value of stumbling across papers they didn’t know in advance that they wanted or needed to read.

Following up on that last bullet, a couple of commenters on the original post made a good point: if your search terms are broad enough, “searching” shades into “not searching” (i.e. searching on very broad terms and then skimming the results for occasional interesting-looking papers) My little survey wasn’t designed to capture such subtleties.

This is mostly just for fun, of course. The respondents aren’t a random sample of any well-defined statistical population. But statistically-dodgy data are just as good for conversation fodder as good data. So comment away!

*You get the statistical rigor you pay for on this blog.