First cut results of poll on manuscript rejections: we deal with a lot of rejection

I recently did a poll asking readers about their experiences with manuscript rejections. This was based on thinking about different submission strategies, including wondering about what the “right” amount of rejection is. In this post, I lay out the big picture results, and then end by asking about what further analyses you’re interested in.

There are lots of figures below, but here’s my summary of the key results:

• respondents to this poll reported a lower acceptance rate at the first journal to which they submitted a manuscript (48.4%) than in the recent Paine & Fox survey (64.8%). They had vastly more respondents (over 12,000!!!), so I trust their number more; other potential factors that might also contribute are discussed below.
• it’s not uncommon for people to need to submit a paper to 3 or more journals before it’s accepted.
• it’s surprisingly common (at least to me) for people to take the “aim high, then drop if rejected” strategy
• people are submitting to stretch journals pretty often—and sometimes it pays off
• there’s a decent amount of uncertainty in terms of how well a manuscript fits a particular journal (on the part of authors, reviewers, and/or editors). This suggests that the concluding advice of Paine & Fox (“We therefore recommend that authors reduce publication delays by choosing journals appropriate to the significance of their research.”) is sometimes easier said than done.
• people aren’t totally giving up on manuscripts as often as I might have thought they might (but this might be explained by the demographics of the poll respondents)

The poll asked people to focus on their three most recent papers. I did some minor data cleaning to rule out things that indicated errors in poll responses (e.g., the number of journals to which someone submitted their most rejected paper should be greater than or equal to the one with the fewest rejections). After that, we had 459 responses to the poll.

How many papers (of the three most recent) were rejected from the first journal to which they were submitted?

The first question asked how many of those were rejected from the first journal to which they were submitted:

If you do the math on this, there were 710 total rejections out of 459*3=1377 possibilities, which is 51.6% rejected from the first journal to which they were submitted. Or, put differently, 48.4% were accepted at the journal to which they were first submitted, which seems notably lower than in the Paine & Fox dataset (“64.8% were published in the first journal to which they were submitted”). Their sample size was much, much larger, which likely explains at least some of the difference. Another possible factor could be if the approach people are taking to publishing (e.g., people being more likely to aim for a “stretch” journal), or the likelihood of rejection at a given journal (e.g., it’s harder to publish in Ecology now than it used to be), has shifted over time: their study focused on articles published between 2009 & 2015. A quick look at acceptance rates suggests it’s possible that is contributing, but I doubt it’s the whole effect, and it seems to vary by journal. I think it’s probably worth a whole post on that topic. (To give a couple of years of data from the ASN Secretary’s reports: the acceptance rate at AmNat was 26.9% in 2005 vs. 20% in 2018.)

How many journals was the least rejected paper (of the three most recent) submitted to?

Fortunately for the sake of our collective egos, most people had at least one paper out of their past three that had been accepted at the first journal to which it was submitted. But 25.5% of respondents reported that their least rejected paper (of their three most recent papers) had been rejected from at least one journal:

How many journals was the most rejected paper (of the three most recent) submitted to?

If we flip that to look at the paper that was rejected the most times (of the three most recent papers), we can see that people deal with rejection pretty regularly: the modal response (reported by 32.2% of respondents) was submission to three journals, and over a quarter of respondents reported submitting to four or more journals.

How often do people take a shot at a selective journal, then drop to a non-selective journal as the next step?

I was a little surprised at how many people reported taking a shot at a relatively highly ranked journal, then moving to a non-selective journal on the next step. I asked about this because I had the impression some people were doing this, but was still surprised by how common it was: 38.9% for one of their three most recent papers, 18.2% for two of them, and 6.3% for all three. (I was surprised because I take more of an incremental approach when moving between journals.)

How many of the three most recent papers were rejected from a stretch journal?

People seem to be submitting to stretch journals pretty often. (Note: I don’t think you can use the figure below to figure out how often people submit to a stretch journal, since the “none” category presumably includes people who never submitted to a journal they considered a stretch *and* people who submitted to a journal they thought was a stretch but had it accepted. But only 33.8% of respondents chose “none”, which means that most people tried a stretch journal at least once.)

How many of the three most recent papers were accepted at a stretch journal?

People seem to be submitting to stretch journals pretty often…and sometimes it pays off! About 35% of respondents said at least one of their three most recent papers was accepted at a journal that they thought was a bit of a stretch. (The two people who had all three of their most recent papers accepted at journals they thought were a stretch are either very lucky or underestimating their work.)

How many of the three most recent papers were rejected from a “safe bet” journal?

Less happily, about half of respondents reported rejection of one or more of their papers from a journal that they thought was a relatively safe bet:

Taken together, I think that the results of the “stretch accept” and “safe bet reject” questions suggest that there’s a decent amount of uncertainty in terms of how well a manuscript fits a particular journal (on the part of authors, reviewers, and/or editors). This suggests that the concluding advice of Paine & Fox (“We therefore recommend that authors reduce publication delays by choosing journals appropriate to the significance of their research.”) is sometimes easier said than done. I know this is sometimes true for me. There are certain papers where I feel pretty certain of what journal is the right target journal, but other times I feel less certain. Add to that that some of the manuscripts where I think there’s a clear “right” journal for them get rejected from those, and things definitely don’t always end up where I first thought they might.

How often are people giving up on papers?

Finally, expanding beyond the three most recent papers, people aren’t totally giving up on manuscripts as often as I might have thought they would, which seems like it’s probably a good thing! Then again, some of this might be related to the demography of respondents, who tended to be relatively early career (see below).

Demographics of poll respondents

I also asked some demographic questions on the survey:

• Most respondents were grad students, postdocs or equivalents, or academic faculty or staff; there was a good spread across those categories, but many fewer in the other categories (e.g., government or museum-based scientist).
• Almost all respondents reported being in positions where research is a major component of how their performance is evaluated.
• Assuming that the person who reported getting their PhD in 206 missed a number when responding, the earliest PhD was from 1973, with most respondents getting their PhDs in 2010 or later. The modal response (10.7% of respondents) was 2017, which is part of why I was wondering above if rejection rates have shifted over time and if that might help explain why the acceptance rate in this poll was lower than in the Paine & Fox study.
• There was a good spread of responses to the question about how contingent the current position is, being split almost evenly between the three response options.
• Most respondents (55%) identified as male, 43% as female, 1% as non-binary, and 1% preferring not to say.
• Most respondents (72%) consider themselves ecologists, 25% do not but work in a closely allied field, and 3% are not and not in a closely allied field.

So, given those results and the things for which we have variation in responses: what further analyses are you interested in? And what stands out to you about the overall results?