First cut results of poll on manuscript rejections: we deal with a lot of rejection

I recently did a poll asking readers about their experiences with manuscript rejections. This was based on thinking about different submission strategies, including wondering about what the “right” amount of rejection is. In this post, I lay out the big picture results, and then end by asking about what further analyses you’re interested in.

There are lots of figures below, but here’s my summary of the key results:

  • respondents to this poll reported a lower acceptance rate at the first journal to which they submitted a manuscript (48.4%) than in the recent Paine & Fox survey (64.8%). They had vastly more respondents (over 12,000!!!), so I trust their number more; other potential factors that might also contribute are discussed below.
  • it’s not uncommon for people to need to submit a paper to 3 or more journals before it’s accepted.
  • it’s surprisingly common (at least to me) for people to take the “aim high, then drop if rejected” strategy
  • people are submitting to stretch journals pretty often—and sometimes it pays off
  • there’s a decent amount of uncertainty in terms of how well a manuscript fits a particular journal (on the part of authors, reviewers, and/or editors). This suggests that the concluding advice of Paine & Fox (“We therefore recommend that authors reduce publication delays by choosing journals appropriate to the significance of their research.”) is sometimes easier said than done.
  • people aren’t totally giving up on manuscripts as often as I might have thought they might (but this might be explained by the demographics of the poll respondents)

The poll asked people to focus on their three most recent papers. I did some minor data cleaning to rule out things that indicated errors in poll responses (e.g., the number of journals to which someone submitted their most rejected paper should be greater than or equal to the one with the fewest rejections). After that, we had 459 responses to the poll.

How many papers (of the three most recent) were rejected from the first journal to which they were submitted?

The first question asked how many of those were rejected from the first journal to which they were submitted:

bar graph showing that most respondents reported having 1 or 2 papers rejected from the first journal

If you do the math on this, there were 710 total rejections out of 459*3=1377 possibilities, which is 51.6% rejected from the first journal to which they were submitted. Or, put differently, 48.4% were accepted at the journal to which they were first submitted, which seems notably lower than in the Paine & Fox dataset (“64.8% were published in the first journal to which they were submitted”). Their sample size was much, much larger, which likely explains at least some of the difference. Another possible factor could be if the approach people are taking to publishing (e.g., people being more likely to aim for a “stretch” journal), or the likelihood of rejection at a given journal (e.g., it’s harder to publish in Ecology now than it used to be), has shifted over time: their study focused on articles published between 2009 & 2015. A quick look at acceptance rates suggests it’s possible that is contributing, but I doubt it’s the whole effect, and it seems to vary by journal. I think it’s probably worth a whole post on that topic. (To give a couple of years of data from the ASN Secretary’s reports: the acceptance rate at AmNat was 26.9% in 2005 vs. 20% in 2018.)

How many journals was the least rejected paper (of the three most recent) submitted to?

Fortunately for the sake of our collective egos, most people had at least one paper out of their past three that had been accepted at the first journal to which it was submitted. But 25.5% of respondents reported that their least rejected paper (of their three most recent papers) had been rejected from at least one journal:

Bar chart showing most people reported submitting their least rejected paper to only 1 journal

How many journals was the most rejected paper (of the three most recent) submitted to?

If we flip that to look at the paper that was rejected the most times (of the three most recent papers), we can see that people deal with rejection pretty regularly: the modal response (reported by 32.2% of respondents) was submission to three journals, and over a quarter of respondents reported submitting to four or more journals.

bar chart showing the modal response was submitting the most rejected paper (of the three most recent) to 3 journals

How often do people take a shot at a selective journal, then drop to a non-selective journal as the next step?

I was a little surprised at how many people reported taking a shot at a relatively highly ranked journal, then moving to a non-selective journal on the next step. I asked about this because I had the impression some people were doing this, but was still surprised by how common it was: 38.9% for one of their three most recent papers, 18.2% for two of them, and 6.3% for all three. (I was surprised because I take more of an incremental approach when moving between journals.)

only 36.5% of respondents said they never took a shot at a high ranked journal then dropped to a non selective one

How many of the three most recent papers were rejected from a stretch journal?

People seem to be submitting to stretch journals pretty often. (Note: I don’t think you can use the figure below to figure out how often people submit to a stretch journal, since the “none” category presumably includes people who never submitted to a journal they considered a stretch *and* people who submitted to a journal they thought was a stretch but had it accepted. But only 33.8% of respondents chose “none”, which means that most people tried a stretch journal at least once.)

bar chart showing the modal response was one paper (of the most recent 3) rejected from a stretch journal

How many of the three most recent papers were accepted at a stretch journal?

People seem to be submitting to stretch journals pretty often…and sometimes it pays off! About 35% of respondents said at least one of their three most recent papers was accepted at a journal that they thought was a bit of a stretch. (The two people who had all three of their most recent papers accepted at journals they thought were a stretch are either very lucky or underestimating their work.)

Bar chart showing over 60% of respondents said no papers (of their last 3) were accepted at a stretch journal

How many of the three most recent papers were rejected from a “safe bet” journal?

Less happily, about half of respondents reported rejection of one or more of their papers from a journal that they thought was a relatively safe bet:

bar chart showing slightly over half of respondents said none of their three most recent papers had been rejected from a journal they thought was a safe bet

Taken together, I think that the results of the “stretch accept” and “safe bet reject” questions suggest that there’s a decent amount of uncertainty in terms of how well a manuscript fits a particular journal (on the part of authors, reviewers, and/or editors). This suggests that the concluding advice of Paine & Fox (“We therefore recommend that authors reduce publication delays by choosing journals appropriate to the significance of their research.”) is sometimes easier said than done. I know this is sometimes true for me. There are certain papers where I feel pretty certain of what journal is the right target journal, but other times I feel less certain. Add to that that some of the manuscripts where I think there’s a clear “right” journal for them get rejected from those, and things definitely don’t always end up where I first thought they might.

How often are people giving up on papers?

Finally, expanding beyond the three most recent papers, people aren’t totally giving up on manuscripts as often as I might have thought they would, which seems like it’s probably a good thing! Then again, some of this might be related to the demography of respondents, who tended to be relatively early career (see below).

most poll respondents said they had never submitted a manuscript to at least one journal then stopped submitting it even though it hadn't been accepted

Demographics of poll respondents

I also asked some demographic questions on the survey:

  • Most respondents were grad students, postdocs or equivalents, or academic faculty or staff; there was a good spread across those categories, but many fewer in the other categories (e.g., government or museum-based scientist).
  • Almost all respondents reported being in positions where research is a major component of how their performance is evaluated.
  • Assuming that the person who reported getting their PhD in 206 missed a number when responding, the earliest PhD was from 1973, with most respondents getting their PhDs in 2010 or later. The modal response (10.7% of respondents) was 2017, which is part of why I was wondering above if rejection rates have shifted over time and if that might help explain why the acceptance rate in this poll was lower than in the Paine & Fox study.
  • There was a good spread of responses to the question about how contingent the current position is, being split almost evenly between the three response options.
  • Most respondents (55%) identified as male, 43% as female, 1% as non-binary, and 1% preferring not to say.
  • Most respondents (72%) consider themselves ecologists, 25% do not but work in a closely allied field, and 3% are not and not in a closely allied field.

So, given those results and the things for which we have variation in responses: what further analyses are you interested in? And what stands out to you about the overall results?

 

 

9 thoughts on “First cut results of poll on manuscript rejections: we deal with a lot of rejection

  1. Interesting. I agree that many of the trends are likely related to the career stage of the respondents. I can’t remember if you asked in demography if faculty were pre- or post-tenure, but I think that would be an interesting split point on publication strategies. Personally, since receiving tenure I’ve all but abandoned the stretch approach–except in the case when I’ve had a graduate student strenuously lobby for a stretch attempt. In one case, the student the student was right (but not in my last 3 pubs). In another case the student was quite wrong but rejection came very quickly and we moved on. I have to admit that in the last case I knew the paper would be rejected, but because the student was so adamant, I figured it would be a good learning experience for him. Often the journal choice now is based on balancing the needs of my students to publish in as reputable/high impact (but also available/not paywalled) journals vs getting published. My answer to the question, “Is it more important to have a publication in a second tier journal or to delay publication by hoping to get it in a top tier journal?” is usually the former. Then you have more time for the second and third pubs and much research access now is through search engines and not directly through journal TOC.

    Also, there are a lot of journals out there. I have a pretty good feel for what is appropriate to submit to the 5-6 journals I read the most. But once I get outside of that realm, and especially with the legitimate open access journal proliferation, my choices are little better than a shot in the dark based on impact, browsing TOC, and potentially an email to an editor if I know one (personally or by reputation).

    • I didn’t ask specifically about tenure or not, but asked “how contingent is your current position” with options of “I am likely to be in more-or-less the
      same position in 5 years”, “I am unlikely to be in more-or-less the
      same position in 5 years”, and “I hope to be in a more secure version
      of the same position in 5 years (e.g., shift from VAP to tenure track, move
      from pre-tenure to tenured, move from short term research contract to long
      term one)”. There was also a question about current position, which included “grad student”, “postdoc”, and “academic” as options, so I think the combination of those two questions would give insight into what you’re getting at. My hypothesis, though, is that there might not be a big effect. Most of my papers have early career folks as lead authors, so I don’t think my publication strategy changed much with tenure.

  2. Pingback: First cut results of poll on manuscript rejections: we deal with a lot of rejection — Dynamic Ecology | The Waterthrush Blog

  3. I think this is a very important message. Ability to deal with rejection and stay motivated is one of the secret key skills to success in academia for sure. I am not at all minimizing that impact or apologizing for the system, but it is not super obvious to me that there is a desirable change to fix it. Peer skepticism is a key part of the scientific method. And of course other professions deal with rejection too (salespeople lose deals, lawyers lose clients and cases, etc). But I do think naming this and talking about and helping people develop skills is super important.

    I agree with Skip above that just expecting people to know what a given journal wants is not a realistic means to avoid rejection. I’ve had cases where after rejection I submit to a higher journal and get accepted. Experience does help, but unless you want to aim low (which I don’t recommend for either one’s science or one’s career) one will miss and get rejected.

  4. I’ve had the anecdotal impression that the “take a shot at a highly selective journal, then if the ms gets rejected go straight to an unselective journal” strategy has increased in frequency over the last 10-15 years. To the point where I wonder if we’re heading towards a future in which there are fewer “tiers” of journals in the minds of many authors than there used to be. Put crudely: is the “middle class” of journals going to vanish in future if authors increasingly don’t bother with them?

    Trying to decide if this poll supports that impression or not. On the one hand, a substantial fraction of respondents report pursuing that strategy with at least one paper. On the other hand, a substantial fraction report needing more than 2 tries to get a recent paper published.

    • I cannot tell how often your jump straight to safe strategy is used, but from the papers I see at GEB plenty of people are still using the one rung (or a couple of rungs) of the ladder at a time approach.

  5. Pingback: My paper was rejected. What are my options? - International Science Editing

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.