Here’s an issue which I’ve encountered occasionally as a referee over the years (though not recently, and not as a handling editor as far as I can recall). It concerns manuscripts for which a student is the lead author, and their supervisor is a co-author. Once in a while I find that such a manuscript contains one or more serious mistakes, such as confusion about basic concepts, an experimental design that completely confounds key factors, failure to measure important response variables that obviously should’ve and easily could’ve been measured, or serious statistical errors such as analyzing a nested design as if it were a factorial design. The nature of the errors is such that I would not expect to encounter them in papers lead-authored by the supervisor.
So my assumption (and I emphasize that it is an assumption) is that one of two things is going on.* Either the supervisor didn’t really read the paper carefully before it was submitted, and so wasn’t fully aware of the mistakes or of their seriousness. Or else the supervisor was fully aware of the mistakes, but decided that “it’s the student’s paper, let him make his own mistakes”. And of course, these possibilities aren’t mutually exclusive, since a supervisor who gives his students a lot of freedom and lets them make their own mistakes is the sort of supervisor who might let students submit an ms without first reading it carefully.
My question to you is: are you bothered by this? Because I am, but I’m not sure if that’s just me. I’m bothered for several reasons. First, either possibility I’ve described would seem to be a violation of the published rules of most journals, which require that all authors take responsibility for everything in the manuscript. Second, even if those journal rules didn’t exist, wouldn’t you still want to make sure that any science with your name on it was correct? Third, I’m most bothered by the apparent willingness of some supervisors to effectively force reviewers to do the training that the supervisors ought to be doing.
Note that the situation is totally different if the supervisor isn’t a co-author. As a reviewer, I’m not the least bit bothered if I’m reviewing a manuscript sole-authored by a student and find serious mistakes that a more experienced author probably wouldn’t make. Note also that I’m all in favor of allowing students a lot of freedom, including the freedom to make mistakes. But that freedom does not extend to the freedom to make serious, clear-cut mistakes with my name on them.
But then again, maybe I shouldn’t be bothered by this. One could take the view that it’s the job of reviewers to identify mistakes, no matter what the source of those mistakes. Further, even very experienced people do sometimes make serious mistakes (like believing in zombie ideas!), so maybe my annoyance here is based on the false premise that there are some mistakes that should just never happen in any paper with an experienced co-author.
What do you think? Should supervisor co-authors let student lead authors make serious mistakes? Should reviewers care if they do? Or is this whole post just based on a false premise?
*Actually, I suppose there are at least two other possibilities: the supervisor is aware of the mistakes and their seriousness, but either hopes the reviewers won’t notice or care, or else hopes to be given the opportunity to fix the mistakes in a revision. But I ignore these possibilities, because considering them is too depressing.
p.s. to my own students: this post was not inspired by you!
Another possibility is the supervisor doesn’t agree with the reviewer as to the seriousness of the problem. Maybe they don’t think it’s a problem at all!
More likely they see it as a shortcoming rather than a fatal flaw, that would be impossible to fix after the experiment is done and the student has moved on. No study is perfect after all — especially experimental ones. Not every flawed experiment is worth publishing, but as long as the shortcomings are openly discussed and there is something of value in the study, it could be published.
“Maybe they don’t think it’s a problem at all!”
Sure, that’s possible. In the post I give the example of supervisors who believe in zombie ideas. Although in practice I’ve never encountered that, it’s always been things like huge glaring experimental design flaws.
I agree that the most likely possibility in most cases is that the supervisor sees the problems as shortcomings, which can’t be fixed because the student has moved on, but they still want to get a paper out of it. Which I can understand, but which I admit still bugs me a little. The fact that the student has moved on but you still want to get a paper out of the work shouldn’t cloud your judgment as to just how serious the problems are.
I completely agree with the first 2 main objections you bring up – a supervisor should be a mentor and teacher, not just someone who provides funding, lab space and sticks their name on the end of the author list.
From the mentoring point of view, as a student, getting strong feedback from your supervisor is likely to be less soul destroying than from an anonymous reviewer. It would have been for me, anyway. It will still be a valuable lesson learnt for the student, and faster than through any review process.
As for the 3rd possibility (supervisors handing off responsibility for highlighting known mistakes to reviewers), this is verging on unethical behaviour. But how common is it? You’d need to rule out a lot of possibilities before being able to prove such behaviour was deliberate.
How common is the third possibility? I don’t know. Probably not very common. And yes, there are lots of other possibilities one would want to rule out before assuming that was the case.