Note from Jeremy: this is a guest post from Mark Vellend.
I feel tired. It’s mentally fatiguing to judge others and to be judged, and it seems that we scientists spend an inordinate proportion of our time doing one or the other. Do we do more than people in other lines of work? Do we do the right amount given our overarching goals? Or do we do too little, or too much? Could we judge in ways that are less exhausting for everyone? My tired mind thinks we do too much, and that there must be better ways of doing it. But that’s just from the gut, and I have no specific ideas to offer. I’m hoping that when you finish reading this you will have some thoughts to share in the comments.
- As an Associate Editor, I’ve got one manuscript awaiting decision, and probably a couple more “out there” in review (I’m not totally sure). I will make a judgement of each, offering my opinion to an Editor about whether a paper should be published in the American Naturalist. There’s at least one major judgement per month. As a reviewer for many different journals, I’ll do the same, one step back in the process, also roughly once per month. The decisions are consequential for the authors.
- Upon submitting my recommendation to the Am Nat Editor, I will also be asked to rate the reviewers of the paper – in other words, to judge the judgements. I’m not sure, but one can imagine that at some level my judgement of the judgements will be judged by other judges. I sure hope someone judges that judgement. Not.
- I’ve currently got an application package for promotion to Full Professor to read, after which I will communicate my judgement as to whether this person deserves promotion. Maybe 2-4 things like this happen in a given year. The outcomes matter for people’s lives.
- The season of NSERC Discovery Grants is upon us, and as a member of the Evolution and Ecology evaluation panel, December-February will involve evaluating some 40-45 full proposals. February will include an intense week of coming up with final judgements as to whether each grant is insufficient, moderate, strong, very strong, outstanding, or exceptional, in each of three different ways: excellence of researcher, merit of the proposal, and training of highly qualified personnel. Over my three-year term, this committee will determine the funding level for more than half of Canada’s ecologists and evolutionary biologists funded through this program.
- For the dozen-plus grad student committees I’m on, each year will involve a handful of oral comprehensive exams, a handful of defenses, and a larger bunch of progress reports. On top of that will be a few turns as external examiner for Ph.D. exams or defenses elsewhere. The committees will decide whether students can continue their degrees, or get their degrees, in addition to judging all the details of their research.
- And also…job applications and interviews, awards committees (I’ve been on two major ones for scientific societies during the past 3 years), discussion groups critiquing papers, grad student seminar evaluations, undergrad papers and exams…and I’m probably forgetting things.
(2) Being judged.
- Every judgement involves both the judge and the judged, so this section can be short (switch roles from the last section). In sum, for every judgement I render, there is a judgement received, on a manuscript, grant proposal, or promotion application of my own. And while no one has an evaluation sheet in hand when I give a seminar, there are most certainly judgements made, sometimes evident during question period (or by someone falling asleep), sometimes not. After a paper is published, sometimes you receive positive feedback, sometimes rather harshly worded e-mails. The positive judgements feel good, but probably not as bad as the negative ones feel bad. The wear you out.
Important point 1: This is likely a fairly typical judgement load for someone at my mid-career stage (with the exception of the NSERC panel, which is not a constant throughout a career). In other words, I’m not claiming to do any more of this stuff than anyone else, but rather that we all spend a tremendous amount of time as judge or judgee. Little stuff happens daily, and formal judgements are probably given or received several times per week, averaged over the year. It’s exhausting.
Important point 2: The cumulative mental fatigue is not just from doing the work, as it is for an activity like preparing a lecture. It also stems from the knowledge that each decision matters for someone else’s future – some in only a small way (a mark on an oral presentation) but some in a major way (e.g., funding, promotion). The fact that essentially all final judgements involve multiple judges eases the stress somewhat, but it’s still there. On the other side, being judged is always stressful.
Important point 3: The work of judgement is extremely important and very often gratifying. The importance hardly requires explanation: ideas turn into data and data into knowledge via judgements. The activities described above involve helping many people to get better at what they do, to think more clearly and critically, and to improve communication. The judge also learns a lot, and all that is quite gratifying. In short, my intention is not to suggest that we’re just running around like chickens with our heads cut off, following the rules of the system blindly. The work is important and rewarding. But…
- Do some people steer clear of scientific careers once they see how often they’ll need to judge and be judged? If so, are we losing out from the absence of those people? (Note that I’m not asking about the fairness of judgements – a very important issue – just the fact that so much judgement happens.)
- Would science be better off with more doing and less judging?
- To return to my initial questions: Are we doing the right/wrong amount? Could it be done in a less exhausting way (for both judges and judged)? To be sure, some strategies are already aimed at reducing our collective judgement load, such as journal editorial decisions to “reject without review”, or reducing the number of times one can apply for a given fellowship/grant. These strategies also come with costs, so solutions will not be a simple matter.
Anyone else feeling judgement fatigue? I look forward to hearing what you think…