Advice on writing reference letters (UPDATED)

Recently Paul Cross wrote to us asking if we had any advice for writing reference letters for people seeking faculty positions, particularly with regard to the tension between wanting to help out the person for whom you’re writing the letter and being totally honest. We decided this would be an interesting post topic*, so here are some thoughts.

Jeremy’s advice:

This is good advice on how to write a reference letter for someone applying for a faculty position, and this and this are good advice for academic reference letters more generally. So go read them and then come back here so I can emphasize a few points I think are particularly important.

[waits]

Welcome back! A few points of emphasis:

• Your job as a reference letter writer is to provide whoever is reading the letter with information and context that isn’t in the applicant’s cv.
• One of the most valuable things you can do as a letter writer is provide context for any features of the applicant’s cv that might need some context (e.g., a gap in publication history due to a parental leave). But don’t reveal any personal details that the applicant doesn’t want to reveal. Talk to the applicant if you’re not sure what it’s ok to say.
• Readers of reference letters are well aware that letter writers have incentives to gush, and they discount gushing accordingly. What matters is concrete evidence of the applicant’s training, accomplishments, skills, and potential, and concrete comparisons to other people you’ve known at the same career stage. Most readers will see through empty praise, especially if it doesn’t match other lines of evidence such as the applicant’s cv. Don’t misunderstand, praise is great–but only if you can demonstrate that it’s merited. You can’t help someone chances by gushing and might well hurt their chances.
• Be honest about those concrete comparisons. It is not ok to just routinely say that everyone for whom you write a letter is in the top X% of their peers, where X is some small number. No, it’s not ok even if you think (falsely) that “that’s what all reference letters say” or “that’s what you have to say to avoid killing their chances.” (Late addition in response to Brian’s comments below: yes, I know it’s probably futile to urge letter writers to be honest when they have strong incentives to be positive. But I figure it can’t hurt…)
• It can count for something to say (truthfully!) that you or your institution would hire this person. But how much it counts for depends on how similar you or your institution are to the person or institution doing the hiring. Different people and institutions look for different things in applicants.
• Reference letter norms differ hugely between North America and Europe (sorry, no idea about the norms in other places). Try to learn and follow the local norms as best you can. The linked advice describes North American practices. Good readers of reference letters will be aware that norms differ and will adjust their reading accordingly. But not all will do so. So when asking for reference letters, you should consider whether your letter writers know and can follow the norms of whoever will be reading the letters.
• Avoid gendered descriptions.

*We don’t always take up suggestions for post topics; see our About page.

Brian’s advice:

I basically agree with what Jeremy said. A few thoughts:

• There IS a tragedy of the commons problem – as an individual writing a letter for a student or for somebody you know there are many incentives to be very positive to help them get a job. As a reader of letters, you want the honest truth. I broadly agree with Jeremy that it is incumbent on us to not give in to the tragedy of the commons and be a good citizen by writing honest letters. But the reality is, I’m sure I’m more honest in letters that I know are going to a colleague to read than to a stranger. I don’t think sweeping this tragedy of the commons under the rug helps. And it leads to all sorts of complex game theoretic analysis. As a reader of a letter, I have to estimate how honest I think the writer is being and discount accordingly. When I am sitting on a departmental or campus committee its pretty easy. All the letter writers are known to me. So I will know that a letter from person A (who is very tough) that makes the candidate sound average is actually far higher praise than a letter from person B that is effusive (but I know that B effuses for everybody). But that only works when I know the letter writers A vs B. Its much harder in inter-campus scenarios. There are no good answers, but acknowledging this exists is helpful.
• This interpretation of what somebody else is really trying to say is that much harder in inter-country scenarios. As Jeremy hinted at, letter writers from the US are much more likely to behave like person B above while letter writers from Europe (and especially Britain) are much more likely to behave like person A. Indeed a puff letter like that from person B could actually be harmful (or at least useless) for somebody in the UK. And beyond those handful of countries we’re familiar with, few know what to write. I recently had a conversation with somebody where we were trying to figure out whether writing letters for somebody in Israel should be more like for the US or more like for the UK. We never did figure it out.
• You can try to help this muddle by signalling at a meta level. By saying something like “I’ve never said this about a student before but they have a better programming skills than any student I’ve seen” is stronger/clearer than “they have some of the best programming skills I’ve seen”. This can of course be gamed and you can say “I’ve never said this before” in every single letter. But I hope most of us are not that cynically gaming the system!
• It is of course easy to write a letter for somebody who is excellent. But what about a letter for somebody who is good? Not the best, but deserving of a job.I think here it is important to carve in bas-relief (what is not said is just as important as what is said). As an example, if I think a person is a top 5% or a top 10% student I will say that in the letter. If I think somebody is a top 50% student, I simply won’t say this. In this way you can write a letter that is positive AND truthful.
• Although many letter writers think what I want is a general assessment (excellent/good/fair), the truth is because of all this complexity of interpretation, I only give so much credence to general assessments (unless its somebody who I know well enough to have a decoder manual of what their general assessments mean). I am much more likely to trust the objective record (papers, test scores) and my own impressions from meeting the candidate for a general assessment. What I find most helpful as a reader of a letter is the details. As Jeremy noted, contextualize work – explain the context – were they working remotely, just had a child (although one has to be careful with this as Jeremy noted, and I certainly think this context applies to men as well as women), etc? But put some dimensions on the person. What are their special strengths. What impressed you most about them. As scientists we shy away from anecdotal evidence, but anecdotes supporting specific claims are almost always the strongest part of a letter. Having spent a lot of time on peer committees (i.e. making tenure recommendations) I can tell you these concrete details and anecdotes are almost invariably what get taken most seriously from the external letters.
• As a consumer of reference letters it is invaluable to actually have a phone conversation with the references (when allowed – its not allowed e.g. for tenure). Much will come out verbally that won’t come out in a letter. Especially with a few leading questions like “if I am going to become this candidates new supervisor, what would you suggest I give them extra help with to make them successful”.
• I think gender bias in letters is subtle but pernicious. A few years back I did intentional review of gender bias in letters I wrote. There was nothing glaring. I was equally likely to use “brilliant” or “top” for men and women. But there were a number of more subtle biases that I was embarrassed to discover. Especially in the area of describing how people work collaboratively. I would mention as praiseworthy when a man worked well with others but only note this in passing for women simply because I subconsciously took it for granted that women always worked well with others (despite examples to the contrary). Conversely, I would single it out as novel if women played leadership roles more often than I did for men. Related to this, it is probably impossible to have a wholly objective interpretation across men and women of whether aggressive behavior is that just right amount that is beneficial (aka leadership) or too far so as to be harmful if you’ve been raised in modern society (which has profound gender-biased judgments on this). And of course there are lots of subtle biases in what we value. For example, is the person who grabs the marker and starts leading the conversation in a group meeting or the person who listens to everybody and provides the perfect summary statement more valuable? I’ll bet the first skill gets remarked upon favorably in letters a lot more often than the 2nd, but I think they’re equally valuable – and while by no means strongly tied to gender, they are not independent of gender either. I don’t have great answers. But it is really instructive once you’ve accumulated a body of letters written to look at them with a really hard nose for these more subtle gender biases. Given all the data on implicit biases (in women as well as men), it is unlikely you are completely escaping gender bias in your letters.

Late additions from Meg (Jun 24, 2015):

I agree with what Jeremy and Brian wrote, so don’t have much to add. A few things I wanted to add are:

1. I find it can be helpful to give a concrete example of something. To go with the leadership example raised by Brian, it might be possible to give an example of when a project was at risk of going off the rails, but that person stepped in and helped the project get back on track.

2. Jeremy and Brian both mentioned the need to consider gendered language in writing letters. As Brian said, we all have implicit biases, and this impacts letters. Occasionally the biases are glaring, but more often they are subtle. This Trix & Psenka study is the classic one on the topic. Here are guidelines for writing letters, provided by the ADVANCE program here at the University of Michigan.

3. Length: for most letters of recommendation, anything less than a page will be seen as “too short”. No one wants to read more than three pages. So, ending somewhere on page two (or three, if really necessary) is a good target.