Advice on writing reference letters (UPDATED)

Recently Paul Cross wrote to us asking if we had any advice for writing reference letters for people seeking faculty positions, particularly with regard to the tension between wanting to help out the person for whom you’re writing the letter and being totally honest. We decided this would be an interesting post topic*, so here are some thoughts.

Jeremy’s advice:

This is good advice on how to write a reference letter for someone applying for a faculty position, and this and this are good advice for academic reference letters more generally. So go read them and then come back here so I can emphasize a few points I think are particularly important.


Welcome back! A few points of emphasis:

  • Your job as a reference letter writer is to provide whoever is reading the letter with information and context that isn’t in the applicant’s cv.
  • One of the most valuable things you can do as a letter writer is provide context for any features of the applicant’s cv that might need some context (e.g., a gap in publication history due to a parental leave). But don’t reveal any personal details that the applicant doesn’t want to reveal. Talk to the applicant if you’re not sure what it’s ok to say.
  • Readers of reference letters are well aware that letter writers have incentives to gush, and they discount gushing accordingly. What matters is concrete evidence of the applicant’s training, accomplishments, skills, and potential, and concrete comparisons to other people you’ve known at the same career stage. Most readers will see through empty praise, especially if it doesn’t match other lines of evidence such as the applicant’s cv. Don’t misunderstand, praise is great–but only if you can demonstrate that it’s merited. You can’t help someone chances by gushing and might well hurt their chances.
  • Be honest about those concrete comparisons. It is not ok to just routinely say that everyone for whom you write a letter is in the top X% of their peers, where X is some small number. No, it’s not ok even if you think (falsely) that “that’s what all reference letters say” or “that’s what you have to say to avoid killing their chances.” (Late addition in response to Brian’s comments below: yes, I know it’s probably futile to urge letter writers to be honest when they have strong incentives to be positive. But I figure it can’t hurt…)
  • It can count for something to say (truthfully!) that you or your institution would hire this person. But how much it counts for depends on how similar you or your institution are to the person or institution doing the hiring. Different people and institutions look for different things in applicants.
  • Reference letter norms differ hugely between North America and Europe (sorry, no idea about the norms in other places). Try to learn and follow the local norms as best you can. The linked advice describes North American practices. Good readers of reference letters will be aware that norms differ and will adjust their reading accordingly. But not all will do so. So when asking for reference letters, you should consider whether your letter writers know and can follow the norms of whoever will be reading the letters.
  • Avoid gendered descriptions.

*We don’t always take up suggestions for post topics; see our About page.

Brian’s advice:

I basically agree with what Jeremy said. A few thoughts:

  • There IS a tragedy of the commons problem – as an individual writing a letter for a student or for somebody you know there are many incentives to be very positive to help them get a job. As a reader of letters, you want the honest truth. I broadly agree with Jeremy that it is incumbent on us to not give in to the tragedy of the commons and be a good citizen by writing honest letters. But the reality is, I’m sure I’m more honest in letters that I know are going to a colleague to read than to a stranger. I don’t think sweeping this tragedy of the commons under the rug helps. And it leads to all sorts of complex game theoretic analysis. As a reader of a letter, I have to estimate how honest I think the writer is being and discount accordingly. When I am sitting on a departmental or campus committee its pretty easy. All the letter writers are known to me. So I will know that a letter from person A (who is very tough) that makes the candidate sound average is actually far higher praise than a letter from person B that is effusive (but I know that B effuses for everybody). But that only works when I know the letter writers A vs B. Its much harder in inter-campus scenarios. There are no good answers, but acknowledging this exists is helpful.
  • This interpretation of what somebody else is really trying to say is that much harder in inter-country scenarios. As Jeremy hinted at, letter writers from the US are much more likely to behave like person B above while letter writers from Europe (and especially Britain) are much more likely to behave like person A. Indeed a puff letter like that from person B could actually be harmful (or at least useless) for somebody in the UK. And beyond those handful of countries we’re familiar with, few know what to write. I recently had a conversation with somebody where we were trying to figure out whether writing letters for somebody in Israel should be more like for the US or more like for the UK. We never did figure it out.
  • You can try to help this muddle by signalling at a meta level. By saying something like “I’ve never said this about a student before but they have a better programming skills than any student I’ve seen” is stronger/clearer than “they have some of the best programming skills I’ve seen”. This can of course be gamed and you can say “I’ve never said this before” in every single letter. But I hope most of us are not that cynically gaming the system!
  • It is of course easy to write a letter for somebody who is excellent. But what about a letter for somebody who is good? Not the best, but deserving of a job.I think here it is important to carve in bas-relief (what is not said is just as important as what is said). As an example, if I think a person is a top 5% or a top 10% student I will say that in the letter. If I think somebody is a top 50% student, I simply won’t say this. In this way you can write a letter that is positive AND truthful.
  • Although many letter writers think what I want is a general assessment (excellent/good/fair), the truth is because of all this complexity of interpretation, I only give so much credence to general assessments (unless its somebody who I know well enough to have a decoder manual of what their general assessments mean). I am much more likely to trust the objective record (papers, test scores) and my own impressions from meeting the candidate for a general assessment. What I find most helpful as a reader of a letter is the details. As Jeremy noted, contextualize work – explain the context – were they working remotely, just had a child (although one has to be careful with this as Jeremy noted, and I certainly think this context applies to men as well as women), etc? But put some dimensions on the person. What are their special strengths. What impressed you most about them. As scientists we shy away from anecdotal evidence, but anecdotes supporting specific claims are almost always the strongest part of a letter. Having spent a lot of time on peer committees (i.e. making tenure recommendations) I can tell you these concrete details and anecdotes are almost invariably what get taken most seriously from the external letters.
  • As a consumer of reference letters it is invaluable to actually have a phone conversation with the references (when allowed – its not allowed e.g. for tenure). Much will come out verbally that won’t come out in a letter. Especially with a few leading questions like “if I am going to become this candidates new supervisor, what would you suggest I give them extra help with to make them successful”.
  • I think gender bias in letters is subtle but pernicious. A few years back I did intentional review of gender bias in letters I wrote. There was nothing glaring. I was equally likely to use “brilliant” or “top” for men and women. But there were a number of more subtle biases that I was embarrassed to discover. Especially in the area of describing how people work collaboratively. I would mention as praiseworthy when a man worked well with others but only note this in passing for women simply because I subconsciously took it for granted that women always worked well with others (despite examples to the contrary). Conversely, I would single it out as novel if women played leadership roles more often than I did for men. Related to this, it is probably impossible to have a wholly objective interpretation across men and women of whether aggressive behavior is that just right amount that is beneficial (aka leadership) or too far so as to be harmful if you’ve been raised in modern society (which has profound gender-biased judgments on this). And of course there are lots of subtle biases in what we value. For example, is the person who grabs the marker and starts leading the conversation in a group meeting or the person who listens to everybody and provides the perfect summary statement more valuable? I’ll bet the first skill gets remarked upon favorably in letters a lot more often than the 2nd, but I think they’re equally valuable – and while by no means strongly tied to gender, they are not independent of gender either. I don’t have great answers. But it is really instructive once you’ve accumulated a body of letters written to look at them with a really hard nose for these more subtle gender biases. Given all the data on implicit biases (in women as well as men), it is unlikely you are completely escaping gender bias in your letters.

Late additions from Meg (Jun 24, 2015):

I agree with what Jeremy and Brian wrote, so don’t have much to add. A few things I wanted to add are:

1. I find it can be helpful to give a concrete example of something. To go with the leadership example raised by Brian, it might be possible to give an example of when a project was at risk of going off the rails, but that person stepped in and helped the project get back on track.

2. Jeremy and Brian both mentioned the need to consider gendered language in writing letters. As Brian said, we all have implicit biases, and this impacts letters. Occasionally the biases are glaring, but more often they are subtle. This Trix & Psenka study is the classic one on the topic. Here are guidelines for writing letters, provided by the ADVANCE program here at the University of Michigan.

3. Length: for most letters of recommendation, anything less than a page will be seen as “too short”. No one wants to read more than three pages. So, ending somewhere on page two (or three, if really necessary) is a good target.

7 thoughts on “Advice on writing reference letters (UPDATED)

  1. A further thought re: the notion that you have to gush in a reference letter (or say that the candidate is in the top 1% of their peers, or whatever) or else you’ll kill the candidate’s chances. That’s the wrong way to think about it. If the candidate’s only chance is for you to gush (or say they’re in the top 1%, or whatever), then the candidate has no chance. You’re not killing their chances by not gushing, because they never had a chance in the first place.

    And if the candidate really is competitive, then the only thing that will kill their chances is a letter that does a crap job of explaining that reality.

    Note that if there are specific respects in which the candidate might appear unsuitable, you can address these head-on in your reference letter. For instance, if it’s a reference for a faculty position, and the candidate has no teaching experience, you can note that up front but then talk about the pedagogy training courses he or she has taken or give examples of how he or she is great at learning new things and preparing for new duties. Identifying specific concerns the readers of your letter might have, and addressing them head-on, is much more useful to the reader (and so is a much more effective way to maximize the candidate’s chances) than just gushing.

  2. Re: Brian’s remark about carving in bas relief, in some contexts this is impossible. For instance, applicants to my department’s graduate program need three references. Those references have to both write a letter, and fill out a form which obliges them to compare the student to others in various respects, checking boxes to indicate top 5%, top 10%, etc. And while there are “unable to judge” boxes, checking off a bunch of those for a student whom you purportedly know well would be a huge red flag.

    In practice, my experience is that everyone who applies to our graduate program is considered by all their references to be top 20% or better in every respect; exceptions are rare. I think that’s somewhat reasonable. Referees are comparing the undergrad applicants to all undergrads, and the applicants who already have a master’s to all master’s students. Undergrads who apply to grad school, and master’s students who apply for doctorates, probably are substantially above-average in most respects compared to all undergrads or all master’s students. But I also think it reflects referees being more positive than is truly warranted, and so as a reader of applications I have to discount accordingly. Which means that as a reader I find the checkbox forms only mildly useful.

Leave a Comment

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s