Why teaching Intro Bio makes me think we need to radically change qualifying exams

When I first arrived at Michigan and began teaching Intro Bio, the course had four exams. In that first semester, I added in clicker questions. Since then, we have added in frequent quizzing, so the students now have four exams, plus two quizzes a week (completed before coming to class), plus clicker questions in class. We have all of that because we know that frequent testing improves student learning. (Here’s one review, here’s another, and here’s a summary of the changes we made in Intro Bio and their impacts on student performance.) As a side bonus, when the testing is low stakes (as with the quizzes and clicker questions), students get those learning benefits without paying a cost in terms of increased anxiety. Given all that, I would never consider changing the format to one where we have just a single, pass/fail, high stakes assessment at the end of the semester.

Now, let’s consider graduate prelim/qualifying exams.

While there is variation in how they work across departments, universities, and disciplines (including in whether “prelim” and “qualifying” exams are two different things), they often include a single, high stakes assessment of knowledge, perhaps in the form of an oral exam. I think there’s general recognition that this is not good for student stress levels and mental health (and a new paper supports that general impression), but I think we maybe don’t reflect enough on what this form of assessment means for student learning. Frequent testing helps people learn. (Make It Stick is a great book that talks about this.) One important goal of qualifying exams is to ensure that students have learned, mastering the background knowledge required to move forward on their independent research. So, why are they structured in a way that is not in line with best practices for learning?

A key reason is that qualifying exams are in many ways a historical relic. But I think there’s a sense that there’s a tradeoff between rigor and student well-being. I’m sure there are many people who think that they wish qualifying exams weren’t so stressful for students, but that it’s unavoidable. But, rather than “rigor” (whatever that is), we should be focusing on student learning, and there’s no tradeoff between learning and well-being – those would both be supported by having qualifying exams that involve frequent, low stakes assessments.

So, what reasons, other than historical ones, are there to keep qualifying exams that involve a single high stakes assessment such as a comprehensive oral exam? Are there any? The main possibility I can come up with is that the workload would probably be higher if the format was one where students, for example, write a series of reviews on different topics. But, if faculty workload is really the reason for keeping qualifying exams that involve a big, high stakes assessment such as a comprehensive oral exam, we need to explicitly acknowledge and discuss that.

Are there any programs that have really radically changed the way they do qualifying exams? (No, moving it one semester earlier or later is not a radical change.) I’d love to hear about programs (ecology or otherwise) that are doing creative work related to qualifying exams, especially those that have re-evaluated the approach they take based on scholarship related to how we learn. 

14 thoughts on “Why teaching Intro Bio makes me think we need to radically change qualifying exams

  1. For sure the stress around comperehensive exams is too high. I’ve written about the not just mental but physical accident consequences of the stress https://dynamicecology.wordpress.com/2013/03/28/surviving-your-comprehensive-exams/

    I think professors should (and I have very actively at several institutions) work to de-stress comps. I think departmental culture can change and make a big difference. Just reminding students how few students actually flunk out, that you get at least two tries in almost every institution, transparency of expectations of scope of material and depth of knowledge, transparency of outcome statistics, and professors not cackling and otherwise enjoying terrifying students around comps, and professors getting it through their heads that comps don’t exist to make the student spit out the answer you would give but to give an intelligent, defensible answer of their own call all help with this.

    So format is by no means the only method to reduce stress. But you raise the specific question of whether the format has justification or goes against our understanding of learning as studied in the context of undergrads.

    As with all teaching, it starts with goals.

    I guess to state the obvious we do have different expectations of advanced PhD students vs. first year undergrads. Specifically I’ve always been taught (and believe) the goal of a PhD is something like “to train and certify as trained a person to be an independent researcher”.

    So some things I think that fit that independent researcher model that do fit the current comp format include:
    – ability to dive in and absorb a new topic independently (this is where the frequent testing model might not be a great fit for grads)
    – ability to converse at a deep intellectual level with colleague including defending ideas, examining the limits of ideas, etc (and this especially happens in most scientific job interviews and the comps+defense are the only prior experience students have of this – at least more than 3 minute blip at the end of a conference talk)
    – and even the high stakes part of it, I’ve seen more than one student (never my own advisees of course!) who really needs to learn a topic and keeps resisting it, but then comps is the time you can actually insist they learn the topic(by transparently putting it on the list of expected knowledge). Aside from having a course available that is a good fit to what they need to learn which can be rare, there is no other mechanism to insist they dig in and learn something entirely knew in their largely independent environment.

    So I guess I agree that traditional comps are toxic and need to change. But I have different expectations (if you will demands) of a PhD student than an undergrad. I’m not sure the simple format per se is a bad fit to the learning objectives of a PhD (which are very different from a BIO 100 course). I personally think the real problem is the culture. And I have been involved in a department that intentionally transformed its culture rather effectively. It doesn’t mean all stress was gone. But a lot was gone. Whether you agree with me or not about the format/goals match, I think it is important not to let the culture piece off the hook for stress.

    And 1000s of grad students studying for comps can now commence throwing rotten tomatoes at me 🙂 …

    • I support Brian’s POV. Comps don’t need hyper stressful, but they do need to be a significant challenge. Success should be a milestone event for students, one that leaves them with a substantial sense of achievement and confidence.

  2. As a PhD Candidate, I think the first thing departments need to decide is what are the goals/reasons for the quals. I agree with Brian that expectations are different for grad students. So, I personally don’t think the goal of the quals is to help learning. Whereas with undergrads, we’re hopefully helping teach them best practices for future independent learning (and if they learn that frequently quizzing themselves actually helps them learn, then that’s a win), grad students should be able to do that already.

    In my program, PhD candidates have to pass a day long written exam and a 3 hour oral exam. I felt like my oral qualifying exam was helpful in giving me practice thinking about scientific ideas and discussing those ideas in depth with other scientists (including using my knowledge to think through potential solutions to questions that I didn’t know the answer to). It was good practice and felt in line with the goals Brian discusses in his comment (not to mention something that turned out to be really fun once I got started and my anxiety settled down – something I try to stress to incoming students).

    I’m honestly not sure the goals my department has for the written portion. It felt like I had to demonstrate that I could learn, integrate, and remember a lot of biology information and be able to communicate what I learned without any references in a stressful, timed setting. I understand that learning independently, integrating information, and communicating are important as a scientist, but I’m not sure when I’d be required to do anything quite like this again. That being said, I think I blanched at Meghan’s idea that instead we should have grad students write a series of in depth reviews. I agree that this might be better practice for future work we will be doing and demonstrate our ability to integrate and communicate lots of information. But how long would this take and how much would it add to the already overburdened work load of grad students (not just faculty), which in itself would create added stress. But as I sit here typing up all the reasons we shouldn’t change to this system, I decided to delete those because really I think there might be workable solutions to those. And the real question, as Meghan, points out is are we having the right conversation concerning quals.

    I’ll sign off on one final note, if your department is discussing changing how you conduct your qualifying exams, please include your graduate students in the conversation. We’re stakeholders in this. Older students have been through your current system, and believe it or not, we have some great ideas and don’t actually care about just taking the easiest route!

    • I think you make a good point about writtens. They are much more variable between universities. And I think not coincidentally it is less obvious (to me anyway) what the goals are of the different formats. My PhD program we had 4 questions, 1 week per question. I found them quite helpful. I was given topics I really needed to think about and got to do a lot of reading & thinking (albeit it was exhausting lasting for a month). The goals of shorter writtens are less obvious to me and perhaps might seem more like an exam of knowledge which I don’t think matches well to my understanding of PhD goals.

      • I actually really “enjoyed” the way my prelims were structured. I was in an interdisciplinary policy program, and it was emphatically made clear to us that our committee would strive to make it a useful experience and not a punishment.

        We had a week to complete our take-home exam that included four questions largely based on our proposed dissertation topic – one on theory, one on methodology, one or two on your content area, and then maybe one “surprise.” Starting a few months before, we would meet with each committee member to come up with a “syllabus” of important topics and readings that we would be expected to know, and then meet with folks as needed to discuss the material. The oral exam, depending on how the written portion went, was meant to be an oral defense of your written work.

        Honestly, it was great – I came out feeling prepared to start my dissertation, knowing I had my areas of interest down pat, got a practice orally defending my work, and learning how to meaningfully talk about academic ideas with colleagues. Plus, I had chunks of my proposal and dissertation already developed. I think the faculty realized that cramming a bunch of material for one exam wasn’t really worth much and was more or less a hazing ritual. And besides, if we weren’t good at cramming we wouldn’t have gotten that far anyway. That model may not work in a narrower disciplinary program, but honestly I think it could.

    • PS – I think it is a given that if Meghan is involved the grad students are being consulted heavily. But your point is completely on target. In the departmental destressing that I mentioned, we spent several hours talking with grad students at a retreat about this one topic along with some other later conversations.

  3. Here at U. Laval (Quebec City), we have tried to reduce the level of anxiety by having the PhD student write a 1-page syllabus for each of three topics (something equivalent to what the syllabus would be for an undergrad course on this topic) chosen among a list of 40+ subjects. The syllabi are pre-approved by the committee members, and serve as a basis (and some sort of a binding contract) on what will be discussed during the oral exam. The president of the jury reins in any members wandering outside the themes of a syllabus (it does happen).
    I believe the students learn a lot when they prepare the syllabi, because they have to organize their thoughts and knowledge. The ‘surprise’ effect is lessened during the exam, the student knows what will be discussed. I often hear students propose rearrangements to a syllabus during the exam. I see this as making a huge leap in metacognition.
    Students on the program committee were involved while this exam format was developped and voted.
    It’s not perfect, but better than the grilling session I heard about.

  4. Overall, my comps experience was 80% really great. While I can’t credit my department for that (sorry department…), I can certainly credit my committee. Here’s some things that made the whole comps experience great for me, which may or may not be helpful in thinking about how to change comps:

    1. Several months to a year prior to my comps, 3 of us set up a study group. We were from 2 different departments (basically the ecology department and the traditional range/wildlife department) and all had vastly different areas of specialty, including working on different continents. I learned a LOT in that study group. We constantly quizzed each other on comps questions we compiled from our respective departments, and literally spent time teaching each other concepts one of us knew, but the others didn’t. It was so helpful, and an experience that has really stuck with me in terms of the knowledge gained and overall experience. From a mental health standpoint, it was also really healthy to go through it together rather than alone. It cut way down on the ability of our “imposter syndromes” to take over. I recognize that not all students would take the initiative to set something like this up, and it may not even be right for every student, but what if similar departments offered independent study classes for students preparing for comps where they had to attend the first class or so? It would create the opportunity for this kind of collaborative learning environment. I can see some pitfalls with that idea, but it’s just an idea.
    2. One of my committee members held weekly comps study meetings for her lab, and I was invited to attend. It was cool idea, and brought down the intimidation factor quite a bit. They were about an hour long, and just focused on a few subject areas for discussion.
    3. Another of my committee members held weekly (or maybe bi-weekly) brown bags. A paper/something or two was distributed, and all were invited to attend and discuss (including other professors). There was a lot of “This usually comes up during comps…” kind of discussions had, which was helpful. It didn’t last but a semester or two, but I liked the idea. It also created a relatively low-pressure space for students to speak up without being nervous about “getting it wrong.” So I guess the caveat with 2 and 3 is that they really have to be facilitated in the right way.
    4. All 4 of my committee members scheduled meetings with me to discuss comps and expectations. The responsibility was on me to schedule and ask questions, but it was helpful and again, helped knock back the intimidation factor a bit. Although, they mostly told me what to expect from my other committee members and minimally about themselves…ha!
    5. All 4 of my committee members also made it a point to ask where I wanted to go in my career, then tailored some of their written and oral questions to focus on things that might come up in my future. For example, they asked quite a few questions about how I would explain X to the public, to the state wildlife commissioners, to the master gardener’s club, etc. since I was very interested in (and now do!) extension/outreach. Speaking from the other side of it – I do genuinely use those skills now!
    6. One of my committee members tailored the written portion like a last minute grant proposal the student had to submit (with a full 8 hours to complete it). Relevant; especially given some of the last minute-opportunities that have come my way now! That being said, I second Brian and others saying that the written portion needs to be bounded by goals. I don’t remember them ever being made clear to me, or any of the other grad students really.

  5. Thread:

  6. Pingback: Recommended reads #155 | Small Pond Science

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.