I wrote a few years ago about our overhaul of Intro Bio at Michigan. We substantially reduced the amount of content we cover in the course (though I suspect current students would be surprised to realize that – it still feels like more than enough). We also added in more in class activities (clicker questions as well as other things such as in class short answer problems and exercises aimed at increasing students’ comfort levels with figures). And, most notably for this post, we added in frequent quizzing. Students are expected to take a quiz before every class, with more basic questions related to the readings for that day, as well as higher order questions related to previous classes. Writing the questions for the quizzes the first semester was overwhelming, but my hope was that, in future semesters, it would be much less work. While it’s been less work, it’s still quite a stressful part of the course for me. After teaching the course multiple times after the semester where we overhauled things, I still feel like I am crawling across the finish line at the end of the semester – and that’s with teaching only half the semester! When I teach Intro Bio the next time, I will teach the whole semester, and I am pretty concerned about what state I will be in by the end of the semester if I teach the course the same way we have in recent years. The current course does not feel sustainable.
In talking with others who use similar approaches, I know I’m not alone in this feeling. Teaching this way takes up a huge amount of time, and we still have our other responsibilities (mentoring students, keeping our research programs going, department service, editorial responsibilities, etc.) Lately, I’ve been in multiple conversations with others where we wondered: what do we do if we’ve made a course demonstrably better for student learning but, at the same time, not sustainable for the faculty teaching it?
To be honest, I’ve actually been a little surprised to realize how many other people feel this way. I assumed that a lot of my problem is my personality. I think that, if I could be less perfectionist, the quizzes would be much less stressful. I’m pretty sure it’s the quizzes that push the course over the edge of sustainability for me. Some of it is that there’s all the assembling of quizzes that we need to do (unfortunately, it never seems to work to just reuse an old quiz, since changes in semester schedules and course content mean there’s always shuffling of materials). On top of that, there’s a non-trivial amount of modification to individual questions that needs to happen – even relatively minor changes to course content lead to needing to read all of the answers on old questions very carefully, modifying ones that will now be confusing to students. As one example: this past year, I cut the phosphorus cycle from class to make space for other material. (Sorry, phosphorus lovers.) That meant that any questions that mentioned phosphorus or weathering or any of the other things that had been in that brief part of the course had to be changed (even if it had just been a wrong answer choice for a question on the nitrogen cycle, for example). This doesn’t sound like a lot, but, with even minor changes to a lecture (e.g., using an updated example in the microbiome lecture) and with two quizzes per week, there are a lot of changes that need to be made. On top of that, we continue to run into technological glitches. This past year, our quizzing platform kept deleting images from questions for reasons that we were never able to understand. (The images would all be there when we checked the quiz before it went live, but then would disappear at some point after that. I would learn this by getting an email from a student who was understandably confused by the question.) And, inexplicably, questions that we’ve used multiple semesters in a row would sometimes end up with the wrong answer keyed on the quiz. The most reasonably hypothesis we could come up with was that gnomes were messing with the quizzes after we assembled them.
Quizzes are live almost all week, to give students as much time as we can to complete the quiz, recognizing that they have lots of things going on in their lives. But this means that I spend most of the week cringing whenever I get an email from a student about a quiz. Usually, when they think there’s a problem, it turns out it’s due to them misunderstanding the material. (Discovering those sources of confusion well before an exam is part of why frequent quizzing is good for student learning!) But sometimes it’s due to a problem with a question, and then we have to go back through and figure out who has already taken a quiz with that question (the questions are drawn at random from a larger bank of questions). Fortunately, our really amazing textbook rep does this for us – I really don’t know how I would find the time to do this! – though it also adds to some of the stress I feel about messing up quizzes. I feel not only like I’ve let down students (again, I realize a lot of that is my perfectionism at work), but also like I’ve created more work for the textbook rep, who already has plenty on her plate.
I feel like I’ve boxed myself into a corner with teaching, feeling like the course is demonstrably better for students, but clearly harder for me personally. As I said, I felt like I was maybe weird in feeling so stressed about the quizzes, but it’s clear that I’m not alone in the general feeling of being boxed into a corner, even if the specific causes for other people differ. At a fundamental level, a lot of it comes down to this style of course requiring much more time than simply showing up and lecturing. One person I spoke with about this noted that it leaves them feeling like a chump: they look at their colleagues who are showing up and doing the sage on the stage model of teaching, investing little effort and therefore having more time for research and everything else. I certainly have thought about how I got the same credit for teaching this course back when all I did was show up and lecture at the students, then go back to my office and focus on other stuff.
I feel fortunate to be at a university that supports faculty doing innovative, interesting things in the classroom – as one example of that, we got extra teaching credit for the course in the semester we overhauled it. But it’s still a research university and, for tenure track and tenured faculty, the primary emphasis is still on research.
You could argue that we should continue to teach these courses in this more intensive way because it’s the right thing to do, and I partially think that. I care a lot about student learning and would have a really hard time choosing to do something that I thought would harm student learning. But, at the same time, by choosing to keep this very intensive model, I am choosing to harm myself. My mental health is generally not in a good place by the end of the semester when I teach Intro Bio. As I said above, that’s when I’ve taught only half the semester! The next time I teach, it will be for the full semester, and I’m worried about the hit my personal health will take during that semester. Is it reasonable for me to continue doing that to myself and my family?
Recently, I discussed all this with a friend at another university. She told me that, motivated in part by my earlier posts on our changes to our course, she considered adding in frequent quizzing to her class. But, when she met with her university’s teaching consultants, they advised against it. That really surprised me — I’d been worried that I’d have to turn in my “Cares about Teaching” badge for even considering getting rid of the quizzes. (Note: as far as I know, those badges don’t actually exist.) The teaching consultant my friend spoke with felt like it was a whole ton of effort for the faculty member, but only a small portion of the grade. In their opinion, it wasn’t worth the investment. It’s true that the quizzes are not a major portion of the final grade in our course, but I think they are valuable for student learning. Part of that is because they more-or-less force students to stay on top of the material, and part is because they get more practice. The latter goal could be accomplished in other ways, though: we could still offer large banks of practice questions for students, and we could make those available weekly. As for forcing students to stay on top of the material: yes, there is value in that, but I wonder if I could change something about the way I do clicker questions in class to accomplish roughly the same goal. And I think it’s also worth considering that, if I find the relentless nature of the quizzes overwhelming, it’s possible that some students do, too. And, finally, another friend noted that having a really stressed out instructor isn’t great for the student learning experience either, which should be factored in.
I think a key question for me is: how do I balance doing what is good for students with being sustainable? I care a lot about teaching, and really don’t want to be someone who phones it in. But there’s also a risk of going to the opposite side of the spectrum and devoting so much time and energy to teaching that it’s not good for my health. I’m much closer to that side of the spectrum than the phoning-it-in side.
In short: there will always be trade-offs (I love these in my research, not in my life!) and I’m starting to question whether the costs of frequent quizzing outweigh the benefits. When I first wrote this post I was truly unsure and looking for ideas and suggestions. But, over the past couple of weeks while it’s been sitting in the queue, I’ve come more and more to think that I really should try running the course without quizzes the next time I teach and see how that goes both for me and for the students.
I’m curious to hear about what other people think and about their experiences. Some things I’m wondering about include: Have you found yourself in a similar situation, where you feel like you improved student learning at a substantial personal cost? If yes, what was it that made it feel that way? Did you make a change in response to that? And how do you make decisions about how to balance student learning vs. all the other things you need and want to do (including maintaining your personal health and work-life balance)?
I’d also be really interested in hearing from others who’ve switched to intensive quizzing about whether you found it stressful, whether you think it helped student learning, whether you plan on sticking with the frequent quizzing model, and whether you think there are things that make it more sustainable.
Dynamic Ecology 
Every. Damn. Day. And, I’d add to this the problem that often those things we know (or at least think or suspect) improves student learning are not the things that the students want or think improve learning…so this is reflected in sour student moods, lower teaching evaluation scores, etc. This, at least in me, builds a further cycle of resentment (“I’m killing myself to make this class even better for you, and this is the thanks I get!?!”). I went from something like 2-3 in-semester exams and a cumulative final to 6 all-cumulative exams (one every 5 lectures) to 4 all-cumulative exams, no final. (And also always with various other low-stakes forms of assessments.) And now I’m trying “specifications grading” as a way to reduce my grading time while still providing a lot of assessment practice. All of this experimentation has come at a great cost to me (very little time spent on hobbies, “self-care”, etc) and by extension, my family. I think it’s very easy to fall into a trap that makes you think “If I only I change this one thing…the class will finally be that thing that is best for all.” but in reality, this may never be true and the feeling will never end. So, yeah, I know exactly how you feel, and no, I have no good advice on how best to design a course! I think I’ve started to realize that some of what I thought was my desire to provide a great course was as much a desire to get my students to say it was a great course. I intend to have a sit-down with myself at the end of this semester (if I make it…) to explore this. I’d like to find a sustainable model that does the job, be confident in that, and allow myself to ignore misguided criticisms on evaluations.
“those things we know (or at least think or suspect) improves student learning are not the things that the students want or think improve learning”
Yup. In a recent linkfest (sorry, can’t find it out), I linked to a new meta-analysis of all the studies that randomly or quasi-randomly assign students to different instructors in multi-instructor courses and then look at how student evaluations of the course and their instructors relate to their mastery of the material. The answer is they don’t; the mean correlation is zero, with a tight confidence interval.
Like you, I use frequent low-stakes quizzes in my flipped intro biostats course, which is something all the pedagogical research advises, for various good reasons. But yeah, students don’t like it. They feel like they’re under constant pressure, even though the quizzes are low stakes.
Many students in my flipped class also don’t like being forced to do background reading outside of class, which is what my quizzes force them to do (that’s one of the main purposes of the quizzes). Many students want to be free to put in as much or little effort as they want outside of class hours, and free to come to class or not as they choose. So they don’t like a course design that basically obliges them to show up to class, and to put in more effort outside of class hours than they otherwise would choose to put in, even if it results in them mastering the material better. I *don’t* say this as a criticism of them; the vast majority of them aren’t lazy! They’re juggling other classes, and part-time jobs in many cases. It’s understandable that many of them don’t like having additional constraints placed on their time allocation.
I was surprised to find that a former student who did well in the course felt like the quizzes were a large portion of their grade, since they’re collectively worth 80 points out of ~740 total, and we drop quizzes and round up and do other things that mean the quizzes generally help students. But, I think some of that is because they are twice a week and so feel like a large portion of the class.
The students I speak to are aware that the quizzes, individually and collectively, aren’t worth a large fraction of their grade. But to some students, every single question that’s worth marks is IMPORTANT. Anecdotally, this attitude seems to be especially common among premed students, but it’s by no means limited to them.
I should also add that by no means do all students feel this way. A non-trivial fraction do, but I wouldn’t venture a guess as to the exact percentage.
This does raise an interesting question about scalability though. If every instructor on campus uses a strongly flipped model (e.g. frequent quizzes to enforce doing reading before class), could today’s students (many of mine have 40 hour a week jobs) actually do that much work? Or are the benefits of flipping really a priority effect of getting more hours out of the student than other courses and therefore unscalable?
Yes, I’ve thought about this with my class. Part of why it works is because we force students to spend more time on the course. I hope we also teach the students about how to study effectively and efficiently, but at some level the benefit comes from the structure forcing them to spend more time on the class. And you are absolutely right that that time has to come from somewhere!
This has come up in old comment threads. It’s definitely something I wonder and worry about. Meghan’s pointed to a bit of pedagogical research trying to separate out how much of the pedagogical benefit of flipped classes comes from obliging students to put in more time/effort, vs. from increasing the pedagogical benefit of each unit of effort put in.
When our Instructional Design people have presented on flipped classrooms, the one point they have emphasised over everything else is that flipping the classroom should not be an excuse to get students to do more work. If the student’s feel that that’s what you’re doing then you and your course are toast. The aim of flipping is to provide time for more meaningful in-class interactions and opportunities for learning. Now, whether all those actually flipping their classrooms stick doggedly to these principles is a different matter.
Whenever you teach a course, you are implicitly competing with other faculty for students’ time. If you want students to learn the material in your course, an important part what you do will be to convince students that your course is worth more time than other courses. You can do that lots of different ways: having tons of work; making the material more interesting; challenging the students more with more sophisticated material and presentation; or just getting them to like you personally so they don’t want to disappoint you.
I personally would have hated the repetitiveness – and constant quizzing – of a flipped course. I strongly prefered courses that challenged me with more sophisticated material.
When you start adding up all the potential influences (I guess what Andrew Gelmen would call “interactions”) on student learning, it gets pretty hard to be convinced that one single change can be isolated and “demonstrated” to have a major impact, and that’s not counting the hit for the obvious trade-offs you make (e.g., covering less material more intensively).
Perhaps that’s why the lecture has survived for millenia – it the best compromise between the many competing interests facing both faculty and students.
Why assume that a flipped course is repetitive or not challenging?
Our flipped intro biostats course covers only a bit less material than the previous unflipped version, and we haven’t reduced the depth or difficulty of the remaining material. And our experience is that even the most advanced students benefit from the flipped course structure, or at least aren’t held back by it.
https://dynamicecology.wordpress.com/2016/02/22/we-flipped-our-huge-intro-biostats-course-heres-how-part-1/
https://dynamicecology.wordpress.com/2016/03/09/we-flipped-our-huge-intro-biostats-course-heres-how-we-did-it/
I will note that the flipped structure seems to make a bigger difference to student mastery of the material when the other instructor teaches it: https://dynamicecology.wordpress.com/2016/05/02/flipping-our-big-intro-biostats-class-didnt-work-for-me-yet/
Jeremy; Thanks for voicing something that’s been bugging me of late. I do similar kind of weekly assessment as you (for us it’s in the form of 10-15 questions based on students analysing a data set similar to one they explored in that weeks lab class). Perhaps we put too much weight on them..? (they’re worth 50% of the final grade, split over 10 assessments; all answers are quiz-like that at most require students to write a sentence, and many questions only require a numeric or multiple choice answer), but in our experience students do much better on these lab-based quizzes than the final exam. Yet over several years the comments we get are similar to those you voice; they feel under constant assessment pressure during the course.
You mention that many students are juggling lots of other courses and activities, a situation that contributes to lack of outside-class reading. To what extent do you think this is, in part, due to the way students do their degrees in N American institutions? In the UK, I had little choice in the courses I could take in my BSc (I did Environmental Geography so my programme was mostly “physical” geography courses, with each year of the three-year programme offering a small range of options beyond the core course for that year) and I couldn’t take a first or second year course in my third year for example. So there were typically no restrictions on class sizes etc, though most were small, especially in 3rd year. Whilst I appreciate that I didn’t get as well-rounded a degree education as students do in our programme here, it did mean that across the curriculum, workloads were well-balanced, there was no stress in trying to fit in the right combination of courses so you could take that specific final-year-level course. The whole experience was much more rigid but it never felt overwhelming. And this was a geography degree, where when they say you are “reading for a degree” the meant it (where my peers in other courses had many more formal hours of lectures and labs, we were expected to spend the time difference in the library and reading).
Basically, I’m wondering if more structure/less choice would help or not with the out-of-class workload bit? (I appreciate the UK system is quite different to the ones in the US or Canada and there are pros and cons to both.)
Our quizzes (which are in lecture, not labs; they do have lab assignments as well) are a bit less frequent than once/week, and collectively are worth only about 15% of the final mark. That 15% is split evenly between quizzes students take individually and quizzes they take as a team (which as an aside are the same quizzes–they take the individual quiz, then immediately get together with their teammates and retake the same quiz, on scratch cards that reveal to the team if they got the right answers). So our students don’t feel constantly pressured because the quizzes cumulatively are a big portion of their course mark. As best I can tell, some of them feel constantly pressured just because there are fairly frequent quizzes, preparing for which obliges them to do assigned textbook readings before coming to class. The weight of the quizzes doesn’t matter. It’s just their frequency, which forces students to frequently give some attention to the course material outside of class hours.
“ou mention that many students are juggling lots of other courses and activities, a situation that contributes to lack of outside-class reading. To what extent do you think this is, in part, due to the way students do their degrees in N American institutions?”
Interesting question to which I don’t know the answer, not having any experience with the UK system. I mean, I knew that it works as you describe, but I have no idea about the consequences for student feelings about their overall workload. Also, students may well pay for their degree programs differently in the UK vs. in North America. “Maintenance grants” from the national government basically aren’t a thing in N. America. But they are in the UK, or used to be until recently, right?
>Why assume that a flipped course is repetitive or not challenging?
Class time for quizzing is repetitive, and if the instructor is spending time generating new quiz questions, then they’re not spending (as much) time refining and improving the course. In my experience, almost any kind of information can be continually refined and compressed to convey more knowledge with fewer words and less time.
>covers only a bit less material than the previous unflipped version
What percentage less? 10%? 5%? 15%? is that a guess or something you’ve assessed by reviewing your older lectures? I say that because it would be pretty easy to miss by 5-10% just guessing. Memory isn’t always a great guide for what you actually covered in class in a course you taught a few years back.
>I will note that the flipped structure seems to make a bigger difference to student mastery of the material when the other instructor teaches it
You indicated that the other instructor invests more time, correct? You also indicated that even with the less time you invest, you’re sacrificing other things that need to get done.
My guess is that when all is said and done, the flipped classroom will wind up in the scrap heap because a) the research is leaving out critical aspects of comparison that make FC’s look better than the really are; b) much of the remaining benefit comes from the extra time invested on the part of both students and profs.
One factor that’s likely being overlooked is that, when you lead students by the nose, they don’t develop the skills they need to learn on their own – much less the motivation to do so. My guess is that other important skill development is being sacrificed to learn less material only slightly better, and mostly for students in the lower 2/3rds of the class.
The only topic we dropped from the course was on the rules of probability. It was 2 lectures out of 36.
And I’m unclear how lecturing teaches students to learn “on their own”. Indeed, if you mean what I think you mean by “learn on your own”, I’m unclear how you could learn that in school at all! No instructor just says to students “this class is on X. Go learn about X on your own, come back to me when you’re done, and I’ll test you.”
Further, students are pretty crap at diagnosing the reasons for their own failures to master course material. “How to learn” is not something most students can teach themselves. For instance, just rereading lecture notes the day before the exam is a lousy way to learn–but left completely to their own devices, that’s how many students study. And when they do badly on an exam in a lecture course, their response is often to just do more rereading of their lecture notes the day before the exam. I wouldn’t expect my students to learn how to learn on their own any more than I’d expect my son to teach himself baseball without any coaching, or teach himself to play an instrument without any lessons.
Terry McGlynn has an old post at Small Pond Science about how he once was assigned to TA a human anatomy lab despite not knowing any human anatomy. By his own admission, he did a terrible job teaching, because in response to every question he’d say “Let’s figure out the answer.” The students hated it, because they were more or less forced to teach themselves anatomy. But his students ended up being the best in the class. I’m guessing this is close to what you have in mind when you say students should be forced to learn “on their own”? I interpret Terry’s anecdote as an argument in favor of active learning–which is precisely what a flipped class is designed to promote and that a lecture isn’t designed to promote. The whole point is that Terry didn’t lecture them on anatomy! Note as well that, left to their own devices, the students would *not* have chosen to learn anatomy that way. They learned anatomy “on their own” (I’d prefer to say “actively”) because the instructor obliged them to. (Terry’s anecdote can of course also be read as evidence that less structured forms of active learning, requiring little or no prep on the part of the instructor, can work as well as the more structured forms of active learning implemented in a typical flipped classroom. But in order to generalize Terry’s experience from that human anatomy class to other topics would require careful thought. I’m not sure what the “unstructured” equivalent of Terry’s human anatomy lab would look like for, say, an intro biostats course.)
As I said above, I agree that flipped classes may not be sustainable for faculty, and that some large chunk of their benefit may come from obliging students to put more time into the course than they otherwise would (which as an aside does *not* mean most students are lazy or unmotivated!). But I’m afraid we part ways when you say that lecturing teaches students the skills and motivation to learn “on their own.”
Jeremy! I’m glad you referenced this post in your year end summary. I apologize for not responding to your comments earlier.
This is nice! Just me here. I won’t be as circumspect as usual. That’s a sign of respect.
“if you mean what I think you mean by “learn on your own”, I’m unclear how you could learn that in school at all”
Obviously people have to learn to learn on their own. Look, that doesn’t mean that you rediscover evolution on your own. It means you work through the material on a daily basis until you understand it inside and out. You take responsibility to do that yourself – to figure out what you understand and don’t understand and, when you don’t understand something, you go find it out by asking the prof or TA. In the courses I took as an undergrad, even the more lecture-disabled profs were usually able to get the basic material on the board and explain it almost sensibly.
The method I used to “learn on my own” was to rewrite and illustrate my notes. I rewrote lecture notes in roughly the original outline form, doing as much detail on illustrations as possible, then expanded it into paragraph form, then recompressed it into the tightest possible outline – one word per line if possible. Working down the outline is really important you to refine the language into parallel language, so that each sentence is the exact same with different words:
A) Complete Bullshit Factors Affecting Formation of Pond Scum:
1) Insulated Soil Base Creates Thermal Inversion in Pond
2) Thermal inversion drives gentle convection
3) convection carries light solids to surface
4) Fog Layer prevents evaporative abduction of light solids
5) Temp inversion at atmosphere interface decreases solubility
6) solubility change causes crystallization
This kind of language structuring makes one concept lead into another. Creating parallel language makes it impossible to force two steps into one Breaking it down to individual steps makes it easy to see when something’s missing. In doing this one invariably runs into problems in ones understanding. You go to the prof or TA and ask for help. Ask the prof/ta to review the parts of your notes where you’re having trouble and make sure you’re clear all the concepts. Once you get it clear, go home and rewrite it again.
This kind of work has many ancillary benefits that don’t come from taking quizzes: students have to read, write and recraft the material on their own. Drawing illustrations forces them to look at the existing illustrations carefully to make sure they understand every detail. They learn better outlining and writing skills. They ensure they understand all the concepts. They use their own agency to work out problems.
With regard to Terry McGlynn’s story, I think that’s hilarious! So what if the students didn’t like it? Waa. And that’s exactly what I would tell them: Waa. There’s work to do, you can all cry later!
I also had to TA a class that I was totally unprepared for: Geodynamics. The prof even told the students “if you want a used car, see jim (he thought I was a bullshitter), if you want to know the material, ask Chen Ho Lee (or whatever his name was)”. But as the course went on I figured out how to help the students even when I didn’t know the material. And in doing that I taught them something more universal that mere geodynamics: general problem solving methods. In the end the old Italian prof was very pleased with my TAing the course and asked me to do it again the next two years.
And really that’s a lot of what’s happening when you’re rewriting your notes. When you do that, you’re encountering problems (holes in your notes), first trying to solve them yourself, then if you can’t solve them, you get help.
Yeah, I just have to add a few more things to this:
Teaching any curriculum isn’t and never has been *just* about the curriculum. There are several core courses in the geology curriculum that virtually no students will ever use again. But they teach important principles, both about the physical world and about how to solve problems so while the information itself isn’t especially useful, the knowledge that’s derived from it is.
Teaching at the university level is about teaching people ***how to think*** first and about the curriculum second, because if you “learn” the curriculum without learning how to think, almost nothing has been accomplished.
Several years ago I got a faculty grant to transform my senior-level Analysis of Populations class into a capstone experiential-learning class, whereby in addition to teaching content related to population estimation and modeling, we added group research projects where teams of students would inherit a real world data set or modeling problem and work with it all semester long, culminating in a formal set of lightning talks, poster presentations, and scientific papers. It’s been wildly successful in terms of student learning objectives, and some students have described it as the highlight of their undergraduate careers, but it’s not very sustainable from a teaching perspective. After running the class in this format the first time, with the addition of team teaching and added TA support, we told our department head that it wasn’t sustainable and if we wanted to continue, we had to split off the new experiential activity as a stand-alone 3-credit class. That never happened, and I’m stuck in the same model as before, sans team teaching and additional TA support. Tomorrow I’m cancelling one of the scheduled scaffolded assignments because I’m already behind in grading the last two, and I’m counting the weeks until the semester is over. So your post resonates with me very much – yes, it is possible to design something that is seemingly effective, but unsustainable for faculty or TAs, which is to say that it’s unsustainable.
Some of the Twitter reaction:
(related to that last one: https://dynamicecology.wordpress.com/2016/04/20/how-much-do-scientists-lecture-and-why-poll-results-and-commentary/)
(interesting idea to not grade homework but let students use completed homework as their only resource for quizzes. Also, I’ll second a couple of the other ideas in that Twitter thread–but although they help, they’re not enough to make our flipped intro biostats course feel sustainable for me or the other instructor.)
Oh man, get out of my head Meghan! This post is the story of my life this term. Even though my survival strategy has been to mail it in as much as I dare, for instance by mostly reusing old quizzes and labs rather than writing new ones. And even though many of the quizzes are multiple choice and marked by scantron machine. And even though I have TA support. And even though my flipped course isn’t nearly as big as your intro bio course. It’s still a ton of work, and it’s still not sustainable.
I think the flipped version of our intro biostats course is better for student mastery of the material than my old lecture-based version. We have various lines of evidence for that. But it’s not a *lot* better, at least not when I teach it (the other instructor is better at teaching a flipped course than I am–in part because she puts in even more effort). Frankly, I’m on the fence about whether I’ll stick with the current structure next fall, or go back to my old approach (mostly lecture but with lots of clicker questions sprinkled in). My research productivity has taken a serious hit (like, a “my grant may not get renewed if I don’t get back to writing papers”-sized hit), my book has been on hiatus for months, I’m stressed, and I’m up late most nights working. I just can’t sustain this.
I recently went back and reread this old post of Terry’s on doing active learning the easy way: https://smallpondscience.com/2014/09/04/efficient-teaching-doing-active-learning-an-easy-way/. Though frankly even his approach would still involve a massive up-front investment in new prep. And I think the time involved in doing all the in-semester logistics would still add up.
I flipped my Ecology course about 5 years ago, and included 3Q “hard” clicker quizzes at class outset: first solo, then w students consulting their 3 person team. Lotta the same problems.
At outset of last semester, when we switched to a new platform, folks were complaining that clickers were even more kludgey. With relief, I told myself: “back to paper and pencil first class of the week”. There was also a bonus of students being able to *draw* their answers. I still reviewed the quiz after, using Q&A. Granted, I have TAs to grade paper quizzes, but there was no decline in grades, it was less stressful, and I better reinforced the visual/graphing component of Ecology (which, of course, is huge).
Here’s the thing: if admin wants more engagement, they have to meet us half-way. In this case, I have excellent TAs who’s responsibility includes attending lectures and grading quizzes.
MD: It seems particularly stressful to have the quiz out there online allowing students to write you emails about it while its out there (when your reply is 90% of the time, after you re and re-read the question, that they don’t know what the answer is). That would give me conniptions. Switching to one time, class time, beginning of the week would accomplish most of your goals, and emphasize that you expect students to have prepared before class.
It does give me conniptions! I’ve thought about switching to in class quizzes, but see two issues (one practical, one philosophical):
1. we have too many students to fit in a single lecture hall, so teach back-to-back sections. Plus there’s a third section that gets all the same materials but in a smaller section. So, if we do in class quizzes, we’d need three versions of each quiz.
2. right now, we encourage class attendance (in part by giving them optional clicker points that almost always end up helping students’ grades) but don’t require it. I kind of like the philosophy of “you decide if coming to class is worth your time” and would want to think about changing that. It might well be worth it.
When you give frequent in class assignments that are part of a student’s grade, what do you do about students who are ill, traveling for a sport, dealing with a major life issue, etc?
For quizzes and marked team activities, students have to give me a documented reason for missing them–illness, sports travel, etc. I also drop the lowest quiz mark, to give them a bit of additional buffer.
For clicker questions asked in class, they only have to answer 75% of the clicker questions asked during the term to get the “participation” mark. That’s to give students a bit of a buffer against life events that prevent them from attending the occasional class. (Of course, there’s usually one student who answers 73%, and then complains to me about how they should get the mark because of some unlucky event that caused them to miss the last class in which clicker questions were asked. That student never seems to have a good answer when I ask “But what about the other ~25% of the classes that you missed?”…)
The last question is easiest: “we drop the lowest X quizzes, assignments, etc” and students have to provide documentation (which all go into a folder and are retrieved at end of semester in case of need (i.e., more than X allowed “0”s)/
The first part…hooboy. Your university has stacked the deck against you. Class quizzes followed by discussion can be very effective. But I understand the constraint of writing 3 quizzes per class meeting.
Two more observations:
Re time management by students, explore the option of not having a final exam, and using, say, 5 short midterms that are cumulative. This is kind of a hybrid model that keeps students engaged w the material (and reduces lectures a bit). They could take half of class time, and then be reviewed in class. I am pretty much convinced that a Final Exam is perhaps the worst pedagogical tool ever invented. But that’s for another day.
Re shorter exams:how many questions are required to tell you how much a student has learned? If you treat it as a sampling problem, probably a lot fewer questions than you are writing.
Further Twitter commentary:
(I agree with that last one. My experience is that it’s not that difficult to teach decently, in the sense of getting most students up to a decent level of mastery. It’s *very* hard to teach better than decently.)
Via Twitter:
Pingback: Balanced teaching. Can we make our teaching more effective for students and ourselves? | Wildlife Genetics
This seems like a search for the least “bad” solution given a system that is out of whack.
I know this is off the topic to some degree, but some departments, especially those that are cramming in as many health professions undergrads as possible, are hiring non-research faculty (sometimes people with M.S degrees) for the intro courses. My impression is that because these people teach the Intro courses each term and are assessed on their teaching outcomes, the students have a better experience.
Related, my own experience with the big life science Intro courses was that I was better off at the local community college than taking the courses at my four year institution. My grades were better because I actually mastered the material. I also spent a lot less money. This was not the fault of the faculty at the 4 year school. I later had basically the same instructors for upper level courses and they were almost all high quality experiences. The problem was the over-stretched system. I am strongly advising my sons to attend our local CC to get all their prerequisite courses out of the way.
Yes – I do think at some point we have to start asking the (to admin) unpopular question of what can you realistically expect out of an x00 student class? Despite heroic efforts, I’m sure its still way less than what you can expect a teacher to accomplish in a 20 person or even 50 person class. At what point does that reality become not the professor’s problem?
Via Twitter:
Thanks for writing this post Meghan. You’ve expressed many of the worries I have with my teaching. Like you I teach an introductory biology class (2 sections, with total enrollment of ~800 students this semester) that also has weekly entrance/exit quizzes, clicker questions and lots of active learning components. Today was my last regular lectures of the semester, and I am exhausted, as there are always a million things to chase after and prepare, as well as adding in new materials/exerices to keep the content fresh. I really love how my class is structured, but it doesn’t seem to be getting simpler in each year. In a few months I start my first sabbatical, and I worry that I have made things too intense for the faculty member who will be substituting for me in my classes next year. While there are clear benefits of this contemporary teaching approach, it does cause issues for both work-life and teaching-research balances, that I am struggling with. Is it sustainable? I’m really not sure (especially after reading this article as I’ve looked to you as a inspiration for my teaching!)
I have never been convinced that if I change everything of what I have done before (lecturing vs. flipping the classroom), I will be better teacher and my students will learn better. Also I know that moving from one teaching style to the other means a lot of extra work that as Meghan said may not be sustainable ( I am also a little bit older that most of you, so maybe no willing to leave my “old ways”). I tend to be a “perfectionist” as Meghan! So I do a combination of both teaching styles. In my upper level course, I used the flipped classroom for 75% of the class and take a lot of my time, because it means a lot of grading, but with 20-25 students it is still sustainable. In my 200 Ecology class ~90, I have resisted the use of weekly quizzes for a low % of the grade. I use the flipped classroom for few portions of the lectures and not every week. I lecture about a topic and ask the student to read some material in the book, provide some questions and discuss the questions instead of lecturing in the next class (no grading involved). I use a total of 4 exam ( one is the comprehensive final) for 80% of the class. Because I hate the idea of all the grade be linked to the exams (mostly multiple choice) I have designed 2-3 group class exercises that required the students to prepare before they class and they complete the exercises during the lecture. In addition I provide some study guide with some questions or problems related to the material covered in each exam. I do not have TAs for this two classes, and teach the class alone, it is still a lot of work but maybe is more sustainable that what Meghan have experienced.
Wow, this really struck a chord with me. I’m a psychology professor teaching our workhorse research methods course to close to 100 undergrads this semester. I’ve taught the course for many years, and over the past few semesters, have steadily adopted new practices. For example, I’ve reduced the amount of lecturing in favor of active learning techniques (including using Learning Catalytics, a classroom response option like clickers), moved to several lower-stakes online quizzes rather than fewer exams, moved to specs-based grading, and moved to having students do pre-lecture activities before every class (mostly these are reading quizzes). Students in this class also write two full empirical reports, one of which is based on a real, ethics-committee approved replication study. I’ve partnered with our writing fellows program so that student work one-on-one with a peer to improve their writing of both reports. (Students also write peer reviews of each other’s work.) The course also entails a lab, the lesson plan for which I create every week – it takes several hours because the things we’re doing this semester are new. Basically, the class feels now like a new prep each time I offer it. I was just recently talking to a couple of colleagues about it and basically said it doesn’t feel sustainable.
Moreover, despite all the work I put into it and the best intentions to maximize learning, my guess is that a good chunk of students don’t love it. It’s already a topic that is, to most students, rather dry – nobody really wants to take it; they do so because they have to. On top of that, many students simply prefer the approach that gives them more latitude to attend or not or do the reading or not, and to sit relatively passively taking notes rather than having systems in place that encourage attending, doing the reading, and doing activities. Also, based on a midterm feedback session that our center for learning and teaching ran at my request this semester, I’ve gotten the feedback that the pace feels too slow to some students; we could certainly cover more ground by spending more of our class time with me lecturing. So, all in all, while all of the above feels to me exactly right in terms of promoting student learning, it has taken a lot out of me; my days are long, they expand into the weekend, I have less time for other things that are really important to me, and it’s all very stressful. I find myself wondering if the trade-off is really ultimately worth it, a feeling magnified by the thought that my course evals are likely to take a hit (injury meet insult).
Thank you for writing this, Meghan! I’m glad to see my own experience echoed in others. Having read this, I’m looking forward to reevaluating my plan for the fall semester (when I teach this again) to come up with something that maintains good pedagogy without sacrificing sanity.
The Tweeter who recommended 5 minutes of silent writing resonates with me. I did a bit of it the last time I taught a giant lecture. It was not that time-consuming and at times even uplifting. It seems do-able in a low-tech+high-tech way if students earn points by writing and uploading a photo of their writing (also check-able by a textbook rep or TA), with the option for paper-less reading by the instructor.
For me, writing is better than quizzes or tests for making biology class memorable after 5, 10, or many more years. Unmeasurable, I know, but doesn’t an an over-emphasis on short-term effects risk killing longer-term goals? Which, respectfully, is probably what you mean when you call quizzes as “better for student learning” mean — because they yield evidence and data, and are short-term?
Here’s another thing I’m sacrificing to keep my head above water teaching this big flipped course (and other courses) this semester: collecting data throughout the term on how the students are finding the flipped course. I’ve decided that for the sake of my sanity, I don’t have the time to do this. I don’t feel like I have time to adjust anything substantial about the course on the fly anyway, so why bother collecting student feedback if you’re not going to use it?
Via Twitter:
Via Twitter:
I can’t seem to find it, but I remember reading a paper that suggested that student mastery responds nonlinearly to the degree to which you flip the class. The key finding was that asking students specific questions that forced them to engage with the material was the most important component of the flipped classroom. The specifics were less important [e.g. whether you used activities, quizzes, clickers, or even more basic approaches]. I wish I could find this paper because I may be misremembering the findings. But if this study was correct, and I am remembering it correctly, there probably isn’t much harm scaling back or modifying the quizzes to protect your sanity.
Via Twitter:
Via Twitter:
Via Twitter:
Are you even sure you are helping your students? One of the easiest traps to fall into is using assessment to compensate for lack of student motivation. With flipped courses we are basically forcing them to study in the classroom instead of providing direction to what they need to study. We end up providing the discipline and energy to get them to class instead of the students using their own will power. This is why over assessment is so rampant, it is compensating for the inability to hold this generation’s attention. All of this leads to the increased work load and stress you are noticing. It sounds to me like you are indirectly teaching your students that they way to get the grade and be successful is to do only and exactly what they are told. our main allegiance is making sure our discipline’s next generation of scholars are provided for. High school teachers may be morally obligated to not leave the bottom behind at the expense of the top students, but we as university teachers are morally obligated to not let the bottom slow down the top. Don’t feel guilty for focusing on the top third at the expense of the bottom third. It is far more more sustainable and fun to focus on helping the top. Instead, focus on making the class fun; present the most beautiful side of the subject. Stop using assessment solely to motivate basic attendance and study habits. Maybe flip a classroom day or two early in the first year to show students how to study on their own, but doing it constantly I think denies the students the chance to grow up to use their own will power to learn how to learn on their own. Let the ones who skip class and don’t do their homework fail and even drop out. In the long run everyone then gets the most important lesson and chance to actually grow.
Why assume that no method of teaching can benefit all students? What data we have, and my own anecdotal impression, is that our flipped intro biostats course works better than the old version for all students, even the strongest ones. Or at least, doesn’t work any worse for any of them. We cover almost all the same material in the flipped course as in the old version, and cover the remaining material in just as much depth and with just as much time devoted to entertaining real-world examples. We’re not holding back the keen, well-prepared students by flipping intro biostats.
Also, students in our flipped class still have to make choices about how to study and how much. I highly doubt they would learn study skills any better in a conventional lecture course.
Pingback: O que é um professor universitário? – Sobrevivendo na Ciência
Via Twitter:
Link to the book:
Via Twitter, Jim Elser’s answer to the question “how do you sustainably flip a huge class?”
Pingback: Efficient teaching: frequent assessments | Small Pond Science
Kudos to Meghan Duffy for writing a great post that has stimulated a wonderful discussion. I’ve enjoyed it both because I periodically teach a General Ecology course using an increasingly flipped model and because I am an author for SimBio, where I have been assisting with the development of an interactive textbook that uses questions to give students practice and to hold them accountable while helping instructors with formative assessment.
As part of my work for SimBio, I’ve spent a bit of time talking with instructors about flipped classes and what has worked for them. The folks I spoken with agree that the flipped model works best if you can hold students accountable for their out-of-class preparation and that quizzes are a good way to do this. Flipped classes already move a bit slower and if you need to spend class time bringing unprepared students up to speed, it’s really easy to have things bog down, which is why holding students accountable–even with low-stakes quizzes–is so important.
What’s more, quizzes have the potential to do more. They can help with formative assessment, allowing us to more effectively use class time. They can offer students a chance to practice with the materials by letting them apply concepts they’ve just learned. They also can help students with meta-cognition, forcing them to think about what they do and don’t understand about the work they’ve just done. People like Anne-Marie Hoskinson, Nichole Barger, and Andrew Martin have published some really great stuff on how meta-cognition can improve student learning in a flipped-classroom setting.
Unfortunately, as Meghan and others have pointed out, developing and grading these quizzes can be overwhelming, especially in large classes. For this reason, I think it’s worth considering what you’re hoping to accomplish with the quizzes and adjusting accordingly. During my conversations with faculty, I’ve received a few suggestions that may be useful and that I want to pass on:
1. Multiple-choice and/or numeric questions are a great way to hold students accountable. These can be offered as part of the homework (i.e., as something completed before class) or as clicker questions in-class. Their advantage is that they can be designed so that grading is quick and easy (though the drafting of the questions is certainly represents a big up-front investment). If the primarily goal is simply to hold students accountable, the questions do not need to be overly-challenging or higher-order. They simply need to be regular and often work best if they are low-stakes.
2. Higher-order/more synthetic questions help students make connections across assignments but tend to be comparatively open-ended and thus require a good bit of grading. To deal with this, some instructors build in-class exercises around these questions rather than using them as homework assignments. Personally, I like like this approach and try to structure portions of my classes around interesting higher-order questions. Additionally, in our text, we’ve been able to build sophisticated algorithms that allow us to provide automated feedback to these a variety of questions.
Other instructors will ask higher-order questions as part of a 5-minute quiz (similar to what Katherine Holmes suggested) and then ask students to exchange papers and grade them based on a rubric provided they provide. There are some issues with student grading one another’s work, but one of the advantages of this approach is that students can see what their peers are doing.
3. Some instructors have touted asking students to reflect and identify what they don’t understand after completing a given assignment. These questions can generate open-ended responses that are comparatively easy to grade (students either have answered the questions or they haven’t) and stimulate meta-cognition, which is valuable in and of itself. Additionally, it’s really useful for formative assessment because a quick review of student answers often allows one to identify common themes that are worth addressing during class.
4. Many instructors have reported that quizzes and assignments that are graded complete/incomplete are often enough to hold students accountable–both for out-of-class preparation and in-class work.
Hope folks find these suggestions useful.
I am a colleague of Maria Gonzales. She brought this discussion to our Department’s attention. I perhaps have the greatest “treadwear” (28 yr) of the commenters so far, so I’ve seen a lot of pedagogical “innovations” over the years, and I’ve tried a lot of them.
I ask the following (re Jim’s comment about competing with other classes), is this really a game – a zero-sum game, even – wherein we are in competition with our colleagues for the students’ time to win recognition (or tenure) from our administrations? Is that even an ethical way to think about it. If we are to “flip” anything, maybe it should be our thinking about why we are doing what we do.
I have always been Socratic: asking questions in lecture, often to specific students, answering questions with other questions, trying to get them all to realize significance and make connections. I think that was the original “flipped classroom” (400 BCE). I have tried major term papers with iterative submissions and those without. I have done traditional labs and tried “inquiry” in the lab and added short, low point-value quizzes on the prep reading (using the questions from the book) at the start of every meeting, in part to drive on-time arrival. I’ve tried being a hard-assed, absolute scale grader, and normalizing scores, dropping low scores, and almost all the stuff that has been mentioned. NONE of these techniques has made any significant difference in final course GPAs or other assessment indicators that I have run over the years.
So, I offer the following. Flipping, quizzing, inquiring – all that stuff – only makes a difference if it is memorable (i.e., makes the material easier to remember). When I was a student (UC-Davis), I learned more easily when my instructors used tactics that made things simply more memorable. These were wide-ranging and often very gimmicky and personal. We have no control over the outside social forces that compete with academic activities, like studying, or students’ motives for being in college. We only have control over ourselves. Focusing on Logos at the expense of Pathos may be misguided (thank you Aristotle).
Pingback: Reporting back from the 1st Symposium of the BES’s Teaching and Learning SIG: Advancing the synergies between Teaching and Research – Functional Ecologists
Pingback: On getting a sense of perspective…or not | Dynamic Ecology