A curmudgeon’s musings on modern pedagogy

(warning this is long – you can skip to the conclusions or even bottom-bottom line at the end if you want)

I am not an expert on pedagogical methods. But I have been on the teacher side of university education for almost 20 years. And I’ve literally taught 100, 200, 300, 400, 500 and 600 level classes. I’ve taught classes ranging from 630 students to 3. Math-oriented to field-based. In short a pretty typical mid-career teaching history. And about 8 years ago, I took over a 600+ student intro bio class (basically BIO 100) and spent a lot of time thinking about goals which led to my introducing clickers which led to my basically being the lead academic (working with the campus learning center) leading clicker introduction in basic science classes across campus. And I was a TA in a class before and after introduction of active learning. (my most recent experience with changing pedagogy in a class is discussed below) So I’ve formed a few opinions along the way.

I am by no means at a settled state of where I think university education should go. But the following are a few thoughts and musings. (NB Meg has a series of good posts on this topic as well: here   here and Friday links here  and Terry has a bunch of good posts over at Small Pond here and here).

Point #1- Buzzword blur – we tend to just lump all the trends together but they are not the same. You can do one without the other. (And there are distinct goals and rationales in each case). Here is a quick tour

• Active learning – activities in which the students are not just passively listening but actively producing knowledge via inquiry, answering questions, discussing, etc. This was one of the earliest movements (in ascendancy in late 90s).
• Peer instruction – a model in which students teach each other. Often students are given a question and then discuss the answer with their peer students. This draws on research showing most people learn better in a social context. When tested via before & after versions of the same question using clickers I am astonished at the improvement (often 10% right to 95% right).
• Flipped classroom – the buzzword du jour – this starts from the notion that lecturing is a relic from the days when textbooks were rare (hand copied). Flipping means students do passive learning (reading, watching lectures) at home on their own schedule, and then uses the classroom with the instructor present to do something more active where the instructor can intervene and assist. This can be as simple as having students do what used to be their homework now done in class and raise their hand for help to much newer approaches like peer instruction.
• Just-in-Time-Teaching – the notion that the teacher will dynamically adapt the material being taught based on real-time feedback on what students are not understanding. This implies an ability to reteach material in a new way. It also implies real time feedback either from quizzes just before class or some in class feedback mechanism (clickers, hands raised) or although nobody talks about it old-fashioned sensitivity to puzzled looks on students faces.
• Inquiry based learning/Investigative learning – instead of teaching material, giving students problems (specifically non-trivial problems) to solve. The teachers role is as a facilitator to help students discover first the process they need to use then the answer to the questions themselves.

Point #2 – Clickers – clickers are just a tool – they can be used for any of the above techniques or for purposes not listed above. At one end clickers can be used to pose simple multiple choice questions and then reward or penalize based on attendance (there is a difference and both are possible) Clickers can also be used in peer instruction (get clicker answers, show what everybody answered, discuss for 2 minutes with peers, then revote – amazing improvement occurs)  Clickers can also be an important tool in just-in-time-teaching if the teacher is flexible enough (i.e they’re a great way to find out if the students really understand what you just taught if you’re brave enough to deal with a no they didn’t answer). Generally one should only expect as much out of clickers as one puts into them. And clickers have real issues about cost – old fashioned methods like hand raising can do many of the same things (although its harder to force 100% participation). Honestly, I think the single biggest value of clickers is to serve as a disruptor and force you to think about how and why you teach. And if you don’t do that thinking, then clickers aren’t doing much.

Point #3 – Remembering why we are doing this – Although often not made explicit the goal of most of the techniques listed in Point 1 is to elevate learning up Bloom’s taxonomy. If this is not the goal, then such techniques are not necessarily the best approach. Bloom’s taxonomy was formulated in three domains: cognitive, emotional & physical, but the most talked about and the relevant one here is the cognitive. This recognizes the simple idea that there are different levels of learning starting with knowledge (memorize facts), then comprehension (extrapolate/understand), then analysis (using knowledge) then synthesis then evaluation. The last sentence is immensely oversimplified of course. But this is the central motivation of all of these techniques: to elevate learning up the taxonomy. Much of the origin of these techniques started in physics when people realized students were memorizing formulas enough to plug and chug on tests, but had major failures in basic intuition about how physics works. So they began teaching to develop higher level mastery.

Learning higher up on the taxonomy is obviously a good thing. But the thing I never hear anybody discuss is that it is part of an inherent trade-off. It is essentially a depth vs breadth trade-off. Any realistic application of active learning etc techniques to elevate learning involves covering less material. Covering better, but covering less. Are there times and places in university courses to cover the breadth rather than the depth? I struggled with this question a lot teaching intro bio. The breadth expected of that course from higher level courses, and indeed the breadth of life gives a strong demand in the breadth direction. But to cover it meant giving up on deeper understanding of higher level concepts like homoplasy. Which is more important: a) truly understanding homoplasy rather than just being able to regurgitate a definition of homoplasy (e.g. being able to produce new examples of homoplasy which would probably be the applying or 3rd level of Bloom’s taxonomy) or b) remembering platyhelminthes and their acoelemate architecture and basal position (level 1 or remembering)? Maybe some of you out there are such fantastic teachers you can achieve both in a semester. But in my experience this trade-off is very real (not on just these two exact topics of course but on these two levels of learning across all of the material to cover in an intro bio class). I never did fully decide what I thought about this and I’d be curious to hear what others say. But I do strongly believe there is a trade-off between breadth and depth (moving up the taxonomy).that is not talked about enough.

Point #4 – Notetaking – I find it ironic that in this day and age of focus on active learning and moving up the taxonomy, teachers have largely capitulated on giving students copies of powerpoint slides and eliminating a very effective method for doing real-time active learning while listening to lectures (with many studies showing that note taking is a very effective learning method). And nobody is calling this out.

Point #5 – You can go halfway (or 10%) in – It seems to me the conversation is very binary. All-in flipped/active learning/peer instruction 100% of the time or boring old traditional. This is totally bogus. If active learning has value, then one five minute exercise per hour (or even every other class) has value. And practically, it is very possible to choose anywhere on the spectrum from 0% to 100% flipped/active. This is also my reason for being pedantic and breaking apart the ideas in point #1. One can flip without inquiry based,  do active learning without just-in-time, etc.

Point #6 – This is not new – Another thing that is not discussed very often is that these techniques are hardly new (but see this link and commentary of Terry’s). Socrates was demanding productive/active learning using inquiry based techniques and peer instruction 2500 years ago. And many teachers have been doing the same for decades (and millenia).

Point #7 – How hard is it to do? – You can find various opinions about how much work it is to flip a class room (see Meg here and Terry here). My main experience was also the first time I taught the class so it is hard to separate the two. I don’t think I have an informed opinion. But I do think that for those of us raised in the traditional lecture mode, it can take more creativity and emotional energy to do something new and different.

Point #8 – Does it work? – My sense of the overall empirical literature on how effective these techniques is that the answer is complex, which matches my own experiences. There is a lot of evidence that active learning etc approaches match what we know from cognitive psychology about how we learn best, but this is indirect evidence for superior learning occurring. Students on average also enjoy these techniques. This is also indirect evidence (but very relevant in its own right). More directly, studies show statistically significant improvements in level of learning with active approaches but the pedagogical significance is tougher to assess. A good recent metanalysis is Freeman et al They show one half standard deviation improvement which amounts to about 6 points out of 100 improvement (less on traditional exams, more on higher level learning concept inventories).  But there are a lot of issues with these studies (e.g. are more motivated teachers more likely to adopt active learning techniques but succeed primarily because of the motivation not the method – or are they likely to teach better because the change in technique is forcing new energy and attention to teaching regardless of technique).

My own experience with a partial commitment to such techniques in the BIO 100 course is that the students scored exactly the same average (and I mean to 2 significant digits) on the final exam as they did in the earlier version of the course. It was a a rewritten exam, and I would like to argue that it was testing further up the taxonomy. But this was not formally measured. And it wasn’t miles up the taxonomy (it was still multiple choice for goodness sake). My overall impression is that there is an improvement in “learning” (hard as that is to define and measure) but it is not stunning or even by obvious amounts (i.e. I would have to use statistics to tease apart the improvements) .Its certainly not like every student is suddenly moving up a grade (e.g. B to A) in class or anything. Freeman suggests 4-5 points on traditional exams which might be a B- to a B. This still sounds a little high compared to the experiences I know of but not outrageously high. But I am more confident (based on experience and literature) that students are enjoying things more, paying attention more, and probably developing a slightly more sophisticated understanding. And that is nothing to sneeze at.

My most recent personal experience with pedagogy reform

This year I abandoned powerpoint (except for occasional graphs and pictures) and did a lot of chalk boarding but in the end you would have to say they were “traditional” lecture classes (in fact really old school lectures without visual aids except the chalk board). But the students took lots of notes (no powerpoints to give). And I spent a lot of time asking and being asked questions (there were <20 students so everybody was involved). Indeed, despite always making dialogue during class a top priority, a lot more happened this year – somehow powerpoint seems to introduce a wall and turns the goal into finishing the slides instead of teaching/learning. I did some peer instruction and also just giving ecological math problems to do individually in class, but most of it was more in the vein of Socratic inquiry (i.e. teacher asking a question and getting multiple responses back). So I wasn’t following too many of the buzzwords, but it felt like a much improved class to me. Was this good pedagogy or bad? NB: I am officially combining point #4 with this experience to launch a new pedagogical movement called “PowerPoint is evil”. If this takes off, you heard it here first! But then again, its possible that getting rid of the powerpoint was just the disruptor (as mentioned above with clickers) that made me pay more attention to my teaching and five years from now adding PowerPoint back in will improve my teaching. (UPDATE – just found this lovely piece on teaching with chalk by Chris Buddle)

Point #9 – Class size – Thinking about class sizes above raises another big point – one that I’m sure administrations won’t like. But how much can pedagogy innovation do to fundamentally change learning (or lack thereof) in a classroom of 600 (or 300 or even 100) students? Teaching a class of 15 students effectively is pretty easy.Teaching a class of 300 effectively is impossible no matter what. The aforementioned meta-analysis by Freeman showed pretty clearly that active learning is most effective in classes with <50 students and decreases in effectiveness pretty quickly in larger classes. Is pedagogy improvement just a giant distraction from the real issue?

Conclusions

Overall, I think the emphasis on pedagogical methods is fantastic (and largely unprecedented in higher ed – most previous reform movements have focused on curricular reform). And I do think there is something real and of value in the active learning movement.But its not ginormous. And I also think we have gotten overly simplistic, reducing teaching to a one-dimensional bad (traditional) vs good (100% active learning) axis. The reality is that even the concept of learning is multidimensional (with the Bloom taxonomy being but a single dimension) and that pro-con trade-offs exist on all of these dimensions. This makes it impossible to to say what the “best” teaching method is without specifying the learning goal. In practice, I think we are better off to think of the traditional vs active/flipped axis as a dial we should tune depending on the situation and goals. And this dial has positions everywhere in between 0 and 100. And it is not one-dimensional it has multiple dimensions including 0-100% flipped, 0-100% just-in-time, 0-100% peer instruction, 0-100% inquiry based learning independent of each other and etc. And, although I haven’t fully worked it out for myself, I believe in some contexts breadth is a more important goal than higher taxonomy learning. We don’t have a set of best practices for breadth-oriented learning yet, but I wish we did.

One big thing I hope comes out of all of this is that we spend a lot more time in our departments and among colleagues having discussions about what our learning goals are (and no I don’t mean the kind my university requires me to list on my syllabus under the heading goals that are just lists of topics covered). I mean talking about how far up the taxonomy should a class go. What breadth is necessary and appropriate in this class to set up future years. Which classes are appropriate for different kinds of learning. Perhaps ecology and genetics should focus on high level learning and BIO 100 should focus on memorizing the phyla of life? Or maybe not? How important is a 6 point increase on an exam (and maybe half of that in a large class)? Would we be better off scrapping exams and lectures and active learning and putting them in hands-on labs? or taking ecology students out in the field to design their own experiments? Recall that there are finite resources so there are trade-offs and limits. How can we measure and assess whether we are succeeding? We need to start having discussions about pedagogical goals in departments. Logically that should proceed decisions about classroom pedagogical methods, but I’m not sure this is how things have happened.

Bottom bottom line – Modern pedagogy (=active learning/flipped class/etc) is not a silver bullet and it should not become the good end of a one-dimensional value judgement (flipped=good, not flipped=bad teaching). But these techniques definitely have some benefits. There are probably other issues we should be talking about equally much ranging from the simple like the declining art of notetaking to the difficult like class sizes. And maybe just mixing up our teaching approach periodically is more important than any specific technique. More broadly we need to think deeply and discuss regularly about our pedagogical goals, especially depth vs breadth, and the best ways to get there.

What are your experiences with the modern pedagogy movement? Has flipping classrooms become a bandwagon? Is this a good thing or a bad thing? Is there a breadth vs depth (=up the taxonomy) tradeoff? Should we ever choose breadth? Which of the techniques in point #1 do you think are most important?