What should high schoolers and undergrads learn about the scientific method?

Note from Jeremy: this is a guest post from Greg Crowther. Greg has a Ph.D. in biology and has held several teaching and research positions at the University of Washington and other Seattle-area colleges. He’s currently working on a master’s in science education.

***************************

I’ve never been inordinately curious about the natural world. As a kid, I did not spend long hours using a telescope or a home chemistry set, nor did I catch frogs in marshes or learn to identify species of local flora.  I got to high school, and then college, without any clear sense that I should become a scientist or that I would enjoy this particular vocation.

In my first four semesters of college, I took the usual variety of courses and grappled with their many fascinating questions.  Why did the Vietnam War start?  What do Buddhists really believe?  How did E.M. Forster’s novel Howards End illustrate his directive of “Only connect”?

Though fascinating, these questions also seemed horribly intractable.  One could cite evidence from a primary or secondary source to support one interpretation or another, but there didn’t seem to be any standard way of resolving disagreements besides deferring to the authority of the professor.

Science was different, though.  Professors presented the so-called “scientific method” as a fair, objective way of evaluating the strength of different possible explanations.  Accrue some background knowledge via reading and observation; pose a hypothesis; design an experiment to test the hypothesis; determine whether the data collected are consistent with the predictions of the hypothesis; and discard, modify, or retain the hypothesis as appropriate.

It all sounded so orderly, so sensible, so feasible.  Even if I did not have a great big hypothesis of my own, I could imagine taking someone else’s hypothesis out for a spin, say, using a species that hadn’t been studied yet.  This “scientific method” seemed simple enough for novices like me to follow, yet powerful enough to reveal fundamental insights about the world.  I was hooked – not on any particular molecule or technique or theory, but on the logical flow of the process itself.  I’ve considered myself a scientist ever since, and I now present the scientific method (often called the process of science) to my own students – because it’s relevant to their futures (whether or not they become scientists), but under-taught and poorly understood – more or less as it was presented to me.

“But wait!” cry various smart, articulate people such as Terry McGlynn and Brian McGill.  “That’s not how scientific research really works!”  Indeed, UC-Berkeley has an entire website, How Science Works, devoted to debunking and revising what it calls the “simplified linear scientific method.”

sciencerecipe

The “simplified linear scientific method,” according to UC-Berkeley’s How Science Works website, http://undsci.berkeley.edu/article/howscienceworks_01.

How Science Works has four principal objections to the “simplified linear” (SL) model:

The simplified, linear scientific method implies that scientific studies follow an unvarying, linear recipe.

But in reality, in their work, scientists engage in many different activities in many different sequences. Scientific investigations often involve repeating the same steps many times to account for new information and ideas.

The simplified, linear scientific method implies that science is done by individual scientists working through these steps in isolation.

But in reality, science depends on interactions within the scientific community. Different parts of the process of science may be carried out by different people at different times.

The simplified, linear scientific method implies that science has little room for creativity.

But in reality, the process of science is exciting, dynamic, and unpredictable. Science relies on creative people thinking outside the box!

The simplified, linear scientific method implies that science concludes.

But in reality, scientific conclusions are always revisable if warranted by the evidence. Scientific investigations are often ongoing, raising new questions even as old ones are answered.

To capture the complexity of science, including its iterative, social nature and its connections with society and technology, UC-Berkeley replaces the SL model with a sprawling four-panel illustration of How Science Works (HSW).  The illustration includes four interacting circles – Exploration & Discovery, Testing Ideas, Community Analysis & Feedback, and Benefits & Outcomes – each of which can be expanded and explored in more detail.

flowchart_noninteractive

UC-Berkeley’s alternative model of the process of science. http://undsci.berkeley.edu/images/us101/flowchart_noninteractive.gif 

So, is the 30-factor HSW model an improvement over the 5-step (or 4-step, or 7-step) SL one?

Well, it depends on one’s pedagogical goals, of course.

If your main goal is to capture the complexity of how science is practiced in the real world, then HSW is the model for you.

But what if you want to inspire confidence that science can provide solid answers to fundamental questions?  Which model more clearly conveys a sense of progress?  Or what if you wish to emphasize that truly scientific theories can be falsified (a la Karl Popper), or that good scientific theories lead to startling predictions of novel facts (a la Imre Lakatos), and you want to focus attention on hypothesis testing?  I vote for SL.

Yet another way to portray the process of science is exemplified by the Next-Generation Science Standards (NGSS) for K-12 education.  The NGSS share some of UC-Berkeley’s concerns about the SL model:

Students are told that there is “a scientific method,” typically presented as a fixed linear sequence of steps that students apply in a superficial or scripted way.  This approach often obscures or distorts the processes of inquiry as they are practiced by scientists.

NGSS’s solution is to present a set of eight key Science and Engineering Practices, as follows: ask questions; develop and use models; plan and conduct investigations; use and interpret data; use math and computational thinking; construct explanations; argue from evidence; and obtain, evaluate, and communicate information.

It’s a nice, compact set capturing the important things that professional scientists really do.  As with the HSW model, though, the NGSS model’s admirable realism obscures any clear sense of how scientific progress is made.  The Practices are not presented as a linear sequence because, in the real world, they are not executed in one particular order. Fair enough, but, as a result, there is no obvious thread of how hypotheses get formulated, get tested, and get rejected or retained.

My preference to keep “linear” hypothesis testing in the foreground is hardly unique.  While it is easy to dismiss this linear path as idealized or unrealistic, its real-world value is underscored by the movement toward preregistration of experiments, for example. One can also ask why we often report on studies as if they were conducted to test hypotheses that were actually thought of afterward (“HARKing”).  It seems that a lot of professional scientists want science to work this way, even if reality does not always cooperate.

I acknowledge the limitations of the SL model, of course.  As a graduate student and as a postdoc, I experienced plenty of nonlinearity and irrationality in my own work.  The old Albert Einstein quote comes to mind: “If we knew what we were doing, it wouldn’t be called research.”  And, as noted by both How Science Works and NGSS, the SL model can certainly be applied in superficial, unhelpful ways.  For my 10-year-old son’s science fair project last winter, he included a hypothesis and a prediction, as required by the rubric, but the two were unrelated!  Still, I believe that the SL model itself is “wrong” mainly in the sense that all models are simplifications and therefore incomplete at best.  I don’t see how this particular simplification of reality is any more misleading or egregious than any other textbook-level explanation we offer to our students.  On the contrary, I see it as a wonderful unifying framework that represents the scientific enterprise at its best.  Thus, my preference and my recommendation is to mostly teach from the SL model.  But, as always, I’m looking forward to your comments.

24 thoughts on “What should high schoolers and undergrads learn about the scientific method?

  1. In high school biology, I’ve normally taught the simple model of scientific method, and then I’ve let the students to plan and execute a very simple study based on their own interests. After this, we’ve discussed how their process of doing science related to the model of scientific method.

    The simple model has its benefits: when we’re discussing philosophical foundations of doing science and how doing science has been formulated as an acceptable way of acquiring new knowledge, it is very important. But when we get to socioscientific issues and how science actually works, it becomes more and more important to understand the real modus operandi of scientific research. And I find little sense in telling students how complex doing science can be as they can (and should!) learn it first-hand. Furthermore, the students’ idea of science equals quite usually to physical sciences – it’s also enlightening to compare these two fields of science and how their approaches and methods can be wildly differing, especially in high school, where physics experiments are quite often straight-forward and leave less room for… let’s say surprises.

    • This approach — talking about the simple linear model and then having students do their own study and compare their process to the simple linear model — sounds great to me.

  2. Why not forgo the algorithmic or piecemeal models altogether and use very simple descriptions such as, “The scientific process is about providing evidence-based explanations of phenomena?” This includes formulating hypotheses, testing them, collecting other observations and data, etc. I worry that an algorithmic approach (what I see in the SL model) can lead to grave misunderstandings about the whole process, and I also see your reservations about the more complicated presentations. I see this in a similar way to how mathematics is taught as a purely cold dead tool, rather than an exciting canvas on which to paint ideas. See, for instance, Lockhart’s Lament about this. https://www.maa.org/external_archive/devlin/LockhartsLament.pdf

    I see things like Bayesian philosophies of science as one possible alternative. Michael Strevens (and many many others) have written a lot about these sorts of approaches, although I think there is some work to be done in making them pedagogically useful (especially for those without an interest in science or mathematics to begin with). Essentially the idea that we update our existing beliefs according to evidence, to me, summarizes a lot of what rational science is about. This includes domain-experts, collaborations, and refutations of well-known theories I think, and can be presented at varying levels of detail to encompass these things. http://www.strevens.org/research/simplexuality/Bayes.pdf

    Perhaps I’m being a bit naive here. These are just some thoughts I had reading your excellent blog on the topic. 🙂

    • It’s interesting and apt that you use the word “algorithmic,” a good adjective for the SL model. Perhaps opinions on this vary according to how people feel about algorithms. For pessimists like me, who tend to doubt my and others’ ability to figure anything out, having an algorithm for advancing scientific knowledge is comforting and helpful — not a guarantee of success, but at least a reasonable, orderly approach to consider. However, some might think that reducing science to an algorithm makes it “cold” and “dead.” Maybe one teaching challenge, then, is to help students understand that any broad 4- to 7-step process leaves many things unspecified and thus requires considerable judgment and creativity on the part of the scientists.

      • More subtly, algorithms have specifiable conditions under which they work. And when they fail they do so in characteristic ways that vary from algorithm to algorithm. For instance, I read Andrew Gelman as identifying some of the “failure modes ” of the SL model. This provides pedagogical opportunities. Don’t just teach the algorithm, teach how it fails and the conditions under which it fails.

  3. Leaving out the initial step – make factual observations – seems to cause the confusion. Lots of science is there with asking questions (see something, poke it, did it change?) rather than further down the chain. Once one has a bulk of well quantified observations, one can begin to theorize how they fit together and then work through the whole falsification process.

    • I agree. I work with species in a small island where in most cases only the surface of bare identification has been scratched. But even so, and implied in you comment, I think, the way I go about gathering basic information is informed by eco and evo foundations, even if they are not explicitly trotted out all the time.

  4. I know for myself and I think for many others what was lost during these formative years in science was the root philosophy behind the entire process. What happened for me early in my career, and I see happen with so many others, is this notion of becoming married to ones ideas (i.e., hypotheses). I think people have a tendency to become so enamored with their thoughts that they often become cheerleaders- thus forgetting what science ought to be all about.

    The development of hypotheses, and especially null hypotheses is something that can never receive enough attention. Good science not only requires a good hypothesis, but an even better null. When science is done properly, we do not endeavor to confirm our ideas but instead labor to refute them. A demonstration of the null should not only be the ultimate goal of science, but it ought to reveal relationships and processes that were previously unknown. I see very few people conducting science this way anymore, and that concerns me deeply.

    So for what it’s worth, I’d say a heavy dose of these fundamental philosophical tenets are desperately needed during the early years of education in science.

  5. IMO the linear model is fine K-8, but after that I think its too shallow on its own. Many experiments (most?) don’t yield definitive conclusions. College-bound HS and BA/BS students definitely have the sophistication to grasp that and learn how to revise hypotheses and redesign experiments to work thru probs

    • Jim, thanks for this comment. I’d almost argue the opposite. That is, my recollections of K-8 and (limited) experiences with these students is that, while they can certainly memorize a sequence of steps presented as the scientific method, they usually lack the sophistication to know what it means to offer a falsifiable hypothesis, to design an experiment that will actually test that hypothesis, etc. To me, high school seems like about the right time to start delving into these issues in an authentic way, using real experiments and data. I vaguely remember testing the law of conservation of energy in 11th grade physics and getting a result that didn’t support the law and not being sure what to make of that, and also struggling in 12th grade AP biology to come up with a good testable hypothesis to explain the propeller-like structure of maple leaf seeds. If I found these things challenging to contemplate in high school, I question whether a typical middle schooler would be ready for them.

      • Hi Greg, well a close friend taught 6-8 science for many years and the linear model is basically the lab write up her kids did weekly (+methods), so I would expect g12 to go beyond and substantially so for AP courses.

      • If your friend is getting good “authentic” lab reports from her middle schoolers, then that proves that they can do it, and that they can then do even more in high school. Still, I’d ask whether your friend is unusual in teaching the scientific method so extensively and authentically. My guess would be yes, in which case the typical high schooler would be less capable due to not having had that great middle school experience.

      • The work of her students varies from xlnt to poor. Just the same students’ apparent ability is as much prep as actual ability. My recollection of intro undergrad geol courses as a TA (in a mid-level school) is that many students have a more sophisticated knowledge of science than the LM, but its not structured knowledge, its a patchwork from many sources.

      • While the observations I report below are purely anecdotal, they also seem fairly consistent based upon my experience. I’ve judged K-12 science fair competitions in rural communities for about a decade now. It seems as though the creative element of their respective projects steadily increases until about the 8th grade, and then appears to level off, if not decline a wee bit thereafter. Alternatively, the technical sophistication of their work seems to continue to increase with age.

        I have also noticed that students from about grades 6 through 8, on the whole, seem to be much more adherent to the linear scientific method than younger or older students. In fact, I am both amazed and impressed with their adherence to these standards, year in and year out.

        I was curious if you have observed similar trends in your professional pursuits, and if you have, what explanation you might offer for these apparent effects.

      • Elliot,

        Those are interesting trends but I haven’t observed them myself. My experience is almost exclusively with undergraduates, but I have a few close friends that teach middle school science, one biology and the other earth sciences.

        A while back though I took a job with an organization that offers 1-week camps for grade 5-8 students during spring and fall, where kids take courses and do group activities that often have an emphasis on science. The kids’ abilities (and behavior) fell across an extremely wide spread.

  6. One of the best ways of identifying what science is, is by contrasting it to what it is not. In my IntroBio courses starting in 2000, I spent the first week on epistemology, calling it “How do we know what we know?”. I carved out three ways of knowing the world:
    Esthetic–you just know something to be true instinctively/intuitively
    Authority–you are told something is true, and believe the teller
    Science–you discover the truth by contrasting the fit of hypotheses with observations that everyone can agree on

    This framework allows for lots of discussion early On and I find far more effective than those deadly dull “Sceintific Method” labs that often start an Intro course. For example
    –If you are looking for your car keys, you typically use the scientific method, unless of course, you ask your roommate where they are first.
    –Most of what students learn in a science class is, ironically enough, by authority in that they never make the observations themselves
    –Learning by authority is how most folks get their religious beliefs
    –Knowing thru Science celebrates finding you were wrong (progress!); the same discovery in knowing thru Authority is akin to betrayal
    –Certain kinds of Esthetics/Authority celebrate “Faith is belief in things unseen”; which is antithetical to science.

    The best part is that students seem to enjoy this. It is reasonably pluralistic (we don’t say authority is bad, science is good) but at the same time begin to show how different ways of knowing fit different parts of the human experience.

    MikeK

      • Not questions like that. But I do have students coming up after class and giving me examples of stuff they know, and asking which of the three is it? Is religion an esthetic if it occurred to you out of the blue? Can the Bible be scientific? Can I scientifically prove that Green Day is better than Oasis? So the best students probe the interstices using examples they know.

        One of the final exercises was a clicker question, asking which of the following was true:

        A) Men have one more rib than women
        B) Women have one more rib than man
        C) Men and women have an equal number of ribs

        The answers and resulting discussion are enhanced by the framework of esthetics, authority, science.

      • Ok, a GenEd survey course. That’s useful context. I agree that the sort of examples I’m suggesting wouldn’t be a great fit for a GenEd survey course.

  7. The biggest shortcoming of the SL model is the last one mentioned: “The simplified, linear scientific method implies that science concludes.” This makes science about the answers, rather than the process of getting the answers, which is what we want them to understand when we are teaching students about the scientific method. A relatively easy fix is simply to emphasize that the simple linear model is actually a simple *circular* model (SC?). The point is that once you get an answer, that is how you find the next question, often one you could not anticipate.

    I appreciate the notion of using different models to teach in different contexts to different audiences, but I will say for undergraduates, I do think SL (and perhaps even SC) is overly simplistic in most cases.

    • It’s interesting that you should single out that 4th objection. When I first read that on the UC-Berkeley site, I said to myself in mock horror, “Yes — God forbid that we ever actually ANSWER the question that we started with!?!” But that is “conclude” in the sense of “draw a conclusion.” You’re absolutely right that we don’t want students to think that science “concludes” in the sense of “ends” at the last step (which I guess is what UC-Berkeley was getting at). I like your fix of making the model “SC.”

  8. I prefer to teach a more expansive version of science. Science has multiple dimensions and many kinds of scientists, a diversity that is reflected in the multifarious methods of science. P.J. Medawar thought deeply about this. “Among scientists are collectors, classifiers, and compulsive tidyers-up; many are detectives by temperament and many are explorers; some are artists and others artisans. There are poet-scientists and philosopher-scientists and even a few mystics.” (Art of the Soluble, 1968 Medowar). He wrote, “a scientist, so far from being a man who never knowingly departs from the truth, is always telling stories… stories which might be about real life but which have to be tested very scrupulously to find out if indeed they are so. This is similar to Kenneth Boulding’s (Science: Our Common Heritage, Science, v207 n4433 p831-36 Feb 1980) vision that the principle function of the human mind is fantasy, and that science is a vigorous application of spinning yarns coupled with a constant bias for the disappointments of real world experience: testing. The simple linear model of science is embedded within these frameworks.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s