Poll: which big ideas in ecology were successful, and which were not? And what’s “success”, anyway?

Wanted to revisit a perennial topic of conversation around here: success and failure in ecology. And the surprisingly difficult task of distinguishing one from the other.

One reason scientific success or progress is sometimes difficult to identify is that scientific ideas have various desirable features that don’t always go hand-in-hand. So an idea can be successful in some ways but unsuccessful in others. Here are some ways in which a scientific idea might be considered successful:

  • Make correct predictions.
  • Make testable predictions, whether or not they’re correct. We learn something from testing predictions that don’t pan out as well as from ones that do.
  • Identify or discover a new possibility or phenomenon. For instance, Edward Lorenz’s discovery of chaos.
  • Explain a pattern or phenomenon.
  • Provide understanding or insight.
  • Unify seemingly-unrelated phenomena or special cases.
  • Ask new questions.
  • Ask better questions. For instance, taking an existing vague question and making it precise.
  • Focus research effort. Insofar as you think scientific progress requires lots of people working on the same problem (whether collaboratively or not), you’re going to want to see scientists focusing their efforts. Arguably, at least some of the credit for focusing research effort should go to the idea on which the effort is focused.
  • Be influential. Scientists tend to have a lot of respect for influential ideas (ideas that prompt a lot of work), even if those ideas turn out to be totally wrong and are eventually abandoned. Personally, I don’t share that respect, because I think that if influential-but-totally-wrong idea X hadn’t been proposed, the scientists who worked on it would’ve just worked on something else instead. And some of that work might’ve turned out to be based on correct ideas. So I don’t think being influential, independent of correctness or other desiderata, is a mark of a successful scientific idea. But I recognize that I’m probably in a minority on this.
  • Other possibilities I haven’t thought of.

(Aside: all of the above require elaboration. For instance, there’s such a thing as too much unification. Sometimes, “focus of research effort” is just a phrase for “bandwagon“. Etc. Those sorts of caveats are another reason why “success” isn’t always easy to identify in science. But I wanted to keep the post short so didn’t elaborate much.)

I’m interested in how different ecologists define “success” in ecology. So as a conversation starter, below is a little poll. For each of a number of different big ideas in ecology, you have to say if it was successful, unsuccessful, a mixed bag, or if it’s too soon to tell (there’s also an option for don’t know/not sure/no opinion). I also ask you to provide your career stage, since I’m curious whether junior and senior people differ in their evaluations.

Don’t read anything into my choice of ideas. I just picked some big ideas that I have opinions about, and on which I’m curious about others’ opinions. I tried to include a range of different sorts of ideas–verbal and mathematical ideas, older and newer ideas, etc.

For purposes of the poll, define “success” however you want. I’m betting we’ll get a pretty wide range of views on most of these ideas, in part because different people define “success” differently. Even though all of the ideas on my list are famous ideas, it’s not obvious that they’re all successful. For instance, you know what I think of the IDH. The hump-backed model of diversity-productivity relationships has been debated for forty years, which arguably isn’t a sign of success. A bunch of prominent ecologists think R* theory is unsuccessful while optimal foraging theory and metabolic theory are successful–but that’s a very debatable view. There are ecologists wondering if neutral theory has just been a distraction. The ideas of r/K selection and limiting similarity have come in for a lot of criticism over the years. Etc. So hopefully this poll will be a good conversation starter. In the comments, I encourage you to share why you voted as you did.

p.s. Note that calling an idea “unsuccessful” doesn’t imply anything negative about those who proposed or worked on the idea. Great scientists can have unsuccessful ideas.

Ask us anything, and we’ll answer!

Here it is again: ask us anything, and we’ll answer!

Got a question about ecology, academia, bird poop, or anything else we blog about? Ask us! Past questions have ranged from the statistical techniques every ecologist needs to know, to how to transition from postdoc to PI, to how much time we spend reading the literature, to how we’d fix the entire scientific funding system.

Submit your questions in the comments on this post, or tweet them to @DynamicEcology. You have a week to submit questions. Ask as many questions as you like. We’ll compile them and answer them in future posts just as soon as we get the chance.* Meg even promises to try to answer this time!**

*Patience is a virtue. :-)

**Though to be sure of that, I recommend baiting her with questions about hippo-sized Daphnia. Or Daphnia-sized hippos, whichever.

Friday links: Price = d’Alembert, the first null model war, and more (UPDATED)

Also this week: the US government vs. frequentist statistics, a survey on measuring excellence in scientific outreach, outlier sheep, the essential thought for anyone giving a talk, and more

From Meg:

Alex Bond had a post reflecting on what he’s learned in his 10 years in science. It’s spot on. (ht: Stephen Heard)

Sigh. There’s always one, isn’t there?:

ht: @McNameeJason

From Jeremy:

The EEB and Flow with a really interesting post on ecology’s first null model war (from the 1920s!)–and how it didn’t prevent the second.

Steven Frank with a new preprint linking the Price equation and Fisher’s fundamental theorem of natural selection to…[wait for it]…d’Alembert’s principle from classical physics (a generalization of Newton’s second law of motion, F=ma). I love this kind of stuff, identifying deep connections between seemingly-unrelated ideas. Here, the connection is that relative fitness is like relative motion (i.e. relative to a frame of reference). Interestingly, Frank seems (?) to be getting away from the idea that the Price equation has deep linkages to information theory, now preferring to think of the conservation of key quantities (e.g., total probability, mean relative fitness, the sum of direct and inertial forces) as what’s truly fundamental. (UPDATE: Just found this new preprint from Frank, linking his new d’Alembert-based perspective back to the information theory perspective. He does indeed now believe that conservation of total probability is the really fundamental thing here, and that MaxEnt is a “useful but sometimes unnatural” way to express the “geometric” constraints imposed by conservation of total probability.)

Advice on academic job hunting in the US for non-resident foreigners. I can’t vouch for it, but it sounds reasonable to me.

The Global Young Academy (an organization of top young scientists, with which my friend Rees Kassen is heavily involved) is conducting a survey on scientific engagement and outreach. They want to determine how universities and other scientific institutions measure and reward engagement and outreach, and how their employees and administrators think they measure and reward engagement and outreach. Click the link to take the survey; there are separate versions for profs/NGO scientists/administrators and students/postdocs.

Deborah Mayo and Andrew Gelman are rightly horrified by the way a US government webpage defines statistical significance and P-values.

Remember: when you give a talk, don’t be scared–you’re the smartest person in the room.

A rare retraction in ecology, due to an innocent mistake arising from the lead author’s serious illness. A very unusual and unfortunate situation; there’s no suggestion of misconduct or incompetence on the part of anyone involved.

Geez, the Google autocomplete suggestions for “my phd” are depressing.

Book review: Theory and Reality: An Introduction to Philosophy of Science by Peter Godfrey-Smith (UPDATED)

In a recent post on philosophy of science for ecologists, Brian identified Harvard philosopher Peter Godfrey-Smith’s Theory and reality: an introduction to the philosophy of science as promising-looking. I thought it looked promising too, so I read it (Kindle edition). Here’s my review. (UPDATE: another review here)

The book is based on introductory lectures in philosophy of science that Godfrey-Smith used to give at Stanford. It assumes no background knowledge of philosophy, and so is perfectly accessible to anyone reading this blog. But it’s aimed at people interested in philosophy, and takes that interest for granted. It doesn’t spend much time trying to argue you into an interest in philosophy you don’t already have. And it’s not aimed at teaching you just the bits of philosophy of science that you need to know in order to be a good scientist, or a better scientist than you are already. For instance, at various points it links philosophy of science to topics in epistemology and metaphysics that scientists have no particular reason to care about. So it’s not the philosophical equivalent of, say, an introductory biostats course or “math for ecologists” or whatever. Whether you find philosophy of science useful in your day-to-day scientific work is up to you.* But if you want to have a better sense of where philosophers of science are coming from, and be able to identify and understand those bits of philosophy of science that are relevant to you as a scientist, then I think you’ll find this book very helpful.

The first 2/3 of the book is a chronological survey of the most important work in philosophy of science from the early 20th century up until almost the present day, with a few nods to important earlier figures. It switches from chronological to topical organization to cover recent work. I think the chronological organization is effective. Features of later work that might otherwise seem puzzling make more sense when you know about the earlier work that later work was either trying to build on or improve upon. The book also includes a couple of chapters on fields on the boundary of philosophy of science (sociology of science, and feminist philosophy of science and “science studies”).

It’s a short and easy read—I knocked it off in a couple of days. It’s the philosophy of science equivalent of one of those bus tours that takes you to, say, Westminster Abbey, the Palace of Westminster, Tower Bridge, the Tower of London, St. Paul’s Cathedral, Stonehenge, and Bath all in one day. Yes, those bus tours only hit the obvious highlights, and yes they only give you a quick superficial glance at the sights you’re seeing, and yes they leave you wanting to go back for more. But they fill a real need. So for “Westminster Abbey, the Palace of Westminster…” read “the logical postivists, Hempel, Quine, Popper, Kuhn, Lakatos, Feyerabend, Laudan, Goodman,” plus various other figures discussed more briefly.

It’s an opinionated survey. Godfrey-Smith always tells you what he thinks of the ideas he discusses, and he uses the final chapters in part to lay out and stump for his own views. I welcomed this. I wouldn’t want a he-said, she-said survey that just describes what philosophers have written without any attempt at evaluation. Reading someone else’s evaluation helps me form my own evaluations, rather than getting in the way of me forming my own evaluations. Especially since Godfrey-Smith always gives a fair (sometimes generous) description before he launches into (often critical) evaluation. Nor does he skip any major ideas he disagrees with. At least, I didn’t notice any obvious gaps in the coverage, and I know enough philosophy that I’m fairly sure I would have. And Godfrey-Smith tells you when his own views are unorthodox, as opposed to when he’s voicing widely-shared opinions, so he never comes off as trying to railroad you towards his own views.

I broadly agree with Godfrey-Smith’s views, and I think most scientists will too. He’s a “naturalist”, which in this context means a philosopher of science who takes as his starting point how actual (rather than hypothetical or idealized) scientists go about their business. (Not that he thinks science as its currently practiced is above criticism, including philosophical criticism.) The only point where I seriously disagreed with him was his explication of subjective Bayesianism, which is surprisingly light on criticism. Godfrey-Smith strongly criticizes many other views, his overall judgement seems quite good, and he’s familiar with current everyday scientific practice. So I don’t understand how he could fail to strongly criticize a view that has been strongly criticized by various recent philosophers of science (e.g., Deborah Mayo), and that has never gotten any traction among practicing scientists or statisticians.** Especially because the subjective Bayesian view grew in part out of Carnap’s work in logical positivism, and Godfrey-Smith follows the rest of philosophy of science in writing off Carnap’s work as a dead end.

For those of you who worry that philosophy of science is remote from the actual practice of science, well, if your read this book you’ll discover that many philosophers of science worry about that too. As the book discusses at length, perhaps the biggest issue in philosophy of science ever since Thomas Kuhn’s Structure of Scientific Revolutions in 1962 has been figuring out the philosophical implications, if any, of how science was and is actually done. Those implications aren’t obvious. There’s an old philosophical dictum, dating back at least to Hume, that “is does not imply ought”. That is, descriptive and normative issues are two different things. How scientists do science, or how they’ve done it in the past, doesn’t on its own imply anything about how science should be done (especially since scientists themselves often disagree with one another on how to do science). But on the other hand, the descriptive and normative aren’t totally independent of one another either. As another old philosophical dictum goes, “ought implies can”. That is, any claim about how things should be presupposes that they could be that way. I have to say that I sometimes found Godfrey-Smith a bit unclear on the relationship between descriptive and normative claims in philosophy of science. Or maybe just a bit unclear on what claims are being made in the first place. In particular, some (not all) recent work at the interface of philosophy of science and other disciplines—sociology of science, “science studies”, feminist philosophy—arguably suffers from muddying descriptive and normative claims, and from lack of clarity about exactly what’s being claimed in the first place. Godfrey-Smith notes this, but his summaries of this work sometimes suffer a little from the same flaw, I think. In his admirable urge to take seriously recent work at the boundary of philosophy of science and other disciplines, I think he’s a bit less critical and demanding of that work than he should be. For instance, there’s a serious discussion of Bruno Latour, whom Andrew Gelman for one frequently mocks, and not without reason. I don’t think Godfrey-Smith should’ve mocked or omitted Latour and related figures. But I do think the chapters on sociology of science and feminism/science studies are a bit too uncritical and drift a bit too far away from philosophy of science sensu stricto. And Godfrey-Smith’s explication of his own naturalism wasn’t totally satisfying to me. I left the book with more of a sense of what his naturalism isn’t than what it is. Bill Wimsatt is one naturalist philosopher of science who’s good at linking how science is actually done to a normative account of how it should be done. For instance, Wimsatt emphasizes how human beings are cognitively limited in all sorts of ways. Many scientific practices involve heuristics and “rules of thumb” that would be suboptimal or even undesirable for a cognitively-unlimited being, but that are optimal given humans’ cognitive limitations. The scientific preference for “simple” or “parsimonious” models is a good example. As Godfrey-Smith notes, philosophers of science have tried mightily—and failed abysmally—to find a universal justification for preferring simple models. After all, the truth might be complicated. And there doesn’t seem to be any other desirable property (testability or whatever) that invariably increases with the simplicity of one’s model. But as Wimsatt (but not Godfrey-Smith) notes, real scientists’ preference for simple models doesn’t have the sort of universal justification philosophers traditionally have sought. Rather, a preference for simple models is justified in many contexts (not all!) for heuristic reasons, such as that real human beings just can’t wrap their heads around complex models, and that simple models often (not always!) provide a good-enough approximation to more complex models.

At the end of every chapter are suggestions for further reading, along with brief comments (e.g., identifying which of the readings are accessible and which are advanced and technical). This is very helpful. There’s also a glossary, which I didn’t really need since I’ve read some philosophy of science already, but which I imagine would be a godsend for someone totally new to the subject. And while we’re on the subject of vocabulary,Godfrey-Smith is good about alerting the reader to (and avoiding) loaded terms that get used in different ways by different philosophers.

The style is clear and readable. There are occasional jokes, often rueful apologies for using well-worn examples. And the book shows its origins as introductory undergraduate lectures in a good way. Godfrey-Smith is good at picking clear examples to illustrate broader points. And he’s good at picking examples that undermine your pre-philosophical intuitions, and so motivate you to stop and think about something that might’ve otherwise seemed obvious.

I took away from the book a better understanding of some philosophical topics that I previously hadn’t understood. For instance, going into the book I’d found Nelson Goodman’s notion of “grue” to be weirdly pointless. I stand corrected on that, at least in part. I now see the point of “grue”, and it was interesting to find that it’s actually scientifically relevant (although I still think Goodman did himself no favors by making his point with such a weird hypothetical***). And I really like Godfrey-Smith’s own resolution of the “grue problem”. I now have a better understanding of Kuhn, including the various tensions and ambiguities in Kuhn’s thought. And I have a better sense of the current lay of the land in philosophy of science.

I’d recommend the book to anyone who wants a quick accessible overview of philosophy of science, including scientists who want an overview so that they can then hone in on the bits of philosophy of science most relevant to their own work.

*At least in part. It might also depend on the state of your field—you might have no choice but to learn and do some philosophy of science. Godfrey-Smith discusses the possibility that philosophical issues loom large for practicing scientists only under certain circumstances, such as during Kuhnian “paradigm shifts”. I think this is right. Part of why I’m interested in philosophy of science is that I think philosophical issues loom larger in a young field like ecology than they do in, say, chemistry.

**“Bayesian” scientists and statisticians come in various stripes, but hardly any are subjective Bayesians in the sense Godfrey-Smith explicates, and most would find that sort of subjective Bayesianism totally bizarre. Andrew Gelman, for instance, is a self-described Bayesian, but is emphatically against the sort of subjective Bayesianism philosophers of science apparently have paid the most attention to.

***And I say that as someone who very much sees a place for ridiculous hypotheticals.

Can anyone help Jeremy specify an asymmetrical “distance” matrix for a partial redundancy analysis?

Trying something new for me: using the blog to get help with a technical issue.

Briefly, I did an microcosm metacommunity experiment in which inter-patch dispersal was experimentally controlled. So I know the rates at which species dispersed from any given patch (microcosm) to any other patch. Some of those rates were zero–not all pairs of patches were connected by dispersal. And dispersal was asymmetrical–the rate of dispersal from patch A to B wasn’t the same as the rate from B to A.

I’d like to code up up the dispersal rates in the form of a “distance” matrix. But the problem is, it wouldn’t be a metric distance matrix since it’d be asymmetrical. It’d just play the same role as a metric distance matrix in the analysis I have in mind. That analysis being a partial redundancy analysis, partitioning the variance in species abundances attributable to environmental variation from that attributable to “space” (here, dispersal). Basically, I want to do the same analysis as in Cottenie 2005 EcoLetts (and many subsequent papers by various authors), the difference being that the “distance” between any two of my patches isn’t determined by their geographic coordinates (since they don’t have any), it’s determined by the experimentally-imposed dispersal rates.

But I can’t figure out how to code up my asymmetrical “distance” matrix  in a form that works with the rda function in the vegan package in R. Note that I’m pretty sure partial redundancy analysis can be done with a non-metric distance matrix (right?), so I think my question here is just a matter of how to get R to do what I want it to do. But I’ve never done any sort of ordination besides PCA, and I only started reading up on redundancy analysis yesterday, and so maybe the problem is that I’m trying to do the impossible. Googling and searching Stack Overflow hasn’t helped, hence my resort to this post.

There’s a beer at the ESA* in it for you if you can help me out, whether in the comments or via email (jefox@ucalgary.ca). Thanks!

*Or equivalent reward

Different types of hands-on projects in a natural history course

In what seems to be becoming my annual post on help me think out loud about my fall teaching assignment (see last year’s post on community ecology classes), I am thinking about a field-oriented natural history course I’ll be teaching this fall and what assignments/evaluation tools I should use. Or more broadly, you hear a lot about best pedagogical approaches to classroom learning (including many great posts from Meg), but less about outdoor pedagogy. I think we all think since we’re ecologists this is obvious. Or maybe outdoor learning is so obviously active-learning, project-based, real-world etc which is what we’re trying to bring into classrooms that we don’t have to worry about it. But really, outdoor pedagogy is pretty much teach as we’ve been taught every bit as much as classrooms have been. I’ve increasingly been appreciating how much deep thinking is required to really get pedagogy right, and since I’m taking over a field course, I’ve been thinking a lot about my goals and how to align them with teaching and evaluation tools outdoors. I’d be really curious to hear your thoughts.

To make this concrete will still keeping this fairly generic, I am looking at a natural history course, the center-piece of which is multiple half-day field trips to a variety of ecosystems, and I am looking for an integrative project that spans the semester. I am considering three different projects:

  1. Do your own research/experiment – this is fairly typical in the OTS type model where you are spending a few weeks at a field station (I’ve taught such a course myself). Here you mentor students through the process of designing, executing and writing up a discrete piece of novel research. Pros – this teaches the scientific method and is fairly open-ended and clearly requires stretching their critical thinking skills and at least one form of writing. Cons – Many students aren’t really ready to do independent research as undergrads (especially lower level) and so often find this assignment more frustrating/intimidating than inspiring and in some cases do such low level work I’m not sure they learn much (or worse learn a very simplistic view of science), and its not particularly integrative (i.e. good at teaching scientific method, bad at helping students make connections and insights in natural history)
  2. Do a digital specimen collection – this is also a fairly typical assignment in “ology” classes (I did one myself in my graduate days in an entomology class). Since my class cuts across many taxa (requiring many types of collection equipment) I would probably have this be a digital collection instead of a physical collection where students take photos, put them into a document and annotate each photo with species ID, location, and notes about the species. Pros – this reinforces the goal of learning to ID species, paying attention while outdoors and seems likely to be retained as a tool useful to students after they graduate. Cons – less integrative than the other two choices, although this comes down a lot to what and how much I make them write in addition to the photos.
  3. Write a natural history journal – I haven’t encountered this one as much but a colleague suggested it. The assignment basics would be: 1) pick a small piece of land, 2) study it in depth from the soil to the sky, 3) make repeated visits, 4) write 5 pages about this location and its dynamics and interconnections in the spirit of Thoreau or McPhee. Pros – very integrative, very open-ended, a lot of emphasis on writing which is good (although like most biology courses we’re not really set up to do extensive mentoring on writing). Cons – pretty risky to expect students to observe and write like Thoreau.

There are some course-specific constraints in my own mind for my personal situation (although I think they’re not untypical of many teaching situations): this is lower division undergraduate (200-level), largish (44 students vs 2 instructors) course so more limited opportunities for mentorship than ideal. It is not a 2-week at a field station type of course so students will be doing this assignment very independently on their own time in the business of the semester (or not doing it until the last minute in some cases). The course is also literally focused on natural history, not principles of ecology or such (we have a separate ecology class). You can, of course, share your thoughts in the context of these constraints or I would be equally interested to hear your thoughts about the three options in your own context.

I personally have two main goals for this assignment: 1) is to be integrative. By integrative I mean they will already have lab exams on species ID and lecture exams on the stages of old-field succession etc. I really want something different that makes them think big picture, have ah-ha moments of connection and develop critical thinking and writing skills in addition to memorization. 2) is just to have fun and inspire. It is shocking how little time the average ecology student spends outdoors in their 4 years (forestry and wildlife do a little better than biology departments but still not great). This is likely to be their primary exposure to in-the-field until their 4th year for many students. I want them feel inspired by the awesomeness of nature that made us all go into the subject (while still being able to evaluate learning and give grades).

How important do you think these goals are? Do you think these assignments meet these goals? Any tips or gotchas you’ve learned the hard way on any of these projects?Other goals are imaginable and I’d be curious to hear them. Of course I’d be curious to hear other suggestions for assignments to meet these goals too.What do you see as the relative merits of these three projects? What do you think should be the primary pedagogical goals in a course that represents many students first exposure to the wonders of nature in a hands-on fashion? More broadly is pedagogy for outdoor teaching easy, or do we need to rethink this too?

 

Friday links: greatest syllabus ever, treemail, and more

Also this week: Andrew Gelman vs. the Wilcoxon signed ranks test, PLUTO, and more.

From Meg:

The Pluto Flyby happened this week, which made it an extra fun week to be on twitter. This BBC piece helps put things in perspective, as does this video from the NY Times. It’s mind-boggling that it went from a massively pixelated blob to this gorgeous image. One of my favorite images of the week has been this one, showing the reaction of NASA scientists to that image linked to in the previous sentence. The expressions on their faces are amazing! We’re also learning lots more about Pluto’s moons, including its biggest moon Charon. Overall, it’s been amazing to follow along. My daughter has been totally captivated, too. This article also has a good summary and a great slide show at the bottom showing images of Pluto through time.

The city of Melbourne, Australia, gave email addresses to trees so that people could report problems such as hanging branches. Instead, people are using the email addresses for all sorts of other purposes, and sometimes the trees reply. Wonderful! :) (ht: @catherineq)

From Jeremy:

Andrew Gelman says you shouldn’t do the Wilcoxon test. Interestingly, he also doesn’t like what I’d think would be the most obvious alternative, a randomization test. The post and comments have an interesting discussion of the pluses and minuses of different alternatives here.

Sticking with Gelman, he passes on the news that there really is a “hot hand” in sports. Of interest to me because I use the example of the hot hand in introductory biostatistics. I may need to revise my lectures in light of the latest results. But first I need to make sure I’ve fully grasped the very interesting-sounding little probability paradox at the core of the latest research on this.

Gelman also passes on news that economists, like psychologists, can now bet real money on whether various experimental studies published in leading economics journals will replicate. Sounds fun and potentially informative. I’d think it would be reassuring if, on the whole, people working in a field are able to predict which results will replicate. In psychology, a commenter on Gelman’s blog reports making money by betting against replicability of small sample studies, counterintuitive studies, and social psychology. A betting market also seems like an improvement over one-on-one bets between opponents, since it can be hard for opponents to agree terms.

Princeton economist Uwe Reinhardt with the greatest syllabus ever. It begins as follows:

After the near‐collapse of the world’s financial system has shown that we economists really do not know how the world works, I am much too embarrassed to teach economics anymore, which I have done for many years. I will teach Modern Korean Drama instead.

Although I have never been to Korea, I have watched Korean drama on a daily basis for over six years now. Therefore I can justly consider myself an expert in that subject.

Click through for the notes from the first lecture. I’m still trying to decide whether it’s completely a joke, or whether he’s going to use Korean drama as a fun way to teach economics.

And finally, an impending increase in realized fitness. :-)

Do we need to teach our students about the scientists as well as the science?

Earlier this year, I needed to cover Intro Bio for a colleague on very short notice due to an emergency. I needed to give two lectures, back-to-back, first thing Monday morning. When I got the email on Sunday, I pulled up the syllabus to see what was on schedule for the week. One lecture was on populations and, while there wasn’t time to prep the students for my preferred population lecture oriented around Pablo Escobar’s hippos, it was straightforward enough for me to work from the existing slides. The second lecture on the schedule was a community ecology lecture. When I pulled up the slides, I saw a lot of emphasis on details that I prefer not to cover — including on people like Clements vs. Gleason. I was a little surprised to realize that, while I taught very similar material just a few years ago, now I felt like it was going to be really hard to cover that, since I now feel pretty strongly that freshman don’t need to know anything about Clements or Gleason.* Apparently I have fully embraced the less-is-more approach to teaching. Instead, I decided to skip ahead one lecture to the behavioral ecology lecture (because, hey, who doesn’t love a class filled with lots of behavior videos?)

This had me wondering, though, about when it’s important to teach students about the scientists as well as their science – and about how that changes as we move from first- and second-year undergrads to upper-level undergrads to graduate students. Ecologists tend to use peoples’ names as shorthand for their ideas – a Tilmanesque view on competition, a Hutchinsonian niche, etc. At what level is it important for students to know that? And who should they know?

In my opinion, Intro Bio students don’t really need to know any names beyond Darwin’s (and they already know his coming in). I care much, much more about whether they know the concepts, and worry that trying to learn the names will distract them. I will mention the names of people who did the studies, and I cover classic experiments by Tilman, Connell, Paine, and others – but I try to use recent studies just as often, in part because I have a goal of using examples from diverse scientists. And I never identify an experiment or concept only by naming the scientist associated with it. Instead, I would say something like “In the experiment Connell did looking at competition between barnacles in the intertidal…” to set up a question on the topic.

I certainly expect grad students to be more familiar with people, and especially to be familiar with people whose work is related to theirs. I don’t currently teach a grad-level course, but when I did in the past, I spent more time on people than I do in my Intro Bio course. And, at qualifying exams, I think some amount of “name game” style questions are fine.

All of this has me wondering: who do you think students need to know? And at what level should they know them?

My original plan was to have a poll here, with a list of names and three columns where you could check if you think first year undergrads should know a person, last year undergrads should, and/or if grad students should. But I couldn’t figure out how to set up such a poll, plus the list quickly got too long. So, instead, I’ll just put a list of names below, and then ask for thoughts in the comments about how much you expect students at different levels to know names, and which names you think are the key ones.

And now, the list, compiled with suggestions from twitter:**

Anderson, Andrewartha & Birch, Brooks & Dodson, Carson, S. Carpenter, Clements, Connell, Cowles, Coyne, Darwin, Dobson, Dobzhansky, Earle, Elton, Endler, Estes, Fairbairn, Fisher, Forbes, Gleason, Goodall, P&R Grant, Grinnell, Haldane, Hamilton, Hanski, Hoekstra, Holling, Holt, HSS, Hubbell, Hudson, Huffaker, Hutchinson, Janzen, Kimura, Kuhn, Lack, Leibold, Lenski, Leopold, Leslie, S. Levin, R. Levins, Lewontin, Likens, Losos, Lotka & Volterra, Lubchenco, MacArthur, Margulis, May, Mayr, Menge, Muir, E&H Odum, Ohta, Paine, Park, Pfennig, Power, Queller, Reznick, Ricklefs, Roughgarden, Schindler, Schoener, Simberloff, John Maynard Smith, Stearns, Stebbins, Strassmann, Tansley, Tilman, Tinbergen, Trivers, Wallace, Watson & Crick (& Franklin!), West-Eberhard, Whittaker, George Williams, E.O. Wilson, Wright, Zuk

Can we pause for a moment to note how amazingly undiverse that list is? Sigh. Moving on…

This leads to a larger question of what historical information is valuable. Why is Connell’s barnacle experiment more-or-less required content in ecology courses? Can you be a competent ecologist without being familiar with the classic experiments? Is teaching about the voyage of the Beagle essential to understanding evolution by natural selection? Alex Bond argues that the narrative that these stories provide is really helpful for teaching, and I agree with that. In fact, the ability to tell the interesting behind-the-scenes stories about classic and recent discoveries – which students find really engaging – is one of the reasons why we have moved to a format where I teach the ecology portion of the Intro Bio course to both sections, and my colleague Trisha Wittkopp teaches the genetics portion to both sections. (We split evolution. In the past, one instructor taught the whole semester, but just to one of the two sections.)

But the stories don’t have to be about the classics (and those classics tend to be overwhelmingly by white men, so, again, my preference is to include more recent work, too). And focusing on the names can make students miss the big picture. So, for Intro Bio, I will continue to try to emphasize the concepts rather than the people.

 

*Jeremy’s recent post argues that Clements’ notion of communities as superorganisms is one of the biggest ideas ecologists have rejected. There’s definitely an interesting post in the question of whether (and at what level) we should continue to teach these sorts of ideas.

**I’m sure some grad students will use this as a checklist for studying for qualifying exams – check the comments for people I missed!

Big data doesn’t mean biggest possible data

For better and for worse, big data has reached and is thoroughly permeating ecology. Unlike many, I actually think this is a good thing (not to the degree it replaces other things but to the degree to which it becomes an another tool in our kit).

But I fear there is a persistent myth that will cripple this new tool – that more data is always better. This myth may exist because “big” is in the name of the technique. Or it may be an innate human trait (especially in America) to value bigger house, car, etc. Or maybe in science we are always trying to find simple metrics to know where we rank in the pecking order (e.g. impact factors), and the only real metric we have to rank an experiment is its sample size.

And there is a certain logic to this. You will often hear that the point of big data is that “all the errors cancel each other out”. This goes back to statistics 101. The standard error (description of the variance in our estimate of the mean of a population) is \sigma/\sqrt{n}. Since n (sample size) is in the denominator the “error” just gets smaller and smaller as n gets bigger. And p-values get corresponding closer to zero which is the real goal. Right?

Well, not really. First \sigma (standard deviation of noise) is in the numerator. If all data were created equally, \sigma shouldn’t change too much as we add data. But in reality there is a lognormal-like aspect to data quality. A few very high quality data sets and many low quality data sets (I just made this law up but I expect most of you will agree with it). And even if we’re not going from better to worse data sets, we are almost certainly going from more comparable (e.g. same organisms, nearby locations) to less comparable organisms. The fact that noise in ecology is reddened (variance goes up without limit as temporal and spatial extent increase) is a law (and it almost certainly carries over to increasingly divergent taxa although I don’t know of a study of this). So as we add data we’re actually adding lower quality and/or more divergent data sets with larger and larger \sigma. So \sigma/\sqrt{n} can easily go up as we add data.

But that is the least of the problems. First, estimating effect size (difference in means or slopes) is often only one task. What if we care about r2 or RMSE (my favorite measures of prediction. These have sigma in the denominator and numerator respectively so the metrics only get worse as variance increases.

And then there is the hardest to fix problem of all – what if adding bad datasets adds bias. Its not too hard to imagine how this occurs. Observer effects is a big one.

So more data definitely does NOT mean a better analysis. It means including datasets that are lower quality and more divergent and hence noisier and problably more biased.

And this is all just within the framework of statistical sampling theory. There are plenty of other problems too. Denser data (in space or time) often means worse autocorrelation. And another problem. At a minimum less observation effort produces smaller counts (of species, individuals, or whatever). Most people know to correct for this crudely by dividing by effort (e.g. CPUE is catch per unit effort). But what if the observation is non-linear (e.g. increasing decelerating function of effort as it often is). Then dividing observed by effort will inappropriately downweight all of those high effort datasets. Another closely related issue that relates to non-linearity is scale. It is extremely common in meta-analyses and macroecology analyses to lump together studies at very different scales. Is this really wise given that we know patterns and processes often change with scale. Isn’t this likely to be a massive introduction of noise?

And it goes beyond statistical framing to inferential framing to what I think of as the depth of the data. What if we want to know about the distribution of a species. It seems pretty obvious that measuring the abundance of that species at many points across its range would be the most informative (since we know abundance varies by orders of magnitude across a range within a species). But that’s a lot of work. Instead, we have lots of datasets that only measure occupancy. But even that is quite a bit of work. We can just do a query on museum records and download often 100s of presence records in 15 minutes.But now we’re letting data quantity drive the question. If we really want to know where a species is and is not found, measuring both sides of what we’re interested in is a far superior approach (and no amount of magic statistics will fix that). The same issues occur with species richness. If we’re really serious about comparing species richness (a good example of that aforementioned case where the response to effort is non-linear), we need abundances to rarify. But boatloads of papers don’t report abundances, just richness. Should we really throw them all away in our analyses?

As a side note, a recurring theme in this post and many previous ones is that complex, magic statistical methods will NOT fix all the shortcomings of the data. They cannot. Nothing can extract information that isn’t there or reduce noise that is built in.

So, returning to the question of two paragraphs ago, should I knowingly leave data on the table and out of the analysis? The trend has been to never say no to a datset. To paraphrase a quote from Will Rogers, “I never met a dataset I didnt’ like”. But is this the right trend? I am of course suggesting it is not. I think we would be better off if we only used high quality datasets that are directly relevant to and support the necessary analytical techniques for our question. Which datasets should we be omitting? I cannot tell you of course. You have to think it through in the particulars. But things like sampling quality (e.g. amount of noise, quality control of observation protocols), getting data that make apples to apples comparisons, and the depth of the data (e.g. abundance vs occupancy vs presence/absence) may well place you in a realm where less is more!

What do you think? Have you had a situation where you turned away data?

Friday Links: how to feel less guilty about time off, time associate profs spend on service, and more!

Also: the webcam showing bears fishing for salmon at Brooks Falls is back, the myth that academic science isn’t biased against women, ways to fail at your job search (or, ideally, things to avoid doing!), and being an academic “other”.

From Meg:

This piece has good suggestions on how to tackle guilt related to taking some time off. This is still something I’m working on!

The Brooks Falls salmon/bear cam is back! Sorry for killing your productivity.

This piece has depressing numbers on the representation of minorities in academia, and argues that higher service burdens on underrepresented minorities contributes to this. It includes this really interesting graphic, showing that white men perform the least service, while women of color perform the most:

service

On a similar topic, I found this response to the recent Williams & Ceci PNAS paper informative. (The W&C paper argued that STEM hiring is biased in favor of women.) The response lays out 5 major problems with the Williams & Ceci study. Well worth a read!

This post from Acclimatrix at Tenure, She Wrote has great suggestions for things you may be doing wrong in job applications or interviews. I really agree with her advice to come up with ideas for local work (or work that takes advantage of local resources/strengths). And I’ve been amazed at how many job candidates act like their interactions with grad students don’t matter. Don’t do that (ideally because you are actually interested in talking to and learning from the grad students!)

Also over at Tenure, She Wrote, there was a powerful guest post from NotYourOther on her experiences as an underrepresented minority in academia. It’s a really powerful post on the challenges and responsibilities of being an “other”, and is also well worth a read.

Jeremy is away this week, so no links from him. And apparently Brian couldn’t top his brony link. ;)