Friday links: Brian wins the linkfest forever. Also, academic whack-a-mole, and more.

There are other links this week, but really the only one you need to click is Brian’s. Greatest. Link. Ever. Seriously, drop whatever you’re doing and click through NOW.

From Meg:

Here’s an argument for giving women faculty a bonus to offset the gender bias that occurs in student evaluations.

I enjoyed this piece by Nathan Hall, who runs the Shit Academics Say (@academicssay) twitter account. I really enjoy the account, and it was neat to read the backstory.

On twitter, Anne Hilborn (@annewhilborn) asked:

I said I would add dialogue to my paper and then have Nina Totenberg read it. Because, as Anne said,

Tweets are now coming in with the #SciPaperNarrator hashtag.

This is a great post on time management in academia, how easy it is to get bogged down in the tedious little things, and how important it is to avoid that. Best yet, it does it with an extended analogy to whack-a-mole. Read it! Here’s one part:

My assumption as a young assistant professor was that if I whacked all the moles fast enough, I would run out of moles and have time to do research. I have since learned the error of my ways and I will let you in on the secret they will never, ever tell you at faculty orientation: there are always more moles. In fact, I am convinced that there are an infinite number of moles. You could spend every waking hour whacking moles and still never run out of moles. Honestly.

Indeed. Also read the comment from Odyssey, who points out:

You have the right approach, although I would go one step further. Not all moles are equal. Try to focus on whacking only the moles worth the most points.

Good advice. (ht: Ethan White)

From Jeremy:

Data Colada with some straight talk on reducing fraud in science, with which I agree. And with which most of Twitter disagrees, as far as I can tell. To give you the flavor, here’s an extended quote, pushing back against the notion that we need to change the incentives scientists face in order to prevent fraud:

First, even if rewarding sexy-results caused fraud, it does not follow we should stop rewarding sexy-results. We should pit costs vs benefits. Asking questions with the most upside is beneficial.

Second, if we started rewarding unsexy stuff, a likely consequence is fabricateurs continuing to fake, now just unsexy stuff.  Fabricateurs want the lifestyle of successful scientists. Changing incentives involves making our lifestyle less appealing. (Finally, a benefit to committee meetings).

Third, the evidence for “liking sexy→fraud” is just not there. Like real research, most fake research is not sexy…That we attend to and remember the sexy fake studies is diagnostic of what we pay attention to, not what causes fraud…

The evidence that incentives causes fraud comes primarily from self-reports, with fabricateurs saying “the incentives made me do it” …To me, the guilty saying “it’s not my fault” seems like weak evidence. What else could they say?

Similarly weak, to me, is the observation that fraud is more prevalent in top journals; we find fraud where we look for it. Fabricateurs faking articles that don’t get read don’t get caught…

Rather than change incentives, they suggest that authors be obliged to post data, materials, and code, and keep a paper trail verifying that they did the claimed work.

Tim Poisot asks: when do scientific debates get vicious? I think his answer (lack of data) is at best incomplete. If only because lack of data seems to be a poor explanation for how scientific debates even get started in the first place. Semi-related: name the most productive and unproductive debates in the history of ecology. And why I love a good argument, and what makes an argument good.

Banning bottled water sales at the University of Vermont had the opposite of its intended effects. Good fodder for an environmental policy class.

Serious, but honest, scientific errors of famous scientists.

Way too early to get your hopes up, but Disney is thinking of making an adventure movie about Charles Darwin. Here’s the trailer (I assume). Dana Carvey is a surprising but effective choice for the lead. :-) (ht Rich Lenski, via Twitter)

From Brian:

Did you wonder why the hotels in Baltimore for ESA were so booked up on Saturday and Sunday night? We’ll have some company at the convention center on Sunday. Get your costumes ready now (HT Jes Hines)

(meme credit: Jeremy)

Jeremy adds: the ecological implications of Brian’s link are thoroughly discussed in this video. Which provides some, um, novel suggestions for Meg’s “organism of the day” project. :-)

How valuable are lab tours for getting undergraduates excited about STEM?

This summer, I’ve been involved in several efforts to recruit students to STEM. For one of these events, I have a total of three hours to introduce incoming freshman to Biology at Michigan (in all its varied forms), to try to make them feel more connected to campus in general and biologists on campus in particular, and, ideally, to get them to consider working in a lab next year. That’s a tall order!

For the first hour, I plan on having a few people give talks that highlight some of the breadth of biology-related research that is done on campus – while realizing that it is hopeless to really try to span that breadth with just four talks, and biasing things a bit towards applied topics, which I think incoming students tend to find more engaging. For the second hour, I plan to have a panel made up of current students. We’ll also have two Program in Biology/Neuroscience Advising Coordinators there, since they will be best equipped to answer questions about the nuts-and-bolts of the different major options.

For the third hour, though, I’m torn. The model that the other departments will use will be to do lab tours during this hour. And that was initially my plan. But then I started to wonder if something focused more on Grand Challenges in Biology* might be more fun and a better use of time. Part of the issue is simply the logistic one of the biologists being spread out across several buildings, so there would be as much time spent walking between labs as there would be touring the labs. Another part of the issue is that I feel like it gives a biased view of biology in general and ecology in particular – watching someone work at a computer isn’t very exciting, so we tend to skip the computational labs, and it isn’t possible to take people to the field during a 10 minute tour. It’s also possible I have physics envy after hearing the physicists talk about which labs have the best lasers for tours!**

This all has me wondering: has anyone studied the effect of short lab tours on recruitment efforts, comparing them to other potential activities? Are they effective? If anyone knows of such studies, I would love it if you posted a link in the comments!

Anecdotally, students sometimes do seem really excited by the tours. I know for other programs that have done something similar, the lab tours were listed as a highlight by some students at the end of the event. But how many of them would have been engaged by sitting down in a small group with faculty and talking about the most exciting areas of current research? And what makes for an engaging lab tour? Presumably some depends on the person leading the tour, but I’m guessing the topic of research in the lab matters, too, as does the presence of fancy equipment (or at least something fun — we stored our kayak in the lab at Georgia Tech, and visitors always loved seeing that).

For my part, I never did one of these short lab tours as a student. I did have the very good fortune of being on a tour of Cold Spring Harbor Labs with just a few other students, when I was a senior in high school. It was a pretty long tour, and really exciting.*** I also remember, when I started at Cornell, doing a freshman research experience that involved spending one evening in a plant biology lab, attempting to give plants tumors. But that was 2-3 hours on its own, not a simple 10 minute tour. And, while I thought it was interesting, I’m not sure it had much of an effect on me beyond me getting a plant for my dorm room.

Over the years, I’ve given several short tours of my lab to various groups of students. I show them some Daphnia, talk about what we do and why it’s important, and answer a few questions. But I never feel like it’s that engaging. Perhaps that’s a sign that I’m doing it wrong? I may be biased by lab outreach things we do where we have students sample lakes and look at plankton — those are really fun and exciting, but also generally with a younger group of students, and right on the dock of a lake.

If given a choice between short lab tours, grand challenges in biology, or something else, what would you choose? Which do you think is most effective at engaging undergraduates? And, again, if anyone knows of research on this area, I would love to hear about it! And, if you’ve given these tours or taken them: do you think they’re effective? What do you think makes them more likely to be effective?


* I’ll have a follow up post on this general topic next week. I recently did an event with this format, and thought it worked really well. The challenges I talked about were linking genotype to phenotype (and figuring out how environment influences that link) and about understanding biodiversity.

** Then again, one of the speakers in the first hour, Micaela Martinez-Bakker, will talk about how ecology can inform disease eradication efforts, so perhaps we can just use this recent xkcd to recruit people to biology!

*** We were supposed to meet with Watson, too, but he was at a funeral. We did get to sit in his office, though, which I thought was pretty cool.


Postscript: After writing this post, I had a different meeting where we were discussing plans for a recruitment weekend for grad students. The topic of whether to do lab tours came up, which I found amusing. Once again, there doesn’t seem to be a consensus on whether they’re valuable. In this case, the issues are a little different — we aren’t trying to get the students interested in EEB or Biology, but rather to convince them that Michigan is a good place for them to be a graduate student. And they’ve almost all already worked in a lab, so there isn’t the gee-whiz-I’m-in-a-lab factor. But we still tend to feature lab tours with these things. So, I will extend my questions above to ask if anyone knows of literature related to grad-recruitment, and what practices are most effective (including for students from underrepresented groups).

Steering the trait bandwagon

Although the notion “bandwagon” technically only means something that is rapidly growing in popularity, calling a scientific research program a bandwagon  carries several more connotations. These include the idea that it will crash (people will abandon it) and that people are piling in because they perceive the research program as a way to do something that is “easy” (or even formulaic) but still get in a good journal (i.e. the proverbial something for nothing). Popular and easy are of course two of the worst reasons to choose a research project, but that seems not to matter in the bandwagon phenomenon.

There is little doubt that functional traits are a bandwagon research program right now:

Papers using "functional trait*"

Papers using “functional trait*” per year

The use of the phrase “functional trait*” (per Web of Science) is rising exponentially with a doubling time of less than 4 years. In less than two decades, there are almost 3000 total publications cited 56000 times, 14000 times last year alone (with an astonishing average citation rate of 19 times/article and an h-index for the field of over 100).

For better and worse, I am probably one of a fairly large group of people responsible for this bandwagon due to this paper which came out simultaneously with a couple of other papers arguing for a trait based approach, although (as likely true of all bandwagons)  the idea has been around much longer and builds on the research of many people.

By calling functional trait research a bandwagon, I am implying (and now making explicit) two things: 1) The popularity of the functional trait program is in part due to the fact that people see it as a simple way to do something trendy. I think there is no doubt of this – there are a lot of papers being published right now that just measure a bunch of functional traits on a community or guild and don’t do much more. 2) That this party is about to come to an end. I predict we will see multiple papers in the next two years talking about how functional trait research is problematic and has not delivered on its promise and many people bailing out on the program.

You might think I am worried about the impending crash, but I am not. I actually relish it. Its after the bandwagon crashes that we lose all the people just looking for a quick paper and the people who are really serious about the research field stay, take the lessons learned (and identify what they are), build a less simple, more complex but more realistic, productive world view. In my own career I have seen this with phylogenetic systematics, neutral theory of biodiversity, and – if we go back to my undergraduate days – neutral theory of genetics and island biogeography.

In an attempt to shorten the painful period and hasten the renewal, what follows are my ideas/opinions about what is being ignored right now on the functional trait bandwagon (although by no means ignored by the researchers I expect will still hang around after the crash and I have tried to give citations where possible), which I predict will become part of the new, more complex view of functional traits version 2.0 in 5-10 years down the road.

(As an aside – I wanted to briefly note as a meta comment on how I think science proceeds, that: a) I think probably many other people are thinking these thoughts right now – they’re in the air, but as far as I know nobody has put them down as a group in ink (or electrons) yet and b) my own thinking on this has been deeply influenced by at least a dozen people and especially by Julie Messier as well as Brian Enquist & Marty Lechowicz – more full acknowledgements are at the bottom c) its not as easy to assign authorship on these thought pieces as it is on a concrete piece of experiment or analysis – if this were a paper I could easily argue for just myself as author or 1 more or 3 more or 10 more)

So without further ado, here are 9 things I think we need to change to steer the bandwagon:

  1. What is a trait? – there are a lot of definitions (both the papers linked to above have them). But the two key aspects are: 1) measured from a single individual and 2) conceivably linked to function or performance (e.g fitness or a component such as growth rate). The 2nd is not a high bar to clear. But a lot of people right now are ignoring #1 by taking values that can only be tied to a species or population (such as population growth rate, geographic range size, mortality rates) and calling them functional traits. They’re not. They’re important and interesting and maybe science will someday decide they’re more important than things you can measure on individuals. But they’re not functional traits if you can’t measure it on one individual. The functional trait program is going from function (behavior and physiology) to communities or ecosystem properties. Its where a lot of the excitement and power of the idea comes from. It is actually in a subtle way a rejection of the population approach that dominated ecology for decades.
  2. Where’s the variance? – I believe that the first step in any domain of science is to know at what scales and levels of measurement variation occurs. Only then can you know what needs to be explained. There has been an implicit assumption for a long time that most of the variance in functional traits is between species and/or along environmental gradients. There is indeed variation at these two levels. But there is also an enormous amount of variation between individuals in the same species (even population). And there is way more variation between members of a community than between communities along a gradient. Finally, although the previous statements are reasonably general, the exact structure of this variance partitioning depends heavily on the trait measured.  Functional traits won’t deliver as a field until we all get our head around these last three facts. And learn a lot more than we already know about where the variance is. A good intro to this topic is Messier et al 2010 and Viollet et al 2012 (warning I’m a coauthor on both).
  3. Traits are hierarchical (can be placed on scale from low level to high level) – we tend to lump all traits together, but traits are hierarchical. Some are very low level (e.g. chlorophyl concentration per leaf volume), one level up (e.g. light absorption), and going on up the ladder from this one trait we have Amax (maximum photosynthetic rate), leaf CO2 fixation/time, CUE (or carbon use efficiency or assimilation over assimilation+respiration), plant growth rate, and fitness. Note that each trait directly depends on the trait listed before it, but also on many other traits not listed in this sequence. Thus traits are really organized in an inverted tree and traits can be identified at any tip or node and performance sits at the top of the tree. We move from very physiological to very fitness oriented as we move up the tree. One level is not more important than the other but the idea of different levels and being closer to physiology or closer to fitness/performance is very real and needs to be accounted for. And we need to pick the right level for the question. All traits are not equivalent in how we should think about them! And learning how to link these levels together is vital. A depressing fact in phenotypic evolution is that the higher up the hierarchy a phenotypic character is, the less heritable it is (with fitness being barely heritable), but so far we seem to be having the opposite luck with functional traits – higher level traits covary more with environment than low level traits (there are a lot of good reasons for this). A good intro paper to this topic is Marks 2007.
  4. Traits aren’t univariate and they’re not just reflections of 1-D trade-offs – How many papers have you seen where trait #1 is correlated with environment. Then trait #2 is correlated with environment, and etc.? This is WRONG! Traits are part of a complex set of interactions. If you’re a geneticist you call this epistasis and pleiotropy. If you’re a physiologist you call this allocation decisions (of resources). If you are a phenotypic evolution person you call this the phenotypic covariance matrix. Of course we are finding that one trait low in the hierarchy is neither predictive of overall performance nor strongly correlated with environment. It is part of an intricate web – you have to know more about the web. The main response to this has been to identify trade-off axes. The most famous is the leaf economic spectrum  (LES) which basically an r-K like trade-off between leaf life span and rate of photosynthesis. Any number of traits are correlated with this trade-off (e.g. high nitrogen concentrations are correlated with the fast photosynthesis, short life end). And several of the smartest thinkers in traits (e.g. Westoby and Laughlin) have suggested that we will find a handful of clear trade-off axes. I hate to contradict these bright people, but I am increasingly thinking that even the idea of multiple trade-off axes is flawed. First the correlations of traits with the LES are surprisingly weak (typically 0.2-0.4). Second, I increasingly suspect the LES is not general across all scales. And the search for other spectra have gone poorly. For example, despite efforts, there has not yet emerged a clear wood economic spectrum that I can understand and explain. So to truly deal with traits we need to throw away univariate and even trade-off axes and start dealing with the full complexity of covariance matrices. This is complex and unfortunate, but it has profound implications. Even the question of maintenance of variation simplifies when we adopt this full-blown multivariate view of phenotype (two nice papers by Walsh and Blows and Blows and Walsh). For a good review of the issue in traits see the newly out just this week in TREE Laughlin & Messier
  5. Any hope to predict the performance consequences of traits requires dealing with the TxE (traitXenvironment) interaction – Does high SLA (specific leaf area or basically thinness of leaf, a trait strongly correlated with the rapid photosynthesis end of the LES) lead to high or low performance? The answer blatantly depends on the environment (e.g. causes lower performance in dry environments or environments with lots of herbivory). Too many studies just look at trait-performance correlations when they really need to look at this in a 3-way fashion with performance as a 3-d surface over the 2-D space of trait and environment. Presumably this surface will usually be peaked and non-linear as well (again see Laughlin & Messier 2015)
  6. Theory – the field of functional traits is astonishingly lacking in motivating theory. When people tell me that natural history or descriptive science is dead, I tell people its just been renamed to functional traits. I personally see descriptive science as essential, but I also see theory and the interaction between theory and description as essential. Key areas we need to develop theory include:
    1. How exactly filtering on traits works – one of the appealing concepts of traits is that we can move from simply saying a community is a filtered set of the species pool to talking about what is being filtered on. But we aren’t thinking much about the theory of filtering. Papers by Shipley et al 2006 and Laughlin et al 2012 are good starts but not referenced by most workers in the field. And nowhere have we got a theory that balances the environmental filter that decreases variance with the biotic competition filter that increases variance within a community (and yes Jeremy, other possibilities are certainly theoretically possible per Mayfield & Levine 2010, but for good empirical reasons, I believe this is the main phenomenon happening in traits).
    2. What is the multivariate structure of trait covariance – This is partly an empirical question but there are many opportunities for theory to inform on this too. In part by thinking about …
    3. Causes of variation – we know variation in traits are due to a combination of genetic variation and adaptive plasticity and that these respond to environments at many scales. But can we say something quantitative?
  7. Individuals – we are very caught up in using traits as proxies for species but I increasingly think that filtering happens on the individual level and that we need to shift away from thinking about traits at the species level. The same given trait value (say the optimal value in some environment) can be provided by any of several species, each species of which shows consider variability in traits and therefore having significant overlap in the trait distributions between species.This idea can be found in Clark 2010 and Messier et al 2010 among many others. This might seem subtle, but it is a pretty radical idea to move away from populations to individuals to understand community structure.
  8. Interaction traits, reproduction traits and other kinds of traits – most of the traits studied are physiological/structural in nature. This is probably because one of the major roots of functional traits has been seeking to predict the ecosystem function of plants (e.g. CO2 fixation, water flux). But if we are going to develop a fully trait-based theory of ecology we need to address all aspects of an organism including traits related to species interactions (e.g. root depth for competition, chemical defenses for herbivory, floral traits for pollination and reproduction, and even behavioral traits like risk aversion).
  9. Traits beyond plants – the trait literature is dominated by botanists. There is a ton of work in the animal world that deals with morphology and behavior. And some of it is starting to be called “functional traits.” The hegemony of one term is not important, but the animal and plant people thinking about these things (whatever they’re called) need to spend more time communicating and learning from each other.

So there you have it. If you want to predict outcomes (e.g. invasion, abundance, being found at location X or in environment Y, and etc) based on traits, its easy. You just have to recognize that it happens in interaction with the environment and many other traits (many of which we haven’t even started studying) and figure out what the appropriate level of traits to study for the scale of the question. Sounds easy right? No, of course not. When is good science ever easy? That’s the problem with bandwagons. Anybody want off the trait bandwagon before we get to that destination? Anybody want on if they know that is the destination?

What do you think? Are traits a bandwagon? Is it about to crash? What will be the story uncovered by those picking up the pieces? Anything I forgot? Anything I should have omitted?

PS – I don’t usually do acknowledgements on informal blog posts, but it is necessary for this one. My thinking on traits has been profoundly influenced by many people. First among them would be Julie Messier who is technically my student but I am sure I have learned more from her than vice versa. And she currently has shared with me several draft ms that make important progress on #2, #4 and #5. I also have to highlight my frequent collaborators, Marty Lechowicz and Brian Enquist. Also influencing me greatly at key points are Cyrille Violle, Marc Westoby, Evan Weiher. And this field is advancing by the work of many other great researchers (some of whom I’ve mentioned above) who were there before the bandwagon started (and many before I got on) and will still be there after it crashes but whom I won’t try to name for fear of leaving somebody out. Despite it being a bandwagon right now, there is no lack of smart people trying hard to steer constructively!


What’s the biggest idea ecologists have ever permanently rejected?

Brian likes to emphasize–and I agree–that ecology as a whole lacks what Brian calls a problem solving mentality. We’re too often satisfied with evidence that merely suggestive or consistent with our hypotheses, and we’re too reluctant to permanently discard ideas that are past their sell-by date.

So here’s a question. Actually, two of them:

  • What’s the biggest idea ecologists have ever permanently rejected? Define “biggest” in any plausible way you like–most important, most influential, most widely-believed, most widely-studied, most fundamental, most cited, etc.
  • How many big ideas have ecologists ever permanently rejected? Can we come up with a complete list? If we can, that would be consistent with Brian’s and my concern–because that would mean the list is short!

Here’s an opening bid: Clements’ notion of communities as superorganisms. Even community ecologists (like me!) who believe species aren’t independent of one another because interspecific interactions matter don’t believe in anything like Clementsian superorganisms as far as I know. And subsequent attempts to revive the idea by talking about speculative possibilities like community-level selection haven’t gotten very far. I suppose there’s an echo of the superorganism idea in the notion that stability constraints “select” for stable communities, but it’s a pretty faint echo.*

Here’s a second bid: “broken stick” models of species abundance distributions. Not all that big an idea in the grand scheme of things. But beggars can’t be choosers–we’ll end up with a really short list if only really big ideas qualify for inclusion.

Hopefully, the list (or lack thereof, if no one can think of any other entries!) will start a conversation about what it takes for an idea to end up on the list. What does it take for ecologists to give up on an idea? What combination of circumstances–features of the idea, the ecologists working on it, the available theory and data, and nature itself–conspire so as to cause an idea to be widely influential in ecology, and then later get permanently rejected?

I predict that at least one comment will be from someone arguing that superorganisms or broken stick models aren’t dead. Thereby confirming the first paragraph of the post. :-)

*Aside: Here’s a brand-spankin’ new TREE paper by a bunch of sharp people on stability constraints possibly selecting for stable ecological systems. I think it’s an interesting topic that’s worth studying. I think you’ll need to look to microcosms or other appropriate model systems to get decent direct evidence (as opposed to highly indirect evidence that is sorta suggestive if you squint at it, wave your arms, and make lots of really strong assumptions). And I predict that the the effects of selection for stability (using whatever sense of stability you think is most relevant) will turn out to be quite weak to nonexistent in practice, because they’ll be swamped by other factors. But it’d be very cool if I turned out to be wrong about that. See this old post for further discussion.

Another attempt to stop or steer the phylogenetic community ecology bandwagon

I’m a bit late to this, which is embarrassing because I was involved in it. Back in May, Functional Ecology published a special feature (well, they call it an “extended spotlight”) on community phylogenetics. I helped edit the special feature, along with Anita Narwani, Patrick Venail, and Blake Matthews. Here’s our introductory editorial, which basically argues that phylogenetic community ecology has gone too far down the well-trodden road dead end of trying to infer process from pattern and that it’s high time for a course correction.

If it sounds rather like some old blog posts of mine (e.g., this and this), well, that’s no accident. It’s because of those old posts that Anita and Patrick invited me to join the team (they were the driving force behind this, having organized the symposium this special feature grew out of). So there’s a tangible benefit of blogging to add to the rather short list–you might get mistaken for an expert and invited to edit a special feature. :-) That my involvement in this project grew out of my blogging is my tissue-thin justification for posting about it.

The four papers in the special feature are quite different in terms of the specific topics addressed and the approaches used to address them. But they’re all nice examples of contrarian ecology, pushing back against the current conventional wisdom.

Kraft et al. use modern coexistence theory to rethink and make precise the disturbingly-popular-for-such-a-vague-idea notion of “environmental filtering”. They then review the literature and find that most studies of “environmental filtering” don’t actually present evidence of environmental filtering, properly defined. They argue that current vague usage of the term overstates the importance of abiotic tolerance in determining community composition. A nice example of something I’ve been thinking about a lot lately–how attempts to quantify vague concepts often just paper over the vagueness, leading to confusion rather than insight. One consequence of their argument (which I agree with 100%, btw) is to undermine a recently-proposed method for generating simulated datasets structured by a specified strength of environmental filtering. Which is kind of a funny coincidence, because the lead author of that method also wrote one of the papers in this special feature.

Gerhold et al. challenge the idea that the phylogenetic relatedness of co-occurring species can be used to infer the mechanisms driving community assembly. They point out that this idea depends on numerous strong assumptions that are weakly supported at best. They suggest more useful things that ecologists can do with phylogenies besides trying (futilely) to use them as a convenient shortcut to discovering community assembly mechanisms.

Venail et al. show, contrary to some recent claims, species richness, not phylogenetic diversity, predicts total biomass and temporal stability of total biomass in BDEF experiments with grassland plants.

Finally, Münkemüller et al. use evolutionary simulations to show that commonly-used measures of “phylogenetic niche conservatism”, such as phylogenetic signal, actually are very hard to interpret, and often are highly misleading guides to the underlying evolutionary processes governing niche evolution.

It will be interesting to see if these papers have much impact. I predict that Venail et al. will. It’s a comprehensive review of a purely empirical topic, and so I think it will quickly become the standard reference on that topic. The impact of Münkemüller et al. is harder to predict. My guess is it’ll get cited in passing a lot, but that people will mostly keep doing what they’ve been doing on the (dubious) grounds that there’s no easy alternative. I think Gerhold et al. and Kraft et al. will have little impact, unfortunately. They’re telling community ecologists to abandon an easy-to-follow recipe that purports to allow inference of process from pattern. Community ecologists only reluctantly abandon such recipes. But a minority of ambitious community ecologists will recognize that there’s an opportunity to do really-high impact work by following the lead of Kraft et al. rather than by following the crowd.

The editorial and the papers are open access, so check them out.

Friday links: Jeremy interviews Rich Lenski, pimp my snail, and more

Also this week: teaching resources for theoretical ecology, trolling the entire Evolution meeting (and the entire ESA meeting), data on sexual assault on campus, the science funding gene, reference letter advice, optimizing field pants, and more.

From Meg:

The ESA Theoretical Ecology Section has started a wiki of teaching resources relevant to theoretical ecology. (ht: Marissa Baskett) The Disease Ecology section also has a similar resource.

The University of Michigan conducted a survey of its students regarding sexual assaults. The results are sobering, and are summarized here. 22.5% of undergraduate women report nonconsensual touching, kissing, or sexual penetration in the past year. 9.7 percent of all women reported nonconsensual sexual penetration. The numbers are appalling, but I think it’s great that Michigan is bringing the issue into the light, and hope more universities do the same.

From Jeremy:

This week PLoS Biology published my interview with Rich Lenski about his Long-Term Evolution Experiment. The EiC there saw my post with questions for Rich, and Rich’s answers, and invited us to publish them. So Rich and I edited our posts to read like an interview; I like how it turned out. Hopefully publishing in Plos Biology will bring it to a new audience that wouldn’t otherwise have seen it. Which would be great, because Rich has a ton of super-interesting things to say. The LTEE is a great case study of how to do science, which anyone can learn from.

Charles Goodnight asks:

This is the week before the Evolution meetings, so the big question of the day is what can I post that I believe to be true, and will rile enough people up to get a good discussion going.

His answer: arguing that there is no genic selection (except in the special case of transposable elements). I like the idea of posting something controversial right before a big conference. May have to give that a go for ESA! With any luck, I’ll make a lot of students nervous that I’m going to ask them tough questions after their talks.* :-)

Charles Goodnight is trying to start an argument, but Trevor Branch is trying to continue one. Several, actually. Here’s his list of key readings on eight high-profile controversies in fisheries research. Good fodder for a lab meeting discussion or seminar course.

Further to our post earlier this week on how to write a reference letter, here are HHMI’s excellent tips on how to do it. I particularly like the tip on checking for subtle gender bias: switch all the pronouns in your letter to the opposite gender and see if it still reads well. If it sounds odd, you need to rephrase. Worth doing this for letters for both men and women. (ht Jessica Theodor, via the comments)

When does switching from a between-subjects design to a within-subjects design improve statistical power? When does it reduce (yes, reduce) your power? Here’s an example from psychology, but the point is broader. Good example for intro stats courses.

Wait, a random sample of 40 recent papers from each of three cancer journals reveals that 25% of papers have duplicated images, the frequency of which doesn’t vary with journal IF? And over half of those duplications involve the same image being used to represent different experimental conditions? Jeebus, that’s a high frequency of some combination of very sloppy errors and fraud. (ht Retraction Watch)

A history of, and ode to, field guides. (ht Not Exactly Rocket Science)

A hypothesis about the ultimate field pants. :-) Not being a field ecologist myself, I assume they look something like this. That’s the sort of thing I used to wear when I had to go into the field. And when I had to go up to bat, of course. :-)

Using the full power of genomics to find the science funding gene. :-) (ht Not Exactly Rocket Science)

Frogs unveil 5 million year plan to move up the food chain. :-) (ht Not Exactly Rocket Science)

And finally, pimp my ride snail. What would E. B. Ford have made of this? :-)

*Just kidding, students.** ;-)


***You now have all the incentive you need to read this old post before you go to the ESA meeting. Dynamic Ecology: causing, and resolving, student anxiety since 2012. :-)

The most common way to fish for statistical significance in ecology

Based on a large random sample of data my own not-inconsiderable but admittedly anecdotal experience, here’s the most common way to fish for statistical significance in ecology: analyze several different measures, indices, or indicators of the “same” thing.

The problem arises because many important concepts in ecology are only vaguely defined. “Diversity”, for instance–ecology is awash in diversity indices. But that’s far from the only example. Indeed, at some point in pretty much any ecology study the investigator will have a choice as to how to quantify something. In my own work, for instance, I have to decide how to measure spatial synchrony of population fluctuations. There are various ways one might do it. One could look at synchrony of abundances, or population growth rates, or per-capita growth rates. One could quantify synchrony with the cross-correlation coefficient, or some other measure of association. Etc. Often, different choices will lead to at least slightly and perhaps substantially different answers. And while in some cases there may be some mathematical, theoretical, or empirical reason to prefer one measure over others, those reasons often aren’t decisive. And in many other cases there’s no obvious reason to prefer one measure over another.

In such cases, it seems reasonable to look at, and report, various different choices. Report results for several different diversity indices, for instance, or several different measures of synchrony, or whatever. This feel less arbitrary than just picking one possible measure out of many. And it probably feels like you’re doing reviewers a favor, or at least pre-empting them. After all, aren’t they just going to ask you to report a bunch of alternative measures anyway? And heck, there are basically no page limits anymore thanks to online appendices, so why not just report results for a bunch of different measures?

This seems reasonable–but on balance, it’s not a good idea. In practice, it’s mostly just a (presumably unintentional) way to fish for statistical significance, by disguising exploratory analyses as hypothesis-testing analyses. I can’t recall ever reading an ecology paper where someone learned something scientifically interesting by comparing and contrasting results for different measures or indices of the “same” thing. Instead, having multiple measures of the same thing just gives authors more chances to come up with a statistically-significant result on which they can then focus. Or at least more excuse to wave their arms about what might be going on.

There’s a deeper issue here as well, that I’m still mulling over. In the past, I’ve said that if different measures of the “same” thing give you different answers, then that shows that they’re not actually measures of the same thing after all, and you don’t really know what you’re trying to measure. And if different measures of the “same” thing give you the same answer, they’re redundant with one another and you shouldn’t report them all. I still think that’s mostly right, but now I worry that it’s a bit misleading. I now think you can have a measurement problem even if various different choices of measure give you the “same” results. So you shouldn’t just rely on your data to warn you when you have a measurement problem. The problem here is different and deeper than just fishing for statistical significance. Andrew Gelman is good on this deeper issue.

Advice on writing reference letters (UPDATED)

Recently Paul Cross wrote to us asking if we had any advice for writing reference letters for people seeking faculty positions, particularly with regard to the tension between wanting to help out the person for whom you’re writing the letter and being totally honest. We decided this would be an interesting post topic*, so here are some thoughts.

Jeremy’s advice:

This is good advice on how to write a reference letter for someone applying for a faculty position, and this and this are good advice for academic reference letters more generally. So go read them and then come back here so I can emphasize a few points I think are particularly important.


Welcome back! A few points of emphasis:

  • Your job as a reference letter writer is to provide whoever is reading the letter with information and context that isn’t in the applicant’s cv.
  • One of the most valuable things you can do as a letter writer is provide context for any features of the applicant’s cv that might need some context (e.g., a gap in publication history due to a parental leave). But don’t reveal any personal details that the applicant doesn’t want to reveal. Talk to the applicant if you’re not sure what it’s ok to say.
  • Readers of reference letters are well aware that letter writers have incentives to gush, and they discount gushing accordingly. What matters is concrete evidence of the applicant’s training, accomplishments, skills, and potential, and concrete comparisons to other people you’ve known at the same career stage. Most readers will see through empty praise, especially if it doesn’t match other lines of evidence such as the applicant’s cv. Don’t misunderstand, praise is great–but only if you can demonstrate that it’s merited. You can’t help someone chances by gushing and might well hurt their chances.
  • Be honest about those concrete comparisons. It is not ok to just routinely say that everyone for whom you write a letter is in the top X% of their peers, where X is some small number. No, it’s not ok even if you think (falsely) that “that’s what all reference letters say” or “that’s what you have to say to avoid killing their chances.” (Late addition in response to Brian’s comments below: yes, I know it’s probably futile to urge letter writers to be honest when they have strong incentives to be positive. But I figure it can’t hurt…)
  • It can count for something to say (truthfully!) that you or your institution would hire this person. But how much it counts for depends on how similar you or your institution are to the person or institution doing the hiring. Different people and institutions look for different things in applicants.
  • Reference letter norms differ hugely between North America and Europe (sorry, no idea about the norms in other places). Try to learn and follow the local norms as best you can. The linked advice describes North American practices. Good readers of reference letters will be aware that norms differ and will adjust their reading accordingly. But not all will do so. So when asking for reference letters, you should consider whether your letter writers know and can follow the norms of whoever will be reading the letters.
  • Avoid gendered descriptions.

*We don’t always take up suggestions for post topics; see our About page.

Brian’s advice:

I basically agree with what Jeremy said. A few thoughts:

  • There IS a tragedy of the commons problem – as an individual writing a letter for a student or for somebody you know there are many incentives to be very positive to help them get a job. As a reader of letters, you want the honest truth. I broadly agree with Jeremy that it is incumbent on us to not give in to the tragedy of the commons and be a good citizen by writing honest letters. But the reality is, I’m sure I’m more honest in letters that I know are going to a colleague to read than to a stranger. I don’t think sweeping this tragedy of the commons under the rug helps. And it leads to all sorts of complex game theoretic analysis. As a reader of a letter, I have to estimate how honest I think the writer is being and discount accordingly. When I am sitting on a departmental or campus committee its pretty easy. All the letter writers are known to me. So I will know that a letter from person A (who is very tough) that makes the candidate sound average is actually far higher praise than a letter from person B that is effusive (but I know that B effuses for everybody). But that only works when I know the letter writers A vs B. Its much harder in inter-campus scenarios. There are no good answers, but acknowledging this exists is helpful.
  • This interpretation of what somebody else is really trying to say is that much harder in inter-country scenarios. As Jeremy hinted at, letter writers from the US are much more likely to behave like person B above while letter writers from Europe (and especially Britain) are much more likely to behave like person A. Indeed a puff letter like that from person B could actually be harmful (or at least useless) for somebody in the UK. And beyond those handful of countries we’re familiar with, few know what to write. I recently had a conversation with somebody where we were trying to figure out whether writing letters for somebody in Israel should be more like for the US or more like for the UK. We never did figure it out.
  • You can try to help this muddle by signalling at a meta level. By saying something like “I’ve never said this about a student before but they have a better programming skills than any student I’ve seen” is stronger/clearer than “they have some of the best programming skills I’ve seen”. This can of course be gamed and you can say “I’ve never said this before” in every single letter. But I hope most of us are not that cynically gaming the system!
  • It is of course easy to write a letter for somebody who is excellent. But what about a letter for somebody who is good? Not the best, but deserving of a job.I think here it is important to carve in bas-relief (what is not said is just as important as what is said). As an example, if I think a person is a top 5% or a top 10% student I will say that in the letter. If I think somebody is a top 50% student, I simply won’t say this. In this way you can write a letter that is positive AND truthful.
  • Although many letter writers think what I want is a general assessment (excellent/good/fair), the truth is because of all this complexity of interpretation, I only give so much credence to general assessments (unless its somebody who I know well enough to have a decoder manual of what their general assessments mean). I am much more likely to trust the objective record (papers, test scores) and my own impressions from meeting the candidate for a general assessment. What I find most helpful as a reader of a letter is the details. As Jeremy noted, contextualize work – explain the context – were they working remotely, just had a child (although one has to be careful with this as Jeremy noted, and I certainly think this context applies to men as well as women), etc? But put some dimensions on the person. What are their special strengths. What impressed you most about them. As scientists we shy away from anecdotal evidence, but anecdotes supporting specific claims are almost always the strongest part of a letter. Having spent a lot of time on peer committees (i.e. making tenure recommendations) I can tell you these concrete details and anecdotes are almost invariably what get taken most seriously from the external letters.
  • As a consumer of reference letters it is invaluable to actually have a phone conversation with the references (when allowed – its not allowed e.g. for tenure). Much will come out verbally that won’t come out in a letter. Especially with a few leading questions like “if I am going to become this candidates new supervisor, what would you suggest I give them extra help with to make them successful”.
  • I think gender bias in letters is subtle but pernicious. A few years back I did intentional review of gender bias in letters I wrote. There was nothing glaring. I was equally likely to use “brilliant” or “top” for men and women. But there were a number of more subtle biases that I was embarrassed to discover. Especially in the area of describing how people work collaboratively. I would mention as praiseworthy when a man worked well with others but only note this in passing for women simply because I subconsciously took it for granted that women always worked well with others (despite examples to the contrary). Conversely, I would single it out as novel if women played leadership roles more often than I did for men. Related to this, it is probably impossible to have a wholly objective interpretation across men and women of whether aggressive behavior is that just right amount that is beneficial (aka leadership) or too far so as to be harmful if you’ve been raised in modern society (which has profound gender-biased judgments on this). And of course there are lots of subtle biases in what we value. For example, is the person who grabs the marker and starts leading the conversation in a group meeting or the person who listens to everybody and provides the perfect summary statement more valuable? I’ll bet the first skill gets remarked upon favorably in letters a lot more often than the 2nd, but I think they’re equally valuable – and while by no means strongly tied to gender, they are not independent of gender either. I don’t have great answers. But it is really instructive once you’ve accumulated a body of letters written to look at them with a really hard nose for these more subtle gender biases. Given all the data on implicit biases (in women as well as men), it is unlikely you are completely escaping gender bias in your letters.

Late additions from Meg (Jun 24, 2015):

I agree with what Jeremy and Brian wrote, so don’t have much to add. A few things I wanted to add are:

1. I find it can be helpful to give a concrete example of something. To go with the leadership example raised by Brian, it might be possible to give an example of when a project was at risk of going off the rails, but that person stepped in and helped the project get back on track.

2. Jeremy and Brian both mentioned the need to consider gendered language in writing letters. As Brian said, we all have implicit biases, and this impacts letters. Occasionally the biases are glaring, but more often they are subtle. This Trix & Psenka study is the classic one on the topic. Here are guidelines for writing letters, provided by the ADVANCE program here at the University of Michigan.

3. Length: for most letters of recommendation, anything less than a page will be seen as “too short”. No one wants to read more than three pages. So, ending somewhere on page two (or three, if really necessary) is a good target.

When is vagueness a virtue in science?

Like a lot of people, I think precision is a virtue in science. One sign of scientific progress is increasing precision, often but not necessarily expressed mathematically. For instance, Darwin got a lot right without using any math–but we’ve since learned a lot more about evolution than we could have otherwise by expressing Darwin’s ideas more precisely and mathematically. Or think of how the vagueness with which ecologists define “biodiversity” and “ecosystem function” leaves us vulnerable to critical scrutiny. Or think of how Bob May’s math undermined and corrected previous verbal intuitions about how “diversity” or “complexity” might relate to “stability” in ecology. Thereby forcing future ecologists working on that topic to quit waving their arms and instead say exactly what they meant. We need math because the consequences of our assumptions often can’t be worked out without math, and because our choice of words often misleads us in subtle but important ways. Some profound scientific ideas can’t even be expressed verbally, at least not without doing the math first.

So in the spirit of being contrarian with myself,* here’s a question: can vagueness rather than precision ever be a virtue in science? Vagueness is sometimes inevitable, such as when some new idea is first being developed. But is it ever desirable, so that we wouldn’t want to get rid of it even if we could?

Some smart people would say yes. For instance, here’s philosopher Ken Waters**, writing in 2013 about vagueness in the definition of “gene”:

What about biologists? What do they say when repeatedly pressed by philosophers to answer the question, ‘What is a gene?’ In my own experience, after being shown that their answers are vague, admit exceptions, or are ambiguous, biologists typically shrug their shoulders. Many quickly concede that they do not know exactly what a gene is. Reflective biologists add that trying to answer the question with the kind of rigor that philosophers demand would be
counterproductive. Progress in genetics, they say, has depended and continues to depend on “muddling through”. Science, they insist, would be stymied if geneticists were forced to agree on using a clear and unambiguous concept of the gene.

The example of “gene” hints at one kind of situation in which vagueness might be a good thing: some bit of biology is messy and complicated. There’s a range of cases that are obviously similar in important ways, but that also differ in various ways and it’s hard to pin down exactly which similarities and differences matter for what purposes. So rather than risk pinning them down in the wrong way, we muddle through with vague definitions. Although personally, I might describe such cases as cases in which vagueness is inevitable rather than advantageous. Or to put it another way, in such cases the greatest level of precision possible or reasonable may not be all that precise in an absolute sense (Peter Godfrey-Smith recently wrote about this in an evolutionary context). Put still another way, it’s not as if we ever want as much vagueness as possible–there’d be no point to defining genes as “molecular thingies that do stuff”. So whether you want to say vagueness is good rather than merely unavoidable probably depends on what baseline you’re comparing it to. I’d say the same about other vaguely-defined terms in ecology and evolution, like “species” and “ecosystem”.

It occurs to me that the best example of a usefully-vague term in ecology might be…[wait for it!]…”ecology”. Ecologists all share some interests, and it’s good for all of us to recognize those shared interests, even though they’re quite vaguely defined. Recognition of shared interests encourages us to see each other as colleagues involved in a shared intellectual project that’s worth pursuing. But if you tried to define those shared interests more precisely, you’d either end up with an empty definition (which is just vagueness by another name), or you’d end up with a definition sufficiently precise as to exclude some people whom everybody considers ecologists. I’m thinking for instance of Cooper’s The Science of the Struggle for Existence, a philosophical work framed around a search for the definition of ecology. As I recall, the definition Cooper ends up with basically defines ecosystem ecology as not ecology. I’m also thinking of various people who’ve defined ecology as “scientific natural history”, or otherwise emphasized the primacy or centrality of natural history or field work to ecology, in a way that defines me and Ben Bolker out of the field (I’m sure that wasn’t the intent***, but that’s the effect). Tyler Cowen has noted that many political unions and other partnerships are based on “creative ambiguity”.**** Perhaps the field of ecology is one such union.

Another way vagueness can be useful is by allowing an idea to draw greater interest and more research effort than it otherwise would. If progress depends on having a critical mass of people working on a problem, and if defining the problem too precisely (at least initially) would discourage lots of people from working on it, then maybe vagueness is a good thing. Of course, you can have too much of any good thing–a vague idea that attracts a lot of research interest is a good way to start a bandwagon.

One important distinction here is between vaguely defined concepts, and concepts that have various precise but contrasting definitions. As noted earlier in the post, “stability” is now an example of the latter, having previously been an example of the former. I don’t think it’s ever helpful to mix up different definitions of stability, or be vague about which definition you’re using. More broadly, I suspect there are various ways scientific ideas can be vague, and it’s probably important to distinguish between them. Maybe somebody needs to write the scientific equivalent of Seven Types of Ambiguity.

*For which this is the best illustration, given my credentials as a zombie fighter. :-)

**A conversation with whom got me thinking about this. And as an aside, if you’re looking for philosophy of science to read and you’re interested in genetics and evolution, you should add some of Ken’s stuff to your reading list.

***At least I hope not! :-)

****Such as academic departments. As Cowen writes:

We find the same in many academic departments.  Things can be going along just fine, but once the department has to write out an explicit plan for future growth and the allocation of slots across different fields or methods, all hell breaks loose.

Friday links: Meg read ALL the things and picked the best ones

This week: the secret to R’s success, the internet vs. Tim Hunt, career polygamy, satisficing vs. stress, the joys of chairing your department, the formula for getting an academic job (is that there isn’t one), ggplot2 cheat sheet, and more. Oh, and Jurassic World lacks a responsible waste diversion program. :-)

From Meg:

I’m late in getting to read this, but this is a great post calling for acceptance of career polygamy in science. It’s a call for acknowledging and accepting that some people will want to pursue science and other pursuits (e.g., writing) at the same time, and that such people have a lot to contribute to science. (ht: Lenny Teytelman)

In a similar vein of redefining what constitutes success in science, academia, and life: at Tenure, She Wrote, a guest blogger wrote about her decision to turn down an offer for a tenure-track position. Her decision was largely based on deciding she wants to stay in the city where she’s set down roots and where she has a family support network. Without a doubt, the hardest part of academia for me has been moving away from great support networks multiple times. I completely understand deciding that it would be better to leave academia than to move away from a strong support network.

Lots has been written on Tim Hunt’s sexist comments, but I found two pieces especially illuminating. First, this piece from Michael Eisen, which puts Hunt’s comments in the context of a meeting of young Indian scientists that Hunt and Eisen both attended, and, in particular, in the context of a session where young women scientists talked about the sometimes horrifying stories about their experiences as women scientists. Second, this piece by journalist Deborah Blum, who was at the event in Korea where Hunt made his sexist remarks, also helps put the events in context. She writes,

I do have sympathy for anyone caught in the leading edge of a media storm. But if we are ever to effect change, sometimes we need the winds to howl, to blow us out of our comfort zones.  Because the real point here isn’t about individuals, isn’t about Tim Hunt or me.
The real point is our failure, so far, to make science a truly inclusive profession. The real point is that that telling a roomful of female scientists that they aren’t really welcome in a male-run laboratory is the sound of a slamming door. The real point is that to pry that door open means change. And change is hard, uncomfortable, and necessary.

That made me think of this series of tweets by Lindsay Waldrop, who offers her perspective as a young woman scientist on the reaction to Hunt’s comments. She points out, quite correctly, that NOT responding to these comments is damaging, because it seems to condone them. (ht for the Blum piece to Gina Baucom)

Here’s a ggplot2 cheat sheet that is searchable by task. It seems like it will be quite handy! (ht: @statsforbios)

I really enjoyed this post by Liz Haswell on how to deal with stress as an academic. (The post says it’s aimed at grad students and postdocs, but I think it applies to everyone.) One part I especially agree with is:

No matter how smart and competent you are, it is not possible to do an excellent and compelling job at everything that comes across your path. So you have to choose. First you must decide what you DO want to accomplish well (give a great talk, make substantial progress on an experiment, make industry connections). Then, you simplify your life and reduce stress by a) “satisficing” or b) jettisoning the rest.

As she says, it can be really hard for a perfectionist to accept when something is good enough but not perfect, but doing so is so important. (ht: Joan Strassmann)

From Jeremy:

Why has R been so successful, despite its quirks? The linked piece omits part of the answer: lots of people use it, which encourages others to do so. Nothing succeeds like success. (ht Brad DeLong)

Data-based debate on the risk factors for corrections and retractions of scientific papers. Haven’t read the papers in question myself so don’t have a view of my own. But given that the data in question are correlational, and given the many factors that affect whether or not a correction or retraction occurs (some of them unmeasurable), I doubt that you can say much about causality with any confidence no matter how you slice and dice the data.

How long are master’s theses and PhD dissertations? Here’s the answer, for the University of Minnesota. It’s broken down by field. I leave it to you to decide if “above average” is a good thing in this context. :-) I believe I’ve linked to an earlier version of this dataset that lacked data on master’s theses.

Princeton’s trustees approve 17 new faculty hires; Michael LaCour is not on the list. I assume that means he’s been unhired. (ht someone on Twitter; sorry, forgot who)

There’s no magic formula for success in academia. Echoes my own thoughts in various old posts. A further (wildly speculative) thought: does academia tend to attract people who want a magic formula for success? I ask because in some ways academia as a career path comprises a very well-defined series of steps. You go to college, then you go to grad school, then you get a postdoc, then you get a faculty position. And at the end of the path is tenure, which gives you more job security than in just about any other occupation. So does academia tend to attract uncertainty-averse people who want to walk a well-defined career path towards a secure destination? (a perfectly fine desire that I share, btw) Which then causes some of those people to get upset or frustrated when they discover that there is no magic formula–that is, that the path to career success in academia isn’t nearly as well-defined as they thought?

Stephen Heard on all the reasons you might actually want to be chair of your department one day.

And finally, a public assembly facilities manager critiques Jurassic World:

While it is never wise to carelessly damage or destroy capital assets, the stated cost of park attractions is substantially less than the potential tort exposure in the event of an attraction-guest consumption event.

:-) (ht Marginal Revolution)