Pre- vs. post-publication review and procedural justice

Wanted to elaborate some ideas from my recent post on “culture clashes” surrounding post-publication peer review. In that post, I suggested the post-publication review isn’t likely to work well unless we develop widely-accepted norms and procedures for how it should be done, just as we currently have for pre-publication review. I just thought of an analogy that I think clarifies and bolsters that argument. It’s an analogy between pre-publication review and procedural justice.

Procedural justice is the idea that justice reflects the fairness of the procedure used to resolve disputes. In A Theory of Justice, philosopher John Rawls distinguished three flavors of procedural justice. Perfect procedural justice consists of an independent criterion for what constitutes a just outcome, plus a procedure that guarantees that the just outcome will be achieved. Obviously, perfect procedural justice is more of a philosophical ideal than a real world possibility. Imperfect procedural justice is like perfect procedural justice, except that the procedure doesn’t guarantee that the just outcome will be achieved. Criminal trials are an example–there are rules and procedures to ensure that both sides get a fair hearing, but there’s no guarantee that the guilty will always be convicted and the innocent always cleared. Finally, pure procedural justice is when there’s no independent criterion for a just outcome; any outcome of a fair procedure is a just outcome by definition. For instance, in some sports a coin flip is used to decide which team gets the ball first. As another example, any outcome of a democratic election conducted under appropriate, fair procedures is a just outcome.

One reason to care about fairness of procedures is that fair procedures often will be more effective than unfair ones at producing just outcomes. For instance, “show trials” with unfair procedures stacked against defendants will mistakenly convict innocent people much more often than fair trials. But if what we ultimately care about is achieving just outcomes, then shouldn’t we prefer whatever procedure produces just outcomes most frequently, even if that procedure is unfair in some sense? Indeed, why should we even care about procedures at all, if what we ultimately care about is achieving just outcomes? Think of how criminal trials often are decried as “miscarriages of justice” in cases where the guilty go free or the innocent are punished, even though fair procedures were followed. That’s why books and movies often portray as heroes individuals who use ad hoc and illegal methods to achieve just outcomes.

Of course, there are other reasons for caring about procedural justice besides its effectiveness at achieving the right outcomes as judged by independent criteria. For instance, in order for a legal system to function effectively, it needs the cooperation of the general public. And that cooperation isn’t likely to be forthcoming if the legal system is widely viewed as unfair (e.g., Tyler 2003). That’s also one of the arguments for democratic elections over other means of transferring governmental power. Because democratic elections are seen as fair, the losing side is more likely to accept the outcome than if power is transferred by, say, a military coup. Put another way, while unfair or ad hoc procedures might seem like they’d lead to better outcomes than fair procedures in particular cases, in the long run they would actually lead to worse outcomes because of the chaos that would result from people refusing to accept them as legitimate. Put yet another way, unfair or ad hoc procedures are in some sense often parasitic on fair, agreed procedures. The only context in which occasional use of unfair or ad hoc procedures can improve the justice of outcomes (as judged by independent criteria) is a context in which those unfair or ad hoc procedures are used in rare, extraordinary circumstances. That way they don’t undermine everyone’s confidence in the fairness of the usual, agreed procedures.

In various ways, pre-publication review is analogous to a system of imperfect procedural justice, and advocates of post-publication review are analogous to critics of imperfect procedural justice. There are independent criteria as to what constitutes a publishable scientific paper, such as technical soundness. And pre-publication review is an imperfect system of procedures for fairly and accurately identifying papers that meet those criteria. In their insistence that what ultimately matters is getting the science right and correcting the record in individual cases by whatever means necessary, the stronger advocates for post-publication review are analogous to those who care little for procedural niceties in other contexts. Instead focusing on achieving the right outcomes by whatever means necessary on an ad hoc, case by case basis. And just as with other examples of imperfect procedural justice, like the criminal justice system, one can argue that any effective peer review system needs to not only achieve the right outcomes at a high rate, but be seen to comprise fair procedures. Otherwise people won’t voluntarily participate in or cooperate with the system, causing the system to break down and cease to function effectively. And finally, one can argue that ad hoc post-publication review is effective only in rare, unusual cases, as a supplement to the existing pre-publication review system (e.g., fraud detection; close and rapid scrutiny of especially important or high-profile papers).

I also wonder if pre-publication review is in some ways analogous to a system of pure procedural justice. This is a new thought I’ve just had, unlike everything above, which is just elaborating that recent post of mine. So bear with me while I think out loud… The idea is basically that we need some way of adjudicating disagreements as to what science is worth publishing (and such disagreements are common; see this old post). We also need some way of coming to final accept or reject decisions, some way of declaring the review process closed so that everyone concerned can move on. Sort of like democratic elections. You can’t expect everyone to come to an agreement about who the best candidate is, no matter how long a political campaign is allowed to go on. There will always be differences of opinion, both because some differences of opinion are reasonable, and because some people will never change their minds under any circumstances. But despite those irresolvable differences, we need a procedure for cutting off debate and picking a winner, because we need to have a government and everyone needs to get on with their lives. What we need, and all we can ask for, is a fair procedure for picking the winner. If the procedure is fair, that makes the outcome fair, whatever the outcome may be.

I think there’s something to this argument, both in the context of democratic elections and in the analogous context of peer review. Which is why I disagree with folks like Andrew Gelman who’d prefer to move to a review system in which publication is never the last word, even provisionally. Who would prefer scientific communication to consist of totally open-ended discussion, with traditional papers either ceasing to exist or else being viewed very differently. Don’t get me wrong, like Andrew I hate it when somebody defends a paper from post-publication criticism by saying “It was peer reviewed”, as if that means the paper must be right. Analogously, I hate it when an elected government defends its policies by saying “We won the election”, as if that means its policies must be wise. But I suspect that those who defend their work against post-publication criticism by saying “It was peer reviewed” generally don’t mean “it must be right.” Instead, what they mean is something like what elected politicians mean when they say “elections have consequences“. The phrase “elections have consequences” is shorthand for the fact that fair elections are a form of pure procedural justice, not imperfect procedural justice. The fairness of the procedure is what makes the outcome (and its consequences) legitimate. So if you don’t like the outcome, well, you’re free to complain, but ultimately there’s nothing you can legitimately do about it besides live with it and try to win the next election.* Analogously, one can argue that science needs fair peer review mechanisms that, for at least some purposes, cut off debate and discussion and allow all concerned to move on. Not all debate and discussion, of course, any more than elections cut off all political debate and discussion. But political discussion and debate aims to change the outcome of the next election, rather than undo the outcome of the previous one. Otherwise politics and government cease to function effectively. Analogously, one can argue that scientific discussion and debate should be directed primarily towards determining what papers get published in future, rather than re-reviewing papers that were published in the past. Because otherwise science will cease to function effectively.

Interestingly, the analogy to pure procedural justice partially undermines attacks on pre-publication peer review based on its inability to predict the subsequent “impact” of papers (however “impact” is measured). That kind of attack only has bite insofar as you think of pre-publication peer review as an imperfect procedure for attaining some independently-defined goal like “identify high impact papers”. It has no bite insofar as you think of pre-publication peer review as analogous to pure procedural justice.

I think the analogies of peer review to imperfect procedural justice and pure procedural justice both have something to them. Which may be what makes debate about peer review reform challenging, and leads to people sometimes talking past one another. Because you need to evaluate pre-publication peer review on totally different and even contradictory grounds depending on whether you see it as a system of imperfect procedural justice or a system of pure procedural justice.

This has been another edition of Jeremy’s Half-Baked Analogies. Looking forward to your comments.**

*Except perhaps in extraordinary circumstances such as when the elected government tries to change the rules so as to enshrine itself in office permanently. And one can of course argue that there’s a scientific analogue of that sort of thing–papers that have especially egregious flaws, maybe.

**I predict that the proposed analogy to pure procedural justice will convince few of you.

Friday links: extinction cartoons, March Mammal Madness, and more

Also featured this week: “quit lit”, “public” universities aren’t so public, fashion advice, and more…

From Meg:

There is a growing body of blog posts in which people document their reasons for leaving academia – Jeremy linked to one of these last week, and there are enough of these emerging that I’ve seen them referred to as a collective “quit lit” genre. Here are two I read recently that I found particularly compelling and powerful. First, this one (by Lenny Teytelman) talks about leaving largely because of concerns over the current funding situation, but also because of the nomadic lifestyle of an academic. I’ve been fortunate to get funding so far, but I certainly worry about my ability to continue to support folks in my lab. And, most definitely, I have found the upheaval of one’s personal life that comes with moving to be very difficult. For me, this has probably been the most difficult aspect of academia. The second post (by Frances Hocutt) deals with issues related to women in science, and, more specifically, with the author’s choice to leave chemistry. One point of the piece is that, in trying to increase the numbers of women (and other underrepresented groups) in the sciences, we need to make sure we don’t guilt those who chose to leave. As she says, “When a pipeline leaks, we don’t blame the water. We fix the pipe and design the next one to leak less. Why do we blame women who leave STEM fields?” It also includes examples of how to be supportive of people who’ve decided to leave science/academia. The comment threads on these posts are also filling up with stories from others who are recounting their own tales.

It’s time for Mammal March Madness! This is run by Katie Hinde, and starts March 10^th, so you only have a few days left to fill out your bracket! Lots more info on the page, but here’s a brief overview of how it works: “Battle outcome is a function of the two species’ attributes within the battle environment. Attributes considered in calculating battle outcome include temperament, weaponry, armor, body mass, fight style, and for social mammals division- group size. Some random error has been introduced into calculating battle outcome & the amount of that error is scaled to the disparity in rankings between combatants. Early rounds, the battle occurs in the better-ranked species’ habitat (home court advantage). BUT once we get to the ELITE EIGHT, battle location will be random: rainforest, savannah, ocean & ocean adjacent, or snowy tundra/mountains.” I’m hoping to get my 3-yr-old to fill out a bracket soon!

From Jeremy:

How the last individuals of various now-extinct species met their end, in cartoon form. (ht Denim and Tweed)

The journal Nature is looking for an Ecology and Evolution Editor for 6 months starting in June, to cover a paternity leave. A rare opportunity to see, and shape, the leading edge of the field. I was briefly tempted to apply, it would be a really interesting and valuable experience (and also a real challenge to do well). But the timing is terrible for me. Anyway, just for laughs: what sort of attributes do you think a Nature E&E editor ought to have? Anyone you know who you think would be really good at it? Maybe we can start a “draft Brian” or “draft Meg” movement! (ht Sociobology)

Anyone who thinks that metrics–impact metrics, altmetrics, whatever–are a way to reduce subjectivity in decision-making (say, about who to hire or what grants to fund) should read this. (ht counterparties.com)

Does the US still have public universities? Or does it effectively have subsidized private universities? Data here. (ht Economist’s View)

Good advice for giving a good talk. One quibble: minimum font size should be equivalent to 24 pt. Arial, not 16 pt.; the latter is way too small for most rooms. My own advice is here. And Zen Faulkes has an ebook of presentation tips. (ht @jennyperry)

A picture gallery of professional wardrobe ideas for female academics. Obviously I’m not really the best person to vouch for these suggestions (FWIW, they seem like reasonable suggestions to me). I’m just passing them on in case anyone’s interested. Advice on this sort of thing is a little tricky, in part because conventions can vary between fields. For instance, when dressing up (say, for a job interview), I think ecologists tend to dress up a bit less than, say, molecular biologists. (ht @jennyperry)

Using Google Documents in Discussion-Based Courses

This semester, I’m teaching a discussion-based course. I’ve taught this sort of course before, and I really like the no-lecture format, as I think the students learn more when they work through the materials themselves and teach each other. However, the major problem with this format of a course is figuring out how to get all the students to engage in the discussions. There are always a few really chatty students who do a lot of the talking, and some who never speak. Most importantly, to quote my University of Michigan colleague Orie Shafer, “the boldest, most confident person is rarely the person who understands things the best.”

Over the years, I’ve tried different strategies to encourage more even participation, but still felt like the evenness of discussion participation was a problem in courses. So, when I met with someone from the University of Michigan Center for Research on Learning and Teaching about the design of the course I’m teaching this semester, this was a topic I was interested in discussing more. She recommended watching this video featuring Orie Shafer talking about how he uses google documents to facilitate group discussion in his class. (Here’s a direct link to the video.) That video is where the quote in the preceding paragraph comes from, and it really resonated with me. So, I decided to give google docs a shot this semester.

How do they work? Well, we use slightly different approaches in different class periods, but the general idea is that the students split up into groups (it seems to work best if I split them up, rather than letting them choose groups themselves). Each group is then given a set of questions to answer or topics to research. They open up a google document, share it with their group mates (groups are usually 2-4 people), and share it with me. Then they get to work on answering the questions or researching the topics. Sometimes, the whole class answers a single set of questions – I am most likely to use this approach if the reading was a difficult reading from the primary literature. In this case, I give them a set of questions aimed at making sure they all understand the main results, where they came from, and what they mean. Other times, each group researches a different topic related to the readings for the day; oftentimes, many of these topics emerge out of the pre-class forum discussions, but other times the topics are things that I think they would be interested in or that would be important for them to understand.

In my opinion, there are several key advantages of using google documents in discussion-based courses:

1. Being able to type out their answers gives the students who are more shy a more even playing field, and allows them to get their thoughts and opinions out.

2. Even for students who aren’t particularly shy, writing out their thoughts before trying to articulate them to a large group can give them a space to work through complicated ideas. I’ve overheard students commenting to each other how much easier it is for them to get things out in writing first, then to discuss it with each other. Of course, google documents aren’t essential for this – you could have the students write down their thoughts on paper first.

3. I can more easily monitor the different groups and provide feedback in a less intrusive way. This is probably the biggest benefit of the google docs, in my opinion. In previous discussion-based courses, I would rotate around the room to check in on the different groups. As soon as I got to a group, they would usually all stop talking and look at me. Now, I monitor the discussions from my laptop at the front of the room, so I can more easily see what they’re confused by. And, more importantly, if they are heading off in the wrong direction or their document reveals a fundamental misconception about something, it is much easier for me to get them back on track in a way that doesn’t embarrass the students. For example, in one class this semester, the answers to one question by one group revealed a really fundamental misunderstanding of human evolution. If this had come up in front of the whole class, it would have been tricky to correct it in a way that didn’t completely stifle remaining discussions. And, if this had happened in a small group before I started using google docs in class, I probably wouldn’t have known they were so confused about this topic, because I wouldn’t have been likely to hear it while circulating the room. But, with the google document, I could see it and just type in a note suggesting that they should do some research on human evolution and patterns of migration. That was enough to get them back on track and then, in a moment that shows the power of this peer-based learning, they then explained what they had just learned to their peers. This is not something I would have thought to include in a lecture on this topic, because I wouldn’t have realized they were confused about this – again, this is one of the strengths of this approach to teaching.

4. It is easy to create a document at the end of class that synthesizes the major topics of discussion. I copy chunks of the different groups’ google documents and post that file to the course website so students can come back to it later. For example, earlier in the semester, I had one group go through the different stages of clinical trials, with the goal of having them understand how long the process is. In a later class, students were wondering why doctors weren’t prescribing a drug based on a result from an experiment done in mice that had just been published last May. Clearly the were forgetting about the lengthy clinical trial process! So, we went back to the document that synthesized our earlier discussions, and could recap all that information.

(As a general note, part of why I think it works so well in my class this semester is that all students bring laptops to class. They are not required to bring laptops, but are strongly encouraged to do so.)

Of course, it’s not all upsides. Some downsides to the approach:

1. While it’s easier to monitor the discussions than by circulating the room, I still find it difficult to keep them on top of all the groups. (We’ve had as many as 9 groups researching different topics.) I don’t know that there’s much I can do about this, other than get better at following different documents at once.

2. Similarly, some level of walking around the groups is still useful, even if the students initially stop talking when I get there. They often have questions about the topic that they ask when I stop by, but not via google document. So, I need to do some combination of them, which means even more for me to juggle.

(These next two are not specific to google documents, but more to this discussions-based approach.)

3. I think some students would prefer that I just tell them what they’re supposed to know. I’ve tried to explain to students that part of the benefit of student-centered learning is that, when they figure it out themselves or learn from a peer, it sticks much better than if I just try to explain it. But my class this semester is made up of freshman, and their other science courses are probably all traditional lectures. Some students prefer that approach (even if the data show that they don’t retain the material as well when it’s presented in lecture format). As Eric Mazur mentioned in his talk at UMich (storified here), there can be a false sense of security after sitting through a lecture – you feel like you’ve learned a lot because of all the information that was just presented to you, even if understanding and retention of that information tend to be poor.

4. Choosing the topics for small group work is difficult and requires lots of last-minute work on my part. As I said above, I try to base the in-class discussion topics on the forum posts by the students. (Their forum posts are due the night before class.) This means that I spend a few hours right before class trying to find topics and sources that are accessible, informative, interesting, and reliable. And then I have to figure out how to group them together and try to make it so that the different groups have roughly similar amounts of material to process and present to their peers. I tend to be the sort of person who prepares things well in advance, so this “just-in-time” teaching can feel like a scramble for me.

5. Sometimes the wireless goes down, and then we have to adjust on the fly in class.

6. By encouraging everyone to bring their laptops and have them open, I am certainly making it more likely that some students will get distracted by instant messages, emails, etc. Most of the students have their laptops open so they can read the notes they wrote and/or take notes on what other students are saying. But I know some of them are sometimes switching to non-class materials. I’m not entirely sure of what to do about this. I think if the students had exams that might potentially cover any topic that was brought up in class, that would encourage some students to pay attention more. (At the same time, it would probably also stress out other students who would then be just trying to memorize things for an exam.)

Overall, I’ve been happy with using google docs in class, and plan on continuing to use them in my discussion-based classes in the future. Participation in my class this semester has been much more even than in previous classes. There are still some students who talk more than others, but no one dominates discussion. And there are other students who are still pretty shy, but everyone participates in class now, which I’ve never had happen before. At the same time, I will probably try to change some aspects of the course, especially to reduce the digital distraction problem. But I suspect those will center more on assessments, and that I won’t give up on google docs in class — I think they really help with discussions.

Questioning the value of biodiversity (UPDATEDx3)

I don’t ordinarily plug individual papers in ecology journals, figuring y’all don’t need my help finding individual papers to read and deciding what you think of them. But I’m making an exception because this is important.

Writing in Trends in Ecology and Evolution, the always-thoughtful Mark Vellend reviews philosopher Donald Maier’s new book What’s So Good About Biodiversity? You should click through and read it, because the review (and by the sound of it, the book) are really sobering.

Maier reports being “stunned” that he “could not find a single argument [for the value of biodiversity] that does not have serious logical flaws, crippling qualifications, or indefensible assumptions.” And before you write Maier off as just ignorant, or as someone saying outrageous things to try to sell books, be aware that Mark thinks he’s got a point. Which doesn’t prove the book is right–but does prove it deserves to be taken seriously rather than dismissed out of hand. For instance, Mark discusses the argument for biodiversity based on the ecosystem services it provides–and notes that nobody ever talks about all the ecosystem disservices that biodiversity also provides, such as increased disease risk. And how come ecologists never argue for promoting species invasions on the grounds that it would increase biodiversity and thereby enhance ecosystem services? As Mark pointedly remarks, when it comes to valuing biodiversity, ecologists basically started with a conclusion, and then looked around for arguments to support it. It’s unsurprising that such a backwards procedure yields unconvincing arguments. It’s the same procedure that Scholastic philosophers used to find “proofs” of the existence of God, for instance, and it’s the same procedure creationists use to argue against evolution. Now in fairness, post hoc rationalizations have their place, and often they’re unavoidable. But if they’re really bad rationalizations, and if they’re not even recognized as rationalizations, then there’s a problem.

Which is something ecologists should talk more about–but we mostly don’t, at least in my experience. That’s maybe the biggest problem here. Maier concludes that not only have ecologists collectively been the victims of confirmation bias when it comes to valuing biodiversity, but they’re not talking about it. Maier concludes that there is “tacit agreement among colleagues not to rock the boat of bad reasoning – perhaps out of fear that there is no other way to defend nature and its value”. As Mark says: ouch.

Anyway, consider this another example of why ecologists should read more philosophy.

p.s. I’m a bit late to this, so for previous commentary on Maier’s book (and Vellend’s review of it), and for commentary on (and links to) a talk by Maier, see here, here, and here.

UPDATE: On Twitter, @frank_burdon asks whether Mark “lacks objectivity” because of his recent PNAS paper showing that local-scale plant biodiversity around the world isn’t changing. I’m confess I’m unclear exactly what “objectivity” means here. But if what’s meant is something like “a thoughtful, knowledgeable, self-aware person who carefully weighs evidence and arguments in order to come to conclusions he’s prepared to defend”, then Mark Vellend is one of the most objective people I know.

UPDATE #2: Just to continue the thought from the previous update, and highlight some things I just said in the comment thread. I’m well aware that Maier’s views are going to be controversial and unpopular. And I’m well aware that some people didn’t like Mark’s recent PNAS paper (though personally I’m not sure why, as I didn’t see any serious technical flaws or wild overstatements in it myself…). It’s for those reasons that I hesitated to post on this; there’s a higher than usual risk of the discussion degenerating into an unproductive shouting match. But I decided to post, as the issue is important, and interesting to me. And because in the past we’ve always been able to trust our commenters to vigorously debate controversial issues without descending into personal attacks, questioning of motives, proof by authority, etc. So let’s all make sure we live up to the high standards set by previous comment threads on controversial topics. One of the best uses of blogs is as a forum for vigorous but productive debate.

UPDATE #3: And just in the interests of being up front about my own views here: I think Maier has an excellent point about general, blanket arguments for the value of “biodiversity”. Insofar as we’re conserving biodiversity in order to achieve some instrumental goal (like maximize the level of some ecosystem service), then I think we’re much better off making narrower, case- and context-specific arguments. Arguments like “In order to ensure provision of ecosystem service X in system Y, we ought to conserve high levels of biodiversity, because alternative ways of ensuring provision of high levels of service X would be more expensive, and because doing so will not have any negative side effects.” If there are any good general, blanket arguments for conserving biodiversity always and everywhere (or preventing species invasions always and everywhere, or pursuing any particular conservation policy always and everywhere), then I suspect they’re based on intrinsic values rather than instrumental values. There’s a large philosophical literature on that, of course, but it’s not one I know well.

Which pre-blog ecologists and evolutionary biologists would’ve made the best bloggers? (UPDATED)

Which ecologists and evolutionary biologists from before blogs existed (so, not that long ago!) would’ve made the best bloggers? A few opening bids:

Steven Jay Gould is an obvious pick. And while I disagree with many of his opinions, he totally would’ve been a great blogger. Prolific, opinionated, entertaining.

John Lawton used to have a column in Oikos called “A View From the Park” (because he was based at Imperial College London’s Silwood Park campus). His essays for that column were like lengthy blog posts. “A View from the Park” is part of what inspired me to start blogging, back when I was an editor at Oikos, and I believe it was part of Brian’s inspiration as well.

Similarly, Bob Holt recently did a series of invited essays for an Israeli journal that would’ve worked equally well as blog posts. And of course, it’s not too late for Bob and John to start blogging if the mood strikes them!

On the evidence of this wonderful little essay, John Maynard Smith would’ve been a fine blogger. (I’m presuming that, given the opportunity provided by blogging, he could’ve cranked out this sort of thing on a regular basis.)

As suggested by the comment that inspired this post, Thomas Henry Huxley (“Darwin’s bulldog”) definitely would’ve had a blog, and it would’ve been awesome. (On the other hand, Darwin himself was far too cautious to have been a blogger. He always wanted to be absolutely sure of his ground before he published anything).

Who have I missed?

UPDATE: In the comments and on Twitter, Dan Janzen is a popular suggestion, and rightly so. Indeed, as noted in the comments, he preceded John Lawton in having a column in Oikos, and so in a sense had a blog before blogs existed. R. A. Fisher is another name suggested on Twitter. But while he certainly was capable of eloquent writing, and willing to take controversial positions, he wasn’t always noted for his ability to explain himself clearly. Think of how long it took people to figure out what the heck he was talking about when it came to his “Fundamental Theorem” of natural selection. So I’m not sure if Fisher would’ve been a great blogger or not.

Friday links: scientific publishing is unfixable, “Big Replication”, new ecology teaching videos, and more

Also featuring this week: the changing landscape of blogging, Darwin’s pet tortoise, Richard Feynman’s comments from beyond the grave on post-publication peer review, and more…

From Meg:

This video is a really excellent introduction to community ecology, trophic cascades, ecosystem engineers, and indirect effects. I added it to my list of videos that can be used while teaching ecology. I need to do a more thorough update of that list. If you have other suggestions for videos to add to that list, please let me know!

The Royal Society is hosting a Wikipedia edit-a-thon on March 4^th, with the goal of increasing the number and quality of entries for women scientists on Wikipedia. The article indicates that a secondary goal is to get more women editing Wikipedia (it reports that currently just around 9% of editors are women). I have very slowly been working on some of the pages for women ecologists (emphasis on “very slowly”), but perhaps I can try to set aside some time on March 4^th to make some more progress on some of them – and hopefully some of you will, too!

And, just for fun, as my videos for teaching ecology post indicates, I’m a big fan of David Attenborough, so I knew I had to watch him narrating a curling event from the Sochi Olympics!

From Jeremy:

I don’t ordinarily plug individual papers in ecology journals, but this one promotes a cause near and dear to my heart so I’m giving it a shout-out. Writing in American Naturalist, Mark McPeek discusses a neglected classic in the theoretical literature on species coexistence: Levin 1970. Modern coexistence theory is way more general than the Lotka-Volterra model or MacArthurian resource overlap models, and way more precise and useful than verbal niche models like Hutchinson’s. And arguably, modern coexistence theory starts with Levin 1970. And not only is Levin’s paper a classic, it has an unusual and amusing epilogue.

The scientific publishing system will never be “fixed”, for the simple reason that people disagree on what’s valuable science (or good science, or interesting science, or important science, or novel science, or high-impact science, or any other metric you care to name). And if you think the “fix” is to just publish everything technically sound and let readers decide what’s valuable, well, click through and think again. Echoes my own thoughts. (ht Denim and Tweed)

But publication bias is fixable. At least, so says this new preprint, which suggests a clever way to obtain an unbiased estimate of a true effect size, solely from knowledge of reported sample sizes and P values from published studies. I’ve only skimmed it, but unless I missed some big problem (did I?), it seems quite clever and useful. (ht a commenter on Andrew Gelman’s blog)

Welcome to the era of “Big Replication” (ht Andrew Gelman). A bunch of psychologists (the “ManyLabs” project) haphazardly picked 13 key studies of a range of psychological phenomena, all of which could be tested with simple experiments that could be administered online. And they replicated all of them with a frickin’ massive sample (total of over 6000 subjects from all over the world). And just as important as the massive sample was the pre-planning: the group decided every aspect of their data processing and analysis protocol in advance, pre-registered it, and stuck to it, so there’s no possibility that the results reflect researcher degrees of freedom or p-hacking or whatever. Excellent, accessible discussions here, here, and here. Turns out some effects replicate while others don’t (as many psychologists expected, social priming effects, like “showing Americans a US flag makes them more patriotic”, are just statistical flukes at best; perhaps they should’ve bet on this!). Another interesting take-home for me was that there’s surprisingly little variation in effect size across countries or cultures. That to me is one of the best arguments for this sort of work. You’re not out to prove the original study wrong, you’re out to discover if the original result is robust to background variation. As one of the linked posts notes, there’s no reason why the same approach can’t be taken to test novel hypotheses:

Here is how this might go.

(1) A group of researchers form a hypothesis (not by pulling it out of thin air but by deriving it from a theory, obviously).

(2) They design—perhaps via crowd sourcing—the best possible experiment.

(3) They preregister the experiment.

(4) They post the protocol online.

(5) They simultaneously carry out the experiment in multiple labs.

(6) They analyze and meta-analyze the data.

(7) They post the data online.

(8) They write a kick-ass paper.

Ecologists were way ahead of psychologists on this, in the form of the NutNet project.

Chris Bertram of Crooked Timber (a very long-running and successful academic blog) on the changing landscape of blogging. Relates to issues we’ve discussed. Stick with it until the end for a funny analogy to the music business.

Ecologists don’t call themselves “ecological biologists”. So why don’t evolutionary biologists call themselves “evolutionists”?

Good advice for pre-tenure faculty members on how to be strategic about your “service” obligations.

Goodbye to all that: a journalist and editor explains why he left academia despite having long been sure that he wanted to be a history prof, and having been on track to achieve that goal. Good thoughtful essay. Not just about the academic job market, but also about risk-taking and the desire to write for a bigger audience. Resonates with one of our best guest posts ever.

Advocates of post-publication review over pre-publication review usually emphasize how only two or three people (whom you don’t know!) read a paper before it’s published, while lots of people read it after it’s published. Surely that large post-publication “sample” is better, right? Well, not if those post-publication readers are just skimming, or just reading the abstract, or just glancing at the figures (which they are, mostly). As Richard Feynman once wrote in a closely-related context:

This question of trying to figure out whether a book is good or bad by looking at it carefully or by taking the reports of a lot of people who looked at it carelessly is like this famous old problem: Nobody was permitted to see the Emperor of China, and the question was, What is the length of the Emperor of China’s nose? To find out, you go all over the country asking people what they think the length of the Emperor of China’s nose is, and you average it. And that would be very “accurate” because you averaged so many people. But it’s no way to find anything out; when you have a very wide range of people who contribute without looking carefully at it, you don’t improve your knowledge of the situation by averaging.

Charles Darwin’s pet Galapagos tortoise “James” was rediscovered earlier this month. Well, its shell was, in a storeroom at the Natural History Museum in London. Will have to file this tidbit away for next year’s Darwin Dinner quiz.

And finally, this has nothing to do with ecology, but it’s really cool (and brings back fond memories for me): historical photos of London merged with modern-day photos of the same locations. The Museum of London has published an app that lets you create your own.

Do you have a lab philosophy?

Do you have a lab philosophy? By “lab philosopy”, I mean a formal, written philosophy that you share with lab members (and, perhaps, prospective lab members) that lays out what they can expect of you and what you expect of them? And, if you have one, is it online?

The most well done philosophy I’ve seen comes from the neuroscience lab of Kay Tye. She clearly put a lot of thought and effort into it, and it’s an impressive document. Some people may disagree with some of the specifics – indeed, there was lots of discussion on twitter about some of the specifics several months ago, and she appears to have updated it based on that dialogue. But, regardless of whether you agree with everything it says, it is clearly very good in terms of laying out a set of expectations for all members in the lab.

The discussion surrounding Tye’s philosophy statement, combined with emailing with prospective students for my lab, got me wondering if it would be useful to write up a philosophy statement for my lab. I tried finding ones online for ecologists, but couldn’t find any. (I did, however, find this information for prospective grad students on Andrew Read’s page, which is certainly a clear – and entertaining, in my opinion – statement of what he expects of his students.) A few people said on twitter that they have them (one of which is available via dropbox), but don’t have them posted on their webpage.

The challenge, of course, is that writing such a document is very difficult. Getting the content and tone correct would be really hard and take up a huge amount of time, I think. Instead, I’ve found that I end up pointing prospective students to some of my blog posts that address things that they ask about (this one on system-based research is probably the one I referred the most prospective students to this year; my post on the importance of working efficiently – rather than huge hours – might be one that comes up often in the future). Though my blog posts leave out other very important things that students should know before joining the lab, such as:

If I come up with a "lab philosophy" statement, it might need to include that I believe in using 1 space after a period, not 2.—
Meghan Duffy (@duffy_ma) February 21, 2014

I would also need to include the most important lesson I teach people in my lab, which is that you should not hyphenate adverbs. Totally-unnecessary.

Does your lab have a formal, written lab philosophy? Is it shared online and/or with prospective lab members? Do you think there’s value in having a written lab philosophy? I’m especially interested in hearing from grad students and postdocs about whether they do/would find such a document valuable.

Post-publication “review”: signs of the times (UPDATE: link fixed)

A couple of recent incidents have gotten me thinking about how norms of scientific criticism are changing. With one side effect being nasty fights.

Here’s the first incident (ht Retraction Watch). Computational biologist and blogger Lior Pachter has a series of posts strongly criticizing the network research of A.-L. Barabási, Manolis Kellis, and others. This is very prominent work, reported in a long string of papers in Nature and other top-tier journals. In particular, Pachter digs into one recent paper by Kellis and co-authors in great detail and argues that it’s “nonsense”. And he goes further and accuses Kellis and co-authors of misconduct: being intentionally-vague about their methods in order to get the paper into a leading journal and then trying to cover their tracks by “correcting” the paper post-publication. Pachter has taken to his blog because his correspondence with the authors left him unsatisfied and journals rejected the comments he submitted. A reply from Kellis is here, and a reply to the reply is here.

The second incident is the recent dust-up over some prominent papers on dinosaur growth rates. Nathan Myhrvold, a millionaire dinosaur hobbyist, has strongly criticized a series of papers by paleobiologist Gregory Erickson and colleagues for numerous omissions of methodological details that make it impossible for others to fully reproduce the data analyses. And also for various mistakes (including two different figures in which the fitted curve shown in the figure doesn’t come close to matching the equation provided in the figure legend). Myhrvold has described his concerns publicly and in great detail in a peer reviewed paper, and apparently has had private correspondence with Erickson. But he also took his criticisms to the New York Times, and in an interview with the Times said the errors in Erickson’s work were “consistent with scientific misconduct”. And while some of the co-authors of the criticized work have said that Myhrvold has identified some important issues, Erickson’s public response has been limited to noting that his papers were peer-reviewed.

I’m not qualified to judge the first dispute, and while I do feel I’m qualified to judge the second I don’t really want to get into that.* I just found both incidents to be striking signs of the times. Whatever the rights or wrongs of these two particular incidents, this kind of incident is going to become more common, I think. Because like Bob Dylan sang, the times they are a changin’. What follows are my admittedly anecdotal impressions and probably-rash generalizations. I’m tossing them out there in the hopes that others will chime in with their own impressions.

It seems like more and more people are increasingly demanding about the level of detail in which published work should be documented. Are increasingly less likely to see any mistake or omitted detail as minor. Instead taking the view that all mistakes and omissions are serious by definition, or at least that they should all be corrected and their seriousness left to the reader to judge. Are increasingly disinclined to trust authors, editors, or the pre-publication peer review process, preferring to at least have the in-principle option of checking everything themselves.** Are increasingly disinclined to care about voicing post-publication criticisms through the “proper” channel (i.e. by submitting formal comments for peer review). Instead taking the view that once something’s published, it’s fair game. And that peer review of comments is at best too slow, and at worst functions as a way to suppress legitimate criticism. So that it’s better to just publish criticisms of peer-reviewed papers in whatever venue seems most convenient and visible. And are increasingly comfortable with strongly-worded language, and disinclined to care about tone. Indeed, often viewing worries about tone as a distraction from discussion of substantive issues.***

I’m actually not sure if more and more people feel this way, or if thanks to the internet people who feel this way are merely more visible and vocal than they used to be. Could be some of both. In any case, I do think times are changing, and that they’re going to keep doing so. So if you don’t like the way things are changing, well, for better or worse I think you’re going to end up on the wrong side of history in the long run.

We’re living through a culture clash, I think. People who feel more or less as described above certainly can give reasons for thinking that science would be better off if everyone felt the same. And conversely, people who don’t feel that way can give reasons for thinking that science will be better off if everyone felt as they do. And I actually think everyone’s reasons (including mine!) mostly are post hoc rationalizations. For instance, people who are used to scientific communication working a certain way would like it to continue working in that familiar way. And people who are used to other things working a certain way (e.g., they’re used to the informality and openness of social media) would like scientific communication to work in that familiar way. Put it this way: do you often see people arguing that scientific communication should change in a way that they personally would hate? And culture clashes are a recipe for arguments.

But having said that, just because people’s proffered reasons are post hoc rationalizations doesn’t mean they’re merely post hoc rationalizations. It’s by giving reasons that we can (sometimes) get others to understand where we’re coming from and appreciate our own point of view. Even if we don’t ultimately hold that point of view for rational reasons. And through trying to rationalize your point of view and thereby justify it to others with different preferences, you’ll often come to realize that you are rationalizing. Which is a useful thing to recognize. So if indeed there is a culture clash here, I don’t think that means there’s no hope for mutual understanding or compromise.

On balance, I think the trends I’ve described are good for science, but my feelings are somewhat mixed. Which isn’t surprising, since I’m something of a ‘tweener when it comes to novel ways of communicating and criticizing science. So here are my post hoc rationalizations for my own views.

I think the bar always gets raised rather than lowered in science. Our standards for everything go up over time, and that’s good. Right now, the bar is getting raised on the detail and completeness with which authors are expected to report their methods. I think that can only be good for science.

I also think it’s good for science if we all get more comfortable with mutual criticism and vigorous debate. As Andrew Gelman recently remarked, the only way for the vaunted self-correcting nature of science to work is if we actually allow corrections. Various lines of evidence show that science self-corrects more slowly than it could or should (e.g., this quite interesting paper I just found [UPDATE: link fixed]). And I don’t like the view that any post-publication criticism of peer reviewed work necessarily is nitpicking, and in any case is never worth publishing because any seriously flawed work will be ignored and so needn’t be publicly criticized. I think that view (which is common though far from universal) is based on false premises, and holds back scientific progress. And I don’t think we need to worry about most authors having to deal with a barrage post-publication criticisms. The vast majority of papers aren’t read often enough or carefully enough to attract any post-publication criticism. (Which still leaves significant concerns about the minority of authors who do have to deal with such criticisms; see below)

And I think it’s a good thing on balance if people have less ability these days to dismiss criticisms on the basis of the critic’s credentials or the venue in which the criticisms were published, or any other basis besides the substantive content of the criticism itself. Science is supposed to be all about logic and evidence. Logic and evidence don’t care what credentials you have, or the venue in which they’re published, or etc.

On the other hand, I do think we need some professional norms or other about post-publication review. Because otherwise post-publication review isn’t going to work well. Pre-publication review works because everybody agrees to participate voluntarily. It’s part of what you’re signing up for when you choose to do science. But while at some level everyone’s aware that once they publish something others are free to read and react to it as they see fit, lots of people don’t feel like they signed up for post-publication “review”, at least not in all its forms. I mean, do you feel like being accused of serious mistakes and possible misconduct in the New York Times just “comes with the territory”?

Pre-publication review also works because it’s a system with formal and informal rules, not a free-for-all. For instance, it’s private, which saves people from the embarrassment of having their mistakes pointed out publicly, thereby encouraging them to participate. And there’s an editor involved, which helps ensure (though of course doesn’t guarantee) that referees and authors address each other professionally, and obliges authors to respond point-by-point to referees rather than just ignoring them or whatever. The fact that it’s a system with rules is a big part of what encourages people to participate voluntarily. People know what they’re signing up for.

Which is why I don’t think you can just tell people to “have a thick skin” or “don’t take it personally” when it comes to post-publication review. As this very good piece about another recent post-publication review dust-up points out, post-publication review these days seems to slip quite easily into personal criticism of the author. Which isn’t something you should be expected to “have a thick skin” about. Nobody should be expected to “have a thick skin” about implications or accusations that they’re incompetent or unethical. And further, sufficiently-strong criticism of someone’s work is awfully hard to distinguish from an attack on their competence or integrity, whether or not you preface it by saying “Don’t take what I’m about to say personally” (again, see the very good piece I just linked to).**** Further still, just as authors sometimes make mistakes, sometimes because they’re incompetent or unethical, so do critics. It seems like advocates for “anything goes” in post-publication review presume that critics are always right. They’re not. So what’s needed isn’t for everyone to have a thick skin. A thick skin is what you need to survive perceived or actual attacks. What’s needed are new norms and practices that everyone buys into.***** So that, as far as reasonably possible, people don’t get attacked and don’t feel they’ve been attacked, and so don’t need to have thick skins.****** For instance, in the post linked above, Andrew Gelman offers some suggestions on changes to how journals publish criticisms. In the absence of widely-accepted norms and practices, effectively critiquing the published literature is a really tricky thing to do.******* This isn’t (just) about sparing people’s feelings or protecting their reputations–it’s about creating a post-publication review system that actually works. That actually does effectively correct the scientific record, and that is seen by all to do so.

I don’t have any answers here–I don’t know what the new norms and practices will be, or should be. And I don’t know how to create a set of norms and practices that everyone would buy into and happily participate in. Attempts to codify rules of good commenting behavior on the internet are infamous failures. So we may have no alternative but just to let things sort themselves out through some sort of quasi-evolutionary process. But until we have wide agreement on new norms and practices, expect more dust-ups like the two described above.

*For the record, I’ve read Myhrvold’s paper carefully, and I feel like I’m competent to judge it because his concerns all relate to data processing and statistical analysis. Assuming that he’s truthfully described what he did (and he does seem to have described it in sufficient detail for anyone to reproduce it), then I think he has indeed identified some mistakes that really ought to be corrected. And while the mistakes may not alter the broadest qualitative conclusion (“dinosaurs grew fast”), the quantitative changes seem big enough to me to be more than trivial. As for the various methodological omissions he’s identified, I suspect that this is an example of changing reporting standards. The work was originally reported in enough detail to satisfy the referees that it was done competently. But like an increasing number of readers, Myhrvold wants to see methods reported in sufficient detail so that others can exactly reproduce all of the results without any need to contact the authors.

**E. D. Deming supposedly said “In God we trust; all others must bring data.” That’s never been more true and it’s going to get truer.

***Although there’s an irony, noted by a commenter on Pachter’s blog, that people who feel comfortable expressing even minor criticisms in strong language sometimes also complain about authors overhyping their results. Leaving one wondering why it’s not ok to hype results, but is ok to hype criticisms of those results.

****I’m probably one of the guilty parties here. Probably one mistake I’ve made at times in the past has been phrasing criticisms of people’s ideas so forcefully that it sounds like I question the competence of anyone who disagrees with me. Even though I’m usually careful to say that I’m not questioning anyone’s competence.

*****Some would disagree. For instance, the comments on that piece I just linked to include the view that not only will scientific debates get personal sometimes, but that they should get personal. I don’t buy that at all. But the fact that that view is even out there is another sign of the times.

******A bit of a thick skin is always going to be needed, no matter what our norms and practices are. Some people find pre-publication review quite hard to take. Which is a feeling they have to learn to get over, since pre-publication review is standard practice.

*******”Effectively” is a key word here. If you want your post-publication critique to actually convince people rather than falling on mostly-deaf ears, you can’t just voice it however you want and then write off anyone who doesn’t like your tone as a wimp. If you want to be convincing, you have to write for the audience you have, not the audience you wish you had. See this old exchange of comments.

Friday links: how to make the most of grad student recruiting visits, Darwin doodles, and more

From Meg:

Darwin’s kids drew on the Origin of Species! The pictures are actually quite well done. My daughter contributes to my work sometimes, as tweeted here:

My three year old's contribution to expt design (from MLK day) pic.twitter.com/k5EAtJM0ru

— Meghan Duffy (@duffy_ma) January 24, 2014

but she has a ways to go before she’s at Darwin’s kids’ level. (It’s okay for me to say that, because she could point out that I have a ways to go before I’m at Darwin’s level. ) ht: Ed Yong

This article in the ASLO bulletin by Hansen et al. (subscription/membership required) actually has data on the proportion ecology PhDs that obtain tenure-track academic positions, as well as on how long it took from obtaining the PhD to obtaining a tenure-track position. As I wrote in my post on training students for non-academic careers, I wasn’t aware of ecology-specific data, so I was very interested when I saw this article. Some of the quick stats are: 1) a 77% increase in ecology PhDs awarded from 2003 to 2010, 2) the median time elapsed between earning a PhD and getting a TT position has actually decreased since the 1980s (when it was 4-7 years); with it taking about 3 years now (as it has more-or-less since the 90s).

This post from Tenure, She Wrote talks about the importance of daycare to remaining productive in academia after having kids. In some ways this sounds obvious, especially if you have attempted to work on a manuscript while watching a fussy baby. But the most interesting point to me was this one: “From what I observed in Germany and Switzerland, it seemed that access to luxuriously long paid family leave was not unequivocally good.” The problem is that, with “luxuriously long” maternity leave, women can be judged harshly for returning to work before they were required to. (Indeed, a friend of mine reported sneaking into the lab at odd hours to get some work done while her child was still an infant.) But, if you take the full leave, it is a long time to be away from academia, and can make restarting hard. Overall, I agree that the leave in the US is often incredibly short, but, at the same time, simply having a long leave is not necessarily going to make things better for women in science. And, unquestionably, having quality, affordable childcare available will make life better for women in science.

From Jeremy:

Good advice for prospective grad students about to attend on-campus recruiting weekends with other prospective students. I’d chime in with additional advice if I could, but I never attended one of these recruitment weekends. Does that make me unusual? All of my visits to prospective supervisors were just me on my lonesome. I did my homework, researched potential supervisors, contacted them early in the fall, and later in the fall or the winter they brought me out to visit their labs. I’ve always thought that that was mostly how things worked in the US and Canada, and that places with recruiting weekends were the exception rather than the rule in ecology and evolution. Am I wrong about that (or was I right back in the Dark Ages when I was a prospective grad student, but now things have changed?) I do think there are advantages to individual visits to specific labs. Ecology and evolution programs mostly don’t do lab rotations, so it’s incumbent on you as a prospective student to figure out who you want to work with before you even apply, much less before you do a site visit. Honest question: do recruitment weekends encourage prospective ecology and evolution students to see themselves as applying to a department or program, much as they previously applied to an undergraduate college? When in fact they should be seeing themselves as applying to work with a specific supervisor, with the rest of the program being an important but secondary consideration?

Good advice on how to get it in gear and write your dissertation.

We missed R. A. Fisher’s birthday on Monday. To make up for it, here’s a potted history of how Fisher revolutionized statistics as an outsider, and overcame active resistance from the statistical establishment at the time.

Ecology is infamous for having lots of jargon, often defined at least somewhat differently by different people. Nathan Lemoine and some other ecology graduate students are conducting a survey of how ecologists define key terms. I don’t know if “type I functional response” is among the terms included in the survey, but maybe it should be.

Rutgers University has suspended famed evolutionary theorist Robert Trivers with pay, apparently for an issue to do with Trivers’ teaching assignment. I have no more details beyond what’s in the article. (ht Retraction Watch)

The rise and fall of cap and trade.

Hoisted from the comments:

See the comments on this post for lots of great recommendations for scientific history and biography books, many of them to do with the founding of particular fields of ecology and related disciplines.

Book review: The Silwood Circle

This is the story of “the Silwood mafia”.

At least, that’s the term I first heard, during my time as a postdoc at the NERC Centre for Population Biology (CPB), located at Imperial College London‘s Silwood Park campus. The CPB no longer exists, but the time it was one of the very best places in the world to do population and community ecology. I arrived there knowing something of the history of the CPB, and of the Silwood Park campus more broadly. For instance, some of the pioneers of modern ecology, folks like Mick Crawley and Mike Hassell, were still on the Imperial College faculty and pursuing active research programs (some of them are still there today). And others like Bob May and John Lawton still maintained close ties to Silwood. But it wasn’t until after I arrived that I fully realized just how important and influential the leading Silwood-associated ecologists had been in shaping the development and direction of ecology, both in Britain and internationally. And it wasn’t until after I arrived that I realized that this influence inspired a fair bit of jealousy and resentment in some quarters. To the point where some outsiders began referring to them as the Silwood mafia.

In The Silwood Circle, Imperial College London historian Hannah Gay charts the careers of a small, close-knit, informal “core” group of ecologists with similar but non-competing research interests: Richard Southwood (the informal leader of the group and a mentor to the others), Robert May, Gordon Conway, Mike Hassell, Roy Anderson, Mick Crawley, John Lawton, John Beddington, John Krebs, and David Rogers. If that sounds like a “Who’s Who” list of many of the most famous senior ecologists in the world, well, that’s no accident. Gay has a longstanding interest in generational groups and how they make their way in science, and how the social dimension of science affects its direction. Given those interests, she could hardly have picked a better case study. This isn’t history by the winners–but it’s history about the winners.

The backdrop of the story is the changing face of ecology as a whole. In the first half of the 20th century, ecology was very much a peripheral discipline, widely seen as the domain of amateurs if indeed it was noticed at all. By the end of the century, it was widely seen as much more central, and had become professionalized. In the 1950s and 60s, growing environmental awareness created a demand and opportunity for professional scientific advice on a broad range of ecological topics. And pioneers like Hutchinson, Slobodkin, MacArthur, and Holling had begun to point the way towards a new ecology, placing a much heavier emphasis on mathematical theory, hypothesis testing, manipulative experiments, and rigorous statistical analysis. The members of the Silwood circle recognized that opportunity early and seized it. Their work blazed the trails that many other ecologists subsequently followed.

A theme of the book is the interplay of chance and determinism in the fates of scientists and their ideas. The Silwood circle emerges as Exhibit A for the truth of the aphorisms that “chance favors the prepared mind” and “it’s best to be good and lucky”. For instance, Mick Crawley was lucky to hear about an upcoming opening on the faculty at Imperial College. He heard about it at a pub lunch with John Maynard Smith and Mike Hassell, an illustration of the importance of social connections. But it wasn’t blind luck that had caused him, years earlier, to do a Ph.D. involving a lot of statistics and computer programming. That was a very unusual choice at the time–but it was a prescient one. Silwood was ahead of the curve in terms of hiring quantitatively-oriented ecologists, but quantitative skills soon were much in demand throughout ecology. Through both the example of his own research and several textbooks, Mick Crawley taught a whole generation of ecologists how to think quantitatively and statistically. And it wasn’t luck that caused him to say, as soon as he heard about the opening, “I could do that job”. Put another way, to say that Crawley had a “lucky opportunity” is to say something about him as well as his luck, since for someone else without Crawley’s skills, hearing about that job opening wouldn’t have been an opportunity at all. Or consider physicist Robert May. He was fortunate to get the mentoring and advice he did, which helped him meet some very good biologists. But those meetings would’ve come to nothing if May hadn’t had some really good ideas and been very good at linking those ideas to big questions biologists wanted to answer. Stories like this come up throughout the book.

Correctly, Gay doesn’t just write off the members of the Silwood circle as naked careerists, or the multifarious formal and informal ways in which they helped each other as a nepotistic conspiracy. For instance, she notes the importance of interpersonal trust for enabling people to make decisions in the face of imperfect information and the inherent unpredictability of the future. For example, later in their careers several members of the Silwood circle became trusted government advisers. And in most cases that trust was justified, though Gray discusses the controversy over an arguable exception (the handling of the vCJD crisis, in which Southwood was involved). And if the joke that the title “F.R.S.” stood for “Friends of Richard Southwood” rather than “Fellow of the Royal Society” has more than a little truth to it, well, ambitious scientists have always worked together to advance their shared causes. Including by nominating one another for awards like membership in the Royal Society. The Origin of Species would hardly have been noticed had Darwin’s many friends not worked behind the scenes to support it, and one another. For better or worse, science is done by people, and so you can’t effectively support ideas without also supporting the people who hold them.

Gay notes that members of the Silwood circle shared various traits, which she tentatively suggests help explain their success. For instance, all were keen naturalists as children–collecting insects, watching birds, etc. I found this interesting because they subsequently took ecology in a direction that (incorrectly in my view) is seen by some as inimical to natural history or even field work. For instance, May’s landmark book Stability and Complexity in Model Ecosystems is famously abstract and divorced from system-specific biological details. And the Ecotron controlled environment facility that John Lawton spearheaded at Silwood was criticized for purportedly being unnatural, unhelpful, and diverting resources from more traditional ecological work (Lawton 1996). Gay suggests that other shared traits include ambition, competitiveness (May in particular is famously competitive), a heightened yet selective awareness of what others are doing and thinking, and a greater than usual willingness to make evaluative judgments about other scientists and their work. For instance, members of the Silwood circle mostly didn’t like the systems ecology work of the Odums.

The book also is a case study in how to build a topnotch research group that’s greater than the sum of its parts. Tellingly, it was not a matter of the people in charge just trying to hire clones of themselves. Richard Southwood didn’t himself pursue the same sort of research as many of the people he hired or mentored. Southwood was no modeler or statistician, for instance. But he was very good at spotting talent, and at nudging people in productive directions.

Gay also tries to give the reader a sense of the science itself. What questions, fundamental and applied, were circle members trying to address, how did they address them, and what was new and important about their approaches? In this I think she’s fairly successful, but it’s tough for me to judge since as a professional ecologist I wasn’t the intended audience for these explications.

The book isn’t hagiography. For instance, Gay is matter of fact about how the Silwood circle included no women. She talks at greatest length about Southwood’s own attitudes towards women. He seems to have viewed women as capable of playing only a supporting role in the advancement of science.

One thing I wonder is whether it would’ve been possible for the Silwood circle, or its equivalent, to achieve equal success in the US. Back when I first heard the pejorative term “Silwood mafia”, I remember thinking how weird it was that a small cohort of people associated with one university campus could ever attain so much influence that they’d be seen as a “mafia”. I mean, a place like UC-Davis has lots of hugely-famous and influential ecologists–but nobody ever talks about a “Davis mafia”! Don’t get me wrong, there are schools of ecological thought associated with particular places in the US. For instance, according to Meg, people sometimes talk about the “Yale school” and “Wisconsin school” of limnology. But the US just seems like too big a pond to ever be dominated by any one “school” of fish, as it were.

I probably read the book with more interest than most, because of my personal connections to Silwood and my pre-existing admiration for several members of the Silwood circle. But I think the book will be of broad interest to many. Certainly, it’s unusual and intriguing read a serious, detailed socio-historical study of people whom you’ve met, and who are mostly still alive. And it was interesting to hear someone articulate not just a description of how science works as a social activity, but why it works that way and why that way of working can often be good for science (rather than bad, or merely inevitable rather than good or bad). You may find it slow-moving in places. Gay describes in great detail the career trajectory of every member of the circle, down to the level of key conferences they attended, specific projects on which they worked, their movements from one job to another, etc. And there’s no juicy gossip and little in the way of personal anecdotes. Gay’s goal was to produce a serious study of the sociology of science, not dig dirt or even write potted biographies. I think she succeeded, and I recommend The Silwood Circle.

*Dick Southwood, the father figure of Silwood ecology, occasionally referred to the group he mentored as the “Silwood Mob”. He meant it affectionately.