Why teaching Intro Bio makes me think we need to radically change qualifying exams

When I first arrived at Michigan and began teaching Intro Bio, the course had four exams. In that first semester, I added in clicker questions. Since then, we have added in frequent quizzing, so the students now have four exams, plus two quizzes a week (completed before coming to class), plus clicker questions in class. We have all of that because we know that frequent testing improves student learning. (Here’s one review, here’s another, and here’s a summary of the changes we made in Intro Bio and their impacts on student performance.) As a side bonus, when the testing is low stakes (as with the quizzes and clicker questions), students get those learning benefits without paying a cost in terms of increased anxiety. Given all that, I would never consider changing the format to one where we have just a single, pass/fail, high stakes assessment at the end of the semester.

Now, let’s consider graduate prelim/qualifying exams.

Newly-hired tenure-track N. American asst. professors of ecology are 59% women (no further updates)

(Last update: June 3, 2018. No substantial changes since the original post, only very very minor quantitative changes. A recent update adds a link and brief discussion of new data on the gender balance of ecology PhD recipients and postdocs. Those data provide useful context for the data I compiled.)

Like last year, this year I once again quantified the gender balance of newly-hired tenure-track asst. professors in ecology and allied fields at N. American colleges and universities. I also conducted a poll asking readers what they expected me to find. Click the link to the poll for details on how I compiled the data, and why I went with a gender binary even though that’s not ideal.

I’ll present the results first, then the poll results, then some discussion.

Warning: long post ahead. That’s because I’ve tried to discuss the results thoroughly and carefully, and to anticipate and address questions that readers are likely to have. You really should read on, but if you just want the headline results:

• 59% of tenure-track asst. professors of ecology hired in N. America in 2016-17 are women. Combined with last year’s data, it’s 57% women over the last two years. If anything, that’s probably a slight underestimate, for reasons explained below. This is good news!
  
• Ecologists as a group remain unaware of this; many think recent faculty hiring in ecology is <50% women.

Hardly any ecology faculty jobs are filled by internal candidates. And you can’t identify the ones that will be. (UPDATED)

If you’ve ever looked at the ecoevojobs.net faculty jobs board, you’ve probably seen speculation that position X has an internal candidate, the implication being that others maybe shouldn’t bother applying because the internal candidate will have an edge or even be a shoo-in. Sometimes, the speculation is not merely that a strong internal candidate exists, but that the position is intended for the internal candidate, so that the entire search is a formality with a pre-determined outcome.

But internal candidates have factors working against them as well as for them. As illustrated by the fact that they don’t always get the job–even when they’re confident they will! For instance, see here, here, and here. Those are anecdotes, though, so it’s hard to say if they’re typical. How often are internal candidates hired for ecology faculty positions? And is there any reliable way for outsiders to identify positions for which internal candidates will be hired?

According to the data I’ve compiled, the answers to those questions are “hardly ever” and “no”.

Ask Us Anything: the demand for field biologists

A while back, we invited you to ask us anything. Here are our answers to our next three interrelated questions, from Jeff Hean plus a related fourth question from Andy Park:

1. Why are biologists paid so little compared to other fields of science and the private sector?
2. Why do the majority of advertised research positions, particularly in N. America and Europe, seem to require a modeling component these days? Especially when so much baseline empirical data still needs to be collected?
3. In my personal experience, field biologists don’t make good modelers, and vice-versa. Do field biologists still have a place in ecology, in light of the high demand for young scientists who can “do it all”?
4. Can we really call someone a biologist if their training has failed to teach them any taxonomic skills whatsoever?

Who should be senior author on papers resulting from collaborations between multiple research groups?

I am pretty much through with revisions to my manuscript on authorship, with one exception. One of the reviewers is (quite reasonably) pushing me to make a stronger recommendation about how authorship decisions should be made in the increasingly common case of collaborations between groups. But, of course, this is a tricky issue, and I’m waffling on what exactly to recommend. This blog post is me trying to work through that, and looking for feedback at the end. I’m quite interested in hearing how others think decisions about authorship should be made when multiple groups collaborate substantially on a project!

I’ll start by recapping some of what my results, since they set up the general question. Then, I’ll give some of my thoughts on what might be a proposed solution. And, as I said above, I’ll end by asking for feedback on what I propose.

There are too many overspecialized R packages

I use R. I like it. I especially like the versatility and convenience it gets from add-on packages. I use R packages to do some fairly nonstandard things like fit vector generalized additive models, and simulate ordinary differential equations and fit them to data.

You can probably tell there’s a “but” coming.

You can’t estimate your odds of getting a faculty job from common quantitative metrics (UPDATED)

The 2016-17 ecology & evolution jobs compilation includes a spreadsheet on which anonymous job seekers can list some common quantitative metrics summarizing their qualifications. Year of PhD, number of years as a postdoc, number of peer-reviewed publications (first-authored and total), h-index, number of major grants held, and number of courses taught (not counting TA positions). Job seekers also can list the number of positions for which they’ve applied this year, the number of interviews they’ve received (phone/skype and on-campus), some personal attributes such as gender, and other information. The purpose presumably is to allow job seekers to determine how competitive they are for faculty positions.

As of Dec. 19, 2016, 73 people had listed their information. Not a massive sample of current ecology & evolution job seekers. Also surely a statistically-biased sample in various ways. But it’s many more current job seekers than anyone not currently sitting on a search committee is likely to have personal knowledge of. So I checked how well quantitative metrics like number of publications and h-index predict the number of interviews job seekers receive. For comparison, I also compiled data on the h-indices of 84 North American ecologists recently hired as assistant professors.

Faculty job seekers understandably want any information they can get on how competitive they are. But how competitive any given individual is for any given position depends on many factors, many of which are only captured coarsely or not at all by common quantitative metrics. You can’t put numbers on fit to the position, quality of your science, strength of your reference letters, and so on. So I suspect that many job seekers tend to overrate the importance to search committees of things you can put numbers on: publication count, h-index, etc. It’s an instance of “looking under the streetlight”. Hence my question: Can you estimate your odds of being interviewed for, or obtaining, a faculty position in ecology and evolution just from common quantitative metrics?

Short answer: No. For the details, read on.

A bit of good news that most ecologists weren’t expecting: recent ecology hires are gender balanced (updated periodically)

Most recent update: Aug. 31, 2017. To date, all updates have introduced only tiny quantitative changes to the original data, no substantive changes. A recent update adds an additional analysis (finite population confidence intervals). A second recent update slightly corrects the analysis of R1 vs non-R1 institutions because I originally misclassified one institution.

Recently I decided to quantify the gender balance of recently-hired ecology faculty in North America. “Recent” being operationally defined as “hired in 2015-16, or in a few cases in 2014”. Data on the gender balance of faculty is widely available only at the level of very broadly-defined fields like “biology”. Current faculty gender balance mostly (not entirely) reflects the long-term legacy of past hiring and tenure practice rather than current hiring practice (Shaw and Stanton 2012; ht Shan Kothari via the comments). And nobody’s anecdotal experience informs them about the outcomes of more than a tiny fraction of all ecology searches in any given year. So this seemed like a topic on which many people would welcome some reasonably comprehensive data. Follow the link for more details on how I compiled the data. In that old post, I also conducted a poll asking readers what they expected me to find.

Here are the answers: what fraction of recently-hired North American ecologists are women, and what do ecologists think that fraction is?

Many of you are going to be pleasantly surprised…

Poll: guess the gender balance of recently-hired North American ecologists (UPDATED; poll now closed)

Many academic fields are staffed by a male-biased mix of faculty. But the existence and degree of faculty gender imbalance varies among fields. Further, those fields often are quite broadly defined in published datasets (e.g., “biology”), which can leave many people wondering how well published data apply to their own, narrower field (e.g., “ecology”). Gender balance of academic fields also changes over time, but only slowly. Published data therefore only give you an imperfect sense of the gender balance of recent hires in your field. And personal anecdotes and experiences provide only a very small sample. Every year there are many dozens of faculty hired in ecology and closely-allied fields, but nobody hears through the grapevine about the outcomes of more than a small fraction of those hires.

So I decided to quantify the gender balance of recently-hired ecology faculty at North American colleges and universities. I’m doing it by going through this very comprehensive list of all ecology & evolution faculty positions advertised in 2015-16, and checking the university websites to identify who was hired. This turns out to be really easy in many cases, and difficult or impossible in the remaining cases (I therefore remove from the dataset). To keep things manageable, I’m skipping positions outside North America, of which there are very few on the linked list. I’m also skipping non-ecology positions, of which there are many. So not, e.g., biology, anatomy & physiology, genomics, evolution, paleontology, museum curator, science education, etc., even though some of those positions might have been filled by ecologists. But I’m defining “ecology” pretty broadly so as to include fields in which people who self-identify as ecologists often apply for and obtain positions. “Ecology” for purposes of this exercise includes wildlife management, conservation, ecological genetics, ecological physiology, evolutionary ecology, microbial ecology, fisheries, etc. My judgments on what constitutes “ecology” obviously are somewhat subjective and arbitrary, but I don’t see why that would affect the results. To focus on new faculty, I’m only looking at assistant professor positions, so ignoring the (very few) ads for heads of department, program directors, endowed senior chairs, etc. See the footnote (*) at the end for more nitty-gritty details on my procedure. UPDATE: To be clear, I’m including positions at all types of institutions, not just R1 universities. And you should do the same when answering the poll below. I’ll present the results broken down by institution type for anyone who’s curious about that.

I’ll present the results in a future post, in a sufficiently-complete form that you can go back and reproduce my work if you wish.

But before I show the results, I’m very curious what you think I’ll find. So below is a little poll. What do you think is the gender balance of recently hired North American ecology faculty? (UPDATE Nov. 10: responses have slowed to a tiny trickle, so I’ve closed the poll so that I can start analyzing the results. We already have 468 respondents–thanks to everyone who responded!)

p.s. Obviously, these data won’t tell you whether the outcome of any particular search was fair, much less whether every individual applicant for every position was evaluated fairly. And I have no way to collect lots of contextual information that you might want in order to interpret the results, such as the gender mix of the applicant pool for every position. In that future post I’ll talk more about what I think we can and can’t learn from these data.

*Failed searches are among those for which I can’t tell who was hired, so they automatically get dropped from the dataset. The difficulty of identifying who was hired mostly has to do with departmental web page design, so I’m confident that the easily-identifiable hires are a random sample of the population with respect to gender balance. A couple of times, I’ve determined that a position that I thought was ecological wasn’t filled by an ecologist; I’m dropping those cases from the dataset.  I’m being careful to remove duplicate ads from the list so that I don’t double-count anyone. I’m including some other recent (2015-16) hires that weren’t on the linked list. I learned about these either from colleagues, or by stumbling across them while checking on listed positions. A few of the hires I stumbled across might actually be 2014 hires, but I’m fine with that because those are still very recent hires. In every case so far, gender has been obvious from the person’s name and photo.

Ten commandments for good data management

Usually when I am asked to give a few words to describe myself I say macroecologist or large-scale-ecologist. And I might on other days say biodiversity scientist or global change scientist. But a lot of days I would say “ecoinformatician”. Ecoinformatics is the subset of bioinformatics that applies to ecology – that is to say informatic (data) techniques applied to ecology. Some of you may know that I spent 9 years in business before returning to my PhD. But not many know that most of what I was doing was business informatics. Helping companies understand their data. It wasn’t planned. I just have always liked seeing what the data has to tell me. But it turned out to be great training as ecology dived into informatics just as I hit graduate school.

Not surprisingly given my background, I spend a lot of time being asked to make recommendations on how to work with data. I’ve also been involved in some very large data projects like BIEN. Here I don’t want to focus on the large (often social) issues of really big projects (my slides from ESA 2015 on the next 100 years of ecoinformatics are on figshare if you’re interested). Here I want to focus on the much smaller single person or lab-scale project. This post is attempts to summarize what I have learned to be best practices over both my business informatics and ecoinformatics careers. I am intentionally going to stay tool or software agnostic. In this post I really want to emphasize a frame of mind and mental approach that can be implemented in literally dozens of different software packages. In a second post tomorrow, I will give a worked example in R since I know that has the highest popularity in ecology.