Also this week: Deborah Mayo vs. Andrew Gelman on statistical power, a new analysis of the leaky pipeline, the verjus theory of blogging, Excel=C, and more.
From Jeremy:
This is a few months old but I missed it (and embarrassingly, can’t recall if we’ve already linked to it): a new preprint analyzing the biennial NSF Survey of Doctoral Recipients, a longitudinal study that follows the career paths of thousands of STEM Ph.D. receipients from the year they received their Ph.D. until age 76 (ht the Chronicle, which has a summary). Using data for the 31,000+ people surveyed who got their doctorates from 1993-2010, the authors find that:
- 20% of STEM Ph.D.s get a tenure track position within 3 years of their Ph.D. (Though I bet that number would be lower if you restricted attention to the most recent Ph.D.s)
- The conditional probability of getting a tenure-track job, given that you haven’t gotten one yet, is highest 2 years post-Ph.D., and declines to less than 1% 10 years post-Ph.D. (Note: this is broadly consistent with survey data from ASLO that Meg’s linked to in the past, indicating IIRC that ecologists who get faculty positions mostly get them 4 years post Ph.D. or less.)
- Post-Ph.D., women are hired into tenure-track positions about 6 months before men on average. Blacks and Hispanics are hired about a year before whites on average, while Asians are hired about 2 years later than whites on average.
- Overall, women are about 10% more likely than men to obtain a tenure-track position. Blacks and Hispanic are 51 and 30% more likely than whites to obtain a tenure-track position, while Asians are 33% less likely.
- Women and Blacks who obtain a tenure-track position are less likely to get tenure than men and whites, respectively, although the sex effect disappears if you control for heterogeneity among disciplines.
- Controlling for marital status, parenthood, and their interactions with one another and with sex has complex effects that I find a little difficult to interpret, but that might make more sense to people who study this stuff. But it looks like there’s a “baby penalty”: women with children under 6 are about 15-22% less likely to get a tenure track position, and to get tenure once they’ve gotten a tenure-track position, than men or other women.
Note that the analysis doesn’t consider lots of other covariates you might want to consider, like career intentions, achievements like publications and awards, etc. So it’s not a complete analysis of the leaky pipeline (that’s probably impossible). As the authors note, it doesn’t show that there’s now “reverse discrimination”. And this sort of analysis obviously doesn’t show that incidents of sexism and racism are a thing of the past. But together with previous studies of pre-1993 cohorts, the results do suggest that, on average, academic job prospects for women, Hispanics, and Blacks have improved a lot, particularly at the hiring and pre-hiring career stages. The authors suggest that the strength of the “baby penalty” indicates a need to focus on child care, family leave, and tenure clock policies to make further progress.
Stephen Heard gave a talk on the history of scientific writing to an English department. He was nervous going in, but it turned out well.
The NSF DEB is great about using its blog to disseminate information and dispel myths about the grant evaluation process (e.g., this). I missed it at the time, but earlier this spring the NSF IOS did the same, presenting a bunch of data on the effects of the new preproposal system. See here, here, and here. Bottom line: the preproposal system is working as intended, and widespread fears about potential bad effects have not come to pass. The only big thing the posts don’t comment on directly (unless I missed it in my quick skim) is whether the advent of the preproposal system had the unintended side effect of leading to a big jump in the number of submissions and an associated drop in success rates, as occurred at DEB. Relatedly, see here and here for some commentary on the difficulty funding agencies have of managing submission volume and success rates.
Economist Mark Thoma taught some online courses and now thinks more highly of them. Here’s his list of their good and bad points. Note that many of the good points depend on students having sufficient self-motivation and study skills to figure things out for themselves. (ht Brad DeLong)
Terry McGlynn is going to stop ignoring ResearchGate. I’m still ignoring it, but with less certainty.
Andrew Gelman links to the latest developments in the ongoing Tim Hunt fight (in his p.s.). I don’t have any opinion on the Hunt incident, at least not one I’m sufficiently confident in to share publicly. I haven’t followed it closely. In general, I personally find it very difficult to sort truth from falsehood and wisdom from its opposite in these sorts of social media-driven fights. As with Andrew, this is an illustration of why I stick to blogging rather than Twitter, and why I personally prefer not to use these sorts of incidents as an occasion to comment on larger issues of unquestioned importance. But I’m sure your mileage may vary on all this. I’m just noting my own personal attitudes, which I wouldn’t necessarily expect others to share.
Oikos Blog, where I got my blogging start, hasn’t posted anything since February. RIP? I’d be sad to see it go, even though I haven’t looked at it much since I left and they switched to posting summaries of forthcoming Oikos papers (which is totally fine, it’s just not what I personally look for in a blog). I still think the original idea for Oikos Blog–all the editors would post interesting, provocative thoughts, somehow related to (but not just summarizing) the journal’s content–is well worth a go for any ambitious journal that wants to try it. But I doubt it’ll happen. Most people don’t want to write that sort of blog post even occasionally, so I doubt you’d be able to get an entire editorial board to start blogging. Much less get the board to keep it up for long enough (months at least; more likely years) to build enough of an audience to make a material difference to the journal.
This is a month old but I missed it at the time: a historian argues that no, Watson and Crick didn’t “steal” Rosalind Franklin’s data, or “forget” to give her credit. And while they certainly treated her data cavalierly, there’s no evidence that they’d have treated data collected by a man less cavalierly. Which isn’t to excuse Watson’s appalling sexist attitude toward Franklin or downplay the importance or quality of her scientific work, of course. Interesting deep dive into the details of a famous moment in the history of science.
Dan Davies’ verjus theory of blogging. Or, why you should worry about audience quality rather than quantity (or really, worry about writing what seems worth writing, and let the audience look after itself). I agree with this. I would only add that, if you’re writing for a niche audience of professionals (as we are), the only way to get traffic is to not try to get it. So somewhat contra Davies, there’s no necessary trade-off between audience quality and quantity for professional niche blogging.
Deborah Mayo with a good post on the interpretation of statistical power. This is a tricky and controversial subject. I need to sit down at some point and figure out how her argument relates to the apparently-opposing argument of Andrew Gelman. I think that they’re asking subtly but importantly different questions, and that the disagreement comes down to which question is the best one to ask. Or maybe they’re just saying the same thing in very different ways, so that their apparent opposition is merely apparent. I’m not sure yet. (And if you care to enlighten me in the comments, please do!)
Maybe profs should haggle over textbook prices.
Here’s why R functions like read.table and read.csv default to reading character strings as factors. I mostly use R for traditional statistical tasks, so I like this default and had no idea anyone found it annoying. I liked this line at the end:
I fully expect that this blog post will now make all R users happy.
And finally, sticking with programming links that may be good or bad news, depending on your point of view: you can convert Excel spreadsheets into C!
Dynamic Ecology 
Interesting — the “reverse discrimination” in the NSF Survey (which they rightly say is not a problem) looks more like a symptom of the leaky pipeline in STEM. Perhaps women and minorities just don’t stick around long enough to get hired for faculty positions at the average time for white males, so on average the ones who do get hired are hired earlier. The likelihood of a woman getting a tenure-track position is so much higher than for a man because there are simply fewer women applying at that point; the “10% more likely” figure is looking at the fraction of women applicants who get a position, not the fraction of new hires that are women. The figures here look good for women and minorities, but in some sense they’re not even addressing the main question about the make-up of new faculty hires.
Re: the NSF Survey of Doctoral Recipients study:
The correlational data support the claim that “it looks like there’s a ‘baby penalty'” based on a 15-22% difference; moreover, the strength of this “baby penalty” indicates the need for remediation policies.
But the same set of data showing a relative 51% advantage for one racial group to a 33% disadvantage for another racial group “doesn’t show that there’s now ‘reverse discrimination'”?
I’m interested in how the same set of correlational data — from apparently the same model — can support the inference of “baby penalty” but cannot support the inference of racial discrimination.
Agreed, “baby penalty” sounds like a causal claim that they’re denied tenure or not hired for tenure-track positions because they have a baby. I think it’s probably the leaky pipeline showing up in the data.
“Baby penalty” is merely the authors’ summary term for a partial regression coefficient in their model. The authors discuss why this term might arise in their statistical model. They do *not* suggest that it arises because, when presented with two otherwise-identical candidates, hiring committees and tenure committees prefer to hire or tenure the one without young kids. Rather, they suggest that the partial regression term in their model likely arises for a complex of reasons having to do with the differential parenting burden experienced by women with young children.
I suggest that if you’re interested in the causal claims that the paper does or doesn’t make, you should read it.
Hi Jeremy,
Before I posted, I read in the preprint: “Wolfinger et al. (2008) mentioned finding the same pattern of advantage and disadvantage among minority doctorate recipients in their analyses but did not discuss these findings. Faculty markets can partly explain these patterns. Asian doctorates continue to be overrepresented among U.S. faculty (8.4% of full-time faculty; 5.6% of U.S. population; Smith et al., 2012) and do not experience the same hiring demand. Similarly, Blacks and Hispanics are underrepresented among U.S. STEM faculty, as are American Indian/Alaska Natives. This pattern of advantage and disadvantage in landing a tenure-track job may be due to hiring policies and practices that strongly encourage consideration of candidates from underrerpresented minority groups.” [emphasis added]
The authors thus acknowledged the possibility that race might be a causal factor that increases the chance of members of underrerpresented minority groups being hired, and the authors provided a theoretical reason for such a causal effect; that passage from the preprint produces a different inference, for me at least, than: “As the authors note, it doesn’t show that there’s now ‘reverse discrimination’.”
L. J., you seem to want to pick a fight. I have no idea why, but I’m not going to indulge you. You’re taking my very brief summary and trying to twist it in order to put words into my mouth that you can then attack. I don’t appreciate it. I’m sure that wasn’t your intent, but I’m afraid that’s how you’re coming across.
You clearly don’t like how I phrased my very brief summary. Fair enough (not that I care, but fair enough). But I’m sorry, I’m not going to be drawn into a debate with someone who’s starts off spending multiple multi-paragraph comments critiquing the wording of a brief summary.
This paper concerns a controversial, hot button topic on which it’s easy for the conversation to go off the rails even when everyone involved is well-intentioned. Frankly, you seem more interested in nit-picking and point scoring. I’m sure that wasn’t your intent, but I read your comment twice and that’s how it reads to me. So I’m not going to reply to you further on this topic.
And by the way, I say this as someone who is open to being convinced of the possibilities you raise.
Hi Jeremy,
My first comment was about what I perceived to be a causal interpretation of correlational data for the baby penalty but not for race discrimination; after all, if there’s no causal dimension of the baby penalty, then there is no need to discuss remediation. My second comment — in response to your suggestion that “…if you’re interested in the causal claims that the paper does or doesn’t make, you should read it” — was that the preprint acknowledged that race might be a causal factor but I did not get that from your summary.
I can understand how critiquing a summary can be perceived as nit-picking, but I presume that some of your readers rely only on the summaries posted here, so I thought that it would be worth discussing the accuracy and completeness of the summaries themselves.
—
I’d agree that it’s easy for discussions of hot-button topics to devolve, but it does not help to claim that the other person is trying to put words into someone else’s mouth without providing an explanation or description of what these words are. One of the reasons why I direct quoted so much is to avoid claims that I am not correctly describing the words of others. Thus, the only words that I attributed to you in my most recent comment were a direct quote of you. Maybe you meant the “reverse discrimination” sentence to apply only to sex, but — given that the next sentence in that paragraph starts with “And” and discusses race and sex — I’m not sure it would be twisting anything to interpret the “reverse discrimination” as applying to race. I certainly might have interpreted the summary differently than you intended, but that’s different than twisting.
I think the most sensible thing to do is summarize the analyses that were done, and the tentative implications they seem to suggest. Without making assumptions about what other analyses that weren’t done would reveal if they were done.
“The likelihood of a woman getting a tenure-track position is so much higher than for a man because there are simply fewer women applying at that point; the “10% more likely” figure is looking at the fraction of women applicants who get a position, not the fraction of new hires that are women. ”
No, that’s incorrect (unless I’ve badly misunderstood the paper). They’re fitting hazard models. So their finding here does not mean that male applicants outnumber female applicants. It means that, on a per-capita basis, women are 10% more likely to obtain a tenure-track position than men, controlling for the factors the authors controlled for.
Whether that effect would strengthen or weaken or vanish or reverse if one were to control for other factors than the ones the authors control for is of course an open question.
The post by Mayo on statistical power is interesting. I think we all need reminders that just because a given test yields a significant result, it does not necessarily mean it’s significant.
To me, though, the bigger issue of statistical power in ecology concerns error. So for example, in my current role as a plant ecologist, we know observer error for ocular estimates of vegetation can, and often do exceed 50%. Similar issues have been reported for ocular estimates of flock and herd sizes. So it’s not at all uncommon.
Even when our estimates are precise, the other issue concerns the actual proportion of the population you have sampled. For most of us, that is exceptionally small. When I studied aquatic microbial ecology, my samples of fungal spores were at times trillionths of what was suspended in the water column. Even though my estimates were reproducible, that was likely more a result of methodology than reality.
The chances of the sample mean differing from the population mean, either because of observer error or small sample size, I think is common in ecological research. Even though your data might be reproducible, it could well be your methodology is particularly good at consistently reproducing a specific kind of error.
This is why I place a great deal of emphasis on assessing the power of the data to predict specific thresholds of change, prior to defining the limits of the hypotheses and testing them. So for example, my collaborators on a recent study asked, begged and then demanded I submit results in terms of assessing 15% or greater change in vegetative communities. Time and again, I refused to comply. When I applied an alpha of 0.90 (error limit of 10%), my data were sufficient to detect 25% changes in community types. I could have lowered the alpha and attained the 15% limit requested, but at that point, you have to wonder if your estimates have any precision.
So while I appreciate the post and arguments presented by Mayo, I think the bigger issues for ecologists when it comes to assessing data has to do with a) the methodology used to gather them and b) understanding within the context of error just how useful your data happen to be. That is to say, you can subject your data to levels of stringency beyond their capability, and still achieve what appears to be a significant outcome… when, in fact, it’s gibberish.
Thanks for the comment David, but I don’t think it’s fair to even gently critique a post about power by saying that measurement error is the more important issue. Her post was about power, period, and her omission of measurement error implies nothing about the relative importance of power vs. measurement error. If we’re going to go down the road you’ve gone down, we’ll have to say that there are many things that are more important than any statistical issue–macroeconomic issues, political issues, etc.
With respect, you often seem to treat comments here as an opportunity to change the subject and talk about your own work. We welcome your on-topic comments. But in future, please try to stay on topic, and please only use your personal experiences to illustrate on-topic points.
With respect, I suppose it would depend upon the area one works in. I certainly by no means consider this a “personal” or “off-topic” point. The literature in plant science is robust and hot concerning this issue. So I do not consider it a personal issue. I did not debase Mayo’s points. They are excellent points.
And with respect, yes, I do make connections between posts and my own work. It happens to be what I study, and know something about. I think it is better to draw upon that and make relevant connections than to pontificate on areas I have little or no experience. For those posts where I have little or no experience, I do not comment.
David, when I told you to stay on topic, I meant it. It wasn’t an invitation for you to try to prove me wrong about whether you’re on topic.
Last warning: stay on topic, now and in future.
Some thoughts regarding ResearchGate (and I hope this is not off-topic, given the only very brief mentioning of RG in this blog).
After having received numerous invitations from close colleagues, I eventually signed up last year. I soon regretted the vast amount of emails that RG sends out about downloads, impact points, etc. By the way, the default email settings are set to automatically send out invitations to your colleagues, making it appear as if you sent out these invitations personally. These default settings can be changed, of course. Even though it is possible to disable these automatic invitations, it should have been disabled to begin with. Also, various unknown people started following my RG profile without my confirmed approval. Deleting undesired followers from a RG network was anything but straightforward. Eventually this became too cumbersome so I deleted my RG account.
Terry McGlynn mentions that an unexpected high number of downloads eventually persuaded him to join RG. Yet those download stats appear just as dodgy and opaque as the calculation of the impact points. Interestingly enough, RG still appears to offer access to a full text PDF to a single author paper of mine. Cautious as I am about copyrights, I certainly never uploaded it. Overall, I perceived RGs sending of unsolicited invitations and their promise of full PDF access as rather undesirable and perhaps even as aggressive. I am therefore going to continue ignoring RG for as long as I possibly can.
Thanks for sharing your experience Jesse. I suspect RG is like Twitter, blogging, and various other newfangled things. It’ll work for some people, and not for others: https://dynamicecology.wordpress.com/2013/03/25/advice-why-some-academics-shouldnt-read-blogs-or-use-twitter-or-facebook/