Also this week: contrarianism! We should worry about over-reproducibility! We should not worry about grade inflation! Good statistical advice can be bad advice! See if you can guess which two of those are well argued, and which is a #slatepitch.
In the non-contrarian division: consensus (?) on p-values, a major change to NSF’s Graduate Research Fellowship rules, and more.
From Meg:
Hope Jahren had a piece in the NY Times this week on the advice she gives to women facing sexual harassment at work, on the dismaying frequency with which she needs to do this, and on the effects this harassment has on women in science. She writes:
IT’S not something I can put on my C.V., but I believe that one of my most important duties is to walk young women through emails like the one my former student received, and I am called upon to do it many times each year. I emphasize to them that the first email is important because it is the one that the powers that be will point to and say, “Why didn’t you do something when you first got this?” I talk to each woman bluntly and advise her to write back immediately, telling (not asking) him to stop.
I teach her to draw strong professional boundaries and then to enforce them, not because she should have to, but because nobody else will. I insist that she must document everything, because someday he will paint this as a two-way emotional exchange. I wearily advise her to stick it out in science, but only because I cannot promise that other fields aren’t worse. And I hope that this is enough to make him stop. But it never, never stops.
It’s a very important piece, and you should read the whole thing carefully and think about what you can do at your institution to try to change the climate for women. Because, as Hope said, “my male colleagues will sputter with gall, appalled by the actions of bad apples so rare they have been encountered by every single woman I know.”
On twitter, Jonathan Eisen asked whether having a conference on a weekend was a good or a bad idea. There were really diverse (and interesting!) answers, which Eisen has summarized and storified here.
NSF announced a major change to the eligibility criteria for Graduate Research Fellowships. Specifically, students can now apply in the first OR second year of grad school, but not both. Terry McGlynn and Zen Faulkes both had blog posts on potential impacts of this change. My personal experience: I did not apply for an NSF GRF as an undergrad, because I didn’t know about them. I applied in my first year in grad school and one of the reviews said “Success is doubtful”. So, I felt incredibly lucky when they changed the rules for the next year to allow second year grad students to apply for the fellowship (before that, only undergrads and first year grad students could apply). I ended up getting one that year. It will be interesting to see what the effects of this rule change end up being, and what people decide to do in terms of applying in year 1 vs. year 2 of grad school. My guess is that most grad students will wait until year 2, making it so that reviewers are comparing senior undergrads and second year grad students. (ht: Terry McGlynn)
From Jeremy:
A while back I published my shadow cv: all my rejected papers and grants, all the jobs I applied for unsuccessfully, etc. Now Andrew Hendry has taken a deep dive into one section of his shadow cv: his rejected papers. Andrew is an obsessive record-keeper; click through for lots of data on how often he gets rejected. It’s often! Which is one argument for Axios Review.
Arjun Raj risks the wrath of the intertubes by arguing (cogently) that there’s such a thing as caring too much about reproducibility. Specifically, he says that, for many tasks and many people, version control, figure scripting, GitHub, and LaTeX are a waste of time. Follow up post here. And before you go jumping to the conclusion he’s hidebound and ignorant: he’s a bioinformatician who cares a lot about reproducibility. Interested in Meg’s thoughts on this, since she’s recently moved to scripting her analyses but hasn’t yet taken the further steps that Raj argues are often time badly spent.
The American Statistical Association’s consensus statement on the use and abuse of P-values. Andrew Gelman has some very good comments on the statement. Deborah Mayo also has good comments.
Sticking with Andrew Gelman, a post on how good statistical advice can lead you to make statistical mistakes. Related to something I’ve been thinking more about, in all kinds of contexts: “failure modes”. The idea that every approach has its own characteristic ways of failing, in those cases where it fails, and that those ways of failing are intimately tied to what makes the approach work in those cases where it works. Philosopher Bill Wimsatt is one person who’s written about this. More on this in a future post, hopefully.
Smart watches that allow students to cheat on exams. I don’t think this is a big worry yet and I doubt it will be for a while, if ever. But it’s still interesting and maybe important to think about what you’d do as an instructor if it ever did become a worry. I’ve heard (apocryphal?) stories of profs who’ve gone to one-on-one oral exams to eliminate any possibility of cheating, but that doesn’t scale. Maybe the solution is to let students use any resource they want but alter the style and difficulty of the questions appropriately? (ht Marginal Revolution)
The latest data on gender balance of college graduates around the world: in 2013, 6 million students across OECD countries graduated with a bachelor’s degree; 58% of them were women. But it varies a lot by field: the percentage of women among 2013 OECD bachelor’s recipients was much higher in education, humanities, and social sciences (about 65%) than it was in STEM fields (31%). Click through to see the data broken down by country. Bet few of you will be able to guess the OECD country with by far the highest percentage of women among 2013 STEM bachelor’s recipients; I couldn’t. (ht Marginal Revolution)
Paging Mark McPeek: an argument that we should give up the fight against grade inflation as already lost and rely instead on letters of reference. I don’t agree, and have a sneaking suspicion I’m falling for a #slatepitch here. Good luck with the whole “we can totally do this; every university just has to hire enough well-paid faculty to reduce the faculty-student ratio to Hampshire College’s level” thing. And what makes you think that a world that relies on letters of reference wouldn’t just recapitulate the problems of a world that relies on grades? Finally, at the end there’s a deeply weird list of other things that, like a world without grades, supposedly seem “too good” and “suspiciously easy” (open marriages?!). Anyway, consider this an excuse to talk about what, if anything, to do about grade inflation. (ht Marginal Revolution)
And finally, expertly trolling Oxford Comma Fundamentalists. 🙂
Hey, Jeremy, have you seen this viral video yet (wallet-stealing fox)? http://www.sbnation.com/lookit/2016/3/10/11192888/golf-fox-steal-wallet-go-golf-fox-go
No, I hadn’t seen it. Thanks!
Ha!
🙂
“what makes you think that a world that relies on letters of reference wouldn’t just recapitulate the problems of a world that relies on grades?”
Spoilers below:
It does.
I like that Arjun Raj post — thanks for linking to it! We are currently working on a manuscript in Google docs. 🙂 One thought I had regarding the point about figures is that it’s hard to know where to do the full reproducibility cost-benefit analysis when initially working on a paper given that, at that point, you have no idea how many times you might need to remake a figure. But I agree that the key is to get all the data points coded, and then after that it’s more a matter of preference. One suggestion I like is to take the coding one step beyond where you could have easily gone — then you learn something new that’s likely to help you again in the future, but without getting bogged down in trying to write the perfectly reproducible figure.
I like that suggestion!
That’s a really nice idea!
Re: grade inflation, one important fact that author neglects is that it’s mostly an issue at Ivy League universities and highly selective liberal arts colleges*. There’s no problem with grade inflation at, say, the University of Calgary. Which suggests that the problem is either eminently soluble, or if it’s insoluble it’s insoluble for reasons unique to those places. Either way, there’s no reason to call for everybody to get rid of grades.
*Sorry, can’t find the link to the data on this at the moment. And if anyone has data showing otherwise, I’m happy to be corrected.
I agree grade inflation is less in Canada so what you say about U Calgary is probably true. But I think grade inflation is alive and well at most institutions in the US – not just Ivy/SLAC. It might be more extreme at those elite institutions, but it is actually probably more justified and representative of student quality than the grade inflation I see at major state universities.
Ok, I stand corrected. Casual googling reveals this data compilation for 230 US colleges and universities, chosen based on data availability:
http://www.gradeinflation.com/
Average GPA across all universities in the dataset has gone from 2.93 in 1991-1992 to 3.11 in 2006-2007. If you break out private institutions, the increase has been a bit larger: from 3.09 to 3.30.
I haven’t checked the data sources myself, and one can certainly imagine this might be a biased sample for various reasons. But yeah, it suggests that inflation is widespread, but faster at elite institutions.
The website author says he’ll be updating with more recent data any day now…