Also this week: what it’s like to visit Congress, in (qualified) praise of impact metrics, research integrity zealot vs. humor, how to talk to people at conferences, journal life list, and more.
UPDATE: Wish I’d seen this earlier, want to link to it now while it’s still timely. A very sensible comment on the kerfuffle over whether to lower the conventional alpha (type I error threshold) from P<0.05 to P<0.005. tl;dr: If we want to reduce the rate at which we publish type I errors, and reduce the file drawer problem, don’t we want to raise the type I error threshold rather than lower it? Click through for the full argument; it’s not trolling.
How to start a conversation with a stranger at a scientific conference. Hint: don’t start with your “elevator pitch”.
The “Caltech rules” for how to structure a scientific paper, particularly one applying a theoretical idea to a particular question. (ht @jtlevy) I’m still mulling over this remark, about why you should not try to model your own papers after great papers that you’ve read:
The reason is that your own experience in reading [great papers] is a bad model of a reader of your own paper. Most paper in this category have already been acclaimed as great. When a reader gets yours (e.g., as a referee, a senior person in your field to whom you have sent the paper), you will be unknown. Most of these readers will therefore read it quickly. A complex, intricate, or discursive argument will confuse such a reader.
Arjun Raj with qualified praise for metrics of scientific impact. Some good points here, especially about everyone’s tendency to refute metrics they don’t like with anecdotes (I’m guilty of that). I do think he overlooks the problem of gaming metrics. There are contexts in which that’s an overrated problem (e.g., people self-citing so as to increase their citation counts; not enough people do that often enough to be worth worrying about). But there are contexts in which it’s a huge deal. I’m thinking of how the British research assessment excercise–a
metrics-based assessment of research at all British university departments–has distorted faculty hiring practices in obviously-suboptimal ways. (Corrected; commenter Jeff Ollerton points out that my memory is faulty and the REF is not explicitly metric-based. Though as Brian points out, even evaluation systems not explicitly based on metrics may produce results similar to the results that would have been produced by a metrics-based system. In any case, the broader point that universities do try to game the REF in a big way still stands, I believe.)
A 2×2 decision matrix to help you decide if your dissertation topic is a good idea. (ht @dandrezner)
Hoisted from the comments: Gregor alerts us that some journals (including Plos Genetics and Genome Biology) are now having their editors scour preprint servers and inviting authors of suitable preprints to submit them to the journal. I’m curious to hear what folks think of this practice. Is it a good thing because it improves efficiency of the peer review system (presumably, it cuts down on rejection)? Is it a good thing because it’s an endorsement of preprints, and any endorsement of preprints is a Good Thing? Or is it at least potentially a bad thing depending on how journals are choosing invitees? For instance, one can imagine that preprints from famous people might tend to garner invitations from selective journals. And one could imagine an unselective author-pays open access journal inviting anybody who posts a preprint in order to attract publication fees. Also: I’m reminded of this old post, from back in my Oikos Blog days, noting that law journals already have a centralized system that allows them to “bid” on unpublished manuscripts. But a key difference with that system is that authors decide which journals to seek bids from, as opposed to having to hope a journal editor favors them with an invitation. I’m also recalling an old link I can’t find just now from someone arguing that you can either have wide uptake of preprints, or you can have double-blind peer review, but not both.
A while back Brian posted on “crowdsourced truth telling” as a highly-imperfect-but-better-than-nothing way to deal with serial bullies in academia. The basic idea was for everyone who knows about a bully to cautiously and truthfully share that information in their own local circles, with the hope that word will get around and allow others to steer clear of the bully. Of course, one major limitation of the approach is that the information may not reach people who are in greatest need of it. Think for instance of prospective grad students who might join a bully’s lab. One commenter suggested a solution to that, but with obvious and serious downsides: something like RateMyProfessor. A site on which anyone could anonymously post statements about any graduate advisor. Well, somebody else thought of the same idea, and is actually doing it. It’s called QCist. I confess that I don’t see it working, because their moderation methods seem both ineffective and problematic. “When a user contacts us about a potential problem [with a supervisor], we will investigate the user who left the comments.” Really? And, um, how do you propose to do that, exactly, while also preserving anonymity? “[W]e also encourage the reviewer to leave words in the “additional comments” section to make the review more authentic. This way, a prospective trainee will be able to tell a bogus review from a truly authentic one.” Again, really? “[A] popular (and truly good) professor should receive multiple reviews. So one or two “bad-mouth” reviews will be drowned out by other good reviews.” Nice thought, but no: that doesn’t even work for RateMyProfessor (the ratings for which are infamously sexist), and they have way more ratings per professor than QCist will ever get. I’m sorry, I know QCist’s founders are tying to deal with a real problem, that’s very much to their credit. But I just can’t get behind their proposed solution. This seems like a much more promising initiative to me.
Several scientists are planning to run for Congress in the next US election; it’s an unfortunate sign of the times that they’re all Democrats. Hats off to them, I wish them best of luck. Running for federal office for your first political campaign is really ambitious. It’s tough to do successfully, though it has been done. I would be interested to hear if there are more scientists planning to run for state and local office. Collectively, state and local governments matter at least as much as the federal government, those offices are easier to run for in many cases, and those offices can be stepping stones to higher office. And a lot of policy that scientists tend to care a lot about is set at the state level (e.g., education and higher education policy).
In other political news, US scientist readers may want to call their senators regarding Sam Clovis, the nominee for USDA chief scientist. He previously accused the Obama administration of paying climate scientists to falsify their work, and that’s far from the only crazy idea he pushed during his years as a conservative talk radio host.
“How can I give honest feedback [on statistics to my lab group] in a way that doesn’t come across as overly negative?” Good discussion thread. (ht Andrew Gelman, who has the good suggestion to frame your criticism as coming from a hypothetical reviewer)
University of Vermont medical school will end all lectures in two years. Because they got a $60+ M donation to build classrooms suited to active learning in small groups.
Stephen Heard’s journal life list. I have a JLL too, but it’s not nearly as long as his.
Philosophy journal corrects 35-year old paper by famous philosopher’s cat. It was actually by the famous philosopher himself, of course, and published under his cat’s name as a joke. A joke of which philosophers were, and are, widely but not universally aware. Recently, a humor-challenged philosopher complained that this harmless decades-old joke contravened “research integrity” because
think of the children! some younger philosophers might be unaware of the joke. So the journal added a note to the article spelling out the joke. I’m reminded of the sad effort to have Onion articles prominently labeled as satire on social media, because occasionally somebody thinks they’re real. Somewhere, a single tear is rolling down Stephen Heard’s cheek. I hope that if anybody ever asks Evolutionary Ecology to attach a note to Vincent, Van & Goh 1996, the journal has the good sense to tell them to go pound sand. It’s ok for some people to be taken in by a joke. Trying to make sure nobody is ever taken in by a joke makes the world a worse place on balance.
And finally, you when you first started on Twitter vs. you now. I defer to Meghan on whether this is universally true. 🙂
Georgia Auteri, a grad student in UMich’s EEB department, wrote a post about her experience on Capitol Hill for the annual Biological and Ecological Sciences Coalition’s (BESC’s) Congressional Visits Day, where she lobbied for support for NSF. The event is co-hosted by AIBS and ESA and sounds like a great opportunity! And, as she notes there, now is a great time to contact legislators about the budget, as this is a key time for it! If you think you don’t have time, perhaps this idea will work for you: