Friday links: how to argue for basic research, Andrew Gelman vs. Amy Cuddy, and more

Also this week: missing Jane Lubchenco, Gilbert & Sullivan vs. digital humanities, the mysterious behavioral ecology of joggers, nerd sniping Stephen Heard, and more.

From Jeremy:

The Trump administration is nominating AccuWeather CEO Barry Myers to run NOAA. Sigh. I remember when Jane Lubchenco ran NOAA. So does Jane Lubchenco.

The right way to argue to national governments that they should support basic research. Canadian, but I think it generalizes.

25% of US STEM grad students at R1 universities lose interest in an academic career during grad school; only 5% become more interested. Interestingly, people who lose interest mostly do not do so because of perceptions of lack of tenure-track job availability, even though awareness of the tight faculty job market increases during graduate school. Rather, they mostly lose interest because they become less interested in research, especially basic research. Another tidbit of interest: men are about 8-10 percentage points more likely than women to start grad school with an interest in an academic career, and that gap persists through grad school. Related: my old post on protecting grad students from their own optimism. (ht @noahpinion)

A very interesting remark from Andrew Gelman: pseudoscience (or bad science; call it what you will) is becoming harder to distinguish from actual science (or good science)–but only at the cost of addressing trivial topics rather than important ones with the potential to fundamentally alter our understanding of the universe. It used to be ESP and perpetual motion machines. Now, says Gelman, it’s bad social psychology. Like some of Gelman’s commenters, I wonder if his Gelman isn’t overgeneralizing from the bits of social science he thinks about most and forgetting about stuff like anti-vax. And I wonder about his fudging of the difference between pseudoscience/bad science that attracts many scientists (like ESP, back in the day), and pseudoscience/bad science that’s mostly only attractive to non-scientists (like anti-vax). But as we’ve discussed in old posts, there aren’t any clear bright lines to be drawn here (see here and here).

Speaking of Andrew Gelman and social psychology: the NYTimes magazine has a feature on how the rapidly-changing research landscape in social psychology (push for replicability, etc.) has affected one of its highest-profile researchers, Amy Cuddy. Andrew Gelman gets a look-in as one of the most prominent critics of Cuddy’s work. Amazing (and very fair) piece, weaves together a lot of stuff we’ve talked about in the past. I’ll also point you to this old post of mine on some of the same issues. Here’s Andrew Gelman’s response to the piece, which attracted a wide range of comments.

Semi-related: Developmental economist Chris Blattman asks what will the revolution be in economics, analogous to the “non-preregistered, small-N experiments are uninformative” revolution in social psychology. Related post from Brian on whether ecology is having (or will shortly have) a scientific crisis, and if so, what kind.

Sticking with Chris Blattman, he used to have a blog that was widely read in his field. Here’s why he stopped blogging (and why he might start again one day).

Machine learning program with no preprogrammed knowledge of Go except the rules of the game, and no access to any Go games except those it played against itself, taught itself to become easily the best Go player ever (human or computer) in three days. (ht a correspondent)

The optical illusion of the year finalists.

How Canadians (well, some of them) decided what college to attend 30 years ago. An interesting artifact from the Time Before The Intertubes. I predict that Stephen Heard will waste the rest of the day trying to track down a copy of the book discussed in the link. 🙂

I think my research falls in the lower left of this graph. Fortunately. 🙂

I am the very model of a modern major general digital humanist“. This is hilarious. A sample:

I know my methods irritate established new historicists

Provoking them to write Op-Eds dyspeptic and quite moralist

But though they write my coding off as empty and reductionist

I’d rather read more Python than some bloody deconstructionist

From Meghan:

Ecological perspective on jogging:

 

From Brian:

Brian Martinson argues in this week’s Nature that every researcher should have a lifetime limit on the number of words they can publish. He acknowledges it would create different perverse incentives, but suggests they would be better than the perverse incentives of the current system. He broadly argues it would encourage replacing quantity with quality. He is most concerned about research quality (e.g. fewer errors), but I think it would apply to a lot of dimensions of quality. It will never happen, but I wish it would!

Hoisted from the comments:

My post on checking your causal inference methods by testing ridiculous causal hypotheses led to a great discussion, including a comment from the author of the paper that inspired the post.

The comment thread on our AUA about the research contributions of ecologists from developing countries is great too. Features numerous comments from researchers from Brazil. Touches on everything from the roles of local vs. international journals, to imposter syndrome, to detailed discussions of the particular challenges faced by researchers in particular countries.

23 thoughts on “Friday links: how to argue for basic research, Andrew Gelman vs. Amy Cuddy, and more

  1. If you look at Figure 6 of the nature paper (here https://www.nature.com/nature/journal/v550/n7676/full/nature24270.html ) it seems to suggest that the new variant (AlphaGo Zero) was able to beat AlphaGo Lee (which could beat the human World Champion) within 3 days of training. But the Figure suggests that it required ~30 days of training before it could beat AlphaGo Master which was another variant trained using some human-generated data. Please correct me if I’ve misunderstood things – the last Neural Network I used was over 6 years ago, so I’m pretty rusty.

    Still, this is an insane accomplishment I think, but the science journalism missed several crucial things, as they usually do.

  2. I’m pretty sure I used that Canadian university (not college!) guide to support my decision of where to go to school. I love the quotations they included – the distillation of Canadian snobbishness.

    “When people say McGill is a great institution, they really mean is that Montreal is a great city. Put McGill in Tuktoyaktuk and they wouldn’t be quite so enthusiastic”

    “Queen’s students are absolutely head over heels in love with their school, not to mention themselves”.

    “If only to keep up appearances, Western has a faculty”

    • The nearest US equivalent I’m aware of (which I didn’t use) was the Princeton guide to colleges. I don’t recall what affiliation it had with Princeton University, if any. Anyway, it was a crowdsourced “insider’s” guide to colleges and universities, which sold because it purported to address parent’s fears (and prospective students hopes) about what each institution was “really” like. Was it a party school, a hotbed of political activism, were there lots of LGBTQ students, etc. In practice, the “information” provided was rubbish. The people who put the guide together would just mail questionnaires to a few arbitrarily-chosen students or students groups at each college or uni, and then quote from whatever responses they got back with no attempt to check them.

      I know this because when I was an undergrad at Williams, a friend of mine who was an editor at the student newspaper got a questionnaire from the Princeton Guide. He put down a bunch of joke responses, like how the student newspaper’s office was the center of campus social life, figuring they were so obviously silly they’d never get printed. They got printed.

      The analogy to modern anonymous online forums, and social media, is left as an exercise for the reader. 😉

  3. If you count blogging, I’m sure I’ve already exceeded any reasonable lifetime word limit and should cease all publishing immediately.

    Really, what you want is cap and trade: set a lifetime word limit, and then let people with more words than they need sell them to the highest bidder.

    Or, you could give everybody an allocation of words, but also oblige everyone to give some fixed fraction of those words (say, 50%) to somebody else.

    /end obviously-bad ideas

    • In seriousness, that’s an interesting piece. And I think at least some of perverse incentives created by a lifetime word limit are fixable. For instance, don’t want people to skimp on their methods sections because they’re under a word limit? Don’t count methods against the limit.

      • I agree that many of the downsides are fixable.

        Personally I think a number of pubs limit is probably less artificial than a word limit. If you think you’ve got the chops to get all 20 of your lifetime allocated pubs in Science or Nature go for it. If you think your paper they reject is rejected because it is too novel and publish it somewhere else go for it. Time will tell. But you’ve got some skin in the game.

      • Agree that a pubs limit makes more sense than a word limit. It automatically fixes a bunch of the perverse incentives raised in the article.

      • Lets begin with only allowing CVs to list 20 pubs…max. [ or maybe 25]. I suspect folks will use broader criteria than status of journal or citation counts in their choice of 20, maybe even put in papers THEY think are important that are not well cited.

  4. That NYT piece on Cuddy vs Gelman is very well done.

    It strikes me that Simmons, Simohnson, Gelman etc were naive in thinking they could unleash the critiques they did without them have personal consequences for the people whose work they were attacking.

    But also what really struck me was the phenomenon of social media pile on where much less thoughtful people quickly feel unleashed to go very ad hominem even if, maybe especially if, the social media authors are subject to the same critiques they’re piling on to. I’ve seen that kind of sanctimonious, schadenfreude pileon happen around retractions in ecology too.

    By the time tenured Ivy League professors are afraid to speak up, the field has gone way beyond healthy critique.

    • I agree with all of this.

      And great point that about others feeling licensed to engage in a sanctimonious pile-on *especially* if folks like Gelman make a point of saying that they themselves make mistakes, have had to retract papers, are happy to be criticized online, etc. I hadn’t thought of that before. Not that it’s totally Gelman’s fault if some people who think they’re following his example just go on the attack–people are responsible for their own choices. But yeah, I do think Gelman’s stylistic choices and ways of explaining/justifying his approach might help create a climate in which sanctimonious pile-ons are more likely to happen.

      I’ve always thought that Gelman’s writings about this are a little tone-deaf. He doesn’t always seem to realize how he sounds to others. His line about how he’s doing Amy Cuddy a “favor” is a good example. As an aside, I now feel the same way about some of my old pieces about online debate, like my old one on why I love a good argument. They’re a little tone deaf.

      Simmons and Simohnson are better than Gelman at this, I think, and I think they’ve gotten better over time. They have a policy of showing anyone whose work they criticize a draft of the post before it runs, revising in light of that feedback, and summarizing those exchanges in footnotes. I think that’s a small illustration of how agreed, formal policies and procedures are helpful for ensuring productive discussions everyone’s comfortable participating in (well, as comfortable as such discussions can ever be, given that they might involve serious criticism of work you put your heart and soul into.) It’s why pre-publication peer review works well for the most part–it’s an agreed, formal process, not a free-for-all.

      This NYTimes story also illustrates why I’m most comfortable making critiques that apply broadly to many papers by many authors, *without* singling anyone out (even “just” as an illustrative example). I hope that I set a positive example in that way, rather than seeming to give license to others to engage in personal attacks. And I hope it helps that we also write about the importance of, e.g., *not* piling on people who made honest mistakes (https://dynamicecology.wordpress.com/2015/04/14/mistakes-happen-in-science/).

      • I agree there is a big difference between critiquing an approach in general vs. pulling out a single paper/persons work as an exemplar. I vastly prefer the former. The latter can sometimes seem more effective because its “edgy” and captures attention. But I think as this example shows, critiquing specific work might ultimately be more distracting from creating real system-wide change by instead creating scapegoats. The Biblical origins of scapegoat is a Jewish ceremony where one goat has laid on it the sins of all the people and then it is left to fend for itself in the desert. That is a more perfect analogy than is comfortable I think.

        Which brings us back to the pileon phenomenon. While I still think SS&G were naive, they all have in common saying that they have made the same mistakes themselves and they don’t think the person behind the work is a bad person or even a bad scientist. Those quickly get lost in the pileon though.

      • Something that is tacit in all this, I think, is whether someone is punching up, down or on the level. I believe that what really irks Gelman are folks with high-status/high-prestige positions who use popular press venues (e.g. TED) to popularize, or ‘hype’ their work. Without getting too far into this discussion, about which a ton could be said, I think it makes a big difference whether somebody like Gelman, who as a statistician sees his role at least partially as a gate-keeper of legitimate analytical methodologies, uses his position to go after the Big Names when they do shoddy work, versus if he were similarly to single out graduate students, post-docs, whatever. In a way, junior researchers/scientists already have to live with an enormous structural “pile-on” in the form of a broken system of incentives and mandates surrounding publications, grantsmanship, constant need to demonstrate and sell “impact” and “novelty”, etc. I don’t think either pile-on is healthy for science, but I see whatever Gelman has initiated as far less problematic…

      • I think this would have to be considered punching on the level. Gelman has his own crowds and is also a “Big Name” (including I think in his own mind), and the others were contemporaries in grad school. Nobody said they were punching down, but honestly to invoke TED talks as a reason to make it all OK sounds to me more like jealousy than evidence of punching up.

        To be clear I didn’t blame SS&G for the ensuing pileon. The people who pileon are responsible for themselves. I just said they were naive in thinking it wouldn’t happen. I also think, as Jeremy pointed out, they had choices about whether they personalized their charges or made them at the field as a whole. As I said, I am sure personalizing them was a more effective choice in the short term, but I believe and think we are seeing playing out it is less effective in the long run because it lets everybody else off the hook in their minds – “we dealt with that problem person” so lets move on. Certainly none of those people piling on have spent a lot of time thinking about their own practices.

      • ” As I said, I am sure personalizing them was a more effective choice in the short term, but I believe and think we are seeing playing out it is less effective in the long run because it lets everybody else off the hook in their minds – “we dealt with that problem person” so lets move on. Certainly none of those people piling on have spent a lot of time thinking about their own practices.”
        Agreed!

  5. Pingback: Friday links: love letters to trees, are invasive species bad, ASN Young Investigator Award applications due soon, Barbara Kingsolver vs. Mary Treat, and more | Dynamic Ecology

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.