Friday links: heavy metal vs. the Origin of Species, peer review vs. Jeremy, and more

Also this week: #bugsR4girls, quadratic regression is not a valid test for humped and u-shaped relationships, a belated shoutout to #MySciComm, karaoke talks, college ≠ Harvard, and more. Including a link from Brian!

From Jeremy:

Quadratic regression is an invalid test of humped and u-shaped relationships between variables. It has a hilariously high false positive rate, routinely misdiagnosing curvilinear relationships with no mode as humped or u-shaped. Ecologists aren’t entirely unaware of this and often deal with it by testing whether the estimated maximum/minimum of the fitted curve is within the observed range of the predictor variable. But still, that’s kind of a kludgy, imperfect workaround. The linked post develops and evaluates an alternative test, a variant on breakpoint regression. Students: your “low hanging fruit” alarm should now be blaring. Want a quick and easy paper that would also be an important contribution? Find some important study or meta-analysis that looked for a humped or u-shaped bivariate relationship using quadratic regression, and redo the analysis with this new approach. Even if you have no plans to ever run a quadratic regression, you should click through. The Q&A at the end of the post is a great example of why it’s important to be crystal-clear about your goals when choosing your statistical technique.

Peer review is younger than you think, although I believe the age of the practice varies by field (?). Indeed, by the linked measure, it’s about my age. Which raises the question:

I’m very late to this, but via a correspondent, I just found out about the ESA Science Communication section’s #MySciComm series of interviews with scientists on how they got into scicomm and how they do it.

A preview of the Wildlife Photographer of the Year contest winners. I used to love going to this exhibition when I was a postdoc in the UK. The standard is much higher than for any other nature photography contest I’ve ever seen.

Your annual reminder, if one were needed, that when it comes to news stories about college and university admissions (or really, any news story to do with undergraduate experiences of college), any story that focuses on current or aspiring students of Harvard and similar institutions is focusing on a very small and unrepresentative minority of all students.

The ASN standalone meeting is going to formally debate the question of whether evolutionary history tells us anything about the functioning of contemporary ecological communities, and is going to have a natural history trivia contest, and is going to have karaoke talks?! Man, I am sooo tempted to cancel the first week of my classes so that I can go. 😦

And finally, commenter max informs us that there is a heavy metal song based on the last line of The Origin of Species (the famous bit about “endless forms most beautiful”). Here’s the video. Have I mentioned lately how much I love our commenters? 🙂 Of course, now I’m going to have a metal song in my head whenever I reread the final paragraph of the Origin, which I consider the most beautiful and eloquent passage ever written. That’s going to be a disconcerting experience. But I still love our commenters.

From Meghan:

This science communication issue of the Annals of the Entomological Society of America looks fantastic, including #bugsR4girls, the use of science communication to combat research isolation, and a piece by Terry McGlynn on recruiting students from minority-serving institutions. (ht: Christine Rose-Smyth) Update: NPR covered #bugsR4girls! It’s a really wonderful story, featuring entomologist Morgan Jackson (who created the hashtag) and 8-year-old entomologist Sophia Spencer (who was the inspiration for the hashtag).

From Brian:

I found this survey* on American college student attitudes towards and knowledge of free speech disturbing. 40% think hate speech is not protected (it is), 60% think allowing a platform for the opposite point of view is required (it isn’t), and a shocking 20% think that physical violence is an appropriate response to objectionable speech (just wow!). Democrats, Republicans and Independents all agree on that last point so don’t go looking to make this partisan. Look, I don’t like the views of a good number of people that have been in the news for getting shut down on college campuses. In fact I hate some of them. But the first amendment is the bedrock of what makes America work**. I’m pretty sure the idea that King George was an odious tax-grubber was considered hate speech at the time it was uttered too (and a few of our British readers might still feel that way …). I don’t mean to make a flippant comparison to some other remarks that are genuinely hate speech (and odious in themselves). But the point of the first amendment is that neither the state nor any one person has the right to decide. It is in the rational communal discourse that bad ideas are outed and good ideas advanced. The alt-right speech in Boston and the peaceful response (guaranteed by some excellent police management) is exactly how free speech should look. It made the alt-right look pitiful and ridiculous and they literally gave up and quit early. And college campuses ought to be especially good at doing this. In this time when freedom of the press is under attack, colleges should teach and serve as models of how free speech works, not be a seed of violent shut down of free speech.

* Caveat emptor – the survey was funded by the Koch brothers but it was designed and run independently by a scholar at the Brookings Institute (a highly respected and non-partisan think tank in DC). So read the survey carefully and draw your own conclusions.

** I am writing this from an American perspective, but some version of free speech is at the core of most modern democracies.

(Jeremy adds: I’m more sanguine about this survey than Brian is, in part for my own reasons, and in part for reasons laid out here and here.)

 

16 thoughts on “Friday links: heavy metal vs. the Origin of Species, peer review vs. Jeremy, and more

  1. The poll results are already clear: Our readers think that peer review is resilient, or that I’m going to die young. This is one of the rare times when I’m glad the poll question was ambiguous. 🙂

  2. About the test of U-shapedness: I haven’t had a chance to dig into the papers behind the blog link you reference, but I think the “problem” is misleading and the solution to be just as bad as the quadratic regression. First, I typically use a quadratic term to test if there is any evidence a relationship is non-linear. Thus I am fine with it detecting y=log(x), or in fact any other relationship that is not linear. So, what is one’s goal is in doing anything other than simple regressions? In this case, there are two underlying parts to answering that. The more superficial is what is really meant by a U-shape? I suspect there are very few relationships out there that are actually precisely U-shaped. The two-pronged regression that is suggested tests for a U-shape no more precisely than a quadratic (we could call it a “tepee” test for assessing V-shapes). But, more fundamentally, the whole idea of frequentist statistical testing is violated here. One doesn’t test for a particular shape, but rather tests if the data are sufficient to reject a null hypothesis of no shape. So the two-regression test is not testing for U or V-shapes, but doing what I think the quadratic should be used for, which is to ask if there is evidence the relationship is nonlinear, since linearity is the null hypothesis for both of the quadratic and tepee tests. I will note in ending that the two regression test could result in neither regression being significant, but in fact the relationship is significantly non-linear, so I suspect the false negative rate on this test is quite high. Thus there is the additional problem with converting one test into two separate statistical tests. Presumably the correct approach here, which I think already exists in a branch of statistics called functional statistics, is to assess the fit of the data to a variety of non-linear functions and do model selection to pick the best mathematical descriptor of the data. Perhaps this has already be used? I’d be interested in hearing about it.

    • “First, I typically use a quadratic term to test if there is any evidence a relationship is non-linear.”

      That’s a different question than testing for a hump or u-shape. Different tests are appropriate for different questions.

      “The two-pronged regression that is suggested tests for a U-shape no more precisely than a quadratic ”

      You’re mixing up describing the shape of a curve as precisely as possible with inferring whether it’s truly humped or u-shaped. The linked post explains why those two tasks require different tests.

      “the two-regression test is not testing for U or V-shapes,”

      It tests whether the bivariate relationship changes direction from increasing at low values of X to decreasing at high values of X (for a hump; it tests the reverse for a U-shape). In many contexts, that’s all that scientists mean when they hypothesize a humped or u-shaped relationship between variables. For instance, in ecology the intermediate disturbance hypothesis is a qualitative hypothesis that species diversity will increase with disturbance rate or intensity at sufficiently low values of disturbance rate or intensity, and decrease with disturbance rate or intensity at sufficiently high values of those variables. That the prediction often is illustrated with a smooth humped diagram is just a matter of convention; the hypothesis isn’t actually that specific. It’s fairly rare for scientists to actually have a more precise quantitative hypothesis–a theory that quantitatively predicts that the bivariate relationship will be quadratic, or linear with a breakpoint, or etc.

      Re: the power of the proposed test to detect a hump or V, the linked post provides data on that. And as the linked post points out, it’s useless for a test to have a low type II error rate if it has a sky-high type I error rate.

    • I agree with Jeremy’s points here; the U test is looking for points where the derivative changes from positive to negative; quadratic regression won’t find that unless the true relationship is quadratic.

      Following up on your point about multiple models though: I got curious about how well this would work using a GAM instead of two lines, by using estimated derivatives for smooth curves, and it turns out it works well! Basically, the idea is to look for intervals where the CI for the first derivative includes zero, and the derivatives at the ends of the intervals have derivatives of opposite sign. I put some sample code up on my Github site.

      • GAMs will have greater uncertainty at the ends of the intervals, true, but so will the split linear regression model (whichever line is close to the end will have a more uncertain slope, increasing the odds of not detecting an extremum). Statistically, it’s just hard to detect an extremum at the end of an interval, no matter what approach you use.

        I also implemented a second test, where I considered any point to be a potential extrema if the CI for the 1st derivative included zero, but the CI for the 2nd didn’t. However, I think this test is actually too anti-conservative, as it results in CIs that are too small, since the uncertainty around the 2nd derivative will generally be greater than that around the 1st derivative. Also, this second test ended up finding a fair number of intervals on the boundaries of the range of x, and I didn’t think that really made sense.

        I’d note: I have done nothing yet to check the error properties of this method, so I’m not sure under what all conditions it works for.

      • Well, except that with Data Colada’s proposed test, youre not trying to estimate slopes at the extremes. Youre estimating average slope over some range of the X variable.

      • The Data Colada test is based on first estimating where a peak might be (by a spline) then splitting the data in two around that peak and estimating one slope to the left and one to the right of the split. A given point is only considered a peak if both slopes are significant and both have different signs.

        This means that any peak near the edge of the data will always have one slope estimated with very little data (the slope closer to the edge), and thus will have low power, since that slope will often be insignificant. It’s a basically inescapable problem; in general, your power to detect peaks will go down as you get close to a boundary.

  3. That metal song based on the Origin suggests a fun post idea: what are the weirdest evolution-inspired cultural artifacts? Besides that song, the horror movie based on the Price equation would be another candidate: https://dynamicecology.wordpress.com/2012/08/26/theres-a-horror-film-based-on-the-price-equation/ And there’s apparently a graphic novel in which Charles Darwin hunts werewolves in Victorian London. What else?

    Lots of products slap “evolution” on the label–I have a bottle of “Evolution” wine in my fridge. Anything like that would have to jump a pretty high bar of creativity and weirdness to make the list, I think.

  4. I am delighted you highlighted this issue, Meghan:

    “the use of science communication to combat research isolation”

    I am not certain what other investigators have experienced during their careers, but I have found that I very much became a hermit as my research activities increased. I can look back on entire blocks of time- months really, where I was so zeroed-in on a particular research problem that I rarely came up for air. Meaning, I did not have a lot of contact (face to face anyway)- with other people.

    While I did not seem to mind the isolation, I also felt I was missing out on a lot of fun and potentially a lot of good ideas- science or otherwise. In recent years I found a way to avoid the trap of the ivory tower by getting involved on a more local basis. So now, I serve on several boards of directors and participate actively in project development for these groups. It is a great way to remain connected and also give back to the society that made my education and success possible. It also provides professional interactions beyond the realm of science, which I have found to be instructive and fulfilling.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.