The post last week on readability of statistics made me realize that the term “statistical machismo” has grown and morphed quite a bit from my original intent. One blog commentor noted that he now hears the phrase statistical machismo thrown at him when he works on developing new statistical methods. And one twitter commentor implied that statistical machismo was equated with “mocking complex statistics”. Both of these usages horrify me. Which has led me to a new word coinage: twitterized – verb – to become overly simlistically black and white as in the phrase “statistical machismo has become twitterized way past its original meaning”. (NB: apparently the word twitterized is already used in another sense but I of course prefer mine).
So I know just as in science, where you don’t have full control over how a paper is perceived once you release it into the wild, I have no control over how the term “statistical machismo” is used. But I at least have to try …
If you read my original post you will see that statistical machismo is not an absolute judgement on any particular statistical technique. And seriously if you are doubting me on that point go read the first few paragraphs of my original post. Any technique can be used with statistical machismo, even ANOVA. And even though I named some candidate techniques, I quite explicitly stated that all of the techniques I named had very valid usages (many of which I have used myself). I did name techniques to start a conversation and because it is my perception that those techniques are more prone to be pushed in a machismo way, but every technique I have ever mentioned in the context of statistical machismo is a perfectly good technique. There is no technique that is in itself statistical machismo.They’re just used sometimes in bad ways.
I am probably guilty of getting careless about this distinction between technique and machismo attitude myself in some of my later posts on the topic. Especially in the titles although I was generally pretty careful (but I’m sure not perfect) in the text. For example, in the first paragraph of my post on detection probabilities I clearly stated “I at no point said these techniques were bad or should never be used. But I did say that we had in many cases reached a point where the techniques had become sine qua non of publishing – reviewers wouldn’t let papers pass if these techniques weren’t applied, even if applying them was very costly and unlikely to change the results. ”
The bottom line is this. Statistical machismo is not a set of complex statistical techniques. Statistical machismo is an attitude. Many users of advanced statistics don’t have it. And many users of some pretty basic statistical methods do have it.
The two key components of the statistical machismo attitude are:
1) My way is the only correct way – statistics is all about shades of gray and judgement. Have you ever run a test assuming normality when the data didn’t fall perfectly on a line in a Q-Q plot? Most people have. That’s because statistics are messy. Ecological data is messy. It is rare to get perfect conformance with all assumptions. And in many cases (like the normality example) there are simulations showing that it doesn’t matter very much as long as the data is not skewed too badly. Statistical machismo is a reviewer who suggests a particular method, and then when the author provides a carefully reasoned explanation of why they didn’t do it that way, the reviewer doubles down with imperious language about “it has to be done that way” without even recognizing that there is a legitimate discussion to be had. Statistical machismo is also about not recognizing that many methods have significant costs in terms of extra work (e.g. generating a phylogeny, running computations that take weeks) or limiting the scope of questions that can be asked because the techniques are only tractable at certain scales. Bottom line is that statistical machismo is not recognizing that it is a question of judgment and there are multiple valid answers.
2) Malfeasance of motive – statistical machismo is typically motivated by some motive other than doing a good job analyzing the data. Statistical machismo motives include:
- An author trying to impress people and distract from the ecology. If a statistical method appears in the title of a paper and it is not a methods paper, that is a bad sign.
- A reviewer trying to gatekeep – using statistical methods as a way of saying no to other people and feeling smug about being part of the “in group”.
- Being unable to have a conversation about when a technique should and shouldn’t be used. If you think a technique should ALWAYS be used that is statistical machismo.
A related issue that I brought up in my original post is whether it will change the ecological conclusions. There are many cases where more complex techniques do change the conclusion in an important way and are more correct for it. But there are also many cases where it doesn’t. The number of macroecology papers I have seen that run a regression with and without phylogenetic regression and get exactly the same answers has to number into the 100s. Now admittedly sometimes its hard to know the outcome in advance until you try, and if trying is cost free go for it. But other times trying comes at a cost in time and can be known in advance to be very likely not to have an effect and is just not worth it. If you are unopen to that last argument (for any statistical method), you are committing statistical machismo.
The final issue I will briefly raise, but save as full a post for another day, is that the more complex the statistics, the more assumptions and the more complex the assumptions that need to be verified and validated. And I worry that we move from a world where most people know how to assess the assumptions to a world where people don’t even have a clue that they should be assessing newer, more complicated assumptions let alone how to do it.
The bottom line is if you are open to a conversation about trade-offs pro and con of multiple techniques, you’re probably not committing statistical machismo. If as an author you hype the technique you use more than the biology that comes out, or, if, as a reviewer, you are absolutely convinced that there is no other acceptable way to do it despite many rational arguments given by the authors, you are committing statistical machismo. If you are so attached to a method you think there is no valid reason for not using it then you are committing statistical machismo.In my experience, most stats experts are not the ones committing statistical machismo. They’re acutely aware that no technique is perfect and every technique has limitations and trade-offs. Its the people who have struggled to learn a technique and often genuinely don’t know the assumptions and limitations of the technique that are most likely to commit statistical machismo. Or in other words, if you are 100% sure you are right in statistics, you’re probably wrong. And you’re probably practicing statistical machismo.
So lets untwiterrize “statistical machismo”. Lets keep it to a description of an unconstructive, inflexible, superior, gatekeeping attitude. And not to a critique of statistical sophistication in itself.
What do you think? Has statistical machismo morphed in meaning since the original post? Has it become twitterized? Has it outlived its usefulness? Can it be meaningfully applied as a description of an attitude rather than a statistical technique? Or do you I think I’m full of it and trying to have it both ways?