Or more precisely, error statistical (“error statistics” being Deborah Mayo‘s term for statistics as a set of tools for detecting and quantifying scientific errors). So argues this very nice article by statisticians Andrew Gelman and Cosma Shalizi. A more systematic and fully-developed statement of views Gelman has expressed in other writings. Argues that Bayesian statistics, properly understood and practiced, is actually hypothetico-deductive, not inductive (!) And it’s not just philosophical navel-gazing; their arguments have strong implications for the actual practice of Bayesian statistics. For instance, Gelman and Shalizi emphasize the importance of “posterior predictive checks” (basically, frequentist-style testing for model misspecification) and deemphasize the importance of posterior probabilities and the usefulness of things like Bayes factors and Bayesian model averaging. If like most ecologists you either think philosophy of statistics doesn’t matter for practitioners, or else have only been exposed to standard philosophy of Bayesian statistics (like the notion that Bayesians have to believe in “subjective” probability), you really ought to read this. I learned a lot and even changed my mind about a few things.
I’m coming around to the view that Bayesian statistics as practiced by Gelman (that’s a crucial qualifier) is not just pragmatic but also principled. Further, they’re principles I agree with. I wouldn’t exactly take back what I’ve written before, but I would say that what I wrote before underrates just how principled Gelman’s unorthodox Bayesianism is.
p.s. I really need to do a post sometime on Deborah Mayo’s ideas about error statistics, particularly her key idea of “severe tests”…
Dynamic Ecology 
“For instance, Gelman and Shalizi emphasize the importance of “posterior predictive checks” (basically, frequentist-style testing for model misspecification) and deemphasize the importance of posterior probabilities and the usefulness of things like Bayes factors and Bayesian model averaging.”
Hmmm. I’ve got a bit of a quibble with the idea that Gelman deemphasizes posterior probabilities. Yes he neither likes Bayes factors nor model averaging. But the way he defines Bayesian analysis is as any use of a posterior distribution to make inferences (http://andrewgelman.com/2012/07/what-is-a-bayesian/)…and consequently this involves lots of posterior probabilities. The posterior predictive checks that he emphasizes make use of Bayesian p-values, which are themselves posterior probabilities (see point 6 on this post: http://andrewgelman.com/2009/02/confusions_abou/).
Less technically, I’m not sure Gelman would agree that how he practices Bayesian statistics would actually be considered frequentist statistics. But I obviously can’t speak for him. What *I* have taken from Gelman is that Bayesian analysis provides useful tools that are best practiced by incorporating *some* ideas that have historically been promoted by frequentists and rejecting *some* ideas that have historically been promoted by Bayesians. Many ideas have been put forward about how to analyze data; some under the heading of Bayesian, some frequentist, some predictivist, some likelihoodist, some in machine learning, then there’s the correspondence analysis and multidimensional scaling stuff etc. But we’re still pretty far from a synthesis…and consequently are all still trying to find the right mix of them for each of our particular problems.
Well, all I can say is read the linked article and see how you think it lines up with other things Gelman has written. See in particular pp. 14-15, where a lot of emphasis is put on learning from mismatches between the model and the data.
I think my remark that Shalizi & Gelman de-emphasize posterior probabilities may have been a bit unclear. Yes, they make use of the posterior distribution–but to simulate from it. They aren’t using it to just slap a number on the probability of the model, as the sole measure of the model’s adequacy, which is what an orthodox Bayesian would do.
I read that article a while ago so I hope I remembered it properly. And I don’t disagree with anything in your reply. I just thought it was going too far to say that Gelman deemphasizes posterior probabilities — he uses them all the time, but just for different tasks than many Bayesians would have historically been interested in (as you point out).
I appreciate your coverage of Gelman’s interesting ideas. I like how he seems to really listen to what scientists want / need…which I think has allowed him to develop his own brand of statistics. This is one of my favourite Gelman quotations on this topic (http://andrewgelman.com/2011/02/a_statistical_v/):
“Unfortunately, when we deal with scientists, statisticians are often put in a setting reminiscent of Arrow’s paradox, where we are asked to provide estimates that are informative and unbiased and confidence statements that are correct conditional on the data and also on the underlying true parameter. Larry Wasserman feels that scientists are truly frequentist, and Don Rubin has told me how he feels that scientists interpret all statistical estimates Bayesianly. I have no doubt that both Larry and Don are correct. Voters want lower taxes and more services, and scientists want both Bayesian and frequency coverage; as the saying goes, everybody wants to go to heaven but nobody wants to die.”
There’s some truth in that quote. Though personally, I’m purely frequentist and don’t interpret my stats in a Bayesian fashion. Whether that means I don’t want to go to heaven, or that I don’t mind dying, I’m not sure. 😉
Well played.
Pingback: Statistical machismo? | Dynamic Ecology
You may be interested in John Kruschke’s reply to Gelman and Shalizi’s paper:
Kruschke, J. K. (in press). Posterior predictive check can and should be Bayesian: Comment on Gelman and Shalizi (2012a). British Journal of Mathematical and Statistical Psychology.
http://doingbayesiandataanalysis.blogspot.com/2012/09/posterior-predictive-check-can-and.html
Pingback: Philosophical ‘basis vectors’ for locating statistical philosophies « ecology & stats
Pingback: Why, and how, to do statistics (it’s probably not why and how you think) | Dynamic Ecology
Pingback: Stuart Hurlbert rips Sokal & Rohlf and the state of biostatistical training | Dynamic Ecology
Pingback: The downside of data sharing: more false results | Dynamic Ecology
Pingback: Why ecologists might want to read more philosophy of science | Dynamic Ecology
On the point of predictive posterior checks, Gelman, Hwang, and Vehtari demonstration (in http://www.stat.columbia.edu/~gelman/research/published/waic_understand3.pdf) how proper information criteria can be used for these purposes.
Pingback: Machismo estatístico (tradução) – Mais Um Blog de Ecologia e Estatística