As an editor, reviewer, and committee member, I have seen many authors and students give the following rationale for their chosen research approach (e.g., choice of study design, analytical method, or response variable):
“This approach has proven useful in other contexts, so we chose to use it in our context.”
This rationale is weak. A skeptic might rephrase it as follows:
“This hammer has proven useful for hammering nails, so we tried hammering on other things to see if any of them were also nails.”
It is not enough to argue that a particular approach is useful in general, or even that it is useful in contexts that vaguely resemble your context. Lots of things vaguely resemble nails, such as screws and one’s own fingers. You should be able to make a very specific case that you have chosen the right tools for your particular job. Choosing your tool first, independent of the job, is a good way to end up using the wrong tool for the job. Your rationale for your research approach should be “We needed to hammer a nail, and so we chose to use a hammer.”
Oh man, literally laughing out loud, that was great.
The number of instances of people using some method or other because Joe Schmoe (insert string of publication dates here) did so, is frightening.
“This hammer has proven useful for hammering nails, so we tried hammering on other things in the off chance that any of them were also nails.”
I especially enjoy papers that conclude, pretty much straight up, that the methods have some potentially pretty serious problems, then insert a bunch of irrelevant justifications and other stuff and go ahead and use them away. And how authors respond in print when challenged on it.
Pingback: Advice: a compilation of all our advice posts, and a call for new advice topics | Dynamic Ecology
Pingback: The fox and the hedgehog | Dynamic Ecology