A previous post on this blog asked why ecologists can’t all just get along. Contrarian that I am, I want to ask whether we get along too well. That is, do we ask each other enough tough questions?
I’ve been thinking about posting on this for a while. I finally decided to do so after a commenter on a previous post asked my view on whether it’s appropriate to ask a speaker at the ESA meeting a question that implies the speaker has made an extremely serious mistake (e.g., a pseudoreplicated experimental design, ignorance of basic facts about one’s study organism, etc.) Questions implying that the speaker has made a serious mistake aren’t the only tough questions, of course. Questions which undermine the motivation for the speaker’s entire study, for instance (which I ask more often than many people, because I don’t take for granted the speaker’s framing of the issues, choice of question, and choice of study system)
Tough questions rarely get asked at the ESA meeting, though anecdotally they’re more common in other contexts. In Canada, UBC and McGill both have the reputation of asking tough questions of invited speakers, and at Duke in the 70s and 80s I’m told the graduate students were really tough on each other. Do we ask tough questions as often as we should? I don’t know the answer, but in the interests of helping to ensure that the answer is “yes”, below are some suggestions on how to ask tough questions.
I should preface my remarks by noting that I have the reputation, at my university and probably elsewhere, of being quite a tough questioner. I invite tough questions as well: as a grad student I found the audience at the graduate student seminar series to be too uncritical, and once asked them to fill out anonymous comment forms after my talk in an attempt to get more critical feedback. Since I’m comfortable getting tough questions, and confident in my ability to answer them, I tend to have little hesitation in asking them and probably don’t always ask them in the best way. I think the advice I offer below is reasonable advice, but I admit I’m not perfect about following it myself (though I’m better than I used to be).
First, tough questions aren’t the same as rude questions. At his PhD defense seminar, my colleague Steve Vamosi was asked “So Steve, you’ve shown that sometimes populations go up, and sometimes they go down. What have we learned here?” And I know of an ecologist who once asked, of an invited speaker, “What was the point of this?” That kind of thing is just asinine; it’s easy to make anyone’s work sound stupid (“So Mr. Darwin, you’ve shown that things change. What have we learned here?”) No matter how bad you think someone’s talk was, don’t ask a rude question. If the purpose of the question is not to elicit an answer, but simply to let the speaker and the audience know how bad you thought the talk was, that’s rude (and redundant, because if the talk really was that bad, everyone in the audience already knows it)
Second, consider the speaker and the setting. If the speaker is a student, or is clearly nervous, or the seminar is some kind of informal or semi-formal practice talk, phrase your question in a positive, helpful way. Prefacing your question with some positive remarks about the talk is one way to do this. I admit I’m probably not as good as I should be about following this particular piece of advice. I asked a pretty challenging question of a student at this year’s meeting, indeed so challenging that the student’s supervisor (who’s a friend of mine) was swivelling in her seat trying to see who was asking the question. In my defense, the student seemed confident and professional during the seminar, and so didn’t seem like someone who would just wilt when pushed a little. And he handled the question fine, although at the time his answer didn’t really satisfy me. So I went and found the student and his supervisor later to talk further about it, and to make sure there were no hard feelings, and we ended up having a really good conversation about different ways to approach the problem the student was working on. So it worked out in the end, but I was probably over-aggressive at first.
The flip side of this second point is that, if the speaker has a PhD, and if the setting is a formal one like the ESA meeting, I tend not to pull many punches. If you have a PhD, and you’re talking in a setting in which speakers are expected to be extremely well-prepared, you should be able to handle any non-rude question anyone asks you. And if the speaker is aggressive about pushing a point with which I disagree, I don’t hesitate to push back, taking the view that the speaker is spoiling for a fight (I did encounter one such speaker at the ESA this year).
Third, phrase your question in such a way as to make it unaggressive, but without changing the content. For instance, if you want to question the use of a pseudoreplicated design, you could phrase the question as a request for the speaker to clarify the design, because maybe you misunderstood but it sounded like it might be pseudoreplicated. This kind of phrasing also hedges against the possibility that you really did misunderstand and the experiment really was replicated properly (or whatever). Or, depending on the nature of the question, you might be able to “defuse” it by simply saying right up front that it’s a tough question, but it’s one you really think is important and are really interested in having answered, so you’re going to ask it. This tack can work well if the question is one that’s been debated in the literature, so you’re basically inviting the speaker to engage in a wider, ongoing debate rather than challenging the speaker as an individual. This tack also can work well if the speaker is a friend, or is someone who you know likes a good back-and-forth debate.
Fourth, if the speaker is not a native English speaker, keep that in mind when deciding how to ask your question.
Fifth, if you really think that there’s no way to ask the question without seriously embarrassing the speaker, contact the speaker later and ask the question privately.
Sixth, once you ask the question, wait for the answer. This is something else I’m not as good at as I should be. If the speaker appears not to be answering my question to my satisfaction, I have to fight the urge to cut them off and ask the question more forcefully, which is bad.
So what do you think? Do we ask enough tough questions of each other? And do we ask them well? What’s the toughest (or rudest!) question you’ve ever gotten? Answer in the comments.
Thanks for the insightful post. As for your questions: I think the format of the ESA talks makes tough questions difficult to ask and answer due to the time limits, so I’m of the opinion of not-enough-tough-questioning. (However, I bet a lot more questioning happens in the hallways after talks when the audience isn’t there to hear it.) I think tough questions are generally asked in a polite manner in our field. And I don’t think I’ve ever gotten a tough or rude question in a public talk; I wish I got tougher ones, though.
Hi Jeremy. I agree, it is rather difficult to ask hard questions, and I think here at UVM there’s a culture against it. I was at a friends talk and asked him a question some regression analysis he had done, and why hadn’t he done it in x-manner (it was several years ago now so I don’t remember the details). Anyway, he was very flustered by it and stopped speaking to me for almost a month, and other friends came up and told me, partially in jest I assume, that I wouldn’t be invited to their talks anymore. Ever since then I just keep my mouth shut, especially about statistics. I find that some ecologists (Students here sadly) still have this attitude that “I’m an ecologist and I’m just following what the statistical help clinic person told me” and that a deeper understanding of the analysis and math isn’t needed. People clam up and say things “Well I’m not a statistician.” I think we should be asking and be prepared to answer hard questions. I know that I always sit and think about “what would I criticize myself for?” and make up more figures and slides to put at the end of my talks just in case I run into a jerk like myself at a talk :). I think your rules are good, but you always run the risk that people are sensitive and be hurt by your question no matter how delicate you are.
You’re not a jerk, and your friends need to grow thicker skins if they plan to get jobs in academia, and probably in a lot of other fields (you think government and NGO ecologists don’t get asked tough questions?) And “I’m not a statistician” is *never* an acceptable answer to a question about your stats (especially at UVM, where I’d think you could pick up lots of stats by osmosis just by hanging around Nick Gotelli…). I may do a follow-up post on how to answer tough questions; the number-one piece of advice will be “know the correct answer.”
At the Montana state Wildlife Society conference, there was a talk on the use of occupancy surveys to evaluate gray wolf population dynamics. These surveys were performed via hunter interviews at check stations. At the end of the talk, I asked the speaker if he had incorporated hunter density as a covariate for their analyses, given that detection probabilities are likely to be higher in areas that have higher numbers of hunters. The speaker had in fact done so, but didn’t mention it during the course of the talk. Afterward, he thanked me for asking the question, as he was sure that other people were thinking the same thing, but for varied reasons had opted not to do so.
I’m not particularly shy about asking questions, though given my status as an undergraduate I tend to only ask questions that I feel safe about my understanding of the topic at hand.
Pingback: On answering tough questions « Oikos Blog
Pingback: Engaging with crackpots at scientific meetings « Oikos Blog
Pingback: Advice from the archives: preparing to present at, and attend, the big summer conferences « Oikos Blog
Pingback: Advice: a compilation of all our advice posts, and a call for new advice topics | Dynamic Ecology
Pingback: Advice: helping you prep for the big summer conferences | Dynamic Ecology
Pingback: ESA 2013 preview | Dynamic Ecology
Pingback: Advice for the summer conference season | Dynamic Ecology