Last week, I highlighted some new results from a paper on detection probabilities and placed detection probabilities in the context of estimator theory. This in turn led to a a reader poll to try to get a sense of how people thought about experimental design with detection issues.
Although I don’t want to spend too much time on it here, I wanted to briefly highlight a great paper that just came out “Assessing the utility of statistical adjustments for imperfect detection in tropical conservation science” by Cristina Banks-Leite and colleagues. They look at several real world scenarios focused on identifying covariates of occupancy (rather than absolute occupancy levels) and show the results are not much different with or without statistical adjustment. They draw a distinction between a priori control for covariates of detection probability in setting up a good study design vs a posteriori statistical control for detection probability and point out that both are valid ways of dealing with detection issues. The take home quote for me was “We do not believe that hard-won field data, often on rare specialist species, should be uniformly discarded to accord with statistical models”. Whereas my last post was very theoretical/statistical this paper is very grounded in real-world, on-the ground conservation, but in many ways makes many of the same points. It is definitely worth a read.
Turning now to the survey … at the time of analysis Wednesday morning there were 168 respondents. You can view the raw results here. There was a reasonably good cross section of career stages and organisms represented although the employment sector skewed very heavily to university. And of course “readers of a blog who chose to respond to a poll” is in no way a scientifically designed sample. If I had to speculate this particular post attracted a lot of people interested in detection probabilities, but what exact bias that would result in is hard to predict.
Recall I presented two scenarios. Scenario A was to visit 150 sites once. Scenario B was to visit 50 sites 3 times each. The goal was to estimate how occupancy varied with four collinear environmental variables.
Probably the lead result is the recommended scenario:
Scenario B (50 sites 3 times) was the most common recommendation but it by no means dominated. Over 10% went for scenario A outright. And 20% noted that choosing required more information – with most people saying the critical information was more knowledge about the species – well represented in this quote on what the choice would depend on: “A priori expectation of potential for detection bias, based on species biology and survey method.”. It should be noted that a non-trivial fraction of those who went for B did it not to support detection probabilities but for reasons of sampling across temporal variability (a goal that is contradictory with detection probability modelling which assumes constant conditions and even constant individuals across the repeat visits). 17% also went for B but with hesitation (either putting statistical expertise of others over their own field intuition or else feeling it was necessary to publish).
There was a trend (but definitely not statistically significant) for more graduate students to recommend B and more senior career people (while still favoring B) to switch to “it depends”. Similarly there was a non-significant trend for people who worked on vertebrates to favor B and for people who worked on plants and inverts to switch a bit to scenario A (with scenario B still a majority).
Quite a few people argued for a mixed strategy. One suggestion was to visit 100 sites with 2 repeat visits to 25 of them. Another suggested visiting 25 sites 3 times, then making a decision how to proceed. And there were quite a few variations along this line.
The story for my question about whether there was pressure or political correctness to use detection probabilities was similar (not surprisingly). There was a weak trend to yes (mean score of 3.09) but not significant (p=0.24). Graduate students were the most likely to think there was PC-ness and senior career people the least likely. People working in verts and plants were more likely to see PC-ness than people working on inverts (again all non-significant).
So the overall pattern is a lean to scenario B but a lot of diversity, complexity and nuance. And not much if any perception of PC-ness around having to use detection probabilities ON AVERAGE (some individuals felt rather strongly about this in both directions).
In short, I think a majority of respondents would have agreed with this quote from one respondent: “… the most important part of study design is…thinking. Each situation is different and needs to be addressed as a unique challenge that may or may not require approaches that differ from those used in similar studies.” Which nicely echoes the emphasis in this blog on the need to think and not just apply black and white universal rules for statistics and study design.
Dynamic Ecology 
Hello Brian!
a copy one statment form your post:
“It should be noted that a non-trivial fraction of those who went for B did it not to support detection probabilities but for reasons of sampling across temporal variability (a goal that is contradictory with detection probability modelling which assumes constant conditions and even constant individuals across the repeat visits)”.
I do not really understand it, specially the part in brackets. Detection probability modelling actually does not assume constant conditions and even constant individuals across the repeated visits, that’s, in any case, an assumption or constrain made by the project researchers. Furthermore, it also allows to work with colonization-extinction processes on the patches, so it is hard to believe that constant individuals may be an asumption.
cheers
It depends of course on which detection probability model one is working with. But the simplest one (described in my first post on detection) assumes that the set of individuals being studied is constant (no immigration/emmigration/death/birth). The simplest models also assume p constant so factors affecting p (fog, water clarity, etc) are also assumed constant. There are more complicated models that let p vary over visits – but those have more parameters and typically require even more data to estimate.
However, the majority of people that I mentioned were I think people who work with things like insects, flowering herbs, plankton who actually have a rule of sampling 3 or 4 times per season exactly because the whole community composition changes over time (Phenology). I’m sure one could build a model of this in a detection context, but I’d hate to think of how much data you’d have to collect. Certainly its not the norm.
That was my thought process.
Thanks
Hi Brian,
Just for some context I’m looking at wildlife use of trails using cameras before and after woody debris spread on the line to limit human use and I keep running into people who insist I account for differences in detectability.
I tried to control for a lot of it beforehand like it was mentioned in Banks-Leite, trails are similar width and much narrower than the detection limits of the cameras, the trails all lack high on trail vegetative cover, cameras are the same model, set up the same. Additionally I state my area I’m sampling is strictly the trail and my study species (moose + caribou ) if they are occupying the trail are not going to suffer from low detectability, either before or after the woody debris is on the trail.
Since I’m not an expert I keep thinking I’m just missing something, but I also feel like they wouldn’t accept any situation where you don’t account for it and I don’t want to acquiesce just to get published in a “reputable” journal.
As for the feeling pressure part when someone states “why should I believe you?” when your trying to explain why you don’t consider it necessary, puts me in the yes camp.
Hi Jeff – that sounds pretty typical. It seems to me like you have thought about detection, been careful, understand your limits and could do very good science without worrying about detection issues (or at least detection would be pretty far down the list of things you should worry about as having much impact on your results). Yet, you are being told that you “have” to do it. That to me is the kind of reflexive, non-thoughtful statistical complexity I have labelled statistical machismo. But I don’t have a solution for your particular situation. Sometimes you just have to “play the game” if you want to get published.