Equity and diversity targets in science, part II: views depend on identity (guest post)

Note from Jeremy: This is a guest post from Françoise Cardou & Mark Vellend.

************

A few weeks ago, we came to you with a knotty lunchtime debate : are quantitative equity and diversity targets in science a good idea, and if so, on what basis? Unequivocally, certain groups of people face unequal challenges and barriers in science. Any specific policy measure to address this issue necessarily comes with both benefits and costs, and we wanted to find out what people (from a broad range of situations) actually think. We want to thank everyone who chimed in: you are now honorary participants in our lunch group. We return today with the results.

First, a brief recap. Our questions were motivated by changes to the diversity targets used in the Canada Research Chair (CRC) program, which have for a few years now been based on the “availability” of people from designated groups (women, Indigenous peoples, people with disabilities, and visible minorities) in the pool of candidates. Following a lawsuit from several Canadian faculty members under the Human Rights Act, the new CRC targets will now be based on the representation of people in these same groups in the general population*. As one example, the target for women will increase from 21% to 50% in the natural sciences and engineering. These represent two ways of setting quantitative targets, and we can imagine a range of rationales for different sets of such targets, each with a different goal and balance of cost and benefits (see the original post).

We intended first to find out whether people perceived quantitative EDI targets as a compromise between two forms of discrimination, and second whether there was any allocation policy around which there was broad agreement. Finally, a few obvious but important points: this poll is not a scientific study (it’s an online poll), and the results do not say anything about what is right or wrong, simply that some people feel this or that way. Should anyone like to have a look for themselves, you will find the full dataset here (please let us know if you dig deeper, we’d love to know what you find!).

Who responded?

We had 128 respondents (Fig. 1). This included 34 grad students (27%), 32 post-docs (25%), 23 faculty for <6 years (18%), 33 faculty of 6 + yrs (26%) as well as 4 professional ecologists (3%) and 2 others (2%). All in all, this seems comparable to demographics for other polls on this blog. More importantly (for the topic at hand), of our respondents, 50 self-identified as women, 4 as Indigenous peoples, 8 as people with disabilities and 12 as visible minorities – proportions that do not represent a random sample of the Canadian population, but that do exceed the CRC’s own availability targets based on applicant pools (with the exception of visible minorities). Therefore, while this sample is probably biased to some extent, it is also likely bigger and more diverse than most people’s regular lunch group, so these results are worth looking into!

Figure 1. Representation of members of designated groups among the poll respondents, and in the availability and population pools.

Are quantitative equity and diversity targets perceived as a compromise between two forms of discrimination?

To answer this question, we asked two yes/no questions:

• A policy for the promotion of equity and diversity in prestigious research positions (henceforth an “allocation policy”) without quantitative diversity targets would effectively discriminate against candidates from underrepresented groups. (Y/N)
• An allocation policy for prestigious research positions based on quantitative diversity targets would discriminate against candidates from already well-represented groups (e.g., white men in Canada). (Y/N)

We predicted a preponderance of Yes-Yes answer combinations. This was based, first, on well-documented discrimination via conscious or unconscious biases in the “old” system for which targets could be seen as a necessary part of the solution; second, that targets themselves almost by definition require discriminating candidates based on identity.

Figure 2. Poll respondents’ answers to the first two poll questions.

We were wrong! Overall, all four combinations of answers were given by at least 20% of respondents (Fig. 2). Still, there was a clear contrast between respondents belonging to one or more of the designated groups, for whom the most common answer combination was yes/no, and all others (mostly likely able-bodied white men), for whom the most common answer was no/yes.

So it does not seem like quantitative targets are broadly seen as a compromise between two forms of discrimination. These results suggest that people are more likely to see discrimination if they are part of the group potentially being discriminated against, and were the first of several to indicate that people in different groups can have very different perceptions about the costs and benefits of associated with different policies. (One could write a lot here speculating on explanations for different combinations of answers, but we will resist this temptation in order to keep things concise.)

Is there clear overall (dis)agreement with any of six different allocation policy options?

To answer this question, we asked whether people thought any of the following allocation policies (in addition to considerations of excellence) would be appropriate (on a 5 point scale, from “strongly disagree” to “strongly agree”).

• No policy.
• Provide guidelines to evaluators to take into consideration, as they see fit, the degree of representation of different groups of people, and possible conscious and unconscious biases they may have as evaluators.
• Establish and enforce diversity targets based on proportional representation in the available pool of potential nominees
• Establish and enforce diversity targets based on proportional representation in the pool of people with a Ph.D. in the appropriate discipline
• Establish and enforce diversity targets based on proportional representation in the pool of students who could benefit from the direct mentorship of researchers in prestigious positions.
• Establish and enforce diversity targets based on proportional representation in the entire population

We predicted that all policy options would have at least some support (i.e., there is a broad range of opinions on these questions) and, because it represents the status quo, the current “availability pool” approach would have the highest average level of agreement.

Fig. 3. Agreement with each allocation policy.

Fig. 3 shows the raw data (grey points, jittered), as well as the average level of agreement (solid line) (yes, we know means are statistically problematic here, but they seem well suited to rough comparisons all the same; alternative graphs/analyses welcome!). Policies with the least spread (and thus, the most consensus) are “no policy”, with which most people either disagreed or strongly disagreed, and “availability pool”, which had little spread, but also lukewarm agreement. “Guidelines” garnered stronger agreement, but with more spread among answers. We do not think this can be read to mean that people prefer guidelines to targets, since the questions did not specify that guidelines and targets were mutually exclusive (i.e., one can have guidelines and targets).

Overall, the more ambitious targets (i.e., generally higher proportions in designated groups) seem to have a bit less support than less ambitious targets. It is striking that non-trivial numbers of people (i.e., more than a few) show almost every level of (dis)agreement with every policy, but we see no obvious polarization in the form of bimodal distributions.

The answer to our question? A strong majority think “no policy” is inappropriate, and that guidelines are appropriate. But for all of the target options, almost as many people disagree as agree, with many feeling neutral. The obvious next question is whether this variation in opinions is driven by differences among designated groups or career stages.

Differences among groups

We predicted higher support for targets from people in designated groups than from those not in these groups.

Fig. 4. Agreement of women vs. others with each allocation policy.

For people who identified as women, there was a similar positive level of agreement for guidelines and all target options (Fig. 4). In contrast, for people who did not identify as women guidelines were popular, but their view on targets was neutral to negative (the latter for population-based targets).

Fig. 5. Agreement of designated groups other than women with each allocation policy.

For designated groups other than women, the story is a little different (Fig. 5). Although each of these groups includes a very small number of respondents, a pattern emerges when they are considered together. More so than the overall average, people in these groups tended to agree more with targets based on the mentor pool (and sometimes the general population) than with targets based on availability or Ph.D. pools.

Overall people in all groups show similar levels of support for guidelines, but support for quantitative targets, and different kinds of targets, appears variable among groups.

These results suggest that people likely to be negatively impacted by targets (e.g., non-women) tend not to be in favour, while people likely to see benefits (i.e., those in designated groups) tend to be in favour of at least some target options. Furthermore, people in designated groups currently most underrepresented are most strongly in favour of more ambitious targets based on representation at earlier stages in the pipeline. Overall, people generally favour policies that provide an advantage (relative to alternative policies) to members of the group(s) with which they identify: this could be due to greater awareness of barriers faced by that group and/or to a general tendency of all people to promote self-interest.

If agreement with any allocation policy indeed reflects different perceptions of associated costs and benefits, then we might expect different patterns for people at different career stages. For instance, we could imagine people who do not have a permanent position yet (grad students and post-docs) to feel tradeoffs more keenly, and therefore to show stronger contrasts between designated and non-designated groups (e.g., women vs. not women),  than people in more secure positions (faculty < 6 yrs, faculty for 6+ yrs, professional ecologists, others). We did see some signs in this direction, with grad-student and postdoc women showing stronger agreement with targets than not-women of any career stage or women with more secure positions (Fig. 6).

Fig. 6. Agreement with each allocation policy, as a function of gender and faculty employment.

Caveats

We are not social scientists and so there should no doubt be a long list of caveats here. We can offer two for now. First, people’s choices can be influenced by the range of options presented to them (the “anchoring effect”). For example, including more ambitious targets as options might essentially provide an “anchor”, making intermediate options seem more palatable to someone who might otherwise disagree with them. Second, by starting with questions about people’s identity, we may have magnified the degree to which subsequent questions were answered through the lens of those identities, thus increasing contrasts among groups.

Final thoughts

Are CRC targets really going to see drastic changes? After attending a recent local EDI workshop where even experts hadn’t heard of the changes, and finding only a somewhat buried reference to the new targets on government websites, we started to wonder whether the new targets would really be implemented. The answer is “yes” they will be, with a target date of 2029, as described clearly here. This is happening and so our questions pertain to a very real situation.

What does this sample of people think about the changes? Overall, there was less agreement with the new CRC targets (pop.pool) than with the old ones (avail.pool), driven by a lower level of agreement from people in non-designated groups, and a similar level of agreement from people in designated groups. In our experience, ecologists tend to be fairly progressive, so it seems unlikely that this sample of respondents is unusual in seeming less than enthused about the policy change. In other words, it seems unlikely that researchers from other natural sciences (chemistry, physics) and engineering might view the policy change positively. Policy is not determined by popular opinion, but nor can popular opinion be ignored entirely, so we think these numbers are useful in the ongoing EDI conversation.

At the same time, people in our sample (dis)agreed with different sets of targets in accordance with their cost and benefits for the group(s) with which they identify. At the very least then, this underscores the need for a broad diversity of voices at the table, since well-represented groups might otherwise simply pull the covers in the direction that happens to benefit them. Thus, for example, simply providing guidelines to a homogenous group of evaluators seems unlikely to successfully promote EDI.

In science, EDI conversations often revolve around the specific challenges and barriers faced by women. Here, women agreed with different kinds of targets than other designated groups. This is not surprising, but neither is it trivial. Other designated groups may be experiencing different sets of challenges, some of them tied to specific geographic regions. Therefore, if targets must be set at a national-level (as they now must, in the case of CRC), one productive direction for this discussion could be to ask at what scale should they be implemented (e.g., within universities, regions, disciplines, etc.) and should this scale be the same for all designated groups?

To conclude this (much longer than intended) post: having now looked at some length into this one narrow issue, we are left with substantial questions that we’d like throw back to you. Specifically: what will constitute success with respect to EDI in the CRC program? Given the strong incentive structure involved, it seems likely that the CRC program will reach its EDI targets. Should we then declare victory? Is the faithful representation of Canadian society in science and our institutions the goal itself, or are we striving for every Canadian to be able to pursue the academic career of their choice? These questions have direct policy implications. The scientific community will be dealing with issues related to equity, diversity and inclusion for a long time yet, but consensus on this point seems necessary for the conversation to move forward.

* The new targets for both people with disabilities and Indigenous peoples are a little more complicated. In the case of people with disabilities, the CRC has set the target somewhat lower than representation in the whole population to account for challenges associated with self-reporting. In the case of Indigenous peoples, while the target does appear to be based on the whole population, possible revisions are open to discussion (see questions 41 and 42 here).

** Based on the figures reported here and here, using the percentage of women specific to NSERC.