Equity and diversity targets in science, part II: views depend on identity (guest post)

Note from Jeremy: This is a guest post from Françoise Cardou & Mark Vellend.

************

A few weeks ago, we came to you with a knotty lunchtime debate : are quantitative equity and diversity targets in science a good idea, and if so, on what basis? Unequivocally, certain groups of people face unequal challenges and barriers in science. Any specific policy measure to address this issue necessarily comes with both benefits and costs, and we wanted to find out what people (from a broad range of situations) actually think. We want to thank everyone who chimed in: you are now honorary participants in our lunch group. We return today with the results.

First, a brief recap. Our questions were motivated by changes to the diversity targets used in the Canada Research Chair (CRC) program, which have for a few years now been based on the “availability” of people from designated groups (women, Indigenous peoples, people with disabilities, and visible minorities) in the pool of candidates. Following a lawsuit from several Canadian faculty members under the Human Rights Act, the new CRC targets will now be based on the representation of people in these same groups in the general population*. As one example, the target for women will increase from 21% to 50% in the natural sciences and engineering. These represent two ways of setting quantitative targets, and we can imagine a range of rationales for different sets of such targets, each with a different goal and balance of cost and benefits (see the original post).

We intended first to find out whether people perceived quantitative EDI targets as a compromise between two forms of discrimination, and second whether there was any allocation policy around which there was broad agreement. Finally, a few obvious but important points: this poll is not a scientific study (it’s an online poll), and the results do not say anything about what is right or wrong, simply that some people feel this or that way. Should anyone like to have a look for themselves, you will find the full dataset here (please let us know if you dig deeper, we’d love to know what you find!).

Who responded?

We had 128 respondents (Fig. 1). This included 34 grad students (27%), 32 post-docs (25%), 23 faculty for <6 years (18%), 33 faculty of 6 + yrs (26%) as well as 4 professional ecologists (3%) and 2 others (2%). All in all, this seems comparable to demographics for other polls on this blog. More importantly (for the topic at hand), of our respondents, 50 self-identified as women, 4 as Indigenous peoples, 8 as people with disabilities and 12 as visible minorities – proportions that do not represent a random sample of the Canadian population, but that do exceed the CRC’s own availability targets based on applicant pools (with the exception of visible minorities). Therefore, while this sample is probably biased to some extent, it is also likely bigger and more diverse than most people’s regular lunch group, so these results are worth looking into!

1_GroupRepresentation

Figure 1. Representation of members of designated groups among the poll respondents, and in the availability and population pools.

Are quantitative equity and diversity targets perceived as a compromise between two forms of discrimination?

To answer this question, we asked two yes/no questions:

  • A policy for the promotion of equity and diversity in prestigious research positions (henceforth an “allocation policy”) without quantitative diversity targets would effectively discriminate against candidates from underrepresented groups. (Y/N)
  • An allocation policy for prestigious research positions based on quantitative diversity targets would discriminate against candidates from already well-represented groups (e.g., white men in Canada). (Y/N)

We predicted a preponderance of Yes-Yes answer combinations. This was based, first, on well-documented discrimination via conscious or unconscious biases in the “old” system for which targets could be seen as a necessary part of the solution; second, that targets themselves almost by definition require discriminating candidates based on identity.

2_DiscriminationOrNot

Figure 2. Poll respondents’ answers to the first two poll questions.

We were wrong! Overall, all four combinations of answers were given by at least 20% of respondents (Fig. 2). Still, there was a clear contrast between respondents belonging to one or more of the designated groups, for whom the most common answer combination was yes/no, and all others (mostly likely able-bodied white men), for whom the most common answer was no/yes.

So it does not seem like quantitative targets are broadly seen as a compromise between two forms of discrimination. These results suggest that people are more likely to see discrimination if they are part of the group potentially being discriminated against, and were the first of several to indicate that people in different groups can have very different perceptions about the costs and benefits of associated with different policies. (One could write a lot here speculating on explanations for different combinations of answers, but we will resist this temptation in order to keep things concise.)

Is there clear overall (dis)agreement with any of six different allocation policy options?

To answer this question, we asked whether people thought any of the following allocation policies (in addition to considerations of excellence) would be appropriate (on a 5 point scale, from “strongly disagree” to “strongly agree”).

  • No policy.
  • Provide guidelines to evaluators to take into consideration, as they see fit, the degree of representation of different groups of people, and possible conscious and unconscious biases they may have as evaluators.
  • Establish and enforce diversity targets based on proportional representation in the available pool of potential nominees
  • Establish and enforce diversity targets based on proportional representation in the pool of people with a Ph.D. in the appropriate discipline
  • Establish and enforce diversity targets based on proportional representation in the pool of students who could benefit from the direct mentorship of researchers in prestigious positions.
  • Establish and enforce diversity targets based on proportional representation in the entire population

We predicted that all policy options would have at least some support (i.e., there is a broad range of opinions on these questions) and, because it represents the status quo, the current “availability pool” approach would have the highest average level of agreement.

3_OverallAgreement

Fig. 3. Agreement with each allocation policy.

Fig. 3 shows the raw data (grey points, jittered), as well as the average level of agreement (solid line) (yes, we know means are statistically problematic here, but they seem well suited to rough comparisons all the same; alternative graphs/analyses welcome!). Policies with the least spread (and thus, the most consensus) are “no policy”, with which most people either disagreed or strongly disagreed, and “availability pool”, which had little spread, but also lukewarm agreement. “Guidelines” garnered stronger agreement, but with more spread among answers. We do not think this can be read to mean that people prefer guidelines to targets, since the questions did not specify that guidelines and targets were mutually exclusive (i.e., one can have guidelines and targets).

Overall, the more ambitious targets (i.e., generally higher proportions in designated groups) seem to have a bit less support than less ambitious targets. It is striking that non-trivial numbers of people (i.e., more than a few) show almost every level of (dis)agreement with every policy, but we see no obvious polarization in the form of bimodal distributions.

The answer to our question? A strong majority think “no policy” is inappropriate, and that guidelines are appropriate. But for all of the target options, almost as many people disagree as agree, with many feeling neutral. The obvious next question is whether this variation in opinions is driven by differences among designated groups or career stages.

Differences among groups

We predicted higher support for targets from people in designated groups than from those not in these groups.

4_WomenNotWomen

Fig. 4. Agreement of women vs. others with each allocation policy.

For people who identified as women, there was a similar positive level of agreement for guidelines and all target options (Fig. 4). In contrast, for people who did not identify as women guidelines were popular, but their view on targets was neutral to negative (the latter for population-based targets).

5_OtherGroups

Fig. 5. Agreement of designated groups other than women with each allocation policy.

For designated groups other than women, the story is a little different (Fig. 5). Although each of these groups includes a very small number of respondents, a pattern emerges when they are considered together. More so than the overall average, people in these groups tended to agree more with targets based on the mentor pool (and sometimes the general population) than with targets based on availability or Ph.D. pools.

Overall people in all groups show similar levels of support for guidelines, but support for quantitative targets, and different kinds of targets, appears variable among groups.

These results suggest that people likely to be negatively impacted by targets (e.g., non-women) tend not to be in favour, while people likely to see benefits (i.e., those in designated groups) tend to be in favour of at least some target options. Furthermore, people in designated groups currently most underrepresented are most strongly in favour of more ambitious targets based on representation at earlier stages in the pipeline. Overall, people generally favour policies that provide an advantage (relative to alternative policies) to members of the group(s) with which they identify: this could be due to greater awareness of barriers faced by that group and/or to a general tendency of all people to promote self-interest.

If agreement with any allocation policy indeed reflects different perceptions of associated costs and benefits, then we might expect different patterns for people at different career stages. For instance, we could imagine people who do not have a permanent position yet (grad students and post-docs) to feel tradeoffs more keenly, and therefore to show stronger contrasts between designated and non-designated groups (e.g., women vs. not women),  than people in more secure positions (faculty < 6 yrs, faculty for 6+ yrs, professional ecologists, others). We did see some signs in this direction, with grad-student and postdoc women showing stronger agreement with targets than not-women of any career stage or women with more secure positions (Fig. 6).

6_CareerStage

Fig. 6. Agreement with each allocation policy, as a function of gender and faculty employment.

Caveats

We are not social scientists and so there should no doubt be a long list of caveats here. We can offer two for now. First, people’s choices can be influenced by the range of options presented to them (the “anchoring effect”). For example, including more ambitious targets as options might essentially provide an “anchor”, making intermediate options seem more palatable to someone who might otherwise disagree with them. Second, by starting with questions about people’s identity, we may have magnified the degree to which subsequent questions were answered through the lens of those identities, thus increasing contrasts among groups.

Final thoughts

Are CRC targets really going to see drastic changes? After attending a recent local EDI workshop where even experts hadn’t heard of the changes, and finding only a somewhat buried reference to the new targets on government websites, we started to wonder whether the new targets would really be implemented. The answer is “yes” they will be, with a target date of 2029, as described clearly here. This is happening and so our questions pertain to a very real situation.

What does this sample of people think about the changes? Overall, there was less agreement with the new CRC targets (pop.pool) than with the old ones (avail.pool), driven by a lower level of agreement from people in non-designated groups, and a similar level of agreement from people in designated groups. In our experience, ecologists tend to be fairly progressive, so it seems unlikely that this sample of respondents is unusual in seeming less than enthused about the policy change. In other words, it seems unlikely that researchers from other natural sciences (chemistry, physics) and engineering might view the policy change positively. Policy is not determined by popular opinion, but nor can popular opinion be ignored entirely, so we think these numbers are useful in the ongoing EDI conversation.

At the same time, people in our sample (dis)agreed with different sets of targets in accordance with their cost and benefits for the group(s) with which they identify. At the very least then, this underscores the need for a broad diversity of voices at the table, since well-represented groups might otherwise simply pull the covers in the direction that happens to benefit them. Thus, for example, simply providing guidelines to a homogenous group of evaluators seems unlikely to successfully promote EDI.

In science, EDI conversations often revolve around the specific challenges and barriers faced by women. Here, women agreed with different kinds of targets than other designated groups. This is not surprising, but neither is it trivial. Other designated groups may be experiencing different sets of challenges, some of them tied to specific geographic regions. Therefore, if targets must be set at a national-level (as they now must, in the case of CRC), one productive direction for this discussion could be to ask at what scale should they be implemented (e.g., within universities, regions, disciplines, etc.) and should this scale be the same for all designated groups?

To conclude this (much longer than intended) post: having now looked at some length into this one narrow issue, we are left with substantial questions that we’d like throw back to you. Specifically: what will constitute success with respect to EDI in the CRC program? Given the strong incentive structure involved, it seems likely that the CRC program will reach its EDI targets. Should we then declare victory? Is the faithful representation of Canadian society in science and our institutions the goal itself, or are we striving for every Canadian to be able to pursue the academic career of their choice? These questions have direct policy implications. The scientific community will be dealing with issues related to equity, diversity and inclusion for a long time yet, but consensus on this point seems necessary for the conversation to move forward.

* The new targets for both people with disabilities and Indigenous peoples are a little more complicated. In the case of people with disabilities, the CRC has set the target somewhat lower than representation in the whole population to account for challenges associated with self-reporting. In the case of Indigenous peoples, while the target does appear to be based on the whole population, possible revisions are open to discussion (see questions 41 and 42 here).

** Based on the figures reported here and here, using the percentage of women specific to NSERC.

9 thoughts on “Equity and diversity targets in science, part II: views depend on identity (guest post)

  1. Thanks again to both of you for doing this very interesting survey.

    I’m struck by the high level of disagreement about any policy besides “no policy” and “guidelines”. And I’m struck by the fact that the CRC’s new population pool target has only neutral support (on average) even among members of designated groups.

    I’m curious whether the (or a) intended goal of the population pool target is to encourage universities to allocate their CRCs to fields in which the *availability* pool already has appreciable representation of members of designated groups. For instance, allocating more CRCs to psychology (a field with higher representation of women than most others) so as to help meet the target of 50% women. On my reading, it certainly sounds like that’s the intent of the new policy. If that’s right, I want to think more about that. I can imagine arguments for and against a policy of diversifying CRCs as a whole by allocating CRCs to fields that are already comparatively diverse.

    Also, it’s not clear to me how the population pool targets will be enforced. The linked regulations mostly concern how institutions must conduct individual CRC searches–concrete steps that must be taken to ensure diverse applicant pools and fair search outcomes. And anytime within 48 months following the submission of a CRC nomination, the institution can be obliged to provide documentation that the search associated with that nomination followed the regulations. But unless I missed it (did I? I was skimming…), there’s nothing in the linked regulations about enforcement of the population pool targets. So it seems like it’s possible that a university that missed the population pool targets could be in the clear so long as it followed the procedures for each individual CRC search. I’d be curious to hear if others are ok with that hypothetical possibility. I think I am ok with it, though it depends just how badly the targets were missed. It certainly is possible for a series of individually-reasonable award decisions to be seriously biased in aggregate: https://dynamicecology.wordpress.com/2014/04/17/when-a-series-of-entirely-reasonable-decisions-leads-to-biased-outcomes-thoughts-on-the-waterman-award/. But if a series of individually-reasonable award decisions is only slightly biased in aggregate, then that seems ok to me. But what do others think?

    • Am I one of those folks you are describing, my reply to the original post might suggest so …

      p = proportion representation of the disenfranchised group among graduates with a PhD in the field
      a = proportion of the disenfranchised group among the pool of potential nominees
      t = proportion of the disenfranchised group among the total population

      Then perhaps a good diversity target is “min( max( a, p ), t )”

      • I wasn’t actually thinking of your very thoughtful comment Matthew, but thank you for reminding me of it. 🙂

        Reflecting further, your suggested target could be seen as a bit unambitious. For instance, for tenure-track faculty hiring in ecology in the US and Canada, p=52%, a=46%, t=50% (all those numbers are +/- one percentage point or so). But tenure-track faculty hires are 57% women, without any detectable cost in terms of the on-paper qualifications of women relative to men. So perhaps a better target is (min(max(a,p),t)+x, where x is some suitable positive percentage (that might itself depend on a or p.)

  2. To re-allocate CRCs from low-diversity disciplines to high-diversity disciplines would seem to just accept the status quo of diversity levels overall, and to work against some of the presumed objectives: e.g., breaking down stereotypes of who “belongs” in a given discipline, having mentors for people in groups underrepresented in a certain discipline. It seems like a way to play the numbers game, rather than to actually promote EDI.

    My understanding of enforcement was that universities would start to lose CRCs if they did not meet diversity targets, in which case it’s not about following a process with each hire, but explicitly about meeting quantitative targets. But I’m not 100% up to date on how this all will work.

    • “To re-allocate CRCs from low-diversity disciplines to high-diversity disciplines would seem to just accept the status quo of diversity levels overall, and to work against some of the presumed objectives: e.g., breaking down stereotypes of who “belongs” in a given discipline, having mentors for people in groups underrepresented in a certain discipline. It seems like a way to play the numbers game, rather than to actually promote EDI.”

      I’m inclined to agree, so I was trying to step outside my comfort zone and imagine a good argument for this. One might be if there are some fields that are undeservedly seen as unprestigious, that have high representation of people in designated groups, maybe those fields should be allocated more CRCs to raise their prestige. But I’d be interested to hear from others on this.

  3. Just looking at the “Guidelines for ensuring a fair and transparent recruitment and nomination process”… although they don’t address the specific point of penalties, they do say that a “proof of transparency of processes” includes showing that the “final offer is in line with the institution’s equity targets.”

    http://www.chairs-chaires.gc.ca/program-programme/equity-equite/guidelines-lignes_directrices-eng.aspx

    Then from question 36 in the FAQ:

    “In the case of an institution not meeting these deadlines or program requirements, the institution will not be permitted to submit new nominations to the program (other than in cases where a nomination contributes to meeting their equity targets) until it is compliant with program requirements.”

    http://www.chairs-chaires.gc.ca/program-programme/equity-equite/faqs-questions_frequentes-eng.aspx#q36

    This rather sounds like enforcement to me, but maybe there’s a loophole I’m not seeing? Presumably more specific mechanisms will be addressed in early 2020, when the new action plan comes out.

    On the other point: in order to meet population-based targets on a 10 year horizon, it seems inevitable that, at least to a some degree, CRCs will shift to disciplines or subdisciplines where designated group representation is comparatively higher. If this is the case, then I agree that the new program will not really be addressing unequal challenges & barriers.

    I’m not so sure that it this means “accepting the status quo” though. Regardless of what discipline or subdiscipline receives a CRC, I think enforcing population-based targets for a group like Indigenous Peoples (with low representation across the board) is likely to lead to increased diversity overall.

  4. One of the lawyers involved in the lawsuit against the CRC that led to these changes has commented on Twitter:

    In response, I’d note that this post and the previous post both recognize–at length–the facts that law and ethics aren’t popularity contests. Neither post says or implies that the new CRC policy should be changed merely because it’s not the most popular option among the poll respondents (even among the poll respondents from designated groups). But as the post also notes, law and ethics aren’t the only things worth thinking about if you want to bring about social change (at least in my view).

    Prof. Attaran notes that popular support for a failed policy–guidelines–is precisely why legally-enforced targets are needed, because otherwise the CRC would just stick with a popular-but-ineffective policy. I think that’s a good point.

    I wish Prof. Attaran had commented on the hypothetical but plausible-seeming (at least to me…) possibility that the population targets will be met by allocating CRCs to fields that already have comparatively high representation of designated groups. I’m sure that he knows and has thought much more about this possibility than me.

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.