Researchers, check your bias on ethnicity data

Researchers’ decisions on how to classify ethnicity in quantitative research matter a lot because findings in turn affect important areas like health and social services, according to recent research from the University of Auckland.

Research fellow Dr Esther Yao’s PhD, which specifically focuses on quantitative research (where data involves numbers) as opposed to qualitative (where data involves words), found that measuring ethnicity is not at all straightforward.

“Researchers can select from around four possible methods to classify ethnicity,” she says.
“There is a misconception that quantitative research, because it’s all about numbers, is objective, but this research empirically shows that it's not.”

Yao found the decisions researchers make on how to classify someone who, for example, identifies as both Māori and Pākehā, or both Asian and Pākēha, can affect research conclusions substantially.

One of her studies involved looking at mental health outcomes in young people using data from the Youth2000 survey series.

“We found that if we chose a particular method to classify ethnicity, the conclusions about whether two groups are different or not, and also the size of that difference, could be substantially different from if we had chosen another classification method.”

And different conclusions, says Yao, inevitably have an impact on equity and social justice.

“Because interventions in areas like mental health, for example, depend on what researchers say the disparities are, how ethnicity is measured can have a large bearing on who is resourced and who isn’t.”

She says a big issue in this field is that many people now, and particularly young people, identify with multiple ethnic groups, but the most common techniques for analysing data favour mutually exclusive groups.

“For example, it wants us to classify someone as either Māori or Pākehā, which means researchers have to make a choice about how we classify people who report more than one ethnicity.”

Even asking people to choose one ethnicity over another can be problematic and make people feel uncomfortable, says Yao.

“Ethnicity is not neutral. I was motivated to look into this further after I read about the history of race, racism and colonisation, and notions of superiority and inferiority.”

Another issue, she discovered, is that depending on which classification method is used, the ethnic groups constructed as a result can have a different demographic composition.

“So for example, one method resulted in a group who live in higher socioeconomic deprivation, and deprivation tends to be associated with poorer mental health outcomes, so that would influence recommendations accordingly.”

Yao says that quantitative research is typically seen as neutral and objective, but her research shows that complacency and presumed objectivity is dangerous and risks coming to false conclusions.

“Researchers can come to substantively different conclusions depending on their methodological decisions and underlying worldview.”

So how can researchers avoid this issue?

“We need to be reflexive about the decisions we make and fit the method to the research question. And in the same way we consider our positionality [e.g., ethnicity, gender, socioeconomic status] when doing qualitative research [which is acknowledged as being subjective], we need to do the same with quantitative research.”

“Researchers need to be asking, ‘Why am I making this decision and what implications will it have for the conclusions being drawn? Who will these conclusions benefit? Are we perpetuating the status quo or advancing social justice?’”

Researchers also need to recognise their position of power, Yao believes.

“The decisions we make can really shape the narrative and the production of knowledge, and this has real impact on people’s lives.”

Esther Yao's PhD, which she completed in the University's Faculty of Education and Social Work, looked at three groups: children, adolescents and adults. She used data with permission from the Growing Up in New Zealand longitudinal study and the Youth2000 survey series.

Her thesis Classifying ethnicity in multi-ethnic contexts: Implications of methodological decisions on quantitative research was nominated for the 2022 Vice-Chancellor’s Prize for Best Doctoral Thesis.