Are we counting on the right evidence?

What our government relies on when it comes to evidence for their social well-being decisions is the question behind a new study, to be led by Dr Eileen Joy from the University of Auckland.

A professional teaching fellow in social work in the University’s Faculty of Education and Social Work, Joy is the recipient of a Mana Tūāpapa Future Leader Fellowship from the Royal Society Te Apārangi, worth $820,000 over four years.

She says one of the concerns that came out of her PhD research, which focused on child protection policies in Aotearoa, was the type of knowledge being prioritised as evidence to inform various policies.

“What I discovered was the privileging of very scientific-based forms of knowledge, which sidelined Te Ao Māori, for example, as well as any other ways of understanding the same issues and problems.”

For example, she says, there are different ways of considering attachment theory, which is based on the idea that a child's early relationship with one primary caregiver, usually their mother, is crucially important for their social development and future relationships.

“This theory is used a lot in child protection, but its critiques, some of which centre on how things are viewed differently in non-western cultures who might have a more collective view of child rearing, weren’t being considered; so certain ways of thinking about families and children were dominating our policies in that area.”

Joy says governments typically use lots of quantitative, rather than qualitative, science, which is favoured in the “hierarchy of evidence”.

“Quantitative data, which is all about numbers, tends not to consider why those numbers are there in the first place; it doesn’t take into account poverty, colonisation … any of the reasons why Māori, for example, might be overrepresented in our poorest statistics.”

She says the New Zealand government often ends up relying on evidence from other countries, from the US and the UK for example, based on research that’s not been done within our own population, which has its own distinct profile and needs.

A related issue is what degree of access policymakers in the social well-being area have to a wide range of evidence, considering the high prevalence of paywalls.

“I believe when taxpayers fund research, via big government grants to universities, that research should then be open access, but that's often not what happens.”

Instead, she says, government agencies and governments that fund research must then pay a second time to get access to it once it’s published; meanwhile, academic publishing companies are making an awful lot of money.

And which government ministry you're in will dictate the type of access you have to academic databases, says Joy.

“The Ministry of Health, for example, is likely to have wide access, but are those policymakers able to access social science databases as well? By contrast, academics generally have access to an immense amount of research.”

Another issue she’ll be investigating is the role of “casual, informal reading” and how this affects what policymakers decide.

“From my doctoral research, I found social work practitioners, for example, were quite informed by this kind of reading; like a book they’d grabbed from the library, or a blog or social media post; so, if that’s happening there, it’s probably happening at the highest levels of government as well.”

And if we've got policy makers who are being fed a diet of algorithm-informed information from social media companies, then what are those informal influences that are then informing the policy they create? asks Joy.

“An example of this might be the recent panic about transgender children and adults which had people responding to ‘informal influences’ as opposed to verified research.”

A further question, she believes, is the role of generative AI in helping write policy, given its known biases and tendency to reflect western ways of viewing the world.

“In an environment where our indigenous population, Māori, are already marginalised, what does that mean for the resulting policy?”

Is it possible for policy to be ‘politically neutral’ is another interesting question, says Joy.

“What happens when good policymakers are ignored, or told the evidence they’re presenting can't be used? We’ve seen the results of that under Trump in the US in particular, and no doubt it’s happening in many other countries.”

Joy’s study plans to focus on “the silences and gaps”.

“That’s always going to be more challenging than examining what's actually there, because you have to consider what’s been left out, and whose voices are not being heard.”