Using augmented reality to revisit ‘first contact’

The awards were established to support women who are conducting research that benefits women and or girls in Aotearoa New Zealand.

Ms Gunn’s PhD is being supervised by Angus Campbell, new head of the Design Programme, co-supervised by Professor Mark Billinghurst, who leads the Empathic Computing Lab at the ABI, and Moana Nepia who is currently a Research fellow at the James Henare Māori Research Centre. She is also a part-time Professional Teaching Fellow on the Design Programme.

She received the award for her research, common/room, an investigation into the potential of XR (extended reality) technologies to support and enhance intercultural social engagement, using Virtual Reality (VR), Augmented Reality (AR) as well as real-world realities.

Her research explores finding ways to reimagine the moment of ‘first contact’ between people from different cultures, including tāngata whenua and tauiwi, newcomers.

“In the context of a colonised world, first encounter dramas echo down the years,” she says. “For Māori, first contact was and is traumatic.”

Ms Gunn had over two decade’s experience as a cinematographer in the NZ film and television industry, including on documentaries for Māori and mainstream television, before enrolling for her PhD in late 2016.

Through her research she aims to better understand social interaction and human psychology and use that in combination with new technologies to draw attention to and improve Māori/Pākehā relations in the real world.
 

She has recently designed and led the project called Haptic Hongi, one of over 20 installations from New Zealand to feature in Ars Electronica 2021, in which she worked with researchers in the Empathic Computing Lab at the ABI and multimedia artist Tania Remana.

Ars Electronica is the world’s largest media arts festival, an event that originated Linz, Austria, which showcases installations at the nexus of art, technology, and society. Due to pandemic travel restrictions the festival moved online in 2020, and this year (partly due to travel restrictions) features 100 “gardens” from around the world, a hybrid of real-world and virtual installations.

Haptic Hongi will feature in Garden Aotearoa, co-hosted by the University of Auckland and Te Herenga Waka-Victoria University of Wellington, a 3D cyber exhibition which can be accessed from anywhere in the world via computer screens, mobile devices or virtual reality headsets.

Haptic Hongi features a wahine Māori sitting at table. Visitors see the woman in pre-recorded volumetric video, seated opposite. (Volumetric video appears in 3-D space and can be viewed in 2-D on flat screens as well as using with 3D displays and AR Hololens II headsets.)

The wahine Māori locks the visitor in a mutual gaze, introduces herself and invites the visitor/viewer to respond.

However, during the response, a computer-generated avatar of the wahine Māori replaces the volumetric video version. Can visitors tell the difference between both versions, and if so, which version feels or is experienced as more real in the virtual world?

The volumetric wahine Māori returns to replace the avatar and invites the visitor to join her in a hongi. Had the exhibition been able to go ahead as initially planned in the ‘real world’ (at the University of Victoria, Wellington), viewers could literally feel the pressure of the hongi through haptic feedback. Bioengineers at the ABI developed actuators fitted to the AR headset that visitors could wear (in the real pre-lockdown world). This would create the experience of touch through vibration.

The real version of the wahine Māori is multi-media and performance artist Tania Remana.

“I would like people who haven’t met Tania to have a sense of connection so that they might be less inclined to objectify or stereotype Māori in future,”, says Ms Gunn. She hopes that even in the virtual world, her work can help build better connections and understanding between people in the real world.

While Haptic Hongi is an exploration of the potential of technology, she says “technology is a tool that I want to work with, but it's only a tool. It’s not a final destination. I just really want to find ways to connect people together.”

Other members of the team behind Haptic Hongi are Prasanth Sasikumar, Huidong Bai, Sachith Muthukumarana (ABI) and UK artist/designer Wendy Lawn (TechnomancyXR, UK).

The Garden Aotearoa cyber exhibition is on 8-12 September, and all online events are free, though some aspects of the Ars Electronica festival originating in other countries carry modest charges. A full programme of New Zealand events is available at ars.nz.