ABI and CAI work together to show their arty-science side

Ars Electronica is an arts and technology festival held every year in Linz, Austria, which attracts upwards of 100,000 visitors over five days. Due to the pandemic, in June the exhibitions were dispersed to 120 'Gardens' around the world, each of them connected through three dimensional digital exhibition spaces, facilitated by the Mozilla Hubs online platform. 

Which means that this year it can be visited by anyone in the world with a computer and good interent connection, from 9 - 13 September. The New Zealand exhibition, Ars Electronica Garden Aotearoa New Zealand, has been co-ordinated by Associate Professor Uwe Rieger of the School of Architecture and Planning at the University.

The purpose of the festival is to explore the connections between art, society, science and technology, and to ask questions about new technologies, their potential impact on society and the environment, both the good and not so good.

ABI has contributed several works, from both the Empathic Computing Lab (ECL) led by Professor Mark Billinghurst, and the Augmented Human Lab (AHL) led by Associate Professor Suranga Nanayakkara.

Art has met science in both the exhibitions, and in the creation of them - the ECL, for instance, has collaborated with students and researchers from the Faculty of Creative Industries. 

Such as in the creation of First Contact, which was designed by Mairi Gunn, who is studying for her PhD in Fine Arts (co-supervised by Professor Billinghurst and Associate Professor Deb Polson, who leads the Design Programme at the University), and in collaboration with the ECL.

In First Contact, visitors are invited to take a seat at a domestic table where, by wearing a head-mounted Augmented Reality (AR) display, they encounter a volumetric (3D) video representation of an indigenous woman seated opposite, who addresses visitors directly, and to provoke a consideration of what might constitute solid foundations for a healthy, ongoing relationship between people from diverse cultural backgrounds.

"I raise the issue of the existing power-imbalance between indigenous and dominant settler narratives,” says Mairi. “My wish is to playfully give visitors to the table an opportunity to have a direct, though digital/virtual, experience of connection with Māori and to practice active listening to indigenous voices.

“First Contact uses Extended Reality (XR) technology as a bridge between people, and invites a wider audience to feed their thoughts on intercultural relationships, and the use of XR in this context, back to us at the Empathic Computing Laboratory in the ABI.”

Another exhibition, Why do we…?  is the result of a collaboration between the Ryo Hajika and Amit Barde and their team at the ECL, and Dr Fabio Morreale and student Natalie Walker, both of the School of Music. Natalie composed the music for the installation, and Amit fine-tuned it.

In Why do we…?  Tweets and web posts have been channelled into an Audio Visual installation, to explore and understand the motivations that fuel a global movement such as the Black Lives Matter movement. Visitors are surrounded by words, voices and images related to the movement

“I wanted to identify what are important matters are to people, and to visualise them in a form of generative art,” says Ryo. “Also, I wanted to observe what those modern machine learning and artificial intelligence technologies reveal by analysing data people posted on the social networking services.”

Dr Morreale adds that the exhibition aims to raise awareness of our ambivalence about the role of technology and social networking in and on social issues. “On the one side, it enables participation and access to information and content production; on the other side it exacerbates issues of surveillance, privacy, and polarises political views.”

Another exhibition created by ECL researchers, Radarmin: An old way to interact, invites users to control a musical track in real-time. That is, to “play” a virtual musical instrument. The installation demonstrates the potential of emerging touch-less interaction interfaces, and how they can facilitate new means of musical expressions. Ryo, also part of the team behind Redarmin, says: “It reads people's dynamic motion into a musical expression. We also used a MR [mixed reality] system, so that a player can get a unique, immersive music playing experience.”

Meanwhile, Dr Nanayakkara’s team at the Augmented Human Lab at the ABI have created Jammify, which uses an interactive light display system to promote impromptu digital art jamming. It has two main components, a jamming canvas (2m x 6m wall made of LED lights) and the arm sleeve that provide haptic feedback – that is, electronic feedback that responds to and stimulates the sense of touch

Users can perform hand gestures in-front of the canvas to create digital drawings on the LED wall and ‘feel’ what they are drawing via the arm sleeve. They can also respond to and work with other users. “As social distancing is becoming the new normal, technology holds the potential to bridge this societal gap through novel technologies that allows multiple users to interact with each other and ‘draw’ and create together,” says Dr Nanayakkara. “Which reflects the philosophy that underpins the Ars Electronica festival.”

Visit Garden Aotearoa New Zealand at Ars Electronica, via ars.nz