The Pulse of the Gulf

IT MAY BE NEW ZEALAND’S largest Marine National Park, but very little is known about what goes on beneath the surface. Which is why Associate Professor Rochelle Constantine from the School of Biological Sciences and the Institute of Marine Science is on a mission to better understand the key drivers behind marine megafauna – like whales – that provide important indicators of the
Gulf’s health.

“The key purpose of this work is to understand the Hauraki Gulf and the megafauna, what makes it function and what’s important to keep it functioning under some quite intense pressures.”

Covering almost 5,000 square kilometres, the relatively shallow waters of the Gulf have long been recognised as a productive and dynamic ecosystem – from the abundant phytoplankton at the beginning of the food chain to the mussels, rays, dolphins, Bryde’s whales and seabirds that were recorded in oral traditions when Māori first arrived.

However, fish and shellfish stocks are only “a shadow” of what they used to be in the Gulf says Rochelle, and she plans to draw on satellite technology that is now being used in marine science to measure everything from whale numbers to sea surface temperatures and phytoplankton. “I want to be able to measure the ocean using some of the remote sensing tools developed for space research.”

Capturing an accurate picture of the gulf

Called the ‘Pulse of the Gulf’, Rochelle’s latest project will focus on the feeding aggregations known in fishing parlance as ‘work ups’ that are a familiar sight to many boaties. “What are the drivers behind these multi species feeding aggregations, what are the things that draw these animals here in this time and place – and how does that change over time?”

In addition to images captured from air surveys, Rochelle wants to “look through the sea” which is where radar, drones and artificial intelligence come in. “The world is a beautiful place when it’s in a controlled lab environment setting,” says Rochelle, but the reality is that the marine animals operate in a 3D environment and drones and underwater cameras will allow her to step into that a lot better than she’s ever been able to.

On Rochelle’s shopping list is a fixed wing drone costing in the order of $200,000, and work is now underway using machine learning algorithms to train computers to recognise a plethora of high resolution images of sea life that have already been taken in the Gulf. But visually tracking a gannet that plummets into the sea at 100 kilometres an hour and leaves a trail of bubbles before emerging a few seconds later is easier said than done. 

Artificial intelligence helps process visual imagery

With a PhD in Computer Vision, Faculty of Science technical manager Dr Wannes van der Mark is providing technical and logistical support for the project. He says that while humans have exceptionally good visual abilities, such as finding a face in a crowd, when it comes to the natural environment “trying to get a computer to do this is another matter”.

Rising to that challenge is Associate Professor Patrice Delmas, head of the Intelligent Vision Systems laboratory within the School of Computer Science, who admits that New Zealand’s harsh light and strong wind present a challenging environment and that gannets do “lots of terrible things to our images. This is not a simple problem”.

Funded by the Vice-Chancellor’s Strategic Research Initiatives Fund, postgraduate students are developing an interface for annotation tablets that will translate and extract data from the images which can then be subjected to artificial intelligence (AI) techniques like deep learning to help understand how the various animals move in time and space.

“I’m quite confident we’ll get something out of it and it will be quite exciting,” says Patrice, whose “end game” is to have the data on marine life available online for anyone to use. “I want to be an enabler of these new technologies so that people don’t need to understand what’s behind it but can still get something interesting.”

For Rochelle, trying new things is “where interesting discoveries are made” and the use of AI tools will be a very efficient way to process visual imagery – and save valuable time. “These new technologies are incredibly powerful and affordable,” she says, “and the more information that you put into machine learning and AI development the better they get.”

The collaboration between the biological, computer and space sciences will hopefully allow Rochelle to build up detailed and minute layers of environmental information – including radar images of ocean-surface micro-eddies that may influence phytoplankton – to better understand the biology and ecosystem connectivity and relationships between the animals and their environment. “There’s this really lovely marriage that’s coming from accessing the tools and techniques that have been developed for space research.” 

Creating a framework to preserve the environment

The inaugural winner of the Blake Environmental Leadership award in 2018, Rochelle successfully fought to slow the passage of ships through the Gulf to prevent the killing of Bryde’s whales and says part of the overall vision of the project is to get people talking about the marine environment beyond it being a great place to hook snapper.

“It’s actually a place with a lot of life that needs to be looked after a lot better than we do and hopefully the health and the mauri of the Gulf will improve a lot as a result of this kind of work.”

Likewise, Patrice says the move towards automated systems could eventually provide people with a snapshot every day and create a framework for future generations to better understand and preserve the environment. “Kids get excited at school to be part of it, it’s very easy now to engage like that.”

The ‘Pulse of the Gulf’ will also put the University of Auckland RV Hawere to good use by deploying more conventional research tools such as hydrophones to record both natural and boat sounds.

Water samples will also be collected to look for dimethyl sulfide – the so-called ‘smell of the sea’ produced by phytoplankton – which some seabirds and maybe whales use to detect prey. And, if Rochelle has her way, an array of cameras and hydrophones will be submerged amongst a work up in order to capture it from within. “That’s a challenging part, but hopefully we’ll get data using that.”

Behind the scenes, Wannes is helping to develop sensible tools and standard operating procedures for use in the marine environment – and smoothing the way through the regulatory framework for the use of drones. As he puts it, “you don’t want headlines about drones crashing into a boat full of school children!”

Funded by generous grants from the Foundation North G.I.F.T. programme, the George Mason Centre for the Natural Environment, the Chisholm Whitney Charitable Trust, the Auckland Council, DOC, and the Waiheke Brewing Company, Rochelle hopes the project will inform people about what’s changed in the Gulf over the next decade “and we’re hoping it’ll change for the better”.