As composed by David Chisholm ... and AI

Notable composer Dr David Chisholm, head of the University of Auckland’s School of Music, decided to collaborate with AI on an ambitious experiment.

The idea came after an informal chat with colleague Professor Rodrigo Cádiz, an expert in creativity and AI at Universidad Católica in Santiago, Chile.

“I asked Rodrigo if he could train the AI and then prompt it to produce four, five-minute original pieces based on four of my own works, essentially asking it to compose something new in what it perceived as my style,” says Chisholm.

“I would then commit to transcribing by ear the audio he sent me into scores for full orchestra.”

And the results were nothing like he’d imagined.

“At first I just couldn’t recognise the audio it produced as anything I would have composed. They took much longer to transcribe than I anticipated, but the more I worked on them, the more, weirdly, they did sound like me.”

Chisholm says the pieces were quite contrasting in style, as each version used a different AI environment and different prompts. In turn, each piece posed a different challenge to orchestrate.

He developed a process that combined the industry standard music notation software programme Sibelius, as well as various analytical tools to locate pitches, timecodes and textural density.

He says the role of an orchestrator or arranger is a process of taking an existing work and arranging it for a specific ensemble, be that a string quartet or a symphony orchestra.

“It’s often regarded as ‘lesser’ in some way that composing something original, but in fact, the process involves an enormous amount of creativity and skill to do it successfully.”
 

The quality of prompts is all-important, he discovered.

“The closer the audio produced by AI sounds like an actual orchestra, the more straight forward the process of transcription. Trying to map electronic-sounding audio onto the acoustic world of an orchestra requires not only a technical understanding of how both media function, but also a huge dose of the imaginary: something that AI doesn’t currently possess.”

Chisholm says he’s found working with AI in this way “a mind-blowing experience” and something that’s really brought into question the notion of authorship and who’s in charge.

“The way I work with AI is, in some ways, the same way I’d work respectfully with any musician or conductor playing my work. I’m open to input, but then I might say, ‘I want more colour here’, or ‘You need more vibrato there’, so at the end of the day, I’m still the boss, the creative force that ultimately drives the work.”

What it showed him, he says, is that he’s as much a product of his own training as is AI.

“AI makes choices based on the knowledge it builds from pattern recognition. It then responds within the limits of what it observes and then speculates on possibilities, not probabilities, when given specific guiding parameters. In a sense, that's exactly what I'm doing too.”

The first movement of Chisholm’s AI-inspired work, ‘Fountain’, was performed in November last year by the Auckland Philharmonia with around 70 musicians, in collaboration with the New Zealand Dance Company, who created an original work set to the music.

He is currently working on completing the four-movement suite to take back to the Philharmonia for a performance next year. And while he will be credited as the composer, his colleague Rodrigo Cádiz and the AI will also be credited for their roles.

Before then however, he’s heading back to Chile for a three-week residency at a new composition and technology laboratory, working with musicians from Chile, Costa Rica and Venezuela, led by his ‘Fountain’ collaborator, Professor Cádiz.

“Since 2012, I’ve been to Chile a lot for family, so to be able to spend a chunk of time there working reinforces my determination to be a composer whose practice remains focused on contemporary Southern hemisphere cultures and perspectives.”

Meanwhile his creative adventure with AI continues.