On Friday, May 1, conductor, cellist, teacher, entrepreneur, and guest lecturer Paul Henry Smith, OC ’86, gave a lecture in Bibbins Hall. He holds degrees in theory, composition, and musicology. He has studied conducting with American legend Leonard Bernstein. Now, his research has shifted to focusing on artificial intelligence. During his talk last Friday, he proposed that AI can be used to generate musical interpretations of preexisting material, an alternative to prevalent conversations on AI generated music composition.
Smith started working with AI in the late 1980s through the MIT Artificial Intelligence Laboratory, where he worked on developing an AI system that could understand and predict how musical lines create motion. As a founding faculty member of Harris Conservatory in the ’90s, he worked on MAPLE, an AI system used to analyze and create species counterpoint, a pedagogical multi-voice harmonic exercise.
Professor of Music Theory and Conservatory Director for AI Innovation and Strategy Joseph Lubben overlapped briefly with Smith at Brandeis University when they were both pursuing Ph.D.s — Lubben in theory and composition and Smith in musicology. Smith reached out to Lubben when he heard of Oberlin’s Year of AI Exploration.
His talk centered around philosophical ideas of the present moment in live music making and an example of AI-generated interpretations of a Bach melody. Smith began by playing a recording of the melody in its concerto context and breaking down what made the melody distinctive, from the rhythmic structure to unexpected harmonic shifts. Then, he showed us what AI produced when fed an audio clip of the melody with a short genre prompt: rave, marching band, or film score. The results were inventive, if not gratuitous, and Smith’s point was that the AI was picking up on these defining features of the melody and augmenting them.
“It doesn’t know anything about what it did,” Smith said at the talk. “It picked up on the idea of a trill and extended it as a kind of variation inside another phrase. We hear that, but it doesn’t know. … A composer would have thought of that, but this is not thinking.”
While this may seem like a pedantic example, Smith has done a lot of practical work using AI to create tools for musicians, rather than replace them. His app, Cadenza, which plays MIDI accompaniment tracks for select concerti, uses AI to listen to performers practicing so that it can adjust to their tempo and turn pages for them so that performers can practice with a flexible accompaniment track, as it would be in performance. The program was developed while in residency at the New England Conservatory. When Cadenza was released 10 years ago, it was named best new iOS app in 22 countries.
“[AI is] a very dangerous thing, and it’s got a lot of irresponsible things coming out of it,” Smith said. “I try to [look] for things that are positive and [make] them happen. What I do think this moment is good for is helping people actually reveal human needs.”
In line with this ambition, Lubben pointed to decades of technological advancement in music.
“There are literally hundreds of tools before [generative] AI that people have used in production, everything from autotune … to stem separation, and some things that are quite useful, and many of them in the service of homogenizing music,” Lubben said.
Smith is currently working with the Symphonic Laboratory, an organization he founded which is geared toward projects like increasing the musicality of human musicians through a pattern recognition mindset and the use of AI-composed music.
“A computer can generate notes, but only a live performance (real people, real sound, real sweat) can transform that into an experience you’ll never forget,” the Symphonic Laboratory website reads. “We’re looking for top musicians who believe that art, at its best, is about more than novelty or technology: it’s about wonder, discovery, and bringing people together for experiences that matter.”
This talk is part of Oberlin’s larger initiative to explore what an artistic world with AI will look like: What are its limitations and uses? One limitation is that music-based AI is technologically leagues behind language- and image-based AI.
“AI is horrible at notation because there’s no defense department grants,” Smith said. “The reason they are so good at understanding language and imagery is because that really helps [the U.S. Defense Department].”
Furthermore, the big corporate players in AI are also disinterested in music.
“At Google and Meta and Adobe, they use music as a way to learn how AI can be built,” he said. “They don’t care about music. It’s just the same way they use art and writing. And then, by the way, they turn it around and make a product you could use — it generates stuff. I’m not interested in that. … I’m interested in any technology that can help a musician.”
