What Will Artificial Intelligence Mean for the Music Industry?
AI programs offering collaborative and cost-efficient advantages to musicians.
Photo Source: Medium
Artificial intelligence has been exploding recently. According to Google Trends, the topic was most popular as a Google search in 2022 between December 4 - 10, when its prevalence as a search term skyrocketed.
This rapid increase coincided with the Nov. 30, 2022, public release of Open AI’s ChatGPT, the conversational chatbot which has taken the world — via Twitter — by storm.
As a college student in the midst of finals week, I was amazed by its potential to assist students writing essays, coding and completing other assignments. You can read more about ChatGPT’s future in academia here.
There now exist questions about how rapidly emerging artificial intelligence processes will impact other sectors, including creative ones. I explore a few possibilities in the music industry below.
AI: The Great Democratizer?
One potential for artificial intelligence in music is its ability to eliminate some of the barriers to entry in the industry. Traditionally, becoming a successful musician is an expensive endeavor. One needs access to studio time, producers and audio engineers. That’s just half the battle, and excludes the costs of the talent management side, such as an agent and marketing costs. Now, AI processes present options to cut some of these creation costs.
MuseNet and ChatGPT: Creative assistance and collaboration
This video from YouTuber and artist Gami is enlightening in a few ways. First, Gami plays a simple piano loop and uses Open AI’s MuseNet to add additional instrumentation. MuseNet interprets the piano loop it's given and — using the dataset of harmonies and rhythms on which it was trained — adds to the loop. Gami indicated that he wanted piano, harp, guitar, winds and strings, so the model adds those instruments, in rhythm, to his loop. As you can see in Gami’s astonished reaction, the results sound great.
The barrier broken here is access to resources (instruments) and collaborators. Say you’re a pianist looking to create more instrumentally layered music, but don’t have the time or money to learn other instruments, and none of your friends play. With the help of AI programs like MuseNet, that’s no longer a problem.
A second barrier broken in the video is writer’s block. Not only can ChatGPT help with homework, but as seen in this video, it can write chord progressions and lyrics for human creators to work with.
I recommend watching the entire video, but I especially enjoyed this monologue where Gami provides a nuanced and sobering assessment of how creators are viewing this rapidly developing technology. He gets into why the tech is scary, but also how it can be used as a collaborative tool.
LANDR: Audio enhancement
Another daunting potential cost for musicians is enlisting the services of an audio engineer to help record, mix and master their tracks. As DIY artists are forsaking the previous industry norm of giving up a sizable portion of revenue to record labels, opting to record in their own homes, AI generated mixing and mastering services become attractive.
LANDR is just one example, but it offers mixing and mastering services on either a case-by-case or subscription basis. It is as simple as dragging a track into the software and choosing from a couple stylistic mastering options before choosing an export option (WAV or MP3).
AI in the Driver’s Seat?
It is now also possible to create music with AI taking the reins. The aforementioned Open AI also runs JukeBox, a generative model which produces raw audio. The linked website is worth checking out, especially the audio samples which are meant to replicate popular artists’ sounds — pop music in the style of Katy Perry, rock in the style of Elvis Presley, etc.
The results are muffled and the lyrics often indecipherable — almost as if you are listening underwater — though you can hear semblances of familiar song structures. While the innovation impresses, this isn’t anything that is going to replace the real Katy Perry, at least not in its current iteration.
For some insight as to how the program works, check out this video, which explains how the neural network produces its output. In this video, the song is essentially a modified riff on the Christmas classic “Jingle Bells.” The actual music begins at 11:37 if you want to skip the explanation.
It’s easy to get lost in the coding weeds, but the essence of the process is as follows: You can either create a song in the program’s ‘ancestral’ mode, which conditions the model to produce audio based on a specified artist and genre. Or, you can use the ‘primed’ mode, which trains the model on a specific audio sample.
Ave Coders, the creator in the above video, uses the latter of the two to make a modified “Jingle Bells.” The end product — like the samples on OpenAI’s site — is a garbled and messy remix of the original song. Not to mention that besides the mangled output, it takes a very long time for the computer to process audio — another reason this technology is fun to think about, but not going to replace musical artists anytime soon. It took several hours for the program to process just 40 seconds of original audio, meaning trying to create an entire three minute song would take days.
Here’s a funny video from 2020 which shows JukeBox’s many creative takes on Eminem and JuiceWRLD’s “Monster.” This video really helps you understand what the program is doing: taking a short excerpt of the song and creating a modified extension to that snippet.
Overall, while the technology exists, the final product doesn’t sound great and it takes quite a while to produce.
Personhood and Scarcity Create Demand — The Artist’s Reprieve
Artificial intelligence can also be used to create songs that sound like already existing artists. In this video, a YouTuber uses ChatGPT to write lyrics for a song in the style of the rapper J. Cole and feeds the lyrics through Uberduck.ai, a model that synthesizes text to sound like a famous musician of the user’s choice. Besides the absurd subject matter (gluten-free cake), the song actually sounds quite a bit like J. Cole, almost like a leaked song that was never mixed or mastered.
A logical conclusion could be that we now have the technology to produce music in styles eerily similar to those of our favorite artists. Why won’t fans seize this opportunity to create music “on behalf” of their favorite artists?
Well, we’ve already seen something similar to this. In 2021, Over the Bridge, a mental health non-profit, released an album of AI-assisted songs meant to replicate the styles of Jimi Hendrix, Kurt Cobain, Jim Morrison and Amy Winehouse — all members of the “27 Club” (artists who infamously died at the age of 27).
According to a Rolling Stone article about the endeavor, Over the Bridge used Google’s Magenta Studio to produce new riffs and soundbites in the style of the 27 Club artists. From there, audio engineers cobbled together the pieces and cover singers were enlisted to sing the lyrics, which were also created with AI programs. Note that none of this would have been possible sans human involvement — AI processes created bits and pieces of the final product while humans wove them together.
The results are stunning. So why hasn’t this imaginative replication become more prevalent?
Take, for example, the ever-elusive Frank Ocean. Frank commands a fan base which rabidly awaits new music, and he hasn’t released an album in seven years. Why haven’t his fans undertaken a similar project?
The answer is twofold:
Primarily, AI processes are — at this juncture — incapable of assuming a distinctive voice or style. By voice, I mean a unique outlook or angle, synthesized in a distinct rhetorical manner — not the tonal patterns of one’s literal voice, which we now know can be replicated.
Fans clamor for more Frank Ocean music because every time he releases something, it feels monumental, even epochal for his fans. They flock in droves to their preferred streaming services to hear Frank, the person, rehash his life since his last release: Who he’s fallen in love with, cars he’s admired, movies he’s watched. While artificial intelligence processes can replicate something similar, their objects of reference will be, of course, nonexistent.
The same logic applies for watch collectors, or any other fine-good connoisseur. Though there are nearly perfect Rolex replicas on the market, most watch collectors would be hesitant to buy one. Just as the watch collector’s true infatuation is with Rolex, the brand, the music superfan’s true infatuation is with the artist’s personhood.
For now, no artificial intelligence model can replace that.
Secondly, scarcity creates demand in the music industry. Fans also clamor for more Frank Ocean music because so little of it exists, and minds are left to wander about what could be in store for the future and what could have been in the past.
If fans saturated the market with rip-off songs, even if they could be manufactured in such a way that evoked similar emotion and responses to the real Frank Ocean’s music, demand would plummet. Fans’ appetite for new music is only really insatiable when they’re rarely eating.
AI as a Jamming Partner?
A recently published paper by Notto J.W. Thelle, at the time a P.h.D fellow at the Norwegian Academy of Music, posits that there are benefits for humans collaborating with AI jamming partners during the creative process. Thelle set up a “Wizard of Oz” experiment — one in which the subject believes they are interacting with an artificial intelligence program, when in reality they are interacting with a human — in order to examine how humans would react to using an AI agent as a co-creator.
In the experiment, the researchers separated the participants (three musicians) from the “AI agent” (in reality, a human keyboardist) in two different rooms. The human musicians each played music for 15-minute periods, during which they provided real-time feedback to the “AI agent,” based on how they felt its musical output complimented their own music. The participants reported the following:
Two participants expressed that they were emboldened by the notion of a non-human system that withheld judgment and allowed them to play without having to worry about sounding good or being correct. They also stated that they were less critical of the system’s output than with human musicians. Apparently, this trade-off—accepting “less musicality” while being free of judgment— was a prospect they found exciting … One participant expressed that the lack of a social dimension gave her a heightened sense of security, and she was “able to check out a lot of things that I never would have done with people.”
AI as a jamming partner seems like an inherently collaborative use for the technology, a finding which contravenes the recent cataclysmic frenzy surrounding its emergence. AI’s advantage over human creators here is quite simply that artificial intelligence agents are not human, and thus do not cast judgment, apparently giving human creators a sense of artistic safety and freedom.
When I talked to Thelle, who is a musician himself, he told me that the social aspect of music creation is strong.
“Most people don’t want to make a fool out of themselves,” Thelle said. “So that actually inhibits a lot of creativity a lot of the time.”
Thelle said one of the participants in the experiment compared her experience to playing music with her best friend.
“That’s when she was able to let her guard down and not worry about being judged, (and know that) it's okay to do something stupid,” Thelle said. “So, I think not needing to worry about sounding good, or being clever, can really bring out something new in people.”
I did find some aspects of this premise concerning. Even though it seems like a net positive for artists to feel freed in taking creative risks, if this becomes the industry norm, we might face a period of creative stasis.
An integral part of producing music in a collaborative setting is constructive feedback. Especially when considering massively famous and successful artists, human feedback is essential to help these artists refine their output. Collaboration can help save artists from their own excesses or egos. In that way, it seems that removing the human counterpart as a part of the creative process has regressive potential, as the feedback structure becomes one-sided.
I asked Thelle if he worried about this kind of regression in collaboration as AI becomes more prominent in music. He said there is a danger there, but only because artists have failed to properly conceptualize AI programs.
“I don’t think it’s accurate to say that humans prefer playing with AI. It’s just fundamentally different to playing with people. The lack of a social dynamic brings out something else, something that doesn’t necessarily happen between people. These new discoveries may actually widen the musicians’ expressive scope, which they in turn can bring back to the domain of human-to-human interactions, emboldened, and enrich those experiences as well. These are not competing worlds, as many people might fear. A familiar example is the drum machine. The drum machine never replaced human drummers or made them less creative, but you see lots of examples of how human drummers have been influenced and inspired by their aesthetics. Others became drum machine converts. You get both. So instead of a standstill, what we might see is a transformation and a diversification. Machines will never replace people completely, but they can alter trajectories.”
Trends I’m keeping an eye on
There are a couple spaces in music that I’ll be watching for upcoming advancements in AI.
Soundtracking: AIVA is an AI composer, which works on a set of parameters decided by the creator. You can choose from a few preset styles (Rock, Cinematic, Pop, etc.) and specify the key and pacing of the song. The system produces an original score within seconds, which can then be downloaded as an MP3 or MIDI file. There are three subscription options on AIVA, depending on how you want to use its songs commercially.
Stem isolation: Stems are the isolated components of songs — vocals, guitars, drums, etc. — and it can be useful for artists to access them individually. The music label Cherry Red, for example, used Audioshake to send one of its artist’s instrumentals to Netflix for a trailer. Artists will frequently be asked to send stems of their songs if they are placed in a movie or TV show. Now, with the help of AI, musicians won’t have to track down an old audio engineer or otherwise dig up the old files themselves. LALA.AI is another program to watch in this space.
Talent acquisition: Talent acquisition is a massive part of the music industry. It’s how artists sign lucrative label contracts and usually how they enter the mainstream. Sodatone, which Warner Music acquired in 2018, uses AI to track up-and-coming artists, including playlist placement, media mentions and venue bookings. In 2020 — which could have been a dead year in the midst of the pandemic — Warner’s CEO announced that, using Sodatone, Warner had actually signed double the number of artists it had in 2019. As more and more music artists are releasing music in the bedroom pop revolution, machine learning algorithms are already having a massive impact on identifying superstars.
So, where are we?
Artificial intelligence is here to stay in the music industry. However, these programs will serve more of a collaborative purpose than anything else. AI programs won’t be replacing real artists anytime soon, as they lack true sentience and personhood. They won’t be making music completely on their own, either, as they require humans to orchestrate and engineer their output.
However, AI programs do present a number of collaborative and cost-efficient advantages to musicians. The ones who learn to harness them quickly will find themselves with a competitive advantage.
Best,
Eli for the Don’t Count Us Out Yet Team