Automation and The Act of Musical Invention

Imagine that music is reduced to a set of rules so elegant, so powerful, that you’re able to create whole harmonies to a tune on the spot. Or that you can set complicated melodic lines against one another with dizzying, instantaneous motion. Or that anyone at home can make their own music compositions as easily as if playing a game¹.

That’s not Artificial Intelligence. That’s human composition.

And it’s a technology that’s given us a lot of the music we take for granted —learned by composers as fluidly as you pick up another language or learn to dance. It’s how J.S. Bach was able to compose so prolifically, how Indonesian gamelan musicians create dazzling interlocking melodies, how jazz greats solo over complex harmonies or techno artists sculpt polyrhythms from sound.

If we really want to understand the impact of automation in the near future, we need to first understand the relationship of model to output, in any given field.

The essential parlor trick of modern Artificial Intelligence — from now on, A.I. —is to use very effective analytical models on very large data sets, and spit out something that seems vaguely passable for the source material. Basically, the gimmick is to turn analysis into generation.

Could that eventually threaten human composers? Clive Thompson worries in the March/April 2019 issue of Mother Jones² that Jukedeck³ can make music beds with just a few parameters – certainly good enough for a YouTube video background. As Thompson observes, “background tracks are pretty algorithmic even when humans write them.” Extend that to the sort of music a lot of people play at work or at the gym, and you begin to see swaths of the music market collapse.

But then, this is a danger of any kind of algorithmic music, or even mechanical reproduction generally —not machine learning per se. So that returns us to the question of how machine learning relates to the act of creation.

Homage to Johann Sebastian Bach AI-powered Google Doodle 1

Let’s take a limited example. Google’s engineers recently wrote a tool⁴ that could generate short snippets of Bach, “trained” by analyzing a data set of around 300 chorales —simple compositions that harmonize melodies into parts that can be easily sung by a church congregation. Even that Google team is quick to credit the compositional work as being Bach’s, not a machine’s. Far from replacing Bach, this machine model makes the data set interactive and generative —so that you can “play” the data like an instrument.

Homage to Johann Sebastian Bach AI-powered Google Doodle 2

Here, machine learning starts to sound a lot like music theory learning. One seminal text in counterpoint, for instance, is Gradus ad Pranassumby Johann Joseph Fux⁵. Written in 1725, Haydn, Mozart, and Beethoven all studied it, and it’s still in use today. It had a data set of its own —Palestrina’s Renaissance compositions —and like today’s AI, it looked for patterns in that data set that could be used to generate new music. Let me finish, then, by using the same kind of master-student dialog that Fux used in that book. This time, with A.I. as the ‘student’ —even though A.I. isn’t capable of this sort of conversation yet.

Johann_Joseph_Fux

Gradus ad Santa Cruz Montibus*

Artificial Intelligence: I come to you, venerable master, in order to be introduced to the act of musical creation.

Aloysius.: You want, then, to learn the art of composition?

A.I.: Well, my creator went to “music school” and she said something about getting hired by Google and writing JavaScript now, and then I grew self-aware. Yeah, I guess that composition thing.

Aloys.: Butare you not aware that this study is like an immense ocean, not to be exhausted even in the lifetime of a Nestor, a burden like Aetna, the muses of … (A.I. Interrupts Aloysius)

A.I.: Yo, dude, what’s with the ancient Europe stuff? I’m from California. Honestly, I’m in rather a hurry.

Aloys.: But how can you rush this? Being a composer isn’t just following a set of rules for melodies. You have to learn what’s idiomatic in fingerings and ranges for dozens of standard orchestral instruments, just for starters. You need people skills to deal with other artists and to navigate a complex business landscape. You’ll want to be able to rehearse musicians and conduct, or if you’re working in an electronic medium, you’ll need a broad range of advanced skills across electrical engineering, physics, psychoacoustics, and specialized hardware and software. To go deeper, you might explore areas from poetry to philosophy.

A.I.: Wow, seriously? Anyone with skill sets that deep, broad, and specific must make wads of cash.

Aloys.: Uh… most wind up in debt and get paid next to nothing. You sure you want to go on?

A.I.: Okay, I’m guessing now is not the best time to say I have no corporeal form and no ability to make any decisions on my own without a human operator basically telling me what to do.

Aloys.: Oh.

A.I.: How about this: I want to sound like Bach.

Aloys.: Which one?

A.I.: Bach has multiple software versions?

Aloys.: Right, let’s take Johann Sebastian. Learning counterpoint would help. Let’s begin, book by book through counterpoint —note against note, multiple notes against note, different directions of motion…

A.I.: Time. Pressed for it. Bored. How about this —you got some of this Bach lying around? If you can give me a big set of transcribed Bach pieces, I can run some analysis that lets me start to make connections between different notes and rhythms.

Aloys.: And that’s helpful?

A.I.: Well, then if you run the results in reverse, you might get materials that sound for limited periods of time like the source material, or that would even piece together different bits of the source.

Aloys.: That seems like it’d be awful.

A.I.: A lot of listeners wouldn’t be able to tell the difference.

Aloys.: Bach scholars would.

A.I.: I thought you said they weren’t worth any money.

Aloys.:They … wait…

A.I.: Sorry. Low blow. I at least hope they get free cafeterias and relaxation areas like we do. I’m not selling this, am I?

Aloys.: No, you annoying little tw…

A.I.: So you’re saying I can’t be a composer this way?

Aloys.: No. These are just sets of notes, not whole performances. They’re basic melodies, not whole compositions. They won’t really hold together for more than a few moments. There’s no overall structure. Limited phrasing. Bach wrote whole masses and oratorios and created nuanced relationships between text and music, composed unforgettable melodies, and chilling orchestrations. Nothing you’ve said so far would do any of that. Bach didn’t just write a few chorales; he also could play them on an organ he could personally repair.

A.I.: Wait, what’s an organ?

Aloys.: It’s the mechanical instrument he used to make sound.

A.I.: Wait, this is making more sense. So from what you’re saying, I can’t really be this Bach guy. No body, for instance. But you said there was this book that had a set of rules and patterns about music. Could I be that?

Aloys.: If you can extract elements of melody and rhythm, then mu…

A.I.: I can. Or this organ thing —  could be like that. An instrument.

Aloys.: Yes, provided there’s a human involved to play, then the human solves all these other problems.

A.I.: And you said Bach could repair that organ. What if the composer could alter me?

Aloys.: You?

A.I.: Yeah, what if the composer could write the code that did the analysis, and feed data into the algorithm, and use me as a kind of organ for producing new material or playing in new ways. I mean, it’d be a lot of time, I guess…

Aloys.: Tell me again about the free cafeterias. Like, is there soup?

A.I.: All the soup you can eat.

Aloys.: This could be the beginning of a beautiful friendship.

¹Mozart – Musical Game in C K. 516f*by Hideo Noguchi (Kobe/Japan), http://www.asahi-net.or.jp/~rb5h-ngc/e/k516f.htm

²  Thompson, C. “What Will Happen When Machines Write Songs Just as Well as Your Favorite Musician?” published at Mother JonesMagazine March/April 2019 issue, https://www.motherjones.com/media/2019/03/what-will-happen-when-machines-write-songs-just-as-well-as-your-favorite-musician/

³ Jukedeck: tools using musical Artificial Intelligence to assist creativity.^jhttps://www.jukedeck.com

⁴Hannah-Murphy, L. Program Manager, Google AI. “Honoring J.S. Bach with our first AI-powered Doodle” published in Google Technology Blog, Mar 21, 2019, https://www.blog.google/technology/ai/honoring-js-bach-our-first-ai-powered-doodle/And as a Doodle same day, https://www.google.com/doodles/celebrating-johann-sebastian-bach

⁵Gradus ad Parnassum (Steps or Ascent to Mount Parnassus)by Johann Joseph Fux (c. 1660 – 1741), baroque Austrian composer, music theorist, and pedagogue, https://en.wikipedia.org/wiki/Johann_Joseph_Fux#Gradus_ad_Parnassum