They Wrote Live AI into the Score

Bright editorial illustration of an orchestra in a gilded hall, with light threading from a clarinet solo across the strings toward a person listening at a console
Hero image: original illustration created for this post.

On Thursday night the Boston Symphony Orchestra will play a new piece whose published scoring includes the words “live AI.” Tod Machover’s already and not yet lasts about twenty minutes. Kent Nagano conducts. It is a BSO commission, and it sits on a subscription program between Mendelssohn’s Hebrides overture and, after the American premiere of a work by Alex Nante, Mendelssohn’s Reformation Symphony. The orchestra gave its first concert on October 22, 1881, at the old Boston Music Hall. This premiere is 145 years later, to the day, in Symphony Hall. Boosey & Hawkes lists the forces in the usual shorthand, and then adds a desk the shorthand did not used to have: three each of flutes, oboes, clarinets, and bassoons; four horns, three trumpets, three trombones, tuba; timpani and three percussion; a MIDI keyboard; live AI; and a full string body of sixteen first violins, fourteen seconds, twelve violas, ten cellos, and eight basses.

Nobody has heard it yet. The BSO’s own page still says the program note is coming soon. What can be read is the composer’s note, dated July 4, and the instrumentation line. That is enough to say what kind of AI event this is. It is not a model asked to finish a dead composer’s symphony, and it is not a track uploaded to a streaming service. A person wrote the piece. A named system, built for this piece, is supposed to listen while the orchestra plays. The interesting question is whether the listening is audible.

What the score claims the system will do

Machover composed the work between the summer of 2025 and the summer of 2026. The title, in his note, is about the state of the country: prepared to be shaped, with much to appreciate, unfinished, imperfect, and in need of being pushed. The movements are named like instructions. SMASH UP is a clash. WAKE UP is an awakening. STAND UP is what he calls a growing picture of a possible future for democracy, and a musical call to action. Electronics run through the whole piece. The live AI, made for this composition at the MIT Media Lab, becomes prominent only in the second and third movements.

In WAKE UP the system spins enhancements and variations on solo lines and sends them through the orchestra. Machover’s purpose for that, as he wrote it, is to emphasize the individuality of each player and the difference between them. In STAND UP the system pulls the many layers of the orchestra into what he hopes will be a new kind of sound: detail and wholeness at once, polyphony and a single texture, with the sound leaving the stage and going out into the hall to include the audience. The verb that matters is “I hope.” A composer who has already heard the rehearsal in his head is still unwilling to claim the blend as a fact. The fact is the assignment. The blend is what Thursday is for.

Symphony Hall is a strange place to ask a computer to embrace an audience. The room opened in 1900, and it has been doing that embrace, by architecture, for longer than any model has existed. Wallace Sabine helped tune it. A solo line in that room already reaches the second balcony without assistance. So the system’s job cannot be amplification in the ordinary sense, or the piece would be a very expensive way to do what the plaster already does. The job, if the note is taken seriously, is selective. It has to take one player’s line, change it, and make the change recognizable as that player’s, and then, a movement later, hold every layer without sanding off the player the previous movement had just isolated. Those two tasks pull in opposite directions. One is about difference. The other is about a whole. A live system that can do only the second will sound like a wash. A system that can do only the first will sound like an effect on a solo. The piece is built so that a listener can tell which one happened.

The names on the new desk

The Media Lab, which is marking its fortieth year, says the live AI was designed by Manaswi Mishra, a PhD student in Machover’s Opera of the Future group. The BSO’s profile of him adds that he is also a student researcher at Google DeepMind, and that his work has been instruments and operas rather than a general chatbot: VALIS, FLOW Symphony, an AI-sitar collaboration. Ana Schon, another Media Lab PhD student, mixes the audio. Machover is both the composer and the faculty director of the lab. The premiere is the center of a BSO week called Technology & Our Humanity, run in parallel with the lab’s anniversary.

That list is the design. The model in the hall is not an unnamed internal system that mailed a pile of drafts and left. It has an author, a builder, and a mixer, and it is printed on a score a publisher will rent. When the sound fails, there is a person to ask. When it succeeds, there is a person to credit. That is a low bar, and a great deal of machine-made music still misses it.

The week around the concerts makes the same point in talks. On Saturday at 6:30, before the evening performance, George Lewis sits with Machover. Lewis has spent decades on the question this score is walking into: his Voyager software has been improvising with live musicians since the 1980s, and the point of that work was never to replace the player. On Sunday afternoon, at MIT rather than in the hall, a panel of Media Lab music alumni includes Teresa Marrin Nakra, Alex Rigopulos, Anna Huang, Ben Bloomberg, and Nicole L’Huillier, with Machover moderating. The lab’s own account of that history runs from the Hypercello and the Brain Opera to Guitar Hero. At 3:30 Holly Herndon talks about AI as a musical practice, with Mishra asking the questions. Herndon, on the lab’s description, will talk about the choices that come before an output: what you train on, what you put in the dataset, how you judge what comes back. Those choices are not in the Boosey scoring line. They are the part of the instrument a listener cannot see, and they are why a talk belongs next to a premiere. A live system is only as particular as the material and the rules someone gave it.

A full string section is the answer to the fear

In August, when the orchestra’s promo first described the piece, Norman Lebrecht’s Slipped Disc ran the skeptical reading in one breath. AI looks, to many musicians, like a threat to the job. A major American orchestra putting it on a subscription concert could be a tipping point. The fear is not foolish. A generated orchestra part is cheaper than a section, and plenty of people are already selling the cheaper thing.

Look at the scoring anyway. The strings are not a sketch of a string section. Sixteen, fourteen, twelve, ten, eight is a symphony orchestra at full height. The winds are in threes. Nobody’s chair was deleted to make room for the model. The new line is additional. For these three performances, the institution hired the players and also hired a system that is supposed to make those players more separable from one another, not less. One commission does not settle the labor question in the rest of the industry. It does fix what this particular use of the tool is for. If the second movement works, the AI is an argument for keeping the oboe chair, because the system has nothing to vary if the oboist is not there, playing, tonight, differently from the oboist who would have sat there on another night.

That is also why a preview cannot pretend to be a review. A fixed tape under an orchestra can be judged in the practice room. A system whose input is the player in front of it cannot. Machover has been building instruments that respond to performers for decades. Hyperinstruments were an earlier version of the same bet: the machine should react to a musician, not sit beside one on a backing track. The words “live AI” on a 2026 score are the current name for that bet, and the current name is doing a lot of work. Anything that listens and answers during a concert can be called live. The test, in the second movement, is narrower than the label. Can you hear a solo come back altered and still know whose solo it was? If the variation could have been written in July and pasted under any principal player, the system was a tape with better marketing. If the variation depends on the attack, the breath, and the intonation of the person who just played, then the desk earned its line in the score.

The other music news this week was paperwork

On October 12, the distributor Symphonic updated its rules for generative music. Artists who use a model are told to disclose it, to name the tool when asked, and to use services that have the rights. Training opt-in stays optional and is never automatic. Disclosure does not by itself get a track onto every platform, and it does not by itself keep a track off them. That is the industry’s ordinary AI problem: a file arrives, and someone has to decide whether a person made enough of it to own it, license it, and sell it.

Symphony Hall is running the other experiment. The composer is on the poster. The system designer is on the orchestra’s website. The mixer has a name. The score says where the machine enters, which is not the first movement. The audience is invited into a room where the input is still a human being holding an instrument. Herndon’s talk on Sunday is about the part the room will not show: the dataset and the judgment behind the instrument. Taken together, the week is a description of how to use the tool without pretending the tool wrote the concert. SMASH UP, WAKE UP, STAND UP may or may not amount to the civic vision Machover attached to them on the Fourth of July. I would not ask a premiere to carry that. I would ask it to do the musical thing the note actually specifies. Keep the players different. Then make a whole that does not erase them. If that happens in the hall, the live AI will have been an instrument. If it does not, the orchestra will still have been there, at full strength, which is more than most machine-made music can say.

References