OpenAI develops AI music tool with Juilliard data labeling

OpenAI is developing an artificial intelligence tool to generate music from text and audio prompts, people familiar with the project said, in a move that could let users add soundtracks to videos, create instrumental accompaniment for vocals, or produce full compositions. The effort, first reported this month, is taking place in collaboration with students at the Juilliard School, who are annotating musical scores to create training data, according to those people.

The tool is designed to accept text and audio cues and produce musical output that can be used for short-form video soundtracks, advertising jingles, guitar accompaniment and longer works, the sources said. It is not clear when OpenAI plans to release the technology or whether it will be a standalone product or integrated into existing services such as ChatGPT or its video-generation application. OpenAI did not immediately respond to requests for comment.

The project builds on OpenAI’s earlier music experiments, including MuseNet and Jukebox, as well as its more recent work on speech-to-text and text-to-speech models. Other technology companies and startups have already introduced generative music systems, and the...