With the release of their new music-generating program, OpenAI’s Jukebox, the company has topped all its competition. The Jukebox models can capture melody, rhythm, and long-range composition. In addition, they can capture the timbres of a variety of instruments, the styles of different singers, and even include lyrics. The future of music-making may lie in the AI, and Jukebox promises to make the process easy, quick, and streamlined.
Jukebox incorporates lyrics for the first time. Users can add their own lyrics or those of the song they’re listening to, or let the model compose the lyrics itself. However, the Jukebox model has issues aligning the words to non-lyrical components of the song. Currently, it depends on a separate auto-encoder to do so. The researchers note that they are still learning the process of incorporating lyrics and other aspects of music.
While many people are skeptical of the jukebox project, the company has released a variety of demos and information. The software also uses a neural network to train its software. To test the software, the user inputs a brief demo of a song, such as 15 seconds. The AI then plays the track indefinitely, based on the input data. And the samples are not cherry-picked. This is why the software is able to resemble the original artists.
Its researchers have been developing AI tools to produce artificial music for a variety of uses. They have produced controversial deepfake videos, artificial music, and text generation, to name a few. In addition to music, OpenAI has developed a model for artificial song composition that can generate songs similar to those produced by popular singers. OpenAI’s AI model is trained on 1.2 million songs from the internet and can output raw audio for several minutes.
While the Jukebox model trains on the distribution of compressed audio tokens, the Transformer model enables researchers to steer it towards any style or genre they desire. The researchers also added a query-using layer to the music decoder, as well as a lyric encoder. These three layers combined result in a model with over two million parameters. This is an impressive feat. But how can these new models beat the current top music-generating software?
The Jukebox is a fascinating example of artificial intelligence in action. The system generates music that sounds like a mushy rendition of familiar songs. The OpenAI team chose music as a challenging domain, and they chose it because it contains recognizable chords, melodies, and words. Whether these artificial intelligence systems can actually generate new tunes under the names of deceased artists is still unclear.
Another impressive piece of AI music from the company is the OpenAI Jukebox. Using 23,000 transcriptions of traditional Irish music, the Jukebox is the first album generated by AI. The project was curated by a UK musician named Daren Banarse, and CDs were sent to music critics in the US and Europe. Artists were even credited on the CD sleeves as part of the O Conaill family. Folk music is an ideal sandbox for AI music experiments.
In an effort to improve the sound quality of music produced by generative AI, OpenAI developed a machine-learning framework called Jukebox. The model can generate rudimentary song samples, including vocals and lyrics. The software is publicly available on GitHub, and the developers have included a tool to explore the generated audio samples. Despite the limitations, the Jukebox project is an enjoyable project with great results. It is worth a look!