Deep learning and other AI algorithms are tailor-made for unstructured data and there is nothing less structured than digitized audio or video. Furthermore, the world is awash in it, with digital media being a significant cause of the explosive growth in data created, captured and replicated. Indeed, computers are both the cause of massive amounts of new music and the solution to the problem of managing it.

Since music became digitized in the 1980s and the emergence of digital mixing software in the 1990s, computers have been essential in music production. The technology has led to an explosion in artistic output and consumption epitomized by the rise of music streaming services serving up more than two billion streams in the U.S. alone each year. Furthermore, musicians and producers are cranking out titles at a furious pace, with Spotify saying it adds more than 40,000 tracks per day to its platform. Producing, organizing, curating and promoting this prodigious output is a Herculean task that several companies now automate by employing AI algorithms.

Music digitization fueled creative output, but facilitates computerized automation

Recommending and categorizing music based on the actual sound, not just metadata about a particular piece, has been an ongoing field of technology development for two decades. The pioneers were Shazam’s audio search algorithm that fingerprinted each audio file using audio samples and Pandora’s Music Genome project that used 400 characteristics to categorize each musical piece. In a 2006 interview, Pandora’s co-founder Tim Westergren said these,

Essentially cover all of the granular details of melody, harmony, rhythm, form, compositional qualities and lyrics. I think of it as the primary colors, the distinct elements that make up a song]. For example, there are over 30 attributes that describe the voice alone; how much vibrato, range, ornamentation, tone, performance.

Both techniques start with signal processing using FFTs (Fast Fourier Transforms) or similar algorithms that were initially developed for processing radar imagery to decompose music into spectral components. Measurements of signal strength at various frequencies over time are then used to generate metadata that categorizes particular passages, instruments, tones and tempos. Augmenting signal processing techniques with deep learning and other AI algorithms is the next phase in improving the accuracy of musical recommendations and predictive correlations used in music production.

CI/CD for music streaming

CI/CD (continuous integration/continuous delivery) is one of the most popular methods for DevOps teams to accelerate software development and release by automating the workflow from code submission to compiled application deployment. NEOS-AI provides a similar set of process automation tools for music curation and distribution to hundreds of streaming platforms.

The rise of streaming services, which accounts for about 80 percent of total music industry revenue, has enabled millions of back catalog titles and thousands of independent artists to be widely distributed and discovered by a broad audience. Unfortunately for indie musicians and small publishers, the overhead in making title or album streaming-ready is significant, costing thousands of dollars for even low-budget releases. NEOS-AI automates the process using machine learning (ML) for sorting, organizing, and structuring music libraries for distribution.

NEOS-AI starts by analyzing a music collection and its metadata for genre, mood and passages. From this dataset, it automates the production and curation of track lists, song titles and album art using five customizable components.

• AI-Pilot: Analyzes and sorts metadata by genre, mood, and moments to create album tracklists.
• AI-Art: Designs artwork according to album title, using a library of more than 50 million images and artist-curated templates.
• AI-Codes: Generates ISRC (an international standard for uniquely identifying music recordings) codes and UPCs using custom or default
• AI-Agenda: Schedules automated tasks up to several months in advance.
• AI-Release: Distributes albums via API to more than 300 digital streaming platforms worldwide.

The entire process takes seconds, speed that is particularly important to companies with large music catalogs seeking to release customized collections for events or special projects. For example, NEOS-AI can make albums based on genre, mood, artist, tempo or other musical characteristics and use behavioral analysis to select the tracks with the highest probability of commercial success. The AI-Art module can also create album art by matching the music with the best font, color scheme and photos from its massive library.

AI and music - a young and evolving technology

NEOS-AI, a division of EMUQ Tech, isn’t alone in applying AI to music creation and production. For example,

• Avia uses AI to create musical works and themes based on 11 preset styles. Its recurrent neural network is trained from a database of classical music that finds patterns in selected tracks used to classify its style. The algorithm refines itself by making temporal predictions based on different parts of a piece about what will come next. It then uses the difference between predicted and actual results to refine the rules for that style of music. It also has plagiarism detection to identify musical segments that are copies of music already in its database. Avia software is famous for composing the haunting theme to NVIDIA’s GTC conferences, I am AI, first used in 2017.
• Musiio has three products that automate metadata classification (Tag), album and playlist creation (Playlist) and catalog search (Search). Its classification and search engine can identify trends useful to music producers looking for titles to promote. For example, by analyzing 5200 songs from 104 Weekly Spotify Viral Charts in the U.S., it found a pronounced decrease in the positive mood of popular songs as the year wore on. Its software can also identify the most popular genres and correlations in popularity swings between different genres.

• Niland (acquired by Spotify in 2017) uses statistical signal processing and AI to classify tracks and identify correlations and musical similarity between different tracks and genres. Like NEOS-AI and Musiio, Niland’s software can automatically classify songs and recommend tracks based on an individual’s preferences or listening history.
• Synchtank is a music asset management system that handles copyrights, multiple formats and versions, metadata, promotion and sales and usage tracking. By consolidating several functions under one UI, Synchtank streamlines the sales and distribution of musical works and can create customized websites for music promotion. According to the company’s CEO, Synchtank use "“AI-like techniques to help run and manage B2B metadata based on audio waveforms, powering advanced search capabilities for our clients. Catalogs on our platform are automatically tagged with semantic and descriptive metadata by default, allowing the right track to be found in a database of thousands and even millions of tracks in seconds.”

My take

The application of AI to music production, organization and catalog management provides a convincing example of the technology’s power to find and classify patterns in unstructured data. By extracting structure from previously intractable data sources like digital music, video, speech and text, deep and machine learning algorithms can identify correlations and make predictions that would be impossible or unfeasible using manual methods. These insights both reduce the overhead and time to manage unstructured data, for example, in streaming music production, and improve the quality of decisions.

Ultimately, AI can change how executives make decisions, engineers develop products and artists create works. Although software like Avia has initially targeted less critically demanding needs like the background tracks for movies, games and live events, AI is already being used by composers and lyricists to explore and refine ideas. Indeed, some estimate that a quarter of all hit songs will be written in whole or part using AI. Likewise, most business executives will rely on AI-powered analysis and predictions when making strategic decisions. Much like robots on the factory floor, AI is a powerful tool for improving the quality and timeliness of intellectual and creative tasks.