Please consider subscribing to LWN Subscriptions are the lifeblood of LWN.net. If you appreciate this content and would like to see more of it, your subscription will help to ensure that LWN continues to thrive. Please visit this page to join up and keep LWN on the net.

The world of music and audio production is largely dominated by proprietary software vendors. Among them, Steinberg stands out as a company that created some of the most-used software, including the Cubase and Nuendo digital audio workstations. Steinberg is also known as the creator of the VST plugin API that, largely due to its licensing policy, has irritated developers enough to inspire multiple attempts at creating an open-source alternative. Even now, when the VST3 SDK is available under the GPLv3 license, the way the company exercises its control over the SDK keeps pushing developers away toward other open-source solutions.

This is an introduction to open-source plugin APIs for musicians and sound engineers alike. It focuses on the options in the larger ecosystem and how their shortcomings led to the creation of new alternatives with liberal licensing.

What audio and MIDI plugins do

Most audio editors, MIDI sequencers, and digital audio workstations (DAWs) are extensible. Sometimes that means writing a script that accesses the public API of a program (e.g. Lua in Ardour, ReaScript in Reaper). But the most common way is to write an extension, or plugin, using an API that is usually not specific to one single program. The audio editor or the DAW then would act as a "host" for that plugin, running it either in its own process or as a separate process (an increasingly popular option).

Two of the most common use cases for plugins in DAWs are processing audio (equalizers, reverbs, etc.), which means there's audio input and audio output, and turning notes into music, which means there's MIDI input and audio output. Most plugin APIs allow both audio and MIDI input and output. Additionally, some plugins exist for the purpose of controlling external gear, such as an audio interface with on-board digital signal processing like a built-in reverberation engine. These plugins are typically made by hardware vendors themselves, though there are some rare exceptions.

Unlike plugins in many image editors, audio and MIDI plugins are usually designed to work non-destructively and run in realtime mode. That means that you don't need to render a processed audio track to listen to it. Instead, you can hook up a MIDI keyboard and play a virtual instrument without ever recording a note, and you can change the value of plugin parameters over time.

A quick overview of current plugin APIs

These plugin APIs are currently in wide use:

• Virtual Studio Technology, or VST2/VST3, by Steinberg. It is supported on Windows, macOS, and Linux, and has broad host/DAW support.
• Real Time AudioSuite, or RTAS, by Digidesign/Avid (Windows, macOS, only supported in Avid Pro Tools).
• Avid Audio eXtension, or AAX, by Avid (Windows, macOS, only supported in Avid Pro Tools).
• Audio Unit, or AU, by Apple (macOS, iOS/iPadOS).
• LV2, which is an open-source project (Linux, Windows, macOS).

Industry-wide, plugins are mostly available as VST2/VST3 for both Windows and macOS and as AU for macOS. In some cases, plugins are also available as AAX and RTAS to target Avid Pro Tools users, because Avid infamously refuses to support any APIs other than its own.

On Linux specifically, LADSPA was the first widely used API for audio plugins; DSSI (Disposable Soft Synth Interface) was developed on top of it by the same team to make software synthesizers and samplers available as plugins too. There are several hundred LADSPA plugins available, but fewer than 50 DSSI plugins. Most audio editors and DAWs on Linux support LADSPA; few support DSSI. Both LADSPA and DSSI have now been largely replaced by LV2, which allows both audio and MIDI processing. There are well over a thousand LV2 plugins out there today.

There is also a growing number of plugins — both proprietary and free — written for the VST API and built natively on Linux, largely thanks to the VST3 SDK being available under the terms of both GPLv3 and a proprietary license. In fact, some spectacular free plugins, such as the Dexed and OB-Xd synthesizers, are only available as VST3 on Linux.

One final API worth mentioning here is the Generalized Music Plugin Interface, or GMPI. The original specification was designed between 2003 and 2005 by a diverse group of developers of both proprietary (SONAR, SynthEdit, FL Studio, etc.) and free (Ardour, etc.) software. Despite their varying backgrounds, these developers all shared the same sentiment: everyone felt fed up with Steinberg's dominance in the industry, which we will talk about in more detail below.

The GMPI specification was turned into an unofficial SDK by Tim Hockin (Google, but acting on his own behalf) and Jeff McClintock (SynthEdit), with Koen Tanghe (engineer at Ghent University at the time) working on his own version. The code by Hockin and McClintock did not receive much peer review, despite many requests. Eventually, the working group dissipated and McClintock used a cut-down version of the SDK for the third iteration of his SynthEdit plugin API. The fourth iteration will have a refactored version of the GMPI SDK.

I've heard a few explanations for what happened. The story from McClintock, shared in a private conversation, is the closest match to the discussions in the public mailing list. His impression is that many stakeholders only wanted a kind of VST2 with liberal licensing and weren't interested in major changes and improvements, so there was no incentive for SDK review. Another view, expressed by Paul Davis (Ardour), is that the MIDI Manufacturers Association, which announced the initiative in the first place, wanted the implementation phase to be closed. The Association reportedly also could not guarantee licensing that everyone would be happy with. So by the time an actual official SDK would have become available, many stakeholders would likely have lost their interest, and stakeholders likely weren't interested in an unofficial SDK.

I'm mentioning GMPI for three reasons. First of all, it was the first time in history that developers of proprietary and free software teamed up to develop an open-source audio/MIDI plugin API. Secondly, while largely unsuccessful, GMPI affected the design of both VST3 and LV2 (here is more info on the latter). And finally, McClintock resumed working on the full GMPI SDK last December. The SDK is available under the terms of the ISC license.

Where existing plugin APIs fall short

First off, if you want to develop an audio/MIDI plugin for use on Linux, your options are limited as compared to Windows and macOS.

AU (both v2 and v3) is not available anywhere but macOS and iOS/iPadOS. AAX and RTAS are cross-platform, but only work in Avid Pro Tools, which is unavailable on Linux. So far, SynthEdit appears to be pretty much the only real software that makes some use of GMPI. That leaves us VST2, VST3, LADSPA, DSSI, and LV2. These APIs are all supported by host applications on Linux, free and proprietary alike, although only two hosts, Qtractor and Zrythm, support all five.

Right off the bat, VST2 is at the end of its life. If you have a license for the VST2 SDK for an existing plugin, you can make a new version of that plugin as long as you want to. But Steinberg won't license the SDK for new plugins, and the need to license the use of this SDK was a major reason why VST2 never took off on Linux anyway. The limited number of native VST2 plugins that did appear for Linux were only possible thanks to a clean-room reverse-engineered VeSTige header file made by some LMMS developers.

LADSPA and DSSI are simple APIs, and the majority of audio hosts on Linux support LADSPA. However, these APIs have a number of technical limitations, they are not well maintained, and pretty much everyone stopped making new LADSPA and DSSI plugins years ago. Thus, we are down to VST3 and LV2.

VST3 has a specification that matches contemporary expectations and allows the creation of capable plugins. It's also extensible, though VST3 extensions are scattered across third-party vendors. However, over the years, Steinberg has managed to irritate developers of both proprietary and FOSS plugins in a major way. The way the company is sunsetting VST2 has upset plugin developers because it's impossible to use the old technology to create new work for hosts that still support VST2. Beyond that, Steinberg's own hosts and plugins will cease supporting VST2 by January 2024, so any Cubase and Nuendo projects that users have will fail to load entirely. Steinberg took some steps to make a VST2-to-VST3 transition possible but, reportedly, neither designed it well nor made developers aware of this feature in a timely manner, resulting in mistakes that cannot be easily undone now.

And then there are the inevitable technical controversies. The VST3 SDK is huge and has a number of hard-to-swallow idiosyncrasies like the enforced use of UTF-16. In particular, VST3 has an entirely new way of dealing with incoming MIDI data through a convenient abstraction layer that makes a lot of sense but which demands rearchitecting existing plugins, with no fallback to the old way. This complicates the task of maintaining both VST2 and VST3 versions of a plugin. To get a sense of how much work it takes to develop and maintain a fully functional VST3 version of a fairly complex plugin, you can read this comment on KVR by Urs Heckmann.

In summary, the industry is tired of living at the mercy of Steinberg. GMPI was the first major attempt to change the balance of forces, but, as described above, it wasn't successful.

LV2 was built in the mid-2000s using many ideas from the GMPI specification. It's modern, extensible, and allows all sorts of wonderful things. However, although well over 1,000 LV2 plugins are now available (here is a fairly recent, yet not exhaustive, list), there is a general sentiment that LV2 is difficult to get started with. Since its inception, I have observed a number of irritations that developers have with this API. Some of the issues are purely technical in nature. For example, plugin metadata is stored separately in text files using the Turtle syntax. For complex plugins, those files tend to become large. To give you an idea, the compiled metadata file for Mix Maxtrix has nearly 32K lines, although that's an extreme example. Other developers cite too much flexibility being the major issue they have with the API.

Then there are organizational issues like the lack of governance body and "too many cooks in the kitchen", which results in plugin developers feeling like they aren't being heard. The feeling that the LV2 maintainers are too defensive about their design decisions and don't like revisiting them does not help either. On an even larger scale, end users want an ecosystem, and LV2 has never established itself as such. That is, however, a very broad claim that, alone, is worth a separate conversation starting with how one defines an ecosystem for a plugin API.

There are more reasons for this API's slow adoption. One is that LV2 is not very popular with host developers outside of the free-software realm. The lack of LV2 support in hosts substantially limits the exposure of available plugins to end-users. Reaper is pretty much the only major DAW (on a global scale) that supports it, but then again Reaper supports almost every plugin API out there.

The other problem is that LV2 is currently unsupported by the major plugin-development frameworks; these allow writing code once and then building it for multiple plugin APIs. There's a ten-year-old fork of JUCE, one of those frameworks, by falkTX that adds native LV2 support; the upstream project is reimplementing this just now, with no planned completion date announced. Also, a third-party effort to add LV2 support to the iPlug2 framework was never merged into the mainstream project.

All of these factors affect the adoption of LV2. Thus we see, for example, OB-Xd developer closing the request for an LV2 version because JUCE6 has VST3 support and doesn't have LV2 support. It's difficult enough to get plugin developers to generate a native VST3 plugin even when they use JUCE, it's almost impossible to get them to release an LV2 version.

I have definitely seen most of the points mentioned here refuted by developers heavily invested with LV2. However even a naive observation suggests that LV2 is neither popular nor known outside of the FOSS ecosystem.

In conclusion

There are two open-source audio/MIDI plugin APIs in active use today: VST3 and LV2. Both APIs provide the means to create sophisticated plugins but, from the standpoint of a number of developers, both are somewhat problematic for technical reasons, as well as for how these APIs are managed by their respective maintainers.

For VST3, developers have to deal with Steinberg's aggressive licensing policy. This essentially means never building new VST2 plugins. For developers of virtual instruments, it means either dropping support for existing VST2 plugins entirely or accepting the burden of maintaining two architecturally different code bases. On top of that, various other design decision in the API have met resistance.

In case of LV2, developers have a steep learning curve and little choice in case they want to develop a plugin for multiple APIs from one code base. They would also be limited to just open-source host applications plus one proprietary digital audio workstation (Reaper), which significantly limits their exposure to users unless they support multiple APIs.

This situation has persisted for years now. In 2015, it resulted in the birth of another open-source API, CLAP. This project was resurrected late last year and is now finding its way to both free and proprietary software. We will talk about it in more detail in a followup article.

[Many thanks to Paul Davis, Robin Gareus, Filipe "falktx" Coelho, William Light, Alexandre Bique, Jeff McClintock for consulting on various technical aspects of audio/MIDI plugin APIs.]

---

(Log in to post comments)