The State of the Open Source Database Industry in 2020: Part Two

A four-part blog series by Matt Yonkovit, Percona Chief Experience Officer.

Read Matt’s first blog in this series on ‘Baseline Data and the Size of the Market’ .

Migrating from Proprietary Software to Open Source

There are two main sources of growth in the open source database space. First is new application development (more on that later). Second is migration and companies moving away from proprietary software.

You can’t go onto a cloud or database vendor website without seeing lots of information on why you should ditch your expensive, old, database technology, and upgrade to a sparkling, new, open source one.

But in reality, how much is this happening? The answer is a fair amount, though not as much as these vendors would like. The reasons for this are complicated, but I have summarized these, based on what I have observed.

What is the Motivation for Migration?

Many of the migrations we see at the moment involve small and medium-sized applications. These migrations are often opportunistic and form part of modernization efforts.

Larger, more complex, applications are not currently migrating in mass unless the systems were due to be completely replaced anyway. This has led to many companies running their proprietary databases alongside their open source databases.

In terms of larger migrations, why have some been held up or slowed down?

Mostly this seems to stem from internal buy-in, and vendor lock-in. The following scenario is common:

• A company executive wants to get away from a proprietary vendor (I’ve used Oracle here as an example) and makes the decision to start replacing systems to reduce costs and dependence on the vendor.
• Middle management goes to their Oracle-focused DBAs (many of whom have been doing this for 10-15 years) and tells them they will be moving to something that is not Oracle and that they should start investigating alternatives. This causes their team to start from a position of fear, as they are concerned their jobs are on the line if they move away from Oracle.
• The application development teams are told of the plan to migrate to a new database.
• Application management and their stakeholders (often business executives who need new applications, features, and new revenue streams) don’t see the value in migration if that move interrupts other revenue-facing work they are engaged in. Complex databases take up development resources, and that is also an issue.
• The company goes back and tries to reduce its licensing bill with Oracle. Instead of coming in with a cost-saving, Oracle says that removing a few servers won’t save much money, because they gave the company a volume discount. So any cost savings here evaporate.
• The migration effort loses steam and is shelved for several years before the cycle repeats, often under new management.

It is also worth noting that not all migrations are migrations. We have observed that people often confuse the term ‘migration’ when discussing modernization projects. Lots of companies are undertaking larger modernization projects, where their database is simply one component.

Changing an application’s backend database by migrating from Oracle to MongoDB is very different to replacing an entire legacy N-tier application with a brand new, or completely redesigned, cloud-native application. The latter option is more about replacing a whole technology stack and involves a top-to-bottom redo. This is where we see more of the changes to complex applications.

Who is Fueling Current Open Source and Cloud Space Growth?

One word: developers .

One of the trends we are seeing is that developers and application architects are more frequently becoming the ones who pick the database technologies that they use in their applications.

In the past, DBAs or infrastructure teams often worked hard to consolidate and reduce the complexity of the technologies they used. This turned many companies into what many would call a “single database shop” (remember when people would say we are an “Oracle Shop” or we are a “SQL Server Shop”?)

This type of consolidation has become a thing of the past. There are so many great, purpose-built, databases out there that you can’t help but want to explore them.

Ourrecent survey data backs up this growing use of multiple databases trend:

So, what is the reason behind this growth in the multi-database and multi-cloud space?

Growth in the multi-database space might be a business strategy, but, like growth in the multi-cloud space, this is also an outcome of enabling users/developers to have easier access to database technology.

Putting more power in the hands of developers and end-users is great as it speeds time to innovation, and can allow companies to achieve higher throughput and increase revenue. But an “inheritance” problem might also be created.

In the current database environment, developers are kings. But, they are kings with more options and choices than ever before.

We have seen the same pattern repeated at numerous large organizations. Different development teams choose different technologies for their applications. The DBAs, SREs, Sysadmins, etc. then inherit, or get stuck with supporting, those technology decisions long-term.

Easier access to different technologies can create an insanely complex matrix of technologies within a company.

We are starting to see infrastructure teams becoming “product development teams,” with the goal of building automation, tools, and the infrastructure to support a varied and rapidly-expanding technology stack. This is an important trend to understand as it can lead to a number of issues, most notably difficulty in consolidating into a single database technology. This can limit company and project growth long-term.

My next blog in this series on The State of the Open Source Database Industry in 2020 is available next week and will discuss the most popular open source databases currently on the market .