The race to replace Redis

Welcome to LWN.net The following subscription-only content has been made available to you by an LWN subscriber. Thousands of subscribers depend on LWN for the best news from the Linux and free software communities. If you enjoy this article, please consider accepting the trial offer on the right. Thank you for visiting LWN.net!

Free trial subscription Try LWN for free for 1 month: no payment or credit card required. Activate your trial subscription now and see why thousands of readers subscribe to LWN.net.

On March 21, Redis Ltd. announced that the Redis "in-memory data store" project would now be released under non-free, source-available licenses, starting with Redis 7.4. The news is unwelcome, but not entirely unexpected. What is unusual with this situation is the number of Redis alternatives to choose from; there are at least four options to choose as a replacement for those who wish to stay with free software, including a pre-existing fork called KeyDB and the Linux Foundation's newly-announced Valkey project. The question now is which one(s) Linux distributions, users, and providers will choose to take its place.

A short history of Redis

Redis has a complicated backstory. Salvatore Sanfilippo (also known as "antirez") started the project to use as "a different kind of database" with a realtime log-analyzer application called LLOOGG, because MySQL was not meeting his needs. Instead of creating a relational database, he designed the project as a simple dictionary database that stored a key-value pair in memory—its name is a contraction of "remote dictionary server". It has, of course, matured and accrued many more features over the years. Redis quickly became popular as part of the NoSQL movement, and he was hired by VMware to work on Redis development in 2010. He moved to VMware's spin-off, Pivotal, in 2013 and continued to work on the project.

Around that time, Redis was growing in popularity, with high-profile use by Twitter and Pinterest, among others, and started to appear in Linux distributions. It was packaged for Ubuntu in the 12.04 (April 2012) release, Fedora 18 (January 2013), and others. Support for Redis was added to Amazon Web Service's (AWS) ElastiCache service in September 2013, which took advantage of, and helped bolster, Redis's popularity.

In early 2013, a startup called Garantia Data started offering Redis services and positioning itself as a better alternative to "open source Redis". Garantia took a first round of funding in November 2013 and floated changing its corporate name to RedisDB. After some pushback from Sanfilippo, the company renamed itself Redis Labs, instead, in early 2014. Sanfilippo joined Redis Labs as the lead for open-source development in 2015. He remained with Redis Labs until stepping down in 2020.

In 2018, Redis Labs adopted a new license for its add-on modules that provide features on top of the core database. The company chose to use a modified version of the Apache License, Version 2.0, with an addition called the Commons Clause. This clause restricted selling the software or charging for services. The rationale given for the switch was that cloud providers were "taking advantage of the open source community" by selling services based on open-source code they didn't develop. At the time, the company pledged that Redis "is BSD and will always remain BSD".

It was not the only company to start experimenting with use-restrictive licenses. Venture-backed database companies, in particular, were starting to run toward new licenses to try to ensure they could exclusively sell services using the software. MariaDB had created the Business Source License (BSL) for a product called MaxScale in 2016, and MongoDB launched the Server Side Public License (SSPL) in late 2018. Eventually, Redis Labs settled on a dual-license scheme of SSPL and its own Redis Source Available License (RSAL) for its modules.

The company dropped "Labs" from its name in mid-2021. In announcing the name change, Redis again committed to open source, and said that the company renaming "will not affect the licensing of open source Redis, which has always been and will continue to be BSD licensed". The company also put in place a governance model that would place major decisions about Redis's "architecture, design, or philosophy" with a community "core team". One would expect that team's mandate to include the license for Redis itself. The governance page is no longer on Redis's web site, but is available on the Internet Archive's Wayback Machine. It listed a core team of five members, three from Redis (Yossi Gotlieb, Oran Agra, and Itamar Haber) as well as Zhao Zhao from Alibaba and Madelyn Olson from AWS.

On March 20, Redis announced that "all future versions of Redis will be released with source-available licenses", specifically the SSPL and RSAL. Rowan Trollope, Redis CEO, wrote that maintaining the BSD license was now "at odds with our ability to drive Redis successfully into the future". Future versions, in this case, means Redis 7.4 and later. The announcement's FAQ says that, following the company's security policy, security patches will be backported to previous versions under the original three-clause BSD license.

Cloud versus open source

Proponents of use-restrictive licenses like the SSPL and Redis's RSAL have tried to position this solely as a battle between giant cloud providers like AWS and open source, where use restrictions are the only logical alternative and cloud providers are the only losers. In 2019, Redis Labs then-CEO Ofer Bengal was quoted as saying that there were "many different views" after Redis adopted its source-available licenses for Redis modules, but that it was necessary to compete with cloud providers:

Some people condemned that [license change]. But after the initial noise calmed down — and especially after some other companies came out with a similar concept — the community now understands that the original concept of open source has to be fixed because it isn't suitable anymore to the modern era where cloud companies use their monopoly power to adopt any successful open source project without contributing anything to it.

In the March 20 announcement, Trollope wrote that "cloud service providers will be able to deliver Redis 7.4 only after agreeing to licensing terms with Redis, the maintainers of the Redis code" but, that "nothing changes for the Redis developer community who will continue to enjoy permissive licensing under the dual license".

The choice of the phrase "permissive licensing" is misleading. Redis is able to adopt a non-free license scheme for 7.4 and later versions because external developers had granted their contributions under the permissive BSD license. The terms of the SSPL and RSAL are incompatible with common usage of the term "permissive" in the open source community.

It is also hard to reconcile the claims that cloud providers do not contribute with the actual commits to the Redis repository. A quick examination of the commits since the 7.0.0 release using gitdm shows 967 commits over that time period:

Top changeset contributions by employer
(Unknown)	331	34.2%
Tencent	240	24.8%
Redis	189	19.5%
Alibaba	65	6.7%
Huawei	50	5.2%
Amazon.com	50	5.2%
Bytedance	19	2.0%
NetEase	13	1.3%

BinBin Wang, of Tencent, is responsible for nearly 25% of the commits to the project. Some of the contributors without a readily identifiable employer surely are Redis employees, but it's clear that the company has not been working alone. (Note that some single-digit contributors were omitted.)

Changing distribution model

So it should be apparent that code contribution is beside the point. Redis is a venture-backed company that has taken more than $350 million in funding over many rounds since 2011. The company, and its investors, seem to have calculated that they can safely move away from open source to try to capture more revenue.

They have some reason to believe this is the case, if MongoDB's results are any guide. The company went public in 2017 and moved to the SSPL a little more than a year later. Shortly afterward, major Linux distributions stopped packaging the database because it no longer met their licensing standards. But, by that time, the company had set its sights on a platform model that would encourage developers (and their employers) to use and pay for MongoDB and ancillary offerings with the as-a-service model. Distributing a source-available version of MongoDB could be seen as a loss-leader strategy to reach developers that the company wagered did not care about open-source.

As Redmonk founder Stephen O'Grady wrote in 2017:

As developers began to assert control over technical selection and direction in increasing numbers, even in situations where a proprietary alternative is technically superior, the sheer accessibility of open source software gave it an enormous market advantage. Choosing between adequate option A that could be downloaded instantly and theoretically superior option B gated by a salesperson was not in fact a choice.

But open source, noted Grady, "is typically less convenient than service-based alternatives" and if convenience is the most important factor then open source has a problem. Especially when vendors like MongoDB have learned from proprietary vendors that "locking in customers is good for business".

Is it good for business? MongoDB has kept growing, adding customers, and brought in $1.68 billion its last fiscal year. That's more than a 30% increase, and its Atlas database service revenue also increased more than 30%, demonstrating that a lot of companies would rather pay to use the service than try to host it themselves. Despite all that, the company is still losing money—more than $345 million in the same time period.

But, investors may be more interested in stock price than actual profit. The company's stock started around $33 a share when it went public, but is now more than $350 a share. Redis's investors would likely be happy if it can produce similar results.

Forks and alternatives

Venture-backed vendors seem to have, as O'Grady wrote last year, reached a consensus that they can move away from open source. Especially if they are not "actively opposed by other commercial interests, foundations and other interested industry participants". Here, Redis may have miscalculated the industry mood.

When Hashicorp adopted BSL for its projects last year, a fork of its Terraform project appeared within days and was embraced by the Linux Foundation under the name OpenTofu. On March 28, the foundation announced that it was supporting Valkey, a direct fork of Redis 7.2.4, with Amazon Web Services (AWS), Google Cloud, Oracle, Ericsson, and Snap named as backers of the effort.

The Valkey fork got its life just a few days after the Redis license change. Olson wrote that she and "various former Redis contributors" had started working on a fork, using the original three-clause BSD license, with "placeholderkv" as a temporary name. Olson, Zhao, Viktor Söderqvist, and Ping Xie were listed as maintainers. According to Olson this was not an AWS fork of Redis, but "just me trying to keep the continuity with the community". KeyDB was considered, but it has diverged to the point that it "is missing a lot of stuff the community is used to".

The KeyDB fork was created in 2019 for technical, rather than licensing, reasons. The project, which billed itself as "a faster drop in alternative to Redis" was created by John Sully and Ben Schermel, who wanted a multithreaded version and were not able to persuade Redis maintainers to go in that direction. Sully and Schermel started a company, also called KeyDB, that offered a proprietary enterprise version. The entire codebase became fully open source under the three-clause BSD license when KeyDB was acquired by Snap in 2022.

The problem with KeyDB as a direct alternative is that it hasn't kept up with Redis since it forked. It still lacks many features found in Redis 7, and Sully indicated that there's little time for him to work on issues "not directly affecting Snap", though the project "would of course welcome outside help and we can certainly name additional maintainers if there is community interest in helping". On March 22, Sully updated another issue and said he was in discussions to "potentially" add maintainers to bring KeyDB closer to Redis 7. It's not clear yet whether Valkey will supplant KeyDB, but Snap's involvement makes that seem likely over the long term.

Drew DeVault, founder and CEO of SourceHut, has also created a fork named Redict based on Redis 7.2.4, but chose to use the LGPLv3. In his announcement post, he said that the choice of license was "a deliberate one which balances a number of concerns". DeVault wanted a license that was copyleft but "as easy as possible for users to comply" with the license and to ease integrations with Redis-compatible modules or Lua plugins that can be used to perform operations within Redis. He also noted that Redict will have no contributor license agreement (CLA), but contributors would be asked to verify contributions with a developer certificate of origin. Despite his connection to SourceHut, DeVault chose to host Redict on Codeberg to "provide a comfortable and familiar user experience for anyone comfortable with the GitHub-based community" of Redis.

Another kind-of contender is Microsoft's Garnet, announced on March 18. According to the announcement, it has been in development by Microsoft Research since 2021. It is a remote cache-store that can cache and manage the same types of data as Redis and is designed to be compatible with the Redis serialization protocol. Garnet is MIT-licensed, written in .NET C#, and is not meant to be a direct drop-in replacement. However, its API compatibility page claims that it can be "regarded as a close-enough starting point" that works "unmodified with many Redis clients". Many, but not all. For example, one user attempted to switch a NodeJS application to Garnet, but found that Redis's FLUSHALL command is not currently supported. Adding support for missing APIs on is on the project's roadmap.

Scramble for alternatives

Once again, Linux distributions are left with a mess to clean up. Neal Gompa opened a discussion on the Fedora development list, noting the license change and the need to remove Redis from Fedora. Jonathan Wright replied that KeyDB might be a replacement; he had been "loosely working on packaging" before the license change. He later said that KeyDB would be "a step backwards and cause headaches" for those looking to replace later versions of Redis. Nevertheless, he wrote on March 23 that he had pushed builds that were ready for testing for Fedora and EPEL 8 and 9.

Shortly after the Valkey announcement, Wright wrote that he would be packaging it as soon as there is a tagged release. Wright also said that he is "anticipating valkey becoming the [de facto] replacement for redis in most places."

Gompa also raised the issue on openSUSE's Factory discussion list. Dominique Leuenberger replied with a list of 18 packages with dependencies on the redis package in Tumbleweed. The initial discussion mentioned Redict and KeyDB as possible replacements, but Valkey had not been announced yet.

Having to find a replacement to ship in place of Redis is not the only problem for community distributions. Jacob Michalskie called out several services in use by the openSUSE project that will need a Redis replacement, including the Pagure code-hosting software (created and used by Fedora as well) used for code.opensuse.org, and the Discourse forum software.

Debian contributor Guillem Jover filed a Request for Package (RFP) for KeyDB as a potential replacement for Redis. Jover said he was not sure if he was up for sole maintainership, but was happy to give a hand. In an email exchange with Jover, he told me that his company had migrated from Redis 6 to KeyDB and it was a "smooth transition". According to Jover, "KeyDB might be lacking some features compared to Redis 7, but we have neither noticed we miss any or felt we were missing out on anything."

Jover said that it was too early to tell whether the newer forks would continue to be maintained, and that Redict's LGPLv3 licensing "might also be problematic for the ecosystem". In a follow-up email after the Valkey announcement, he said "I think we'll probably go further with packaging KeyDB for Debian at least, if it dies out it can always be removed or transitioned out from there."

Path forward

It is, of course, too soon to predict whether one or more of the forks will gain significant traction—but it seems likely that Valkey will be a credible alternative. The possibility of a swift fork with widespread community and industry backing should give pause to vendors who expect a smooth path after abandoning open source.

Did you like this article? Please accept our trial subscription offer to be able to see more content like it and to participate in the discussion.

---

(Log in to post comments)