Voice.AI: GPL Violations with a Side of DRM

Voice.AI: GPL Violations with a Side of DRM

04 Feb, 2023

Since the release of Eleven Labs’s Prime Voice AI platform, AI-based voice synthesis and modification has seen a revival in popularity. Browsing online forums, I stumbled upon Voice.ai, which bills itself as a “Real-Time AI Voice Changer” software program and community platform. Since the program seemed to run offline (more on that later), I was more interested in it than Eleven Labs’s offering. I figured it would be possible to pipe the output of a regular TTS like espeak into Voice.ai in order to obtain better results.

As a curious software developer and privacy-conscious person, after installing version 0.1.25.0 (I later updated to 0.1.25.1, which is the latest version as of my writing this post on February 4, 2023), I naturally dug into the program files to see exactly what was installed and how it worked.

GPL Violation Saga

After running strings on the files in the \Program Files\Voice.ai directory, I discovered some of the third-party components they were using: Praat and libgcrypt. These were statically linked into the VoiceAILib.dll library.

Truncated output from strings

% strings VoiceAILib.dll | grep -iE '^C:'
C:\Users\D\Desktop\PraatLib_CMake\PraatLib_CMake\external\gsl\gsl_specfunc__bessel_In.c
C:\Users\D\Desktop\PraatLib_CMake\PraatLib_CMake\external\gsl\gsl_specfunc__bessel_Kn.c
C:\Users\D\Desktop\PraatLib_CMake\PraatLib_CMake\external\gsl\gsl_specfunc__gamma.c
C:\Users\D\Desktop\PraatLib_CMake\PraatLib_CMake\external\gsl\gsl_specfunc__gamma_inc.c
C:\Users\D\Desktop\PraatLib_CMake\PraatLib_CMake\external\gsl\gsl_specfunc__beta_inc.c
C:\Users\D\Desktop\PraatLib_CMake\PraatLib_CMake\external\gsl\gsl_specfunc__beta.c
...
C:\Users\D\Downloads\libgcrypt-libgcrypt-1.10.1\cipher\cipher.c
C:\Users\D\Downloads\libgcrypt-libgcrypt-1.10.1\cipher\md.c
C:\Users\D\Downloads\libgcrypt-libgcrypt-1.10.1\cipher\mac.c
C:\Users\D\Downloads\libgcrypt-libgcrypt-1.10.1\cipher\primegen.c
...

This is concerning, since Praat is licensed under the GPLv3 and libgcrypt is licensed under the LGPLv2.1. These licenses are not included with the software at all; in fact, Voice.ai’s Terms of Service1 has sections which explicitly violate these licenses:

We retain all right, title and ownership to the Beta product. You agree the Beta Product is for personal use only. You may not sell, transfer, assign, pledge or in any way encumber or convey the Beta product or any portion or component thereof to any third party or use it in any manner to produce, market or support your own products. You shall not copy, sell or market Beta product to any third party; or modify, reuse, disassemble, decompile, reverse engineer or otherwise translate the beta product or any portion thereof

Meanwhile, the GPLv3 states plainly:

When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work’s users, your or third parties’ legal rights to forbid circumvention of technological measures.

In order to confirm that these strings were in fact evidence of copied GPL code, I fired up the Ghidra reverse engineering tool and searched for references to these strings. After decompiling VoiceAILib.dll, I found many functions that matched code from the Praat GitHub repository.

Function gsl_sf_lnbeta_e

Decompiled

undefined8 FUN_18016b350(undefined8 param_1,undefined8 param_2,undefined8 *param_3)
{
  undefined8 uVar1;
  double local_res20;
  
  uVar1 = FUN_18016b3b0();
  if (local_res20 == -1.0) {
    *param_3 = 0x7ff8000000000000;
    param_3[1] = 0x7ff8000000000000;
    FUN_1801431f0("domain error",
                  "C:\\Users\\D\\Desktop\\PraatLib_CMake\\PraatLib_CMake\\external\\gsl\\gsl_specfun c__beta.c"
                  ,0x2c);
    uVar1 = 1;
  }
  return uVar1;
}

Original Code

int
gsl_sf_lnbeta_e(const double x, const double y, gsl_sf_result * result)
{
  double sgn;
  int status = gsl_sf_lnbeta_sgn_e(x,y,result,&sgn);
  if (sgn == -1) {
    DOMAIN_ERROR(result);
/* expands to:
    do { (result)->val = GSL_NAN; (result)->err = GSL_NAN; GSL_ERROR ("domain error", GSL_EDOM); } while(0)
*/
  }
  return status;
}

See more evidence…

Reaching Out for an Explanation

In order to figure out the best contact method to discuss the license issues, I joined the Voice.ai Discord and asked a simple question:

I was then told I could either email [email protected] or direct message Heath, the founder of the company. I opted to send an email, since that was the more “formal” communication method.

On February 2, 2023, I sent an email to [email protected] requesting source code. Originally I had only identified libgcrypt, and so I cited relevant sections of the LGPLv2.1 in my message. The full text of my message can be found here, but here are the most important parts:

To the developers of Voice.ai:

I have recently downloaded and installed the Voice.ai Software on my PC. I have also discovered that the Voice.ai software includes a software component licensed under the Lesser GNU General Public License version 2.1 (LGPL v2.1) - more specifically, libgcrypt within VoiceAILib.dll.

In accordance with the LGPL v2.1, I am requesting a copy of the libgcrypt source code plus the source code of VoiceAILib.dll and of any other components, tools, and/or scripts necessary to reproduce a working executable with my own version of libgcrypt. Your requirement to fulfill this request is explained in Section 6 of the LGPL v2.1 text

Thank you for your time.

After later identifying the presence of the GPLv3-licensed Praat code, I sent a more succint follow-up email:

I have also discovered that VoiceAILib.dll also includes Praat, which is strictly licensed under the GNU General Public License version 3 (GPLv3). As a result, I am requesting the full Voice.ai client software source code, which you must release in order to comply with the GPLv3 license.

As of February 6, 2023 at 11 AM, I have not received a response to my emails.

Lashing Out in Return?

February 6, 2023 update: The actual reason I was banned was for “ToS violations” in regard to discussing DRM evasion. Regardless, those provisions are not practically enforceable due to the GPL code and the license’s requirement to allow DRM circumvention.

A couple of days later, on February 4, I suddenly found myself banned from the Voice.ai Discord server.

I received no warning from any moderator or developer, and I had sent fewer than ten messages during my time in the server; therefore, I have no reason to believe I broke any legitimate rule.

Invasive DRM Alert

Voice.ai developers are utterly insistent that their software is not malware at all, but the widespread warnings from antivirus software do raise some questions. Now, Voice.ai certainly is not a cryptominer, and modern antiviruses are overly paranoid and annoying, but Voice.ai does collect a concerning amount of data from the systems it runs on.

The Voice.ai software is heavily obfuscated, and the main components of interest are the aforementioned VoiceAILib.dll and the main VoiceAI.exe executable, a C/C++ DLL and .NET assembly respectively. VoiceAI.exe collects at least the following information:

• Motherboard and CPU info
• Audio interfaces
• OS version
• Enabled network interfaces, along with IP address and MAC address
• Computer hostname
• Voice.ai install path

While some of this information has obvious legitimate uses for debugging or otherwise (audio interfaces, OS version, install path), other information such as the computer hostname and network interface metadata is completely irrelevant to Voice.ai’s primary function. This information is sent to the Voice.ai servers and used to derive an encryption key to encrypt and decrypt later communications with the API. Note that all communications with the API happen over HTTPS and are already secured, so this only serves as obfuscation to dissuade reverse engineering.

Other users have reported in the official Discord server that the software also contains virtual machine detection routines. It’s no wonder that antivirus software often detects it as malware: no other class of software is this heavily obfuscated, gathers this much information, attempts to avoid being executed in a virtual machine, and sends what it gathers to a central server. While this isn’t to say that Voice.ai is the next Zeus trojan, it is very understandable why it seems to be a victim of antivirus false positives.

Because of this “DRM spyware,” it is not possible to run the Voice.ai software offline, even though it is clearly technically possible to do so, since it requires a local GPU for live AI processing.

Reflection

Voice.ai developers claim that such obfuscation is necessary in order to protect their proprietary secrets (which, by the way, are not allowed to be secrets due to the included GPL code); however, anyone should be wary of feeding their text and voice recordings to a “blacker than black” box program.

In Other News

Last updated February 17, 2023.

• Apparently Voice.ai got around to putting their Terms of Service in the application. Unfortunately for them, actually reading the terms was broken until they later fixed it when someone on Hacker News pointed out the issue. This likely means nobody has actually “agreed” to their ToS, since it wasn’t actually presented to them. Good to know Voice.ai devs are paying attention now, though.

• Voice.ai also violates permissive licenses, including but not limited to libFLAC’s license and OpenH264. These licenses require attribution, but none is given.

• The moderators of the Voice.ai Discord report that the Voice.ai developers are now supposedly looking into the license issues with their legal team.

• Voice.ai has officially responded to these accusations.2 Removing the GPLv3 code is perfectly acceptable; however, in order to comply with the license, they still need to release the source code to version 0.1.25.1. They claim that they will be making the “relevant source code” available on GitHub. I am not sure what that will include, but I do hope it fulfills the requirements. In addition, their statement mentions their mission to “democratize AI”; however, I feel that closed-source, proprietary software is rarely the way to democratize anything. There is no way to learn or explore outside of the bounds of an overly strict and arguably paranoid license agreement.

• I find it important to clarify that this blog post was not published until February 6, 2023. Most of the content, however, was written on February 4, 2023.

• I fixed the download link.

• On February 14, 2023, the Voice.ai developers released 0.1.26.1, which seems to remove Praat; however, it still includes LGPL’d libgcrypt and they still violate the license requirements of their open source dependencies. Furthermore, they have not released the code for 0.1.25.1.

• Since the 0.1.26.1 update, some users on the Voice.ai Discord are reporting lower quality results.

• On February 17, 2023, with the 0.1.27.1 update, the Voice.ai developers have finally moved libgcrypt to its own DLL and included the licenses of their open-source dependencies. They have still yet to release the code for 0.1.25.1.

Closing Thoughts

To The Reader

I personally recommend downloading a copy of the Voice.ai software and contacting the developers to request source code. Showing community intolerance for GPL violations is one of the best strategies for combatting them. If you intend to actually run and use the software, you should launch it in a sandbox to protect your privacy.

To The Voice.ai Developers

I implore you to release the code to your software under the GPLv3 as the license requires in order to avoid future trouble. It is not too late to do so, and the community will thank you. I also implore you to better respect your users’ privacy, even if that is at the expense of protecting “proprietary secrets.”

#AI #Voice Synthesis #Licensing #Spyware #GPL #DRM #Privacy #Reverse Engineering

Copyright © 2021-2023 undeleted, Ronsor Labs.
The content on this site is provided as is, without any warranty.

Made with Hugo ʕ•ᴥ•ʔ Bear. Hosted on Fly.