AI voice cloning tool ElevenLabs has been abused before. Now the company is thinking about how to prevent these abuses.
A few days ago, AI startup ElevenLabs launched a beta version of its platform that allows users to create brand new synthetic voices for text-to-speech or clone someone’s voice. Unfortunately, it only took a few days for the Internet to start using this tool maliciously. The company tweeted that it is seeing “a growing number of instances of abuse of voice cloning”and is considering ways to address the issue by “implementing additional security measures.”
AI voice cloning tool ElevenLabs is already being abused
Although ElevenLabs didn’t specify what was meant by “case of abuse,”MotherBoard found posts on 4chan with audio clips in which voices were generated that sounded very much like celebrity voices reading or saying something very controversial. One of them, for example, has a voice similar to that of Emma Watson reading an excerpt from Mein Kampf. Users also posted videos with homophobic, transphobic, violent or racist comments. No one knows if all these clips really use ElevenLabs technology, but a post with a large collection of audio files on 4chan included a link to the startup’s platform.
This appearance of an audio “deepfake”is perhaps not a surprise, after all, insofar as a few years ago we happened to observe such a phenomenon. Advances in artificial intelligence and machine learning have led to an increase in deepfake videos, in particular deepfake pornography, with existing porn content altered to show celebrity faces. And yes, some have even used Emma Watson’s face in videos like this.
Now the company is thinking about how to prevent these abuses.
ElevenLabs is currently gathering feedback from users and professionals on how to prevent users from misusing this technology. There is currently talk of adding additional levels of account verification to enable voice cloning, such as making it mandatory to enter a payment method or ID. The company is also considering checking the copyright of the voice the user wants to clone, for example by sending a sample with the text read. Finally, the startup is considering withdrawing its Voice Lab tool from the market and only dealing with queries that it could check manually. To be continued!
Crazy weekend – thanks to everyone who tried our beta platform. While we are seeing our technology overwhelmingly put to good use, we are also seeing an increase in the misuse of voice cloning. We want to reach out to the Twitter community for thoughts and feedback!
— ElevenLabs (@elevenlabsio) January 30, 2023