Experts have long predicted that generative artificial intelligence would lead to a tsunami of fake photos and videos. An audio crisis is looming.
File image
From Pranshu Verma And Will we pray?
DAYS before a crucial national election in Slovakia last month, a seemingly damning audio clip went viral on social media. A voice that sounded like the leader of the country’s Progressive Party, Michal Šimecka, described a plan to manipulate the vote, including by bribing members of the country’s marginalized Roma population.
Two weeks later, another apparent political scandal erupted: the leader of Britain’s Labor Party was apparently caught on tape insulting a staff member in an obscenity-laden tirade posted on X, formerly Twitter.
Both clips were soon debunked by fact-checkers as likely fakes, as the voices bore telltale signs of being generated or manipulated by artificial intelligence software. But the posts remain on platforms like Facebook and X, generating outraged comments from users who assume they are real.
Rapid advances in artificial intelligence have made it easy to generate believable audio content, allowing everyone from foreign actors to music fans to imitate another person’s voice – leading to a flood of fake content online that is causing discord, confusion and anger stokes.
Last week, actor Tom Hanks warned his social media followers that bad actors were using his voice to falsely imitate him touting dental plans. Over the summer, TikTok accounts used AI narrators to display fake news reports that falsely linked former President Barack Obama to the death of his personal chef.
On Thursday, a bipartisan group of U.S. senators announced a bill called the No Fakes Act that would penalize people who produce or distribute an AI-generated replica of a person in an audiovisual recording or voice recording without their consent.
While experts have long predicted that generative artificial intelligence would lead to a tsunami of fake photos and videos, creating a disinformation landscape in which no one can trust what they see, an audio crisis is looming.
“This is not hypothetical,” said Hany Farid, a professor of digital forensics at the University of California, Berkeley. “You’re talking about violence, you’re talking about election theft, you’re talking about fraud – [this has] real consequences for individuals, for societies and for democracies.”
Voice cloning technology has advanced rapidly over the past year, and the proliferation of cheap, easily accessible tools online makes it possible for almost anyone to launch a sophisticated audio campaign from their bedroom.
It’s difficult for the average person to spot fake audio campaigns, while images and videos still have notable oddities – like deformed hands and garbled words.
“Obama still looks a little plasticky when bad actors use his face,” said Jack Brewster, a researcher at NewsGuard, which tracks online misinformation. “But the sound of his voice is pretty good – and I think that’s the big difference here.”
Social media companies also find it difficult to moderate AI-generated audio content because human fact-checkers often struggle to spot fakes. Nowadays, only a few software companies have protective measures in place to prevent illegal use.
Previously, voice cloning software produced robotic, unrealistic voices. But computing power has become stronger and the software has become more sophisticated. The result is a technology that can analyze millions of voices, recognize patterns in elementary units of language – called phonemes – and reproduce them within seconds.
With online tools like voice cloning software company Eleven Labs, almost anyone can upload a person’s voice for a few seconds, type out what they want to say, and quickly create a fake voice – all for a monthly subscription of $5 -Dollar.
For years, experts have warned that AI-powered “deepfake” videos could be used to make it appear that political figures have said or done harmful things. And the flood of misinformation in Slovakia offered a taste of how this will play out – with AI-generated audio playing a starring role instead of videos or images.
On Facebook, the audio clip of Šimečka and the journalist was played over a still image of their respective faces. Both condemned the audio as fake, and a fact-check by news agency Agence France-Presse found it was likely generated in whole or in part by AI tools. Facebook put a warning label on the video ahead of the Sept. 30 election, saying it had been debunked. “When content is fact-checked, we flag it and demote it in the feed,” said Meta spokesman Ryan Daniels.
However, the company did not remove the video and Daniels said it did not violate Facebook’s manipulated media policy. Facebook’s policy specifically targets manipulated videos, but in this case it was not the video that was altered, only the audio .
Research by Reset, a London-based nonprofit that studies the impact of social media on democracy, has uncovered several other examples of fake audio on Facebook, Instagram, Telegram and TikTok in the days leading up to the election. This included an ad for the country’s far-right Republika party, in which a voice that sounds like Šimečka’s says he “used to believe in 70 genders and pregnant men” but now supports Republika. A disclaimer at the end states: “The voices in this video are fictional.”
This video appears on Facebook without fact checking and was advertised on the platform as an ad from a Republika party leader. According to the Facebook Ads Library, it received between 50,000 and 60,000 views in the three days before the election.
About three million people voted in the parliamentary election, with the country’s pro-Russian populist party beating Šimečka’s Progressive Party for the most seats. Slovakia stopped its military aid to Ukraine after the election.
What impact, if any, the AI-generated speech falsifications had on the outcome is unclear, said Rolf Fredheim, a data scientist and expert on Russian disinformation who worked with Reset on its research. But the fact that they are spreading “like wildfire” in Slovakia means the technique is likely to be tried more frequently in future elections across Europe and elsewhere.
Meanwhile, the allegedly fake audio clip of British Labor leader Keir Starmer, who has a chance of becoming the next prime minister, remains on X, with no fact-checking or warning label.
Fears that AI-generated content could mislead voters are not limited to Europe. On October 5, U.S. Sen. Amy Klobuchar (D-Minn.) and Rep. Yvette D. Clarke (DN.Y.) sent an open letter to the CEOs of Meta and AI-generated content in political ads on their platforms. The two politicians introduced a law in May that would require a disclaimer for political ads that use AI-generated images or videos.
EU Commissioner Thierry Breton asked Meta boss Mark Zuckerberg in a letter on Wednesday to outline what steps his company would take to prevent the spread of deepfakes as countries such as Poland, the Netherlands and Lithuania go to the EU in the coming months Go to the ballot box.
Conspiracy theories generated by AI audio are also proliferating on social media platforms. In September, NewsGuard identified 17 accounts on TikTok that use AI text-to-speech software to create videos that spread misinformation, amassing more than 336 million views and 14.5 million likes.
In recent months, these reports used AI narrators to create fake news reports claiming that Obama was linked to the death of his personal chef, Tafarin Campbell; TV show personality Oprah Winfrey is a “sex trafficker”; and that actor Jamie Foxx was left paralyzed and blind by the coronavirus vaccine. Only after TikTok became aware of some of these videos were they removed, according to NewsGuard.
Ariane de Selliers, a spokeswoman for TikTok, said in a statement that the company “requires creators to label realistic, AI-generated content and is the first platform to develop tools to help creators do this and recognize this.” “How AI can increase creativity.”
Brewster, whose company conducted the study and specializes in misinformation, said voice deepfakes present a unique challenge. They don’t show their “glitches” as easily as AI-generated videos or images, which often show people oddities like eight Give fingers.
Although companies developing AI text-to-voice tools have software to detect whether a voice sample is AI-generated, these systems are not widely used by the public.
Speech software has also improved in replicating foreign languages as more and more datasets of non-English language audio exist.
The result will be more AI voice deepfake campaigns in countries that may be experiencing war or instability, the experts added. In Sudan, for example, alleged leaked voice recordings of former head of state Omar al-Bashir circulated on social media platforms, causing confusion among citizens as Bashir is believed to be seriously ill, according to the `.
In countries where social media platforms are essentially replacing the internet, there is no robust network of fact-checkers to ensure people know that a viral sound clip is fake, which makes these foreign-language deepfakes especially makes harmful.
“We’re definitely seeing these audio recordings resonate around the world,” Farid said. “And in such worlds, fact-checking is a much more difficult matter.”
More recently, Harry Styles fans have been confused. In June, alleged “leaked” snippets of Styles and One Direction songs surfaced on messaging channel Discord, selling to eager fans for sometimes hundreds of dollars apiece. But several “superfans” quickly analyzed the music and argued that the songs were AI-generated audio.
Broadcaster 404 Media conducted its own investigation into the audio and found that some samples sounded legitimate and others sounded “sketchy.” Representatives for Harry Styles did not respond to a request for comment on whether the leaked audio was real or an AI-generated fake.
UC Berkeley’s Farid said the ultimate responsibility lies with social media companies as they are responsible for disseminating and amplifying the content.
Although millions of posts are uploaded to their websites every day, the most sophisticated disinformation comes from a handful of profiles with large followings. It is not in the companies’ interest to remove them, Farid added.
“They could turn off the tap immediately if they wanted to,” he said. “But it’s bad for business.”
– THE WASHINGTON POST
Comments are closed.