Report: AI-generated fake and manipulated content about Black celebrities is spreading on YouTube

The report coincided with a wave of intense media scrutiny of AI-generated pornographic images of pop star Taylor Swift that spread across X (formerly Twitter), Instagram and Facebook.

404 Media traced Swift's sexually explicit AI-generated images to 4chan and a Telegram channel “dedicated to creating non-consensual AI-generated sexual images of women.” Members of this group, they wrote, “generate similarly explicit images of dozens of female celebrities, not just Swift.” (Media Matters previously reported that 4chan users have used Microsoft AI tools to develop other disturbing or hateful images and spread, including by Swift.)

The increasing prevalence of AI-generated content on social media poses a number of challenges. AI researchers have observed that study participants tend to “overestimate” their own abilities to detect deepfake audio and video. Meanwhile, developers of AI tools have admitted that in some cases they have difficulty identifying whether a creator has used their software to generate misleading media. Platforms also find it difficult to consistently moderate misinformation in general, which often leads to widespread dissemination of that information as users reproduce and reshare the content across platforms.

On the 404 Media Podcast, Samantha Cole and Emanuel Maiberg noted that Taylor Swift's “visibility could help raise awareness” about AI tools being used as a weapon against people, and that such abuse “happens to minor celebrities, it happens “It happens to regular people every day – we talk to these people every day – and the White House is not rallying around an Instagram influencer who has been faked or just a random person.”

As YouTube veteran Hank Green explained, “These models are getting better and better,” and whatever “tricks” people are using “right now to figure out if an image was created by artificial intelligence, you won't be able to use them.” Tricks in only a year.” He added: “We will not scrutinize every single one of the hundreds of images that we come across every day.”