Meta panel says deepfake policies need update amid controversy over nude images

Meta’s policies on non-consensual deepfake images need updating, including wording that is “not sufficiently clear”, a company panel said in a decision on cases involving AI-generated explicit depictions of two famous women.

The quasi-independent Oversight Board said that in one of the cases, the social media giant failed to take down the deepfake intimate image of a famous Indian woman, whom it did not identify, until the company’s review board became involved.

Deepfake nude images of women and celebrities including Taylor Swift have proliferated on social media because the technology used to make them has become more accessible and easier to use, and online platforms have been facing pressure to do more to tackle the problem.

The board, which Meta set up in 2020 to serve as a referee for content on its platforms including Facebook and Instagram, has spent months reviewing the two cases involving AI-generated images depicting famous women, one Indian and one American.

The board did not identify either woman, describing each only as a “female public figure”.

Meta said it welcomed the board’s recommendations and is reviewing them.

One case involved an “AI-manipulated image” posted on Instagram depicting a nude Indian woman shown from the back with her face visible.

The board said a user reported the image as pornography but the report was not reviewed within a 48-hour deadline so it was automatically closed. The user filed an appeal to Meta, but that was also automatically closed.

It was not until the user appealed to the Oversight Board that Meta decided its original decision not to take the post down was made in error.

Meta disabled the account that posted the images and added them to a database used to automatically detect and remove images that violate its rules.

In the second case, an AI-generated image depicting the American women nude and being groped were posted to a Facebook group. It was automatically removed because it was already in the database. A user appealed to the board over the removal but it upheld Meta’s decision.

The board said both images violated Meta’s ban on “derogatory sexualised photoshop” under its bullying and harassment policy.

However it added that its policy wording was not clear to users and recommended replacing the word “derogatory” with a different term like “non-consensual” and specifying that the rule covers a broad range of editing and media manipulation techniques that go beyond “photoshop”.

Deepfake nude images should also fall under community standards on “adult sexual exploitation” instead of “bullying and harassment”, it said.

When the board asked Meta why the Indian woman was not already in its image database, it was alarmed by the company’s response that it relied on media reports.

“This is worrying because many victims of deepfake intimate images are not in the public eye and are forced to either accept the spread of their non-consensual depictions or search for and report every instance,” the board said.

The board also said it was concerned about Meta’s “auto-closing” of appeals over image-based sexual abuse after 48 hours, saying it “could have a significant human rights impact”.

Meta, then called Facebook, launched the Oversight Board in 2020 in response to criticism that it was not moving fast enough to remove misinformation, hate speech and influence campaigns from its platforms.

The panel has 21 members, a multinational group that includes legal scholars, human rights experts and journalists.