Meta is facing pressure to strengthen its rules on AI-generated and manipulated content after its Oversight Board ordered the company to remove two videos from Facebook. The Board said Meta’s current rules do not do enough to deal with harmful AI-generated content used to target people.
In decisions published on September 17, the Board reviewed two different cases. One involved a Scottish Labour councillor who was falsely shown making offensive comments about refugees. The other involved a young Muslim woman whose likeness was used in AI-generated videos that mocked her appearance and behavior. The Board said both cases showed problems with Meta’s policies and how they were enforced.
The Board also asked Meta to broaden its definition of “unwanted manipulated imagery.” Under the proposed change, the rules would also cover deepfakes that falsely show private people saying or doing things they never said or did.
Two Deepfake Cases Put Meta’s Safety Rules Under Scrutiny
The Oversight Board recently reviewed two cases involving AI-generated videos and images of real people. The cases involved different types of harmful manipulated content and raised questions about how Meta handles deepfakes on its platforms.
1. First Case Involved AI Video of Scottish Councillor
The first case involved a Facebook video posted in November 2025 that appeared to show a Scottish Labour councillor making offensive comments about refugees.
Meta’s Oversight Board said the video appeared to have been created or altered using AI. One sign was that the audio did not fully match the woman’s facial movements. The video was viewed more than 5,000 times and received more than 50 comments, 20 reactions and over 10 shares.
Two users reported the video to Meta, but the company’s systems did not send it for human review. Meta later told the Oversight Board that the video did not break its rules and did not meet the requirements for an AI-generated content label.
The Oversight Board reached a different conclusion and ordered Meta to remove the video. It said the post violated Meta’s hate speech rules because it falsely linked refugees as a group to criminal and sexually abusive behavior. The Board also said Meta should have placed a “high risk AI” label on the video.
2. Second Case Involved AI Images of Muslim Woman
The second case involved a young Muslim woman who had appeared in news coverage while campaigning for better menstrual health education.
After the coverage, people used her likeness to create AI-generated videos and images that mocked her appearance and behavior. Some of the manipulated content gained tens of millions of views across social media platforms, including Meta’s services.
The Oversight Board reviewed one of the videos, which showed the woman exercising in an exaggerated way and eating unhealthy food. It found that the video violated Meta’s bullying and harassment rules and ordered the company to remove it.
The case also highlighted a wider problem: people can use AI tools to create realistic images and videos of real individuals without their consent and then spread that content quickly across social media.
Oversight Board Pushes Meta to Strengthen Deepfake Protections
In its September 17 decisions, Meta’s Oversight Board said the company’s current definition of “unwanted manipulated imagery” does not cover enough types of AI-generated content.
Meta’s rules already cover some manipulated images involving private individuals and minors. However, the Board said the company’s enforcement guidance makes a distinction between changes to a person’s appearance and fake content that shows them saying or doing something they never actually said or did.
The Board said this distinction is becoming harder to justify as generative AI tools can create highly realistic images and videos. A person’s face, voice and actions can be digitally altered in ways that make fake content appear genuine.
The Board recommended that Meta clearly expand its definition of unwanted manipulated imagery. The change would cover deepfakes that falsely show private individuals saying or doing something, as well as manipulated content that changes their appearance without their consent.
The recommendation could also affect how Meta handles a wider range of deepfake content. The same AI tools are increasingly being used to create non-consensual sexual images and videos, including fake nude content involving real people.
ALSO READ: UK Bans Ads for AI Nudify Apps That Sexualize Real Women
Meta Deepfake Cases Reflect a Growing Online Abuse Problem
The two cases reviewed by Meta’s Oversight Board are part of a wider problem involving AI-generated images and videos used to target real people.
A 2024 study based on more than 16,000 people across 10 countries examined non-consensual synthetic intimate imagery, including deepfake pornography. The researchers found that 2.2% of respondents said they had been victims of deepfake pornography, while 1.8% said they had engaged in behavior linked to creating or sharing it. The study also found that awareness of this type of abuse was still relatively low.
The research also found that having laws in place did not prevent all cases of victimization or perpetration. The researchers said stronger platform rules, better tools to detect and remove harmful content, and greater digital awareness could help reduce the problem.
More recent research shows how the problem can also affect people in public life. Research reported by WIRED in September 2026 examined around 160 websites that host deepfake abuse and other non-consensual sexual content. It found that at least 138 women members of parliament from 22 European Union countries had appeared or been mentioned on the sites. Nine male MPs were also identified in the same research.
ALSO READ: AI Has Made Fake Nude Abuse Easier to Scale, New Berkeley Report Warns
Meta Faces Questions Over How Deepfake Reports Are Handled
The Oversight Board also raised concerns about how people are expected to report manipulated content on Meta’s platforms.
In the case involving the Muslim woman, Meta’s rules required the person targeted by the manipulated content to report it as unwanted. However, she had not reported the specific post reviewed by the Board. As a result, Meta did not apply that part of its policy.
The Board said this approach can put too much responsibility on victims, especially when the same fake images or videos are posted by several accounts. A person may have to report the same content again and again as it spreads across the platform.
The Board recommended that Meta add violating unwanted manipulated imagery to its Media Matching Banks. These systems can help Meta identify and remove copies or near-identical versions of content that has already been flagged.
It also recommended that Meta consider signs that content was created or altered using AI when deciding which reports should be sent for human review.
Board Calls for Stronger Action Against Harmful AI Content
In the case of a Scottish Labour councillor, the Oversight Board has made several recommendations after reviewing the case.
The Board called for Meta to use “high risk AI” labels more widely and take steps to limit the spread of deceptive AI-generated content. It also recommended stronger action against accounts that repeatedly share harmful deepfakes and greater transparency about how Meta labels AI-generated material.
One recommendation would reduce the visibility of content that Meta identifies as high-risk AI-generated material. Another would add warning screens that users would have to pass before viewing certain AI-generated content.
The Growing Challenge of Stopping AI Deepfake Abuse
Generative AI has made it much easier to create realistic fake images and videos. People no longer need advanced editing skills to change someone’s appearance or create a fake scene using their face or likeness.
The same image or video can also be copied, edited and shared across different websites and social media platforms. This makes it harder for platforms to remove harmful content completely. Taking down one post does not automatically remove copies that have already been shared elsewhere.
The Oversight Board’s recommendation to use Media Matching Banks is aimed at dealing with this problem. The system could help Meta find copies or similar versions of content that has already been identified as violating its rules.
The Board also highlighted the wider impact of online abuse on women and girls. It cited research showing that 38% of women across 51 countries reported personally experiencing online violence. The figure comes from a study by the Economist Intelligence Unit that surveyed 4,561 women.
Meta’s Deepfake Decisions Could Lead to Wider Policy Changes
The September 17 decisions do not create a new law on deepfakes. Instead, they put pressure on Meta to review how it identifies, labels and handles AI-generated and manipulated content on its platforms.
The Oversight Board can make binding decisions on the individual cases it reviews. Its wider policy recommendations are different. Meta must respond to those recommendations, but they do not have the same binding effect as decisions on specific pieces of content.
The recommendations could still lead to changes in how Meta handles deepfakes. The Board wants the company to look beyond whether an image or video has been changed using AI. Meta could also consider what the content falsely shows, whether the person gave consent, the potential harm and how widely the material is being shared.
These changes could affect how Meta deals with different types of manipulated content, including fake videos, harassment and non-consensual sexual images. They could also influence how quickly harmful material is identified, reviewed and removed.
The decisions also highlight a wider challenge for social media companies. As AI tools make realistic fake content easier to create and share, platforms will need systems that can identify harmful material and limit its spread without relying on victims to repeatedly report the same content.
