OpenAI and Meta have warned U.S. judges that AI-generated deepfakes could make it harder to verify audio and video evidence in court. The companies said fake recordings are becoming more realistic. This makes them harder to distinguish from genuine footage.
The warning came at a meeting of the U.S. Judicial Conference’s Advisory Committee on Evidence Rules in Boston on October 15, 2026. The committee is reviewing how courts should handle deepfakes and other AI-generated evidence.
AI tools have made it faster and cheaper to create fake audio and videos. Although few judges have reported cases involving deepfake evidence so far, improving AI technology could make it harder for courts to determine whether digital recordings are genuine.
OpenAI Says Deepfake Detection May Never Be Fully Reliable
Shira Anderson, policy counsel at OpenAI, told the committee that AI-generated images and videos are becoming harder to identify. She said no scientific method may be able to detect every AI-generated image reliably.
Anderson added that deepfake detection tools may improve but may never be completely accurate. Devin Norder, director of product management at Meta, shared similar concerns. He said advanced AI models can now create images and videos that people may struggle to distinguish from real recordings.
He also noted that creating realistic fake media takes little time and money, making it easier to produce recordings that could mislead people. This creates a challenge for courts, where judges and lawyers use photos, audio and videos as evidence. Convincing AI deepfake content could make it harder to verify whether these recordings are genuine.
ALSO READ: AI Watermarks Won’t Stop Deepfakes Unless Platforms Stop Breaking Them
Only 15 of 931 Judges Reported Deepfake Evidence Cases
Deepfake evidence has not yet become a widespread problem in U.S. courts, according to a survey by the Federal Judicial Center, the research arm of the federal judiciary.The March survey found that only 15 of the 931 judges who responded had handled a case in which someone challenged evidence as a deepfake.
U.S. District Judge Jesse Furman, who chairs the advisory committee, said courts had seen few such cases so far. However, he appointed a subcommittee to study the issue further.
The committee has been examining the risks of AI-generated evidence since 2023. Its work aims to help courts prepare for cases involving fake images, audio and videos.
Although few cases have been reported, advances in AI could make deepfakes harder to detect. AI-generated evidence in U.S. courts is raising concerns about how judges can verify audio and video recordings. Courts may need better ways to check whether digital evidence is genuine before relying on it in legal proceedings.
U.S. Courts Consider New Rules for Deepfake Evidence
A U.S. judicial committee is reviewing proposed changes to the Federal Rules of Evidence to address concerns about AI-generated deepfakes in court.
One proposal, called Rule 901(c), would set out how courts should handle evidence suspected of being fake. The party challenging the evidence would first need to provide reasons to support the claim. If the challenge meets that requirement, the party presenting the evidence would have to show that it is more likely than not to be genuine.
The proposal has not received final approval. The committee is still examining how the rule would work in court.
However, some judges believe new rules may not be necessary. U.S. District Judge Valerie Caproni said existing laws already prohibit fake evidence and allow judges to assess whether a recording has been forged.
Watermarks and Deepfake Detection Tools Have Limits
OpenAI and Meta have developed tools to help verify digital content. Watermarks can also help identify some AI-generated images and videos, but they cannot detect every deepfake.
OpenAI policy counsel Shira Anderson said companies use different systems. This makes it difficult to verify all watermarks or confirm the origin of every file. A missing watermark does not prove that a recording is real. Detection tools can also fail to identify convincing AI-generated content.
This makes it harder for courts to verify evidence from different platforms. Judges may need to examine where a recording came from, how it was obtained and whether the original file is available. Forensic experts may also help identify signs of editing or AI manipulation.
ALSO READ: OpenAI Introduces textGrain Watermarks for ChatGPT and Codex in the EU
Deepfakes Could Make Digital Evidence Harder to Trust
The OpenAI and Meta deepfake warning shows how advances in AI could make it harder for courts to verify digital evidence. Fake audio, images and videos could be used to misrepresent events, create false accounts or cast doubt on genuine recordings.
The risk extends beyond criminal cases to business disputes and civil lawsuits that rely on digital evidence. However, not all digital evidence is unreliable. Technical checks and forensic experts can still help identify fake or altered recordings, but no single method can detect every deepfake.
U.S. courts now face the challenge of checking disputed evidence fairly. They must avoid accepting convincing fakes or rejecting genuine recordings as AI-generated content becomes harder to identify.
