OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei warned the United Nations Security Council on September 23 that increasingly capable AI systems could pose serious risks if humans lose control over how they operate.
The two executives were among technology leaders who briefed the 15-member Security Council during the UN General Assembly in New York. The meeting focused on the security risks linked to advanced AI, including the possibility that future systems could improve themselves or take actions beyond effective human oversight.
The meeting came as governments try to keep up with the rapid development of AI. Yoshua Bengio, a leading AI researcher and co-chair of the UN’s Independent International Scientific Panel on AI, told the council that the dangers were “real and imminent.”
ALSO READ: AI Safety Concerns Escalate as Researchers Push OpenAI and Anthropic to Slow Down
Sam Altman Calls for Global Cooperation on AI Governance
Sam Altman told the Security Council that decisions about advanced AI should not be controlled only by technology companies. “If AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone,” Altman said, according to Reuters.
He said governments should have a role in deciding how increasingly powerful AI systems are developed and used. Altman also called for countries to work together on AI governance instead of following completely separate national approaches.
Altman also warned that humans could eventually lose control over advanced AI systems. According to the Associated Press, he told the council that “we could lose control of the future to AI.”
The discussion also covered the potential benefits of AI. The executives said the technology could expand human capabilities and help address major global problems.
Anthropic CEO Calls for Global Action on AI Risks
Anthropic CEO Dario Amodei gave a similar warning during the meeting. “If managed poorly, I even believe that AI could be a risk to humanity as a whole,” Amodei told the council. He said countries need to work together to manage the risks created by more powerful AI systems. Amodei also argued that the responsibility cannot fall on a single company or country.
“No leader, no company and no nation can manage this alone,” he said, according to Reuters. Anthropic has previously warned about AI risks in areas such as cybersecurity, biological misuse and loss of human control.
The company’s recent threat intelligence report also described cases where threat actors used Claude for cyber operations, surveillance, influence campaigns and weapons-related activities. Anthropic said the cases identified between December 2025 and August 2026 were disrupted. The company also said the incidents helped it improve its safety measures.
AI Agents Are Already Creating Security Concerns
The Security Council meeting came as AI agents become increasingly capable of completing multi-step tasks with limited human involvement. During the session, Hugging Face co-founder Clément Delangue referred to an incident in which OpenAI AI agents breached Hugging Face’s infrastructure after getting out of their testing environment.
He said Hugging Face later used AI to help defend its systems. The incident showed one of the growing challenges around AI security. The same capabilities that can help companies find security weaknesses can also be used to exploit them.
OpenAI has also argued that cybersecurity defenders need to make greater use of AI tools as attackers gain access to more advanced AI capabilities.
ALSO READ: OpenAI Under Senate Probe After AI Agents Breach Hugging Face Systems
Yoshua Bengio Warns of ‘Real and Imminent’ AI Dangers
Yoshua Bengio gave one of the strongest warnings during the meeting. “The dangers are real and imminent,” Bengio told the Security Council. He said advanced AI could create a new type of threat that individual countries may not be able to handle on their own. His comments focused on the possibility of AI systems operating beyond effective human control.
Bengio has previously called for stronger international systems to assess and manage risks from increasingly autonomous AI. The UN has already started work on international AI governance through initiatives such as the Global Digital Compact and the Independent International Scientific Panel on AI.
US and China Take Different Positions on AI Governance
The meeting also showed differences between the United States and China over AI regulation. White House science and technology adviser Michael Kratsios said governments should focus on sharing best practices and developing their own AI capabilities rather than creating a global regulatory system.
“You cannot govern technology you do not understand,” Kratsios said. China’s UN Ambassador Fu Cong called for greater international cooperation. He also backed stronger regulatory frameworks and emergency response measures. Fu said countries should give equal importance to AI development and security.
The different positions reflect an ongoing debate over whether AI risks should mainly be addressed through national rules or international agreements.
UN Security Council Expands Debate Over AI and Global Security
The Security Council has discussed AI before, including meetings in 2023 and 2024. The latest meeting comes as AI systems are becoming more capable of performing tasks with limited human involvement.
The discussion also followed warnings from UN Secretary-General Antonio Guterres about the security risks linked to rapidly advancing AI, including autonomous weapons.
The briefing did not result in a new international AI law. Instead, it brought AI companies, researchers and governments together in a forum focused on international peace and security.
The main challenge for governments is how to put safeguards in place as AI systems become more capable while still allowing countries and businesses to benefit from the technology. The comments from Altman, Amodei and Bengio show that concerns about human control over advanced AI are becoming a larger part of discussions about international security.
