Anthropic researcher quits over ‘out-of-control’ AI fears

illuminem summarises for you the essential news of the day, reviewed by our editorial team. Read the full piece on The Wall Street Journal or enjoy below:

🗞️ Driving the news: Jacob Coxon, a researcher at Anthropic specialising in training models on large data volumes, said Tuesday he is leaving the company and the AI industry
• He does not want to participate in what he describes as an industry-wide rush to build self-improving AI systems, which he believes could spiral out of control and destroy humanity
• The WSJ frames the departure as a sign of mounting safety concerns inside leading AI companies

🔭 The context: The resignation follows a period of disclosed safety incidents across the sector
• Coxon concern is centered on competitive dynamics overriding internal caution, the structural problem that self-authored safety frameworks are least equipped to address
• Anthropic CEO Dario Amodei has publicly warned about risks from rogue AI models
• In September, the company said it had paused some training workloads while strengthening safety and security practices
• UN human rights chief Volker Türk told the Human Rights Council this week that advanced AI could pose an existential risk and called for independent verification and agreed red lines

🌍 Why it matters for corporate governance and sustainability: An individual resignation is the weakest available governance mechanism: it registers dissent by removing the dissenter, leaving the decision-making unchanged
• However, such departures are becoming the visible signal of internal disagreement indicates how little external accountability currently exists over these decisions.

⏭️ What's next: Coxon has urged colleagues at other labs to speak publicly and called for a temporary moratorium on capability improvement
• The resignation lands ahead of Anthropic's planned Nasdaq listing under the ticker ANTH, expected in October,
• It follows the departure of Mrinank Sharma, who led the company's Safeguards Research team and cited economic pressure producing decisions he could not reconcile with his values

💬 One quote: "I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives" — Jacob Coxon, post on X

📈 One stat: Coxon is 27 and spent three years in pretraining research across OpenAI and Anthropic before leaving the industry.

Subscribe to our free newsletters to never miss a beat in sustainability.

The world needs sustainability knowledge. At illuminem, no interest group or shareholder can influence our work. Thank you for supporting our mission to make high-quality and independent sustainability information free for all. Every contribution helps. Thank you for donating today