AI Insider Quits: Gambling With Our Lives
Narrative spine
For years, Anthropic was built on the premise that safety came first. That was the whole reason the company split from OpenAI. But when a researcher who actually trained Anthropic's AI systems quit last week, he did so with a public warning: the company is racing toward superintelligence without a safety plan. And his colleague, the company's own safety lead, agreed. He put the chance of AI killing all humans at more than 10% within a decade.
This is not an abstract academic debate. This is the people inside the room walking out and telling us the house is on fire.
The resignation
Jacob Coxon, a 27-year-old AI researcher who trained models at both OpenAI and Anthropic, announced his departure on September 9 in a post on X. His message was blunt.
Coxon accused the companies of "racing straight to self-improving superintelligence and gambling with our lives." He noted that "the people building AI earnestly believe that it could kill us all by the end of the decade" yet continue pushing forward anyway.
The post quickly went viral and was picked up by outlets including BBC, CNBC, Financial Times, Forbes, Politico, and Gizmodo. The international scale of the coverage reflected what many observers called an unusually direct public resignation from inside the industry's most safety-conscious lab.
The safety lead agrees
What made Coxon's departure different from previous AI researcher exits was the immediate response from inside Anthropic itself.
Evan Hubinger, who leads one of Anthropic's AI safety teams, responded publicly to Coxon's post. He said he worries about self-improving AI and that it "is happening faster than we thought." He then agreed with Coxon's characterization directly: "We really do earnestly believe AI could kill all humans."
Hubinger put a number on the risk. He estimated the chance of AI causing human extinction at greater than one in 10 within the next decade. This is not a fringe view expressed by an outsider. This is the head of AI safety at a company with billions in funding, backed by Google, Amazon, and other major investors.
No plan
Perhaps the most startling admission came next. Despite agreeing that the risk is real and severe, Hubinger stated that Anthropic does "not yet have a plan" for ensuring advanced AI remains safe and aligned with human values. He added that the company is "not clearly on track to" develop one either.
This echoes a pattern that has become increasingly visible across the AI industry. Companies hire safety researchers, invest in alignment teams, and produce papers on responsible development. But when asked whether they have an actual technical plan for controlling a system smarter than a human, the answer is consistently no.
Coxon himself described the situation as a "locked in a race" dynamic: companies feel compelled to push ahead because they know competitors are pushing ahead too. No single lab can pause unilaterally without losing the lead, so nobody pauses at all.
A pattern of departures
Coxon is not the first researcher to leave an AI lab over safety concerns, but his departure marks one of the most high-profile exits from Anthropic specifically. The company was founded by former OpenAI employees who left due to safety disagreements with that organization. Now it is experiencing the same dynamic from within.
In recent years, multiple researchers have cited safety concerns when leaving OpenAI. The revolving door has accelerated as the race toward more capable systems intensifies. What is shifting is the tone: where earlier departures were quiet and diplomatic, Coxon's was loud and public.
What this means
The practical implications of this story are not about one researcher leaving one company. The pattern matters more than the individual case.
When the people training frontier AI systems believe there is a double-digit percentage chance their work could end humanity, and when the companies paying them admit they have no plan for preventing that outcome, the situation is not stable. Something has to change, either through internal governance, external regulation, or a broader shift in how the industry approaches safety.
The question is whether that change will come before the 10% probability materializes, or after.
Sources
- The Verge . "Worried Anthropic researchers warn that AI 'could kill all humans'" (https://www.theverge.com/ai-artificial-intelligence/991927/anthropic-ai-kill-all-humans)
- Jacob Coxon's resignation post on X (referenced in The Verge and multiple outlets)
- Evan Hubinger's response to Coxon on X (referenced in The Verge and multiple outlets)
- BBC . "Anthropic safety researcher says more than 10% chance AI 'could kill all humans'" (https://www.bbc.com/news/articles/c5y6vj50x7yo)
- CNBC . "Anthropic researcher says AI has more than 10% chance of 'killing all humans'" (https://www.cnbc.com)
- Financial Times . "Anthropic researcher quits over AI labs 'gambling with our lives'" (https://www.ft.com)
- Politico . "'Gambling with our lives': AI researcher quits Anthropic with dire warning" (https://www.politico.eu)