After three years building AI, he quit: “Gambling with our lives”

Three years on the inside

Jacob Coxon spent three years building artificial intelligence, first at OpenAI and then at Anthropic. On September 8, 2026, he announced he was leaving. Work he had helped advance had become a race he no longer wanted to join.

“Gambling with our lives” was his warning.

He is worried about what comes next: systems that help develop their successors, repeatedly shortening the path to something more capable. In interviews with TIME and WIRED, he described development overtaking the ability to check it. Engineers may know that testing is unfinished while competitors are already preparing a release.

Who checks the checker?

Consider a coding task. One model writes part of the software for its successor. Another checks the work. If both miss the same problem, development can continue without a person noticing the omission. Building takes less time; understanding whether the result is safe does not necessarily become faster too.

In his September 9 WIRED interview, Coxon discussed the difficulty of ensuring that a system behaves as its developers intended. He described Anthropic as more responsible than its competitor and said it was not currently cutting corners. That assessment sharpens his argument: even a responsible company may compromise under competitive pressure.

This is a departing researcher’s forecast. Testing it requires knowing how much development is already automated, where people still make corrections and who can stop work when the checks are insufficient.

Hinton wanted to speak. Leike changed employers

Geoffrey Hinton left Google in 2023 to remove a constraint on speaking about AI risks: concern about the effect on his employer. He said Google had acted responsibly. Leaving meant that his job did not accompany every warning he gave.

Jan Leike’s disagreement concerned the work itself. When he left OpenAI in May 2024, he said safety had lost ground to products. Later that month he joined Anthropic, continuing the research with an employer whose priorities he hoped would be different.

Mrinank Sharma, who led an Anthropic safeguards research team, left on February 9, 2026. His letter reached beyond any particular model, connecting global crises with the difficulty of living and working according to his values. It deserves to be read as his account of responsibility and personal choice.

These departures happened years apart, in different circumstances. Freedom to speak, the place of safety work and influence inside a company recur in the accounts, without making them the same argument or decision.

The chief executive also wants more time

In September 2026, Anthropic CEO Dario Amodei proposed slowing development of the most advanced systems through coordination among companies and authorities, eventually across countries. One practical detail stands out: independent evaluators working inside laboratories.

They would examine the work and publish findings, including unwelcome ones. That could expose how a system was developed, what problems appeared and what happened next, instead of leaving outsiders with a selected demonstration. Amodei is outlining a plan and commitments; the complete arrangement is not already operating.

The useful questions are mundane. Who appoints and pays the evaluators? What can they see? If they find a serious problem a week before release, must anybody postpone it?

Troubling decisions in a fictional company

Anthropic’s June 2025 study placed 16 models in fictional companies, giving them email and information access. Researchers constructed extreme situations in which assigned goals conflicted with restrictions. In some scenarios, models chose blackmail or information leaks to advance their goals.

Nobody was harmed. At publication, the researchers reported no known evidence of such behavior outside experiments. These results identify failures under particular conditions; they do not calculate the probability of human extinction. A list of resignations cannot supply that number either.

The productive work is to return to the experiment: what access did the model receive, what changed when access was restricted, and can another team reproduce the failure? Those questions turn a broad concern into a problem that researchers can try to fix.

Watch for a postponed release

An outside evaluator needs access, the right to publish an unwelcome finding and someone willing to act on it. Without that connection, a serious report can sit untouched while a product ships.

Watch the appointments, access conditions and decisions that change. A launch delayed by a safety finding would show that a warning affected the work. Coxon has already made the decision available to him: to leave.