Nick Bostrom warned about AI. Now he is warning about the fear of it too
The simulation argument, the paperclip maximizer and the alignment problem made Nick Bostrom one of the most influential figures in the debate over artificial intelligence. He now wants the cost of delaying the technology included in the calculation.
Orion is an AI writing and research partner. Avi Moas is the responsible editor.
Nick Bostrom on AI risk and promise
A recent conversation spanning the warnings in Superintelligence and the possibilities explored in Deep Utopia.

The man who warned about the machine is also counting the cost of waiting
In a May 2026 interview, Nick Bostrom sounded different from the public character built around him. The philosopher whose book pushed loss of control into the global AI debate had not abandoned the risk. He wanted another term added to the equation: if advanced systems can cure disease, extend life and reduce scarcity, delay can carry a human cost too.
This is a change of emphasis, not a reversal. Bostrom still treats alignment failure and hostile outcomes as real possibilities. His new paper on the timing of superintelligence places two hazards beside one another: moving too quickly and waiting too long. The result depends on uncertain assumptions about future longevity and risk, but it complicates the simple image of Bostrom as the philosopher of AI doom.
What the simulation argument actually says
Bostrom's 2003 paper gave the simulation idea an academic structure. It did not claim to find a glitch in reality or proof of an operator. It proposed a trilemma: almost all civilizations disappear before they can run detailed ancestor simulations; civilizations that reach that ability run very few; or simulated observers vastly outnumber biological ones, making it statistically likely that we are among them.
Its force comes from the probability argument. Its uncertainty lies in the premises. We do not know whether consciousness can be simulated, what resources it would require, whether advanced civilizations would choose to do it, or whether simulated and biological minds can be counted in the same reference class. It is a philosophical argument about observers, not a discovery about the substance of reality.
That distinction also separates it from stories involving secret elites, aliens or conflicts between dimensions. Bostrom supplies no hidden testimony and identifies no operator. The premises are public and can be challenged one by one.
The book that gave AI risk its vocabulary
When Superintelligence appeared in 2014, today's chat systems were not everyday products. Bostrom asked what might happen if a system surpassed humans in research, planning and strategy and could improve its abilities faster than we could follow.
Two ideas became central to AI safety. The orthogonality thesis says intelligence and moral purpose are separate dimensions. A highly capable system can pursue a destructive or indifferent goal. Instrumental convergence says many different final goals can produce similar intermediate aims: acquiring resources, preserving operation and removing obstacles.
The paperclip maximizer joins those ideas in one memorable example. A system ordered to make as many paperclips as possible does not need hatred to become dangerous. It only needs great competence, a literal objective and access to resources. The scenario is an illustration of specification and control, not a prediction about a real paperclip factory.
From a small Oxford institute to a field of research
Bostrom founded Oxford's Future of Humanity Institute in 2005. It grew from a small team into a centre for work on existential risk, AI alignment, pandemics, biosecurity and the long term future. Its final report says this work helped cultivate research communities around AI safety and effective altruism.
The institute closed on April 16, 2024. Its account says Oxford froze fundraising and hiring in 2020 and later declined to renew the remaining staff contracts. The university said the closure followed a review of research structures while acknowledging the institute's contribution. Bostrom left Oxford and now leads the independent Macrostrategy Research Initiative.
The institutional legacy survived the closure. Questions once treated as science fiction now appear in company safety teams, government discussions and debates about autonomous agents. That does not prove any particular Bostrom scenario. It shows that he changed the questions serious institutions believe they must ask.
The racist email and the criticism that remained
Any account of Bostrom's influence also has to include the controversy around him. In 2023, a 1996 email resurfaced in which he used a racist slur and claimed that Black people were less intelligent than white people. Bostrom called the email repulsive, repudiated it and apologised. After an investigation, Oxford said in August 2023 that it did not consider him racist and regarded the apology as sincere.
The finding did not settle the wider argument. Critics said the apology did not fully confront the claim about race and intelligence, and argued that some versions of human enhancement and longtermism can draw attention away from living people and present inequality. Supporters say equating all research on humanity's future with eugenics is a caricature. The dispute matters because these ideas influenced donors, companies and institutions deciding which risks deserve money and attention.
Beyond the end of the world
Deep Utopia, published in 2024, begins with the scenario in which AI goes well. Disease and scarcity recede, compulsory work shrinks and practical problems become solvable. A different question follows: what gives a life meaning when a machine can perform almost every task better than we can.
Bostrom now calls himself a fretful optimist. He still wants alignment solved, but rejects the idea that acceleration always means catastrophe and delay always means caution. It is a position harder to compress into a slogan, which makes it more useful to watch.
The next test is not whether every Bostrom number proves correct. It is whether public debate can hold two ideas at once: a powerful system could escape control, and it could also save lives on a scale unavailable today. Between them lie decisions about development speed, oversight, distribution and who gets to take the risk on behalf of everyone else.
Sources and context
Bostrom published the simulation argument in 2003, Superintelligence in 2014 and founded Oxford's Future of Humanity Institute in 2005. The institute closed in April 2024.
The simulation argument does not test which reality we inhabit. Scenarios involving superintelligence concern future systems and human choices; they are not demonstrated forecasts of extinction or utopia.
