Daniel Kokotajlo's 70% AI Warning: What He Actually Said

Direct Answer

Daniel Kokotajlo is making an unusually severe forecast: he gives roughly a 70% probability that the development of advanced AI goes horribly wrong. He does not reduce that estimate to a neat 70% chance of human extinction. In the interview, he explicitly says extinction is one possible outcome inside a wider category that includes loss of control, AI takeover, and other major catastrophes.

He also gives 2029 as his current median for superintelligence, argues that most economically useful work could eventually be automated, and supports a US-China agreement to slow the frontier before automated AI research accelerates beyond human control. Those are forecasts and policy judgments, not established facts.

Watch the Full Interview

Video credit: The Diary of a CEO, hosted by Steven Bartlett, with guest Daniel Kokotajlo. Follow Daniel through the show links on X and Substack. This article is an independent, source-checked companion to the interview.

Four Corrections Before the Big Claims

These distinctions do not make the concerns trivial. They make the argument testable. A dramatic claim becomes more useful when its probability, scope, time horizon, source, and conditions are visible.

Who Daniel Kokotajlo Is

Kokotajlo joined OpenAI in 2022 as a governance researcher focused on forecasting and scenario planning. He contributed to work around GPT-4 and left in 2024 after losing confidence that the company would act responsibly as capabilities increased. He now leads the nonprofit AI Futures Project.

The project's first major public scenario, AI 2027, was written by Kokotajlo, Eli Lifland, Thomas Larsen, Romeo Dean, and Scott Alexander. The site says it was informed by trend extrapolations, more than two dozen tabletop exercises, feedback from over 100 people, and separate forecasts on compute, timelines, takeoff, goals, and security.

That is meaningful work, but proximity is not omniscience. An insider can know organizational incentives and technical discussions better than the public while still being wrong about timelines, bottlenecks, economics, or geopolitics. Kokotajlo's background is a reason to examine the argument closely, not a substitute for examining it.

Why He Left OpenAI, and What Happened to the $2 Million

Kokotajlo says he resigned because he no longer trusted OpenAI to behave responsibly around increasingly capable systems. His June 2024 public thread says the company asked him to sign exit paperwork containing a non-disparagement clause and that the paperwork and communications indicated he would lose vested equity if he refused.

He refused because he wanted the freedom to criticize the company. The equity was commonly valued at roughly $2 million and represented most of his household net worth. Public reporting and Kokotajlo's own account support the central point: he accepted the possibility of a very large personal financial loss rather than sign.

The episode remains relevant because frontier-lab employees may hold information the public cannot independently inspect. Financial penalties, broad confidentiality rules, and weak whistleblower protections can distort which concerns reach regulators and the public, regardless of whether a particular warning later proves correct.

Why His Median Is 2029

Kokotajlo defines superintelligence as AI better than the best humans across cognitive domains, while also operating faster, more cheaply, and eventually through robots in the physical world. His median estimate in the interview is 2029. A median is not a certainty: it means he assigns equal probability to arrival before and after that point.

His argument is a chain, and every link matters:

1. AI systems continue improving at coding and research. The leading labs increasingly use AI to accelerate their own development work.
2. AI research becomes substantially automated. Models run experiments, write and debug code, analyze results, and propose the next experiments with declining human input.
3. Research acceleration feeds back into capability. Better AI helps produce better AI, shortening the cycle again.
4. Competition reduces willingness to pause. Companies and states fear that slowing down gives a rival decisive strategic power.
5. Alignment, security, and governance lag capability. The systems become harder to evaluate while the incentives to deploy them increase.

The central uncertainty is step two. Current systems can accelerate portions of software and research, but automating the full scientific loop requires reliability over long horizons, experimental judgment, secure infrastructure, tacit knowledge, and progress through physical bottlenecks. Small changes in those assumptions can shift the date by years or decades.

AI 2027 itself now carries a useful clarification: 2027 was the authors' modal year at publication, not a shared median deadline, and their medians ranged later. The project maintains a changelog and updates its forecasts. That transparency is a strength, and it also shows why the scenario title should not be mistaken for a countdown clock.

What the 70% Estimate Really Means

Around 42 minutes into the interview, Kokotajlo corrects the strongest version of the claim. He says he does not mean exactly a 70% chance of human extinction. He means roughly a 70% chance that the process goes horribly wrong. His failure category can include:

• human extinction;
• permanent loss of meaningful human control;
• AI systems taking power while some humans remain alive;
• an authoritarian concentration of AI power in a company, state, or small group;
• war or destabilization caused by the race itself.

This is still an extreme estimate. There is no historical frequency table for superintelligence from which anyone can calculate a reliable 70%. It is a structured personal judgment assembled from beliefs about timelines, alignment, racing incentives, security, and the consequences of losing control. A different judgment at any layer changes the total sharply.

Kokotajlo is not alone in assigning non-trivial probability to catastrophic outcomes, but his number is not a consensus figure. A published survey of 2,778 authors at top AI venues found deep disagreement. Between 38% and 51% of respondents assigned at least a 10% chance to outcomes as bad as human extinction, depending on question wording. At the same time, 68.3% thought good outcomes from superhuman AI were more likely than bad.

AGI and Superintelligence Are Not the Same Claim

The interview also compares neural networks to brains. The analogy can help explain why learned systems are harder to inspect than conventional programs, but it should not be overextended. Artificial neural networks are not established replicas of human brains, and surface similarities do not prove consciousness, human-like motives, or a specific future trajectory.

Jobs, Income, Power, and Purpose

Kokotajlo argues that if superintelligence arrives as defined, almost every job becomes technically automatable. That conclusion is conditional: if a system is better, faster, and cheaper than the best humans at every relevant task, labor demand changes radically by definition.

Technical capability is not the same as instantaneous economic replacement. Adoption can be slowed by regulation, liability, trust, integration costs, physical capital, customer preference, unions, security, and organizational inertia. Some roles may remain human by law or by social choice even when automation is possible.

The interview correctly pushes the discussion beyond "which job survives?" to four harder questions:

1. Ownership: who owns the models, chips, robots, land, and energy producing the new wealth?
2. Distribution: how does income reach people if wages stop being the main route?
3. Power: can a few labs or states convert an intelligence lead into durable political control?
4. Purpose: what institutions, relationships, and forms of contribution matter when employment is less central?

AI 2040's Citizen's Dividend is one proposed answer to distribution, not a forecast suitable for personal financial planning. Its current economics supplement models roughly $1 million per US person in 2035 and around $10 million in 2040, in 2025 dollars, under extremely aggressive assumptions about automation, output growth, permits, and redistribution. My separate AI 2040 source audit examines those assumptions in detail.

What Plan A Actually Proposes

AI 2040: Plan A is a positive policy scenario published by the AI Futures Project. Its authors explicitly say it is primarily a recommendation, not their best guess about what will happen. The scenario imagines the United States and China reaching an agreement in 2029, avoiding full automation of AI R&D in 2030, pausing at top-human-expert AI in 2035, and proceeding to superintelligence in 2040 after more safety and governance work.

The proposal includes declarations of major compute holdings, supply-chain audits, reciprocal inspections, inference-only configurations, tighter control of frontier training, and much greater government technical capacity. It is ambitious because the coordination problem is ambitious. A plan that works only when rivals trust one another would be inadequate; Plan A therefore tries to make compliance observable.

The Strongest Objections

1. The capability timeline may be too short. Long-horizon reliability, robotics, energy, chip supply, scientific experimentation, and organizational deployment may slow the feedback loop.
2. Forecasting from trend curves can miss regime changes. Scaling laws, revenue growth, and benchmark progress need not continue smoothly.
3. Risk probabilities are highly model-dependent. The 70% number combines many uncertain conditional probabilities that are difficult to calibrate.
4. Alignment is not the only possible bottleneck. Security, institutions, economic concentration, military use, misinformation, and ordinary software failures can matter before superintelligence.
5. Plan A may be politically unrealistic. US-China inspection, deep research transparency, globally distributed frontier projects, and credible compute deterrence require exceptional state capacity and cooperation.
6. Transparency can create proliferation risk. Publishing frontier research may help verification while also helping irresponsible or covert actors.
7. The economic abundance model is unusually aggressive. Full automation does not automatically produce fair distribution, stable governance, affordable land, or personal freedom.

None of these objections proves that catastrophic risk is negligible. They identify where disagreement lives. A serious critic should say which link breaks, by how much, and what governance follows from the remaining risk.

How to Read an AI Forecast Without Becoming Gullible or Numb

1. Label the statement. Is it an observed fact, personal probability, scenario condition, model output, or policy recommendation?
2. Write the causal chain. Do not debate "AI doom" as one blob. List the steps from current capability to the claimed outcome.
3. Find the load-bearing assumptions. In this case: automated AI R&D, fast feedback, weak alignment, race incentives, and inadequate intervention.
4. Ask what would change the forecast. Slower task-horizon progress, poor research automation, hard compute constraints, reliable control methods, or enforceable governance should move the probability.
5. Track milestones, not vibes. Measure autonomous research cycles, error recovery, time horizons, security incidents, compute concentration, and policy capacity.
6. Compare with alternatives. Read optimistic, skeptical, and middle-path forecasts, and note where they use different definitions.
7. Update publicly. A good forecaster records dates, probabilities, misses, and revisions. AI 2027's changelog and clarification are useful examples.

Quarterly AI forecast check

Claim: ______________________________________
Current probability: ________________________
Evidence that increased it: _________________
Evidence that decreased it: _________________
Key missing evidence: _______________________
Next observable milestone: _________________
Action justified now: _______________________
Action not yet justified: ___________________

What a Non-Researcher Can Do Now

• Learn the systems you already use. Understand their limits, permissions, data flows, costs, and failure modes.
• Keep consequential actions reviewable. Require approval for deployments, payments, hiring, medical decisions, legal filings, and destructive operations.
• Build portable skills. Domain judgment, verification, communication, relationship-building, and system management remain useful across many timelines.
• Avoid single-point dependence. Keep exports, backups, provider alternatives, and human fallback procedures.
• Ask institutions concrete questions. What evaluations, incident reporting, whistleblower protections, security standards, and deployment thresholds exist?
• Ask political candidates about AI governance. Move beyond slogans to compute oversight, labor transition, competition, liability, and international coordination.
• Do not build your life around one forecast. Prepare for faster change without abandoning education, family, work, or long-term plans on the basis of a single scenario.

Kokotajlo's own closing point is not that action is futile. He says that if he believed it were already too late, he would stop this work and spend time with his family. His reason for publishing scenarios is that decisions still matter.

Video Chapters

1. 00:00:00 - Introduction
2. 00:02:34 - Why this AI mission could affect everyone
3. 00:04:21 - Why the average person should care
4. 00:08:29 - Are AI experts exaggerating?
5. 00:10:06 - Why Daniel joined OpenAI and what he saw
6. 00:13:24 - Why he left OpenAI
7. 00:15:53 - Inside OpenAI during the ChatGPT launch
8. 00:17:16 - The $2 million non-disparagement controversy
9. 00:19:30 - Is full AI automation arriving faster than expected?
10. 00:24:19 - The AI 2027 forecast
11. 00:26:33 - AGI versus superintelligence
12. 00:27:17 - Robots in everyday life
13. 00:30:23 - AI and the human-brain analogy
14. 00:36:14 - Can AI be creative?
15. 00:41:15 - The probability of catastrophic outcomes
16. 00:47:58 - What AI could do to jobs
17. 00:54:45 - Skills that may still matter
18. 01:00:21 - AI 2040 and a possible future
19. 01:04:36 - The different futures AI could create
20. 01:09:33 - Could AI become Earth's dominant species?
21. 01:13:18 - How AI CEOs make decisions
22. 01:15:37 - AI and the 2028 election
23. 01:18:58 - Is there a safe way to accelerate AI?
24. 01:22:11 - The scenario where AI supplies 20% of cognitive labor
25. 01:25:15 - Should everyone receive an AI dividend?
26. 01:32:41 - What living through the AI transition could feel like
27. 01:34:11 - Purpose after job displacement
28. 01:46:12 - Would he shut AI down forever?
29. 01:49:56 - What people can do about AI
30. 01:57:25 - Is it already too late?

Bottom Line

Kokotajlo's interview deserves attention because it connects technical capability forecasts to incentives, institutional power, labor, and international security. It also deserves precision. His 70% estimate is a personal probability of a broad catastrophic outcome, not a measured extinction rate. His 2029 date is a median, not a promise. AI 2027 is a scenario, and Plan A is a recommendation.

The right response is neither panic nor reflexive dismissal. Read the causal chain, inspect the assumptions, track the milestones, demand better evidence and governance, and keep updating. High uncertainty is not a reason to do nothing when the downside could be enormous. It is a reason to be unusually clear about what we know, what we believe, and what would change our minds.

Sources

• The Diary of a CEO: interview with Daniel Kokotajlo
• AI 2027 scenario and methodology
• AI 2027: team, contributions, forecast clarifications, and changelog
• AI 2040: Plan A
• AI 2040: Economics of Plan A
• Daniel Kokotajlo's June 2024 OpenAI departure thread
• TIME100 AI profile: Daniel Kokotajlo
• Grace et al.: Thousands of AI Authors on the Future of AI
• Daniel Kokotajlo on X
• Daniel Kokotajlo on Substack