
Paul Christiano
Founder, Alignment Research Center
Worried, this week
Capability 5Outlook 2
Where they stand
Against all 37 people scored in Week 46. Bars count people at each score; black is Christiano, the dashed line is the average.
What they've said
- Monday, Nov 16Week 47
On the Dwarkesh Podcast, Paul Christiano said
I think we’re looking at something like maybe a 1/3 chance that we have superintelligence within one year on the current development trajectory, if we do not slow
, put the risk of AI takeover over the next year atmore like a 10% risk than a 1% risk
, and saidit would be better right now, in the current environment, for AI developers to pause further algorithmic progress
until their security can protect it.The episode ran under the title
We could be 12 months away from AIs that can overthrow the government
. Christiano, who recently joined OpenAI's board, argued that labs cannot safely slow down later becausethere is a good chance those systems would leak or be stolen
, and saidI’m currently extremely worried about the trajectory, even on the six-month timeline.
Asked what a scaled-up Hugging Face incident would mean, he saidI think it would most likely be hard for humanity to recover.
Details and sources
He gave similar takeover-risk warnings at the Hinton Lectures in Toronto on November 9 and 10. Dwarkesh Patel noted his OpenAI board seat; Christiano said he was speaking for himself. No OpenAI response to the interview was found.
Dwarkesh Podcast - Tuesday, Nov 10Week 46
Giving the second Hinton Lecture, on oversight of self-improving AI, Paul Christiano said
We cannot proceed at the current pace indefinitely
but thatthere is also really no evidence that that slowdown will occur in the next four years
, and warned that AI improving AIcould cause rapid acceleration
.Christiano said another ten years of progress is
reasonably likely
and that recursive self-improvement bringsvery significant risks
. He pointed to AI's recent breakthroughs in mathematics, which he calledan extremely alarming warning shot
for many people, and asked what happens if the same thing happens in machine learning.Details and sources
BetaKit also quotes him saying
BetaKitI think there is material risk even within the next six to 18 months
of takeover; a clip of Monday's lecture shared by Rob Wiblin hasmaterial risk of AI takeover in the next 6-12 months
, so that line may come from either night. BetaKit blocks direct reading; quotes were read through a fetch tool. - Monday, Nov 9Week 46
Opening the 2026 Hinton Lectures, Paul Christiano said
I think there is a high risk of an AI takeover within the next three years.
and addedI think for most people in this room, the most likely reason they will die is because of an AI accident.
The co-creator of RLHF called that his
most important sentence
and said building AI along the fastest possible path would meansomething like a coin flip of an AI takeover.
He said the field's trajectoryhas been extremely alarming over the last three to six months
and that there ismeaningful risk even in the next six to twelve months.
Details and sources
He closed:
The Singju Post, AI Safety FoundationThis is not a deal within a couple of years problem. This is a deal with this year problem.
He said the next two nights would cover why AI could soon overpower human institutions and what to do about it. The wording comes from a third-party transcript of the livestream published Nov 10; no news report of the lecture had been published at the time of this run. In an October preview interview with BetaKit he spoke of a six-to-18-month horizon.