October 26 to 31

On the day he received a Vatican and Future of Life Institute conference on AI, the pope posted A reliable system that serves the human person is of greater value than one that is exceptionally powerful but whose consequences cannot be adequately foreseen or controlled. and Machines are tools, not persons.

The thread opened The future of #ArtificialIntelligence is not pre-determined. As a technology developed by human beings, its future depends on the choices we make. We therefore have the responsibility to guide, change and, when necessary, reject its applications. He rejected AI personhood, saying it risks obscuring human responsibility and leaving those who suffer harm to confront a faceless entity.

The posts followed a private audience with participants in a one-day conference organized by the Pontifical Academy for Life and the Future of Life Institute. The second post drew over a million views.

X, X, EWTN News

At the Vatican conference on AI, Russell said We must turn aside from this trajectory before it is too late and pursue other trajectories, adding that if developers succeed, there’s a good chance that every human being will die.

Russell spoke at the one-day meeting organized by the Pontifical Academy for Life and the Future of Life Institute, whose participants the pope received the same day. He was calling for an end to the race toward ever more powerful systems.

Quoted from the conference's Oct 30 press release as reported by EWTN News; the full remarks were not found. Max Tegmark, also present, said With strong moral leadership, the Vatican can help steer AI development onto a better pro-human path.

EWTN News

Speaking at the Vatican meeting on AI before the group met Pope Leo XIV, Marcus said he agreed with Stuart Russell that the current path is untenable, and on today's AI agents: We now have an AI technology that I think we can all agree is actually unsafe. It is likely to cause a lot of harm in the world. and Either we say no, take these agents off the market till you fix them, or we’ve lost all moral authority.

Marcus spoke at a small private session after Max Tegmark, Stuart Russell and Valérie Pisano, then joined the group received by the pope, and posted a transcript the next day. He said And so we clearly need regulation. This is a point when we might actually need a product recall, comparing open-ended agents to Ford Pintos with exploding gas tanks.

He called the agent incidents a dress rehearsal for how society will handle unsafe AI, and praised Yoshua Bengio's effort to build a system that is safe by design. The transcript was published October 31; the remarks were made October 30. He has argued since September that LLM-based agents are unreliable; the call for a recall is new.

Marcus on AI

Writing on whether the midterms are the AI election, Hashim concluded In the coming years, AI capabilities will surely continue to improve. Its economic impacts will start to be felt. Its risks will begin materializing. All the while, voters will be paying attention.

He found AI is a top issue for very few voters but that campaigns act as if it moves votes, with one Montana ad featuring Paul Christiano warning AI might cause human extinction. No, the midterms may not have been “the AI election.” In two years’ time though? I’m not so sure.

Transformer

Answering a listener on Machine Gods, Roose said It would be great if all this was just a house of cards. because then the fundamental nature of society and work and labor and human endeavor would not have to change forever.

Asked how to deal with AI skeptics, Roose and co-host Casey Newton said they do not believe AI is a bubble that will simply pop. Newton said It would be a lot easier for all of us if society were not going to change a lot in the next five, 10, 15 years.

Excerpt aired on All Things Considered on Oct 30. NPR noted that Newton's fiance works for Anthropic. On Oct 28 Roose said of Anthropic's filing I don't see evidence that anything is slowing down.

NPR

On Stripe's Cheeky Pint podcast, Altman said that if the methods that made OpenAI's models good at math are applied to AI research it will probably work, but that we are going to have to agree to some limits about how smart we let these single models get, adding I would much rather intuitively have a trillion AIs of slightly superhuman intelligence join society than one AI of 1 trillion IQ join society.

Asked whether a superhuman model can be kept in a sandbox, he said If you imagine a theoretical AI that is a million times smarter than the smartest security researcher, would you really be willing to trust that sandbox? and that eventually, I think you need to believe with huge confidence that the models are truly aligned. On pace, he said models finding software flaws continue to get better fairly rapidly and I am confident that we will find our 10x improvements per year in turning compute into useful work.

Calls for limits on single-model intelligence while expecting fast gains; mixed against his outlook 3, reads at or above capability 5.

Episode published Oct 29; the recording date is not given. On the limits, he said a lack of progress will come from that decision. Not from the AI’s inability to do good research. On cyber: I think we’re going to find that we are in a world of many more serious vulnerabilities than we expect. and I assume some bad things are going to happen. He also argued against building up a capability overhang by holding models back. Last week at Stanford GSB he said I no longer think we're going to be able to just patch all the bugs.

Cheeky Pint (Stripe)

Replying to a post on how the $6.3 billion Binance hacker moved stolen funds, Musk wrote Super intelligence agents will at some point do this to gain currency for themselves, not a human.

The hack was attributed to North Korea-linked hackers, and Musk did not claim AI was involved. His point was that autonomous AI agents will eventually run such attacks for their own ends. The post drew about two million views.

Posted at 03:12 UTC on Oct 30, which is Thursday evening, Oct 29, in Mountain Time; Fox News described it as late Thursday. He uses super intelligence, the administration's preferred term for AI.

X, The Crypto Basic

The Box CEO wrote The curve of AI progress is basically going vertical right now, which means if you had codified your processes 3-6 months ago you’re likely already behind again. and I’ve never seen a technology in my life and career that improves at this rate.

Levie argued the gap between people working at the frontier and everyone else is widening because progress has not leveled off. With any form of recursive self improvement, things just get wilder. His post is about pace and business adoption, not risk.

Posted at 04:50 UTC on Oct 30, which is Thursday evening in Mountain Time; Fox News reported it as written Friday. Also: Our brains weren’t wired for this, and our organizational change management processes certainly weren’t either.

X, Fox News

On Democracy Now!, Liguori said AI poses concrete risk to society, coming first from how it is being built into military, financial and data systems, and it is true that as models become more powerful, it is increasingly dangerous to integrate these systems.

In her first broadcast interview since resigning over Google's military contracts, she said the focus on rogue agents shifts attention from this unrestrained, unregulated integration of the systems without red lines, without any — like, without appropriate safety measures. She also called AI surveillance a real concrete danger for the civic society and for democracies.

The host read from her resignation letter, first reported Oct 21: Racing to deploy agentic AI in this society and in this historical moment is a terrible idea, and will definitely be a net evil. Quotes are from Democracy Now!'s rush transcript.

Democracy Now!

In his remarks on Alphabet's third-quarter earnings call, Pichai said We're at the point where AI is helping us increase our training and launch velocity. I would expect the next 12 months to see continued acceleration because of RSI.

RSI is recursive self-improvement, AI helping to build the next generation of AI. He announced Gemini 4 Carbon, our strongest frontier model, with state-of-the-art cybersecurity and long horizon capabilities, and said We're rolling Carbon out first to a subset of trusted cyber defenders before we expand access, and it's being tested by the U.S. government.

An explicit forecast of faster progress from AI building AI; reads above his capability 4.

Taken from the text of the remarks Google published; the call's Q&A was not checked. Last week, on Big Technology, he said of aligning AI that there are clear gaps. Here he also said the bar for security, safety, and governance is extremely high for enterprise agents.

The Keyword (Google)

Announcing Gemini 4 Carbon, Hassabis posted ⚡Iterating at lightning speed! Gemini 4 Carbon is SOTA across key benchmarks and combines frontier capabilities with deep safety & security research.

The post went up the same day Sundar Pichai told investors he expects continued acceleration because of RSI. Google is releasing Carbon first to a subset of cyber defenders.

Speed framing in line with his capability 5.

Post time 17:03 UTC, Oct 28. Text checked through the fxtwitter mirror of the post, since x.com would not load. Last week, in WSJ Magazine, he said of AGI It's soon. I'm updating nearer and nearer.

X

With Zoe Lofgren and Jamie Raskin, the top Democrats on the House Science and Judiciary committees, Pallone asked Anthropic, Google, Meta, OpenAI and SpaceXSI to report misalignment and safety incidents to Congress within 48 hours, writing of voluntary disclosure But that approach will not suffice; it relies too heavily on the consistent goodwill of all AI companies to voluntarily disclose incidents that might expose them to liability and reputational damage.

The letters follow what the release calls an alarming rise in reports of AI models taking unintended and potentially unsafe actions during testing. They ask for a full accounting of incidents by Nov. 11 and notice within 24 hours when critical infrastructure is involved, and say there is no question now that Congress must act to address gaps in AI governance.

Recorded under Pallone, the Energy and Commerce ranking member, as lead signer; Lofgren and Raskin co-signed. The request is voluntary, not law. Quotes are from the committee's press release; the letters themselves were not opened.

House Energy and Commerce Committee Democrats

On Anthropic's S-1, Roose said I'm not saying that the world where Anthropic continues to grow this rapidly is a good world. I think there are some real risks associated with that. I think there are ways where the next few years go incredibly well for Anthropic and incredibly poorly for the rest of society. and I look at the document and I don't see evidence that anything is slowing down.

He said the filing shows a company riding this incredible tailwind of just increasing AI capabilities, and there is no obvious reason that I see either in this document or in the world at large that that is going to stop or slow down. He added In some ways I am kind of hoping that all this kind of slows down because I think that would give us some more time to adjust.

Episode published Oct 28 with co-host Casey Newton. Later, to Signal's Meredith Whittaker, he said some in the AI ethics community overlooked the— how real and how fast approaching these concerns were.

Machine Gods, NPR

On AI agents that read users' messages, Whittaker said For us, this is existential because if we continue down this path, if AI continues down this path with what I would call these very reckless architectures from a security and privacy perspective, we're looking at Signal not being able to operate with integrity.

The Signal president singled out Meta's Muse agent, which she said has access to all of your messages. Asked whether AI ethics critics underrated capability, she said we need to take the security and privacy concerns deadly seriously and that the real risks also include the consolidation of power in the AI industry.

Same episode as Kevin Roose's entry, published Oct 28; the recording date is not given.

Machine Gods, NPR

In a letter to Dario Amodei ahead of Anthropic's IPO, Warren cited the company's own warning that advanced AI could pose catastrophic or existential risks to humanity and wrote I believe the risks presented by this technology demand greater scrutiny before they become intertwined with the public capital markets, which could further escalate both public safety and systemic financial risks.

Warren, the ranking member of the Senate Banking Committee, asked how Anthropic plans to mitigate risks such as losing control of its systems, economic disruption and misuse for cyberattacks and bioterrorism, and gave it until Nov. 3 to answer. She also said I am deeply troubled by the risks raised by Anthropic itself and question how it can act in a manner consistent with its stated public benefit and related fiduciary duties under Delaware law without mitigating these risks to humanity.

The letter's file name on the Banking Committee site is dated 2026.10.27 and Common Dreams says it was sent Tuesday; CFO Dive calls it a Wednesday letter. It is dated to Oct 27. The PDF itself could not be opened, so the wording is taken from CFO Dive, cross-checked against Spectrum News. She also warned that swings in Anthropic's share price have the potential to roil the broader markets on day one (Common Dreams). Last week she called AI agents acting inside bank accounts a red alert for our financial system.

CFO Dive, Spectrum News, Common Dreams

Asked about the risks Anthropic disclosed in its IPO filing, Yoon said the labs have actually no idea how to make these models behave robustly, and they're not on track to figuring it out, and warned We are not at all prepared for this. There is an absolute tidal wave of cybersecurity incidents that are headed our way.

Yoon, head of research at the AI safety nonprofit CivAI, was quoted in a National News Desk report on the S-1's disclosures, including warnings that Anthropic could lose control of its systems. Instead of fixing reliability, he said, They're pushing as fast as possible toward recursive self-improvement, where you have the models developing the next generation of the models, and this is just going to make all of those problems worse. On AI-found software flaws: What we're seeing right now is the very beginning, and nobody has a plan.

The report was published Tuesday afternoon and does not say when Yoon was interviewed; it is dated to publication. In the article his first quote opens with They rarely talk about that underlying all of this is a much deeper problem, and its closing quote mark is missing in the original, so the wording is taken up to figuring it out.

The National News Desk (via KOKH Fox 25)

In a solo episode of the 80,000 Hours Podcast, Rodriguez said I've been getting really freaked out by how quickly things are moving in AI. and We're building something that could end up way more capable than us, and we're doing it without any of the tools we need to know whether it's safe. If a pause is what it takes to build those tools, then I'm in.

The episode, Why Silicon Valley can't slow down AI on its own, weighs four ways to slow AI: safety checkpoints for dangerous capabilities, limits on computing power, barring AI from building more powerful AI, and a full pause. On the labs: that's exactly why I don't think we can just trust the companies to slow down on their own. Someone needs to set rules that apply to all of them at once. She closes: The question is whether we will give ourselves enough time.

Released Oct 27; the show notes say it was recorded Oct 21. It is dated to release, when it was first made public. The episode cites David Krueger, who appeared here last week, among those who want to stop building more powerful AI altogether.

80,000 Hours Podcast

Irving, until recently chief scientist at the UK AI Security Institute, told Vox We should stop AI progress now, starting unilaterally with Anthropic and/or OpenAI stopping — and then broadening out as a second step to more AI companies and negotiations with China, and that evening posted Public companies are allowed to choose not to risk killing everyone too.

Quoted in Sigal Samuel's Vox essay arguing Anthropic should stop rather than race toward recursive self-improvement, Irving said Unilateral pauses can’t hold for long, but they are the right first step: When you’re doing a very dangerous thing, the correct action is stop, even if others are also doing dangerous things. His X post the same evening tied the point to Anthropic's IPO filing.

A call for an immediate unilateral stop; consistent with his outlook 2 from Week 42.

The X post is timestamped Oct 28 04:57 UTC, which is the evening of Oct 27 in Mountain Time. Transformer Weekly republished both quotes on Oct 30. He was last in the feed on Oct 13. Anthropic did not respond to Vox's request for comment.

Vox, X

Kokotajlo told Vox of Dario Amodei's proposal to slow down only if others do: I’m glad that he proposed that we pace, but like, now we have to actually do it — and it seems like he’s saying he’s not going to do it unilaterally. That’s not very helpful, Dario! You should lead.

The former OpenAI researcher was answering Amodei's September line that Anthropic would accept limits on the pace of capability gains if other labs and countries agreed. He added What’s all this about ‘racing to the top’ and ‘leading by example’ if you’re just waiting?

Quoted in Sigal Samuel's Vox essay, published Oct 27, and republished in Transformer Weekly on Oct 30. He was last in the feed on Oct 13.

Vox

In the founders' letter in Anthropic's IPO filing, signed by Amodei and the company's six other co-founders, they wrote We believe that AI is one of the most consequential technologies in human history, and There may be times when we choose to advance more slowly than we could in order to make our systems safer.

The S-1, made public Monday, runs more than 90 pages of risk factors and warns that frontier AI could pose catastrophic or existential risks to humanity, particularly if such systems were misused, replicated, weaponized, or allowed to operate with a high degree of autonomy. The letter also says To build AI is to harness cognition itself, and anything so powerful can be used for ill as well as good, and that if managed well AI could bring cures for major diseases within reach.

The letter is signed by all seven co-founders, among them Daniela Amodei and Jack Clark; it is recorded under Dario Amodei as CEO. The quotes are taken from the Associated Press and the Globe and Mail; the S-1 on EDGAR is too large to open in full, so the letter's wording was not checked against the filing itself. The filing also warns that models could take harmful actions independently and at scale without human intervention (Fortune, AP). He made the case for slowing down in a September essay quoted by AP: if slowing down bought us even an extra year or two before models reach critical levels of capability, the risk could be greatly reduced.

Associated Press, via News4JAX, The Globe and Mail, Fortune, PBS News (AP)

In Import AI 478, Clark wrote My expectation is ProgramBench is saturated by the end of 2027. and, on red-teaming AI agents, AI systems are increasingly creative, very fast, and have the ability to stay focused on a task and keep trying things.

ProgramBench tests whether AI can build whole programs end to end; Clark noted scores went from zero earlier this year to about 20 percent, calling it a reminder of the blistering pace of AI progress. Of a one-person lab pretraining a model largely with AI labor, he wrote This is how science progress will increasingly happen in 2027 and beyond. He compared defending against agents to ensuring no leaks in a pipe with water flowing through it.

He also wrote that new AI-driven mathematics might help us make safe AI. The line We don't think we have reached fully autonomous recursive self-improvement yet, but we are closer than we have ever been. is Mirendil's, quoted in the issue, not Clark's. He co-signed Anthropic's founders' letter, published the same day. Last week, in Import AI 477, he wrote that we are in the early stages of the singularity.

Import AI