The politics of AI risk

For years the worry about advanced AI was argued among researchers. Since 2023 it has become a political question, and politicians now face a range of proposals: transparency rules, slowing down, a ban on superintelligence, or a ban on AI that builds AI. This path reads through each proposal in turn, then looks at who stands where. Dated news is collected at the end.

1. Getting it said

Before anyone proposed a law, the worry had to become something a public figure could say.

The pause letter. The Future of Life Institute’s open letter (March 2023) asked labs to stop training systems beyond GPT-4 for six months. No lab paused, but it put the idea of stopping on the table.
The extinction statement (May 2023) put AI risk “alongside other societal-scale risks such as pandemics and nuclear war”, and was signed by the heads of the leading labs. Many serious people say AGI is dangerous covers it and the other statements.
The superintelligence statement. The October 2025 statement asks for a prohibition on superintelligence until it can be built safely and the public supports it. Its signatories, from Steve Bannon to Susan Rice, showed that the concern does not divide along party lines.
A movement. PauseAI turned the 2023 letter into a protest movement asking governments for an international pause treaty, and recruits supporters to lobby their own representatives.

2. Transparency, and who gets to regulate

The first binding rules were modest. California’s SB 53 (2025) and New York’s RAISE Act require the largest AI developers to publish safety plans and report serious incidents. They are the first laws written for frontier models, and they slow nothing down.

The bigger fight was over who regulates at all. The industry wants one federal rule that overrides the states. The Senate rejected a ten-year ban on state AI laws 99 to 1 in 2025, and the White House turned to the courts instead. This fight decides whether any of the stronger proposals below could start in a single state.

3. Slowing down from inside

In September 2026 Dario Amodei argued in “We Must Pace the Frontier” that labs should slow the rate of capability gains: outside evaluators inside labs first, then shared standards among democratic countries, then agreement with China. Sam Altman and Elon Musk said they agreed. It is a proposal to pace, not to stop, made by a lab that keeps building. Read it with Should safety-focused labs keep building.

A related demand is that every lab leader say in public that they would stop if the others did. Campaigners press for these conditional commitments as groundwork: they cost little now, but put on record that the obstacle is coordination, not willingness.

4. Banning superintelligence

The strongest proposals ban the goal itself.

In the US, Senator Bernie Sanders and Representative Greg Casar’s Ban Artificial Superintelligence Act (September 2026) would prohibit superintelligence permanently and pause advanced AI development until a new Department of AI is set up to oversee it. The bill text shows what such a law would have to define and enforce.
In the UK, a private member’s bill would ban developing superintelligence, and more than 70 MPs and peers wrote to the Prime Minister asking the government to back it and to seek an international agreement.

After Democrats won the November 2026 midterms, Representative Yassamin Ansari wrote that the “top of the Democratic agenda on day 1 needs to be a pause on advanced AI development, regulation of this out of control industry, and a ban on artificial superintelligence.” Musk’s mocking reply is on his page.

5. Banning AI that builds AI

A narrower proposal targets Recursive self-improvement: the point at which AI systems do the research that builds their successors, which the leading labs openly aim for. Alex Bores, a co-author of the RAISE Act, made the case to Congress in November 2026, arguing that “any mistake or misalignment in current AIs could be multiplied in each generation”, and that “we don’t get another election before AI companies reach this threshold.” He listed supporters of some version of it from Bernie Sanders to Steve Bannon, including Yoshua Bengio and Daniel Kokotajlo, and left open how a ban would work and how long it would last.

His evidence that the threshold is close is METR’s report that AI was already roughly doubling the pace of progress at Anthropic. Whether such a ban could be defined and enforced is part of Can AI progress be paused.

6. Who stands where

Three kinds of concern overlap, and none belongs to one party:

Loss of control. The argument in The case for AI risk, step by step, carried by researchers and campaigners and now by some legislators.
Distrust of big tech. On the populist right, Senator Josh Hawley presents AI as a threat to workers, children and human dignity. On the left the same distrust comes with demands to share the gains; see Wealth and power.
Local costs. Data centers, power bills and jobs reach voters who will never read about superintelligence, and in some races mattered more than arguments about risk.

The case against rests on two claims: that slowing American labs hands the lead to China, and that safety rules protect the incumbents who can afford them. The second is aimed above all at Anthropic, which funds the push for rules while competing to build what they would restrict. AI companies now fund political groups on both sides, and the first elections fought partly over AI gave no clear verdict on whether voters reward or punish regulating it.

The questions underneath

Each proposal rests on an answer to a crux. Whether stopping is possible at all is Can AI progress be paused. Whether the race with China rules it out is Can the US and China agree to limit AI. Whether a safety-minded lab should keep building while it asks for limits is Should safety-focused labs keep building. Proposals that stop at a threshold, such as a ban on AI that builds AI, are bets on When should AI development pause; A pause only buys time asks what any of them is for.

Latest

Newest first.

2026-11-05 — Alex Bores calls on Congress to ban recursive self-improvement, citing METR’s report on AI-driven acceleration at Anthropic.
2026-11-04 — After the midterms, Rep. Yassamin Ansari calls for a pause and a superintelligence ban as day-one priorities; Musk mocks it.
2026-11-03 — Midterms: AI money on both sides mostly backs safe winners.
2026-09-23 — Sanders and Casar introduce the Ban Artificial Superintelligence Act.
2026-09-12 — Amodei’s “We Must Pace the Frontier”.
2026-09-11 — More than 70 UK MPs and peers ask the Prime Minister to back a superintelligence ban.
2026-09-08 — Jacob Coxon resigns from Anthropic with a public warning; more than twenty members of Congress call for new AI rules.
2026-06-23 — Bores, the main target of a pro-industry super PAC, loses his congressional primary.

The Last Invention

Explorer