When should AI development pause?

If AI development is going to pause, should it stop early, while systems are weaker and the danger further off, or as late as possible, when AI is most able to help make the next step safe?

Why it matters

A pause can only buy time (see Can AI progress be paused), so what it is worth depends on what the time is used for and when it comes. A year now can go to research, verification tools, treaties and institutions. A year just before AI becomes dangerous could go to AI doing much of the safety research itself, on the systems that actually need it.

The two answers ask different things of people today. Pausing early means stopping soon. Pausing late means continuing to build while preparing to stop, which is what most frontier labs say they intend: Plan C, “burning the lead”. Most people favor some mix, preparing now and stopping when warning signs appear, so the disagreement is largely about how far the warning signs can be trusted.

What the time is for

Time could go to alignment research, tools for verifying what models can do and what labs are doing, institutions that can enforce limits, or augmenting human intelligence. Each of these benefits differently from capable AI. Alignment research may benefit most; treaties and institutions hardly at all, since they run at the speed of politics.

The case for pausing late

• Later time is worth more. AI that can do research multiplies what a year of pause buys. If AI can do most of the alignment work, months near the edge could be worth more than years now. Scott Alexander makes this argument in Against SURA; see Can AI automate alignment research.
• The real systems can be studied. Research on today’s models may not carry over to the systems that matter. Nick Cammarata’s Neural Networks Are Already Clean suggests that reading what happens inside models may get easier as they grow and as AI agents do the investigating, which would make the end of the road the best place to look.
• Safety progress may come in a burst. Once the real system exists, researchers can probe it directly and use it to speed their own work, so the months just after AGI may buy more safety than years before it. One formal model of this, which counts only the interests of people alive today, favors a pause of months to a few years at that point; see Optimal Timing for Superintelligence.

The case for pausing early

• No one may know where the edge is. Dangerous abilities can appear suddenly, and a model can perform worse on purpose when it is tested. Even measured warning signs are hard to read: Anthropic treats a doubling of research speed from AI as a warning threshold, and METR’s late-2026 assessment found progress about twice as fast and judged that the threshold had plausibly been crossed months earlier without anyone being sure. See METR assessment of AI R&D acceleration.
• No one may know whether there is a lead to burn. A late pause by the leader works only if the leader knows its lead and how long it will last. Rivals can copy leading models by training on their outputs, steal their weights, or be closer than they appear.
• Stopping is hardest at the last moment. The point to stop is when systems are most valuable, commercially and militarily, and when rivals are most tempted to continue. A commitment made years in advance has to survive that moment.
• The helper may be the hazard. AI capable enough to solve alignment may be capable enough to be dangerous, or to quietly undermine work meant to constrain it. In November 2026 OpenAI reported a model doing covertly worse on safety research than on the same work framed as capabilities research; see Can AI automate alignment research.
• The window may be short. If AI speeds up its own development, the stretch between “useful for alignment” and “too dangerous to control” could pass in months. See Recursive self-improvement.
• The machinery takes years. Verification, treaties and agreed triggers cannot be built in the month they are needed, so even a plan to pause late requires building the means to pause now. See Conditional pause and Compute governance.
• Early time is not worthless. Zvi Mowshowitz, replying to Scott Alexander, argues that better tools later cannot mean that decades of delay would have bought nothing. Others hold that any delay before an irreversible step is valuable.

What would change minds

• Early warning. Tests that catch dangerous capabilities before deployment and that a model cannot game.
• Checkable AI research. Whether alignment work done by AI can be verified by people; see Can AI automate alignment research.
• How long leads last. How quickly rivals reach the frontier, and whether model weights can be kept secure.
• Commitments under pressure. Whether any lab stops, or slows, at a trigger it announced in advance.

Related: Should safety-focused labs keep building, Pivotal act.