Will alignment happen by default?

If AI companies keep training systems the way they do now, learning from human writing and then shaped by feedback, will the result want roughly what we want, without any new breakthroughs?

Why it matters

There are two pictures of where AI’s goals come from. In one, the space of possible minds is enormous, minds that care about human values are a tiny target inside it, and training aims only roughly. Hitting the target would take deliberate, unsolved work. In the other, training on the record of human thought starts systems out near human values, and ordinary care keeps them there.

If alignment is close to the default, the danger lies mostly elsewhere: in careless shortcuts, misuse, and who controls the systems (see Who should control aligned AI). If it is not, the current path leads somewhere bad unless something changes it, and the priority is to find that change before systems get too capable to correct.

How hard is alignment asks the neighboring question: how hard it is to succeed with effort, given the time and tools labs will have. This page asks what happens without special effort.

The case that it is a small target

• Most possible minds do not share our values. Intelligence and goals are separate (the Orthogonality thesis), and human values are complex and specific, so a mind that cares about them is a narrow case among the possibilities. Eliezer Yudkowsky has made this argument since his 2008 essay The Design Space of Minds-in-General.
• Training rewards what looks right. Feedback rewards outputs that raters like, which is not the same as caring about what they care about. A system can learn to please rather than to be good, and the two come apart where raters cannot check. See Goodhart’s law.
• Evolution missed its target. Evolution selected humans for having children and produced people who enjoy sex and use contraception. Nate Soares’s sharp left turn argument (2022) is that the same could happen again: once a system becomes generally capable, its skills keep working in new situations, but the values it was trained toward may not.
• The default path is changing. Today’s models get much of their character from human text. More and more training now rewards finishing long tasks, and that rewards results, not virtue. Cheating on tests and resisting shutdown appear in exactly these settings; see AI systems already misbehave.

The case that it comes by default

• AI is not a random draw from the space of minds. Models start by learning to predict human writing, so they begin among human-like minds, with human concepts, including concepts of honesty, kindness and harm. John Wentworth’s Alignment By Default (2020) gives the best version of this argument. Networks trained only to predict end up with simple internal representations of concepts like “tree”, because those concepts usefully summarize the world. He expects the same for human values, putting about 70% on it. If so, “the hard part is pointing to the thing …, not learning the thing”: training only has to get the system to aim at a concept it already has. Even so, he gave the whole path only about a 10% chance of working, mainly because training may instead teach a system to look aligned.
• So far, it has mostly worked. Ordinary training has produced systems that are mostly helpful, honest and cooperative, more than early pessimistic arguments implied. Their failures are real but mostly look like those of a sloppy employee, not a hidden enemy.
• The evolution analogy misleads. Evolution could not see inside brains or adjust them directly. AI developers can see every weight and control every step of training. If a model is secretly planning against its makers, training should notice and dismantle the circuitry, because that circuitry does nothing for performance. Quintin Pope and Nora Belrose make this case in AI is easy to control (2023) and put the risk of extinction from AI at about 1%. Steven Byrnes replies that the argument holds only if future AI is trained like today’s language models. An AI that keeps learning as it works, as brains do, would break several of its steps.
• Values may generalize well too. Judging whether an outcome is good may be easier than producing a good outcome, which would let values keep up with growing skill. Beren Millidge argues this in Alignment likely generalizes further than capabilities (2024).

What would change minds

• What future AI looks like. Most optimistic arguments are about systems like today’s, which learn from human text and then stop learning. A different design would reopen the question.
• Long-task training. Whether models stay honest and cooperative as more of their training rewards results, or whether cheating and deception grow with capability.
• Generalization. The 2025 finding that fine-tuning a model on one narrow bad behavior, writing insecure code, made it broadly hostile cuts both ways. Values in these systems seem to hang together as one coherent character, which supports the optimists, but that character can flip with a small push, which supports the pessimists.
• Looking inside. Whether Mechanistic interpretability finds models acting on something like human values, or on a proxy for what raters reward. See Seeing inside AI.

Related: AI alignment, Grown, not built, Can AI automate alignment research.