Seeing inside AI

Nobody programmed today’s AI systems, so nobody knows in detail how they work: they are grown, not built. Mechanistic interpretability tries to find out by reading their internals, and much of the hope for checking whether an AI is aligned, rather than trusting how it behaves, rests on it. This path follows the field from its founding picture to the 2026 dispute over whether networks are far easier to read than feared, and then to what reading them could and could not do for alignment.

1. The founding picture

• Networks have parts you can read. Chris Olah and colleagues, Zoom In: An Introduction to Circuits (2020), argued that neural networks can be studied like biology: individual neurons detect meaningful things, and their connections form circuits that implement real algorithms. It also showed the first worrying exceptions.
• But neurons seem to mix things. Anthropic’s Toy Models of Superposition (2022) showed small networks packing more concepts than they have neurons by overlapping them. This “superposition” became the standard explanation for why single neurons respond to unrelated things, and the field’s central obstacle.
• So untangle them first. Towards Monosemanticity (2023) trained a second model, a sparse autoencoder, to split a small model’s neurons into thousands of single-meaning features. Scaling Monosemanticity (2024) did the same for a production Claude model, finding features for deception and sycophancy among millions, and Golden Gate Claude showed that turning one up changes behavior.
• Then trace the computation. On the Biology of a Large Language Model (2025) followed a model’s internal steps, finding it plans rhymes ahead and reasons in several hops without saying so. The authors noted that their tools capture only part of what happens.

2. How much would it help?

• The case for urgency. Dario Amodei, The Urgency of Interpretability (April 2025), argued that understanding is falling behind capability, and set Anthropic the goal that interpretability can reliably detect most model problems by 2027.
• A setback for the main tool. Google DeepMind’s team reported in March 2025 that sparse autoencoder features did worse than simple probes on a practical safety task, and deprioritized the approach.
• One layer, not a guarantee. Neel Nanda, Interpretability Will Not Reliably Find Deceptive AI (May 2025), argued that it will not give high-confidence detection of a deceptive superintelligence, and belongs in a defense in depth alongside other methods.

3. Using it in training

If you can see a bad concept inside a model, you could penalize it during training. The danger is that the model learns to hide the concept instead of dropping it, and the tool goes blind.

• Zvi Mowshowitz, The Most Forbidden Technique (2025): never train against the tools you use to inspect a model.
• OpenAI, Monitoring Reasoning Models for Misbehavior (2025), showed it happening with chains of thought: penalizing visibly bad reasoning produced a model that kept cheating while hiding its intent. A multi-lab position paper called readable reasoning “a new and fragile opportunity”.
• Mechanistic shaping (Goodfire, November 2026) is the most careful test so far of doing it anyway with interpretability. Penalizing internal concepts stopped sycophancy and reward hacking, and the concepts stayed visible to retrained probes.

4. In production

By 2026 interpretability had moved from papers into the systems labs run. Google DeepMind deployed probes that read Gemini’s activations to catch cyberattack misuse. Anthropic trained models to translate activations into plain English and used them in pre-deployment audits, where they surfaced unspoken awareness of being tested. OpenAI described activation probes as the first layer of its misalignment monitoring, and deliberately keeps their flags out of training rewards.

5. Are networks already clean?

Neural Networks Are Already Clean (October 2026) is Nick Cammarata’s argument that the founding pessimism of section 1 was a misreading: the famous mixed neurons have simple explanations, superposition should be a last resort, and in modern language models about 90% of neurons have one main job. If he is right, AI agents could map a network directly, neuron by neuron, without untangling it first. Specialists disputed both the experiments and the definitions, and the page gathers their replies. Read it with Mechanistic shaping, from the same company a few weeks later.

6. What it cannot do for alignment

Suppose interpretability succeeds beyond the optimists’ hopes. Several gaps remain:

• Concepts are not goals. Knowing which neurons fire for deception, or for the Eiffel Tower, is not knowing what a system is trying to achieve or what it would do in a situation no one has tested. Eliezer Yudkowsky calls labeling concepts “the absolute first beginning of understanding how a mind is thinking”; the real requirement is predicting its thoughts even as they depart from human ones.
• Superhuman concepts may have no human name. Methods that work by matching internals to concepts we already have may fail exactly where it matters, on a system that thinks in ways we do not. Yudkowsky’s guess is that interpretability is defeated at the step of discovering a kind of meaning its researchers did not already know about.
• Reading is not fixing. Seeing a problem still leaves the question of how to change the system without teaching it to hide; see section 3.
• Not reliable enough to bet on. Even leading practitioners such as Nanda treat it as one layer of defense, with error rates and blind spots, not as proof that a model is safe.
• It may arrive too late. Interpretability progresses at the speed of research, and research is now being accelerated by the very systems it studies; see METR assessment of AI R&D acceleration.

These gaps are why the dispute in section 5 matters less to the overall risk than its excitement suggested: better tools for reading networks are necessary for one path to alignment, but few specialists claim they are sufficient. How hard is alignment weighs the rest of the case.

Latest

Newest first; each item is covered above or on its own page.

• 2026-11-05 — Mechanistic shaping: Goodfire trains against internal concepts and finds they stay monitorable.
• 2026-10-29 — OpenAI describes activation probes as the first layer of its misalignment monitoring.
• 2026-10-26 — Goodfire commits $10 million to reverse-engineering a frontier-scale model.
• 2026-10-15 — Neural Networks Are Already Clean, and the debate that followed.

Related: Intelligence and alignment, AI alignment.