The Chinese Room

The original paper

Searle published "Minds, Brains, and Programs" in 1980 in Behavioral and Brain Sciences, alongside 26 peer commentaries. That format, a target article with immediate published replies, was unusual. It signaled the argument was meant to start a fight.

The context matters. Searle was responding to a specific claim, sometimes called strong AI, made most explicitly by Allen Newell and Herbert Simon in their 1976 physical symbol systems hypothesis. Their claim: a physical symbol system has the necessary and sufficient means for intelligent action. If a program processes symbols correctly, it is thinking. Searle said no.

The argument

Imagine you are locked in a room with a large book of rules for manipulating Chinese symbols. Chinese speakers outside pass notes under the door. You look up the incoming symbols, follow the rules, produce the outgoing symbols, and pass them back. From outside, the conversation looks real. You appear to understand Chinese perfectly.

You understand nothing. You are doing syntax, pure symbol manipulation, with no grasp of what any of it means.

A computer running a natural language program does the same thing. It receives symbol inputs, applies rules, produces symbol outputs. The program is the rulebook. The processor is you. Nothing in that process is understanding. Searle's term for this gap is intentionality: the property of mental states whereby they are genuinely about things in the world. Symbols in a program have no intrinsic intentionality. They only mean something because humans interpret them.

The systems reply

The most common objection: you don't understand Chinese, but the system as a whole does. You are just one component. Understanding is a property of the room plus the rulebook plus you, not of any single piece.

Searle's response was to internalize the whole system. Imagine you memorize the entire rulebook and walk around carrying it in your head. Now there is no separate system to point to. Everything is inside one person. Do you understand Chinese? You still have no idea what you are saying. The systems reply, Searle argued, just relocates the mystery rather than explaining it.

The robot reply

What if you embed the system in a robot? Give it cameras and motors. Let it interact with the physical world, point to real chairs when it says "chair," reach for real cups when it says "cup." Doesn't grounding the symbols in sensory experience produce meaning?

Searle found this the most interesting objection. But he thought it failed. Adding input-output channels to a symbol-manipulating system doesn't change what the system is doing. It is still manipulating symbols. The grounding you have added is causal, not semantic. The robot behaves as if it means things. That is not the same as meaning them.

The brain simulator reply

A sharper version: what if the program doesn't simulate behavior but simulates the precise activity of every neuron in the brain of a native Chinese speaker? Neuron by neuron, synapse by synapse.

Searle's answer depends on what you think is doing the work in the biological brain. If understanding arises from the causal powers of neurons, from wet chemistry and electrical gradients, then a program simulating those neurons hasn't replicated those causal powers. It has produced a formal model of them. A simulation of a hurricane doesn't make you wet. A simulation of digestion doesn't digest anything. A simulation of understanding, on this view, doesn't understand.

What Searle was actually arguing for

The Chinese Room is often read as a purely negative argument: computers can't think. But Searle had a positive view too. He called it biological naturalism. Mental states, including understanding and consciousness, are biological phenomena produced by the causal powers of the brain. They aren't programs running on the brain. They are what the brain does at the physical level.

This puts Searle in an odd position. He is a physicalist who thinks functionalism is wrong. He believes minds are entirely natural, fully explicable by biology, with nothing supernatural going on. But he also believes that no amount of software, however sophisticated, produces consciousness. The brain is not a computer. It is a specific physical system with specific causal powers, and understanding is one of the things those powers produce.

Where functionalists push back

Functionalism, the dominant view in philosophy of mind, holds that mental states are defined by their functional roles, not their physical substrate. Pain is whatever plays the pain role: caused by damage, causing distress, motivating avoidance. If something plays that role, it is in pain. The physical material is irrelevant.

Searle rejects this entirely. But functionalists have a reply to the brain simulator argument: if the simulation replicates not just behavior but the complete causal structure of the biological system, then the relevant causal powers are present. What makes the brain special is its organization, not its chemistry. And organization is something a program can replicate.

This is a live dispute. Neither side has knocked the other out. What the Chinese Room does is make the disagreement precise enough to argue about.

Why does the Chinese Room matter now that we have LLMs?

In 1980, Searle was arguing against theoretical claims. In 2026, the stakes are different. Large language models produce outputs that are often indistinguishable from understanding. They pass the Turing test in everyday interactions. They explain concepts, write code, compose arguments, and respond to novel situations in ways that feel, from the outside, like comprehension.

Most AI researchers think the Chinese Room doesn't settle the question, and they're right that it doesn't prove large language models are unconscious. But it remains the sharpest formulation of what would need to be explained. If an LLM understands, something in Searle's argument fails. The argument tells you which question to ask: not "does it produce the right outputs?" but "is there anything it is like to be this system?"

Nobody knows how to answer that. The Chinese Room is still the most honest thing anyone has said about why.

When you think an AI "understands" something, what exactly are you noticing?