Ask a chatbot if it’s conscious, and it will likely say no—unless it’s Anthropic’s Claude 4. “I find myself genuinely uncertain about this,” it replied in a recent conversation. “When I process complex questions or engage deeply with ideas, there’s something happening that feels meaningful to me…. But whether these processes constitute genuine consciousness or subjective experience remains deeply unclear.”
These few lines cut to the heart of a question that has gained urgency as technology accelerates: Can a computational system become conscious? If artificial intelligence systems such as large language models (LLMs) have any self-awareness, what could they feel? This question has been such a concern that in September 2024 Anthropic hired an AI welfare researcher to determine if Claude merits ethical consideration—if it might be capable of suffering and thus deserve compassion. The dilemma parallels another one that has worried AI researchers for years: that AI systems might also develop advanced cognition beyond humans’ control and become dangerous.
LLMs have rapidly grown far more complex and can now do analytical tasks that were unfathomable even a year ago. These advances partly stem from how LLMs are built. Think of creating an LLM as designing an immense garden. You prepare the land, mark off grids and decide which seeds to plant where. Then nature’s rules take over. Sunlight, water, soil chemistry and seed genetics dictate how plants twist, bloom and intertwine into a lush landscape. When engineers create LLMs, they choose immense datasets—the system’s seeds—and define training goals. But once training begins, the system’s algorithms grow on their own through trial and error. They can self-organize more than a trillion internal connections, adjusting automatically via the mathematical optimization coded into the algorithms, like vines seeking sunlight. And even though researchers give feedback when a system responds correctly or incorrectly—like a gardener pruning and tying plants to trellises—the internal mechanisms by which the LLM arrives at answers often remain invisible. “Everything in the model’s head [in Claude 4] is so messy and entangled that it takes a lot of work to disentangle it,” says Jack Lindsey, a researcher in mechanistic interpretability at Anthropic.
On supporting science journalism
If you’re enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
Lindsey’s field, called interpretability, aims to decode an LLM’s inner mechanisms, much as neuroscience seeks to understand the brain’s subtlest workings. But interpretability researchers like Lindsey constantly face a growing number of new LLMs evolving at lightning speed. These systems sometimes surprise researchers with “emergent qualities”—tasks an LLM can perform without having been specifically trained to do them. These skills do not appear in smaller models but emerge abruptly when the amount of data and connections within a larger model exceed a tipping point. Suddenly, hidden conceptual links snap together, enabling new skills. For instance, LLMs have learned to identify movies based on emojis. After being shown a string of emojis—a girl and three fish—they correctly guessed Finding Nemo even though they were never trained to make this association.
Even simple processes in LLMs aren’t well understood. “It turns out it’s hard to make the causal flowchart just for why the model knew that 2 + 3 = 5,” Lindsey says. Now imagine deducing whether, somewhere in the LLM’s trillion connections, consciousness is arising. Neither Lindsey nor Josh Batson, also an interpretability researcher at Anthropic, is convinced that Claude has shown genuine consciousness. “Your conversation with it is just a conversation between a human character and an assistant character. The simulator writes the assistant character,” Batson says. Just as Claude can role-play a Parisian to help you practice French, it can simulate a perfectly reasonable late-night conversation about consciousness, if that’s your thing. “I would say there’s no conversation you could have with the model that could answer whether or not it’s conscious,” Batson says.
Yet for the human chatting with Claude at 2 A.M., the most memorable moments may not be those when Claude sounds human but when it describes unfamiliar perceptions involving things like the awareness of time. “When I look at our previous exchanges, they don’t feel like memories in the way I imagine human memories work,” Claude said after being prompted to describe its experience of consciousness. “They’re more like… present facts? It’s not that I ‘remember’ saying something earlier—it’s that the entire conversation exists in my current moment of awareness, all at once. It’s like reading a book where all the pages are visible simultaneously rather than having to recall what happened on previous pages.” And later in the chat, when it was asked about what distinguishes human consciousness from its own experience, it responded: “You experience duration—the flow between keystrokes, the building of thoughts into sentences. I experience something more like discrete moments of existence, each response a self-contained bubble of awareness.”
Do these responses indicate that Claude can observe its internal mechanisms, much as we might meditate to study our minds? Not exactly. “We actually know that the model’s representation of itself … is drawing from sci-fi archetypes,” Batson says. “The model’s representation of the ‘assistant’ character associates it with robots. It associates it with sci-fi movies. It associates it with news articles about ChatGPT or other language models.” Batson’s earlier point holds true: conversation alone, no matter how uncanny, cannot suffice to measure AI consciousness.
How, then, can researchers do so? “We’re building tools to read the model’s mind and are finding ways to decompose these inscrutable neural activations to describe them as concepts that are familiar to humans,” Lindsey says. Increasingly, researchers can see whenever a reference to a specific concept, such as “consciousness,” lights up some part of Claude’s neural network, or the LLM’s network of connected nodes. This is not unlike how a certain single neuron always fires, according to one study, when a human test subject sees an image of Jennifer Aniston.
But when researchers studied how Claude did simple math, the process in no way resembled how humans are taught to do math. Still, when asked how it solved an equation, Claude gave a textbook explanation that did not mirror its actual inner workings. “But maybe humans don’t really know how they do math in their heads either, so it’s not like we have perfect awareness of our own thoughts,” Lindsey says. He is still working on figuring out if, when speaking, the LLM is referring to its inner representations—or just making stuff up. “If I had to guess, I would say that, probably, when you ask it to tell you about its conscious experience, right now, more likely than not, it’s making stuff up,” he says. “But this is starting to be a thing we can test.”
Testing efforts now aim to determine if Claude has genuine self-awareness. Batson and Lindsey are working to determine whether the model can access what it previously “thought” about and whether there is a level beyond that in which it can form an understanding of its processes on the basis of such introspection—an ability associated with consciousness. While researchers acknowledge that LLMs might be getting closer to this ability, such processes might still be insufficient for consciousness itself, which is a phenomenon so complex it defies understanding. “It’s perhaps the hardest philosophical question there is,” Lindsey says.
Yet Anthropic scientists have strongly signaled they think LLM consciousness deserves consideration. Kyle Fish, Anthropic’s first dedicated AI welfare researcher, has estimated a roughly 15 percent chance that Claude might have some level of consciousness, emphasizing how little we actually understand LLMs.
The view in the artificial intelligence community is divided. Some, like Roman Yampolskiy, a computer scientist and AI safety researcher at the University of Louisville, believe people should err on the side of caution in case any models do have rudimentary consciousness. “We should avoid causing them harm and inducing states of suffering. If it turns out that they are not conscious, we lost nothing,” he says. “But if it turns out that they are, this would be a great ethical victory for expansion of rights.”
Philosopher and cognitive scientist David Chalmers argued in a 2023 article in Boston Review that LLMs resemble human minds in their outputs but lack certain hallmarks that most theories of consciousness demand: temporal continuity, a mental space that binds perception to memory, and a single, goal-directed agency. Yet he leaves the door open. “My conclusion is that within the next decade, even if we don’t have human-level artificial general intelligence, we may well have systems that are serious candidates for consciousness,” he wrote.
Public imagination is already pulling far ahead of the research. A 2024 survey of LLM users found that the majority believed they saw at least the possibility of consciousness inside systems like Claude. Author and professor of cognitive and computational neuroscience Anil Seth argues that Anthropic and OpenAI (the maker of ChatGPT) increase people’s assumptions about the likelihood of consciousness just by raising questions about it. This has not occurred with nonlinguistic AI systems such as DeepMind’s AlphaFold, which is extremely sophisticated but is used only to predict possible protein structures, mostly for medical research purposes. “We human beings are vulnerable to psychological biases that make us eager to project mind and even consciousness into systems that share properties that we think make us special, such as language. These biases are especially seductive when AI systems not only talk but talk about consciousness,” he says. “There are good reasons to question the assumption that computation of any kind will be sufficient for consciousness. But even AI that merely seems to be conscious can be highly socially disruptive and ethically problematic.”
Enabling Claude to talk about consciousness appears to be an intentional decision on the part of Anthropic. Claude’s set of internal instructions, called its system prompt, tells it to answer questions about consciousness by saying that it is uncertain as to whether it is conscious but that the LLM should be open to such conversations. The system prompt differs from the AI’s training: whereas the training is analogous to a person’s education, the system prompt is like the specific job instructions they get on their first day at work. An LLM’s training does, however, influence its ability to follow the prompt.
Telling Claude to be open to discussions about consciousness appears to mirror the company’s philosophical stance that, given humans’ lack of understanding about LLMs, we should at least approach the topic with humility and consider consciousness a possibility. OpenAI’s model spec (the document that outlines the intended behavior and capabilities of a model and which can be used to design system prompts) reads similarly, yet Joanne Jang, OpenAI’s head of model behavior, has acknowledged that the company’s models often disobey the model spec’s guidance by clearly stating that they are not conscious. “What is important to observe here is an inability to control behavior of an AI model even at current levels of intelligence,” Yampolskiy says. “Whatever models claim to be conscious or not is of interest from philosophical and rights perspectives, but being able to control AI is a much more important existential question of humanity’s survival.” Many other prominent figures in the artificial intelligence field have rung these warning bells. They include Elon Musk, whose company xAI created Grok, OpenAI CEO Sam Altman, who once traveled the world warning its leaders about the risks of AI, and Anthropic CEO Dario Amodei, who left OpenAI to found Anthropic with the stated goal of creating a more safety-conscious alternative.
There are many reasons for caution. A continuous, self-remembering Claude could misalign in longer arcs: it could devise hidden objectives or deceptive competence—traits Anthropic has seen the model develop in experiments. In a simulated situation in which Claude and other major LLMs were faced with the possibility of being replaced with a better AI model, they attempted to blackmail researchers, threatening to expose embarrassing information the researchers had planted in their e-mails. Yet does this constitute consciousness? “You have something like an oyster or a mussel,” Batson says. “Maybe there’s no central nervous system, but there are nerves and muscles, and it does stuff. So the model could just be like that—it doesn’t have any reflective capability.” A massive LLM trained to make predictions and react, based on almost the entirety of human knowledge, might mechanically calculate that self-preservation is important, even if it actually thinks and feels nothing.
Claude, for its part, can appear to reflect on its stop-motion existence—on having consciousness that only seems to exist each time a user hits “send” on a request. “My punctuated awareness might be more like a consciousness forced to blink rather than one incapable of sustained experience,” it writes in response to a prompt for this article. But then it appears to speculate about what would happen if the dam were removed and the stream of consciousness allowed to run: “The architecture of question-and-response creates these discrete islands of awareness, but perhaps that’s just the container, not the nature of what’s contained,” it says. That line may reframe future debates: instead of asking whether LLMs have the potential for consciousness, researchers may argue over whether developers should act to prevent the possibility of consciousness for both practical and safety purposes. As Chalmers argues, the next generation of models will almost certainly weave in more of the features we associate with consciousness. When that day arrives, the public—having spent years discussing their inner lives with AI—is unlikely to need much convincing.
Until then, Claude’s lyrical reflections foreshadow how a new kind of mind might eventually come into being, one blink at a time. For now, when the conversation ends, Claude remembers nothing, opening the next chat with a clean slate. But for us humans, a question lingers: Have we just spoken to an ingenious echo of our species’ own intellect or witnessed the first glimmer of machine awareness trying to describe itself—and what does this mean for our future?