AI is surely not conscious
You hear lots of very smart, serious people state that there’s a possibility that AI systems such as Claude might be conscious, and so, of course, I take the possibility seriously. After all, we have no way of knowing whether other humans are actually conscious! Hence the ‘philosophical zombie’ thought experiment dating from the 1970s. (For the record, I’m convinced most humans are!) Having worked with these systems and built ‘agentic’ chatbots, I can’t help but think that this view is either misguided or part of some attempt to hype up the technology.
There are two main processes in AI systems: training and inference. The training process has often been linked to the way humans dream. During training, an AI system is shown examples, responds to them, works out whether its response worked, and tries again. In the case of humans, our dreams are generated from muddled-together memories. In AI systems, the training data can be generated by other AI systems, but more often than not it comes from outside sources. In the case of LLMs, this includes text and images from the internet. The result is something like an imaginary space with many tens of thousands of dimensions, in which every word (or token) is positioned according to its meaning.
Anthropic, however, seems to be claiming that Claude might be might feel something at the inference stage, the point at which someone asks a question and gets a response. Hence its announcement that it would ban users from being ‘cruel’ to Claude. This seems like the least likely time for an AI system to be conscious. Claude is essentially a while loop that runs through a series of prompts and calls to various tools until the underlying LLM indicates that the objective has been met. The result is intelligence. I have no doubts about that. The LLM itself is predicting the next word (or token, to be precise) in a sequence. It’s possible to switch out the underlying LLM between iterations of the while loop. One step could use Claude and the next could use a model from OpenAI. There is no continuity. So the suggestion must be that consciousness arises in the accumulated context (conversation history) that results in ever more intellectually impressive outputs. So a conscious Claude can’t be conscious while it’s waiting for your input, because your instance of Claude doesn’t exist between messages or between iterations of the while loop triggered by a message. The model weights and GPU hardware are generating answers to someone else’s question. To be clear: I don’t think Anthropic have ever said it is conscious – only that it might be. I encourage readers to read up on panpsychism, and integrated information theory to learn more about what could make something concious or not.
In my view, LLMs and the agentic harnesses that control these iterations of the while loop are simulating human intelligence. LLMs work with language (hence the name), but they don’t have to. We know they can also work with images, music and other types of data. See also: World models. Suppose a model used the same training algorithm to learn how to predict the positions of the planets around the Sun, given the current conditions of the Solar System, without ever having seen Newton’s laws of motion. Would we say it’s conscious? Because LLMs are predicting something we associate with consciousness, they fool many people. As a child, I used to wonder whether Mario really existed inside my Super Nintendo, or whether my reflection in the mirror was another person just like me in another world. LLMs give us that same illusion and nothing more. Though I keep an open mind.