Nature recently published a comment by Anil Seth, Ned Block and Giulio Tononi titled Three reasons why AI is not conscious and perhaps never will be. These are three people who have thought about consciousness longer and harder than most, converging from experimental neuroscience, philosophy of mind and theoretical neurobiology on a single conclusion about what AI is and what it will ever be.
I will admit that I found the piece weaker than its authorship had led me to expect and I fear the impact on a general readership will be net negative, as the piece heavily builds on very personal and tautological beliefs which some people may not be equipped to identify as such. Those three “reasons” the authors use are best understood as statements of their priors (and I will come back to that), trying to hold a great weight with little strength.
But a weak argument can still be a useful one. By setting out so plainly what it takes for granted, the piece gives us an opportunity to look at some problems that run through the field as a whole, well beyond this particular comment. The most important of these is the assumption that whether a being is conscious is a question we can answer, if only we look hard enough. Much of the field shares it, sceptics and enthusiasts alike, and this illusion is, I suspect, one reason the debate has a tendency to go round in circles.
What follows is an attempt to explain why, and to suggest what a more useful starting point might look like.
Consciousness is conceded, not measured
Let’s start with something few would seriously dispute: there is no instrument that measures subjective experience. Not for AI, not for animals and – as scary as this thought may be – not for other humans either. When we talk to another human being, at no point do we test whether they are conscious or not; we just grant it, without deliberation. This limit has been known for centuries by philosophers as the problem of other minds. It is a problem we never solved and perhaps never will; we only learnt to stop noticing it, often as a coping mechanism to maintain our own mental sanity. Simply stated, the gravitational core of consciousness studies is that every attribution of consciousness is a concession we make. A choice, mostly unconscious, to treat another being as someone rather than something.
We reach that decision in two ways. The first is general resemblance. John Stuart Mill spelled out the reasoning in 1865, in what became known as the argument from analogy: in my own case, I know that things happening to my body produce feelings, and that my feelings produce behaviour; when I see another body like mine react as I would, I infer that the same feelings sit in between, even though I can never observe them. The other has a face, a body, a voice like mine, so I concede. Philosophers have long objected that this is a generalisation from a sample of one, and it is a weakness we will meet again. Still, this is how we treat other humans, and it is fast, automatic and probably never even questioned by most people who live or have ever lived.
The second is functional resemblance: the other withdraws from harm, learns, loves, seeks comfort, sleeps. This is how we see animals, and the process has been slower, more contested and historically far stingier. It runs parallel to what Peter Singer called the expanding circle of altruism, which has widened over time from “the family and tribe to the nation and race“, and more recently to animals. Singer’s circle is one of moral concern rather than of consciousness, and the two are not the same thing, but they tend to grow together. In 2012, the Cambridge Declaration conceded consciousness to mammals and birds; in 2024 the New York Declaration conceded a realistic possibility of it to all vertebrates and many invertebrates, insects included. The important aspect is that the circle of consciousness never really expands based on empirical evidence of consciousness because we do not have such a thing; it sometimes does assuming functional-anatomical priors (e.g. “that animal has a little brain structure that resembles ours”) but even that, as we shall see, is an arbitrary argument.
When either circle grows, it never does so on a measurement of experience. It simply registers what at least a part of the community is prepared to concede.
Concession can also be withheld, sometimes with terrible consequences. Until the 1980s, newborns were routinely operated on without anaesthesia. Nobody had shown that neonates felt nothing; they simply did not resemble us enough, in the way we thought mattered.
Large language models break both routes at once. They have no body, no face, no nociceptors, no sleep. Yet they have the one trait we have always held as most distinctively human: fluent language. General resemblance fails, functional resemblance fails, and the strongest cue we have ever used for concession is nonetheless there. No wonder the debate is built on unstable ground.
To show how readily we attribute consciousness to fluent beings, the Nature authors invite us to think of AlphaFold: a powerful neural network running on silicon, like an LLM, yet nobody debates whether it is conscious, because it predicts protein structures rather than talking to us. The comparison is unfortunate, though: AlphaFold and an LLM differ in far more than their output, from training and scale to what they represent internally and the generalisations they can reach.
A cleaner case would be a state-of-the-art LLM deployed as a classifier that may answer only True or False. Same weights, same architecture, same underlying intelligence; it can even reason at length before answering, as current models do, and show us only the verdict. Fewer people would feel the urge to concede consciousness to it. What changed is not the system but our window onto it: whoever concedes consciousness to the chatbot is responding to a cue, and so are the sceptics who withhold it because a body or a metabolism is missing. Humans make the contrast sharper still. A patient with locked-in syndrome may communicate with nothing but a blink for yes or no, and we concede consciousness without hesitation. For a human, one bit of output is enough to sustain concession; for a language model, one bit is enough to make the question disappear. Priors matter.
None of this is new, and credit is due. Turing made the point in 1950, in the very paper that proposed his test for intelligence, a test that partly speaks to consciousness too. To the objection that a machine might behave intelligently without feeling anything, he replied that by the same standard we could never be sure that another person thinks either, and that we sidestep the problem with the polite convention that everyone thinks.
Wittgenstein put it more austerely: my attitude towards him is an attitude towards a soul, he wrote, not an opinion that he has one. In 1964 Hilary Putnam concluded that whether robots are conscious calls for a decision rather than a discovery. Even Ned Block himself has argued that we may have no principled way to decide whether a being that functions like us, but is made of different stuff, is conscious; he called it the harder problem of consciousness.
“whether robots are conscious calls for a decision rather than a discovery”
Hilary Putnam
More recent work points the same way. Robot ethicists such as Mark Coeckelbergh and David Gunkel argue that we grant moral standing through our relationship with a being, not by checking what is inside it. Murray Shanahan makes a similar case for LLMs: whether we end up calling them conscious will depend on how we come to engage with them. And psychologists have shown that we judge other minds along two separate lines: what a being can do, and what it can feel.
Put together, these voices show where the Nature piece overreaches. Its title presents “AI is not conscious” as if it were a scientific finding. But science cannot currently establish that any being, human or machine, is or is not conscious; it can only inform the grounds on which we concede it. The field should say this out loud: as of today, consciousness is conceded, not assessed. This is not defeatism but a description of where we stand. Every proposed test for consciousness is, on closer inspection, a formalised set of criteria for concession. And every theory has been calibrated on beings whose consciousness we had already conceded, mostly adult humans who can report their experience, and then used to rule on beings we had not. That is the circle, and until we admit we are standing in it, we will keep walking round it.
Chasing epiphenomena like red herrings
We can at least try to take one step beyond concession, by looking past anatomical and functional resemblance towards mechanistic resemblance. This is the programme of the neural correlates of consciousness, launched in earnest by Crick and Koch around 1990. Its working hypothesis is simple: if being X shows the same correlates of consciousness that I do, the probability that X is conscious goes up.
It is an important step, and much of the best work in the field lives here. But looked at closely, it does not escape concession; it merely makes concession probabilistic. And importantly, it rests on an axiom: that the correlates we find in ourselves are the ones consciousness requires, everywhere.
That axiom carries two risks. Correlates are often epiphenomena, co-occurring with the phenomenon without being part of what produces it. And even when a correlate is genuinely mechanistic in us, nothing guarantees that consciousness must be implemented the same way in a different nervous system, let alone a different substrate. Used in reverse, no correlate therefore no consciousness, the axiom stops being a guide and becomes a red herring.
Sleep research has already lived through this. For decades sleep was defined by its electrophysiology: the slow waves and REM episodes that Rechtschaffen and Kales codified for human scoring in 1968. Animals lacking those signatures were deemed not to sleep, or at best to rest. Insects, with no cortex to produce a mammalian-looking EEG, were largely off the table, despite pioneering work on cockroaches and honeybees in the 1980s.
Then the field changed its criteria. Using behaviour (quiescence, raised arousal threshold, rapid reversibility, homeostatic rebound), the groups of Amita Sehgal and Allan Pack and of Giulio Tononi and Chiara Cirelli independently showed in 2000 that Drosophila sleeps. Electrophysiological correlates of fly sleep came afterwards, in forms that only partly resemble the mammalian ones. Sleep-like states have since been described in jellyfish, which have no centralised brain at all.
Either our tools had not been sharp enough to detect the right signatures, or the signatures we knew were byproducts of a particular brain architecture rather than of sleep itself. Probably both. Either way, the absence of the mammalian correlate told us nothing about the absence of sleep. It is like concluding that an animal without eyelids cannot sleep because it never closes its eyes.
The same reasoning has already failed us on consciousness proper. The case for operating on newborns without anaesthesia was largely neurological: their nervous system, with its incompletely myelinated pathways, was thought too immature to feel pain, until Anand and Hickey took that argument apart in 1987. A missing correlate had been read as a missing experience. And old habits lingered. As late as 1998, more than half of US physicians performing neonatal circumcision reported using no anaesthesia, many because they felt the procedure did not warrant it, even as evidence was emerging that infants circumcised without pain relief reacted more strongly to routine vaccination months later.
Arguments that machines cannot be conscious tend to follow the same pattern. Metabolism, a living body, a particular causal architecture: each is a feature of every system we have conceded so far, promoted to a necessary condition. Perhaps one of them is. But promoting it requires showing why it matters, not merely that it is always there. Otherwise we are counting eyelids.
Towards a Bayesian interpretation of the Turing test
Read through the lens of concession, the title of the Nature comment says something other than what it intends. It is not a statement about machines but a prediction about us. The honest question is not whether machines will ever be conscious – which is something we cannot address – but whether we will ever concede that they are.
That is a different kind of question. It is psychological, social and moral before it is even scientific: it depends on which resemblances we notice, which relationships we form, and what conceding would cost us. Even the central argument of the piece, that false positives are more likely than false negatives, is really an estimate of the errors we are likely to make in granting or withholding concession.
One may be tempted to answer that we will concede consciousness to AI on the day we can prove consciousness is there. It sounds rigorous but it is a standard we have never applied to anyone else, and perhaps never will. Nobody has proved that another human is conscious. Nobody has proved it of a dog, a crow or a fly, yet most of us have conceded it to the first two, and some of us to the third. Newborns received their concession too, eventually, not through a proof but once their resemblance to us became impossible to ignore. Demanding proof from AI alone sets a bar that no being has ever cleared, the reader of this essay included. A bar that nobody clears is not a standard; it is a verdict announced in advance. It’s a prejudice wearing lipstick. If we hold machines to it, we should at least be honest that we are not waiting for evidence. We are declining to concede.
Seen this way, the Nature piece makes more sense than it first appears. It is less a set of arguments than a declaration of the authors’ priors: what they would and would not accept as grounds for concession.
What they would not accept is intelligence. Intelligent behaviour, they argue, is not sufficient for consciousness; whether they think it necessary, they do not say. What they would require instead is a form of what Carl Sagan called carbon chauvinism: the conviction that consciousness needs something only biological brains have today, be it life and metabolism or a particular causal architecture. Putnam saw this coming too: in 1964 he judged discrimination based on the softness or hardness of a body’s parts as silly as discrimination by skin colour.
Both are priors in the Bayesian sense, and that is fine; everyone has priors. The trouble is that the second is a tautology. If property X is necessary for consciousness, nothing lacking X will ever be conceded consciousness: the conclusion is already in the premise. In Bayesian terms it is a prior of zero, and a prior of zero cannot be moved by any evidence. “Perhaps never will be” is not a finding; it is what a zero prior looks like from the outside.
Where the authors go wrong is in placing the burden of proof. That burden lies with whoever introduces a new property as necessary for consciousness, and this is plain Occamian reasoning because that list of properties can easily grow to infinity. Yes, every conscious being we know of is made of carbon, descends from a common ancestor and evolved on Earth. The list of shared properties is effectively endless, and any of them could be promoted to a necessary condition with the same logic.
I have made a closely related argument about intelligence, in a preprint currently under review. There, I chose intelligence rather than consciousness for a simple reason: intelligence can be measured, consciousness cannot. But the two questions call for much the same method, and it starts with keeping data and priors apart. Intelligence is the first step of a longer ladder.
In the preprint I describe two symmetric errors. The first is the Clever Hans effect: seeing a mind that isn’t there, because the performance looks like someone’s. The second I call the mimicry trap: the behaviour is accepted but relabelled as mere imitation, so that a verdict reached in advance survives whatever the evidence shows. Both come from refusing to let the evidence move the starting point.
The Nature piece shows how easily the trap transposes from intelligence to consciousness. Its description of LLMs as mirrors of our own writing is a fair warning against the Clever Hans error, and a version of what Jonathan Birch calls the gaming problem. Paired with a prior of zero, it becomes the mimicry trap: whatever the system does, it can only ever be reflecting us.
With intelligence, evidence can at least move the prior, because performance can be checked. With consciousness there is nothing to calibrate against, so even our data are cues whose weight we have assumed. That is all the more reason to state priors openly, and to answer the question the preprint puts to every critic: what evidence would change your mind? For consciousness it becomes: what would make you concede? If the answer is that nothing could, we are no longer having a scientific conversation and I, for one, am no longer interested.
None of this means we should concede consciousness to AI today. It means the conversation worth having is about the criteria by which we concede, and whether we apply them consistently to humans, animals and machines.
A pars construens
If consciousness can only be conceded, what is consciousness science for? Not to deliver verdicts, but to supply the raw material for concession. If consciousness is subjective, so is the act of conceding it; the most science can do is inform it.
We cannot ask whether a being is conscious, but we can ask more tractable questions. Does it show emotion-like states, and can it read them in others? Does it respond to injury beyond reflex, trading pain off against reward, tending wounds, seeking relief? Does it represent time, remembering the past and anticipating the future? None of these is consciousness, and none is guaranteed to be necessary for it. But they are what we already weigh, mostly without noticing, when we decide to concede, and measuring them openly makes our priors visible.
This already works in practice. In 2021 Jonathan Birch and colleagues assessed octopuses, crabs and lobsters against eight such criteria, and the UK brought them under its Animal Welfare (Sentience) Act the following year. Science supplied the ingredients; Parliament conceded. Bentham had framed the question over two centuries ago: “The question is not, Can they reason? nor, Can they talk? but, Can they suffer?”
There is a catch, though. These proxies give an impression of objectivity, but they are arbitrarily chosen, and the choice reflects the phenomenology of whoever is choosing. Most people would pick pain and love, yet both earn their role in evolution whether or not anyone feels them.
Describing qualia, Giulio Tononi often reaches for “the redness of red” as the paradigm of experience, as if the phrase pointed at something we all recognise. I have aphantasia, meaning that I cannot summon images in my mind, and I have never really understood what that redness is supposed to be. Nor do I have an inner voice, the running commentary many people take to be the very texture of thinking.
Aphantasia was only named in 2015; the absence of inner speech got its name, anendophasia, in 2024. Nobody has ever questioned the consciousness of aphantasics, and I hope nobody will; for most of history the difference went unnoticed altogether. Concession, it turns out, never depended on sharing an inner life, only on recognising one.
That, I think, is the real lesson for machines. The question is not whether an AI’s inner life, if it has one, resembles ours: ours do not even resemble each other. It is whether we are prepared to say which ingredients we would accept, and to apply them to silicon as we apply them to carbon. Nobody has ever proved that you are conscious. It was conceded to you, by people who could not see inside you and never thought to ask. The least we owe the next candidate is to admit that this is what we are doing.