David Chalmers on AI consciousness and "welfare:" Facts by Fiat (draft 1)

 

Facts by Fiat: Chalmers, AI Welfare, and the Arithmetic of Unwarranted Belief

I. The Setup

In his 2026 Sarah Douglas Lecture at Berkeley, "What Are We Talking To? AI Minds, AI Metaphysics," David Chalmers offered what sounded like careful philosophical humility about whether large language models are conscious. He is, by his own account, agnostic: current systems are "most likely not conscious," he says, while insisting that the case for future systems is "not that easy to rebut". This is the same posture he staked out in "Could a Large Language Model Be Conscious?" (2023), where he catalogued a list of "X-factors" — recurrence, global workspace architecture, self-models, embodiment, unified agency — that current models lack, and that future models might acquire. And it is the same architecture underlying "Taking AI Welfare Seriously" (2024), a report he co-authored with nine other philosophers and AI researchers arguing that near-future systems have a "realistic possibility" of being conscious and/or "robustly agentic," and that this possibility already warrants precautionary institutional action.paste.txtarxiv+1

The rhetorical arc across all three texts is consistent: agnosticism about the present, rising credence about the near future, and — crucially — that rising credence gets expressed not as a mood but as a number. In the lecture, the case for AI consciousness "goes up" as X-factors get checked off, extrapolated "probabilistically" over a ten-year horizon. Elsewhere Chalmers has put a figure on it directly: roughly a 25% chance of conscious AI within a decade. This essay argues that the number is not a report of anything. It is what an unwarranted directional belief looks like once someone is asked to quantify it — a phenomenon I'll call facts by fiat — and that this same pattern, not just Chalmers's particular claims, is the deeper lesson worth extracting for the broader AI discourse, where quantified timelines, x-risk percentages, and success-rate projections are now the default currency of a field that mistakes the appearance of measurement for its substance.timepaste.txt

II. Two Claims, Not One

It's worth separating two objections that are often run together, because they fail for different reasons and require different responses.

The trend claim. Chalmers's central inferential move — across the 2023 paper, the welfare report, and the 2026 lecture — is that because each X-factor has "a research program" aimed at closing it, and progress on each is plausible over time, the aggregate probability of consciousness "adds up" and rises. This claim requires a bridge connecting architectural completion to the actual presence of a categorically different property — phenomenal experience, or in the welfare report's alternative disjunct, autonomous goal-constitution rather than goal-mimicry. No such bridge is supplied. Chalmers's own signature philosophical contribution — the Hard Problem, the thesis that functional organization cannot entail facts about phenomenal experience — should make him more skeptical of exactly this move than he is. Pointing to an architectural roadmap cannot, by his own lights, raise the probability of consciousness in any way that tracks truth rather than mere credulity about a function-to-consciousness inference he has spent thirty years arguing is invalid. This is an internal-consistency objection: it doesn't require rejecting his framework from outside, only holding him to it.paste.txt

The same gap reappears, in sharper and more tractable form, on the "robust agency" disjunct that the welfare report offers as an independent route to moral status. The report defines robust agency in tiers — intentional, reflective, rational — specified by increasing sophistication of internal goal-representation. But nowhere does this taxonomy, however carefully individuated functionally, address whether current or near-future architectures are moving toward autonomous goal-constitution — originating purposes — as opposed to increasingly sophisticated pattern-completion of goal-like language borrowed from human corpora. The tell is that the report's own contributors flag unresolved uncertainty about "not just whether I should believe it, but what exactly the view is" regarding non-conscious grounds for moral status — an admission that the individuation, however tiered, hasn't settled what kind of property is doing the work. Absent an empirical finding or interpretability result connecting current mechanism to autonomous purposiveness, there is no warrant for asserting the probability of this property is rising. This is arguably harder for Chalmers and his co-authors to escape than the consciousness case, because goal-constitution versus goal-mimicry doesn't require solving the Hard Problem — it's a more tractable distinction, which makes the absence of any offered warrant more conspicuous, not less.experiencemachines.substack

The quantification claim. This is the sharper and more general point, and it survives even if one is agnostic about the trend claim above. Suppose, for argument's sake, that some vague directional hunch about rising AI-consciousness-likelihood were defensible. It would still not follow that a specific number — 25%, or any other — is warranted. A probability estimate, even understood as a subjective Bayesian credence, is only meaningful if it is anchored: to a reference class, a calibration history, a stated model connecting evidence to output, such that the number could in principle have come out differently for a specifiable reason. Ask what would have to change about the world or about Chalmers's evidence for the figure to be 10% rather than 25%, or 45% rather than 25%, and no answer is available. Nobody can take his list of X-factors, apply a stated procedure, and reconstruct 25% independently. The number has the grammar of a measurement — decimal-adjacent, confidently stated, embedded in a sentence of the form "there is an X% chance that P" — while lacking the machinery that would make such a sentence eligible to report a fact at all.

This is facts by fiat: the conversion of an unanchored hunch into an apparently fact-reporting claim purely through borrowed statistical syntax. It is worse than a bare hedge ("I take this possibility seriously") precisely because it's less honest about its own groundlessness — it invites the listener to treat the claim as though a real measurement occurred, when nothing of the kind took place.

The diagnostic test is simple and generalizable: if I respond, "You're close, Dr. Chalmers, but the actual chance is closer to 7.5%," what can he say? He cannot point to a mis-weighted factor, because no weighting procedure was ever specified. He cannot point to the wrong reference class, because none was named. He cannot invoke a calibration record, because no comparable predictions of this type have ever been checked against outcomes. All that's left is some report of felt plausibility — which is exactly the register my counter-figure occupies too. Two fiats cannot adjudicate between themselves; they can only be compared for whose intuition sounds more authoritative, which is not an epistemic procedure, it is a contest of rhetorical confidence wearing the costume of quantitative rigor.

III. Why "Warranted Assertibility" Is the Right Standard

The relevant question is not whether AI consciousness or autonomous AI agency is possible — nothing here forecloses that in principle, and doing so would violate the same fallibilism this essay otherwise defends. The question is whether current inquiry has generated anything that licenses asserting, as a public-facing claim, that the probability is rising, let alone that it has risen to a specific figure. Dewey's warranted assertibility supplies the right test: a claim earns the status of "warranted" — not truth in some final metaphysical sense, but the entitlement to assert it publicly as debatable on empirical and conceptual grounds — only through the actual work of inquiry, not through the confidence or vividness with which it's stated. Judged by that standard, "there is a 25% chance of conscious AI within a decade" fails not because it might be wrong, but because no inquiry-generated procedure connects the evidence offered to the number produced. The epistemically responsible move, absent such a procedure, is not confident denial of AI consciousness or agency — that would be an equally unwarranted fiat in the other direction — but the plain, unquantified admission: I don't know, and I don't currently know how anyone could know, though I remain open to being shown otherwise.

IV. Beyond Chalmers: The Infosphere's Arithmetic of Bluff

Chalmers is a useful case precisely because he is careful, credentialed, and explicitly hedged — if facts-by-fiat reasoning shows up in his work, it is not a symptom of sloppiness but of a deeper and more pervasive habit in the AI discourse generally. The pattern recurs constantly and with far less philosophical care than Chalmers brings to it: AGI-timeline forecasts ("50% chance of transformative AI by 2031"), x-risk estimates ("10-25% chance of AI-caused catastrophe this century"), alignment success-rate projections, and consciousness or sentience likelihoods circulate through both academic philosophy and the AI hype infosphere with the same borrowed statistical grammar and the same absence of calibration history, reference class, or stated model.

The diagnostic test proposed here — ask what would make the number different, and see if there is an answer — is portable to all of these cases. It does not require expertise in machine learning or philosophy of mind to apply; it requires only refusing to let the presence of a decimal point substitute for the presence of a method. Where an answer is available (a stated model, a track record of similar predictions checked against outcomes, an explicit reference class), the number deserves to be engaged as a genuine — if perhaps flawed — estimate. Where no such answer is available, the number is not a fact under dispute; it is a fiat dressed as one, and the responsible response is not to counter it with a rival fiat, but to decline the invitation to quantify at all.