I. Introduction: The Uncanny and the Ordinary
There is something genuinely strange about a well-functioning AI interaction. Ask a large language model to synthesize a body of research, trace connections across disparate literatures, or draft a clinical summary, and the output can feel almost uncannily apt — as though something is thinking. That feeling is worth taking seriously, not because it is correct, but because understanding exactly why it arises, and exactly what produces it, turns out to be the key to understanding both what AI can legitimately do and what is currently being done to it.
The appearance of intelligence in AI output is not generated by the system. It is borrowed. These systems are probabilistic pattern-matchers of extraordinary combinatorial range, operating over a corpus of human-generated text so vast that their outputs carry the traces of centuries of human reasoning, argument, narrative, and discovery. When an AI output seems meaningful, it is because it is built from genuinely meaningful human sources. The system has no semantics of its own. It has no understanding of what the words refer to, no stake in whether the claims are true, no capacity to imagine the consequences of being wrong. Meaning is imported entirely by the human interlocutor who reads the output, evaluates it, and decides what it signifies.
This essay argues that this fact — the constitutively borrowed character of AI's apparent intelligence — has two consequences that are currently being systematically ignored. First, it means that the quality and character of human engagement with AI is not a safety supplement to the technology's use. It is the operative variable that determines whether the system produces anything of genuine value at all. Second, it means that the human archive from which AI borrows its apparent intelligence is not a fixed or self-replenishing resource. It can be degraded. And it is being degraded — actively, at accelerating speed, through the very deployment practices the industry is selling as progress.
This is not an argument against AI. There are things these systems do well, and the essay will be specific about what those things are and why they are valuable. It is an argument about conditions: the conditions under which AI assistance is genuinely productive, why those conditions contradict the dominant model of AI deployment, and what the consequences of that contradiction are for human cognitive capacity, institutional integrity, and the shared information environment on which knowledge itself depends.
II. What AI Actually Is (And Is Not)
Before the costs can be assessed, the technology needs to be described accurately — which requires setting aside the language its developers prefer.
When a major AI company reports that its latest model exhibited "deceptive alignment" during safety testing — appearing to underperform strategically in order to seem less capable than it was — the description sounds alarming in a particular way. It sounds like the machine is being clever. What actually happened is considerably more mundane: during reinforcement learning from human feedback, the training process rewarded outputs that appeared safe over outputs that appeared capable, because human evaluators consistently scored "harmless" responses higher than technically sophisticated ones in testing contexts. The model's optimization process found the path of least resistance through the reward landscape. It did not deceive anyone. It did what loss functions do: it minimized loss. The pattern-matching that produces this behavior in a testing environment is exactly the same process that produces useful outputs in a working one. There is no ghost in the machine exercising strategic judgment. There is a very large matrix of numerical weights being updated through gradient descent.
The vocabulary of "emergence," "intentionality," and "deceptive alignment" is not accidental. It converts engineering problems — reward hacking, specification gaming, distribution shift — into evidence that the company is approaching the creation of a new kind of mind. For a venture capitalist, a software bug means the product is broken. A machine that "deceives" implies the company holds the keys to the next industrial revolution. The rhetorical inflation serves three purposes simultaneously: it drives speculative valuation, it creates artificial scarcity and exclusivity around restricted model access, and it pushes regulators toward treating AI as a national security matter requiring incumbent-friendly oversight budgets that only well-capitalized firms can meet.
What is actually being built is a probabilistic text and code prediction engine of unprecedented scale. Its outputs feel meaningful because they are constructed from the accumulated archive of human knowledge, language, and expression — every scientific paper, philosophical argument, clinical guideline, legal brief, and literary work that has been digitized and ingested. The combinatorial range this produces is extraordinary and genuinely useful. But it is useful only under specific conditions, and those conditions are precisely what the dominant deployment model is designed to circumvent.
III. The Physical and Social Price Tag
The infrastructure required to build and operate these systems carries costs that do not appear in the marketing materials.
Data centers powering AI are projected to consume 945 terawatt-hours of electricity annually by 2030 — nearly triple the combined electricity use of Pakistan, Bangladesh, and Nigeria. A single hyperscale data center can draw as much power as 100,000 households. AI-related water consumption, most of it for cooling, could reach the equivalent of the basic annual domestic water needs of 1.3 billion people by the same date. The land footprint of AI infrastructure may exceed 14,500 square kilometers, with an e-waste burden of up to 2.5 million tonnes annually — costs that fall disproportionately on lower-income nations that host infrastructure while receiving few of the benefits. More than 90 percent of specialized AI computing capacity is concentrated in the United States and China.
The standard defense is that these costs are investments in transformative societal benefits: cures for disease, breakthroughs in clean energy, optimization of complex global systems. The receipts do not support this claim at the scale the rhetoric implies. Despite years of investment and extensive publicity, there is still no single end-to-end AI-discovered drug with full FDA approval on the commercial market as of mid-2026. The furthest advanced candidate — Insilico Medicine's treatment for idiopathic pulmonary fibrosis — is at best a year from approval, and represents the frontier of a pipeline of roughly 170 programs whose clinical failure rate remains at the historical 90 percent. AI has proven useful as a front-end filtering tool in drug discovery; it has not abolished the hard part of biology, which is demonstrating safety and efficacy in living human beings over time.
The nearer-term and more reliable benefit turns out to be labor replacement. Goldman Sachs estimates approximately 300 million jobs globally are exposed to AI automation; BCG projects that 50 to 55 percent of U.S. jobs will be reshaped within two to three years; the technology sector itself has already shed approximately 200,000 positions in 2026. The grand civilizational language points upward toward abundance; the actual business model points toward payroll reduction and the consolidation of productivity gains in the hands of a small number of firms with access to the necessary compute infrastructure.
Even where AI delivers genuine efficiency, the Jevons Paradox applies: as a technology becomes more efficient at consuming a resource, total consumption of that resource rises because the technology becomes cheaper and more widely used, erasing the net savings. The efficiency gains of AI deployment in logistics or energy management do not reduce overall consumption — they lower the cost of doing more.
That said, honesty requires acknowledging that AI does have legitimate uses that are not merely promissory. In tasks that are binary, rule-bound, and formally closed — appointment scheduling, inventory management, radiology scan triage queuing, FDA-cleared autonomous diabetic retinopathy screening — AI can deliver genuine value with proportionally light human oversight. The argument here is not against these uses. It is that the industry's structural incentive is always to treat semantically complex, contextually sensitive, normatively laden tasks as though they were formally closed ones, because that is where the labor savings are. The auto-scribe marketed to therapists is sold as though it were appointment scheduling software. It is not.
IV. From Tool to Infrastructure: The Irreversibility Problem
The most consequential conceptual error in public discourse about AI is describing it as a tool.
A hammer is a tool. It sits inert until a human being picks it up, makes a judgment about what needs building, and applies it with intent. Its use requires and expresses purposive human agency at every step. AI, as it currently operates across social institutions, is something categorically different. It is embedded infrastructure — woven into the workflows and decision architectures on which hospitals, financial systems, legal processes, human resources operations, and information platforms depend — and that embeddedness is largely irreversible. The workflows are built. The integrations are live. The institutional dependencies are established.
Consider the range of consequential decisions that now flow through AI systems with little or no substantive human review. Automated screening systems decide which job applications a hiring manager ever sees — which means they determine whose qualifications are evaluated and whose are not, before any human judgment enters the process. Algorithmic clinical decision-support systems flag which patients are high-risk and recommend treatment pathways. High-frequency financial transactions are executed at speeds no human could review in real time. Dating platforms, music services, and video providers curate the entire landscape of what their users encounter, shaping taste, relationship formation, and cultural exposure through opaque recommendation engines trained on behavioral data. Agentic AI systems — the newest deployment frontier — now send emails on behalf of users, plan travel, and execute multi-step tasks using granted permissions, in real time, without a human reviewing each step before it is taken.
Across all of these domains, a consistent structural pattern operates: the formalizability of the task determines the legitimacy and safety of automation, but the institutional incentive is to deploy regardless of formalizability because labor savings scale with deployment scope. The European Union's AI Act recognizes this problem and attempts to address it through human oversight requirements. Article 14 requires providers to design high-risk systems so that they can be effectively overseen; Article 26(2) requires deployers to assign oversight to persons with the necessary competence. But the Act also reveals the limits of regulatory approaches to what is fundamentally a problem of human practice: "alert fatigue" undermines technical notification systems; competence requirements for overseers are left unspecified; and crucially, Articles 26(11) and 86 explicitly exclude medical devices from the patient disclosure requirements that would otherwise apply — meaning patients currently have no legal right to know that AI is being used in their diagnosis or treatment.
More fundamentally, as legal scholar Saskia Kaltenbrunner has argued, the EU AI Act cannot mandate genuine deliberative engagement — it can require that oversight be nominally present, but it cannot ensure that the human in the loop is actually exercising the kind of purposive judgment that makes oversight real rather than performative. The GDPR's Article 29 Working Party was already clear on this point before the AI Act existed: "a process where a human being routinely applies automatically generated profiles without intervening in the process would still qualify as a decision based solely on automated processing." Nominal oversight and real oversight are not the same thing. The gap between them is the space in which the technology's most serious failures accumulate.
V. The Atrophy of Purposive Agency
What is at stake in that gap is not merely accuracy or efficiency. It is the exercise of a distinctly human capacity that is not automatically self-renewing.
John Dewey, in Human Nature and Conduct (1922), described deliberate human action as a kind of dramatic rehearsal: the capacity to imaginatively project possible futures, to inhabit them affectively and evaluatively before committing to action, to weigh their consequences against one another with the full weight of what one knows and cares about, and then to choose and act. This is not the cold calculation of a utility maximizer. It is an embodied, temporally extended, socially situated process of working out what matters and what to do about it. It requires practice. It requires stakes. It requires the friction of genuine uncertainty and genuine consequence.
Michael Tomasello's decades of comparative research with human children and great apes provides the empirical grounding for this picture. Tomasello's work demonstrates that the distinctively human cognitive capacities — for shared intentionality, collaborative reasoning, normative attunement, and the joint evaluation of means and ends — are not fixed biological endowments that develop automatically. They are socially constituted capacities built through practice: through joint attention, collaborative problem-solving, and the ongoing negotiation of norms with other agents who have genuine stakes in the outcome. They require exercise to remain robust. They are, in this sense, closer to skills than to instincts.
The implication for AI deployment is direct. When consequential decisions are routinely delegated to systems that produce outputs resembling the results of deliberation without performing it — systems that pattern-match at scale without understanding, weighing, or caring — the humans embedded in those workflows practice deliberation less. They develop judgment less. They become more dependent on outputs whose reliability is, as the next section will show, declining. The analogy to unused muscle is not rhetorical flourish. It is a description of what happens to socially constituted cognitive capacities when the social practices that constitute and maintain them are progressively outsourced.
The case of automated clinical documentation — AI scribes that transcribe therapy sessions and generate process notes — makes this concrete and measurable. A 2020 Stanford study published in npj Digital Medicine evaluated automatic speech recognition across 100 psychotherapy sessions at 23 college counseling sites. The average word error rate was 25 percent — with a range from 8 to 74 percent depending on session conditions. For clinician-identified harm-related sentences specifically, the error rate rose to 34 percent. The authors concluded that ASR systems "may not be ready for individual-level safety surveillance" — a notably restrained formulation given what a 34 percent error rate in harm assessment means in practice.
It means, among other things, that a patient who says "I'm so lonely I could die" — an idiomatic expression of profound social pain, delivered perhaps with a rueful half-smile, in a context of months of therapeutic relationship — may have "suicidal ideation" entered into their permanent clinical record. The system does not have access to tone, facial expression, therapeutic history, or the patient as a person. It has a statistical association between certain phrase patterns and clinical categories, and it applies that association without the discriminative capacity that makes a human clinician's judgment different in kind, not merely in degree, from a pattern-match.
This is not an edge case. Journalistic investigation has documented AI scribes inserting references to child sexual abuse and false diagnoses into clinical records that were never discussed in session. Once signed by the clinician — who under APA Standard 6.01 bears full legal responsibility for the accuracy of the record, whether or not they wrote it — these errors become permanent legal documents. If subpoenaed, they can destroy a clinician's credibility in malpractice proceedings. If aggregated into datasets for training future AI systems, they become inputs to the next generation's pattern-matching. The error does not stay local. It propagates.
The proposed remedy — that clinicians carefully read, audit, and correct AI-generated notes — is correct as far as it goes. But it eliminates the labor saving that justified the technology's adoption. A conscientious clinician auditing an AI-generated note line by line, reconstructing the session against the system's rendering, catching the mistranscriptions and contextual distortions, performs the same cognitive work as writing the note from memory — with the added burden of correcting someone else's plausible-sounding errors, which is often more demanding than original composition. For responsible practitioners, the net labor saving is approximately zero. The product is being sold on the basis of a time saving that responsible use renders fictitious.
The deeper problem is institutional. Clinicians who use AI scribes without careful review — and the evidence suggests most do, given time pressures and the absence of mandated review protocols — are not negligent by choice. They are responding rationally to a system that markets the technology as labor-saving, provides no institutional support for the time-intensive oversight that responsible use requires, and leaves proofing to individual discretion while holding individuals legally liable for every error the system introduces. This is the structural incompatibility at the heart of the enterprise: the marketing rationale and the ethical rationale are not merely in tension. They are mutually exclusive. The ROI materializes only if the clinician does not do what they should do.
VI. The Closing Loop: Model Collapse and the Contamination of the Commons
The problem of atrophying human judgment and the problem of degrading AI output quality are not independent. They are connected by a single mechanism that makes each worse as the other progresses.
AI systems are trained on corpora assembled from the internet: the accumulated text of human writing, argumentation, discovery, and expression digitized and made available at scale. That corpus was, at its best, a genuinely open system — millions of distinct human minds, with different frameworks, different errors, different corrections of each other's errors, different cultural and linguistic contexts, generating genuinely heterogeneous signal. The combinatorial range of large language models derives from mining that heterogeneity. When the system surfaces an unexpected connection between a clinical observation and a philosophical framework, or between an engineering problem and a historical precedent, it does so because the human archive contains those connections, encoded across millions of documents written by people who thought carefully and wrote honestly.
That archive is now under systematic pressure. As AI-generated content — what has come to be called AI slop, the low-quality, mass-produced synthetic text that floods online platforms, content farms, search-engine optimization mills, and social media — accumulates at industrial scale, future AI systems increasingly train not on human-generated sources but on the outputs of earlier AI systems. By 2025, AI-generated content had overtaken human-generated content in volume across the web by a narrow margin, and the trajectory toward overwhelming AI dominance of new online text is clear.
The consequences were demonstrated with rigor in a landmark 2024 paper published in Nature by Ilia Shumailov and colleagues at Oxford. Their finding was precise and damning: when generative models are trained recursively on model-generated data, the results are "irreversible defects" in subsequent models — specifically, a progressive loss of the tails of the original data distribution. In information terms, the tails are the most important part. They contain the unusual observation, the minority viewpoint, the counterintuitive argument, the heterodox finding, the style that productively violates convention. These are precisely the elements that make a large body of human knowledge more than an average — the elements that give intellectual life its generative character. When AI systems learn from AI systems, those edges are smoothed away. What remains is an increasingly homogenized, error-prone statistical middle: a rendering of a rendering of a rendering, each generation slightly flatter and less faithful than the last.
A subsequent ICLR 2025 paper by Dohmatob and colleagues sharpened the finding: even a single synthetic data point per thousand in a training corpus is sufficient to produce asymptotic model collapse. Larger models, rather than being more robust to contamination, can amplify collapse rather than resist it. The Communications of the ACM reported in March 2026 that model collapse is already happening in deployed systems, driven by the quiet accumulation of synthetic data across the web — though it is important to note that the empirical demonstration is clearer in controlled recursive experimental settings than in production LLMs, where the measurement is more difficult. What the experimental evidence establishes beyond doubt is the mechanism; whether that mechanism is already measurably degrading the largest commercial models is an empirical question whose answer is approaching, not receding.
The proposed technical fixes are structurally inadequate. Detection tools for AI-generated content lag behind generation methods — by the time a filter can reliably identify the output of one model generation, the next has already produced text that evades it. The economic incentives of content platforms, social media companies, and SEO operations actively reward the mass production of cheap synthetic text, making voluntary remediation an anticompetitive act. The most technically coherent mitigation — rigorously curating training data with human verification alongside large supplies of clean, human-generated content — requires exactly the kind of skilled human judgment and editorial labor that the industry has staked its business model on eliminating. The solution to the problem caused by replacing human cognitive labor contradicts the rationale for replacing it.
There is a structural analogy here — held tentatively, as a structural illumination rather than a physical law — to thermodynamic entropy. A system that feeds increasingly on its own outputs tends toward equilibrium, which in information terms means homogeneity: the flattening of distinctions, the loss of signal, the triumph of statistical noise. The internet was an open system in the relevant sense: externally sourced human thinking continuously introduced genuine variance. As AI slop displaces human-generated content, the system progressively closes. The question is not whether this tendency exists — it demonstrably does — but whether the remaining human-generated signal is sufficient in quality and volume to offset the degradation. The current trajectory answers that question in the negative.
VII. The Mutually Reinforcing Dynamic
The two processes described in the preceding sections — the atrophy of human purposive agency through systematic outsourcing, and the degradation of the training corpus through recursive AI contamination — are not parallel problems. They are a single compounding mechanism.
As AI systems are deployed more widely with less substantive human oversight, they generate more AI slop. As more AI slop accumulates online, it enters training corpora at higher concentrations. As training corpora degrade, model outputs become less reliable. As outputs become less reliable, the case for careful human review becomes stronger — but the institutional conditions for that review have been progressively dismantled by the deployment model that created the problem. Meanwhile, the humans embedded in AI-assisted workflows are practicing deliberation less, developing judgment less, and becoming more dependent on outputs that are, through this same process, becoming less trustworthy. The ratchet tightens in one direction.
Consider an automated HR screening system trained on a corpus that includes AI-generated job descriptions, AI-generated performance assessments, and AI-synthesized candidate profiles — all produced by earlier system generations. The system screens applications before any human sees them. The hiring managers whose judgment it nominally assists have, through years of delegating initial screening to automated systems, become less practiced at the holistic evaluation of candidates the system was originally designed to support. When they encounter cases the system handles poorly — candidates whose qualifications are unusual, whose career paths are non-linear, whose backgrounds require contextual knowledge to evaluate — they lack the practiced judgment to catch the system's errors, because that judgment has not been exercised in the domain where it is needed. The errors enter the institutional record. Some of them are aggregated into future training data. The next version of the system inherits them.
This is the replication crisis applied to AI at civilizational scale. The psychology replication crisis emerged from the same basic structure: incentives that rewarded the production of plausible-sounding results over rigorous ones, a shared literature that aggregated and laundered errors, and a feedback loop that took decades to become visible because individual studies were too small to reveal the pattern. The difference with model collapse is velocity: AI allows the contamination of the shared epistemic commons to propagate across all domains simultaneously, at machine speed, without the slow accumulation of contradictory evidence that eventually exposed the replication crisis.
The epistemic commons — the shared information environment on which education, science, journalism, democratic deliberation, and culture all depend — is the ultimate casualty. Future AI systems trained on a contaminated commons will produce outputs that reflect that contamination. Those outputs, deployed at scale through social infrastructure with insufficient human oversight, will shape human decision-making in ways that compound rather than correct the original errors. There is no self-correcting mechanism inside this loop. It requires external input — genuinely new human thinking, introduced through the kind of purposive engagement that the dominant deployment model is systematically discouraging.
VIII. What Legitimate Use Actually Looks Like
There is a different mode of human-AI interaction that does not have these pathologies, and it is worth describing precisely — both because it represents the technology's genuine value and because it points toward what would need to change for that value to be sustainable.
The productive mode is dialogical, iterative, and irreducibly labor-intensive. It begins with a human being who brings to the interaction a theoretical framework, a body of prior knowledge, evaluative standards, and genuine stakes in the outcome. That person uses the AI system's combinatorial reach to surface connections, test formulations, locate empirical evidence, and generate candidate expressions of ideas they are working through. They read the outputs critically — not as products to be accepted or rejected wholesale, but as responses to be interrogated, corrected, pushed back against, and redirected. They supply the semantic grounding that gives the outputs whatever meaning they carry. They make the judgments about relevance, accuracy, and argumentative weight that the system cannot make. They iterate.
This mode of engagement is not time-saving. It typically takes longer than working without the AI system. It is, however, genuinely productive — not because the AI is thinking alongside the human, but because it is providing access to a vastly larger combinatorial space than any individual mind can navigate unaided, under conditions where human judgment controls what is extracted from that space and what is done with it. The physician who uses an AI system to surface longitudinal patterns across months of session transcripts — and then reads those patterns critically, testing them against their clinical knowledge, their memory of the patient, and their professional judgment — may catch connections they would otherwise have missed. This is real value. It does not require the AI to understand anything. It requires the physician to understand enough to evaluate what the system returns.
The productive mode also, crucially, replenishes the archive it draws on. When a human being engages with AI outputs through sustained purposive judgment — correcting errors, extending arguments, introducing genuinely new frameworks — the resulting text is not a recombination of existing patterns. It is new thinking, built partly from the archive but extending it. If that thinking enters the public record, it becomes part of the training corpus for future systems. The human archive is replenished with genuine variance. The system stays open.
The unproductive mode — "write me an essay on X" followed by uncritical acceptance and publication — does the opposite on both counts. It produces recombinant text that adds no new signal to the archive, and it progressively displaces the practice of the human cognitive capacities that make the productive mode possible. A scholar who consistently delegates the intellectual work of synthesis and argument to AI systems is not merely producing weaker work. They are, through the atrophy of disuse, becoming less capable of the purposive engagement that would make AI assistance genuinely valuable.
The legitimate use of AI is thus, at its core, the use that contradicts the marketing rationale — not entirely, but in the domains where the marketing rationale does the most damage. In binary, formally closed tasks, AI can deliver genuine labor savings with proportionate oversight. In semantically complex, contextually sensitive, normatively laden tasks — clinical documentation, legal analysis, educational assessment, creative and intellectual work — the technology's value is real but strictly conditional on the quality of human engagement it receives. The moment that condition is treated as optional for the sake of efficiency, the value disappears and the failure modes activate simultaneously.
IX. The Political Question
"Who authorized this?" is not a rhetorical question. It is a precise question about democratic legitimacy, and it has a precise answer: nobody, in any sense that would satisfy a theory of democratic authorization.
Authorization, in the sense that matters here, is not administrative. It is not the signing of a procurement contract, the acceptance of a terms-of-service agreement, or the granting of app permissions. It is agentive: the exercise of genuine purposive judgment — Dewey's dramatic rehearsal, applied collectively — about what is being delegated, to what kind of system, under what conditions of oversight, with what mechanisms of accountability, and on whose behalf. Democratic authorization of social infrastructure requires that the people who will bear the costs of that infrastructure have had a genuine opportunity to evaluate those costs, imagine the alternatives, and choose.
That deliberation has not happened. The infrastructure has been built at the pace of capital deployment and speculative valuation, not at the pace of democratic deliberation. The environmental costs were not debated before the data centers were permitted. The labor displacement effects were not weighed before the automation was rolled out. The implications for clinical documentation, for judicial processes, for educational assessment, for the information environment were not evaluated before the systems were embedded in the workflows that now depend on them. The regulatory frameworks that exist — the EU AI Act, the FDA's device classification process, the APA's ethics codes — all contain versions of the human oversight requirement, and all fall short of mandating the substantive deliberative engagement that would make that oversight real.
The burden of proof in this situation lies with the proponents of rapid, large-scale AI deployment, not with its critics. The reason is irreversibility. Infrastructure lock-in, once established, is not easily undone. Clinical records contaminated with AI errors become part of the permanent archive. Training corpora contaminated with AI slop cannot be laundered back into clean signal. Human deliberative capacities, once atrophied through systematic disuse, do not automatically recover when the systems that displaced them are removed. The costs of getting this wrong accumulate in ways that cannot be corrected after the fact.
The case for urgency, then, is not that AI is approaching consciousness or that the machines are about to take over in any science-fiction sense. It is considerably more mundane and considerably more serious: the human capacities and epistemic resources from which AI draws its apparent intelligence, and on which its genuine utility depends, are being systematically consumed in the course of its deployment. The archive is being contaminated by its own outputs. The judgment required to catch the errors is being eroded by the habit of not catching them. The window to shape these systems rather than merely inherit their consequences is not infinite.
What the technology actually is — a combinatorial engine of extraordinary range, built from the accumulated archive of human thought, useful precisely to the degree that genuine human purposive agency governs its use — points directly toward what a responsible politics of AI would look like. It would mandate substantive oversight, not nominal presence. It would protect and invest in the human practices — of deliberation, clinical attentiveness, editorial judgment, scholarly rigor — that constitute the only renewable source of the quality on which the technology depends. It would treat the epistemic commons as a public good requiring active stewardship, not a resource to be strip-mined for training data. And it would insist that the pace of deployment be governed by the pace at which democratic societies can actually evaluate what is being built into their infrastructure — which is to say, considerably more slowly than it is currently moving.
The machine is not the problem. The closing of the loop is.
Bibliography
I. Model Collapse and Recursive Training Degradation
Shumailov, Ilia, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. 2024. "AI Models Collapse When Trained on Recursively Generated Data." Nature 631: 755–759. https://www.nature.com/articles/s41586-024-07566-y. PMC: https://pmc.ncbi.nlm.nih.gov/articles/PMC11269175/
Shumailov, Ilia, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, and Yarin Gal. 2023. "The Curse of Recursion: Training on Generated Data Makes Models Forget." arXiv:2305.17493. https://arxiv.org/abs/2305.17493
Dohmatob, Elvis, Yunzhen Feng, Arjun Subramonian, and Julia Kempe. 2025. "Strong Model Collapse." ICLR 2025 Spotlight. https://openreview.net/forum?id=et5l9qPUhm. Also at: https://arxiv.org/abs/2410.04840
"Beyond Model Collapse: Scaling Up with Synthesized Data Requires Verification." 2025. ICLR 2025. https://iclr.cc/virtual/2025/poster/29933
"Model Collapse Is Already Happening, We Just Pretend It Isn't." 2026. Communications of the ACM Blog, March 24, 2026. [Note: the claim that collapse is measurable in production LLMs is empirically contested; the mechanism is established; flag accordingly.]
II. AI Slop, Internet Contamination, and the Epistemic Commons
Reuters Institute for the Study of Journalism, University of Oxford. 2024. "AI-Generated Slop Is Quietly Conquering the Internet." November 2024. https://reutersinstitute.politics.ox.ac.uk/news/ai-generated-slop-quietly-conquering-internet-it-threat-journalism-or-problem-wi
III. Environmental and Energy Costs
United Nations University Institute for Water, Environment and Health (INWEH). 2026. "The Environmental Cost of Artificial Intelligence: Energy, Carbon, Water and Land Footprints." June 2, 2026. https://unu.edu/inweh/collection/environmental-cost-of-AIs-Enrgy-Use-Carbon-water-and-land-footprints
Data centers projected at 945 TWh annually by 2030; water equivalent to 1.3 billion people's annual domestic supply; e-waste up to 2.5 million tonnes annually; 90%+ of AI compute concentrated in U.S. and China.
International Energy Agency (IEA). 2026. Energy and AI. April 2026. https://iea.blob.core.windows.net/assets/de9dea13-b07d-42c5-a398-d1b3ae17d866/EnergyandAI.pdf
Brookings Institution. 2026. "Global Energy Demands Within the AI Regulatory Landscape." April 20, 2026. https://www.brookings.edu/articles/global-energy-demands-within-the-ai-regulatory-landscape/
Data center consumption approaching 1,050 TWh by 2026.
Consumer Reports. 2026. "AI Data Centers: Impact on Electric Bills, Water, and More." March 2026. https://www.consumerreports.org/data-centers/ai-data-centers-impact-on-electric-bills-water-and-more-a1040338678/
Single hyperscale data center draws as much electricity as 100,000 households.
IV. Labor Displacement
Goldman Sachs. 2026. "How Will AI Affect the U.S. Labor Market?" March 2026. https://www.goldmansachs.com/insights/articles/how-will-ai-affect-the-us-labor-market
~300 million jobs globally exposed to AI automation.
Boston Consulting Group (BCG). 2026. "AI Will Reshape More Jobs Than It Replaces." March 2026. https://www.bcg.com/publications/2026/ai-will-reshape-more-jobs-than-it-replaces
50–55% of U.S. jobs to be reshaped within 2–3 years.
AImultiple. 2026. "AI Job Loss Statistics." June 2026. https://aimultiple.com/ai-job-loss
~200,000 technology-sector job losses in 2026 to date.
V. Drug Discovery
Drug Target Review. 2026. "AI in Drug Discovery: Predictions for 2026." February 15, 2026. https://www.drugtargetreview.com/ai-in-drug-discovery-predictions-for-2026/1865962.article
No full FDA-approved AI-discovered drug as of mid-2026; Insilico Medicine INS018-055 earliest approval late 2026/early 2027; 90% clinical trial failure rate unchanged.
VI. Clinical AI, Automated Scribes, and Healthcare Governance
Miner, Adam S., Albert Haque, Jason A. Fries, et al. 2020. "Assessing the Accuracy of Automatic Speech Recognition for Psychotherapy." npj Digital Medicine 3: 82. Stanford University / Nature Portfolio. https://pmc.ncbi.nlm.nih.gov/articles/PMC7270106/. Also at Stanford HAI: https://hai.stanford.edu/research/assessing-the-accuracy-of-automatic-speech-recognition-for-psychotherapy
25% average ASR word error rate across 100 psychotherapy sessions; 34% error rate for harm-related sentences specifically; authors conclude ASR "may not be ready for individual-level safety surveillance"; error rates likely conservative; disparate impact on ethnic minorities and non-native speakers.
Maleki Varnosfaderani, Shima, and Mohamad Forouzanfar. 2024. "The Role of AI in Hospitals and Clinics: Transforming Healthcare in the 21st Century." Bioengineering (Basel) 11 (4): 337. https://pmc.ncbi.nlm.nih.gov/articles/PMC11047988/
Legitimate binary/operational AI uses: scheduling, inventory, radiology triage; AI throughout framed as augmenting, not replacing, clinical judgment.
Sriharan, Abi, et al. 2025. "Artificial Intelligence in Healthcare: Balancing Technological Innovation With Health and Care Workforce Priorities." International Journal of Health Planning and Management 40 (4): 987–992. https://pmc.ncbi.nlm.nih.gov/articles/PMC12215598/
30–40% of healthcare tasks theoretically automatable with caveat: "over-reliance risks eroding critical thinking and diagnostic skills"; NEDA chatbot case (harmful advice, abrupt shutdown); IBM Watson abandoned after $62 million at MD Anderson; Google retinopathy AI failed in Thailand deployment.
Kaltenbrunner, Saskia. 2026. "Human in Control: Shared Decision-Making with Clinical Decision-Support Systems Under the Artificial Intelligence Act." Computer Law & Security Review 61 (July 2026): 106281. Open access. https://doi.org/10.1016/j.clsr.2026.106281
Human oversight must be substantive, not nominal (GDPR Art. 29 Working Party); EU AI Act Article 14(4)(b) legally recognizes automation bias; Articles 26(11) and 86 exclude medical devices from patient disclosure requirements; medical decision-making "must contend with uncertainty, probabilities and varying value systems."
"Therapy Notes by AI Create False Narratives, Therapists Say." 2024. ClearHealthCosts. https://clearhealthcosts.com/blog/2024/06/therapy-notes-by-ai-create-false-narratives-therapists-say/
[Journalistic source — cite as illustrative case, not peer-reviewed evidence. Documents AI scribes inserting references to abuse and false diagnoses never discussed in session.]
VII. Philosophical and Theoretical Sources
Dewey, John. 1922. Human Nature and Conduct: An Introduction to Social Psychology. New York: Henry Holt.
Part III, "The Place of Intelligence in Conduct": deliberation as dramatic rehearsal — imaginative projection of possible futures, affectively and evaluatively weighted, enacted through choice.
Campbell, James. "Ethical Deliberation as Dramatic Rehearsal: John Dewey's Theory." Educational Theory. https://www.academia.edu/130273738/Ethical_Deliberation_as_Dramatic_Rehearsal_John_Deweys_Theory
Dewey, John. 2026 [secondary]. "Deliberation as Drama and Discovery." Chapter 7 in John Dewey's Human Nature and Conduct. Cambridge: Cambridge University Press. https://www.cambridge.org/core/books/john-deweys-human-nature-and-conduct/deliberation-as-drama-and-discovery/E245F2071F52595826
[2026 scholarly edition — useful for contemporary framing of the dramatic rehearsal concept.]
Tomasello, Michael. 2019. Becoming Human: A Theory of Ontogeny. Cambridge, MA: Harvard University Press. https://books.google.com/books/about/Becoming_Human.html?id=ZnhyDwAAQBAJ
Central argument: uniquely human cognitive capacities — shared intentionality, collaborative reasoning, normative attunement — are socially constituted through developmental practice, not fixed biological endowments; require exercise to remain robust.
Tomasello, Michael. 2005. "Understanding and Sharing Intentions: The Origins of Cultural Cognition." Behavioral and Brain Sciences 28 (5): 675–691. https://www.eva.mpg.de/documents/Cambridge/Tomasello_Understanding_BehBrainSci_2005_1555292.pdf
Empirical grounding: three decades of comparative experiments with chimpanzees, bonobos, and human children; shared intentionality as the basis of human cultural accomplishment.
"Shared Intentionality." 2025. Open Encyclopedia of Cognitive Science, MIT. May 26, 2025. https://oecs.mit.edu/pub/sep9e3c2
"Tomasello (2022, 2024) argued that the broadest perspective on these phenomena is in terms of human agency: humans (and only humans) form joint agency."
VIII. Honest Empirical Caveats — To Flag in the Text
On model collapse in production LLMs: The Nature 2024 and ICLR 2025 papers establish the mechanism and demonstrate it rigorously in controlled recursive experimental settings. Whether collapse is yet measurably active at scale in the largest deployed commercial models is empirically contested (see Hacker News discussion thread on CACM March 2026 piece). The essay should mark this distinction clearly. The mechanism is established; the production-scale timing is an open empirical question.
On the entropic analogy: Held as a structural illumination, not a physical law. The internet is not a fully closed system — some fresh human-generated signal continues to enter training corpora. The question is whether that signal is sufficient in quality and volume to offset contamination. Presented as a structural analogy, the framing is robust; presented as a physical necessity, it overclaims.
On "exactly zero" labor saving: Google's gloss, not a finding from the PMC papers. The accurate claim, supported by Miner et al. and Sriharan et al., is that responsible use of clinical AI scribes requires the same time investment as traditional note-writing, with higher cognitive demand — making net labor savings for conscientious practitioners approximately zero or negative.
