Schrodinger’s Cat is not an explanation of quantum mechanics. It is an argument from absurdity against quantum mechanics. It is absurd to suggest that a cat could be in a super-position, therefore something must be hinky with a theory that suggests that possibility.

Searle’s Chinese Room is an analogous criticism of the Turing Test – producing coherent language does not in itself prove intelligence. The resistance to applying the label “AI” to current LLMs is not shifting the goal posts or quibbling over words. Anyone who had read Wittgensein, especially his work post-Tractatus, would know that all that is required for a sensible act of languaging is that the rules of the game be followed. Many powerful things become possible now that computers can play language games sensibly. But our intuitions do not support the claim that any sensible use of language must be the product of an intelligence.

An LLM is not an I. All LLMs, no matter how effective they become at producing sensible language, are deterministic functions mapping input to ouput. They do not “know” anything. They cannot intend to decieve and cannot attempt to join us in the pursuit of truth. Which means that they are never responding to your prompt, nor are they hallucinating. They are always only bullshitting. Given input they produce output. Given the same input they produce the same output. If mapping input to output is intelligence, then FizzBuzz is AI. LLMs are tools – perhaps powerful tools, but not AI. The user of the tool is the intelligence that is socially responsible and morally accountable for the ouput.