Communications of the ACM
Cargo Cult AI
Credit: Ella Svetlaya
Astronomer Carl Sagan once wrote "science is more than a body of knowledge; it is a way of thinking." This type of thinking requires skeptical rigor and brutal honesty to thoroughly investigate, reason, and seek to invalidate hypotheses before jumping to conclusions. But it is all too easy to jump to conclusions. Despite our self-proclaimed intelligence, humans are apt to believe remarkable fallacies based on a paucity of correlated information rather than rigorously seek to determine causal foundations.
This propensity of humans to believe wonderful, fanciful things so easily is what another physicist, Richard Feynman, called cargo cult science. Feynman named this phenomenon after a "cargo cult" of people in the Pacific Islands who believed that building replicas of landing strips and control towers would ensure supply planes continued to land after World War II.6 The planes never came. These people missed the fact that it was the advent of war, not the presence of landing strips, that caused the planes to land there.
Comments
David Tonhofer
While agreeing with the gist of the article, I will just add a not regarding this except:
"This approach [i.e. multilayer neural networks] was invented in the 1940s..."
and reference is given to "McCulloch, W.S., Pitts, W. A logical calculus of the ideas immanent in nervous activity."
Well, I dug out that paper and there is, unsurprisingly, nothing in there about "multilayer neural networks" as they are understood now. McCulloch and Pitts describe networks of integrating, firing "neurons" where the connections are inhibitory or excitatory, with "circles", i.e. recurrent, or not. They show that a circuit without circles is equivalent to a formula in propositional logic (as expected) and an argument is made that circle-less networks can replace the control logic of Turing machine, while a circle-full network is equivalent to a Turing machine, tape and all. This is followed by some philosophical musings.
However, there is nothing about "learning" for example, so saying that "the approach was invented" in that paper sounds imbued with excessive Platonicism.
A better early reference might be Frank Rosenblatt (then at Cornell Aeronautical Laboratory) "The Perceptron: A Probabilistic Model for Information Storage and Organization" (Psychological Review, Vol. 65, No. 6, 1958). It contains some beautiful graphs too.
David Tonhofer
As a sidenote, in the report
"ChatGPTs Astonishing Fabrications about Percy Ludgate"
by Brian Coghlan, Brian Randell and Noel OBoyle
we find an pithy description of the "Stochastic Parrot" by Ted Chiang (Ted Chiang: "ChatGPT Is a Blurry JPEG of the Web", New Yorker, 9th February, 2023, see: https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web )
"Think of ChatGPT as a blurry jpeg of all the text on the Web. It retains much of the information on the Web, in the same way that a jpeg retains much of the information of a higher-resolution image, but, if youre looking for an exact sequence of bits, you wont find it; all you will ever get is an approximation. But, because the approximation is presented in the form of grammatical text, which ChatGPT excels at creating, its usually acceptable. [...] Its also a way to understand the "hallucinations", or nonsensical answers to factual questions, to which large language models such as ChatGPT are all too prone. These hallucinations are compression artifacts, but [...] they are plausible enough that identifying them requires comparing them against the originals, which in this case means either the Web or our own knowledge of the world. When we think about them this way, such hallucinations are anything but surprising; if a compression algorithm is designed to reconstruct text after ninety-nine per cent of the original has been discarded, we should expect that significant portions of what it generates will be entirely fabricated."
We might have a plausible model of "Confabulation in Dementia" if nothing else.
Displaying all 2 comments