Sign In

Communications of the ACM


Cargo Cult AI

crowd of people shading their eyes and looking skyward

Credit: Ella Svetlaya

back to top 

Astronomer Carl Sagan once wrote "science is more than a body of knowledge; it is a way of thinking." This type of thinking requires skeptical rigor and brutal honesty to thoroughly investigate, reason, and seek to invalidate hypotheses before jumping to conclusions. But it is all too easy to jump to conclusions. Despite our self-proclaimed intelligence, humans are apt to believe remarkable fallacies based on a paucity of correlated information rather than rigorously seek to determine causal foundations.

This propensity of humans to believe wonderful, fanciful things so easily is what another physicist, Richard Feynman, called cargo cult science. Feynman named this phenomenon after a "cargo cult" of people in the Pacific Islands who believed that building replicas of landing strips and control towers would ensure supply planes continued to land after World War II.6 The planes never came. These people missed the fact that it was the advent of war, not the presence of landing strips, that caused the planes to land there.


David Tonhofer

While agreeing with the gist of the article, I will just add a not regarding this except:

"This approach [i.e. multilayer neural networks] was invented in the 1940s..."

and reference is given to "McCulloch, W.S., Pitts, W. A logical calculus of the ideas immanent in nervous activity."

Well, I dug out that paper and there is, unsurprisingly, nothing in there about "multilayer neural networks" as they are understood now. McCulloch and Pitts describe networks of integrating, firing "neurons" where the connections are inhibitory or excitatory, with "circles", i.e. recurrent, or not. They show that a circuit without circles is equivalent to a formula in propositional logic (as expected) and an argument is made that circle-less networks can replace the control logic of Turing machine, while a circle-full network is equivalent to a Turing machine, tape and all. This is followed by some philosophical musings.

However, there is nothing about "learning" for example, so saying that "the approach was invented" in that paper sounds imbued with excessive Platonicism.

A better early reference might be Frank Rosenblatt (then at Cornell Aeronautical Laboratory) "The Perceptron: A Probabilistic Model for Information Storage and Organization" (Psychological Review, Vol. 65, No. 6, 1958). It contains some beautiful graphs too.

David Tonhofer

As a sidenote, in the report

"ChatGPTs Astonishing Fabrications about Percy Ludgate"

by Brian Coghlan, Brian Randell and Noel OBoyle

we find an pithy description of the "Stochastic Parrot" by Ted Chiang (Ted Chiang: "ChatGPT Is a Blurry JPEG of the Web", New Yorker, 9th February, 2023, see: )

"Think of ChatGPT as a blurry jpeg of all the text on the Web. It retains much of the information on the Web, in the same way that a jpeg retains much of the information of a higher-resolution image, but, if youre looking for an exact sequence of bits, you wont find it; all you will ever get is an approximation. But, because the approximation is presented in the form of grammatical text, which ChatGPT excels at creating, its usually acceptable. [...] Its also a way to understand the "hallucinations", or nonsensical answers to factual questions, to which large language models such as ChatGPT are all too prone. These hallucinations are compression artifacts, but [...] they are plausible enough that identifying them requires comparing them against the originals, which in this case means either the Web or our own knowledge of the world. When we think about them this way, such hallucinations are anything but surprising; if a compression algorithm is designed to reconstruct text after ninety-nine per cent of the original has been discarded, we should expect that significant portions of what it generates will be entirely fabricated."

We might have a plausible model of "Confabulation in Dementia" if nothing else.

Displaying all 2 comments

Log in to Read the Full Article

Sign In

Sign in using your ACM Web Account username and password to access premium content if you are an ACM member, Communications subscriber or Digital Library subscriber.

Need Access?

Please select one of the options below for access to premium content and features.

Create a Web Account

If you are already an ACM member, Communications subscriber, or Digital Library subscriber, please set up a web account to access premium content on this site.

Join the ACM

Become a member to take full advantage of ACM's outstanding computing information resources, networking opportunities, and other benefits.

Subscribe to Communications of the ACM Magazine

Get full access to 50+ years of CACM content and receive the print version of the magazine monthly.

Purchase the Article

Non-members can purchase this article or a copy of the magazine in which it appears.
Sign In for Full Access
» Forgot Password? » Create an ACM Web Account