It has been three years since the arrival of GPT-3,2 a neural-network-based large language model (LLM) that could generate polished, stylish text with ease. What followed was an intense race among tech giants like Google, OpenAI, and Meta to come up with larger and larger models, often trained using texts appearing anywhere on the Internet. While storming the AI research communities, all these were much outside the scope of usual public discourse. And then came OpenAI’s ChatGPT3—an AI chatbot running on top of an LLM. The advancement was evident, shortcomings were subtle. But the most drastic change that ChatGPT brought forth was the sensation among the non-researchers (or researchers far from language technology or artificial intelligence) and ordinary citizens. With the online interface available to everyone on the Internet, it did not take much to become the talk of the town. ChatGPT can explain quantum computing to a non-expert, write poems in Keats’ style, write and debug JavaScript codes, and many more. And very soon we started wondering: Can ChatGPT replace search engines such as Google, simplified knowledge sources like Wikipedia, or community-powered Q&A platforms like StackExchange? To get an idea of the mammoth popularity, ChatGPT garnered more than 100 million users during just two months of launching; it took 2.5 years for Instagram and nine months for TikTok to achieve the same amount of users.8 Beyond the shock and awe, it is important to identify the potential impact of the large-scale deployment of such AI-based chatbots on the production and consumption of scientific knowledge. Specifically, how would one envision the role of an AI chatbot being used by non-experts as a source to quench their thirst for scientific knowledge?
Popular Majority vs. Authentic Minority
It is very much accepted in the LLM research that however proficient these models are in generating polished, confident textual responses to queries, they lack factual consistency.6 Scaling to even larger models has significantly decreased these inconsistencies from the earlier models. However, we are not just talking about blunt factual errors here; ChatGPT is smarter in that sense. When the goal is to communicate hardcore scientific facts to non-expert audiences, misconceptions can percolate with apparently synonymous phrasings—very often overlooked by even the experts. For example, consider the conversation with ChatGPTa per the following text:
The question is erroneous in itself from the beginning since spacetime is modeled as a Lorentzian manifold and not a Riemannian one, with the positive-definiteness condition relaxed.5 Ignoring the erroneous definition, ChatGPT continues to explain why this is indeed true. Since locating what such LLMs actually know, it is even harder to correct such inconsistencies in a trustworthy manner. The condition becomes even more strange as the lack of self-consistency surfaces. When asked about the rate of expansion of the universe, ChatGPT declares the universe is not expanding at a speed faster than light.
Yet, to explain the cosmic horizon, it imposes that light from parts beyond the cosmic horizon cannot reach us because the expansion rate is increasing faster than the speed of light. The arguments in both responses are quite interesting. For it answers that faster-than-light motion is impossible, the assertion is to the point. However, the reasoning is inconsistent as it cannot demarcate between the notions of scale and distance. This misconception is commonplace in popular science literature. It is not unlikely that a vast presence of such poorly conceived articles in the training data of ChatGPT pushed the model to bear such misconceptions. However, there is no way to investigate this other than OpenAI itself since the pretraining dataset is not open source. The two examples mentioned here primarily point toward the inconsistencies present in LLMs that might have originated as a result of noisy training data—scientific misconceptions abundantly present on the Internet. We show another similar example here.
1. Fractal dimension …
When we ask ChatGPT the definition of fractals, it quickly generates a paragraph that reverberates the notion of self-similarity and fractality juxtaposed together. As the interaction proceeds, the chatbot puts light on the ambiguous definitions it spat out earlier. We can see it finally shifting toward the requirement of a Hausdorff dimension of the object to be greater than its topological dimension.9 Interestingly, Mandelbrot’s paper9 contains a rigorous definition of the fractal. Yet, the sheer number of popular definitions of fractals prevail in ChatGPT’s answer. The underlying issue is quite straightforward to perceive—a language model learns from the majority of examples while rigorous scientific truth is mostly a minority. Such miscommunications are bound to manifest from the text such models generate in reply to a scientific query unless it learns to demarcate between an authoritative text on a scientific phenomenon and popular literature trying to communicate scientific findings in simpler terms. Certainly, ChatGPT would need to learn science communication from the latter, but not at the expense of disregarding the facts and reasoning presented in the former.
Multiple Sources, Multiple Versions
And then there are questions that cannot be answered in one specific way. Some of these questions might arise from a philosophical viewpoint of natural and social sciences, and there are multiple contending models that are well applicable. For example, there are multiple schools of thought on the interpretation of quantum uncertainty and what happens when we measure (or do not measure) a quantum particle. The Copenhagen Interpretation (CI)7 assumes quantum states of systems evolve unitarily until measured; measurement breaks the unitary evolution, often described as the wavefunction collapse. On the other hand, Many-Worlds Interpretation (MWI)4 assumes measurement only introduces multiple branches of reality; the quantum states remain unitarily evolving in all possible branches. Both CI and MWI produce the same, consistent predictions of quantum systems—none of these two can be assumed to be closer to the truth. And there are multiple other models of interpretation.10 Yet, as we can see that ChatGPT assumes CI and does not mention MWI or other thoughts in relation to the following question:
This does not come as a surprise, given the fact that many assume the Copenhagen Interpretation to be the dominant one, and as we argued earlier, LLMs are indeed biased towards majority opinion simply due to statistical consequence. While this example is more of a philosophical argument and a certain choice is unlikely to alter the empirical understanding, it is difficult to believe that such majority biases are not present elsewhere. The most troublesome attribute is that models like ChatGPT, in their current state, are not able to cite their sources. With the vastness of human knowledge, it is impossible for an AI-based chatbot to list down all possible interpretations, models, and schools of thought in one single answer. Without showing the sources, their knowledge distribution is essentially a one-step process. The user must remain content with whatever the chatbot produces. One may argue that no one is claiming that ChatGPT will be the only source of knowledge, and hence, why bother? Definitely, the Internet will be there. But so are the public libraries in the age of the Internet. Yet, most tend to access the Internet for its ease and speed. Given that AI-based chatbots are able to decrease the search effort even more, it would be shortsighted to reject the idea of a similar dominance.
Conclusion
We must keep in mind that the examples shown here are cherry-picked and definitely not a wholesome representative of ChatGPT’s capabilities. In fact, the degree of critics ChatGPT has received1 is only signaling the capabilities and expectations that come with such an ambitious project. The arguments we presented are rather focused on better design principles of how an AI chatbot should interact with daily users. Definitely, a fatter column space in popular media demands human-like AI. Language fluency is probably the quickest path to mimic human-like capabilities. But beyond those shiny pebbles, one must ask the question, is a human-like AI the best aid to humans? We argue that the focus of AI research needs to be concentrated on the latter, especially when we are talking about aiding people with easy access to scientific knowledge. In our hitherto non-automated past, seldom a single individual teacher has served the purpose of the only source of knowledge. It is very unlikely that any (even unintentional) push toward an automated persona as a single-point source of knowledge will do us any good. An interesting development in this direction worth mention is the latest Bing chatb from Microsoft: a search engine augmented with an automated chat assistant. Given that this interface provides both the answers from the AI assistant as well as typical search engine results together for a query, it can provide a quantitative insight into the actual impact of AI assistants as knowledge sources on the average users. We hope the research community will make a sincere effort toward these investigations.
Join the Discussion (0)
Become a Member or Sign In to Post a Comment