The ever-increasing number of scientific publications poses a challenge. Liang et al. (2024) suggest a growing trend of AI-assisted paper writing (nearly 20% in computer science). This raises questions about the future of citation metrics and the effectiveness of peer review in a world where authorship may not solely reside with humans.
The term ‘peer review,’ often viewed as a cornerstone of science, has a surprisingly recent origin (1969, according to Merriam-Webster). While giants like Einstein and Newton thrived without it, concerns about its limitations have been voiced. Dijkstra (EWD1018, 1987) saw peer review as potentially stifling originality.
The current emphasis on publication quantity, as Stonebraker (2018) argues, incentivizes researchers to prioritize publishable, but potentially irrelevant, work. This leads to a “diarrhea of papers” that may not address critical problems.
AI’s ability to generate vast amounts of text raises concerns about a potential flood of irrelevant theoretical papers, further straining the evaluation system. Stonebraker’s (2018) call for rewarding problem-solving over publication needs revisiting. Perhaps the emphasis should be on the impact and significance of research, not just its passage through peer review — a skill replicable by AI.
AI can pave the way for a “golden age” of scientific progress if we can develop new evaluation methods focused on problem-solving and real-world impact.
The scientific community must adapt to the evolving landscape. By recognizing the limitations of peer review and prioritizing the pursuit of meaningful solutions, we can ensure that AI becomes a catalyst for scientific advancement, not a detriment.
References
M. Stonebraker, “My Top Ten Fears about the DBMS Field,” 2018 IEEE 34th International Conference on Data Engineering (ICDE), Paris, France, 2018, pp. 24-28, doi: 10.1109/ICDE.2018.00012.
Liang, W., Zhang, Y., Wu, Z., Lepp, H., Ji, W., Zhao, X., … & Zou, J. Y. (2024). Mapping the increasing use of LLMs in scientific papers. arXiv preprint arXiv:2404.01268.
E. W. Dijkstra, EWD1018, Nuenen, 21 December 1987
Daniel Lemire is a computer science professor at the Data Science Laboratory of the Université du Québec (TÉLUQ) in Montreal. His research is focused on software performance and data engineering. He is a techno-optimist and a free-speech advocate.
Join the Discussion (0)
Become a Member or Sign In to Post a Comment