Computing Profession

Why Peer Review Matters

Carnegie Mellon Professor Jeannette Wing

At the most recent Snowbird conference, where all the chairs of computer science departments in the U.S. meet every two years, there was a plenary session during which the panelists and audience discussed the peer review processes in computing research, especially as they pertain to a related debate on conferences versus journals. It’s good to go back to first principles to see why peer review matters, to inform how we then would think about process.

In research we are interested in discovering new knowledge. With new knowledge we push the frontiers of the field. It is through excellence in research that we advance our field, keeping it vibrant, exciting, and relevant. How is excellence determined? We rely on experts to distinguish new results from previously known, correct results from incorrect, relevant problems from irrelevant, significant results from insignificant, interesting results from dull, the proper use of scientific methods from being sloppy, and so on. We call these experts our peers. Their/our judgment assesses the quality and value of the research we produce. It is important for advancing our field to ensure we do high-quality work. That’s why peer review matters.

In science, peer review matters not just for scientific truth, but, in the broader context, for society’s perception of science. Peer review matters for the integrity of science. Scientific integrity is the basis for public trust in us, in our results, in science. Most people don’t understand the technical details of a scientific result, let alone how it was obtained, what assumptions were made, in what contexts the result is applicable, or what practical implications it has. When they read in the news that “Scientists state X,” there is an immediate trust that “X” is true. They know that science uses peer review to vet results before they are published. They trust this process to work. It is important for us, as scientists, not to lose the public trust in science. That’s why peer review matters.

“Public” includes policymakers. Most government executives and congressional members are not scientists. They do not understand science, so they need to rely on the judgment of experts to determine scientific truth and how to interpret scientific results. We want policymakers in the administration and Congress to base policy decisions on facts, on evidence, and on data. So it is important for policymakers that, to the best of our ability, we, as scientists, publish results that are correct. That’s why peer review matters.

  Jeannette M. Wing
While I argue peer review matters, ithtml5-dom-document-internal-entity1-rsquo-ends a whole other question of what the best process is for carrying out peer review. In this day and age of collective intelligence through social networks, we should think creatively about how to harness our own technology to supplement or supplant the traditional means used by journals, conferences, and funding agencies. Peer review matters, and now is the time to revisit our processes–not just procedures and mechanisms, but what it is we review (papers, data, software, and tools), our evaluation criteria, and our incentives for active participation. html5-dom-document-internal-entity1-nbsp-end

Join the Discussion (0)

Become a Member or Sign In to Post a Comment

The Latest from CACM

Shape the Future of Computing

ACM encourages its members to take a direct hand in shaping the future of the association. There are more ways than ever to get involved.

Get Involved

Communications of the ACM (CACM) is now a fully Open Access publication.

By opening CACM to the world, we hope to increase engagement among the broader computer science community and encourage non-members to discover the rich resources ACM has to offer.

Learn More