Sign In

Communications of the ACM


The Importance of Reviewing the Code

The Importance of Reviewing the Code illustration

Credit: Vasily Smirnov

Highlighting the significance of the often overlooked underlying software used to produce research results.

The full text of this article is premium content


Florina Ciorba

Finally, someone who hits the nail on the head. As a reviewer, I have always asked the "uncomfortable" questions regarding the software used to produce the results reported in the paper(s). Very few authors dignified these questions with answers, and not all of them were enlightening.
Reproducibility of experiments is what every scientist wants and fears at the same time. Those who are interested in experiments published in a paper by other researchers, most certainly would want to reproduce them, or leverage them for future work. After all, there is no point in "re-inventing the wheel". However, those that published the experiments, fear that reproducibility of their experiments would take away their vantage point on the specific topic and that they would be overcome by others who would be faster in producing new research results based on the reproduced experiments.
Here's hoping that this article raises & increases the awareness of the significance of the means (in this case the code) used to achieve the goal (in this case the research results) and reproducibility of any published research work & results.

Displaying 1 comment

Log in to Read the Full Article

Sign In

Sign in using your ACM Web Account username and password to access premium content if you are an ACM member, Communications subscriber or Digital Library subscriber.

Need Access?

Please select one of the options below for access to premium content and features.

Create a Web Account

If you are already an ACM member, Communications subscriber, or Digital Library subscriber, please set up a web account to access premium content on this site.

Join the ACM

Become a member to take full advantage of ACM's outstanding computing information resources, networking opportunities, and other benefits.

Subscribe to Communications of the ACM Magazine

Get full access to 50+ years of CACM content and receive the print version of the magazine monthly.

Purchase the Article

Non-members can purchase this article or a copy of the magazine in which it appears.