The Pros and Cons of the ‘PACM’ Proposal: Point

Make no mistake, computer science research is a resounding success. Its research advances, peer reviewed and published in conferences, created new billion-dollar industries, changing how we do science, business, government, entertain ourselves, and communicate.

While many universities accommodate the rigorous, conference publication culture of CS, others do not. Our practices differ from other sciences that only peer review articles in journals, using conferences for unreviewed talks, posters, and articles.

I recommend the ACM Publications Board proposal because it articulates best principles for research publication—rigorous peer reviewing and no page restrictions—yet is flexible enough to accommodate our conference culture. Adopting it will further improve our process and recognize our archival quality conferences as such, harmonizing computer science with the international science community.

Computer scientists use a conference reviewing process with annual deadlines to combine peer review with timely publication (typically five to seven months from submission to publication). Compared to journal reviewing, I believe our conference process has distinct advantages that this proposal maintains.

"I recommend the proposal because it articulates best principles for research publication, yet is flexible enough to accommodate our conference culture."

ACM conferences are organized by Steering Committees, with general and program chairs and SIG representatives. Program chairs generally serve once for one year. They select Program Committee (PC) members, who each review 10 to 30 submissions and act as editors, choosing accepted papers. Because of increasing numbers of submissions, many conferences have an additional committee with fewer responsibilities. They each review 1 to 10 papers and do not attend the PC meeting. When well deployed, these reviewers deliver highly expert reviews. The most important job of program chairs and journal editors is the same—to find expert reviewers, going beyond the committee and associate editors as necessary. While individual reviews are blinded, the committees are publicly acknowledged. Reviewers are known to each other, increasing accountability.

Because reviewers and leadership change regularly and every submission is reviewed, no one editor or associate editor exerts influence and scientific biases for many years or by desk rejecting submissions. Consequently, conferences are more likely to include diverse problems and approaches.

A disadvantage of conference reviewing is page-limiting submissions to control reviewer workload, which may cause omission of material, such as proofs and methodology. With this proposal, reviewers can judge page-limited submissions, but require additional material, reviewing it or not, as appropriate. Concision is a virtue that reviewers may also require. With one to three months to revise accepted papers (and more for rejected submissions), I think the result will be articles with appropriate content without overburdening reviewers.

Conferences often produce more reviews than journals, providing authors richer feedback from diverse points of view. Submissions to conferences, such as SIGGRAPH, SOSP, ICSE, and OOPSLA, receive four to six reviews in rounds—papers that remain under consideration are assigned more reviewers. This process limits reviewer workload, while accommodating the growing numbers of submissions many conferences are experiencing.

The conference process fosters our research communities; a benefit I believe is underappreciated. For instance, in-person PC meetings structure research discussions on problem selection, contributions, approaches, methodology, and other scientific values. Committees include and train fresh Ph.D’s. Every researcher, from a first-time PC member to the most famous in the field, has an equal opportunity to express scientific viewpoints. A common experience after your first PC meeting, which I distinctly remember, is that young researchers feel respected. What better way to welcome researchers to our scholarly community?

Program committee meetings also build cross-institution relationships at all levels. Reviewing alone in your office does not foster community.

One remaining issue is citations. Many ACM conferences have very highly regarded and established brands. Unfortunately, the ACM Digital Library ambiguously specifies how to cite some of them. For example, see the Export Formats for PLDI, ISCA, and older SIGMOD papers. They have two DOI identifiers, one conference and one SIG Notices. ISI indexes SIG Notices, so some authors prefer them. This duplication is a historical accident due to SIGs that made conference proceedings issues of their SIG Notices as a member benefit, producing duplicate, ambiguous, and non-branded bibliographic entries. Worse, some conference names were not standardized from year to year. I strongly recommend that ACM work with the SIGS to develop citation formats that ensure the bibliographic record avoids confusion, the new citation clearly features conference brand names, and consistently apply this citation format retrospectively and thenceforth.

Back to Top