Sign In

Communications of the ACM

Communications of the ACM

Should Conferences Meet Journals and Where?: A Proposal For 'PACM'


View as: Print Mobile App ACM Digital Library Full Text (PDF) In the Digital Edition Share: Send by email Share on reddit Share on StumbleUpon Share on Hacker News Share on Tweeter Share on Facebook
Joseph A. Konstan and Jack W. Davidson

Joseph A. Konstan (left) and Jack W. Davidson (right) are co-chairs of the ACM Publications Board.

Credit: University of Minnesota, University of Virginia

For many months now, the ACM Publications Board Conferences Committee has been working on a proposal that brings together conference and journal publishing. Here, we offer some background leading up to this proposal, summarize the plan, and solicit your input. We are also pleased to present two leading computer scientistsKathryn S. McKinley and David S. Rosenblumwho argue for and against the proposal beginning on page 43.

This proposal has been driven by strong input from the CS research community where the prevailing feeling is, despite strong review processes and selective acceptance, publishing in conference proceedings puts CS researchers at a disadvantage with respect to researchers in other scientific disciplines, where journal publication predominates, and is hence fundamentally detrimental to the CS field.

While some claim U.S.-based efforts by the National Research Council and the Computing Research Association have helped make top-quality conference papers "count" as much or more than peer-reviewed journal papers, others claim conference papers still have second-class status. Indeed, funding and merit systems in many countries do not fully recognize conference papers. Many believe the conference-centric culture of CS puts researchers at a disadvantage when competing with researchers in other science disciplines for top science awards and career progression.

The relationship between conferences and journals is complex. Many computer scientists lament the inherent flaws and limitations of conference publishing: hard deadlines, page limits, limited review and revision time, and overloaded review committees. Attending conferences comes at a cost, limiting access. And some of the initial reasons CS publishing favored conferencesthe lack of CS journals, the slow speed of journal publishing, and the high cost of journal papershave mostly disappeared. Still, many in our community argue that conferences are where the most exciting research is published; they seek to maintain the vibrant exchange that happens there. For this reason, we charged a diverse task force to explore whether the field would be better served through new models of conference-journal interaction.

Back to Top

What Is the Status Quo?

ACM currently offers three ways to publish conference papers in journals.

Revision of the paper by adding at least 25%33% new content and submitting to a journal for review. Disadvantages include "citation splitting" and lag time.

Journal-first publication where the paper is reviewed by the journal and published there, but the authors are invited to present at the conference. This option is the basis of the successful HiPEAC conference, whose papers are submitted directly to ACM TACO, and is increasingly common as an option in several SIG-sponsored conferences.

Journal-integrated publication where the conference integrates its committee review process with a journal. Papers accepted by the conference-review process are published in the journal and presented at the conference. Papers that need additional rounds of review are transferred to the journal-review process. Papers accepted through the journal-review process can be presented at a later conference. This model is used by SIGGRAPH in conjunction with ACM TOG.

Back to Top

New Proposal: A Proceedings-Focused Journal Series

The committee proposes a fourth alternativea journal series specifically created to publish the proceedings of ACM's highest quality conferences. Tentatively called Proceedings of the ACM, it would parallel ACM Transactions with a set of journals publishing high-quality research vetted by research communities through conferences. A few key features and criteria:

  • A series would cover several conferences related by SIG or theme (for example, PACM Programming Languages or PACM Distributed Systems).
  • All included proceedings papers would be reviewed for correctness, accuracy, completeness, and impact through a selective peer-review process. This process would require written reviews by a minimum number of three qualified reviewers. Documented policies would ensure the integrity of the review process.
  • The review process must permit at least one substantive revision by the authors and review of that revision by the reviewers to determine whether their concerns are adequately met.
  • Papers may not have artificial limits on length that would prevent full disclosure of references, documentation of methods, and so on. Supplemental materials would be clearly labeled as to whether they underwent peer review.
  • PACM series may invite submissions or second and subsequent revisions outside the conference timeline and review cycle.
  • Inclusion of a conference proceeding in a PACM series is subject to initial and periodic review by the series steering committee and the ACM Publications Board.

We want to hear from you! What do you think? Please read the arguments for and against this proposal on p. 43 and tell us what you think at https://www.surveymonkey.com/r/PACM2015 by Sept. 20 if possible!

Back to Top

Authors

Joseph A. Konstan (konstan@umn.edu) is co-chair of the ACM Publications Board.

Jack W. Davidson (jwd@virginia.edu) is co-chair of the ACM Publications Board.


Copyright held by authors.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2015 ACM, Inc.


Comments


David Kaeli

The top quality conferences in my field are either co-sponsored by ACM and IEEE-CS, or are sponsored by only one of these organizations. Would it be possible to have ACM and IEEE-CS work together on the ambitious idea put forth by ACM, but to be inclusive and carry the endorsements of both organizations? I know it would benefit my research community!


Christian Cachin

It is great to see such a proposal and discussion at the ACM level. I sympathize with its goals but am missing some crucial information.

As PACM would "publish the proceedings of ACM conferences", would its reviewers fully replace the conference program committees? If yes, the criteria in the bullets would simply be conditions on the review board (alias program committee - PC) of a conference. This would change some of the nature of the PC work, which McKinley's viewpoint heralds.

Would the current operation of PVLDB (Proceedings of the VLDB Endowment), with its continuous submission option, be compatible with PACM?

Much of the conference culture revolves around the perceived prestige or competitiveness of a venue. If multiple conferences are part of the same PACM for XYZ, would a distinction of which conference selected the paper be made and encouraged, or would this be hidden? Would there be special issues by conference venue so that authors can still cite their work referring to the conference?


Joseph Konstan

Thanks for your comments (and we'll keep trying to respond).

David, the idea of ACM and IEEE-CS working together is intriguing -- right now both organizations are experimenting with different ideas, but if we hit the right thing for the field, it is clear that we (and perhaps AAAI, SIAM, and many others) should coordinate.

Christian, the intent is not that PACM has reviewers, but that the reviews are done by conference program committees (and then the criteria become conditions on the review process). For some conferences, this would be a major change; for some, no change at all. And, of course, we're still seeking input on what the important factors are for those standards. The question of PVLDB is an interesting one (that motivated some of this exploration). One could argue that it fits this model, or that it fits the "journal-first" model (albeit with a journal whose model doesn't quite fit the ACM Transactions approach). But the concept of year-round submission is one that the task force was certainly in support of. Indeed, there are no rules preventing today's ACM conferences from moving to that model, though the logistics are different and potentially challenging. As for the individual conference proceedings within a series, the idea is to have an issue for each one, but it is also to ensure that the overall quality of a series is maintained at a high-prestige level. The task force's hope (and that of many commenters) is that peer pressure would lead to maintaining high standards while at the same time encouraging SIGs and conferences to open up venues for less thoroughly reviewed work, such as more workshops.


Robert Harper

In my view it is simply not the case that conference papers are equivalent to journal papers in other disciplines. Nor is it the case that they ought to be. Conferences satisfy a timeliness requirement; journals satisfy a reliability requirement. Results presented at conferences should be regarded as tentative until a backing journal article is published. Conference papers are not and should not be considered archival. If a tenure committee wishes not to consider conference papers as suitable evidence of scholarship, that is a rational and reasonable decision on its part (as would be the opposite decision). It is wrong to try to jigger the system so that conference publications look like journal publications. You've really got the whole thing backwards!


Amir ben-Amram

Many ACM conference papers are already quite long; a 12-page two column POPL paper
that I refomatted to TOPLAS format filled as many as 25 pages. And PC members have
their hands full; some PCs near or exceed 40 papers per PC member.
My opinion is that removing page limits from conference publications is not going to
help with the difficulties of conference reviewing and the quality of published papers.
Such an experiment was actually conducted at the FOCS conference series. The experiment has been abandoned by FOCS 2015 and the page limit reinstated.

The PACM proposal is somewhat ambiguous in this respect. It suggests to remove page
limits but also allows for non-refereed supplementary material, an option which allows
for the common practice of submitting appendices together with the conference submission.
It seems clear to me that the proposal does not try to change the main characteristics of
conferences. Reviewing will still be done by PCs (and perhaps their ERC or subreviewers)
and subject to the customary deadlines (more or less).
Also the quantity of conference papers is not expected to
change (unless it grows to accommodate the demand for publication in PACM).

I conclude that PACM papers are conference papers and should not be seen as the same
kind of entity as traditional journal papers, which undergo a more thorough, slower
refereeing cycle.
I believe that for many papers, this is not a problem. For example, some conference
papers may represent steps in a big project which might usefully culminate in a single
journal paper. Other papers are time-dependent. Some conference papers may actually be
self-contained, within page limits.

But there are also papers that cannot serve their scientific purpose without the long
(and sometimes hard) process of traditional journals. Papers dependent on subtle
mathematics require careful refereeing. With such papers, a conference paper - undergoing
traditional conference reviewing - has to be considered a preliminary announcement of
results (traditionally, such papers were often referenced as "extended abstracts").
I am wondering how the PACM model can accommodate this kind of papers.
Perhaps the two-track idea now adopted by ICLP should be considered - see
http://booleconferences.ucc.ie/iclp2015. Note that they allow both the author and the PC
to put a submission into the "non-journalized" track. This may alleviate the conflict
of the PC when presented with a paper that may leave some things to be desired (perhaps
even after a single revision as permitted by the PACM proposal), or that they feel they
cannot check in detail, while they also feel that the paper is significant and its
presentation to the public should not be delayed.


Displaying all 5 comments