Sign In

Communications of the ACM

Communications of the ACM

Incentivizing Quality and Impact in Computing Research


Communications Editor-in-Chief Moshe Y. Vardi

Over the past few years, the computing-research community has been conducting a public conversation on its publication culture. Much of that conversation has taken place in the pages of Communications. (See http://cra.org/scholarlypub/.) The underlying issue is that while computing research has been widely successful in developing fundamental results and insights, having a deep impact on life and society, and influencing almost all scholarly fields, its publication culture has developed certain anomalies that are not conducive to the future success of the field. A major anomaly is the reliance of the fields on conferences as the chief vehicle for scholarly publications.

While the discussion of the computing-research publication culture has led to general recognition that the "system is suboptimal," developing consensus on how the system should be changed has proven to be exceedingly hard. A key reason for this difficulty is the fact the publication culture does not only establish norms for how research results should be published, it also creates expectations on how researchers should be evaluated. These publication norms and research-evaluation expectations are complementary and mutually enforcing. It is difficult to tell junior researchers to change their publication habits, if these habits have been optimized to improve their prospects of being hired and promoted.


Comments


Rizos Sakellariou

The question is: after all this inflationary pressure, are we in a good position to judge quality? Or, for as long as quantity and metric-based assessment are favored by high-level management (let alone the buzz, big data and the like), eventually there will be allies who win the debate? Isn't this something similar to "bad currency drives out good currency"?


Ben Shneiderman

While we are all in favor of high quality research publications, the dilemma is that hiring, tenure and promotion committee consist of senior faculty whose definitions of quality may not be sympathetic to the topics and methods used by younger researchers, thereby slowing innovation in our fast moving field. By balancing subjective impressions with objective data on download counts, social media discussion, and eventually citation counts, young researchers can demonstrate the value of their work to skeptical senior faculty.


Roberto Sebastiani

"We should not be impressed by large research grants, but ask what the actual yield of the funded projects has been." Two thumbs up for this sentence (among others). In an era in which -- at least in Europe -- careers are evaluated mostly for the amount of money one brings in, it was time some very-authoritative scientist raised this issue.

My two cents on the issue "A major anomaly is the reliance of the fields on conferences as the chief vehicle for scholarly publications."

IMHO the main problem is, most CS journals -- some with notable exception -- have reviewing times which are geological eras wrt. the evolution speed of CS (e.g., I've personally experienced 18-months for having the first review) so that journals are often no more considered by a computer scientist as a vehicle to spread the visibility of his/her results, but only as a cumbersome and incredibly slow way of improving his/her own CV.


Joshua Grochow

Although I agree with the opinion, I have often wondered how to reconcile the desire to recognize true quality, rather than to count beans, with scale? My understanding is that the number of applications per faculty position is often in the hundreds, and the Malthusian problem in academia is not getting any better (even with the recent increase in CS tenure lines, following increasing enrollments in CS undergraduate majors).


Displaying all 4 comments

Log in to Read the Full Article

Sign In

Sign in using your ACM Web Account username and password to access premium content if you are an ACM member, Communications subscriber or Digital Library subscriber.

Need Access?

Please select one of the options below for access to premium content and features.

Create a Web Account

If you are already an ACM member, Communications subscriber, or Digital Library subscriber, please set up a web account to access premium content on this site.

Join the ACM

Become a member to take full advantage of ACM's outstanding computing information resources, networking opportunities, and other benefits.
  

Subscribe to Communications of the ACM Magazine

Get full access to 50+ years of CACM content and receive the print version of the magazine monthly.

Purchase the Article

Non-members can purchase this article or a copy of the magazine in which it appears.
Read CACM in a free mobile app!
Access the latest issue, plus archived issues and more
ACM Logo
  • ACM CACM apps available for iPad, iPhone and iPod Touch, and Android platforms
  • ACM Digital Library apps available for iOS, Android, and Windows devices
  • Download an app and sign in to it with your ACM Web Account
Find the app for your mobile device
ACM DL Logo