Technology Has Social Consequences

A conference program committee (PC) member received a paper for review. He distributed the manuscript to his research group to "solicit their opinions of the paper" and the group embarked on improving the results of the paper under review. The research group then submitted their own paper to another conference, their submission occurring three months before the first paper was to be presented at a conference. When eventually confronted (the short gap between the appearance of the two papers triggered questions), the PC member responded with "Was that wrong? Should I have not done that?" (The reader may want to search for this phrase on YouTube.)

Amazingly, this PC member was not aware that a conference paper submission constitutes privileged communication. In theory, reviewers should immediately "forget" what they have read. For reviewers to use such privileged material for their own work immediately creates a blatant conflict of interest. How did this PC member, a full professor in a respected university, not know such a fundamental rule of scholarly reviewing?

To understand how the ethics of program committees has declined, one must review the history of computer science program committees over the last 50 years. Until the mid-1990s, program committees met in face-to-face meetings. This had two significant consequences. First, PC members bore the cost of attending PC meetings, leading them to be careful with accepting PC service commitments. It was rare then for one to serve on more than one PC per year. Second, junior PC members had a chance to interact intensively with senior PC members. There was nontrivial social pressure on junior PC members to demonstrate their competence in PC meetings. Thus, PC service provided important socializing experience, where unwritten norms and customs were learned by observation.

With the emergence of the WorldWide Web in the mid-1990s, physical PC meetings suddenly seemed wasteful, as it became possible to conduct virtual meetings without incurring travel expenses and headaches. Conference-management software tools emerged and many communities abandoned physical meetings in favor of virtual ones. I was very much in favor of this change back then! It took, however, a few years for the adverse consequences of this change to become visible.

Economists would tell you that a commodity priced too low would end up being overconsumed. PC service is a commodity with positive utility. Our community views PC service as a form of professional recognition; in fact, it is one of the few markers of professional recognition available to junior researchers. Since the "cost" of PC service has dropped with the switch to virtual meetings, "consumption" has gone up. Indeed, it is quite common today to see researchers serving on several PCs per year.

Of course, one cannot expect the same level of effort from someone who serves on one PC per year as compared to someone who serves on multiple PCs per year. Indeed, in the 1980s it was typical to see every submission read by five to six PC members, today the norm is often three to four reviewers for submission. Furthermore, these reviewers are often not PC members but "subreviewers." In fact, while the concept of subreviewer was originally developed with the purpose of bringing additional expertise to PCs, today it is viewed as an opportunity to train junior researchers in the art of paper reviewing. Thus, the role of a PC member seems to have evolved from that of a reviewer to that of "review orchestrator."

What has been the outcome of this development? Many of us are quite familiar with this outcome. The quality of conference reviewing has declined and the selection process has become far more random. Two years ago, I wrote in this space about "Conferences vs. Journals in Computing Research." The declining quality of conference reviewing was one of triggers that spurred me to write that editorial.

The loss in quality of conference reviewing is just one result of the move to virtual PC meetings. Another outcome is the loss of socialization that took place in PC meetings. It is this lost socialization that contributed to a senior researcher being ignorant of one of the most basic rules of scholarly reviewing.

We all know that technology has social consequences. This applies to us as well. The switch to purely virtual meetings did not serve us well. Many communities have already realized that and are combining virtual and physical meetings to merge the strengths of both formats. Such practices, I believe, should be widely adopted. Technology can be managed!

Moshe Y. Vardi, EDITOR-IN-CHIEF

Back to Top