Technology Has Social Consequences

Dear Moshe,

First of all, many thanks and deep appreciation for putting this serious problem on *the* table of the community, CACM.

The problem is indeed very serious. I'll allow myself to add my 2 cents on why we all agree to serve on multiple PCs, with the situation getting worse with every year.

While WWW allowed for virtual PC meetings, Latex (and to some extent Word) allowed for preparing an X-page paper in virtually no time. As a result, the number of conference submissions continuously and rapidly grows. While most of these submissions would be "summary rejected" at any reasonable journal, in most (all?) conferences they are put through a reviewing process, wasting time of 3-4 reviewers. As a result, the demand for PC members keeps growing, while naturally the pool of qualified reviewers per topic remains more or less of the same size (or at least grows much slower).

Understanding of this situation, makes each of us dealing with the same dilemma every year: joining more than one PC means everything that you point on in this and previous articles, but refusing to help means leaving some PC chairs (that we usually know and appreciate) working with a ... less professional team. And this is a serious dilemma.

The only partial solution I see to the overall problem (and I dedicated quite a lot of thought to it in the last couple of years) is to (re-?)introduce a mechanism of "summary reject" to the conferences. If only a small portion of submissions will require a careful reviewing of details and actual writing of the review, then only a fraction of the current PCs will be needed, and PC chairs will get legitimacy to demand high quality of reviewing.

With warmest regards!
Carmel Domshlak (Technion, Israel)

Carmel, there is some truth to what you say, but I am not convinced that it is paper proliferation that drives conference proliferation. It sometimes seems the other way around. I have seen many conferences that the number of PC members is larger than the number of submissions :-)

Moshe Vardi

Moshe,

Point taken ;), but at least in part, this is yet another side of the same coin. PC chairs are worried these days not to get the most qualified PC members if these will be asked for too much work. And so they expand the PC ...

But of course, everything I wrote was to add my 2 cents, not to replace with them your $2 ;) Again: many thanks for the article!

Carmel

See Jeff Naughton's ICDE'10 slides at
http://lazowska.cs.washington.edu/naughtonicde.pdf

Dear Moshe,
Thank you for bringing up the issue. I fear that your observations are quite correct.
These implications of technology to the conference culture further exacerbates the ills that you have already discussed in past articles.

I would like to add another example to the discussion.

Proceedings papers are necessarily limited in size, and often omit improtant information such as proofs, experimental data, and so on.
It used to be a common practice to require appendices to be submitted together with the conference paper, so that, at least, the material would be in the hands of the referees. As you remarked, technology affects practices: I have the impression that it is a growing practice, at least in certain conferences, that papers point the reader to web resouces for missing information, which seems a most natural use of the current technology.

But technology strikes again: I have heard on few occasions that reviewers are known to avoid looking those web resources up, assuming that visitors to the web site might be tracked, and fearing for their anonymity. The natural development: submissions referring to web resources that do not even exist (and the reviewers not noticing it). It happenned.

Journals should not have this problem. First, the length of an article's text is not so severely restricted. Secondly, digital libraries enable the addition of electronic appendices to journal articles, which could include, for example, experimental data. I believe that this is an important consideration for journals to fulfill their roles in archiving research. Web resources are notoriously volatile, even if at the time of submission everything was proper.

The issue of ensuring that a reviewer has access to all the infomation they need is, however, pertinent both to journals and conferences, and I think that it should be considered by journal editors / conference committees.

Amir Ben-Amram

Dear Moshe,
Thank you very much for writing about the declining of ethics of program committees. There is another ethics problem which is the result of the one you mentioned. If a person is a member of programming committees at two - three – four closely related conferences and is considered to be the most suitable person for reviewing papers in some branch of computer science then, to very high extent, he rules by himself the fate of all publications in this area because, with the very high probability, the same person will be the member of the editorial board of the specialized magazine (transactions).
Another example of the declining ethics is the rewarding of the programming committee members with the Best Paper Award. This became the practice on programming conferences, while in all other areas of human activities it's unthinkable to be a judge in the competition where you participate.
My third remark is not on your text but on the first comment from Carmel Domshlak. The number of the conference submissions is not the result of using faster printing equipment but the result of evaluation of the scientific results by the number of publications. For the previous generation of scientists it would be a very good result of the long and very successful 40 or 50 years career in the University to have 70 – 80 publications (I am not writing about extraordinary cases). Now you can look through the web sites of contemporary assistant professors who are 35-40 years old and after 10-15 years of career they demonstrate you a list of 80 - 100 or more publications. Does anyone really think that there are new ideas in all these papers? "Something is rotten in the state of Denmark."
Thank you for your article,
Sergey Andreyev