Last week at ECSS 2010, the annual meeting of Informatics Europe in Prague, we heard a fascinating keynote by Moshe Vardi, editor-in-chief of Communications of the ACM, on "The Tragedy of the Computing-Research Commons" . Professor Vardi talked about the importance of engaging in activities that benefit the community even if they bring no huge immediate reward to the individuals who participate in them. He lamented the degradation of the computer science culture, due among other causes to shoddy refereeing practices.
Lamenting about the reviewing process is common, and everyone has horror stories. Yet the simple solution is almost never considered: turn the default refereeing mode to open, rather than anonymous.
Some cases may still justify anonymity, but they should be the exception, calling for a specific justification. Refereeing should be what it was before science publication turned into a business: scientists giving their polite but frank opinion on the work of other scientists. Anonymity just encourages power games, back-stabbing and, worst of all, poor quality: since non one can call your bluff, you are not encouraged to be a good referee. Of course many people do an excellent job anyway, but they do not necessarily prevail: in the highly competitive world of computer science publications, conference publication in particular with its schedule pressures, one incompetent but arrogant negative review typically outweighs five flattering and carefully considered analyses.
By revealing who you are you force yourself to write reviews that you can defend.
More than two decades ago I started refusing to do anonymous reviews. This stance may not have only brought me new friends (which may not be a big deal, as I am not sure people who hate you because you found flaws in one of their papers are worth having as scientific friends), but it has certainly made me a better reviewer, and in fact did bring me some friends Normal 0 false false false EN-US X-NONE X-NONE — people who are grateful for having gained new insights, positive or negative, into their own work.
A more complete discussion and rationale can be found in a Web page, to which I regularly refer editors asking for reviews. That text, written several years ago, is verbose and should be rewritten, but it does include the basic analysis.
The decision to perform open refereeing was personal and until now I have always refrained from proselytizing. Seeing the degradation in refereeing, however, I believe such reserve is no longer appropriate. Establishing open refereeing as the default strategy is the first step towards fixing the flawed culture of computer science refereeing.
Join the Discussion (0)
Become a Member or Sign In to Post a Comment