In the two years since we launched the revitalized Communications of the ACM, I have received hundreds of email messages from readers. The feedback has been mostly, but not universally, positive. Many people do note places where we can do better. Some readers point out errors in published articles. Nothing in life is perfect. Communications is an ongoing project; continuous improvement is the name of the game.
At the same time, I have also received a fair number of notes with nothing short of withering criticism. For example, six issues into the revitalized Communications, I received this comment from a leading computer scientist: "Although I have looked at every issue and at least glanced at every article, I have not yet found one good one."
Do you find this statement harsh? It surely pales in comparison to this: "The level is unbelievably poor. It reads sometimes like a PR article for big companies. Donation to the ACM seems to be the main reviewing criterion. I would call the policy of ACM scientific prostitution, and I don’t want to pay for a prostitute."
I believe most of us have received at some point very harsh reviews—though, hopefully, not that harsh—on papers or proposals we have written. If you are an experienced researcher, you have undoubtedly dealt with papers and proposals being declined. Still, the harsh tone of negative reviews can be quite unsettling even to experienced authors. When I talk to colleagues about this, they just shrug, but I think this phenomenon, which I call "hypercriticality," deserves our collective attention. Other people recently commented on this issue. In the context of proposal reviewing, Ed Lazowska coined the phrase "circling the wagons and shooting inwards," and John L. King, in a recent CCC blog, referred to such verbal assaults as "Fratricide." Jeff Naughton, referring to conference paper reviewing, said in a recent invited talk that "bad reviewing" is "sucking the air out of our community."
The "hypercriticality" claim is not just based on anecdotes; we actually have data that supports it. Proposals submitted to the Computer and Information Science and Engineering (CISE) Directorate of the U.S. National Science Foundation (NSF) are rated, on the average, close to 0.4 lower (on a 1-to-5 scale) than the average NSF proposal. In his blog entry, King discussed the harmful effects of such harshness.
What is the source of this harshness within our discipline? Here one can only speculate. Let me offer two possible explanations. My first theory refers to the intrinsic nature of our discipline. Computing systems are notoriously brittle. Mistyping one variable name can lead to a catastrophic failure. Computing embodies the principle of "For lack of a nail, the kingdom was lost." This makes us eternally vigilant, looking for the slightest flaw. In our eternal hunt for flaws, we often focus on the negative and lose perspective of the positive.
What is the source of this harshness within our discipline?
My second theory refers to the sociology of our field. We typically publish in conferences where acceptance rates are 1/3, 1/4, or even lower. Reviewers read papers with "reject" as the default mode. They pounce on every weakness, finding justification for a decision that, in some sense, has already been made. It is particularly easy to be harsh when reviewing proposals. If the proposal is not detailed enough, then the proposer "does not have a clear enough plan of research," but if the proposal is rich in detail, then "it is clear that the proposer has already done the work for which funding is sought."
What is to be done? Remember, we are the authors and we are the reviewers. It is not "them reviewers;" it is "us reviewers." Hillel the Elder, a Jewish scholar, 30 B.C.-10 A.D., said "What is hateful to you, do not do to your fellow." This is known as the Silver Rule in moral philosophy. The Golden Rule, which strengthens the Silver Rule, asserts "do unto others as you would have them do to you." Allow me to rephrase this as the Golden Rule of Reviewing: "Write a review as if you are writing it to yourself." This does not mean that we should not write critical reviews! But the reviews we write must be fair, weighing both strengths and weaknesses; they must be constructive, suggesting how the weaknesses can be addressed; and, above all, they must be respectful.
After all, these are the reviews that we would like to receive!
Moshe Y. Vardi’s
EDITOR-IN-CHIEF
Join the Discussion (0)
Become a Member or Sign In to Post a Comment