Hypercriticality

Indeed, it's really annoying, if not disrespectful, to have a work of months or years rejected without further explanation and _constructive_ criticism.

For instance, I'd suggest to 'review the reviewer': why cannot the authors assign a rate to the reviewer of their paper? Then, for future reviews, those low rated ones would not be called to the committee (unless they improve their reviewing skills).

Of course, the real issue to tackle is the lack of (proper) education. Most of it is geared towards formalisms, models, and abstraction, often used in the virtual world of our own and cold-hearted machines. Topics such as moral, and ethics, are not only shallowly handled, but not present in ours daily life. Proper education onto this topics, and the engagement of computer scientists into real-world issues in their neighborhood (not necessarily computer related) would improve this situation.

I am one of the rare researchers practicing multidisciplinary research. Therefore, I experience the differences in review practices at first hand. This letter from the editor confirms my experience. I have been reducing my submissions to computer science forums. Publication acceptance increasingly requires me to squander the tax payers’ money on matters that bring no benefit to society nor contribute to our knowledge. It takes several weeks of writing to make a submission compliant with the unwritten rules of IT research communities where absolutely no added value is generated by this effort.

In contrast to the editor’s letter, I perceive more than hypercriticality. I see reviewers behaving like old-style school teachers correcting exam papers, not like researchers that are keen to learn something new and to contribute their substantial expertise in this process. Allow me to elaborate.

Reviewers fail to appreciate truly innovative contributions. If Herbert Simon, Nobel Prize winner, were to submit a paper on how ‘bounded rationality’ has implications on software engineering methodologies, it most likely would be rejected in a blind review. Formally because the paper failed to discuss work-by-others, etc. In reality, the submission is rejected because the reviewer failed to understand it, because it is perceived as competition undermining his or her own work, etc.

Radical innovation should be welcomed by researchers that are longing for fascinating new insights. This hunger to learn something new is absent, more in IT than in other domains. In particular, reviews for workshops must select innovation over other criteria. Especially, reviewers must behave more humbly if they do not understand a paper and show respect. Moreover, IT researchers must be cautious when they fail to perceive a contribution, which often means that it is situated in the border area crossing IT with an application domain.

Reviewers seem to shut down their own brain when it comes to identifying the research contribution in a paper submission. Papers are rejected when they fail to cite work-by-others, fail to discuss how they contribute in meticulous detail, etc. Papers are rejected because they constitute a fresh start, addressing long-standing problems, when there is no substantial work-by-others to discuss. Reviewers are supposed to be experts who, at least, should be able to be precise about what is missing or wrong in this respect. Note that ‘pretending not to understand’ is a key mechanism to abuse power/authority. Today’s non-cooperative hostile reviewing attitude involuntarily implements this mechanism.

Overall, this situation is more than a nuisance. It is truly alarming. This attitude destroys civilizations; at the very least, it is correlated with their demise. I recommend reading “The sleepwalkers: …” by Koestler to learn just how worried we need to be.

Our society is facing new challenges when it comes to science and technology. The cost and effort to perform scientific experiments increases as we need to investigate infrastructures rather than components, systems of systems rather than a single machine… The present attitude makes it impossible to have cooperative research with joint results in preparation of those experiments. Indeed, smooth access to publications requires us to perform the experiments first. As a result, most will be toy experiments and the valuable bigger ones will be selected based on politics rather than scientific interactions. A lottery would be a superior and certainly more robust mechanism to allocate resources to perform research.

At a time where we need to reinvent the scientific discussion, using all facilities that IT and psychology may offer, we are freezing into a hostile, formalistic and conservative community. In conclusion, I applaud this letter of the editor and want to point out the seriousness of this matter. It is not about being at a disadvantage as an IT researcher, it is not about being more civilized colleagues. It actually is about preserving our civilization for future generations.

See http://www.sigcomm.org/ccr/papers/2011/July/2002250.frontmatter
and http://jbiol.com/content/8/3/24