Hiding referees

This article in the Chronicle of Higher Education is about the ways in which authors can find out who reviewed their paper for academic journals prompted me to wonder about why referees do not 'sign' their reviews. Why is there anonymity?

We know why sometimes journals make reviewers blind to an author's identity -- to remove reputation and affiliation from the accept/reject decision. But what good does making authors blind to a reviewer's identity do. After all, everywhere else in academia we value attribution.

The key issue is whether anonymity improves the 'honesty' of the review. The two type of errors that could occur (Type I and Type II) are that poor articles may receive favourable reviews while good articles might receive unfavourable reviews.

Why would a reviewer of a bad article, give it a favourable review? With anonymity, there is no reason to distort the truth while without it, a reviewer may be concerned about other effects. Of course, that effect could be embarrasment, so reviewers would be more cautious about 'getting it wrong.' But the other effect could be fear of retaliation. There may be a concern that reviewers who make unfavourable reports may receive retaliation from authors (or their colleagues). How might this retaliation come? When the author becomes reviewer, then there may be in an issue in getting published or securing jobs, etc. Thus, anonymity involves a trade-off: it removes a fear of retaliation for a cost of 'shirking' and not as thorough a review.

Do similar things explain why a reviewer of a good article, would give it a bad review? Again, fear of embarassment is an issue. I wrote a paper years ago (Journal of Economic Perspectives, 1994) about the classic articles in economics that had been initially rejected. There were lots of them and I wondered if, had reviewers been non-anonymous, would this have occured. (Actually economics is somewhat of an outlier in terms of the sheer volume of rejections compared with other sciences).

But surely fear of retaliation is less of an issue? Maybe, the reverse is not. You could give a favourable review to 'pay it forward' for favourable reviews in the future. Thus, a club or cartel might form and too many good papers would be accepted.

In each case, with transparency, the potential for distortion arising from punishments and gifts is traded-off from short-run costs associated with embarassment in giving 'off the mark' reviews. This suggests to me that, if it is becoming harder to retain anonymity as the Chronicle argues, then the potential for embarassment needs to rise. In particular, editors will need to get more and more reviewers for a given paper. And let me tell you, as an editor, this is not without cost.

But the current issue is that transparency may be limited to authors and reviewers and not to anyone external. This would be a real problem -- little chance of external embarassment -- but lots of opportunities for cartelisation.