Speaker: Lowell S. Brown (University of Washington, PRD)

Panel Discussion: Peer-Review

The following is an account of what I said or perhaps should have said at the Workshop. But before giving this account, I should first explain why I presented these remarks. The reason is that many of the people who are interested in the electronic media appear to believe that the refereeing process is trivial and of little consequence. My experience after several years as an editor is that the converse is the case.

What referees and editors do, first of all, is to set some basic standards for their field in accepting papers for publication. Secondly, but still importantly, the referee process, guided by editorial decisions, improves a large fraction of the manuscripts before they are published. I shall present a brief statistical survey of how this process works in Physical Review D15. This journal covers the more theoretical apsects of elementary particle physics including general relativity and the cosmology of the early universe. It is the most contentious of the Physical Review journals with about 40% of the manuscripts that it receives being rejected.

The figure shows the story. In 1992, Physical Review D15 received 1020 manuscripts. Of these, 620 were accepted for publication, with 370 accepted after only one review, while the remaining 250 were accepted after several reviews. Manuscripts with two or more reviews often involved more than one referee. The successive refereeing improved a substantial number of the papers. Indeed, even some of the papers that received only one referee report were improved by the authors' response to the referee report. Changes are often made to improve the clarity of the presentation and to sharpen the presentation. Changes are often made to remove misleading statements. Occasionally, substantial errors are corrected. These improvements could appear to be minor to a superficial observer comparing, say, the number of pages or the number of equations of the original and final versions. They do not appear to be minor to a student or a researcher who is trying to understand the work.

Of the 1020 manuscripts received, 400 were rejected. The editor rejected 160 without review. Many of the manuscripts rejected by the editor were outside the scope of the journal. A substantial fraction, however, did not meet minimum standards. Of the remaining 240 manuscripts, 100 authors chose not to resubmit after receiving a negative report, while 140 manuscripts involved multiple reports before they were finally rejected. It is often hard to find good referees. It is especially difficult to find referees for "questionable closed author cliques", small groups of people who work on topics outside the mainstream of research which are often topics that were long ago found to be unproductive. It is hard to find referees not in these groups who are willing to give careful reviews of the work.

The 860 manuscripts that were sent out to referee entailed 1,500 referee reports. The 620 papers that were accepted entailed 1,000 referee reports. The 240 papers that were rejected by referees required 500 reports. I believe that this process did set standards and did often improve the published work.