In public peer review of submitted manuscripts, how do reviewer comments differ from comments written by interested members of the scientific community? A content analysis of comments written for Atmospheric Chemistry and Physics

Whereas in traditional peer review a few selected researchers (peers) are included in the manuscript review process, public peer review includes both invited reviewers (who write ‘reviewer comments’) and interested members of the scientific community who write comments (‘short comments’). Available to us for this investigation are 390 reviewer comments and short comments assessing 119 manuscripts submitted to the journal Atmospheric Chemistry and Physics (ACP). We conducted a content analysis of these comments to determine differences in the main thematic areas considered by the scientists in their assessment comments. The results of the analysis show that in contrast to interested members of the scientific community, reviewers focus mainly on (1) the formal qualities of a manuscript, such as writing style, (2) the conclusions drawn in a manuscript, and (3) the future “gain” that could result from publication of a manuscript. All in all, it appears that ‘reviewer comments’ better than ‘short comments’ by interested members of the scientific community support the two main functions of peer review: selection and improvement of what is published.