Abstract

Manuscript reviews are intended to be objective, empirical assessments of the scientific worth of papers submitted for publication. However, critics have charged that manuscript reviews are unreliable, unconstructive, and biased in a number of ways . A review of the empirical literature in this area indicates: that inter-reviewer reliability in manscript assessments is clearly inadequate, that reviewer bias can sometimes influence manuscript assessments, and that there is a dearth of empirical data supporting the predictive and discriminant validity of manuscript assessment procedures. Based on the available evidence it seems that manuscript reviews are more strongly influenced by chance factors than by systematic reviewer or editorial bias. Nonetheless, our desire to conceptualize manuscript reviews in psychology as objective, empirical assessments has produced a number of undesired results. An alternative approach to manuscript review based on an adversery model rather than a scientific model is presented. Advantages of an adversary model as a method for identifying sound research are discussed. Changes in current publication policies that would allow research findings to be disseminated more efficiently are also described