Philosophy of Science 79 (5):859-870 (2012)

Authors
Carole J. Lee
University of Washington
Abstract
Psychometrically oriented researchers construe low inter-rater reliability measures for expert peer reviewers as damning for the practice of peer review. I argue that this perspective overlooks different forms of normatively appropriate disagreement among reviewers. Of special interest are Kuhnian questions about the extent to which variance in reviewer ratings can be accounted for by normatively appropriate disagreements about how to interpret and apply evaluative criteria within disciplines during times of normal science. Until these empirical-cum-philosophical analyses are done, it will remain unclear the extent to which low inter-rater reliability measures represent reasonable disagreement rather than arbitrary differences between reviewers.
Keywords peer review  social epistemology  science of science
Categories (categorize this paper)
DOI 10.1086/667841
Options
Edit this record
Mark as duplicate
Export citation
Find it on Scholar
Request removal from index
Revision history

Download options

PhilArchive copy


Upload a copy of this paper     Check publisher's policy     Papers currently archived: 58,467
Through your library

References found in this work BETA

View all 11 references / Add more references

Citations of this work BETA

Bias in Peer Review.Carole J. Lee, Cassidy R. Sugimoto, Guo Zhang & Blaise Cronin - 2013 - Journal of the American Society for Information Science and Technology 64 (1):2-17.
Is Peer Review a Good Idea?Remco Heesen & Liam Kofi Bright - 2020 - British Journal for the Philosophy of Science:axz029.
The Social Dimensions of Scientific Knowledge.Helen Longino - 2008 - Stanford Encyclopedia of Philosophy.

View all 9 citations / Add more citations

Similar books and articles

Bias in Peer Review.Carole J. Lee, Cassidy R. Sugimoto, Guo Zhang & Blaise Cronin - 2013 - Journal of the American Society for Information Science and Technology 64 (1):2-17.
The Principles and Practices of Peer Review.Ronald N. Kostoff - 1997 - Science and Engineering Ethics 3 (1):19-34.
Evidence for the Effectiveness of Peer Review.Robert H. Fletcher & Suzanne W. Fletcher - 1997 - Science and Engineering Ethics 3 (1):35-50.
Using a Dialectical Scientific Brief in Peer Review.Arthur E. Stamps - 1997 - Science and Engineering Ethics 3 (1):85-98.
‘Peer Review’ Culture.Malcolm Atkinson - 2001 - Science and Engineering Ethics 7 (2):193-204.

Analytics

Added to PP index
2012-07-24

Total views
56 ( #180,972 of 2,421,247 )

Recent downloads (6 months)
4 ( #192,306 of 2,421,247 )

How can I increase my downloads?

Downloads

My notes