Cognitive Science 39 (2):405-433 (2015)
AbstractResearch into expertise is relatively common in cognitive science concerning expertise existing across many domains. However, much less research has examined how experts within the same domain assess the performance of their peer experts. We report the results of a modified think-aloud study conducted with 18 pilots . Pairs of same-ranked pilots were asked to rate the performance of a captain flying in a critical pre-recorded simulator scenario. Findings reveal considerable variance within performance categories, differences in the process used as evidence in support of a performance rating, different numbers and types of facts identified, and differences in how specific performance events affect choice of performance category and gravity of performance assessment. Such variance is consistent with low inter-rater reliability. Because raters exhibited good, albeit imprecise, reasons and facts, a fuzzy mathematical model of performance rating was developed. The model provides good agreement with observed variations
Added to PP
Historical graph of downloads
References found in this work
Categorization and Representation of Physics Problems by Experts and Novices.M. T. H. Chi, P. J. Feltovich & R. Glaser - 1981 - Cognitive Science 5 (2):121-52.
Facing the Sunrise: Cultural Worldview Underlying Intrinsic-Based Encoding of Absolute Frames of Reference in Aymara.Rafael E. Núñez & Carlos Cornejo - 2012 - Cognitive Science 36 (6):965-991.
Citations of this work
No citations found.
Similar books and articles
Teaching Domain-Specific Skills Before Peer Assessment Skills is Superior to Teaching Them Simultaneously.M. J. van Zundert, K. D. Könings, D. M. A. Sluijsmans & J. J. G. van Merriënboer - 2012 - Educational Studies 38 (5):541-557.
Application of Fuzzy Comprehensive Evaluation on COGAG Power Plant of Performance.Menglin Shao - 2014 - Journal of Power and Energy Engineering 2:29-34.
Assessing Quality and Evaluating Performance in Higher Education: Worlds Apart or Complementary Views?Cláudia S. Sarrico, Maria J. Rosa, Pedro N. Teixeira & Margarida F. Cardoso - 2010 - Minerva 48 (1):35-54.
Peer Assessment of Children's Prosocial Behaviour.Susan Hayes Greener - 2000 - Journal of Moral Education 29 (1):47-60.
Future of Ethically Effective Leadership.Chaudhary Imran Sarwar - 2013 - Journal of Business Ethics 113 (1):81-89.
Expanding Expertise: Investigating a Musician’s Experience of Music Performance.Andrew Geeves, Doris Mcllwain, John Sutton & Wayne Christensen - 2010 - ASCS09: Proceedings of the 9th Conference of the Australasian Society for Cognitive Science:106-113.
The Link Between Corporate Social and Financial Performance: Evidence From the Banking Industry. [REVIEW]W. Gary Simpson & Theodor Kohers - 2002 - Journal of Business Ethics 35 (2):97 - 109.
Performance, Promotion and Information.Zajac Jaroslav - 2004 - Journal of Business Ethics 50 (2):187-198.
Use and Misuse of Metrics in Research Evaluation.Ronald N. Kostoff - 1997 - Science and Engineering Ethics 3 (2):109-120.
Sustainable Supply Chains: Governance Mechanisms to Greening Suppliers. [REVIEW]Cristina Gimenez & Vicenta Sierra - 2013 - Journal of Business Ethics 116 (1):189-203.