In search of good probability assessors: an experimental comparison of elicitation rules for confidence judgments

Theory and Decision 80 (3):363-387 (2016)
  Copy   BIBTEX

Abstract

In this paper, we use an experimental design to compare the performance of elicitation rules for subjective beliefs. Contrary to previous works in which elicited beliefs are compared to an objective benchmark, we consider a purely subjective belief framework. The performance of different elicitation rules is assessed according to the accuracy of stated beliefs in predicting success. We measure this accuracy using two main factors: calibration and discrimination. For each of them, we propose two statistical indexes and we compare the rules’ performances for each measurement. The matching probability method provides more accurate beliefs in terms of discrimination, while the quadratic scoring rule reduces overconfidence and the free rule, a simple rule with no incentives, which succeeds in eliciting accurate beliefs. Nevertheless, the matching probability appears to be the best mechanism for eliciting beliefs due to its performances in terms of calibration and discrimination, but also its ability to elicit consistent beliefs across measures and across tasks, as well as its empirical and theoretical properties.

Links

PhilArchive



    Upload a copy of this work     Papers currently archived: 91,423

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Similar books and articles

Subjective probability estimates and confidence ratings.Lee R. Beach & James A. Wise - 1969 - Journal of Experimental Psychology 79 (3p1):438.
Some Notes on Probability and Confidence.Paul Ryl - 1987 - Dissertation, The University of New Mexico
Three prepositional calculi of probability.Herman Dishkant - 1980 - Studia Logica 39 (1):49 - 61.
Comments on Quantum Probability Theory.Steven Sloman - 2014 - Topics in Cognitive Science 6 (1):47-52.

Analytics

Added to PP
2015-07-11

Downloads
48 (#324,723)

6 months
8 (#347,798)

Historical graph of downloads
How can I increase my downloads?