Aggregating forecasts of chance from incoherent and abstaining experts
| Abstract | Decision makers often rely on expert opinion when making forecasts under uncertainty. In doing so, they confront two methodological challenges: the elicitation problem, which requires them to extract meaningful information from experts; and the aggregation problem, which requires them to combine expert opinion by resolving disagreements. Linear averaging is a justifiably popular method for addressing aggregation, but its robust simplicity makes two requirements on elicitation. First, each expert must offer probabilistically coherent forecasts; second, each expert must respond to all our queries. In practice, human judges (even experts) may be incoherent, and may prefer to assess only the subset of events about which they are comfortable offering an opinion. In this paper, a new methodology is developed for combining expert assessment of chance. The method retains the conceptual and computational simplicity of linear averaging, but generalizes the standard approach by relaxing the requirements on expert elicitation. The method also enjoys provable performance guarantees, and in experiments with real-world forecasting data is shown to offer both computational efficiency and competitive forecasting gains as compared to rival aggregation methods. This paper is relevant to the practice of decision analysis, for it enables an elicitation methodology in which judges have freedom to choose the events they assess. | |||||||||
| Keywords | No keywords specified (fix it) | |||||||||
| Categories | No categories specified (fix it) | |||||||||
| Options |
|
|||||||||
Download options| PhilPapers Archive |
Upload a copy of this paper Check publisher's policy on self-archival Papers currently archived: 5,709 |
| External links |
  Try with proxy. |
| Through your library | Only published papers are available at libraries |
Similar books and articlesStephen John (2011). Expert Testimony and Epistemological Free-Riding: The Mmr Controversy. Philosophical Quarterly 61 (244):496-517.
Axel Gelfert (2011). Expertise, Argumentation, and the End of Inquiry. Argumentation 25 (3):297-312.
Jeryl L. Mumpower & Thomas R. Stewart (1996). Expert Judgement and Expert Disagreement. Thinking and Reasoning 2 (2 & 3):191 – 212.
Guanchun Wang, Sanjeev Kulkarni & Daniel N. Osherson, Improving Aggregated Forecasts of Probability.
Roger M. Cooke (1991). Experts in Uncertainty: Opinion and Subjective Probability in Science. Oxford University Press.
Analytics
Monthly downloads |
Added to index2009-01-28Total downloads4 ( #178,844 of 550,802 )Recent downloads (6 months)1 ( #63,425 of 550,802 )How can I increase my downloads? |
My notes
Discussion
